{"html_url": "https://github.com/simonw/datasette/issues/1#issuecomment-338523957", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1", "id": 338523957, "node_id": "MDEyOklzc3VlQ29tbWVudDMzODUyMzk1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2017-10-23T01:09:05Z", "updated_at": "2017-10-24T02:42:12Z", "author_association": "OWNER", "body": "I also need to solve for weird primary keys. If it\u2019s a single integer or a single char field that\u2019s easy. But what if it is a compound key with more than one chat field? What delimiter can I use that will definitely be safe?\r\n\r\nLet\u2019s say I use hyphen. Now I need to find a durable encoding for any hyphens that might exist in the key fields themselves.\r\n\r\nHow about I use URLencoding for every non-alpha-numeric character? That will turn hyphens into (I think) %2D. It should also solve for unicode characters, but it means the vast majority of keys (integers) will display neatly, including a compound key of eg 5678-345\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 267513424, "label": "Addressable pages for every row in a table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1#issuecomment-338524454", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1", "id": 338524454, "node_id": "MDEyOklzc3VlQ29tbWVudDMzODUyNDQ1NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2017-10-23T01:15:24Z", "updated_at": "2017-10-23T01:15:24Z", "author_association": "OWNER", "body": "Table rendering logic needs to detect the primary key field and turn it into a hyperlink. If there is a compound primary key it should add an extra column at the start of the table which displays the compound key as a link", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 267513424, "label": "Addressable pages for every row in a table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1#issuecomment-338857568", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1", "id": 338857568, "node_id": "MDEyOklzc3VlQ29tbWVudDMzODg1NzU2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2017-10-24T02:57:12Z", "updated_at": "2017-10-24T02:57:12Z", "author_association": "OWNER", "body": "I can find the primary keys using:\r\n\r\n PRAGMA table_info(myTable)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 267513424, "label": "Addressable pages for every row in a table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1#issuecomment-338861511", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1", "id": 338861511, "node_id": "MDEyOklzc3VlQ29tbWVudDMzODg2MTUxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2017-10-24T03:24:17Z", "updated_at": "2017-10-24T03:24:17Z", "author_association": "OWNER", "body": "Some tables won't have primary keys, in which case I won't generate pages for individual records.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 267513424, "label": "Addressable pages for every row in a table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1#issuecomment-338872286", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1", "id": 338872286, "node_id": "MDEyOklzc3VlQ29tbWVudDMzODg3MjI4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2017-10-24T04:46:06Z", "updated_at": "2017-10-24T04:46:06Z", "author_association": "OWNER", "body": "I'm going to use `,` as the separator between elements of a compound primary key. If those elements themselves include a comma I will use `%2C` in its place.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 267513424, "label": "Addressable pages for every row in a table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1#issuecomment-338882207", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1", "id": 338882207, "node_id": "MDEyOklzc3VlQ29tbWVudDMzODg4MjIwNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2017-10-24T05:56:04Z", "updated_at": "2017-10-24T05:56:04Z", "author_association": "OWNER", "body": "Next step: generate links to these.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 267513424, "label": "Addressable pages for every row in a table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/10#issuecomment-341938424", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/10", "id": 341938424, "node_id": "MDEyOklzc3VlQ29tbWVudDM0MTkzODQyNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2017-11-04T23:48:57Z", "updated_at": "2017-11-04T23:48:57Z", "author_association": "OWNER", "body": "Done: https://github.com/simonw/stateless-datasets/commit/edaa10587e60946e0c1935333f6b79553db33798", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 267517381, "label": "Set up Travis"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/100#issuecomment-344771130", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/100", "id": 344771130, "node_id": "MDEyOklzc3VlQ29tbWVudDM0NDc3MTEzMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2017-11-16T00:06:00Z", "updated_at": "2017-11-16T00:06:00Z", "author_association": "OWNER", "body": "Aha... it looks like this is a Jinja version problem: https://github.com/ansible/ansible/issues/25381#issuecomment-306492389\r\n\r\nDatasette depends on sanic-jinja2 - and that doesn't depend on a particular jinja2 version: https://github.com/lixxu/sanic-jinja2/blob/7e9520850d8c6bb66faf43b7f252593d7efe3452/setup.py#L22\r\n\r\nSo if you have an older version of Jinja installed, stuff breaks.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 274160723, "label": "TemplateAssertionError: no filter named 'tojson'"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705889120", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705889120, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTg4OTEyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-08T23:59:01Z", "updated_at": "2020-10-08T23:59:01Z", "author_association": "OWNER", "body": "Needs tests and documentation.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705899629", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705899629, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTg5OTYyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T00:37:02Z", "updated_at": "2020-10-09T00:37:02Z", "author_association": "OWNER", "body": "I'm going to route the existing `TestClient` through this mechanism to exercise it during the tests.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705902902", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705902902, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkwMjkwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T00:50:49Z", "updated_at": "2020-10-09T00:50:49Z", "author_association": "OWNER", "body": "Almost all of the tests are passing:\r\n```\r\n=========================== short test summary info ============================\r\nFAILED tests/test_api.py::test_table_with_slashes_in_name - assert 404 == 200\r\nFAILED tests/test_api.py::test_row_strange_table_name - assert 404 == 200\r\nFAILED tests/test_html.py::test_row_strange_table_name_with_url_hash - assert...\r\nFAILED tests/test_html.py::test_css_classes_on_body[/fixtures/table%2Fwith%2Fslashes.csv-expected_classes5]\r\nFAILED tests/test_html.py::test_templates_considered[/fixtures/table%2Fwith%2Fslashes.csv-table-fixtures-tablewithslashescsv-fa7563.html, *table.html]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys/a,a,a-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/paginated_view-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/facetable-https://example.com/]\r\nFAILED tests/test_messages.py::test_messages_are_displayed_and_cleared - KeyE...\r\nFAILED tests/test_plugins.py::test_hook_register_magic_parameters - Assertion...\r\n============ 11 failed, 718 passed, 6 warnings in 225.77s (0:03:45) ============\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705918844", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705918844, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkxODg0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:46:06Z", "updated_at": "2020-10-09T01:46:06Z", "author_association": "OWNER", "body": "For this failing test I'm suspicious that the AsyncClient may be persisting cookies in between requests:\r\n```\r\n def test_actor_cookie(app_client):\r\n \"A valid actor cookie sets request.scope['actor']\"\r\n cookie = app_client.actor_cookie({\"id\": \"test\"})\r\n response = app_client.get(\"/\", cookies={\"ds_actor\": cookie})\r\n> assert {\"id\": \"test\"} == app_client.ds._last_request.scope[\"actor\"]\r\nE AssertionError: assert {'id': 'test'} == {'id': 'root'}\r\nE Differing items:\r\nE {'id': 'test'} != {'id': 'root'}\r\nE Use -v to get the full diff\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705920055", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705920055, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkyMDA1NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:51:05Z", "updated_at": "2020-10-09T01:51:05Z", "author_association": "OWNER", "body": "The topic of disabling cookie persistence is discussed a little here: https://github.com/encode/httpx/issues/422#issuecomment-537906693\r\n\r\n> We have a cookiejar abstraction, I think setting it to an always-empty jar like you describe is best. :)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705920228", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705920228, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkyMDIyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:51:44Z", "updated_at": "2020-10-09T01:51:44Z", "author_association": "OWNER", "body": "I'm going to switch back to having each request run through a new client. I'm worried about the impact on test performance though. I'll run a microbenchmark before and after.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705921006", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705921006, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkyMTAwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:55:01Z", "updated_at": "2020-10-09T01:55:01Z", "author_association": "OWNER", "body": "With the single client that is reused for all tests:\r\n```\r\n% time pytest tests/test_api.py\r\n...\r\n6.73s user 9.91s system 81% cpu 20.365 total\r\n```\r\nAfter switching back to this class:\r\n```python\r\nclass DatasetteClient:\r\n def __init__(self, ds):\r\n self.app = ds.app()\r\n\r\n def _fix(self, path):\r\n if path.startswith(\"/\"):\r\n path = \"http://localhost{}\".format(path)\r\n return path\r\n\r\n async def get(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.get(self._fix(path), **kwargs)\r\n\r\n async def options(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.options(self._fix(path), **kwargs)\r\n\r\n async def head(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.head(self._fix(path), **kwargs)\r\n\r\n async def post(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.post(self._fix(path), **kwargs)\r\n\r\n async def put(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.put(self._fix(path), **kwargs)\r\n\r\n async def patch(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.patch(self._fix(path), **kwargs)\r\n\r\n async def delete(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.delete(self._fix(path), **kwargs)\r\n\r\n async def request(self, method, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.request(method, self._fix(path), **kwargs)\r\n```\r\nThe time taken is:\r\n```\r\n% time pytest tests/test_api.py\r\n...\r\n7.26s user 10.02s system 82% cpu 21.014 total\r\n```\r\nThat's close enough that I don't feel I need to investigate this further.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705926035", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705926035, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkyNjAzNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T02:14:14Z", "updated_at": "2020-10-09T02:14:14Z", "author_association": "OWNER", "body": "Still need to handle these six failing tests:\r\n```\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys-https://example.com/] - AssertionError: {'base_url': 'https://example.com/', 'elemen...\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys/a,a,a-https://example.com/] - AssertionError: {'base_url': 'https://example.com/', '...\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/paginated_view-https://example.com/] - AssertionError: {'base_url': 'https://example.com/', 'element_parent': '<...\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/facetable-https://example.com/] - AssertionError: {'base_url': 'https://example.com/', 'element_parent': '

FAILED tests/test_messages.py::test_messages_are_displayed_and_cleared - KeyError: 'ds_messages'\r\n\r\nThat one is caused by `response.cookies` skipping cookies that were set to the empty string. Same fix as this: https://github.com/simonw/datasette/blob/a1687351fb75b01f737fda4ad07e0781029de05c/tests/test_auth.py#L90-L95", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705937696", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705937696, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkzNzY5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T02:52:53Z", "updated_at": "2020-10-09T02:52:53Z", "author_association": "OWNER", "body": "These failures are giving me a severe \"how did this ever work in the first place?\" vibe:\r\n```\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys/a,a,a-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/paginated_view-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/facetable-https://example.com/]\r\n```\r\nI have a fix for them, no idea why they weren't already failing though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705940507", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705940507, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTk0MDUwNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T03:04:15Z", "updated_at": "2020-10-09T03:04:15Z", "author_association": "OWNER", "body": "This is really weird: new set of test failures that I wasn't seeing before, and those tests aren't failing on my laptop:\r\n```\r\n=========================== short test summary info ============================\r\nFAILED tests/test_api.py::test_table_with_slashes_in_name - assert 404 == 200\r\nFAILED tests/test_api.py::test_row_strange_table_name - assert 404 == 200\r\nFAILED tests/test_html.py::test_row_strange_table_name_with_url_hash - assert...\r\nFAILED tests/test_html.py::test_css_classes_on_body[/fixtures/table%2Fwith%2Fslashes.csv-expected_classes5]\r\nFAILED tests/test_html.py::test_templates_considered[/fixtures/table%2Fwith%2Fslashes.csv-table-fixtures-tablewithslashescsv-fa7563.html, *table.html]\r\n================== 5 failed, 738 passed in 194.73s (0:03:14) ===================\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705941580", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705941580, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTk0MTU4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T03:08:43Z", "updated_at": "2020-10-09T03:08:43Z", "author_association": "OWNER", "body": "Most likely reason for those failures is that `path` and `raw_path` are not being simulated correctly. I used to do those here:\r\n\r\nhttps://github.com/simonw/datasette/blob/402cf870b7d65f9b5fba9e23aa99433294bd4523/datasette/utils/testing.py#L116-L125\r\n\r\nBut now I'm delegating that to `httpx` to handle.\r\n\r\nWEIRD that it passes on my laptop but fails in GitHub Actions CI though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705945591", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705945591, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTk0NTU5MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T03:24:48Z", "updated_at": "2020-10-09T03:24:48Z", "author_association": "OWNER", "body": "I'm testing this with a `print(scope)` and `pytest -k test_table_with_slashes_in_name -s`.\r\n\r\nAgainst the `main` branch:\r\n\r\n`{'type': 'http', 'http_version': '1.0', 'method': 'GET', 'path': '/fixtures/table/with/slashes.csv', 'raw_path': b'/fixtures/table%2Fwith%2Fslashes.csv', 'query_string': b'_shape=objects&_format=json', 'headers': [[b'host', b'localhost']], 'csrftoken': ._asgi_csrf_decorator..app_wrapped_with_csrf..get_csrftoken at 0x10e2e6040>}`\r\n\r\nAgainst the broken branch:\r\n\r\n`tests/test_api.py {'type': 'http', 'asgi': {'version': '3.0'}, 'http_version': '1.1', 'method': 'GET', 'headers': [(b'host', b'localhost'), (b'accept', b'*/*'), (b'accept-encoding', b'gzip, deflate'), (b'connection', b'keep-alive'), (b'user-agent', b'python-httpx/0.15.0')], 'scheme': 'http', 'path': '/fixtures/table%2Fwith%2Fslashes.csv', 'query_string': b'_shape=objects&_format=json', 'server': ('localhost', None), 'client': ('127.0.0.1', 123), 'root_path': '', 'csrftoken': ._asgi_csrf_decorator..app_wrapped_with_csrf..get_csrftoken at 0x109e0eca0>}`\r\n\r\nThis is on my laptop though so both of those pass the tests.\r\n\r\nKey difference: the `httpx` version doesn't set a `raw_path` at all. BUT.. it does set `path` and sets it to `'/fixtures/table%2Fwith%2Fslashes.csv'`\r\n\r\nThe non-httpx version sets `raw_path` to `b'/fixtures/table%2Fwith%2Fslashes.csv'` and `path` to `'/fixtures/table/with/slashes.csv'`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705946120", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705946120, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTk0NjEyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T03:27:05Z", "updated_at": "2020-10-09T03:27:05Z", "author_association": "OWNER", "body": "I may need to fuss around with how the `httpx` client sends things to the ASGI app.\r\n\r\nhttps://github.com/encode/httpx/blob/92ca4d0cc654859fc2257c492e55d8752370d427/httpx/_transports/asgi.py#L26 is relevant:\r\n\r\n Alternatively, you can setup the transport instance explicitly.\r\n This allows you to include any additional configuration arguments specific\r\n to the ASGITransport class:\r\n ```\r\n transport = httpx.ASGITransport(\r\n app=app,\r\n root_path=\"/submount\",\r\n client=(\"1.2.3.4\", 123)\r\n )\r\n client = httpx.AsyncClient(transport=transport)\r\n ```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705946360", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705946360, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTk0NjM2MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T03:28:08Z", "updated_at": "2020-10-09T03:28:08Z", "author_association": "OWNER", "body": "Here's where `httpx` sets up the ASGI scope: https://github.com/encode/httpx/blob/92ca4d0cc654859fc2257c492e55d8752370d427/httpx/_transports/asgi.py#L82-L97\r\n\r\n```python\r\n # ASGI scope.\r\n scheme, host, port, full_path = url\r\n path, _, query = full_path.partition(b\"?\")\r\n scope = {\r\n \"type\": \"http\",\r\n \"asgi\": {\"version\": \"3.0\"},\r\n \"http_version\": \"1.1\",\r\n \"method\": method.decode(),\r\n \"headers\": [(k.lower(), v) for (k, v) in headers],\r\n \"scheme\": scheme.decode(\"ascii\"),\r\n \"path\": unquote(path.decode(\"ascii\")),\r\n \"query_string\": query,\r\n \"server\": (host.decode(\"ascii\"), port),\r\n \"client\": self.client,\r\n \"root_path\": self.root_path,\r\n }\r\n```\r\nSure enough, it doesn't set the `raw_path`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-706263157", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 706263157, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjI2MzE1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T15:57:15Z", "updated_at": "2020-10-09T15:57:15Z", "author_association": "OWNER", "body": "My `httpx` pull request adding `raw_path` support was just merged: https://github.com/encode/httpx/pull/1357 - but it's not in a release yet.\r\n\r\nI'm going to mark these tests as `xfail` so I can land this change - I'll remove that once an `httpx` release comes out that I can use to get the tests passing.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-706269271", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 706269271, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjI2OTI3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T16:08:49Z", "updated_at": "2020-10-09T16:08:49Z", "author_association": "OWNER", "body": "I'm going to document this in a separate issue.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1001#issuecomment-705904566", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1001", "id": 705904566, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkwNDU2Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T00:58:08Z", "updated_at": "2020-10-09T00:58:08Z", "author_association": "OWNER", "body": "To get a traceback:\r\n```\r\ndatasette . -p 8009 --pdb\r\n```\r\nAnd then:\r\n```\r\ncurl -XOPTIONS http://127.0.0.1:8009\r\n```\r\nThis causes the server to open a debugging prompt:\r\n```\r\nINFO: 127.0.0.1:59514 - \"OPTIONS / HTTP/1.1\" 500 Internal Server Error\r\n> /Users/simon/Dropbox/Development/datasette/datasette/views/base.py(115)dispatch_request()\r\n-> return await handler(request, *args, **kwargs)\r\n(Pdb) list\r\n110 \t def database_color(self, database):\r\n111 \t return \"ff0000\"\r\n112 \t\r\n113 \t async def dispatch_request(self, request, *args, **kwargs):\r\n114 \t handler = getattr(self, request.method.lower(), None)\r\n115 ->\t return await handler(request, *args, **kwargs)\r\n116 \t\r\n117 \t async def render(self, templates, request, context=None):\r\n118 \t context = context or {}\r\n119 \t template = self.ds.jinja_env.select_template(templates)\r\n120 \t template_context = {\r\n(Pdb) \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717768441, "label": "OPTIONS requests return a 500 error"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1001#issuecomment-705904679", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1001", "id": 705904679, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkwNDY3OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T00:58:32Z", "updated_at": "2020-10-09T00:58:32Z", "author_association": "OWNER", "body": "So the bug is in this code here: https://github.com/simonw/datasette/blob/703439bdc37e724b01bc6d7a1fc1d955795132f2/datasette/views/base.py#L113-L115", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717768441, "label": "OPTIONS requests return a 500 error"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1001#issuecomment-705904759", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1001", "id": 705904759, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkwNDc1OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T00:58:47Z", "updated_at": "2020-10-09T00:58:47Z", "author_association": "OWNER", "body": "What should an OPTIONS request return, anyway?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717768441, "label": "OPTIONS requests return a 500 error"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1001#issuecomment-705904917", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1001", "id": 705904917, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkwNDkxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T00:59:25Z", "updated_at": "2020-10-09T00:59:25Z", "author_association": "OWNER", "body": "```\r\n~ % curl -XOPTIONS https://www.google.com/\r\n\r\n\r\n \r\n \r\n Error 405 (Method Not Allowed)!!1\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717768441, "label": "OPTIONS requests return a 500 error"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1001#issuecomment-705905121", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1001", "id": 705905121, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkwNTEyMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:00:07Z", "updated_at": "2020-10-09T01:00:07Z", "author_association": "OWNER", "body": "The www.djangoproject.com site returns an empty page:\r\n```\r\n~ % curl -vv -XOPTIONS https://www.djangoproject.com/\r\n* Trying 151.101.42.217:443...\r\n* Connected to www.djangoproject.com (151.101.42.217) port 443 (#0)\r\n* ALPN, offering http/1.1\r\n* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256\r\n* Server certificate: osff2.map.fastly.net\r\n* Server certificate: GlobalSign CloudSSL CA - SHA256 - G3\r\n* Server certificate: GlobalSign Root CA\r\n> OPTIONS / HTTP/1.1\r\n> Host: www.djangoproject.com\r\n> User-Agent: curl/7.70.0\r\n> Accept: */*\r\n> \r\n* Mark bundle as not supporting multiuse\r\n< HTTP/1.1 200 OK\r\n< Connection: keep-alive\r\n< Content-Length: 0\r\n< Server: nginx\r\n< Content-Type: text/html; charset=utf-8\r\n< Allow: GET, HEAD, OPTIONS\r\n< Content-Language: en\r\n< X-Frame-Options: SAMEORIGIN\r\n< X-Content-Type-Options: nosniff\r\n< X-XSS-Protection: 1; mode=block\r\n< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload\r\n< Access-Control-Allow-Origin: https://code.djangoproject.com\r\n< Accept-Ranges: bytes\r\n< Date: Fri, 09 Oct 2020 00:59:42 GMT\r\n< Via: 1.1 varnish\r\n< X-Served-By: cache-sjc10047-SJC\r\n< X-Cache: MISS\r\n< X-Cache-Hits: 0\r\n< X-Timer: S1602205182.833493,VS0,VE305\r\n< Vary: Accept-Language, Accept-Encoding\r\n< \r\n* Connection #0 to host www.djangoproject.com left intact\r\n~ % \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717768441, "label": "OPTIONS requests return a 500 error"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1001#issuecomment-705905418", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1001", "id": 705905418, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkwNTQxOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:00:54Z", "updated_at": "2020-10-09T01:00:54Z", "author_association": "OWNER", "body": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717768441, "label": "OPTIONS requests return a 500 error"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1001#issuecomment-705916614", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1001", "id": 705916614, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkxNjYxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:37:06Z", "updated_at": "2020-10-09T01:37:06Z", "author_association": "OWNER", "body": "I'm tempted to imitate Django Rest Framework here: https://github.com/encode/django-rest-framework/blob/2e721cdbc85a924d0b0f093b86fe1497b58fe287/rest_framework/views.py#L514-L521\r\n```python\r\n def options(self, request, *args, **kwargs):\r\n \"\"\"\r\n Handler method for HTTP 'OPTIONS' request.\r\n \"\"\"\r\n if self.metadata_class is None:\r\n return self.http_method_not_allowed(request, *args, **kwargs)\r\n data = self.metadata_class().determine_metadata(request, self)\r\n return Response(data, status=status.HTTP_200_OK)\r\n```\r\nThat `determine_metadata()` default method does this: https://github.com/encode/django-rest-framework/blob/335054a5d36b352a58286b303b608b6bf48152f8/rest_framework/metadata.py#L29\r\n\r\nNote the comment at the top:\r\n```\r\n There are not any formalized standards for `OPTIONS` responses\r\n for us to base this on.\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717768441, "label": "OPTIONS requests return a 500 error"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1001#issuecomment-705917015", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1001", "id": 705917015, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkxNzAxNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:38:49Z", "updated_at": "2020-10-09T01:38:49Z", "author_association": "OWNER", "body": "I actually have a sensible `OPTIONS` implementation here:\r\n\r\nhttps://github.com/simonw/datasette/blob/a648bb82bac201c7658f6fdb499ff8ac17ebd2e8/datasette/views/base.py#L154-L165\r\n\r\nI'm going to set the default one to return a 405 (Method Not Allowed) like Google does.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717768441, "label": "OPTIONS requests return a 500 error"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1002#issuecomment-706306214", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1002", "id": 706306214, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjMwNjIxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T17:23:51Z", "updated_at": "2020-10-09T17:23:51Z", "author_association": "OWNER", "body": "I can start by combining the release notes for the 0.50 alphas.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717783692, "label": "Release notes for Datasette 0.50"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1003#issuecomment-706272322", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1003", "id": 706272322, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjI3MjMyMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T16:14:56Z", "updated_at": "2020-10-09T16:14:56Z", "author_association": "OWNER", "body": "Yes I think that makes sense. I added `json` to the template context in Dogsheep Beta just a few days ago because I needed that: https://github.com/dogsheep/dogsheep-beta/blob/bed9df2b3ef68189e2e445427721a28f4e9b4887/dogsheep_beta/__init__.py#L176", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718238967, "label": "from_json jinja2 filter"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1003#issuecomment-706273211", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1003", "id": 706273211, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjI3MzIxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T16:16:38Z", "updated_at": "2020-10-09T16:16:38Z", "author_association": "OWNER", "body": "I'm not a huge fan of `from_json` as the name for this. Some other options:\r\n\r\n- Expose `json` directly so templates can do `json.loads()` and `json.dumps()` - this allows for outputting JSON too, which is useful. But is there anything else on the `json` module that shouldn't be exposed in templates?\r\n- `json_dumps()` and `json_loads()` template functions. I quite like that.\r\n- Something else?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718238967, "label": "from_json jinja2 filter"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1003#issuecomment-706281451", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1003", "id": 706281451, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjI4MTQ1MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T16:33:01Z", "updated_at": "2020-10-09T16:33:01Z", "author_association": "OWNER", "body": "I think `json_dumps()` and `json_loads()` as aliases for `json.dumps()` and `json.loads()` is the way to go here.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718238967, "label": "from_json jinja2 filter"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1005#issuecomment-706473306", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1005", "id": 706473306, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjQ3MzMwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T02:45:46Z", "updated_at": "2020-10-10T02:45:46Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/blob/9f6dd985bc0eff70f8a9ce65c6578bc43d2e172b/tests/test_html.py#L145-L149\r\n\r\nhttps://github.com/simonw/datasette/blob/9f6dd985bc0eff70f8a9ce65c6578bc43d2e172b/tests/test_html.py#L539-L564\r\n\r\nhttps://github.com/simonw/datasette/blob/8f97b9b58e77f82fef1f10e9c9f6754b993544b6/tests/test_api.py#L1190-L1194\r\n\r\nhttps://github.com/simonw/datasette/blob/8f97b9b58e77f82fef1f10e9c9f6754b993544b6/tests/test_api.py#L742-L746", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718259202, "label": "Remove xfail tests when new httpx is released"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1005#issuecomment-747209115", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1005", "id": 747209115, "node_id": "MDEyOklzc3VlQ29tbWVudDc0NzIwOTExNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-12-17T05:11:04Z", "updated_at": "2020-12-17T05:11:04Z", "author_association": "OWNER", "body": "Tracking ticket for the next HTTPX release is https://github.com/encode/httpx/pull/1403", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718259202, "label": "Remove xfail tests when new httpx is released"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1005#issuecomment-787536267", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1005", "id": 787536267, "node_id": "MDEyOklzc3VlQ29tbWVudDc4NzUzNjI2Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-28T22:30:37Z", "updated_at": "2021-02-28T22:30:37Z", "author_association": "OWNER", "body": "It's out! https://github.com/encode/httpx/releases/tag/0.17.0", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718259202, "label": "Remove xfail tests when new httpx is released"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1006#issuecomment-706270877", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1006", "id": 706270877, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjI3MDg3Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T16:12:09Z", "updated_at": "2020-10-09T16:12:09Z", "author_association": "OWNER", "body": "This can become a section on https://docs.datasette.io/en/stable/internals.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718264811, "label": "Documentation for datasette.client"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1006#issuecomment-706305601", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1006", "id": 706305601, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjMwNTYwMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T17:22:31Z", "updated_at": "2020-10-09T17:22:31Z", "author_association": "OWNER", "body": "https://docs.datasette.io/en/latest/internals.html#client", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718264811, "label": "Documentation for datasette.client"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1007#issuecomment-706276831", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1007", "id": 706276831, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjI3NjgzMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T16:23:51Z", "updated_at": "2020-10-09T16:23:51Z", "author_association": "OWNER", "body": "I don't appear to be using these anywhere, not sure why I spotted a warning (which I now can't find).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718272593, "label": "set-env and add-path commands have been deprecated"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1010#issuecomment-706450418", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1010", "id": 706450418, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjQ1MDQxOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T00:08:27Z", "updated_at": "2020-10-10T00:09:13Z", "author_association": "OWNER", "body": "This commit broke it: https://github.com/simonw/datasette/commit/8f97b9b58e77f82fef1f10e9c9f6754b993544b6#diff-ed4bb3a97b7efa4d45be6a6ea6f4636cL114-R165\r\n\r\n\"Banners_and_Alerts_and_datasette_client_internal_requests_mechanism_\u00b7_simonw_datasette_8f97b9b\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718484082, "label": "json / CSV links are broken in Datasette 0.50"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1010#issuecomment-706455202", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1010", "id": 706455202, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjQ1NTIwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T00:33:46Z", "updated_at": "2020-10-10T00:33:46Z", "author_association": "OWNER", "body": "Since this is an urgent fix that needs to go out I've marked the `base_url` tests as `xfail`. I need to solve those separately, see #900 and various other bug reports.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718484082, "label": "json / CSV links are broken in Datasette 0.50"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1010#issuecomment-706455505", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1010", "id": 706455505, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjQ1NTUwNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T00:35:41Z", "updated_at": "2020-10-10T00:35:41Z", "author_association": "OWNER", "body": "I missed the links on the query page: https://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+pk3%2C+content+from+compound_three_primary_keys+order+by+pk1%2C+pk2%2C+pk3+limit+101", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718484082, "label": "json / CSV links are broken in Datasette 0.50"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1011#issuecomment-706480763", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1011", "id": 706480763, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjQ4MDc2Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T03:42:58Z", "updated_at": "2020-10-10T03:42:58Z", "author_association": "OWNER", "body": "Oh no! It's the same bug as #1010. Thanks for the report, I shall push out 0.50.2 as quickly as possible.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718521469, "label": "column name links broken in 0.50.1"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1011#issuecomment-706480791", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1011", "id": 706480791, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjQ4MDc5MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T03:43:13Z", "updated_at": "2020-10-10T03:43:13Z", "author_association": "OWNER", "body": "Demo: click column headers on https://latest.datasette.io/fixtures/compound_three_primary_keys", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718521469, "label": "column name links broken in 0.50.1"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1011#issuecomment-706482542", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1011", "id": 706482542, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjQ4MjU0Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T03:58:58Z", "updated_at": "2020-10-10T03:58:58Z", "author_association": "OWNER", "body": "The fix is live on the demo: https://latest.datasette.io/fixtures/compound_three_primary_keys?_sort_desc=pk2", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718521469, "label": "column name links broken in 0.50.1"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1011#issuecomment-706483680", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1011", "id": 706483680, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjQ4MzY4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T04:09:32Z", "updated_at": "2020-10-10T04:09:32Z", "author_association": "OWNER", "body": "OK, Datasette 0.50.2 is live on PyPI now.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718521469, "label": "column name links broken in 0.50.1"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1012#issuecomment-969602825", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1012", "id": 969602825, "node_id": "IC_kwDOBm6k_c45yvcJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-16T01:21:14Z", "updated_at": "2021-11-16T01:21:14Z", "author_association": "OWNER", "body": "I'd been wondering how to get new classifiers into Trove - thanks, I'll give this a go.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718540751, "label": "For 1.0 update trove classifier in setup.py"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1012#issuecomment-969613166", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1012", "id": 969613166, "node_id": "IC_kwDOBm6k_c45yx9u", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-16T01:27:25Z", "updated_at": "2021-11-16T01:27:25Z", "author_association": "OWNER", "body": "Requested here:\r\n\r\n- https://github.com/pypa/trove-classifiers/pull/85", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718540751, "label": "For 1.0 update trove classifier in setup.py"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1014#issuecomment-706626796", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1014", "id": 706626796, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjYyNjc5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T23:47:23Z", "updated_at": "2020-10-10T23:48:26Z", "author_association": "OWNER", "body": "The GitHub API has these:\r\n\r\n`link: ; rel=\"next\", ; rel=\"last\"`\r\n\r\nWordPress API: https://css-tricks.com/wp-json/wp/v2/posts\r\n\r\n`link: ; rel=\"next\"`\r\n\r\nIt also has these:\r\n\r\n```\r\nx-wp-total: 5162\r\nx-wp-totalpages: 517\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718723543, "label": "Add Link: pagination HTTP headers"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1014#issuecomment-706626913", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1014", "id": 706626913, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjYyNjkxMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T23:49:08Z", "updated_at": "2020-10-10T23:49:08Z", "author_association": "OWNER", "body": "This should be documented on https://docs.datasette.io/en/stable/json_api.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718723543, "label": "Add Link: pagination HTTP headers"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1014#issuecomment-706626934", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1014", "id": 706626934, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjYyNjkzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T23:49:25Z", "updated_at": "2020-10-10T23:49:25Z", "author_association": "OWNER", "body": "I'm just going to implement the 'next' one.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718723543, "label": "Add Link: pagination HTTP headers"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1014#issuecomment-706626971", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1014", "id": 706626971, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjYyNjk3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T23:49:57Z", "updated_at": "2020-10-10T23:49:57Z", "author_association": "OWNER", "body": "I won't offer a total-number header. That's expensive to calculate - no need to calculate it unless it's explicitly asked for.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718723543, "label": "Add Link: pagination HTTP headers"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1014#issuecomment-706627035", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1014", "id": 706627035, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjYyNzAzNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-10T23:50:40Z", "updated_at": "2020-10-10T23:50:40Z", "author_association": "OWNER", "body": "First example page to add this to: `?_shape=array`, e.g. https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718723543, "label": "Add Link: pagination HTTP headers"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1014#issuecomment-706631006", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1014", "id": 706631006, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjYzMTAwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-11T00:36:43Z", "updated_at": "2020-10-11T00:36:43Z", "author_association": "OWNER", "body": "Demo using [paginate-json](https://github.com/simonw/paginate-json):\r\n```\r\n% paginate-json 'https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array' | jq '. | length' \r\nhttps://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array\r\nhttp://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cd%2Cv\r\nhttp://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Ch%2Cr\r\nhttp://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cl%2Cn\r\nhttp://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cp%2Cj\r\nhttp://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Ct%2Cf\r\nhttp://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cx%2Cb\r\nhttp://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Ca%2Cx\r\nhttp://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Ce%2Ct\r\nhttp://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Ci%2Cp\r\nhttp://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Cm%2Cl\r\n1001\r\n```\r\nNew documentation: https://docs.datasette.io/en/latest/json_api.html#pagination", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718723543, "label": "Add Link: pagination HTTP headers"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1015#issuecomment-706756879", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1015", "id": 706756879, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjc1Njg3OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-11T19:35:03Z", "updated_at": "2020-10-11T19:35:03Z", "author_association": "OWNER", "body": "Since plugins are installed via pip this would require Datasette to be restarted. This StackOverflow thread looks relevant to that: https://stackoverflow.com/questions/11329917/restart-python-script-from-within-itself\r\n\r\nThis recipe looks promising:\r\n```python\r\nimport os\r\nimport sys\r\nimport psutil\r\nimport logging\r\n\r\ndef restart_program():\r\n \"\"\"Restarts the current program, with file objects and descriptors\r\n cleanup\r\n \"\"\"\r\n\r\n try:\r\n p = psutil.Process(os.getpid())\r\n for handler in p.get_open_files() + p.connections():\r\n os.close(handler.fd)\r\n except Exception, e:\r\n logging.error(e)\r\n\r\n python = sys.executable\r\n os.execl(python, python, *sys.argv)\r\n```\r\nhttps://docs.python.org/3/library/os.html#os.execl says about `os.execl`:\r\n\r\n> These functions all execute a new program, replacing the current process; they do not return. On Unix, the new executable is loaded into the current process, and will have the same process id as the caller\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718910318, "label": "Research: could Datasette install its own plugins?"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1016#issuecomment-706788010", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1016", "id": 706788010, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjc4ODAxMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-11T23:50:39Z", "updated_at": "2020-10-11T23:50:39Z", "author_association": "OWNER", "body": "For consistency can reuse the icon used on selected facets:\r\n\r\n\"evernote__notes__8_rows_where_where_application-data____ACD4B543-4663-40B2-BAF5-666FA54D4204__and_source-url_is_not_blank_sorted_by_id\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718953669, "label": "Add a \"delete\" icon next to filters (in addition to \"remove filter\")"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1016#issuecomment-706821561", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1016", "id": 706821561, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjgyMTU2MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-12T02:33:52Z", "updated_at": "2020-10-12T02:33:52Z", "author_association": "OWNER", "body": "Should this do the equivalent of clicking Apply for you? I don't think so - that wouldn't fit with how the rest of that block of controls works.\r\n\r\nBut if it makes the row vanish it may not be clear to the user that they still need to click Apply.\r\n\r\nEasiest thing would be for it to select the \"remove filter\" column and blank out the boxes. I will try that first.\r\n\r\nThe code can go in `table.js`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718953669, "label": "Add a \"delete\" icon next to filters (in addition to \"remove filter\")"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1016#issuecomment-706828375", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1016", "id": 706828375, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjgyODM3NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-12T03:01:58Z", "updated_at": "2020-10-12T03:01:58Z", "author_association": "OWNER", "body": "Demo: https://latest.datasette.io/fixtures/facetable?_facet=created&created=2019-01-17+08%3A00%3A00&_facet=state&state=MI\r\n\r\n![x-button](https://user-images.githubusercontent.com/9599/95700744-9b4b4e80-0bfc-11eb-85a3-6da82d9e0cbf.gif)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718953669, "label": "Add a \"delete\" icon next to filters (in addition to \"remove filter\")"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1019#issuecomment-708113989", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1019", "id": 708113989, "node_id": "MDEyOklzc3VlQ29tbWVudDcwODExMzk4OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-14T02:22:40Z", "updated_at": "2020-10-14T02:22:40Z", "author_association": "OWNER", "body": "I've found myself wanting this too.\r\n\r\nIt should respect the allow-sql permission so it doesn't display if the user isn't able to run custom SQL.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721050815, "label": "\"Edit SQL\" button on canned queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1019#issuecomment-708114636", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1019", "id": 708114636, "node_id": "MDEyOklzc3VlQ29tbWVudDcwODExNDYzNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-14T02:24:56Z", "updated_at": "2020-10-14T02:24:56Z", "author_association": "OWNER", "body": "So it would appear on this page: https://latest.datasette.io/fixtures/neighborhood_search#fragment-goes-here\r\n\r\nProbably as a blue \"Edit SQL\" link next to those buttons.\r\n\r\n![575FB160-4F75-43B2-B2AB-D75E60F10477](https://user-images.githubusercontent.com/9599/95936143-c2835680-0d89-11eb-9cf3-06f86d923103.jpeg)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721050815, "label": "\"Edit SQL\" button on canned queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1019#issuecomment-708127294", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1019", "id": 708127294, "node_id": "MDEyOklzc3VlQ29tbWVudDcwODEyNzI5NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-14T03:08:13Z", "updated_at": "2020-10-14T03:08:13Z", "author_association": "OWNER", "body": "Maybe like this:\r\n\r\n\"fixtures__select_neighborhood__facet_cities_name__state_from_facetable_join_facet_cities_on_facetable_city_id___facet_cities_id_where_neighborhood_like_________text________order_by_neighborhood_\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721050815, "label": "\"Edit SQL\" button on canned queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1019#issuecomment-708128286", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1019", "id": 708128286, "node_id": "MDEyOklzc3VlQ29tbWVudDcwODEyODI4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-14T03:11:33Z", "updated_at": "2020-10-14T03:14:07Z", "author_association": "OWNER", "body": "Another edge-case: https://latest.datasette.io/fixtures/pragma_cache_size (`PRAGMA cache_size;`) isn't an allowed query usually, so linking to \"Edit SQL\" for it would link to an error page: https://latest.datasette.io/fixtures?sql=PRAGMA+cache_size%3B\r\n\r\nCan use `datasette.utils.validate_sql_select(sql)` to check for that - it raises `datasette.utils.InvalidSql` if there's a problem.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721050815, "label": "\"Edit SQL\" button on canned queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1019#issuecomment-708130290", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1019", "id": 708130290, "node_id": "MDEyOklzc3VlQ29tbWVudDcwODEzMDI5MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-14T03:17:57Z", "updated_at": "2020-10-14T03:17:57Z", "author_association": "OWNER", "body": "One last edge-case (I think) - magic parameters aren't supported for non-canned-queries, so if a query includes those it shouldn't show an Edit SQL link: https://latest.datasette.io/fixtures/magic_parameters\r\n\r\nI don't have a clean way of detecting if a query contains defined magic parameters, but I can instead do a dumb substring match for `:_` and skip showing the Edit SQL link if that is present.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721050815, "label": "\"Edit SQL\" button on canned queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1019#issuecomment-708139822", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1019", "id": 708139822, "node_id": "MDEyOklzc3VlQ29tbWVudDcwODEzOTgyMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-14T03:53:21Z", "updated_at": "2020-10-14T03:53:21Z", "author_association": "OWNER", "body": "Demos:\r\n- https://latest.datasette.io/fixtures/neighborhood_search has the link\r\n- https://latest.datasette.io/fixtures/neighborhood_search?text=ber has the link, and the link passes through the parameter that has already been entered\r\n- https://latest.datasette.io/fixtures/%F0%9D%90%9C%F0%9D%90%A2%F0%9D%90%AD%F0%9D%90%A2%F0%9D%90%9E%F0%9D%90%AC has the link\r\n- https://latest.datasette.io/fixtures/pragma_cache_size does not have the link (`PRAGMA` is not allowed in regular editable queries)\r\n- https://latest.datasette.io/fixtures/magic_parameters does not have the link (magic parameters)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721050815, "label": "\"Edit SQL\" button on canned queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/102#issuecomment-754192267", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/102", "id": 754192267, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDE5MjI2Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T20:13:19Z", "updated_at": "2021-01-04T20:13:19Z", "author_association": "OWNER", "body": "I'm more likely to do Lambda than Elastic Beanstalk, especially now the size limit for Lambdas has been increased as part of their support for Docker.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 274264175, "label": "datasette publish elasticbeanstalk"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1020#issuecomment-708669178", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1020", "id": 708669178, "node_id": "MDEyOklzc3VlQ29tbWVudDcwODY2OTE3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-14T21:26:37Z", "updated_at": "2020-10-14T21:26:37Z", "author_association": "OWNER", "body": "One option: add an optional request=... parameter which can be passed the current request, and will use that to populate the mock request with the exception of the bits that are passed explicitly (like the path):\r\n\r\n```python\r\nresponse = await datasette.client.get(\"/db/table.json\", request=request)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721068929, "label": "Method for datasette.client() to forward on authentication"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1020#issuecomment-708669778", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1020", "id": 708669778, "node_id": "MDEyOklzc3VlQ29tbWVudDcwODY2OTc3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-14T21:27:58Z", "updated_at": "2020-10-14T21:27:58Z", "author_association": "OWNER", "body": "Maybe these internal requests should have some kind of flag that lets the underlying code tell that it's being called internally.\r\n\r\nOne option: add a `x-internal: 1` request header - and ensure that any requests from outside Datasette have that header stripped.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721068929, "label": "Method for datasette.client() to forward on authentication"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1020#issuecomment-708670392", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1020", "id": 708670392, "node_id": "MDEyOklzc3VlQ29tbWVudDcwODY3MDM5Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-14T21:29:22Z", "updated_at": "2020-10-14T21:29:22Z", "author_association": "OWNER", "body": "I should also verify (and probably unit-test) that things like the `?_trace=1` mechanism work across the internal request boundary.\r\n\r\n`/-/permissions` appears to work across this boundary, but again a test would be useful confirmation.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721068929, "label": "Method for datasette.client() to forward on authentication"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1020#issuecomment-712481568", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1020", "id": 712481568, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjQ4MTU2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-19T22:41:59Z", "updated_at": "2020-10-19T22:41:59Z", "author_association": "OWNER", "body": "It turns out this works just fine:\r\n```python\r\nresponse = await datasette.client.get(path, cookies=request.cookies)\r\n```\r\nSo I don't need a mechanism for this. I'm going to add this to the documentation instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721068929, "label": "Method for datasette.client() to forward on authentication"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1020#issuecomment-712482015", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1020", "id": 712482015, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjQ4MjAxNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-19T22:43:24Z", "updated_at": "2020-10-19T22:43:24Z", "author_association": "OWNER", "body": "... unless I want to support authentication mechanisms that work based on incoming IP address instead, in which case there's an argument for copying more over from the incoming request.\r\n\r\nTailscale is a good example of a system where authentication based on IP address can actually work well, so this is worth doing. Also, there might be authentication mechanisms which work by setting a custom header on the incoming request (not to mention the `Authorization` header).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721068929, "label": "Method for datasette.client() to forward on authentication"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1020#issuecomment-712482504", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1020", "id": 712482504, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjQ4MjUwNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-19T22:45:01Z", "updated_at": "2020-10-19T22:45:01Z", "author_association": "OWNER", "body": "I'm having trouble coming up with the syntax for this. Here's one option:\r\n```python\r\nresponse = await datasette.client.get(path, request=request)\r\n```\r\nBut this feels confusing to me. We're not using the `request=` argument as a request - we're using it as a source of some default request values (the cookies and incoming headers, but not the path).\r\n\r\nWe're essentially combining that request with the other arguments passed to `.get()`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721068929, "label": "Method for datasette.client() to forward on authentication"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1022#issuecomment-708718475", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1022", "id": 708718475, "node_id": "MDEyOklzc3VlQ29tbWVudDcwODcxODQ3NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-14T23:46:38Z", "updated_at": "2020-10-14T23:46:38Z", "author_association": "OWNER", "body": "Good catch, thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 721830990, "label": "Fix table name in spatialite example command"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1023#issuecomment-712604541", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1023", "id": 712604541, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjYwNDU0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-20T05:39:44Z", "updated_at": "2020-10-20T05:39:44Z", "author_association": "OWNER", "body": "Here's the alpha with most of this work ready for people to preview: https://github.com/simonw/datasette/releases/tag/0.51a0", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722673818, "label": "Fix issues relating to base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1023#issuecomment-712607608", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1023", "id": 712607608, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjYwNzYwOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-20T05:47:42Z", "updated_at": "2020-10-20T05:47:42Z", "author_association": "OWNER", "body": "Requested alpha testers in https://github.com/simonw/datasette/issues/838#issuecomment-712604364", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722673818, "label": "Fix issues relating to base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1023#issuecomment-719986922", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1023", "id": 719986922, "node_id": "MDEyOklzc3VlQ29tbWVudDcxOTk4NjkyMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-31T20:51:01Z", "updated_at": "2020-10-31T20:51:01Z", "author_association": "OWNER", "body": "This should all be working correctly now.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722673818, "label": "Fix issues relating to base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1024#issuecomment-709589297", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1024", "id": 709589297, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTU4OTI5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T21:04:31Z", "updated_at": "2020-10-15T21:04:31Z", "author_association": "OWNER", "body": "I think nginx or Apache would be the best tools for this. I'm inclined to try with nginx first since I know it better.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722674708, "label": "Figure out how to run an environment that exercises the base_url proxy setting"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1024#issuecomment-709590337", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1024", "id": 709590337, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTU5MDMzNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T21:06:24Z", "updated_at": "2020-10-15T21:07:19Z", "author_association": "OWNER", "body": "From https://stackoverflow.com/questions/32549684/nginx-proxy-and-remove-proxy-pass-prefix/32550251 it looks like the config I should use is:\r\n```\r\nserver {\r\n listen 80;\r\n server_name example.com;\r\n\r\n location /datasette/ {\r\n proxy_pass http://127.0.0.1:8001;\r\n proxy_set_header X-Real-IP $remote_addr;\r\n proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\r\n proxy_set_header X-Forwarded-Proto $scheme;\r\n proxy_read_timeout 90;\r\n }\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722674708, "label": "Figure out how to run an environment that exercises the base_url proxy setting"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1024#issuecomment-709590941", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1024", "id": 709590941, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTU5MDk0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T21:07:47Z", "updated_at": "2020-10-15T21:07:47Z", "author_association": "OWNER", "body": "On macOS I ran `brew install nginx`. I'm going to try running it on port 8000 so I don't have to run it as root.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722674708, "label": "Figure out how to run an environment that exercises the base_url proxy setting"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1024#issuecomment-709595960", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1024", "id": 709595960, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTU5NTk2MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T21:18:14Z", "updated_at": "2020-10-15T21:18:14Z", "author_association": "OWNER", "body": "Typing `nginx` starts it running as a daemon listening on port `http-alt` aka 8080. It uses the config file from ` /usr/local/etc/nginx/nginx.conf`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722674708, "label": "Figure out how to run an environment that exercises the base_url proxy setting"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1024#issuecomment-709597589", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1024", "id": 709597589, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTU5NzU4OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T21:21:53Z", "updated_at": "2020-10-15T21:23:25Z", "author_association": "OWNER", "body": "Here's a recipe for running nginx against a custom config file: https://gist.github.com/simonw/35f0ebf9c1d6df158759\r\n\r\n```\r\ndaemon off;\r\n\r\nevents {\r\n worker_connections 1024;\r\n}\r\n\r\nhttp {\r\n access_log /dev/stdout;\r\n error_log /dev/stderr;\r\n\r\n types {\r\n text/html html htm shtml;\r\n text/css css;\r\n image/gif gif;\r\n image/jpeg jpeg jpg;\r\n application/javascript js;\r\n }\r\n\r\n server {\r\n listen 8002;\r\n index index.html;\r\n root app;\r\n }\r\n}\r\n```\r\n```\r\nnginx -p `pwd` -c `pwd`/nginx.conf\r\n```\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722674708, "label": "Figure out how to run an environment that exercises the base_url proxy setting"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1024#issuecomment-709598324", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1024", "id": 709598324, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTU5ODMyNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T21:23:33Z", "updated_at": "2020-10-15T21:26:55Z", "author_association": "OWNER", "body": "Combining these two examples, here's the config file I am going to use for this. I'll save this as `nginx.conf`:\r\n```\r\ndaemon off;\r\n\r\nevents {\r\n worker_connections 1024;\r\n}\r\n\r\nhttp {\r\n server {\r\n listen 8000;\r\n\r\n location /datasette {\r\n proxy_pass http://127.0.0.1:8001;\r\n proxy_set_header X-Real-IP $remote_addr;\r\n proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\r\n }\r\n }\r\n}\r\n```\r\nThen start the server with:\r\n```\r\nnginx -p `pwd` -c `pwd`/nginx.conf\r\n```\r\nAnd start Datasette like this:\r\n```\r\ndatasette fixtures.db --config base_url:/datasette/\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722674708, "label": "Figure out how to run an environment that exercises the base_url proxy setting"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1024#issuecomment-709600335", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1024", "id": 709600335, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTYwMDMzNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T21:28:02Z", "updated_at": "2020-10-15T22:25:43Z", "author_association": "OWNER", "body": "This is working OK so far:\r\n\r\n\"fixtures__compound_three_primary_keys__1_001_rows_where_sorted_by_pk2\"\r\n\r\nI'll try crawling it with `wget -r` to see if I get any errors.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722674708, "label": "Figure out how to run an environment that exercises the base_url proxy setting"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1024#issuecomment-709622973", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1024", "id": 709622973, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTYyMjk3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T22:27:31Z", "updated_at": "2020-10-15T22:27:31Z", "author_association": "OWNER", "body": "Here's how I tested it:\r\n```\r\ntime wget -r 'http://localhost:8000/datasette/' 2>&1 | grep -i -C 5 \"failed\\|error\" > /tmp/errors.txt\r\n```\r\nThis wrote out any errors (plus context) to the `errors.txt` log - and reported that the full crawl took 33s.\r\n\r\nHere's what I got in `errors.txt`:\r\n```\r\n\r\n 0K . 71.6M=0s\r\n\r\n2020-10-15 15:23:09 (71.6 MB/s) - \u2018localhost:8000/datasette/index.html\u2019 saved [1276]\r\n\r\nLoading robots.txt; please ignore errors.\r\n--2020-10-15 15:23:09-- http://localhost:8000/robots.txt\r\nReusing existing connection to localhost:8000.\r\nHTTP request sent, awaiting response... 404 Not Found\r\n--\r\n--2020-10-15 15:23:09-- http://localhost:8000/robots.txt\r\nReusing existing connection to localhost:8000.\r\nHTTP request sent, awaiting response... 404 Not Found\r\n2020-10-15 15:23:09 ERROR 404: Not Found.\r\n\r\n--2020-10-15 15:23:09-- http://localhost:8000/datasette/-/static/app.css?b576be\r\nReusing existing connection to localhost:8000.\r\nHTTP request sent, awaiting response... 200 OK\r\nLength: 8563 (8.4K) [text/css]\r\n--\r\n--\r\n2020-10-15 15:23:13 (7.90 MB/s) - \u2018localhost:8000/datasette/fixtures/primary_key_multiple_columns_explicit_label.json?_shape=object\u2019 saved [58]\r\n\r\n--2020-10-15 15:23:13-- http://localhost:8000/-/static-plugins/datasette_cluster_map/datasette-cluster-map.js\r\nReusing existing connection to localhost:8000.\r\nHTTP request sent, awaiting response... 404 Not Found\r\n2020-10-15 15:23:13 ERROR 404: Not Found.\r\n\r\n--2020-10-15 15:23:13-- http://localhost:8000/datasette/fixtures?sql=select+pk%2C+name%2C+address%2C+latitude%2C+longitude+from+roadside_attractions+order+by+pk+limit+101\r\nReusing existing connection to localhost:8000.\r\nHTTP request sent, awaiting response... 200 OK\r\nLength: unspecified [text/html]\r\n--\r\n--\r\n2020-10-15 15:23:13 (84.3 MB/s) - \u2018localhost:8000/datasette/fixtures/roadside_attractions.json?_shape=object\u2019 saved [619]\r\n\r\n--2020-10-15 15:23:13-- http://localhost:8000/datasette/fixtures/%5C%22https://www.openstreetmap.org/copyright%5C%22\r\nReusing existing connection to localhost:8000.\r\nHTTP request sent, awaiting response... 404 Not Found\r\n2020-10-15 15:23:13 ERROR 404: Not Found.\r\n\r\n--2020-10-15 15:23:13-- http://localhost:8000/datasette/fixtures?sql=select+pk%2C+text1%2C+text2%2C+%5Bname+with+.+and+spaces%5D+from+searchable+order+by+pk+limit+101\r\nReusing existing connection to localhost:8000.\r\nHTTP request sent, awaiting response... 200 OK\r\nLength: unspecified [text/html]\r\n--\r\n--\r\n2020-10-15 15:23:14 (28.6 MB/s) - \u2018localhost:8000/datasette/fixtures/searchable_view_configured_by_metadata.json?_shape=array&_nl=on\u2019 saved [180]\r\n\r\n--2020-10-15 15:23:14-- http://localhost:8000/\r\nReusing existing connection to localhost:8000.\r\nHTTP request sent, awaiting response... 404 Not Found\r\n2020-10-15 15:23:14 ERROR 404: Not Found.\r\n\r\n--2020-10-15 15:23:14-- http://localhost:8000/datasette/fixtures?sql=select+pk1%2C+pk2%2C+pk3%2C+content+from+compound_three_primary_keys+order+by+pk1%2C+pk2%2C+pk3+limit+101&_hide_sql=1\r\nReusing existing connection to localhost:8000.\r\nHTTP request sent, awaiting response... 200 OK\r\nLength: unspecified [text/html]\r\n--\r\n--\r\n2020-10-15 15:23:21 (64.1 MB/s) - \u2018localhost:8000/datasette/fixtures.csv?sql=select+pk,+name,+address,+latitude,+longitude+from+roadside_attractions+order+by+pk+limit+101&_size=max\u2019 saved [403]\r\n\r\n--2020-10-15 15:23:21-- http://localhost:8000/datasette/%5C%22https://www.openstreetmap.org/copyright%5C%22\r\nReusing existing connection to localhost:8000.\r\nHTTP request sent, awaiting response... 404 Not Found\r\n2020-10-15 15:23:21 ERROR 404: Not Found.\r\n\r\n--2020-10-15 15:23:21-- http://localhost:8000/datasette/fixtures?sql=select+pk%2C+name%2C+address%2C+latitude%2C+longitude+from+roadside_attractions+order+by+pk+desc+limit+101\r\nReusing existing connection to localhost:8000.\r\nHTTP request sent, awaiting response... 200 OK\r\nLength: unspecified [text/html]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722674708, "label": "Figure out how to run an environment that exercises the base_url proxy setting"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1024#issuecomment-709625063", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1024", "id": 709625063, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTYyNTA2Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T22:33:22Z", "updated_at": "2020-10-15T22:33:22Z", "author_association": "OWNER", "body": "Of those errors...\r\n\r\n`http://localhost:8000/robots.txt` 404 is fine.\r\n\r\n`http://localhost:8000/datasette/%5C%22https://www.openstreetmap.org/copyright%5C%22` looks to me like a `wget` parsing bug where it got confused by this JavaScript:\r\n\r\n```\r\nwindow.DATASETTE_CLUSTER_MAP_TILE_LAYER_OPTIONS = {\"maxZoom\": 19, \"detectRetina\": true, \"attribution\": \"© OpenStreetMap contributors\"};\r\n```\r\n\r\n`http://localhost:8000/-/static-plugins/datasette_cluster_map/datasette-cluster-map.js` is a real bug. It's a bug in `datasette-cluster-map` but also requires me to solve #988 - mechanism for plugins to construct URLs that obey `base_url`.\r\n\r\nI'm not sure why I'm getting a hit to `http://localhost:8000/` since I wouldn't expect to link to `/` anywhere.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722674708, "label": "Figure out how to run an environment that exercises the base_url proxy setting"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1025#issuecomment-709629920", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1025", "id": 709629920, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTYyOTkyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T22:48:20Z", "updated_at": "2020-10-15T22:48:20Z", "author_association": "OWNER", "body": "Also these:\r\n```\r\ndatasette % git grep '\"/' -- '*.html' ':(exclude)*/patterns.html'\r\ndatasette/templates/allow_debug.html:

\r\ndatasette/templates/base.html: \r\ndatasette/templates/error.html: home\r\ndatasette/templates/logout.html:\r\ndatasette/templates/messages_debug.html:\r\ndatasette/templates/query.html: home /\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722724086, "label": "Fix last remaining links to \"/\" that do not respect base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1025#issuecomment-709632136", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1025", "id": 709632136, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTYzMjEzNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T22:55:44Z", "updated_at": "2020-10-15T22:55:44Z", "author_association": "OWNER", "body": "It looks like there are also some generated redirect responses that don't take `base_url` into account:\r\n```\r\ndatasette % git grep '\"/' -- '*.py' ':(exclude)*test_*.py' ':(exclude)datasette/app.py'\r\ndatasette/_version.py: for i in cfg.versionfile_source.split(\"/\"):\r\ndatasette/utils/asgi.py: path=\"/\",\r\ndatasette/views/base.py: should_redirect = \"/{}-{}\".format(name, expected)\r\ndatasette/views/base.py: should_redirect += \"/\" + urllib.parse.quote_plus(kwargs[\"table\"])\r\ndatasette/views/base.py: should_redirect += \"/\" + kwargs[\"pk_path\"]\r\ndatasette/views/special.py: response = Response.redirect(\"/\")\r\ndatasette/views/special.py: return Response.redirect(\"/\")\r\ndatasette/views/special.py: response = Response.redirect(\"/\")\r\ndatasette/views/special.py: return Response.redirect(\"/\")\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722724086, "label": "Fix last remaining links to \"/\" that do not respect base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1025#issuecomment-709632314", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1025", "id": 709632314, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTYzMjMxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T22:56:25Z", "updated_at": "2020-10-15T22:56:34Z", "author_association": "OWNER", "body": "That `utils/asgi.py` line is the default path for setting cookies. That should likely take `base_url` into account too:\r\nhttps://github.com/simonw/datasette/blob/4f7c0ebd85ccd8c1853d7aa0147628f7c1b749cc/datasette/utils/asgi.py#L331-L342", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722724086, "label": "Fix last remaining links to \"/\" that do not respect base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1025#issuecomment-712481127", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1025", "id": 712481127, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjQ4MTEyNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-19T22:40:37Z", "updated_at": "2020-10-20T01:21:36Z", "author_association": "OWNER", "body": "Was blocked on #904 - now unblocked.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722724086, "label": "Fix last remaining links to \"/\" that do not respect base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1025#issuecomment-712525557", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1025", "id": 712525557, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjUyNTU1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-20T01:07:02Z", "updated_at": "2020-10-20T01:07:02Z", "author_association": "OWNER", "body": "I fixed the `queries.html` one. I'm not going to fix these two:\r\n```\r\ndatasette/templates/error.html: home\r\ndatasette/templates/patterns.html: home /\r\n```\r\nBecause the `error.html` one does not get passed a context (which makes sense since an error has occurred) and the pattern portfolio doesn't need to link to anywhere in particular.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722724086, "label": "Fix last remaining links to \"/\" that do not respect base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1025#issuecomment-712579674", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1025", "id": 712579674, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjU3OTY3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-20T04:24:10Z", "updated_at": "2020-10-20T04:24:10Z", "author_association": "OWNER", "body": "Changed my mind, `error.html` needs access to `urls` in order to link to its CSS file. Passing it after all (it already got passed `ds.config(\"base_url\")` so `ds` was available previously).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722724086, "label": "Fix last remaining links to \"/\" that do not respect base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1025#issuecomment-712593790", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1025", "id": 712593790, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjU5Mzc5MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-20T05:12:36Z", "updated_at": "2020-10-20T05:12:36Z", "author_association": "OWNER", "body": "I'm going to leave the cookies code setting cookies to default to the `\"/\"` top level.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722724086, "label": "Fix last remaining links to \"/\" that do not respect base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1026#issuecomment-709636372", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1026", "id": 709636372, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTYzNjM3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T23:09:34Z", "updated_at": "2020-10-15T23:09:34Z", "author_association": "OWNER", "body": "I'm inclined to say that internal requests should ignore `base_url` - since that seems like the right thing for plugins that need to access default Datasette APIs.\r\n\r\nThe one catch here is plugins that might want to proxy the current incoming URL for some reason - where that incoming `request.path` could include the `base_url`.\r\n\r\nActually those should be fine - because it will have been stripped off earlier:\r\n\r\nhttps://github.com/simonw/datasette/blob/4f7c0ebd85ccd8c1853d7aa0147628f7c1b749cc/datasette/app.py#L963-L968", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722738988, "label": "How should datasette.client interact with base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1026#issuecomment-712607227", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1026", "id": 712607227, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjYwNzIyNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-20T05:46:44Z", "updated_at": "2020-10-20T05:46:44Z", "author_association": "OWNER", "body": "We have a solution for this now: `datasette.urls` from #1033 can be used by plugins to assemble the correct URLs to pass to `.get()` and friends.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722738988, "label": "How should datasette.client interact with base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1026#issuecomment-712959034", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1026", "id": 712959034, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjk1OTAzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-20T16:03:33Z", "updated_at": "2020-10-20T16:03:55Z", "author_association": "OWNER", "body": "Reconsidering this: I think the `.get()` etc methods should automatically add the `base_url` prefix for you, since these APIs are only intended to make internal calls.\r\n\r\nThe clincher on this is when I went to add a section to the `datasette.client` documentation recommending you use `datasette.urls.path()` for every call to them that you make.\r\n\r\nBut there's a problem: to handle table name escaping users are likely to want to use `datasette.urls.table()` anyway, like this:\r\n\r\n response = await datasette.client.get(datasette.urls.table(\"db\", \"table\") + \".json\")\r\n\r\nThis risks adding the `base_url` prefix twice. Maybe the `.table()` method could return a string-like object that is marked as already having the `base_url` prefix added, so the `client.get()` method knows not to add it again.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722738988, "label": "How should datasette.client interact with base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1026#issuecomment-712962517", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1026", "id": 712962517, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjk2MjUxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-20T16:09:12Z", "updated_at": "2020-10-20T16:09:12Z", "author_association": "OWNER", "body": "That `datasette.urls.table(\"db\", \"table\") + \".json\"` example is bad because if the table name contains a `.` it should be `?_format=json` instead.\r\n\r\nMaybe `.table()` should have a `format=\"json\"` option that knows how to do this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722738988, "label": "How should datasette.client interact with base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1026#issuecomment-719959754", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1026", "id": 719959754, "node_id": "MDEyOklzc3VlQ29tbWVudDcxOTk1OTc1NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-31T16:56:35Z", "updated_at": "2020-10-31T16:56:35Z", "author_association": "OWNER", "body": "#1041 can also benefit from the string subclass that shows that `base_url` has been added.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722738988, "label": "How should datasette.client interact with base_url"}, "performed_via_github_app": null}