{"html_url": "https://github.com/simonw/datasette/issues/1238#issuecomment-855270917", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1238", "id": 855270917, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTI3MDkxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T17:32:29Z", "updated_at": "2021-06-05T17:32:29Z", "author_association": "OWNER", "body": "This looks like the cause: https://github.com/simonw/datasette/blob/6e9b07be92905011211d8df7a872fb7c1f2737b2/datasette/app.py#L1087-L1092\r\n\r\nNote how `path` is modified... but then we create a new `Request()` that uses the old scope, which has unmodified `scope[\"path\"]` - and then the code later on looks at `request.scope[\"path\"]` when deciding if the request matches:\r\n\r\nhttps://github.com/simonw/datasette/blob/afed51b1e36cf275c39e71c7cb262d6c5bdbaa31/datasette/app.py#L1154-L1155", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 813899472, "label": "Custom pages don't work with base_url setting"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1238#issuecomment-855272693", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1238", "id": 855272693, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTI3MjY5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T17:45:42Z", "updated_at": "2021-06-05T17:45:42Z", "author_association": "OWNER", "body": "Applying this fix worked when I manually tested it:\r\n```diff\r\n         base_url = self.ds.setting(\"base_url\")\r\n         if base_url != \"/\" and path.startswith(base_url):\r\n             path = \"/\" + path[len(base_url) :]\r\n+            scope = dict(scope, path=path, raw_path=path.encode(\"utf-8\"))\r\n         request = Request(scope, receive)\r\n```\r\nBut... the test I wrote still failed. My hunch is that this is because deep within the test framework requests go through `ds.client` which may be applying its own changes relevant to `base_url`:\r\n\r\nhttps://github.com/simonw/datasette/blob/6e9b07be92905011211d8df7a872fb7c1f2737b2/datasette/utils/testing.py#L139", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 813899472, "label": "Custom pages don't work with base_url setting"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1238#issuecomment-855278540", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1238", "id": 855278540, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTI3ODU0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T18:33:25Z", "updated_at": "2021-06-05T18:33:25Z", "author_association": "OWNER", "body": "Got the test to pass by ensuring the tests don't accidentally double-rewrite the path.\r\n\r\nRan into a new problem:\r\n```\r\n    @pytest.mark.asyncio\r\n    @pytest.mark.parametrize(\r\n        \"prefix,expected_path\", [(None, \"/asgi-scope\"), (\"/prefix/\", \"/prefix/asgi-scope\")]\r\n    )\r\n    async def test_client_path(datasette, prefix, expected_path):\r\n        original_base_url = datasette._settings[\"base_url\"]\r\n        try:\r\n            if prefix is not None:\r\n                datasette._settings[\"base_url\"] = prefix\r\n            response = await datasette.client.get(\"/asgi-scope\")\r\n            path = response.json()[\"path\"]\r\n>           assert path == expected_path\r\nE           AssertionError: assert '/asgi-scope' == '/prefix/asgi-scope'\r\nE             - /prefix/asgi-scope\r\nE             ? -------\r\nE             + /asgi-scope\r\n```\r\nThat test confirms that messing around with the `base_url` doesn't modify the ASGI scope... but the fix I'm using for this issue DOES modify the ASGI scope.\r\n\r\nThe question raised here is: should the ASGI scope stay unmodified when `base_url` is used?\r\n\r\nI think it should. It doesn't make sense to obscure the \"real\" path just to get custom pages to work properly.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 813899472, "label": "Custom pages don't work with base_url setting"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1238#issuecomment-855278998", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1238", "id": 855278998, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTI3ODk5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T18:37:16Z", "updated_at": "2021-06-05T18:37:16Z", "author_association": "OWNER", "body": "Alternative idea: populate `request.scope` with a new `route_path` which is the base-url-stripped version, which we then use for other routing operations.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 813899472, "label": "Custom pages don't work with base_url setting"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1357#issuecomment-855281774", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1357", "id": 855281774, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTI4MTc3NA==", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2021-06-05T18:59:07Z", "updated_at": "2021-06-05T18:59:07Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\n> Merging [#1357](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (1b27643) into [main](https://codecov.io/gh/simonw/datasette/commit/6e9b07be92905011211d8df7a872fb7c1f2737b2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (6e9b07b) will **increase** coverage by `0.00%`.\n> The diff coverage is `100.00%`.\n\n[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1357/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n```diff\n@@           Coverage Diff           @@\n##             main    #1357   +/-   ##\n=======================================\n  Coverage   91.71%   91.72%           \n=======================================\n  Files          34       34           \n  Lines        4333     4336    +3     \n=======================================\n+ Hits         3974     3977    +3     \n  Misses        359      359           \n```\n\n\n| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [datasette/utils/testing.py](https://codecov.io/gh/simonw/datasette/pull/1357/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL3Rlc3RpbmcucHk=) | `95.38% <\u00f8> (\u00f8)` | |\n| [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1357/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `95.71% <100.00%> (+0.01%)` | :arrow_up: |\n\n------\n\n[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n> `\u0394 = absolute <relative> (impact)`, `\u00f8 = not affected`, `? = missing data`\n> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [6e9b07b...1b27643](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 912394511, "label": "Make custom pages compatible with base_url setting"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1356#issuecomment-855282466", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1356", "id": 855282466, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTI4MjQ2Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T19:05:06Z", "updated_at": "2021-06-05T19:05:06Z", "author_association": "OWNER", "body": "Yeah that's a good point. I avoided making them sub-commands because `datasette serve` already supports the multitude of other arguments they also need... but actually that was just me being lazy - I can easily share arguments between multiple functions like I do in `sqlite-utils` itself: https://github.com/simonw/sqlite-utils/blob/d1a372b3006e6cf7d2017b3ddc484bf5c033e45d/sqlite_utils/cli.py#L46", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 910092577, "label": "Research: syntactic sugar for using --get with SQL queries, maybe \"datasette query\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1291#issuecomment-855287200", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1291", "id": 855287200, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTI4NzIwMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T19:48:36Z", "updated_at": "2021-06-05T19:48:36Z", "author_association": "OWNER", "body": "This is great, thank you.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 849582643, "label": "Update docs: explain allow_download setting"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1358#issuecomment-855288228", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1358", "id": 855288228, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTI4ODIyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T19:57:18Z", "updated_at": "2021-06-05T19:57:18Z", "author_association": "OWNER", "body": "There's also a security fix I need to make, so I'm not going to block this on wrapping up the work on the new Docker image testing from #1344 before putting out this release.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 912418094, "label": "Release Datasette 0.57"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1358#issuecomment-855302320", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1358", "id": 855302320, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTMwMjMyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T22:08:06Z", "updated_at": "2021-06-05T22:08:06Z", "author_association": "OWNER", "body": "See #1360 for the security fix.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 912418094, "label": "Release Datasette 0.57"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1358#issuecomment-855302339", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1358", "id": 855302339, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTMwMjMzOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T22:08:16Z", "updated_at": "2021-06-05T22:08:16Z", "author_association": "OWNER", "body": "Release notes are in, pushing the release now.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 912418094, "label": "Release Datasette 0.57"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1360#issuecomment-855303649", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1360", "id": 855303649, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTMwMzY0OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T22:22:06Z", "updated_at": "2021-06-05T22:22:06Z", "author_association": "OWNER", "body": "I've released fixes in both 0.56.1 and 0.57.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 912464443, "label": "Security flaw, to be fixed in 0.56.1 and 0.57"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1360#issuecomment-855303776", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1360", "id": 855303776, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTMwMzc3Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T22:23:23Z", "updated_at": "2021-06-05T22:23:23Z", "author_association": "OWNER", "body": "Worth noting that I found this issue myself, and to my knowledge it has not been uncovered by anyone else prior to the patch being released.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 912464443, "label": "Security flaw, to be fixed in 0.56.1 and 0.57"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1361#issuecomment-855306347", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1361", "id": 855306347, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTMwNjM0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T22:49:30Z", "updated_at": "2021-06-05T22:49:30Z", "author_association": "OWNER", "body": "Stack Overflow: https://stackoverflow.com/a/49367679/6083\r\n\r\n> The answer was that `os.chdir()` had been set to the deleted directory by accident. The directory was missing, but the error happened (it seems) at the attempt to get it with `os.getcwd()`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 912485040, "label": "Intermittent CI failure: restore_working_directory FileNotFoundError"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1361#issuecomment-855306497", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1361", "id": 855306497, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTMwNjQ5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T22:51:01Z", "updated_at": "2021-06-05T22:51:01Z", "author_association": "OWNER", "body": "I'm going to try removing that `restore_working_directory` fixture entirely.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 912485040, "label": "Intermittent CI failure: restore_working_directory FileNotFoundError"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1361#issuecomment-855307292", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1361", "id": 855307292, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTMwNzI5Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T22:59:35Z", "updated_at": "2021-06-05T22:59:35Z", "author_association": "OWNER", "body": "That broke things. Here's how `pytest-cov` fixed a similar issue: https://github.com/pytest-dev/pytest-cov/commit/7ccb1783bf8290447df58deeb41eae74296a6d9b\r\n\r\nSee also https://github.com/nedbat/coveragepy/issues/881 and https://github.com/pytest-dev/pytest-cov/issues/306 and https://github.com/nedbat/coveragepy/issues/824", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 912485040, "label": "Intermittent CI failure: restore_working_directory FileNotFoundError"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1361#issuecomment-855308811", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1361", "id": 855308811, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTMwODgxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-05T23:16:21Z", "updated_at": "2021-06-05T23:16:21Z", "author_association": "OWNER", "body": "That seems to have fixed it. I'd love to get rid of this `restore_working_directory` hack entirely.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 912485040, "label": "Intermittent CI failure: restore_working_directory FileNotFoundError"}, "performed_via_github_app": null}