{"html_url": "https://github.com/simonw/datasette/issues/1534#issuecomment-1028461220", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1534", "id": 1028461220, "node_id": "IC_kwDOBm6k_c49TRKk", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T23:39:33Z", "updated_at": "2022-02-02T23:39:33Z", "author_association": "OWNER", "body": "I've decided not to do this, because of the risk that Cloudflare could cache the JSON version for an HTML page or vice-versa.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1065432388, "label": "Maybe return JSON from HTML pages if `Accept: application/json` is sent"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1626#issuecomment-1028423514", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1626", "id": 1028423514, "node_id": "IC_kwDOBm6k_c49TH9a", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2022-02-02T22:36:37Z", "updated_at": "2022-02-02T22:39:52Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1626?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\n> Merging [#1626](https://codecov.io/gh/simonw/datasette/pull/1626?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (4b4d0e1) into [main](https://codecov.io/gh/simonw/datasette/commit/b5e6b1a9e1332fca3effe45d55dd06ee4249f163?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (b5e6b1a) will **not change** coverage.\n> The diff coverage is `n/a`.\n\n[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1626/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1626?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n```diff\n@@ Coverage Diff @@\n## main #1626 +/- ##\n=======================================\n Coverage 92.16% 92.16% \n=======================================\n Files 34 34 \n Lines 4531 4531 \n=======================================\n Hits 4176 4176 \n Misses 355 355 \n```\n\n\n\n------\n\n[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1626?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n> `\u0394 = absolute (impact)`, `\u00f8 = not affected`, `? = missing data`\n> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1626?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [b5e6b1a...4b4d0e1](https://codecov.io/gh/simonw/datasette/pull/1626?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1122451096, "label": "Try test suite against macOS and Windows"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1626#issuecomment-1028420821", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1626", "id": 1028420821, "node_id": "IC_kwDOBm6k_c49THTV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T22:32:26Z", "updated_at": "2022-02-02T22:33:31Z", "author_association": "OWNER", "body": "That broke on a macOS test: https://github.com/simonw/datasette/runs/5044036993?check_suite_focus=true\r\n\r\nI'm going to remove macOS and Ubuntu and just try Windows purely to see what happens there.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1122451096, "label": "Try test suite against macOS and Windows"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1616#issuecomment-1028414871", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1616", "id": 1028414871, "node_id": "IC_kwDOBm6k_c49TF2X", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T22:23:45Z", "updated_at": "2022-02-02T22:23:45Z", "author_association": "OWNER", "body": "First stable Black release!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1119413338, "label": "Bump black from 21.12b0 to 22.1.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1623#issuecomment-1028397935", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1623", "id": 1028397935, "node_id": "IC_kwDOBm6k_c49TBtv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T21:59:43Z", "updated_at": "2022-02-02T21:59:43Z", "author_association": "OWNER", "body": "Here's the new test: https://github.com/simonw/datasette/blob/23a09b0f6af33c52acf8c1d9002fe475b42fee10/tests/test_html.py#L927-L936", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1122416919, "label": "/-/patterns returns link: alternate JSON header to 404"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1624#issuecomment-1028396866", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1624", "id": 1028396866, "node_id": "IC_kwDOBm6k_c49TBdC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T21:58:06Z", "updated_at": "2022-02-02T21:58:06Z", "author_association": "OWNER", "body": "It looks like this is because `IndexView` extends `BaseView` rather than extending `DataView` which is where all that CORS stuff happens:\r\n\r\nhttps://github.com/simonw/datasette/blob/23a09b0f6af33c52acf8c1d9002fe475b42fee10/datasette/views/index.py#L18-L21\r\n\r\nAnother thing I should address with the refactor project in:\r\n- #878 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1122427321, "label": "Index page `/` has no CORS headers"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1620#issuecomment-1028393259", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1620", "id": 1028393259, "node_id": "IC_kwDOBm6k_c49TAkr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T21:53:02Z", "updated_at": "2022-02-02T21:53:02Z", "author_association": "OWNER", "body": "I ran the following on https://www.google.com/ in the console to demonstrate that these work as intended:\r\n\r\n```javascript\r\n[\r\n \"https://latest.datasette.io/fixtures\",\r\n \"https://latest.datasette.io/fixtures?sql=select+1\",\r\n \"https://latest.datasette.io/fixtures/facetable\"\r\n].forEach(async (url) => {\r\n response = await fetch(url, {method: \"HEAD\"});\r\n console.log(response.headers.get(\"Link\"));\r\n});\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1121618041, "label": "Link: rel=\"alternate\" to JSON for queries too"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1623#issuecomment-1028389953", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1623", "id": 1028389953, "node_id": "IC_kwDOBm6k_c49S_xB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T21:48:34Z", "updated_at": "2022-02-02T21:48:34Z", "author_association": "OWNER", "body": "A few other pages do that too, including:\r\n- https://latest.datasette.io/-/messages\r\n- https://latest.datasette.io/-/allow-debug", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1122416919, "label": "/-/patterns returns link: alternate JSON header to 404"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1622#issuecomment-1028387529", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1622", "id": 1028387529, "node_id": "IC_kwDOBm6k_c49S_LJ", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2022-02-02T21:45:21Z", "updated_at": "2022-02-02T21:45:21Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1622?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\n> Merging [#1622](https://codecov.io/gh/simonw/datasette/pull/1622?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (fbaf317) into [main](https://codecov.io/gh/simonw/datasette/commit/8d5779acf0041cfd0db7f68f468419f9008b86ec?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (8d5779a) will **not change** coverage.\n> The diff coverage is `n/a`.\n\n[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1622/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1622?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n```diff\n@@ Coverage Diff @@\n## main #1622 +/- ##\n=======================================\n Coverage 92.11% 92.11% \n=======================================\n Files 34 34 \n Lines 4525 4525 \n=======================================\n Hits 4168 4168 \n Misses 357 357 \n```\n\n\n\n------\n\n[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1622?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n> `\u0394 = absolute (impact)`, `\u00f8 = not affected`, `? = missing data`\n> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1622?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [8d5779a...fbaf317](https://codecov.io/gh/simonw/datasette/pull/1622?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1122414274, "label": "Test against Python 3.11-dev"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1620#issuecomment-1028385067", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1620", "id": 1028385067, "node_id": "IC_kwDOBm6k_c49S-kr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T21:42:23Z", "updated_at": "2022-02-02T21:42:23Z", "author_association": "OWNER", "body": "```\r\n% curl -s -I 'https://latest.datasette.io/' | grep link\r\nlink: https://latest.datasette.io/.json; rel=\"alternate\"; type=\"application/json+datasette\"\r\n% curl -s -I 'https://latest.datasette.io/fixtures' | grep link\r\nlink: https://latest.datasette.io/fixtures.json; rel=\"alternate\"; type=\"application/json+datasette\"\r\n% curl -s -I 'https://latest.datasette.io/fixtures?sql=select+1' | grep link\r\nlink: https://latest.datasette.io/fixtures.json?sql=select+1; rel=\"alternate\"; type=\"application/json+datasette\"\r\n% curl -s -I 'https://latest.datasette.io/-/plugins' | grep link \r\nlink: https://latest.datasette.io/-/plugins.json; rel=\"alternate\"; type=\"application/json+datasette\"\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1121618041, "label": "Link: rel=\"alternate\" to JSON for queries too"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1620#issuecomment-1028374330", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1620", "id": 1028374330, "node_id": "IC_kwDOBm6k_c49S786", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T21:28:16Z", "updated_at": "2022-02-02T21:28:16Z", "author_association": "OWNER", "body": "I just realized I can refactor this to make it much simpler.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1121618041, "label": "Link: rel=\"alternate\" to JSON for queries too"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1618#issuecomment-1028294089", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1618", "id": 1028294089, "node_id": "IC_kwDOBm6k_c49SoXJ", "user": {"value": 770231, "label": "strada"}, "created_at": "2022-02-02T19:42:03Z", "updated_at": "2022-02-02T19:42:03Z", "author_association": "NONE", "body": "Thanks for looking into this. It might have been nice if `explain` surfaced these function calls. Looks like `explain query plan` does, but only for basic queries.\r\n\r\n```\r\nsqlite-utils fixtures.db 'explain query plan select * from pragma_function_list(), pragma_database_list(), pragma_module_list()' -t\r\n id parent notused detail\r\n---- -------- --------- ------------------------------------------------\r\n 4 0 0 SCAN pragma_function_list VIRTUAL TABLE INDEX 0:\r\n 8 0 0 SCAN pragma_database_list VIRTUAL TABLE INDEX 0:\r\n 12 0 0 SCAN pragma_module_list VIRTUAL TABLE INDEX 0:\r\n```\r\n\r\n\r\n```\r\nsqlite-utils fixtures.db 'explain query plan select * from pragma_function_list() as fl, pragma_database_list() as dl, pragma_module_list() as ml' -t\r\n id parent notused detail\r\n---- -------- --------- ------------------------------\r\n 4 0 0 SCAN fl VIRTUAL TABLE INDEX 0:\r\n 8 0 0 SCAN dl VIRTUAL TABLE INDEX 0:\r\n 12 0 0 SCAN ml VIRTUAL TABLE INDEX 0:\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1121121305, "label": "Reconsider policy on blocking queries containing the string \"pragma\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1533#issuecomment-1027672617", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1533", "id": 1027672617, "node_id": "IC_kwDOBm6k_c49QQop", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T07:56:51Z", "updated_at": "2022-02-02T07:56:51Z", "author_association": "OWNER", "body": "Demos - these pages both have ` Table-valued functions exist only for PRAGMAs that return results and that have no side-effects.\r\n\r\nSo it's possible I'm being overly paranoid here after all: what I want to block here is people running things like `PRAGMA case_sensitive_like = 1` which could affect the global state for that connection and cause unexpected behaviour later on.\r\n\r\nSo maybe I should allow all pragma functions. I previously allowed an allow-list of them in:\r\n- #761", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1121121305, "label": "Reconsider policy on blocking queries containing the string \"pragma\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1618#issuecomment-1027653005", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1618", "id": 1027653005, "node_id": "IC_kwDOBm6k_c49QL2N", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T07:22:13Z", "updated_at": "2022-02-02T07:22:13Z", "author_association": "OWNER", "body": "There's a workaround for this at the moment, which is to use parameterized SQL queries. For example, this:\r\n\r\nhttps://fivethirtyeight.datasettes.com/polls?sql=select+*+from+books+where+title+%3D+%3Atitle&title=The+Pragmatic+Programmer\r\n\r\nSo the SQL query is `select * from books where title = :title` and then `&title=...` is added to the URL.\r\n\r\nThe reason behind the quite aggressive pragma filtering is that SQLite allows you to execute pragmas using function calls, like this one:\r\n\r\n```sql\r\nSELECT * FROM pragma_index_info('idx52');\r\n```\r\nThese can be nested arbitrarily deeply in sub-queries, so it's difficult to write a regular expression that will definitely catch them.\r\n\r\nI'm open to relaxing the regex a bit, but I need to be very confident that it's safe to do so.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1121121305, "label": "Reconsider policy on blocking queries containing the string \"pragma\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1586#issuecomment-1027648180", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1586", "id": 1027648180, "node_id": "IC_kwDOBm6k_c49QKq0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T07:13:31Z", "updated_at": "2022-02-02T07:13:31Z", "author_association": "OWNER", "body": "Running it as part of `datasette publish` is a smart idea - I'm slightly nervous about modifying the database file that has been published though, since part of the undocumented contract right now is that the bytes served are the exact same bytes as the ones you ran the publish against.\r\n\r\nBut there's no reason for that expectation to exist, and I doubt anyone is relying on that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1096536240, "label": "run analyze on all databases as part of start up or publishing"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1619#issuecomment-1027647257", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1619", "id": 1027647257, "node_id": "IC_kwDOBm6k_c49QKcZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T07:11:43Z", "updated_at": "2022-02-02T07:11:43Z", "author_association": "OWNER", "body": "Weirdly the bug does NOT exhibit itself on this demo: https://datasette-apache-proxy-demo.datasette.io/prefix/fixtures/no_primary_key/1 - which correctly links to https://datasette-apache-proxy-demo.datasette.io/prefix/fixtures/no_primary_key/1.json", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1121583414, "label": "JSON link on row page is 404 if base_url setting is used"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1619#issuecomment-1027646659", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1619", "id": 1027646659, "node_id": "IC_kwDOBm6k_c49QKTD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T07:10:37Z", "updated_at": "2022-02-02T07:10:37Z", "author_association": "OWNER", "body": "It's not just the table with slashes in the name. Same thing on http://127.0.0.1:3344/foo/bar/fixtures/attraction_characteristic/1 - the `json` link goes to a JSON-rendered 404 on http://127.0.0.1:3344/foo/bar/foo/bar/fixtures/attraction_characteristic/1.json", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1121583414, "label": "JSON link on row page is 404 if base_url setting is used"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1576#issuecomment-1027635925", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1576", "id": 1027635925, "node_id": "IC_kwDOBm6k_c49QHrV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T06:47:20Z", "updated_at": "2022-02-02T06:47:20Z", "author_association": "OWNER", "body": "Here's what I was hacking around with when I uncovered this problem:\r\n```diff\r\ndiff --git a/datasette/views/table.py b/datasette/views/table.py\r\nindex 77fb285..8c57d08 100644\r\n--- a/datasette/views/table.py\r\n+++ b/datasette/views/table.py\r\n@@ -1,3 +1,4 @@\r\n+import asyncio\r\n import urllib\r\n import itertools\r\n import json\r\n@@ -615,44 +616,37 @@ class TableView(RowTableShared):\r\n if request.args.get(\"_timelimit\"):\r\n extra_args[\"custom_time_limit\"] = int(request.args.get(\"_timelimit\"))\r\n \r\n- # Execute the main query!\r\n- results = await db.execute(sql, params, truncate=True, **extra_args)\r\n-\r\n- # Calculate the total count for this query\r\n- filtered_table_rows_count = None\r\n- if (\r\n- not db.is_mutable\r\n- and self.ds.inspect_data\r\n- and count_sql == f\"select count(*) from {table} \"\r\n- ):\r\n- # We can use a previously cached table row count\r\n- try:\r\n- filtered_table_rows_count = self.ds.inspect_data[database][\"tables\"][\r\n- table\r\n- ][\"count\"]\r\n- except KeyError:\r\n- pass\r\n-\r\n- # Otherwise run a select count(*) ...\r\n- if count_sql and filtered_table_rows_count is None and not nocount:\r\n- try:\r\n- count_rows = list(await db.execute(count_sql, from_sql_params))\r\n- filtered_table_rows_count = count_rows[0][0]\r\n- except QueryInterrupted:\r\n- pass\r\n-\r\n- # Faceting\r\n- if not self.ds.setting(\"allow_facet\") and any(\r\n- arg.startswith(\"_facet\") for arg in request.args\r\n- ):\r\n- raise BadRequest(\"_facet= is not allowed\")\r\n+ async def execute_count():\r\n+ # Calculate the total count for this query\r\n+ filtered_table_rows_count = None\r\n+ if (\r\n+ not db.is_mutable\r\n+ and self.ds.inspect_data\r\n+ and count_sql == f\"select count(*) from {table} \"\r\n+ ):\r\n+ # We can use a previously cached table row count\r\n+ try:\r\n+ filtered_table_rows_count = self.ds.inspect_data[database][\r\n+ \"tables\"\r\n+ ][table][\"count\"]\r\n+ except KeyError:\r\n+ pass\r\n+\r\n+ if count_sql and filtered_table_rows_count is None and not nocount:\r\n+ try:\r\n+ count_rows = list(await db.execute(count_sql, from_sql_params))\r\n+ filtered_table_rows_count = count_rows[0][0]\r\n+ except QueryInterrupted:\r\n+ pass\r\n+\r\n+ return filtered_table_rows_count\r\n+\r\n+ filtered_table_rows_count = await execute_count()\r\n \r\n # pylint: disable=no-member\r\n facet_classes = list(\r\n itertools.chain.from_iterable(pm.hook.register_facet_classes())\r\n )\r\n- facet_results = {}\r\n- facets_timed_out = []\r\n facet_instances = []\r\n for klass in facet_classes:\r\n facet_instances.append(\r\n@@ -668,33 +662,58 @@ class TableView(RowTableShared):\r\n )\r\n )\r\n \r\n- if not nofacet:\r\n- for facet in facet_instances:\r\n- (\r\n- instance_facet_results,\r\n- instance_facets_timed_out,\r\n- ) = await facet.facet_results()\r\n- for facet_info in instance_facet_results:\r\n- base_key = facet_info[\"name\"]\r\n- key = base_key\r\n- i = 1\r\n- while key in facet_results:\r\n- i += 1\r\n- key = f\"{base_key}_{i}\"\r\n- facet_results[key] = facet_info\r\n- facets_timed_out.extend(instance_facets_timed_out)\r\n-\r\n- # Calculate suggested facets\r\n- suggested_facets = []\r\n- if (\r\n- self.ds.setting(\"suggest_facets\")\r\n- and self.ds.setting(\"allow_facet\")\r\n- and not _next\r\n- and not nofacet\r\n- and not nosuggest\r\n- ):\r\n- for facet in facet_instances:\r\n- suggested_facets.extend(await facet.suggest())\r\n+ async def execute_suggested_facets():\r\n+ # Calculate suggested facets\r\n+ suggested_facets = []\r\n+ if (\r\n+ self.ds.setting(\"suggest_facets\")\r\n+ and self.ds.setting(\"allow_facet\")\r\n+ and not _next\r\n+ and not nofacet\r\n+ and not nosuggest\r\n+ ):\r\n+ for facet in facet_instances:\r\n+ suggested_facets.extend(await facet.suggest())\r\n+ return suggested_facets\r\n+\r\n+ async def execute_facets():\r\n+ facet_results = {}\r\n+ facets_timed_out = []\r\n+ if not self.ds.setting(\"allow_facet\") and any(\r\n+ arg.startswith(\"_facet\") for arg in request.args\r\n+ ):\r\n+ raise BadRequest(\"_facet= is not allowed\")\r\n+\r\n+ if not nofacet:\r\n+ for facet in facet_instances:\r\n+ (\r\n+ instance_facet_results,\r\n+ instance_facets_timed_out,\r\n+ ) = await facet.facet_results()\r\n+ for facet_info in instance_facet_results:\r\n+ base_key = facet_info[\"name\"]\r\n+ key = base_key\r\n+ i = 1\r\n+ while key in facet_results:\r\n+ i += 1\r\n+ key = f\"{base_key}_{i}\"\r\n+ facet_results[key] = facet_info\r\n+ facets_timed_out.extend(instance_facets_timed_out)\r\n+\r\n+ return facet_results, facets_timed_out\r\n+\r\n+ # Execute the main query, facets and facet suggestions in parallel:\r\n+ (\r\n+ results,\r\n+ suggested_facets,\r\n+ (facet_results, facets_timed_out),\r\n+ ) = await asyncio.gather(\r\n+ db.execute(sql, params, truncate=True, **extra_args),\r\n+ execute_suggested_facets(),\r\n+ execute_facets(),\r\n+ )\r\n+\r\n+ results = await db.execute(sql, params, truncate=True, **extra_args)\r\n \r\n # Figure out columns and rows for the query\r\n columns = [r[0] for r in results.description]\r\n```\r\nIt's a hacky attempt at running some of the table page queries in parallel to see what happens.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1087181951, "label": "Traces should include SQL executed by subtasks created with `asyncio.gather`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1611#issuecomment-1027635175", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1611", "id": 1027635175, "node_id": "IC_kwDOBm6k_c49QHfn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T06:45:47Z", "updated_at": "2022-02-02T06:45:47Z", "author_association": "OWNER", "body": "Prototype, not sure that this actually works yet:\r\n```diff\r\ndiff --git a/datasette/database.py b/datasette/database.py\r\nindex 6ce8721..0c4aec7 100644\r\n--- a/datasette/database.py\r\n+++ b/datasette/database.py\r\n@@ -256,18 +256,26 @@ class Database:\r\n # Try to get counts for each table, $limit timeout for each count\r\n counts = {}\r\n for table in await self.table_names():\r\n- try:\r\n- table_count = (\r\n- await self.execute(\r\n- f\"select count(*) from [{table}]\",\r\n- custom_time_limit=limit,\r\n- )\r\n- ).rows[0][0]\r\n- counts[table] = table_count\r\n- # In some cases I saw \"SQL Logic Error\" here in addition to\r\n- # QueryInterrupted - so we catch that too:\r\n- except (QueryInterrupted, sqlite3.OperationalError, sqlite3.DatabaseError):\r\n- counts[table] = None\r\n+ print(table.lower())\r\n+ if table.lower() == \"knn\":\r\n+ counts[table] = 0\r\n+ else:\r\n+ try:\r\n+ table_count = (\r\n+ await self.execute(\r\n+ f\"select count(*) from [{table}]\",\r\n+ custom_time_limit=limit,\r\n+ )\r\n+ ).rows[0][0]\r\n+ counts[table] = table_count\r\n+ # In some cases I saw \"SQL Logic Error\" here in addition to\r\n+ # QueryInterrupted - so we catch that too:\r\n+ except (\r\n+ QueryInterrupted,\r\n+ sqlite3.OperationalError,\r\n+ sqlite3.DatabaseError,\r\n+ ):\r\n+ counts[table] = None\r\n if not self.is_mutable:\r\n self._cached_table_counts = counts\r\n return counts\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1113384383, "label": "Avoid ever running count(*) against SpatiaLite KNN table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1607#issuecomment-1027634490", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1607", "id": 1027634490, "node_id": "IC_kwDOBm6k_c49QHU6", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T06:44:30Z", "updated_at": "2022-02-02T06:44:30Z", "author_association": "OWNER", "body": "Prototype:\r\n```diff\r\ndiff --git a/datasette/app.py b/datasette/app.py\r\nindex 09d7d03..e2a5aea 100644\r\n--- a/datasette/app.py\r\n+++ b/datasette/app.py\r\n@@ -724,6 +724,47 @@ class Datasette:\r\n sqlite_extensions[extension] = None\r\n except Exception:\r\n pass\r\n+ # More details on SpatiaLite\r\n+ if \"spatialite\" in sqlite_extensions:\r\n+ spatialite_details = {}\r\n+ fns = (\r\n+ \"spatialite_version\",\r\n+ \"spatialite_target_cpu\",\r\n+ \"rcheck_strict_sql_quoting\",\r\n+ \"freexl_version\",\r\n+ \"proj_version\",\r\n+ \"geos_version\",\r\n+ \"rttopo_version\",\r\n+ \"libxml2_version\",\r\n+ \"HasIconv\",\r\n+ \"HasMathSQL\",\r\n+ \"HasGeoCallbacks\",\r\n+ \"HasProj\",\r\n+ \"HasProj6\",\r\n+ \"HasGeos\",\r\n+ \"HasGeosAdvanced\",\r\n+ \"HasGeosTrunk\",\r\n+ \"HasGeosReentrant\",\r\n+ \"HasGeosOnlyReentrant\",\r\n+ \"HasMiniZip\",\r\n+ \"HasRtTopo\",\r\n+ \"HasLibXML2\",\r\n+ \"HasEpsg\",\r\n+ \"HasFreeXL\",\r\n+ \"HasGeoPackage\",\r\n+ \"HasGCP\",\r\n+ \"HasTopology\",\r\n+ \"HasKNN\",\r\n+ \"HasRouting\",\r\n+ )\r\n+ for fn in fns:\r\n+ try:\r\n+ result = conn.execute(\"select {}()\".format(fn))\r\n+ spatialite_details[fn] = result.fetchone()[0]\r\n+ except Exception:\r\n+ pass\r\n+ sqlite_extensions[\"spatialite\"] = spatialite_details\r\n+\r\n # Figure out supported FTS versions\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1109783030, "label": "More detailed information about installed SpatiaLite version"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1533#issuecomment-1027633686", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1533", "id": 1027633686, "node_id": "IC_kwDOBm6k_c49QHIW", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-02T06:42:53Z", "updated_at": "2022-02-02T06:42:53Z", "author_association": "OWNER", "body": "I'm going to apply the hack, then fix it again in:\r\n- #1518", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1065431383, "label": "Add `Link: rel=\"alternate\"` header pointing to JSON for a table/query"}, "performed_via_github_app": null}