{"html_url": "https://github.com/simonw/datasette/issues/859#issuecomment-647135713", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/859", "id": 647135713, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NzEzNTcxMw==", "user": {"value": 3243482, "label": "abdusco"}, "created_at": "2020-06-21T14:30:02Z", "updated_at": "2020-06-21T14:30:02Z", "author_association": "CONTRIBUTOR", "body": "Oops, the same method is called from both index and database pages. But removing select count queries speed up the page load quite a bit.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 642572841, "label": "Database page loads too slowly with many large tables (due to table counts)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/357#issuecomment-647189045", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/357", "id": 647189045, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NzE4OTA0NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-21T22:19:58Z", "updated_at": "2020-06-21T22:19:58Z", "author_association": "OWNER", "body": "I'm going to take this in a different direction.\r\n\r\nI'm not happy with how `metadata.(json|yaml)` keeps growing new features. Rather than having a single plugin hook for all of `metadata.json` I'm going to split out the feature that shows actual real metadata for tables and databases - `source`, `license` etc - into its own plugin-powered mechanism.\r\n\r\nSo I'm going to close this ticket and spin up a new one for that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 348043884, "label": "Plugin hook for loading metadata.json"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/860#issuecomment-647189535", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/860", "id": 647189535, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NzE4OTUzNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-21T22:25:16Z", "updated_at": "2020-06-21T22:25:27Z", "author_association": "OWNER", "body": "This is also relevant to #639, and may mean I can close that ticket in place of this one. I'm going to get this at least to a proof-of-concept stage first though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 642651572, "label": "Plugin hook for instance/database/table metadata"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/859#issuecomment-647189666", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/859", "id": 647189666, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NzE4OTY2Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-21T22:26:55Z", "updated_at": "2020-06-21T22:26:55Z", "author_association": "OWNER", "body": "This makes a lot of sense. I implemented the mechanism for the index page because I have my own instance of Datasette that was running slow, but it had a dozen database files attached to it. I've not run into this with a single giant database file but it absolutely makes sense that the same optimization would be necessary for the database page there too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 642572841, "label": "Database page loads too slowly with many large tables (due to table counts)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/859#issuecomment-647189948", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/859", "id": 647189948, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NzE4OTk0OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-21T22:30:12Z", "updated_at": "2020-06-21T22:30:43Z", "author_association": "OWNER", "body": "I'll write a little script which generates a 300MB SQLite file with a bunch of tables with lots of randomly generated rows in to help test this.\r\n\r\nHaving a tool like that which can generate larger databases with different gnarly performance characteristics will be useful for other performance work too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 642572841, "label": "Database page loads too slowly with many large tables (due to table counts)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/687#issuecomment-647190144", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/687", "id": 647190144, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NzE5MDE0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-21T22:32:13Z", "updated_at": "2020-06-21T22:32:13Z", "author_association": "OWNER", "body": "So the new plan is NOT to have a `plugins/` folder, but instead have several top-level pages:\r\n\r\n- Plugins (exists)\r\n- Writing plugins\r\n- Plugin hooks\r\n- Testing plugins\r\n- Internals for plugins (exists)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 572896293, "label": "Expand plugins documentation to multiple pages"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/687#issuecomment-647190177", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/687", "id": 647190177, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NzE5MDE3Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-21T22:32:36Z", "updated_at": "2020-06-21T22:32:36Z", "author_association": "OWNER", "body": "I'm going to break out the plugin hooks first in a single commit to make for a cleaner commit history (since that way git can hopefully detect that the content moved).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 572896293, "label": "Expand plugins documentation to multiple pages"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/859#issuecomment-647194131", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/859", "id": 647194131, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NzE5NDEzMQ==", "user": {"value": 3243482, "label": "abdusco"}, "created_at": "2020-06-21T23:15:54Z", "updated_at": "2020-06-21T23:26:09Z", "author_association": "CONTRIBUTOR", "body": "I'm not sure if table counts are to blame. There shouldn't be a ~3 orders of magnitude difference.\r\n\r\n```fish\r\nuser@klein /a/w/scrapyard (master)> set sql \"select count(*) from table_1; select count(*) from table_2; select count(*) from table_3;\"\r\nuser@klein /a/w/scrapyard (master)> time sqlite3 scrapyard.db \"$sql\"\r\n187489\r\n46492\r\n2229\r\n\r\n________________________________________________________\r\nExecuted in   25.57 millis    fish           external\r\n   usr time    3.55 millis    0.00 micros    3.55 millis\r\n   sys time   22.42 millis  1123.00 micros   21.30 millis\r\n```\r\n\r\nbut not letting datasette count the tables definitely helps.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 642572841, "label": "Database page loads too slowly with many large tables (due to table counts)"}, "performed_via_github_app": null}