{"html_url": "https://github.com/simonw/datasette/issues/526#issuecomment-1260355224", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/526", "id": 1260355224, "node_id": "IC_kwDOBm6k_c5LH36Y", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-28T04:01:25Z", "updated_at": "2022-09-28T04:01:25Z", "author_association": "OWNER", "body": "The ultimate protection against those memory bombs is to support more streaming output formats. Related issues:\r\n\r\n- #1177 \r\n- #1062", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459882902, "label": "Stream all results for arbitrary SQL and canned queries"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1826#issuecomment-1260357583", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1826", "id": 1260357583, "node_id": "IC_kwDOBm6k_c5LH4fP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-28T04:05:16Z", "updated_at": "2022-09-28T04:05:16Z", "author_association": "OWNER", "body": "This is deliberate. The Datasette plugin system allows you to specify only a subset of the parameters for a hook - in this example, only the `value` is needed so the others can be omitted.\r\n\r\nThere's a note about this at the very top of that documentation page: https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hooks\r\n\r\n> When you implement a plugin hook you can accept any or all of the parameters that are documented as being passed to that hook.\r\n> \r\n> For example, you can implement the `render_cell` plugin hook like this even though the full documented hook signature is `render_cell(value, column, table, database, datasette)`:\r\n> ```python\r\n> @hookimpl\r\n> def render_cell(value, column):\r\n> if column == \"stars\":\r\n> return \"*\" * int(value)\r\n> ```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1388631785, "label": "render_cell documentation example doesn't match the method signature"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1826#issuecomment-1260357878", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1826", "id": 1260357878, "node_id": "IC_kwDOBm6k_c5LH4j2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-28T04:05:45Z", "updated_at": "2022-09-28T04:05:45Z", "author_association": "OWNER", "body": "Though now I notice that the copy right there needs to be updated to reflect the new `row` parameter to `render_cell`!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1388631785, "label": "render_cell documentation example doesn't match the method signature"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1825#issuecomment-1260368537", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1825", "id": 1260368537, "node_id": "IC_kwDOBm6k_c5LH7KZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-28T04:21:18Z", "updated_at": "2022-09-28T04:21:18Z", "author_association": "OWNER", "body": "This is great, thank you very much!\r\n\r\nhttps://datasette--1825.org.readthedocs.build/en/1825/deploying.html#running-datasette-using-openrc", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1388227245, "label": "Add documentation for serving via OpenRC"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1624#issuecomment-1261194164", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1624", "id": 1261194164, "node_id": "IC_kwDOBm6k_c5LLEu0", "user": {"value": 38532, "label": "palfrey"}, "created_at": "2022-09-28T16:54:22Z", "updated_at": "2022-09-28T16:54:22Z", "author_association": "NONE", "body": "https://github.com/simonw/datasette-cors seems to workaround this", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1122427321, "label": "Index page `/` has no CORS headers"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1062#issuecomment-1260829829", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1062", "id": 1260829829, "node_id": "IC_kwDOBm6k_c5LJryF", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2022-09-28T12:27:19Z", "updated_at": "2022-09-28T12:27:19Z", "author_association": "CONTRIBUTOR", "body": "for teaching `register_output_renderer` to stream it seems like the two options are to\r\n\r\n1. a [nested query technique ](https://github.com/simonw/datasette/issues/526#issuecomment-505162238)to paginate through\r\n2. a fetching model that looks like something\r\n```python\r\nwith sqlite_timelimit(conn, time_limit_ms):\r\n c.execute(query)\r\n for chunk in c.fetchmany(chunk_size):\r\n yield from chunk\r\n```\r\ncurrently `db.execute` is not a generator, so this would probably need a new method?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 732674148, "label": "Refactor .csv to be an output renderer - and teach register_output_renderer to stream all rows"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1062#issuecomment-1260909128", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1062", "id": 1260909128, "node_id": "IC_kwDOBm6k_c5LJ_JI", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2022-09-28T13:22:53Z", "updated_at": "2022-09-28T14:09:54Z", "author_association": "CONTRIBUTOR", "body": "if you went this route:\r\n\r\n```python\r\nwith sqlite_timelimit(conn, time_limit_ms):\r\n c.execute(query)\r\n for chunk in c.fetchmany(chunk_size):\r\n yield from chunk\r\n```\r\n\r\nthen `time_limit_ms` would probably have to be greatly extended, because the time spent in the loop will depend on the downstream processing.\r\n\r\ni wonder if this was why you were thinking this feature would need a dedicated connection?\r\n\r\n---\r\n\r\nreading more, there's no real limit i can find on the number of active cursors (or more precisely active prepared statements objects, because sqlite doesn't really have cursors). \r\n\r\nmaybe something like this would be okay?\r\n\r\n```python\r\nwith sqlite_timelimit(conn, time_limit_ms):\r\n c.execute(query)\r\n # step through at least one to evaluate the statement, not sure if this is necessary\r\n yield c.execute.fetchone()\r\nfor chunk in c.fetchmany(chunk_size):\r\n yield from chunk\r\n```\r\n\r\nthis seems quite weird that there's not more of limit of the number of active prepared statements, but i haven't been able to find one.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 732674148, "label": "Refactor .csv to be an output renderer - and teach register_output_renderer to stream all rows"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1825#issuecomment-1260368122", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1825", "id": 1260368122, "node_id": "IC_kwDOBm6k_c5LH7D6", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2022-09-28T04:20:28Z", "updated_at": "2022-09-28T04:20:28Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1825?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nBase: **91.58**% // Head: **91.58**% // No change to project coverage :thumbsup:\n> Coverage data is based on head [(`b16eb2f`)](https://codecov.io/gh/simonw/datasette/pull/1825?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`5f9f567`)](https://codecov.io/gh/simonw/datasette/commit/5f9f567acbc58c9fcd88af440e68034510fb5d2b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> Patch has no changes to coverable lines.\n\n> :exclamation: Current head b16eb2f differs from pull request most recent head e7e96dc. Consider uploading reports for the commit e7e96dc to get more accurate results\n\nAdditional details and impacted files
\n\n\n```diff\n@@ Coverage Diff @@\n## main #1825 +/- ##\n=======================================\n Coverage 91.58% 91.58% \n=======================================\n Files 36 36 \n Lines 4444 4444 \n=======================================\n Hits 4070 4070 \n Misses 374 374 \n```\n\n\n\nHelp us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n