{"html_url": "https://github.com/simonw/sqlite-utils/issues/355#issuecomment-991386841", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/355", "id": 991386841, "node_id": "IC_kwDOCGYnMM47F1zZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T00:14:11Z", "updated_at": "2021-12-11T00:15:15Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/sqlite-utils/blob/7a43af232e4bc00bd227307665163614e225948b/sqlite_utils/cli.py#L2128-L2135\r\n\r\nOne way to implement this would be to look to see if the code starts with `def ...` - but that's not going to work for proper module that start with a docstring or imports.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077322009, "label": "Allow users to pass a full convert() function definition"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/355#issuecomment-991387044", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/355", "id": 991387044, "node_id": "IC_kwDOCGYnMM47F12k", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T00:14:45Z", "updated_at": "2021-12-11T00:14:45Z", "author_association": "OWNER", "body": "Maybe attempt to compile their code, and if it fails try again after adding `def fn(value):` to the start?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077322009, "label": "Allow users to pass a full convert() function definition"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/355#issuecomment-991393684", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/355", "id": 991393684, "node_id": "IC_kwDOCGYnMM47F3eU", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T00:42:19Z", "updated_at": "2021-12-11T00:49:49Z", "author_association": "OWNER", "body": "Ideally I'd like to show the perfect syntax error messages to the user - but I don't know if it's possible to do this cleanly because the error might occur with their originally entered code OR it might occur after I add `def fn(value)` to it.\r\n\r\nI'm going to punt on that for the moment and tolerate slightly confusing syntax errors.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077322009, "label": "Allow users to pass a full convert() function definition"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/355#issuecomment-991395494", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/355", "id": 991395494, "node_id": "IC_kwDOCGYnMM47F36m", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T00:50:22Z", "updated_at": "2021-12-11T00:51:15Z", "author_association": "OWNER", "body": "Here's an example of the new (slightly confusing) error message:\r\n```bash\r\nsqlite-utils convert fixtures.db roadside_attractions name '\r\ndef foo(value) \r\n bar\r\n baz\r\n'\r\nError: Syntax error in code:\r\n\r\n def foo(value)\r\n\r\ninvalid syntax\r\n```\r\nAnother:\r\n```\r\nsqlite-utils convert fixtures.db roadside_attractions name '$'\r\nError: Syntax error in code:\r\n\r\n return $\r\n\r\ninvalid syntax\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077322009, "label": "Allow users to pass a full convert() function definition"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/354#issuecomment-991395919", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/354", "id": 991395919, "node_id": "IC_kwDOCGYnMM47F4BP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T00:52:31Z", "updated_at": "2021-12-11T00:52:31Z", "author_association": "OWNER", "body": "It turns out `rebuild` does indeed work against content tables, so I can put that in the test instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077243232, "label": "Test failure in test_rebuild_fts"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/347#issuecomment-991397907", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347", "id": 991397907, "node_id": "IC_kwDOCGYnMM47F4gT", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T01:01:40Z", "updated_at": "2021-12-11T01:01:40Z", "author_association": "OWNER", "body": "The change I made to that test in #354 might help with this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1066603133, "label": "Test against pysqlite3 running SQLite 3.37"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/354#issuecomment-991398367", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/354", "id": 991398367, "node_id": "IC_kwDOCGYnMM47F4nf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T01:03:14Z", "updated_at": "2021-12-11T01:03:14Z", "author_association": "OWNER", "body": "The new test: https://github.com/simonw/sqlite-utils/blob/ee13f98c2c7ca3b819bd0fc55da3108cb6a6434a/tests/test_fts.py#L270-L277", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077243232, "label": "Test failure in test_rebuild_fts"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/354#issuecomment-991399604", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/354", "id": 991399604, "node_id": "IC_kwDOCGYnMM47F460", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T01:08:46Z", "updated_at": "2021-12-11T01:08:46Z", "author_association": "OWNER", "body": "That passed!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077243232, "label": "Test failure in test_rebuild_fts"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991399782", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/353", "id": 991399782, "node_id": "IC_kwDOCGYnMM47F49m", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T01:09:37Z", "updated_at": "2021-12-11T01:09:37Z", "author_association": "OWNER", "body": "OK, this is implemented. Updated documentation is here: https://sqlite-utils.datasette.io/en/latest/cli.html#converting-data-in-columns", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077102934, "label": "Allow passing a file of code to \"sqlite-utils convert\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991400016", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/353", "id": 991400016, "node_id": "IC_kwDOCGYnMM47F5BQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T01:10:52Z", "updated_at": "2021-12-11T01:11:02Z", "author_association": "OWNER", "body": "This won't be in a release for a little while, but you can install it to try it out using:\r\n\r\n pip install https://github.com/simonw/sqlite-utils/archive/ee13f98c2c.zip", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077102934, "label": "Allow passing a file of code to \"sqlite-utils convert\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/356#issuecomment-991517209", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/356", "id": 991517209, "node_id": "IC_kwDOCGYnMM47GVoZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T07:46:41Z", "updated_at": "2021-12-11T07:46:41Z", "author_association": "OWNER", "body": "By default this will accept single lines, but maybe there could be a `--all` option which instead grabs all of stdin into a single string against which the conversion function runs - like `git-history file`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077431957, "label": "`sqlite-utils insert --convert` option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1549#issuecomment-991752486", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1549", "id": 991752486, "node_id": "IC_kwDOBm6k_c47HPEm", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T19:09:15Z", "updated_at": "2021-12-11T19:09:15Z", "author_association": "OWNER", "body": "That's what this option does:\r\n\r\n![EAB1B9E8-38E9-4C6D-8854-BD1935F163D9](https://user-images.githubusercontent.com/9599/145688531-668bafa1-e287-4bbd-84d6-157241fb1f68.jpeg)\r\n\r\nThe usability of this is pretty terrible though (including \"stream all rows\" - how are people meant to understand what that does?) so it can definitely do with some rethinking.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077620955, "label": "Redesign CSV export to improve usability"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1549#issuecomment-991754794", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1549", "id": 991754794, "node_id": "IC_kwDOBm6k_c47HPoq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T19:16:33Z", "updated_at": "2021-12-11T19:16:33Z", "author_association": "OWNER", "body": "Good call! I'm doing a refactor #1518 right now which will hopefully bring the functionality of those two much closer - I'll make a note to consider this there too.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077620955, "label": "Redesign CSV export to improve usability"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/617#issuecomment-991755013", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/617", "id": 991755013, "node_id": "IC_kwDOBm6k_c47HPsF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T19:17:11Z", "updated_at": "2021-12-11T19:17:11Z", "author_association": "OWNER", "body": "This work is now happening in #1518 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 519613116, "label": "Refactor TableView.data() method"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1549#issuecomment-991755245", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1549", "id": 991755245, "node_id": "IC_kwDOBm6k_c47HPvt", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T19:17:54Z", "updated_at": "2021-12-11T19:17:54Z", "author_association": "OWNER", "body": "Also relevant:\r\n- #1062 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077620955, "label": "Redesign CSV export to improve usability"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1550#issuecomment-991761635", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1550", "id": 991761635, "node_id": "IC_kwDOBm6k_c47HRTj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T19:39:01Z", "updated_at": "2021-12-11T19:39:01Z", "author_association": "OWNER", "body": "I wonder if this could work for public instances too with some kind of queuing mechanism?\r\n\r\nI really need to use benchmarking to figure out what the right number of maximum SQLite connections is. I'm just guessing at the moment.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077628073, "label": "Research option for returning all rows from arbitrary query"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1550#issuecomment-991805516", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1550", "id": 991805516, "node_id": "IC_kwDOBm6k_c47HcBM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T23:43:24Z", "updated_at": "2021-12-11T23:43:24Z", "author_association": "OWNER", "body": "I built a tiny Starlette app to experiment with this a bit:\r\n```python\r\nimport asyncio\r\nimport janus\r\nfrom starlette.applications import Starlette\r\nfrom starlette.responses import JSONResponse, HTMLResponse, StreamingResponse\r\nfrom starlette.routing import Route\r\nimport sqlite3\r\nfrom concurrent import futures\r\n\r\nexecutor = futures.ThreadPoolExecutor(max_workers=10)\r\n\r\n\r\nasync def homepage(request):\r\n return HTMLResponse(\r\n \"\"\"\r\n \r\n