html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app https://github.com/simonw/sqlite-utils/issues/550#issuecomment-1556250236,https://api.github.com/repos/simonw/sqlite-utils/issues/550,1556250236,IC_kwDOCGYnMM5cwn58,9599,2023-05-21T18:25:26Z,2023-05-21T18:25:26Z,OWNER,"Relevant issues: - https://github.com/python/importlib_metadata/issues/406 - https://github.com/PyCQA/flake8/issues/1701 It looks to me like this is only a problem for `flake8` on Python 3.7 - 3.8 and higher work OK.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1718595700, https://github.com/simonw/sqlite-utils/issues/550#issuecomment-1556249984,https://api.github.com/repos/simonw/sqlite-utils/issues/550,1556249984,IC_kwDOCGYnMM5cwn2A,9599,2023-05-21T18:24:48Z,2023-05-21T18:24:48Z,OWNER,"This is blocking: - #549","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1718595700, https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556247818,https://api.github.com/repos/simonw/sqlite-utils/issues/545,1556247818,IC_kwDOCGYnMM5cwnUK,9599,2023-05-21T18:17:46Z,2023-05-21T18:17:46Z,OWNER,Draft documentation: https://sqlite-utils--549.org.readthedocs.build/en/549/cli.html#cli-tui,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1718517882, https://github.com/simonw/sqlite-utils/pull/549#issuecomment-1556242262,https://api.github.com/repos/simonw/sqlite-utils/issues/549,1556242262,IC_kwDOCGYnMM5cwl9W,9599,2023-05-21T18:00:05Z,2023-05-21T18:00:05Z,OWNER,"Failing `mypy` test: https://github.com/simonw/sqlite-utils/actions/runs/5038983349/jobs/9036828465 ``` sqlite_utils/cli.py:37: error: Skipping analyzing ""trogon"": module is installed, but missing library stubs or py.typed marker [import] sqlite_utils/cli.py:37: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports Found 1 error in 1 file (checked 52 source files) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1718586377, https://github.com/simonw/sqlite-utils/pull/549#issuecomment-1556241812,https://api.github.com/repos/simonw/sqlite-utils/issues/549,1556241812,IC_kwDOCGYnMM5cwl2U,9599,2023-05-21T17:58:25Z,2023-05-21T17:58:25Z,OWNER,Documentation: https://sqlite-utils--549.org.readthedocs.build/en/549/cli.html#cli-tui,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1718586377, https://github.com/simonw/sqlite-utils/issues/548#issuecomment-1556231832,https://api.github.com/repos/simonw/sqlite-utils/issues/548,1556231832,IC_kwDOCGYnMM5cwjaY,9599,2023-05-21T17:24:13Z,2023-05-21T17:24:13Z,OWNER,"Oh, I see why that is now: https://github.com/simonw/sqlite-utils/blob/6027f3ea6939a399aeef2578fca17efec0e539df/sqlite_utils/cli.py#L2670-L2679 This is because of the following command: sqlite-utils analyze-tables table1 table2 --column x Since you can pass multiple tables AND multiple columns, the tool currently assumes that the column(s) you specify may be available on a subset of the provided tables. I'm going to change this so if the column is not on ANY of those tables you get an error.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1718576761, https://github.com/simonw/sqlite-utils/issues/547#issuecomment-1556228395,https://api.github.com/repos/simonw/sqlite-utils/issues/547,1556228395,IC_kwDOCGYnMM5cwikr,9599,2023-05-21T17:11:15Z,2023-05-21T17:11:15Z,OWNER,"This will be a cosmetic change to the CLI output only - the options to save data to the database and the Python API function will continue to return `[(None, 158)]`. I can add an optimization though to avoid running the SQL count query if we know that it's all `null`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1718572201, https://github.com/simonw/sqlite-utils/issues/544#issuecomment-1556225788,https://api.github.com/repos/simonw/sqlite-utils/issues/544,1556225788,IC_kwDOCGYnMM5cwh78,9599,2023-05-21T17:02:05Z,2023-05-21T17:02:05Z,OWNER,"New docs: - https://sqlite-utils.datasette.io/en/latest/cli.html#cli-analyze-tables - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#analyze-tables - https://sqlite-utils.datasette.io/en/latest/python-api.html#analyzing-a-column - https://sqlite-utils.datasette.io/en/latest/reference.html#sqlite_utils.db.Table.analyze_column New help output: ``` % sqlite-utils analyze-tables --help Usage: sqlite-utils analyze-tables [OPTIONS] PATH [TABLES]... Analyze the columns in one or more tables Example: sqlite-utils analyze-tables data.db trees Options: -c, --column TEXT Specific columns to analyze --save Save results to _analyze_tables table --common-limit INTEGER How many common values --no-most Skip most common values --no-least Skip least common values --load-extension TEXT Path to SQLite extension, with optional :entrypoint -h, --help Show this message and exit. ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1718515590, https://github.com/simonw/sqlite-utils/pull/546#issuecomment-1556213396,https://api.github.com/repos/simonw/sqlite-utils/issues/546,1556213396,IC_kwDOCGYnMM5cwe6U,9599,2023-05-21T15:58:12Z,2023-05-21T16:18:46Z,OWNER,"Documentation preview: - https://sqlite-utils--546.org.readthedocs.build/en/546/cli.html#cli-analyze-tables - https://sqlite-utils--546.org.readthedocs.build/en/546/cli-reference.html#analyze-tables - https://sqlite-utils--546.org.readthedocs.build/en/546/python-api.html#analyzing-a-column - https://sqlite-utils--546.org.readthedocs.build/en/546/reference.html#sqlite_utils.db.Table.analyze_column","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1718550688, https://github.com/simonw/sqlite-utils/issues/544#issuecomment-1556211643,https://api.github.com/repos/simonw/sqlite-utils/issues/544,1556211643,IC_kwDOCGYnMM5cwee7,9599,2023-05-21T15:48:17Z,2023-05-21T15:48:17Z,OWNER,I generated the commit message in https://github.com/simonw/sqlite-utils/commit/1c1991b447a1ddd3d61d9d4a8a1d6a9da47ced20 using `git diff | llm --system 'describe this change'`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1718515590, https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556210844,https://api.github.com/repos/simonw/sqlite-utils/issues/545,1556210844,IC_kwDOCGYnMM5cweSc,9599,2023-05-21T15:44:10Z,2023-05-21T15:44:10Z,OWNER,"It looks like `nargs=-1` on a positional argument isn't yet supported - opened an issue here: - https://github.com/Textualize/trogon/issues/4","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1718517882, https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556191894,https://api.github.com/repos/simonw/sqlite-utils/issues/545,1556191894,IC_kwDOCGYnMM5cwZqW,9599,2023-05-21T14:20:14Z,2023-05-21T14:20:14Z,OWNER,"Opened a feature request for customizing the help and command name: - https://github.com/Textualize/trogon/issues/2","{""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1718517882, https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556190531,https://api.github.com/repos/simonw/sqlite-utils/issues/545,1556190531,IC_kwDOCGYnMM5cwZVD,9599,2023-05-21T14:13:43Z,2023-05-21T14:13:43Z,OWNER,"OK, this works! ![trogon](https://github.com/simonw/sqlite-utils/assets/9599/2ae194c5-ec82-471a-9d1b-b01b3f2632f3) To try it out, install that branch from GitHub: pip install https://github.com/simonw/sqlite-utils/archive/refs/heads/trogon.zip Then run this: sqlite-utils install trogon And this: sqlite-utils tui ","{""total_count"": 5, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 3, ""eyes"": 0}",1718517882, https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556189823,https://api.github.com/repos/simonw/sqlite-utils/issues/545,1556189823,IC_kwDOCGYnMM5cwZJ_,9599,2023-05-21T14:09:59Z,2023-05-21T14:09:59Z,OWNER,"I don't want to add `trogon` as a default dependency because it's a little heavy - it pulls in all of Rich and Textual as well. People who use `sqlite-utils` just for its Python API won't benefit from this - it's a CLI feature only. But I have a `sqlite-utils install ...` command for helping people to install packages into the same virtual environment as `sqlite-utils` no matter how they installed that tool: https://sqlite-utils.datasette.io/en/stable/cli.html#cli-install So I can treat Trogon as an optional dependency and add the `sqlite-utils tui` command only if that package is also installed.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1718517882, https://github.com/simonw/datasette/issues/2073#issuecomment-1546119773,https://api.github.com/repos/simonw/datasette/issues/2073,1546119773,IC_kwDOBm6k_c5cJ-pd,9599,2023-05-12T18:24:07Z,2023-05-12T18:24:07Z,OWNER,"Here's a demo of this breaking in Datasette Lite: https://lite.datasette.io/?sql=https://gist.github.com/simonw/261564c0ca01567df6eeb9b222b8be84&json=https%3A%2F%2Fcdn.jsdelivr.net%2Fnpm%2Fweb-features%2Findex.json#/data/baseline?_filter_column_1=is_baseline&_filter_op_1=exact&_filter_value_1=1&_filter_column_2=&_filter_op_2=notnull__1&_filter_value_2=1&_filter_column=&_filter_op=exact&_filter_value=&_sort=&_facet=is_baseline Here's a SQL query that demonstrates the underlying issue: ```sql select 'working', count(*) from baseline where is_baseline = 1 union all select 'broken', count(*) from baseline where is_baseline = '1' ``` https://lite.datasette.io/?sql=https://gist.github.com/simonw/261564c0ca01567df6eeb9b222b8be84&json=https%3A%2F%2Fcdn.jsdelivr.net%2Fnpm%2Fweb-features%2Findex.json#/data?sql=select+%27working%27%2C+count%28*%29+from+baseline+where+is_baseline+%3D+1%0Aunion+all%0Aselect+%27broken%27%2C+count%28*%29+from+baseline+where+is_baseline+%3D+%271%27 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1708030220, https://github.com/simonw/datasette/issues/2073#issuecomment-1546117538,https://api.github.com/repos/simonw/datasette/issues/2073,1546117538,IC_kwDOBm6k_c5cJ-Gi,9599,2023-05-12T18:21:38Z,2023-05-12T18:21:38Z,OWNER,"https://latest.datasette.io/fixtures doesn't currently have a view with any integer columns in it, making this bug harder to demonstrate there. I can't replicate the bug using https://datasette.io/content/plugins?_facet=stargazers_count&stargazers_count=3 - I would expect that not to work correctly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1708030220, https://github.com/simonw/datasette/issues/2070#issuecomment-1540494121,https://api.github.com/repos/simonw/datasette/issues/2070,1540494121,IC_kwDOBm6k_c5b0hMp,9599,2023-05-09T16:25:00Z,2023-05-09T16:25:00Z,OWNER,Can now be used here: https://github.com/simonw/datasette/actions/workflows/deploy-branch-preview.yml,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1702354223, https://github.com/simonw/datasette/issues/2070#issuecomment-1540491751,https://api.github.com/repos/simonw/datasette/issues/2070,1540491751,IC_kwDOBm6k_c5b0gnn,9599,2023-05-09T16:23:12Z,2023-05-09T16:23:12Z,OWNER,Added a actions `BRANCH_PREVIEW_VERCEL_TOKEN` secret to this repository.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1702354223, https://github.com/simonw/sqlite-utils/pull/537#issuecomment-1539157643,https://api.github.com/repos/simonw/sqlite-utils/issues/537,1539157643,IC_kwDOCGYnMM5bva6L,9599,2023-05-08T22:45:09Z,2023-05-08T22:45:21Z,OWNER,"Here's an example from the new tests: https://github.com/simonw/sqlite-utils/blob/a75abeb61b91a28650d3b9933e7ec80ad0d92529/tests/test_create.py#L291-L307","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1665200812, https://github.com/simonw/sqlite-utils/issues/448#issuecomment-1539109816,https://api.github.com/repos/simonw/sqlite-utils/issues/448,1539109816,IC_kwDOCGYnMM5bvPO4,9599,2023-05-08T22:01:00Z,2023-05-08T22:01:00Z,OWNER,"This is being handled in: - #520","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1279144769, https://github.com/simonw/sqlite-utils/issues/520#issuecomment-1539109587,https://api.github.com/repos/simonw/sqlite-utils/issues/520,1539109587,IC_kwDOCGYnMM5bvPLT,9599,2023-05-08T22:00:46Z,2023-05-08T22:00:46Z,OWNER,"> Hey, isn't this essentially the same issue as #448 ? Yes it is, good catch!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1516644980, https://github.com/simonw/sqlite-utils/issues/525#issuecomment-1539108140,https://api.github.com/repos/simonw/sqlite-utils/issues/525,1539108140,IC_kwDOCGYnMM5bvO0s,9599,2023-05-08T21:59:41Z,2023-05-08T21:59:41Z,OWNER,That original example passes against `main` now.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1575131737, https://github.com/simonw/sqlite-utils/issues/525#issuecomment-1539101853,https://api.github.com/repos/simonw/sqlite-utils/issues/525,1539101853,IC_kwDOCGYnMM5bvNSd,9599,2023-05-08T21:52:44Z,2023-05-08T21:52:44Z,OWNER,I like the `lambda-{uuid}` idea.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1575131737, https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539100300,https://api.github.com/repos/simonw/sqlite-utils/issues/514,1539100300,IC_kwDOCGYnMM5bvM6M,9599,2023-05-08T21:50:51Z,2023-05-08T21:50:51Z,OWNER,Seeing as `sqlite-utils` doesn't currently provide mechanisms for adding `check` constraints like this I'm going to leave this - I'm happy with the fix I put in for the `not null` constraints.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1465194249, https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539099703,https://api.github.com/repos/simonw/sqlite-utils/issues/514,1539099703,IC_kwDOCGYnMM5bvMw3,9599,2023-05-08T21:50:06Z,2023-05-08T21:50:06Z,OWNER,"Applying the fix from the PR here doesn't fix the above problem either: - https://github.com/simonw/sqlite-utils/pull/515 So it looks like these kinds of `check` constraints currently aren't compatible with how `upsert()` works.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1465194249, https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539094287,https://api.github.com/repos/simonw/sqlite-utils/issues/514,1539094287,IC_kwDOCGYnMM5bvLcP,9599,2023-05-08T21:44:11Z,2023-05-08T21:44:11Z,OWNER,"OK, this fails silently: ```python import sqlite_utils db = sqlite_utils.Database(memory=True) db.execute('''CREATE TABLE employees ( id INTEGER PRIMARY KEY, name TEXT, age INTEGER, salary REAL, CHECK (salary is not null and salary > 0) );''') db[""employees""].upsert({""id"": 1, ""name"": ""Bob""}, pk=""id"") list(db[""employees""].rows) ```` It outputs: ```python [] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1465194249, https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539079507,https://api.github.com/repos/simonw/sqlite-utils/issues/514,1539079507,IC_kwDOCGYnMM5bvH1T,9599,2023-05-08T21:28:37Z,2023-05-08T21:28:37Z,OWNER,"> This means that a table with NON NULL (or other constraint) columns that aren't part of the pkey can't have new rows upserted. Huh... on that basis, it's possible my fix in https://github.com/simonw/sqlite-utils/commit/2376c452a56b0c3e75e7ca698273434e32945304 is incomplete. I only covered the 'not null' case.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1465194249, https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539078429,https://api.github.com/repos/simonw/sqlite-utils/issues/514,1539078429,IC_kwDOCGYnMM5bvHkd,9599,2023-05-08T21:27:40Z,2023-05-08T21:27:40Z,OWNER,"Dupe of: - #538","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1465194249, https://github.com/simonw/sqlite-utils/pull/515#issuecomment-1539077777,https://api.github.com/repos/simonw/sqlite-utils/issues/515,1539077777,IC_kwDOCGYnMM5bvHaR,9599,2023-05-08T21:27:10Z,2023-05-08T21:27:10Z,OWNER,I should have spotted this PR before I shipped my own fix! https://github.com/simonw/sqlite-utils/commit/2376c452a56b0c3e75e7ca698273434e32945304,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1465194930, https://github.com/simonw/sqlite-utils/pull/519#issuecomment-1539058795,https://api.github.com/repos/simonw/sqlite-utils/issues/519,1539058795,IC_kwDOCGYnMM5bvCxr,9599,2023-05-08T21:12:52Z,2023-05-08T21:12:52Z,OWNER,"This is a really neat fix, thank you.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1505568103, https://github.com/simonw/sqlite-utils/pull/537#issuecomment-1539055393,https://api.github.com/repos/simonw/sqlite-utils/issues/537,1539055393,IC_kwDOCGYnMM5bvB8h,9599,2023-05-08T21:10:06Z,2023-05-08T21:10:06Z,OWNER,Thanks!,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1665200812, https://github.com/simonw/sqlite-utils/pull/528#issuecomment-1539053230,https://api.github.com/repos/simonw/sqlite-utils/issues/528,1539053230,IC_kwDOCGYnMM5bvBau,9599,2023-05-08T21:08:23Z,2023-05-08T21:08:23Z,OWNER,"I fixed this in: - #527 Will fully remove this misfeature in: - #542","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1578793661, https://github.com/simonw/sqlite-utils/issues/542#issuecomment-1539052467,https://api.github.com/repos/simonw/sqlite-utils/issues/542,1539052467,IC_kwDOCGYnMM5bvBOz,9599,2023-05-08T21:07:41Z,2023-05-08T21:07:41Z,OWNER,"Relevant commits (will mostly revert these): - https://github.com/simonw/sqlite-utils/commit/455c35b512895c19bf922c2b804d750d27cb8dbd - https://github.com/simonw/sqlite-utils/commit/e0ec4c345129996011951e400388fd74865f65a2","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1700936245, https://github.com/simonw/sqlite-utils/issues/527#issuecomment-1539051724,https://api.github.com/repos/simonw/sqlite-utils/issues/527,1539051724,IC_kwDOCGYnMM5bvBDM,9599,2023-05-08T21:07:04Z,2023-05-08T21:07:04Z,OWNER,"Updated documentation: - https://sqlite-utils.datasette.io/en/latest/python-api.html#converting-data-in-columns - https://sqlite-utils.datasette.io/en/latest/cli.html#converting-data-in-columns - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#convert","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1578790070, https://github.com/simonw/sqlite-utils/issues/527#issuecomment-1539035838,https://api.github.com/repos/simonw/sqlite-utils/issues/527,1539035838,IC_kwDOCGYnMM5bu9K-,9599,2023-05-08T20:55:00Z,2023-05-08T20:55:00Z,OWNER,"I'm going to go with `--no-skip-false` as the CLI option. It's ugly, but this whole thing is ugly. I'm going to make a note to remove this misfeature in `sqlite-utils` 4.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1578790070, https://github.com/simonw/sqlite-utils/issues/527#issuecomment-1539033736,https://api.github.com/repos/simonw/sqlite-utils/issues/527,1539033736,IC_kwDOCGYnMM5bu8qI,9599,2023-05-08T20:52:51Z,2023-05-08T20:52:51Z,OWNER,"OK, I implemented that at the Python API level. I need to decide how it should work for the `sqlite-utils convert` command too: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#convert","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1578790070, https://github.com/simonw/sqlite-utils/issues/530#issuecomment-1539018912,https://api.github.com/repos/simonw/sqlite-utils/issues/530,1539018912,IC_kwDOCGYnMM5bu5Cg,9599,2023-05-08T20:39:00Z,2023-05-08T20:39:00Z,OWNER,"I think the natural place to add these in the Python library API would be https://sqlite-utils.datasette.io/en/stable/python-api.html#adding-foreign-key-constraints - maybe something like this: ```python db[""books""].add_foreign_key(""author_id"", ""authors"", ""id"", on_delete=RESTRICT) ``` Then for the CLI tool could be added to https://sqlite-utils.datasette.io/en/stable/cli-reference.html#add-foreign-key ``` sqlite-utils add-foreign-key my.db books author_id authors id --on-update SET_NULL ``` I wouldn't support these for the other methods of adding foreign keys - `foreign_keys(...)` for the various `.insert()` etc methods and the `add_foreign_keys(...)` bulk menthod.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 1, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1595340692, https://github.com/simonw/sqlite-utils/issues/530#issuecomment-1539015064,https://api.github.com/repos/simonw/sqlite-utils/issues/530,1539015064,IC_kwDOCGYnMM5bu4GY,9599,2023-05-08T20:35:07Z,2023-05-08T20:35:07Z,OWNER,"Wow, this is a neat feature I didn't know about. Looks like there are a bunch of options: - NO ACTION (default) - RESTRICT: application is prohibited from deleting a parent key when there exists one or more child keys mapped to it - SET NULL: when a parent key is deleted the child key columns of all rows in the child table that mapped to the parent key are set to contain SQL NULL values - SET DEFAULT: set a specific default - CASCADE: propagates the delete or update operation on the parent key to each dependent child key","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1595340692, https://github.com/simonw/sqlite-utils/issues/532#issuecomment-1539009453,https://api.github.com/repos/simonw/sqlite-utils/issues/532,1539009453,IC_kwDOCGYnMM5bu2ut,9599,2023-05-08T20:30:29Z,2023-05-08T20:30:42Z,OWNER,"Here's an improvement: ``` % sqlite-utils insert /tmp/b.db blah /tmp/blah.txt [####################################] 100% Error: Invalid JSON - use --csv for CSV or --tsv for TSV files JSON error: Expecting value: line 1 column 1 (char 0) ```","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1620254998, https://github.com/simonw/sqlite-utils/issues/532#issuecomment-1539006509,https://api.github.com/repos/simonw/sqlite-utils/issues/532,1539006509,IC_kwDOCGYnMM5bu2At,9599,2023-05-08T20:28:56Z,2023-05-08T20:28:56Z,OWNER,Was this a newline-delimited JSON file perhaps?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1620254998, https://github.com/simonw/sqlite-utils/issues/541#issuecomment-1538963959,https://api.github.com/repos/simonw/sqlite-utils/issues/541,1538963959,IC_kwDOCGYnMM5burn3,9599,2023-05-08T19:59:34Z,2023-05-08T19:59:34Z,OWNER,"There are 8 failing tests left: ``` ==== short test summary info ==== FAILED tests/test_cli_memory.py::test_memory_csv[False-test] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_csv[False-t] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_csv[False-t1] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_tsv[False] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_dump[extra_args0] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_two_files_with_same_stem - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_recipes.py::test_dateparse_errors[None-parsedate] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: .convert_value at 0x106bcaca0> FAILED tests/test_recipes.py::test_dateparse_errors[None-parsedatetime] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: .convert_value at 0x106bc9620> ERROR tests/test_cli.py::test_upsert_analyze - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> ==== 8 failed, 894 passed, 4 skipped, 1 error in 6.27s ==== ``` Full traceback here: https://gist.github.com/simonw/b40b3e814729d6c02a0302a84ce54d9e","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1700840265, https://github.com/simonw/sqlite-utils/issues/534#issuecomment-1538933540,https://api.github.com/repos/simonw/sqlite-utils/issues/534,1538933540,IC_kwDOCGYnMM5bukMk,9599,2023-05-08T19:34:37Z,2023-05-08T19:34:37Z,OWNER,"On macOS this shows the same warning: ``` % python -Wdefault $(which sqlite-utils) insert dogs.db dogs dogs.csv --csv [############------------------------] 35% [####################################] 100%/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py:1187: ResourceWarning: unclosed file <_io.TextIOWrapper name='dogs.csv' encoding='utf-8-sig'> insert_upsert_implementation( ResourceWarning: Enable tracemalloc to get the object allocation traceback ``` The file itself is a `click.File` which is automatically closed - https://click.palletsprojects.com/en/8.1.x/api/#click.File - but it looks like it's the `_io.TextIOWrapper` which is not being closed: https://github.com/simonw/sqlite-utils/blob/2376c452a56b0c3e75e7ca698273434e32945304/sqlite_utils/cli.py#L949-L956","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1622640374, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538921774,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538921774,IC_kwDOCGYnMM5buhUu,9599,2023-05-08T19:24:41Z,2023-05-08T19:24:41Z,OWNER,That fix seems to work!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538910894,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538910894,IC_kwDOCGYnMM5buequ,9599,2023-05-08T19:16:52Z,2023-05-08T19:17:00Z,OWNER,"How about if I had logic which checked that all not-null columns were provided in the call to `upsert_all()` - and if they were, modified the `INSERT OR IGNORE INTO` to include a placeholder value for those columns that would then be fixed by the later `UPDATE`? Something like this: ```python [ ('INSERT OR IGNORE INTO [comments]([id], name) VALUES(?, ?);', [1, '']), ('UPDATE [comments] SET [name] = ? WHERE [id] = ?', ['Cleo', 1]) ] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538903556,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538903556,IC_kwDOCGYnMM5buc4E,9599,2023-05-08T19:11:24Z,2023-05-08T19:13:23Z,OWNER,"I could detect if this happens using `cursor.rowcount` - not sure how I would recover from it though. This would also require some major re-engineering, since currently it all works by generating a list of SQL queries in advance and applying them inside a loop in `.insert_chunk()`: https://github.com/simonw/sqlite-utils/blob/80763edaa2bdaf1113717378b8d62075c4dcbcfb/sqlite_utils/db.py#L2839-L2878 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538893329,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538893329,IC_kwDOCGYnMM5buaYR,9599,2023-05-08T19:04:47Z,2023-05-08T19:04:47Z,OWNER,"This feels like a fundamental flaw in the way upserts are implemented by `sqlite-utils`. One fix would be to switch to using the `UPSERT` feature in SQLite: https://www.sqlite.org/lang_UPSERT.html But... > UPSERT syntax was added to SQLite with version 3.24.0 (2018-06-04). I still want to support SQLite versions earlier than that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538889482,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538889482,IC_kwDOCGYnMM5buZcK,9599,2023-05-08T19:02:38Z,2023-05-08T19:02:38Z,OWNER,"Here's the code at fault: https://github.com/simonw/sqlite-utils/blob/80763edaa2bdaf1113717378b8d62075c4dcbcfb/sqlite_utils/db.py#L2774-L2788","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538887361,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538887361,IC_kwDOCGYnMM5buY7B,9599,2023-05-08T19:01:20Z,2023-05-08T19:01:20Z,OWNER,"Here's the problem: ```python import sqlite3 db = sqlite3.connect("":memory:"") db.execute('create table foo (id integer primary key, name not null)') db.execute('insert into foo (id) values (1)') ``` Produces: ``` IntegrityError: NOT NULL constraint failed: foo.name ``` But this: ```python db.execute('insert or ignore into foo (id) values (1)') ``` Completes without an exception.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538801855,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538801855,IC_kwDOCGYnMM5buEC_,9599,2023-05-08T18:00:17Z,2023-05-08T18:00:17Z,OWNER,"From time in the debugger, after creating the table it ends up doing this: ``` (Pdb) queries_and_params [ ('INSERT OR IGNORE INTO [comments]([id]) VALUES(?);', [1]), ('UPDATE [comments] SET [name] = ? WHERE [id] = ?', ['Cleo', 1]) ] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538793817,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538793817,IC_kwDOCGYnMM5buCFZ,9599,2023-05-08T17:55:10Z,2023-05-08T17:55:10Z,OWNER,"Confirmed - I added this test and it fails: ```python def test_upsert_all_not_null(fresh_db): # https://github.com/simonw/sqlite-utils/issues/538 fresh_db[""comments""].upsert_all( [{""id"": 1, ""name"": ""Cleo""}], pk=""id"", not_null=[""name""], ) assert list(fresh_db[""comments""].rows) == [{""id"": 1, ""name"": ""Cleo""}] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537514610,https://api.github.com/repos/simonw/sqlite-utils/issues/539,1537514610,IC_kwDOCGYnMM5bpJxy,9599,2023-05-07T18:43:24Z,2023-05-07T18:43:24Z,OWNER,"Documentation: - https://sqlite-utils.datasette.io/en/latest/cli.html#returning-raw-data-such-as-binary-content - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#query - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#memory","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1699174055, https://github.com/simonw/sqlite-utils/issues/540#issuecomment-1537514069,https://api.github.com/repos/simonw/sqlite-utils/issues/540,1537514069,IC_kwDOCGYnMM5bpJpV,9599,2023-05-07T18:40:18Z,2023-05-07T18:40:18Z,OWNER,"https://docs.readthedocs.io/en/stable/config-file/v2.html suggests: ```yaml build: os: ubuntu-22.04 tools: python: ""3.11"" ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1699184583, https://github.com/simonw/sqlite-utils/issues/540#issuecomment-1537513912,https://api.github.com/repos/simonw/sqlite-utils/issues/540,1537513912,IC_kwDOCGYnMM5bpJm4,9599,2023-05-07T18:39:29Z,2023-05-07T18:39:29Z,OWNER,"https://readthedocs.org/projects/sqlite-utils/builds/20513034/ said: > Problem in your project's configuration. Invalid ""python.version"": expected one of (2, 2.7, 3, 3.5, 3.6, 3.7, 3.8, pypy3.5), got 3.11","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1699184583, https://github.com/simonw/sqlite-utils/issues/540#issuecomment-1537513653,https://api.github.com/repos/simonw/sqlite-utils/issues/540,1537513653,IC_kwDOCGYnMM5bpJi1,9599,2023-05-07T18:37:59Z,2023-05-07T18:38:19Z,OWNER,"Useful comment here: - https://github.com/urllib3/urllib3/issues/2168#issuecomment-1537360928 > I faced the same issue. I switched to a different Python kernel (3.11.2) and it worked.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1699184583, https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537507676,https://api.github.com/repos/simonw/sqlite-utils/issues/539,1537507676,IC_kwDOCGYnMM5bpIFc,9599,2023-05-07T18:09:43Z,2023-05-07T18:09:54Z,OWNER,"This worked: ```bash sqlite-utils memory /tmp/books3.json:nl \ 'select name from books3' --raw-lines > titles.txt ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1699174055, https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537507525,https://api.github.com/repos/simonw/sqlite-utils/issues/539,1537507525,IC_kwDOCGYnMM5bpIDF,9599,2023-05-07T18:09:09Z,2023-05-07T18:09:09Z,OWNER,"I'm tempted to upgrade `--raw` to do this instead, but that would be a breaking change.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1699174055, https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537507394,https://api.github.com/repos/simonw/sqlite-utils/issues/539,1537507394,IC_kwDOCGYnMM5bpIBC,9599,2023-05-07T18:08:44Z,2023-05-07T18:08:44Z,OWNER,"Prototype: ```diff diff --git a/docs/cli-reference.rst b/docs/cli-reference.rst index 153e5f9..c830518 100644 --- a/docs/cli-reference.rst +++ b/docs/cli-reference.rst @@ -124,6 +124,7 @@ See :ref:`cli_query`. --json-cols Detect JSON cols and output them as JSON, not escaped strings -r, --raw Raw output, first column of first row + --raw-lines Raw output, first column of each row -p, --param ... Named :parameters for SQL query --functions TEXT Python code defining one or more custom SQL functions @@ -192,6 +193,7 @@ See :ref:`cli_memory`. --json-cols Detect JSON cols and output them as JSON, not escaped strings -r, --raw Raw output, first column of first row + --raw-lines Raw output, first column of each row -p, --param ... Named :parameters for SQL query --encoding TEXT Character encoding for CSV input, defaults to utf-8 diff --git a/sqlite_utils/cli.py b/sqlite_utils/cli.py index d25b1df..da0e4b6 100644 --- a/sqlite_utils/cli.py +++ b/sqlite_utils/cli.py @@ -1653,6 +1653,7 @@ def drop_view(path, view, ignore, load_extension): ) @output_options @click.option(""-r"", ""--raw"", is_flag=True, help=""Raw output, first column of first row"") +@click.option(""--raw-lines"", is_flag=True, help=""Raw output, first column of each row"") @click.option( ""-p"", ""--param"", @@ -1677,6 +1678,7 @@ def query( fmt, json_cols, raw, + raw_lines, param, load_extension, functions, @@ -1700,7 +1702,19 @@ def query( _register_functions(db, functions) _execute_query( - db, sql, param, raw, table, csv, tsv, no_headers, fmt, nl, arrays, json_cols + db, + sql, + param, + raw, + raw_lines, + table, + csv, + tsv, + no_headers, + fmt, + nl, + arrays, + json_cols, ) @@ -1728,6 +1742,7 @@ def query( ) @output_options @click.option(""-r"", ""--raw"", is_flag=True, help=""Raw output, first column of first row"") +@click.option(""--raw-lines"", is_flag=True, help=""Raw output, first column of each row"") @click.option( ""-p"", ""--param"", @@ -1773,6 +1788,7 @@ def memory( fmt, json_cols, raw, + raw_lines, param, encoding, no_detect_types, @@ -1879,12 +1895,36 @@ def memory( _register_functions(db, functions) _execute_query( - db, sql, param, raw, table, csv, tsv, no_headers, fmt, nl, arrays, json_cols + db, + sql, + param, + raw, + raw_lines, + table, + csv, + tsv, + no_headers, + fmt, + nl, + arrays, + json_cols, ) def _execute_query( - db, sql, param, raw, table, csv, tsv, no_headers, fmt, nl, arrays, json_cols + db, + sql, + param, + raw, + raw_lines, + table, + csv, + tsv, + no_headers, + fmt, + nl, + arrays, + json_cols, ): with db.conn: try: @@ -1903,6 +1943,13 @@ def _execute_query( sys.stdout.buffer.write(data) else: sys.stdout.write(str(data)) + elif raw_lines: + for row in cursor: + data = row[0] + if isinstance(data, bytes): + sys.stdout.buffer.write(data + b""\n"") + else: + sys.stdout.write(str(data) + ""\n"") elif fmt or table: print( tabulate.tabulate( ``` Needs tests and more documentation.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1699174055, https://github.com/simonw/datasette/issues/2065#issuecomment-1524709988,https://api.github.com/repos/simonw/datasette/issues/2065,1524709988,IC_kwDOBm6k_c5a4Tpk,9599,2023-04-27T05:09:36Z,2023-04-27T05:09:36Z,OWNER,"That fixed it - after installing `main.zip` again I ran this and it worked: ~/.rye/tools/main-zip/bin/datasette install datasette-graphql ``` % ~/.rye/tools/main-zip/bin/datasette plugins [ { ""name"": ""datasette-graphql"", ""static"": true, ""templates"": true, ""version"": ""2.2"", ""hooks"": [ ""database_actions"", ""extra_template_vars"", ""menu_links"", ""register_routes"", ""startup"", ""table_actions"" ] } ] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652, https://github.com/simonw/datasette/issues/2065#issuecomment-1524707628,https://api.github.com/repos/simonw/datasette/issues/2065,1524707628,IC_kwDOBm6k_c5a4TEs,9599,2023-04-27T05:06:44Z,2023-04-27T05:06:44Z,OWNER,"I need `pip` as a dependency too: ``` % ~/.rye/tools/main-zip/bin/datasette install datasette-graphql Traceback (most recent call last): File ""/Users/simon/.rye/tools/main-zip/bin/datasette"", line 8, in sys.exit(cli()) ^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1130, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1055, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 760, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/datasette/cli.py"", line 365, in install run_module(""pip"", run_name=""__main__"") File """", line 222, in run_module File """", line 142, in _get_module_details ImportError: No module named pip ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652, https://github.com/simonw/datasette/issues/2065#issuecomment-1524699863,https://api.github.com/repos/simonw/datasette/issues/2065,1524699863,IC_kwDOBm6k_c5a4RLX,9599,2023-04-27T04:56:22Z,2023-04-27T04:56:22Z,OWNER,Turned this into a TIL: https://til.simonwillison.net/python/rye,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652, https://github.com/simonw/datasette/issues/2066#issuecomment-1524680160,https://api.github.com/repos/simonw/datasette/issues/2066,1524680160,IC_kwDOBm6k_c5a4MXg,9599,2023-04-27T04:27:50Z,2023-04-27T04:27:50Z,OWNER,That fixed it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269, https://github.com/simonw/datasette/issues/2066#issuecomment-1524675817,https://api.github.com/repos/simonw/datasette/issues/2066,1524675817,IC_kwDOBm6k_c5a4LTp,9599,2023-04-27T04:21:03Z,2023-04-27T04:21:03Z,OWNER,I went with this to generate the long string: https://github.com/simonw/datasette/blob/0b0c5cd7a94fe3f151a3e10261b5c84ee64f2f18/tests/test_csv.py#L157-L176,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269, https://github.com/simonw/datasette/issues/2066#issuecomment-1524669124,https://api.github.com/repos/simonw/datasette/issues/2066,1524669124,IC_kwDOBm6k_c5a4JrE,9599,2023-04-27T04:10:44Z,2023-04-27T04:10:52Z,OWNER,"I need an alternative way of generating a long string with a shorter URL. ```sql select group_concat('abcabcabc', '') from json_each(json_array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)) ``` https://latest.datasette.io/_memory?sql=select+group_concat%28%27abcabcabc%27%2C+%27%27%29+from+json_each%28json_array%281%2C+2%2C+3%2C+4%2C+5%2C+6%2C+7%2C+8%2C+9%2C+10%29%29","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269, https://github.com/simonw/datasette/issues/2066#issuecomment-1524666049,https://api.github.com/repos/simonw/datasette/issues/2066,1524666049,IC_kwDOBm6k_c5a4I7B,9599,2023-04-27T04:06:18Z,2023-04-27T04:06:18Z,OWNER,"Most recent `httpx` release is 0.24 a couple of weeks ago. Here's what changed in that version: https://github.com/encode/httpx/compare/0.23.3...0.24.0 It looks like that URL limit is new: https://github.com/encode/httpx/commit/57daabf673705954afa94686c0002801c93d31f3#diff-78d8d93b5dd4c77d99c3e2b46b7286ba71a8fd60e92d8bd4eee5fb200b4f87bfR149-R155 ```python def urlparse(url: str = """", **kwargs: typing.Optional[str]) -> ParseResult: # Initial basic checks on allowable URLs. # --------------------------------------- # Hard limit the maximum allowable URL length. if len(url) > MAX_URL_LENGTH: raise InvalidURL(""URL too long"") ``` https://github.com/encode/httpx/blob/32e25497a36e6222cc64a758c98275b450dac28d/httpx/_urlparse.py#L153-L155","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269, https://github.com/simonw/datasette/issues/2066#issuecomment-1524660603,https://api.github.com/repos/simonw/datasette/issues/2066,1524660603,IC_kwDOBm6k_c5a4Hl7,9599,2023-04-27T04:02:55Z,2023-04-27T04:02:55Z,OWNER,"In the debugger: ``` (Pdb) MAX_URL_LENGTH 65536 ``` Weird this only seems to be a problem with `httpx` on Python 3.7 though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269, https://github.com/simonw/datasette/issues/2066#issuecomment-1524659084,https://api.github.com/repos/simonw/datasette/issues/2066,1524659084,IC_kwDOBm6k_c5a4HOM,9599,2023-04-27T04:02:07Z,2023-04-27T04:02:07Z,OWNER,"This is the failing test: https://github.com/simonw/datasette/blob/249fcf8e3e2a90e763f41b080c1b9ec8017f5005/tests/test_csv.py#L156-L167","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269, https://github.com/simonw/datasette/issues/2066#issuecomment-1524655203,https://api.github.com/repos/simonw/datasette/issues/2066,1524655203,IC_kwDOBm6k_c5a4GRj,9599,2023-04-27T03:59:56Z,2023-04-27T03:59:56Z,OWNER,"In a fresh Datasette checkout I ran: pipenv shell --python /Users/simon/.pyenv/versions/3.7.16/bin/python That gave me a virtual environment with 3.7.16 Python. Then I ran: pip install -e '.[test]' Weirdly that gave me a `which pytest` of `/opt/homebrew/bin/pytest` which ran the tests on 3.11. I figured out the location of the virtual environment with `which python` and then ran this: ``` % /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/bin/pytest tests/test_csv.py ============================================================================================== test session starts =============================================================================================== platform darwin -- Python 3.7.16, pytest-7.3.1, pluggy-1.0.0 SQLite: 3.39.5 rootdir: /private/tmp/datasette configfile: pytest.ini plugins: asyncio-0.21.0, timeout-2.1.0, xdist-3.2.1, anyio-3.6.2 asyncio: mode=strict collected 15 items tests/test_csv.py ..........F.... [100%] ==================================================================================================== FAILURES ==================================================================================================== ________________________________________________________________________________________________ test_max_csv_mb _________________________________________________________________________________________________ app_client_csv_max_mb_one = def test_max_csv_mb(app_client_csv_max_mb_one): response = app_client_csv_max_mb_one.get( ( ""/fixtures.csv?sql=select+'{}'+"" ""from+compound_three_primary_keys&_stream=1&_size=max"" > ).format(""abcdefg"" * 10000) ) /private/tmp/datasette/tests/test_csv.py:161: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/asgiref/sync.py:240: in __call__ return call_result.result() /Users/simon/.pyenv/versions/3.7.16/lib/python3.7/concurrent/futures/_base.py:428: in result return self.__get_result() /Users/simon/.pyenv/versions/3.7.16/lib/python3.7/concurrent/futures/_base.py:384: in __get_result raise self._exception /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/asgiref/sync.py:306: in main_wrap result = await self.awaitable(*args, **kwargs) /private/tmp/datasette/datasette/utils/testing.py:76: in get headers=headers, /private/tmp/datasette/datasette/utils/testing.py:167: in _request content=post_body, /private/tmp/datasette/datasette/app.py:1787: in request method, self._fix(path, avoid_path_rewrites), **kwargs /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:1528: in request extensions=extensions, /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:346: in build_request url = self._merge_url(url) /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:376: in _merge_url merge_url = URL(url) /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_urls.py:113: in __init__ self._uri_reference = urlparse(url, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ url = ""http://localhost/fixtures.csv?sql=select+'abcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcde...gabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefg'+from+compound_three_primary_keys&_stream=1&_size=max"" kwargs = {} def urlparse(url: str = """", **kwargs: typing.Optional[str]) -> ParseResult: # Initial basic checks on allowable URLs. # --------------------------------------- # Hard limit the maximum allowable URL length. if len(url) > MAX_URL_LENGTH: > raise InvalidURL(""URL too long"") E httpx.InvalidURL: URL too long /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_urlparse.py:155: InvalidURL ============================================================================================ short test summary info ============================================================================================= FAILED tests/test_csv.py::test_max_csv_mb - httpx.InvalidURL: URL too long ========================================================================================== 1 failed, 14 passed in 1.83s ========================================================================================== (datasette) simon@Simons-MacBook-Pro datasette % ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269, https://github.com/simonw/datasette/issues/2066#issuecomment-1524648995,https://api.github.com/repos/simonw/datasette/issues/2066,1524648995,IC_kwDOBm6k_c5a4Ewj,9599,2023-04-27T03:56:42Z,2023-04-27T03:57:11Z,OWNER,"I don't have 3.7 locally. Trying to install it with `pyenv`. brew install pyenv Then: pyenv install --list | grep 3.7 Installing: pyenv install 3.7.16 Output: Installed Python-3.7.16 to /Users/simon/.pyenv/versions/3.7.16 So the executable is `/Users/simon/.pyenv/versions/3.7.16/bin/python`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269, https://github.com/simonw/datasette/issues/2066#issuecomment-1524638233,https://api.github.com/repos/simonw/datasette/issues/2066,1524638233,IC_kwDOBm6k_c5a4CIZ,9599,2023-04-27T03:50:51Z,2023-04-27T03:50:51Z,OWNER,Failure was on 3.7. I'll try that.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269, https://github.com/simonw/datasette/issues/2066#issuecomment-1524637376,https://api.github.com/repos/simonw/datasette/issues/2066,1524637376,IC_kwDOBm6k_c5a4B7A,9599,2023-04-27T03:50:19Z,2023-04-27T03:50:19Z,OWNER,"Having trouble replicating this on my laptop. I tried a fresh virtual environment with fresh packages (in case this is a `httpx` change) but this passed: pytest tests/test_csv.py","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269, https://github.com/simonw/datasette/issues/2065#issuecomment-1524616740,https://api.github.com/repos/simonw/datasette/issues/2065,1524616740,IC_kwDOBm6k_c5a384k,9599,2023-04-27T03:38:21Z,2023-04-27T03:38:21Z,OWNER,"Tried this: rye install https://github.com/simonw/datasette/archive/refs/heads/main.zip But got this error: Error: Expected one of `@`, `(`, `<`, `=`, `>`, `~`, `!`, `;`, found `:` I instead downloaded that file and ran: rye install main.zip This worked! And now: ``` ~/.rye/tools/main-zip/bin/datasette --version ``` ``` datasette, version 1.0a2 ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652, https://github.com/simonw/sqlite-utils/issues/527#issuecomment-1506223848,https://api.github.com/repos/simonw/sqlite-utils/issues/527,1506223848,IC_kwDOCGYnMM5Zxybo,9599,2023-04-13T02:08:16Z,2023-04-13T02:08:16Z,OWNER,"I agree, this is a design flaw. It's technically a breaking change. As such, I would need to bump to v4 to responsibly release this. I'd rather bundle in a few more breaking changes before shipping that version. One thing we could do here is add an extra argument to `.convert()` - something like this: ```python table.convert(col, lambda x: x+1, skip_false=False) ``` This would trigger the new, improved behaviour without breaking existing code that depends on how it works at the moment. Then in `sqlite-utils` 4 we can change the default of that option. What do you think? (I'm open to suggestions for better names for this parameter too)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1578790070, https://github.com/simonw/datasette/pull/2056#issuecomment-1506179555,https://api.github.com/repos/simonw/datasette/issues/2056,1506179555,IC_kwDOBm6k_c5Zxnnj,9599,2023-04-13T01:21:05Z,2023-04-13T01:22:08Z,OWNER,"OK, I'm sold - this is a really neat improvement. One thing to change in the PR: right now it runs `pip install --user ruff` on every commit, which hits PyPI to install the package. I prefer to avoid hitting PyPI every time, so I like to use the GitHub Actions cache. My usual pattern for that looks like this: https://github.com/simonw/datasette/blob/5890a20c374fb0812d88c9b0ef26a838bfa06c76/.github/workflows/test-pyodide.yml#L16-L20 Then a separate command that runs `pip install ...` will benefit from that cache. Are you OK to make that change?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1661860507, https://github.com/simonw/datasette/pull/2056#issuecomment-1506177857,https://api.github.com/repos/simonw/datasette/issues/2056,1506177857,IC_kwDOBm6k_c5ZxnNB,9599,2023-04-13T01:18:18Z,2023-04-13T01:18:18Z,OWNER,"Cool - and now https://github.com/simonw/datasette/pull/2056/files is showing me those inline annotations: ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1661860507, https://github.com/simonw/datasette/pull/2056#issuecomment-1506177115,https://api.github.com/repos/simonw/datasette/issues/2056,1506177115,IC_kwDOBm6k_c5ZxnBb,9599,2023-04-13T01:17:16Z,2023-04-13T01:17:16Z,OWNER,"Here are the failures: https://github.com/simonw/datasette/actions/runs/4684460653/jobs/8300630794?pr=2056 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1661860507, https://github.com/simonw/datasette/pull/2056#issuecomment-1506175208,https://api.github.com/repos/simonw/datasette/issues/2056,1506175208,IC_kwDOBm6k_c5Zxmjo,9599,2023-04-13T01:14:13Z,2023-04-13T01:14:13Z,OWNER,https://github.com/simonw/datasette/actions/runs/4664796647/jobs/8300596121?pr=2056 it's pretty fast - that finished in 9s.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1661860507, https://github.com/simonw/datasette/issues/2059#issuecomment-1506174353,https://api.github.com/repos/simonw/datasette/issues/2059,1506174353,IC_kwDOBm6k_c5ZxmWR,9599,2023-04-13T01:13:00Z,2023-04-13T01:13:00Z,OWNER,"Can you provide a URL to an example, and/or a screenshot of this? Is it a browser warning or is it a warning from Heroku itself?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1665053646, https://github.com/simonw/datasette/issues/2058#issuecomment-1504426792,https://api.github.com/repos/simonw/datasette/issues/2058,1504426792,IC_kwDOBm6k_c5Zq7so,9599,2023-04-12T02:02:42Z,2023-04-12T02:02:42Z,OWNER,"I tightened up the benchmark (it was measuring the time taken to create the tables too) and got this: ![image](https://user-images.githubusercontent.com/9599/231328328-85ca35ac-a11b-46f4-b132-dae367103570.png) ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1663399821, https://github.com/simonw/datasette/issues/2058#issuecomment-1504328395,https://api.github.com/repos/simonw/datasette/issues/2058,1504328395,IC_kwDOBm6k_c5ZqjrL,9599,2023-04-12T00:28:38Z,2023-04-12T00:28:38Z,OWNER,"Here's a much better chart, which shows that MD5 performance unsurprisingly gets worse as the number of tables increases while `schema_version` remains constant: ![image](https://user-images.githubusercontent.com/9599/231316778-513bd99f-5ea4-495c-b86d-c572a7106369.png) ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1663399821, https://github.com/simonw/datasette/issues/2058#issuecomment-1504315697,https://api.github.com/repos/simonw/datasette/issues/2058,1504315697,IC_kwDOBm6k_c5Zqgkx,9599,2023-04-12T00:16:22Z,2023-04-12T00:27:12Z,OWNER,"I got ChatGPT (code execution alpha) to run a micro-benchmark for me. This was the conclusion: > The benchmark using `PRAGMA schema_version` is approximately 1.36 times faster than the benchmark using `hashlib.md5` for the case with 100 tables. For the case with 200 tables, the benchmark using `PRAGMA schema_version` is approximately 2.33 times faster than the benchmark using `hashlib.md5`. Here's the chart it drew me: ![image](https://user-images.githubusercontent.com/9599/231315366-3a12b6d3-08d7-419d-a1fd-36eb24da0d85.png) (It's a pretty rubbish chart though, it only took measurements at 100 and 200 and drew a line between the two, I should have told it to measure every 10 and plot that) And the full transcript: https://gist.github.com/simonw/2fc46effbfbe49e6de0bcfdc9e31b235 The benchmark looks good enough on first glance that I don't feel the need to be more thorough with it. `PRAGMA schema_version` is faster, but not so fast that I feel like the MD5 hack is worth worrying about too much. I'm tempted to add something to the `/-/versions` page that tries to identify if this is a problem or not though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1663399821, https://github.com/simonw/datasette/issues/2058#issuecomment-1504298448,https://api.github.com/repos/simonw/datasette/issues/2058,1504298448,IC_kwDOBm6k_c5ZqcXQ,9599,2023-04-12T00:04:01Z,2023-04-12T00:04:01Z,OWNER,"Here's a potential workaround: when I store the schema versions, I could also score an MD5 hash of the full schema (`select group_concat(sql) from sqlite_master`). When I read the schema version with `PRAGMA schema_version` I could catch that exception and, if I see it, I could calculate that MD5 hash again as a fallback and use that to determine if the schema has changed instead. The performance overhead of this needs investigating - how much more expensive is `md5(... that SQL query result)` compared to just `PRAGMA schema_version`, especially on a database with a lot of tables?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1663399821, https://github.com/simonw/datasette/issues/2058#issuecomment-1504295345,https://api.github.com/repos/simonw/datasette/issues/2058,1504295345,IC_kwDOBm6k_c5Zqbmx,9599,2023-04-12T00:01:42Z,2023-04-12T00:02:26Z,OWNER,"Here's the relevant code: https://github.com/simonw/datasette/blob/5890a20c374fb0812d88c9b0ef26a838bfa06c76/datasette/app.py#L421-L437 This function is called on almost every request (everything that subclasses `BaseView` at least - need to remember that for the refactor in #2053 etc). https://github.com/simonw/datasette/blob/5890a20c374fb0812d88c9b0ef26a838bfa06c76/datasette/views/base.py#L101-L103 It uses `PRAGMA schema_version` as a cheap way to determine if the schema has changed, in which case it needs to refresh the internal schema tables. This was already the cause of a subtle bug here: - #1231 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1663399821, https://github.com/simonw/datasette/issues/2058#issuecomment-1504292145,https://api.github.com/repos/simonw/datasette/issues/2058,1504292145,IC_kwDOBm6k_c5Zqa0x,9599,2023-04-11T23:58:59Z,2023-04-11T23:58:59Z,OWNER,Asked on the SQLite Forum if anyone has seen this before: https://sqlite.org/forum/forumpost/793a2ed75b,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1663399821, https://github.com/simonw/datasette/issues/2058#issuecomment-1504291892,https://api.github.com/repos/simonw/datasette/issues/2058,1504291892,IC_kwDOBm6k_c5Zqaw0,9599,2023-04-11T23:58:45Z,2023-04-11T23:58:45Z,OWNER,"I thought it might relate to the ""defensive mode"" issue described here: - https://github.com/simonw/sqlite-utils/issues/235 But I have since determined that the Datasette official Docker image does NOT run anything in defensive mode, so I don't think it's related to that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1663399821, https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1504288134,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1504288134,IC_kwDOCGYnMM5ZqZ2G,9599,2023-04-11T23:55:06Z,2023-04-12T03:34:32Z,OWNER,"Also checked the official Datasette Docker image - I had to run that in Codespaces because it doesn't currently work on my M2 Mac: ``` codespace@codespaces-112c61:/workspaces/sqlite-utils$ docker pull datasetteproject/datasette Using default tag: latest ... codespace@codespaces-112c61:/workspaces/sqlite-utils$ docker run -it datasetteproject/datasette / bin/bash root@75ba34f501ec:/# python Python 3.11.0 (main, Dec 6 2022, 13:31:55) [GCC 10.2.1 20210110] on linux Type ""help"", ""copyright"", ""credits"" or ""license"" for more information. >>> import sqlite3 .executescript("""""" PRAGMA writable_schema = 1; UPDATE sqlite_master SET sql = 'CREATE TABLE [foos] (id integer primary key)'; PRAGMA writable_schema = 0; """""")>>> db = sqlite3.connect("":memory:"") >>> db.executescript("""""" ... PRAGMA writable_schema = 1; ... UPDATE sqlite_master SET sql = 'CREATE TABLE [foos] (id integer primary key)'; ... PRAGMA writable_schema = 0; ... """""") >>> ``` So that confirms that the official image also has a Python with a SQLite that's not in defensive mode.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495, https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1504245029,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1504245029,IC_kwDOCGYnMM5ZqPUl,9599,2023-04-11T23:13:41Z,2023-04-11T23:14:39Z,OWNER,"I also tested this against the current `ubuntu:latest` Docker image (on an M2 Mac), in Python 3.10 and 3.11: ``` docker run -it ubuntu:latest /bin/bash ``` Then in the container: ``` apt-get update apt-get install python3 python3 # pasted in the above recipe apt install software-properties-common add-apt-repository ppa:deadsnakes/ppa apt install python3.11 python3.11 # pasted it in again ``` In both cases the Python code did not raise an exception, which suggests that on Ubuntu those two Python versions do not have the defensive mode set.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495, https://github.com/simonw/datasette/issues/2057#issuecomment-1503838640,https://api.github.com/repos/simonw/datasette/issues/2057,1503838640,IC_kwDOBm6k_c5ZosGw,9599,2023-04-11T17:48:23Z,2023-04-11T17:48:23Z,OWNER,"> This looks wrong to me - I would expect something like `is_directory()` not `is_file()` for telling if `static/` is a directory. I was right about that: ```pycon >>> importlib.resources.files('datasette_graphql') PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql') >>> importlib.resources.files('datasette_graphql').joinpath(""static"") PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql/static') >>> p = importlib.resources.files('datasette_graphql').joinpath(""static"") >>> p PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql/static') >>> p.is_ p.is_absolute() p.is_char_device() p.is_fifo() p.is_mount() p.is_reserved() p.is_symlink() p.is_block_device() p.is_dir() p.is_file() p.is_relative_to( p.is_socket() >>> p.is_dir() True >>> p.is_file() False ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875, https://github.com/simonw/datasette/issues/2057#issuecomment-1503833906,https://api.github.com/repos/simonw/datasette/issues/2057,1503833906,IC_kwDOBm6k_c5Zoq8y,9599,2023-04-11T17:44:16Z,2023-04-11T17:45:45Z,OWNER,"Another prompt: > How to fix this: > > `pkg_resources.get_distribution(package).version` Response: ```python import importlib.metadata # Get the version number of the specified package package_version = importlib.metadata.version(package) ``` That seems to work: ```pycon >>> import importlib.metadata >>> importlib.metadata.version(""datasette"") '0.64.2' >>> importlib.metadata.version(""pluggy"") '1.0.0' >>> importlib.metadata.version(""not-a-package"") ... importlib.metadata.PackageNotFoundError: No package metadata was found for not-a-package ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875, https://github.com/simonw/datasette/issues/2057#issuecomment-1503832422,https://api.github.com/repos/simonw/datasette/issues/2057,1503832422,IC_kwDOBm6k_c5Zoqlm,9599,2023-04-11T17:42:57Z,2023-04-11T17:46:42Z,OWNER,"I ran this prompt against ChatGPT with the Browsing alpha: > ```python > if pkg_resources.resource_isdir(plugin.__name__, ""static""): > static_path = pkg_resources.resource_filename( > plugin.__name__, ""static"" > ) > if pkg_resources.resource_isdir(plugin.__name__, ""templates""): > templates_path = pkg_resources.resource_filename( > plugin.__name__, ""templates"" > ) > ``` > This code gives a deprecation warning in Python 3.11 - fix it It looked up the fix for me: And suggested: ```python import importlib.resources # Replace pkg_resources.resource_isdir with importlib.resources.files().is_file() if importlib.resources.files(plugin.__name__).joinpath(""static"").is_file(): static_path = importlib.resources.as_file( importlib.resources.files(plugin.__name__).joinpath(""static"") ) if importlib.resources.files(plugin.__name__).joinpath(""templates"").is_file(): templates_path = importlib.resources.as_file( importlib.resources.files(plugin.__name__).joinpath(""templates"") ) ``` This looks wrong to me - I would expect something like `is_directory()` not `is_file()` for telling if `static/` is a directory.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875, https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502559442,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1502559442,IC_kwDOCGYnMM5ZjzzS,9599,2023-04-11T01:32:30Z,2023-04-11T01:33:27Z,OWNER,"This seems to work: ```python import sqlite3 db = sqlite3.connect("":memory:"") db.executescript("""""" PRAGMA writable_schema = 1; UPDATE sqlite_master SET sql = 'CREATE TABLE [foos] (id integer primary key)'; PRAGMA writable_schema = 0; """""") ``` It succeeds on my Python 3.11 and raises the following exception on my broken Python 3.9: ``` sqlite3.OperationalError: table sqlite_master may not be modified ``` Removing the `PRAGMA writable_schema = 1;` causes the same exception to be raised on both Pythons.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495, https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502557629,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1502557629,IC_kwDOCGYnMM5ZjzW9,9599,2023-04-11T01:30:12Z,2023-04-11T01:30:12Z,OWNER,"I'll ask on the SQLite forum if it's possible to toggle that mode on and off using regular SQL. My hunch is that it isn't. In which case `sqlite-utils` should at least know how to catch this error and display a much more readable error message, maybe with a link to further documentation. A utility function that can detect this mode would be really useful too. I'd probably have to do a test that tries to modify `sqlite_master` on a new in-memory database to catch if it's possible or not.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495, https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502556111,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1502556111,IC_kwDOCGYnMM5Zjy_P,9599,2023-04-11T01:28:41Z,2023-04-11T01:28:41Z,OWNER,"Investigating this one now. The `sqlite-utils` test suite passes without errors on my Python 3.11.2 installation... but it fails with this error on a Python 3.9.6 installation. In the broken version's virtual environment directory I ran this: ``` cat pyvenv.cfg home = /Applications/Xcode.app/Contents/Developer/usr/bin implementation = CPython version_info = 3.9.6.final.0 virtualenv = 20.17.1 include-system-site-packages = false base-prefix = /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9 base-exec-prefix = /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9 base-executable = /Applications/Xcode.app/Contents/Developer/usr/bin/python3 ``` So it looks like the Xcode `python3` has ""defensive"" mode turned on for SQLite. As far as I can tell there's no way to turn it OFF again in Python. My virtual environment that DOES work has this: ``` home = /opt/homebrew/opt/python@3.11/bin implementation = CPython version_info = 3.11.2.final.0 virtualenv = 20.17.1 include-system-site-packages = false base-prefix = /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11 base-exec-prefix = /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11 base-executable = /opt/homebrew/opt/python@3.11/bin/python3.11 ``` So the Python 3.11 I installed through Homebrew doesn't exhibit this bug.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495, https://github.com/simonw/datasette/issues/2054#issuecomment-1500608101,https://api.github.com/repos/simonw/datasette/issues/2054,1500608101,IC_kwDOBm6k_c5ZcXZl,9599,2023-04-07T20:14:38Z,2023-04-07T20:14:38Z,OWNER,"Ooh that one's really interesting - very different from the others: ```ruby # app.rb require ""roda"" class App < Roda route do |r| r.root do ""Home page"" end r.on ""pages"" do r.get "":slug"" do |slug| ""Page: #{slug}"" end end r.on ""news"" do r.get "":yyyy/:mm/:dd"" do |yyyy, mm, dd| ""News for #{yyyy}/#{mm}/#{dd}"" end end end end # config.ru require_relative ""app"" run App.freeze.app ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1657861026, https://github.com/simonw/datasette/issues/2054#issuecomment-1499604822,https://api.github.com/repos/simonw/datasette/issues/2054,1499604822,IC_kwDOBm6k_c5ZYidW,9599,2023-04-06T20:48:19Z,2023-04-06T20:48:55Z,OWNER,"I actually quite like that. I could even use `@classmethod` and have utility methods defined on that class that both `get()` and `post()` could call. The crucial rule here is NO INSTANCE STATE - that's what makes routing to classes particularly damaging, and encourages code that's hard to maintain.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1657861026, https://github.com/simonw/datasette/issues/2054#issuecomment-1499604066,https://api.github.com/repos/simonw/datasette/issues/2054,1499604066,IC_kwDOBm6k_c5ZYiRi,9599,2023-04-06T20:47:30Z,2023-04-06T20:47:30Z,OWNER,"I'm contemplating a new approach: using a class with static methods. Something like this: ```python class TableView(MethodRouter): @staticmethod async def get(request): return Response.text(""GET"") @staticmethod async def post(request): return Response.text(""POST"") ``` So effectively the class is just there to bundle together verb implementations, and to provide a `route(request)` method which knows how to dispatch them to the right place. It can offer default HEAD and OPTIONS methods too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1657861026, https://github.com/simonw/datasette/issues/2054#issuecomment-1499596941,https://api.github.com/repos/simonw/datasette/issues/2054,1499596941,IC_kwDOBm6k_c5ZYgiN,9599,2023-04-06T20:39:35Z,2023-04-06T20:39:35Z,OWNER,"I think there are two main approaches: - Map path to a function, that function then behaves differently on GET/POST/etc - Map method AND path to a function - the `app.get('/example')` v.s. `app.post('/example')` ones The decorator approach is calling out to me a bit now.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1657861026, https://github.com/simonw/datasette/issues/2054#issuecomment-1499530921,https://api.github.com/repos/simonw/datasette/issues/2054,1499530921,IC_kwDOBm6k_c5ZYQap,9599,2023-04-06T19:41:52Z,2023-04-06T19:42:09Z,OWNER,Turned this into a TIL: https://til.simonwillison.net/gpt3/gpt4-api-design,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1657861026, https://github.com/simonw/datasette/issues/2054#issuecomment-1499491258,https://api.github.com/repos/simonw/datasette/issues/2054,1499491258,IC_kwDOBm6k_c5ZYGu6,9599,2023-04-06T19:00:16Z,2023-04-06T20:40:16Z,OWNER,"I asked GPT-4 to compare URL routing based on path and verb approaches in different languages and frameworks and spit out some examples. Full transcript here: https://gist.github.com/simonw/fa2379b97420404a81b0fcdb4db79657 Initial prompt: > Show me code examples of different web frameworks in Python and JavaScript and Go illustrating how HTTP routing works - in particular the problem of mapping an incoming HTTP request to some code based on both the URL path and the HTTP verb > > Don't show installation instructions, just get straight to the code snippets Flask: ```python from flask import Flask app = Flask(__name__) @app.route('/example', methods=['GET']) def get_example(): return ""This is a GET request."" @app.route('/example', methods=['POST']) def post_example(): return ""This is a POST request."" if __name__ == '__main__': app.run() ``` Express: ```javascript const express = require('express'); const app = express(); app.get('/example', (req, res) => { res.send('This is a GET request.'); }); app.post('/example', (req, res) => { res.send('This is a POST request.'); }); app.listen(3000, () => { console.log('Server is running on port 3000'); }); ``` Gin: ```go package main import ( ""github.com/gin-gonic/gin"" ) func main() { r := gin.Default() r.GET(""/example"", func(c *gin.Context) { c.String(200, ""This is a GET request."") }) r.POST(""/example"", func(c *gin.Context) { c.String(200, ""This is a POST request."") }) r.Run() } ``` Actix-web: ```rust use actix_web::{web, App, HttpResponse, HttpServer, Responder}; async fn get_example() -> impl Responder { HttpResponse::Ok().body(""This is a GET request."") } async fn post_example() -> impl Responder { HttpResponse::Ok().body(""This is a POST request."") } #[actix_web::main] async fn main() -> std::io::Result<()> { HttpServer::new(|| { App::new() .route(""/example"", web::get().to(get_example)) .route(""/example"", web::post().to(post_example)) }) .bind(""127.0.0.1:8080"")? .run() .await } ``` Koa: ```javascript const Koa = require('koa'); const Router = require('koa-router'); const app = new Koa(); const router = new Router(); router.get('/example', (ctx) => { ctx.body = 'This is a GET request.'; }); router.post('/example', (ctx) => { ctx.body = 'This is a POST request.'; }); app.use(router.routes()); app.use(router.allowedMethods()); app.listen(3000, () => { console.log('Server is running on port 3000'); }); ``` Django: ```python from django.http import HttpResponse from django.views import View class ExampleView(View): def get(self, request): return HttpResponse(""This is a GET request."") def post(self, request): return HttpResponse(""This is a POST request."") ``` ```python from django.urls import path from .views import ExampleView urlpatterns = [ path('example/', ExampleView.as_view(), name='example'), ] ``` Rocket: ```rust #![feature(proc_macro_hygiene, decl_macro)] #[macro_use] extern crate rocket; use rocket::response::content; #[get(""/example"")] fn get_example() -> content::Plain<&'static str> { content::Plain(""This is a GET request."") } #[post(""/example"")] fn post_example() -> content::Plain<&'static str> { content::Plain(""This is a POST request."") } #[launch] fn rocket() -> _ { rocket::build().mount(""/"", routes![get_example, post_example]) } ``` Elixir/Phoenix: ```elixir defmodule ExampleAppWeb.Router do use ExampleAppWeb, :router pipeline :api do plug :accepts, [""json""] end scope ""/api"", ExampleAppWeb do pipe_through :api get ""/example"", ExampleController, :get_example post ""/example"", ExampleController, :post_example end end ``` Bottle: ```python from bottle import Bottle, run app = Bottle() @app.route('/example', method='GET') def get_example(): return ""This is a GET request."" @app.route('/example', method='POST') def post_example(): return ""This is a POST request."" if __name__ == '__main__': run(app, host='localhost', port=8080) ``` FastAPI: ```python from fastapi import FastAPI app = FastAPI() @app.get(""/example"") async def get_example(): return {""message"": ""This is a GET request.""} @app.post(""/example"") async def post_example(): return {""message"": ""This is a POST request.""} if __name__ == ""__main__"": import uvicorn uvicorn.run(app, host=""127.0.0.1"", port=8000) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1657861026, https://github.com/simonw/datasette/issues/2054#issuecomment-1499467703,https://api.github.com/repos/simonw/datasette/issues/2054,1499467703,IC_kwDOBm6k_c5ZYA-3,9599,2023-04-06T18:37:22Z,2023-04-06T18:57:22Z,OWNER,"I've been hoping to move entirely away from class-based views, but now I'm wondering if that's the right decision given the need to support HTTP verbs. It is cleaner to have a class that has methods for each verb, or to have a single function that can behave differently depending on the verb? Or should I have a mechanism for dispatching to separate functions based on the verbs that isn't implemented in a base class? The trouble with using base classes is that I've already shown that they tempt me to put logic in weird places, which makes code harder to modify later on. The thing here is that the ONLY thing I want to use the base class for is as an HTTP verb switcher - which makes me doubt that I should risk taking on the other temptations of having a base class.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1657861026, https://github.com/simonw/datasette/issues/2054#issuecomment-1499465648,https://api.github.com/repos/simonw/datasette/issues/2054,1499465648,IC_kwDOBm6k_c5ZYAew,9599,2023-04-06T18:35:03Z,2023-04-06T18:35:03Z,OWNER,"There are actually five classes that subclass `DataView`: https://github.com/simonw/datasette/blob/8b9d7fdbd8de7e74414cc29e3005382669a812dc/datasette/views/row.py#L16 https://github.com/simonw/datasette/blob/8b9d7fdbd8de7e74414cc29e3005382669a812dc/datasette/views/database.py#L34 https://github.com/simonw/datasette/blob/8b9d7fdbd8de7e74414cc29e3005382669a812dc/datasette/views/database.py#L172 https://github.com/simonw/datasette/blob/8b9d7fdbd8de7e74414cc29e3005382669a812dc/datasette/views/database.py#L215 https://github.com/simonw/datasette/blob/8b9d7fdbd8de7e74414cc29e3005382669a812dc/datasette/views/table.py#L72 I don't think `DatabaseView` and `DatabaseDownload` should have subclassed that at all, since they don't return a table of data.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1657861026, https://github.com/simonw/datasette/issues/2054#issuecomment-1499462324,https://api.github.com/repos/simonw/datasette/issues/2054,1499462324,IC_kwDOBm6k_c5ZX_q0,9599,2023-04-06T18:31:56Z,2023-04-06T18:31:56Z,OWNER,"The `DataView` class does a LOT of work - mostly involving CSV responses. https://github.com/simonw/datasette/blob/8b9d7fdbd8de7e74414cc29e3005382669a812dc/datasette/views/base.py#L160-L544 It has a `redirect()` method with some complex logic and CORS handling: https://github.com/simonw/datasette/blob/8b9d7fdbd8de7e74414cc29e3005382669a812dc/datasette/views/base.py#L163-L172 It uses this method a lot, which has to be over-ridden in the `TableView` and `RowView` and `QueryView` subclasses: https://github.com/simonw/datasette/blob/8b9d7fdbd8de7e74414cc29e3005382669a812dc/datasette/views/base.py#L174-L175 This method: https://github.com/simonw/datasette/blob/8b9d7fdbd8de7e74414cc29e3005382669a812dc/datasette/views/base.py#L180 Is the bulk of the complexity, because it knows how to both turn a list of SQLite rows into a CSV file but also knows how to call `.data()` repeatedly with different pagination arguments in order to stream CSV back for a large table. The `async def get()` method for GET requests is also very complicated. It mainly handles format stuff - knowing how to render HTML v.s. JSON v.s. CSV v.s. other formats specified using this plugin hook: https://docs.datasette.io/en/1.0a2/plugin_hooks.html#register-output-renderer-datasette Plus it catches interrupted queries and returns a special error page for those (and other error messages too): https://github.com/simonw/datasette/blob/8b9d7fdbd8de7e74414cc29e3005382669a812dc/datasette/views/base.py#L381-L408 It adds the time taken to execute the queries: https://github.com/simonw/datasette/blob/8b9d7fdbd8de7e74414cc29e3005382669a812dc/datasette/views/base.py#L410-L411","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1657861026,