html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/sqlite-utils/issues/416#issuecomment-1116684581,https://api.github.com/repos/simonw/sqlite-utils/issues/416,1116684581,IC_kwDOCGYnMM5Cj0El,638427,mattkiefer,2022-05-03T21:36:49Z,2022-05-03T21:36:49Z,NONE,"Thanks for addressing this @simonw! However, I just reinstalled sqlite-utils 3.26.1 and get an `ParserError: Unknown string format: None`: ``` sqlite-utils --version sqlite-utils, version 3.26.1 ``` ``` sqlite-utils convert idfpr.db license ""Original Issue Date"" ""r.parsedate(value)"" Traceback (most recent call last): File ""/home/matt/.local/lib/python3.9/site-packages/sqlite_utils/db.py"", line 2514, in convert_value return fn(v) File """", line 2, in fn File ""/home/matt/.local/lib/python3.9/site-packages/sqlite_utils/recipes.py"", line 19, in parsedate parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst) File ""/usr/lib/python3/dist-packages/dateutil/parser/_parser.py"", line 1374, in parse return DEFAULTPARSER.parse(timestr, **kwargs) File ""/usr/lib/python3/dist-packages/dateutil/parser/_parser.py"", line 649, in parse raise ParserError(""Unknown string format: %s"", timestr) dateutil.parser._parser.ParserError: Unknown string format: None Traceback (most recent call last): File ""/home/matt/.local/bin/sqlite-utils"", line 8, in sys.exit(cli()) File ""/usr/lib/python3/dist-packages/click/core.py"", line 829, in __call__ return self.main(*args, **kwargs) File ""/usr/lib/python3/dist-packages/click/core.py"", line 782, in main rv = self.invoke(ctx) File ""/usr/lib/python3/dist-packages/click/core.py"", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/usr/lib/python3/dist-packages/click/core.py"", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/usr/lib/python3/dist-packages/click/core.py"", line 610, in invoke return callback(*args, **kwargs) File ""/home/matt/.local/lib/python3.9/site-packages/sqlite_utils/cli.py"", line 2707, in convert db[table].convert( File ""/home/matt/.local/lib/python3.9/site-packages/sqlite_utils/db.py"", line 2530, in convert self.db.execute(sql, where_args or []) File ""/home/matt/.local/lib/python3.9/site-packages/sqlite_utils/db.py"", line 463, in execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: user-defined function raised exception ``` I definitely have some invalid data in the db. Happy to send a copy if it's helpful.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1173023272,Options for how `r.parsedate()` should handle invalid dates, https://github.com/simonw/sqlite-utils/issues/430#issuecomment-1116336340,https://api.github.com/repos/simonw/sqlite-utils/issues/430,1116336340,IC_kwDOCGYnMM5CifDU,9308268,rayvoelker,2022-05-03T17:03:31Z,2022-05-03T17:03:31Z,NONE,"So, the good news is that it appears that setting one of those PRAGMA statements fixed the issue of `table.extract()` method call on this large database completing (that I described above.) The bad news is that I'm not sure which one! I wonder if it's something system / environment specific about SQLite, or maybe something else going on.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1224112817,Document how to use `PRAGMA temp_store` to avoid errors when running VACUUM against huge databases, https://github.com/simonw/datasette/issues/1739#issuecomment-1116215371,https://api.github.com/repos/simonw/datasette/issues/1739,1116215371,IC_kwDOBm6k_c5CiBhL,9599,simonw,2022-05-03T15:12:16Z,2022-05-03T15:12:16Z,OWNER,"That worked - both DBs are 304 for me now on a subsequent load of the page: ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1223699280,.db downloads should be served with an ETag, https://github.com/simonw/datasette/issues/1739#issuecomment-1116183369,https://api.github.com/repos/simonw/datasette/issues/1739,1116183369,IC_kwDOBm6k_c5Ch5tJ,9599,simonw,2022-05-03T14:43:14Z,2022-05-03T14:43:14Z,OWNER,Relevant tests start here: https://github.com/simonw/datasette/blob/d60f163528f466b1127b2935c3b6869c34fd6545/tests/test_html.py#L395,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1223699280,.db downloads should be served with an ETag, https://github.com/simonw/datasette/issues/1739#issuecomment-1116180599,https://api.github.com/repos/simonw/datasette/issues/1739,1116180599,IC_kwDOBm6k_c5Ch5B3,9599,simonw,2022-05-03T14:40:32Z,2022-05-03T14:40:32Z,OWNER,"Database downloads are served here: https://github.com/simonw/datasette/blob/d60f163528f466b1127b2935c3b6869c34fd6545/datasette/views/database.py#L186-L192 Here's `AsgiFileDownload`: https://github.com/simonw/datasette/blob/d60f163528f466b1127b2935c3b6869c34fd6545/datasette/utils/asgi.py#L410-L430 I can add an `etag=` parameter to that and populate it with `db.hash`, if it is populated (which it always should be for immutable databases that can be downloaded).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1223699280,.db downloads should be served with an ETag, https://github.com/simonw/datasette/issues/1739#issuecomment-1116178727,https://api.github.com/repos/simonw/datasette/issues/1739,1116178727,IC_kwDOBm6k_c5Ch4kn,9599,simonw,2022-05-03T14:38:46Z,2022-05-03T14:38:46Z,OWNER,"Reminded myself how this works by reviewing `conditional-get`: https://github.com/simonw/conditional-get/blob/db6dfec0a296080aaf68fcd80e55fb3f0714e738/conditional_get/cli.py#L33-L52 Simply add a `If-None-Match: last-known-etag` header to the request and check that the response is a status 304 with an empty body.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1223699280,.db downloads should be served with an ETag, https://github.com/simonw/datasette/issues/1739#issuecomment-1115760104,https://api.github.com/repos/simonw/datasette/issues/1739,1115760104,IC_kwDOBm6k_c5CgSXo,9599,simonw,2022-05-03T05:50:19Z,2022-05-03T05:50:19Z,OWNER,Here's how Starlette does it: https://github.com/encode/starlette/blob/830f3486537916bae6b46948ff922adc14a22b7c/starlette/staticfiles.py#L213,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1223699280,.db downloads should be served with an ETag, https://github.com/simonw/datasette/issues/1732#issuecomment-1115542067,https://api.github.com/repos/simonw/datasette/issues/1732,1115542067,IC_kwDOBm6k_c5CfdIz,52649,tannewt,2022-05-03T01:50:44Z,2022-05-03T01:50:44Z,NONE,"I haven’t set one up unfortunately. My time is very limited because we just had a baby. On Mon, May 2, 2022, at 6:42 PM, Simon Willison wrote: > > > Thanks, this definitely sounds like a bug. Do you have simple steps to reproduce this? > > > — > Reply to this email directly, view it on GitHub , or unsubscribe . > You are receiving this because you authored the thread.Message ID: ***@***.***> > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1221849746,Custom page variables aren't decoded, https://github.com/simonw/datasette/issues/1732#issuecomment-1115533820,https://api.github.com/repos/simonw/datasette/issues/1732,1115533820,IC_kwDOBm6k_c5CfbH8,9599,simonw,2022-05-03T01:42:25Z,2022-05-03T01:42:25Z,OWNER,"Thanks, this definitely sounds like a bug. Do you have simple steps to reproduce this?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1221849746,Custom page variables aren't decoded,