html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1606411508,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1606411508,IC_kwDOCGYnMM5fv-T0,9599,simonw,2023-06-26T01:42:10Z,2023-06-26T01:42:22Z,OWNER,https://sqlite-utils.datasette.io/en/stable/changelog.html#v3-33 - upgrading to `sqlite-utils>=3.33` and then installing both `sqlean.py` and `sqlite-dump` in the same virtual environment as `sqlite-utils` should fix this issue.,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified, https://github.com/simonw/sqlite-utils/issues/532#issuecomment-1539009453,https://api.github.com/repos/simonw/sqlite-utils/issues/532,1539009453,IC_kwDOCGYnMM5bu2ut,9599,simonw,2023-05-08T20:30:29Z,2023-05-08T20:30:42Z,OWNER,"Here's an improvement: ``` % sqlite-utils insert /tmp/b.db blah /tmp/blah.txt [####################################] 100% Error: Invalid JSON - use --csv for CSV or --tsv for TSV files JSON error: Expecting value: line 1 column 1 (char 0) ```","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1620254998,Show more information when JSON can't be imported with sqlite-utils insert, https://github.com/simonw/datasette/issues/1886#issuecomment-1312898318,https://api.github.com/repos/simonw/datasette/issues/1886,1312898318,IC_kwDOBm6k_c5OQT0O,19851673,eigenfoo,2022-11-14T00:52:16Z,2022-11-14T00:52:16Z,NONE,"I'm a cryptic crossword enthusiast and have spent a lot of time scraping and parsing cryptic crossword clues from various blogs, forums and publications. The result is over **half a million clues from cryptic crosswords over the past twelve years**, including the clue, answer, puzzle date, puzzle name and a link to the original source. This is all hosted using Datasette, which has been a delight to use: https://cryptics.georgeho.org/ This dataset is a significant work of crossword archivism and scholarship, as acquiring historical crosswords and structuring their contents require focused effort and tedious cleaning that few are willing to do for such trivial data - for example, according to [this 2004 selection guide](https://cryptics.georgeho.org/static/documents/Selection_AppendixE_v2.pdf), the Library of Congress explicitly does not collect crossword puzzles. Anecdotally, I know that many constructors/setters of cryptic crosswords use this dataset as a resource, and some even simply call it ""the database"" - this is probably one of the most impactful data projects I've worked on!","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1447050738,"Call for birthday presents: if you're using Datasette, let us know how you're using it here", https://github.com/simonw/datasette/issues/1817#issuecomment-1258756231,https://api.github.com/repos/simonw/datasette/issues/1817,1258756231,IC_kwDOBm6k_c5LBxiH,9599,simonw,2022-09-26T23:19:34Z,2022-09-26T23:19:34Z,OWNER,"This is a good idea - it's something I should do before Datasette 1.0. I was a tiny bit worried about compatibility (Datasette is 3.7+) but it looks like they have been in Python since 3.0!","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1384273985,Expose `sql` and `params` arguments to various plugin hooks, https://github.com/simonw/sqlite-utils/pull/480#issuecomment-1232089808,https://api.github.com/repos/simonw/sqlite-utils/issues/480,1232089808,IC_kwDOCGYnMM5JcDLQ,9599,simonw,2022-08-30T19:43:02Z,2022-08-30T19:43:02Z,OWNER,"Yeah this seems like a reasonable addition to me. Needs a test, which can go next to this one: https://github.com/simonw/sqlite-utils/blob/087753cd42c406f1e060c1822dcd9b5fda3d60f4/tests/test_fts.py#L561","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1355433619,search_sql add include_rank option, https://github.com/simonw/sqlite-utils/issues/467#issuecomment-1224386951,https://api.github.com/repos/simonw/sqlite-utils/issues/467,1224386951,IC_kwDOCGYnMM5I-qmH,9599,simonw,2022-08-23T17:20:07Z,2022-08-23T17:20:07Z,OWNER,"Example of that prototype working: ```pycon >>> from sqlite_utils import Database >>> db = Database(memory=True) >>> db[""dogs""].create({""id"": int, ""name"": str}, pk=""id"") >>> db[""dogs""].create({""id"": int, ""name"": str, ""age"": int}, pk=""id"", transform=True)
```","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1348169997,Mechanism for ensuring a table has all the columns, https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1080141111,https://api.github.com/repos/simonw/sqlite-utils/issues/420,1080141111,IC_kwDOCGYnMM5AYaU3,9599,simonw,2022-03-28T03:25:57Z,2022-03-28T03:54:37Z,OWNER,"So now this should solve your problem: ``` echo '[{""name"": ""notaword""}, {""name"": ""word""}] ' | python3 -m sqlite_utils insert listings.db listings - --convert ' import enchant d = enchant.Dict(""en_US"") def convert(row): global d row[""is_dictionary_word""] = d.check(row[""name""]) ' ```","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1178546862,Document how to use a `--convert` function that runs initialization code first, https://github.com/simonw/sqlite-utils/issues/415#issuecomment-1073468996,https://api.github.com/repos/simonw/sqlite-utils/issues/415,1073468996,IC_kwDOCGYnMM4_-9ZE,9599,simonw,2022-03-21T04:14:42Z,2022-03-21T04:14:42Z,OWNER,"I can fix this like so: ``` % sqlite-utils convert demo.db demo foo '{""foo"": ""bar""}' --multi --dry-run abc --- becomes: {""foo"": ""bar""} Would affect 1 row ``` Diff is this: ```diff diff --git a/sqlite_utils/cli.py b/sqlite_utils/cli.py index 0cf0468..b2a0440 100644 --- a/sqlite_utils/cli.py +++ b/sqlite_utils/cli.py @@ -2676,7 +2676,10 @@ def convert( raise click.ClickException(str(e)) if dry_run: # Pull first 20 values for first column and preview them - db.conn.create_function(""preview_transform"", 1, lambda v: fn(v) if v else v) + preview = lambda v: fn(v) if v else v + if multi: + preview = lambda v: json.dumps(fn(v), default=repr) if v else v + db.conn.create_function(""preview_transform"", 1, preview) sql = """""" select [{column}] as value, ```","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1171599874,Convert with `--multi` and `--dry-run` flag does not work, https://github.com/simonw/sqlite-utils/issues/413#issuecomment-1065245831,https://api.github.com/repos/simonw/sqlite-utils/issues/413,1065245831,IC_kwDOCGYnMM4_flyH,9599,simonw,2022-03-11T15:59:14Z,2022-03-11T15:59:14Z,OWNER,"Hint from https://twitter.com/AdamChainz/status/1502311047612575745 > Try: > > `autodoc_typehints = 'description'` > > For a list-of-arguments format > > https://sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_typehints","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166587040,Display autodoc type information more legibly, https://github.com/simonw/sqlite-utils/pull/407#issuecomment-1040959312,https://api.github.com/repos/simonw/sqlite-utils/issues/407,1040959312,IC_kwDOCGYnMM4-C8dQ,9599,simonw,2022-02-16T00:58:32Z,2022-02-16T00:58:32Z,OWNER,This is honestly one of the most complete PRs I've ever seen for a feature of this size. Thanks so much for this!,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1138948786,Add SpatiaLite helpers to CLI, https://github.com/simonw/datasette/issues/93#issuecomment-974765825,https://api.github.com/repos/simonw/datasette/issues/93,974765825,IC_kwDOBm6k_c46Gb8B,9599,simonw,2021-11-21T07:00:21Z,2021-11-21T07:00:21Z,OWNER,Closing this in favour of Datasette Desktop: https://datasette.io/desktop,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",273944952,Package as standalone binary, https://github.com/simonw/sqlite-utils/issues/335#issuecomment-968380387,https://api.github.com/repos/simonw/sqlite-utils/issues/335,968380387,IC_kwDOCGYnMM45uE_j,9599,simonw,2021-11-14T22:55:56Z,2021-11-14T22:55:56Z,OWNER,"OK, this should fix it.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1042569687,sqlite-utils index-foreign-keys fails due to pre-existing index, https://github.com/simonw/datasette/issues/1380#issuecomment-967801997,https://api.github.com/repos/simonw/datasette/issues/1380,967801997,IC_kwDOBm6k_c45r3yN,7094907,Segerberg,2021-11-13T08:05:37Z,2021-11-13T08:09:11Z,NONE,"@glasnt yeah I guess that could be an option. I run datasette on large databases > 75gb and the startup time is a bit slow for me even with -i --inspect-file options. Here's a quick sketch for a plugin that will reload db's in a folder that you set for the plugin in metadata.json. If you request /-reload-db new db's will be added. (You probably want to implement some authentication for this =) ) https://gist.github.com/Segerberg/b96a0e0a5389dce2396497323cda7042 ","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",924748955,Serve all db files in a folder, https://github.com/simonw/datasette/pull/1458#issuecomment-943620649,https://api.github.com/repos/simonw/datasette/issues/1458,943620649,IC_kwDOBm6k_c44PoIp,9599,simonw,2021-10-14T18:38:58Z,2021-10-14T18:38:58Z,OWNER,"This is a great idea, thanks.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",988555009,Rework the `--static` documentation a bit, https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895587441,https://api.github.com/repos/simonw/sqlite-utils/issues/309,895587441,IC_kwDOCGYnMM41YZRx,9599,simonw,2021-08-09T22:15:45Z,2021-08-09T22:15:45Z,OWNER,"``` OverflowError: Python int too large to convert to SQLite INTEGER >>> import sys >>> def find_variables(tb, vars): to_find = list(vars) found = {} for var in to_find: if var in tb.tb_frame.f_locals: vars.remove(var) found[var] = tb.tb_frame.f_locals[var] if vars and tb.tb_next: found.update(find_variables(tb.tb_next, vars)) return found ... >>> find_variables(sys.last_traceback, [""sql"", ""params""]) {'params': [34223049823094832094802398430298048240], 'sql': 'INSERT INTO [row] ([v]) VALUES (?);'} ```","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",963897111,"sqlite-utils insert errors should show SQL and parameters, if possible", https://github.com/simonw/datasette/issues/1127#issuecomment-852712106,https://api.github.com/repos/simonw/datasette/issues/1127,852712106,MDEyOklzc3VlQ29tbWVudDg1MjcxMjEwNg==,9599,simonw,2021-06-02T04:28:55Z,2021-06-02T04:28:55Z,OWNER,This became resizable in #1236.,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",756818250,Make the custom SQL query text box larger or resizable, https://github.com/simonw/datasette/issues/619#issuecomment-850057694,https://api.github.com/repos/simonw/datasette/issues/619,850057694,MDEyOklzc3VlQ29tbWVudDg1MDA1NzY5NA==,9599,simonw,2021-05-28T02:03:05Z,2021-05-28T02:03:05Z,OWNER,"I nearly got this working, but I ran into one last problem: the code path for when an error is raised but the user specified `?_shape=array`. I'll open a draft PR with where I've got to so far.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",520655983,"""Invalid SQL"" page should let you edit the SQL", https://github.com/simonw/datasette/issues/1320#issuecomment-847237524,https://api.github.com/repos/simonw/datasette/issues/1320,847237524,MDEyOklzc3VlQ29tbWVudDg0NzIzNzUyNA==,9599,simonw,2021-05-24T18:15:56Z,2021-05-24T18:15:56Z,OWNER,Added some new documentation about that here: https://github.com/simonw/datasette/blob/c0a748e5c3f498fa8c139b420d07dd3dea612379/docs/installation.rst#installing-plugins,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",884952179,Can't use apt-get in Dockerfile when using datasetteproj/datasette as base, https://github.com/simonw/datasette/issues/1310#issuecomment-829885904,https://api.github.com/repos/simonw/datasette/issues/1310,829885904,MDEyOklzc3VlQ29tbWVudDgyOTg4NTkwNA==,3747136,ColinMaudry,2021-04-30T06:58:46Z,2021-04-30T07:26:11Z,NONE,"I made it work with openpyxl. I'm not sure all the code under `@hookimpl` is necessary... but it works :) ```python from datasette import hookimpl from datasette.utils.asgi import Response from openpyxl import Workbook from openpyxl.writer.excel import save_virtual_workbook from openpyxl.cell import WriteOnlyCell from openpyxl.styles import Alignment, Font, PatternFill from tempfile import NamedTemporaryFile def render_spreadsheet(rows): wb = Workbook(write_only=True) ws = wb.create_sheet() ws = wb.active ws.title = ""decp"" columns = rows[0].keys() headers = [] for col in columns : c = WriteOnlyCell(ws, col) c.fill = PatternFill(""solid"", fgColor=""DDEFFF"") headers.append(c) ws.append(headers) for row in rows: wsRow = [] for col in columns: c = WriteOnlyCell(ws, row[col]) if col == ""objet"" : c.alignment = Alignment(wrapText = True) wsRow.append(c) ws.append(wsRow) with NamedTemporaryFile() as tmp: wb.save(tmp.name) tmp.seek(0) return Response( tmp.read(), headers={ 'Content-Disposition': 'attachment; filename=decp.xlsx', 'Content-type': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet' } ) @hookimpl def register_output_renderer(): return {""extension"": ""xlsx"", ""render"": render_spreadsheet, ""can_render"": lambda: False} ``` The key part was to find the right function to wrap the spreadsheet object `wb`. `NamedTemporaryFile()` did it! I'll update this issue when the plugin is packaged and ready for broader use.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",870125126,I'm creating a plugin to export a spreadsheet file (.ods or .xlsx), https://github.com/simonw/sqlite-utils/issues/227#issuecomment-778854808,https://api.github.com/repos/simonw/sqlite-utils/issues/227,778854808,MDEyOklzc3VlQ29tbWVudDc3ODg1NDgwOA==,9599,simonw,2021-02-14T22:46:54Z,2021-02-14T22:46:54Z,OWNER,Fix is released in 3.5.,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",807174161,Error reading csv files with large column data, https://github.com/simonw/datasette/issues/1091#issuecomment-756453945,https://api.github.com/repos/simonw/datasette/issues/1091,756453945,MDEyOklzc3VlQ29tbWVudDc1NjQ1Mzk0NQ==,9599,simonw,2021-01-07T23:42:50Z,2021-01-07T23:42:50Z,OWNER,"@henry501 it looks like you spotted a bug in the documentation - I just addressed that, the fix is now live here: https://docs.datasette.io/en/latest/deploying.html#running-datasette-behind-a-proxy","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",742011049,.json and .csv exports fail to apply base_url, https://github.com/simonw/datasette/issues/983#issuecomment-753219407,https://api.github.com/repos/simonw/datasette/issues/983,753219407,MDEyOklzc3VlQ29tbWVudDc1MzIxOTQwNw==,9599,simonw,2020-12-31T22:38:45Z,2020-12-31T22:39:10Z,OWNER,"You'll be able to add JavaScript plugins using a bunch of different mechanisms: - In a custom template, dropping the code in to a `