View Datasette (Click on the stop button to close the Datasette server)
')) # Launch Datasette with Popen( [ 'python', '-m', 'datasette', '--', database, '--port', str(port), '--config', f'base_url:{proxy_url}' ], stdout=PIPE, stderr=PIPE, bufsize=1, universal_newlines=True ) as p: print(p.stdout.readline(), end='') while True: try: line = p.stderr.readline() if not line: break print(line, end='') exit_code = p.poll() except KeyboardInterrupt: p.send_signal(SIGINT) ``` Ideally, I'd like some extra magic to notify users when they are leaving the closing the notebook tab and make them terminate the running datasette processes. I'll be looking for it.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",396212021,base_url configuration setting, https://github.com/simonw/datasette/issues/791#issuecomment-640116494,https://api.github.com/repos/simonw/datasette/issues/791,640116494,MDEyOklzc3VlQ29tbWVudDY0MDExNjQ5NA==,9599,simonw,2020-06-06T20:50:41Z,2020-06-06T20:50:41Z,OWNER,"I have a better idea: a feed reader! You can insert URLs to feeds, then have a command which fetches the latest entries from them into a separate table. Then implement favorites as a canned query, let you search your favorites, etc.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",628572716,Tutorial: building a something-interesting with writable canned queries, https://github.com/simonw/sqlite-utils/issues/26#issuecomment-501541902,https://api.github.com/repos/simonw/sqlite-utils/issues/26,501541902,MDEyOklzc3VlQ29tbWVudDUwMTU0MTkwMg==,9599,simonw,2019-06-13T04:15:22Z,2019-06-13T16:55:42Z,OWNER,"So maybe something like this: ``` curl https://api.github.com/repos/simonw/datasette/pulls?state=all | \ sqlite-utils insert git.db pulls - \ --flatten=base \ --flatten=head \ --extract=user:users:id \ --extract=head_repo.license:licenses:key \ --extract=head_repo.owner:users \ --extract=head_repo --extract=base_repo.license:licenses:key \ --extract=base_repo.owner:users \ --extract=base_repo ``` Is the order of those nested `--extract` lines significant I wonder? It would be nice if the order didn't matter and the code figured out the right execution plan on its own.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",455486286,Mechanism for turning nested JSON into foreign keys / many-to-many, https://github.com/simonw/datasette/issues/594#issuecomment-552276247,https://api.github.com/repos/simonw/datasette/issues/594,552276247,MDEyOklzc3VlQ29tbWVudDU1MjI3NjI0Nw==,9599,simonw,2019-11-11T03:13:00Z,2019-11-11T03:13:00Z,OWNER,#622,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",506297048,upgrade to uvicorn-0.9 to be Python-3.8 friendly, https://github.com/simonw/datasette/issues/594#issuecomment-547373739,https://api.github.com/repos/simonw/datasette/issues/594,547373739,MDEyOklzc3VlQ29tbWVudDU0NzM3MzczOQ==,2680980,willingc,2019-10-29T11:21:52Z,2019-10-29T11:21:52Z,NONE,"Just an FYI for folks wishing to run datasette with Python 3.8, I was able to successfully use datasette with the following in a virtual environment: ``` pip install uvloop==0.14.0rc1 pip install uvicorn==0.9.1 ``` ","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",506297048,upgrade to uvicorn-0.9 to be Python-3.8 friendly, https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1606415188,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1606415188,IC_kwDOCGYnMM5fv_NU,9599,simonw,2023-06-26T01:46:47Z,2023-06-26T01:47:01Z,OWNER,"I just tested this in a brand new virtual environment using the macOS Python 3: ```bash pipenv shell --python /Applications/Xcode.app/Contents/Developer/usr/bin/python3 ``` Then in that virtual environment I ran: ```bash pip install sqlite-utils # Confirm the right one is on the path: which sqlite-utils curl ""https://data.nasa.gov/resource/y77d-th95.json"" | \ sqlite-utils insert meteorites.db meteorites - --pk=id sqlite-utils extract meteorites.db meteorites recclass ``` This threw the same error reported above. Then I did this: ```bash rm meteorites.db pip install sqlean.py curl ""https://data.nasa.gov/resource/y77d-th95.json"" | \ sqlite-utils insert meteorites.db meteorites - --pk=id sqlite-utils extract meteorites.db meteorites recclass ``` And that second time it worked correctly.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified, https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556210844,https://api.github.com/repos/simonw/sqlite-utils/issues/545,1556210844,IC_kwDOCGYnMM5cweSc,9599,simonw,2023-05-21T15:44:10Z,2023-05-21T15:44:10Z,OWNER,"It looks like `nargs=-1` on a positional argument isn't yet supported - opened an issue here: - https://github.com/Textualize/trogon/issues/4","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1718517882,Try out Trogon for a tui interface, https://github.com/simonw/sqlite-utils/pull/519#issuecomment-1539058795,https://api.github.com/repos/simonw/sqlite-utils/issues/519,1539058795,IC_kwDOCGYnMM5bvCxr,9599,simonw,2023-05-08T21:12:52Z,2023-05-08T21:12:52Z,OWNER,"This is a really neat fix, thank you.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1505568103,Fixes breaking DEFAULT values, https://github.com/simonw/sqlite-utils/issues/490#issuecomment-1258437060,https://api.github.com/repos/simonw/sqlite-utils/issues/490,1258437060,IC_kwDOCGYnMM5LAjnE,9599,simonw,2022-09-26T18:24:44Z,2022-09-26T18:24:44Z,OWNER,Just saw your great write-up on this: https://jeqo.github.io/notes/2022-09-24-ingest-logs-sqlite/,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1382457780,Ability to insert multi-line files, https://github.com/simonw/sqlite-utils/pull/480#issuecomment-1232419522,https://api.github.com/repos/simonw/sqlite-utils/issues/480,1232419522,IC_kwDOCGYnMM5JdTrC,9599,simonw,2022-08-31T03:33:27Z,2022-08-31T03:33:27Z,OWNER,"Tests look great, thank you!","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1355433619,search_sql add include_rank option, https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065440445,https://api.github.com/repos/simonw/sqlite-utils/issues/411,1065440445,IC_kwDOCGYnMM4_gVS9,9599,simonw,2022-03-11T19:52:15Z,2022-03-11T19:52:15Z,OWNER,"Two new parameters to `.create_table()` and friends: - `generated={...}` - generated column definitions - `generated_stored={...}` generated stored column definitions These columns will be added at the end of the table, but you can use the `column_order=` parameter to apply a different order.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1160034488,Support for generated columns, https://github.com/simonw/sqlite-utils/issues/397#issuecomment-1030730108,https://api.github.com/repos/simonw/sqlite-utils/issues/397,1030730108,IC_kwDOCGYnMM49b7F8,9599,simonw,2022-02-06T01:30:46Z,2022-02-06T01:30:46Z,OWNER,Updated documentation is here: https://sqlite-utils.datasette.io/en/latest/python-api.html#explicitly-creating-a-table,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1123903919,Support IF NOT EXISTS for table creation, https://github.com/simonw/datasette/pull/1967#issuecomment-1368267484,https://api.github.com/repos/simonw/datasette/issues/1967,1368267484,IC_kwDOBm6k_c5Rjhrc,9599,simonw,2022-12-31T19:15:50Z,2022-12-31T19:15:50Z,OWNER,"My Firefox tab before: And after: ","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1503010009,Add favicon to documentation, https://github.com/simonw/datasette/issues/1886#issuecomment-1313052863,https://api.github.com/repos/simonw/datasette/issues/1886,1313052863,IC_kwDOBm6k_c5OQ5i_,9599,simonw,2022-11-14T03:40:50Z,2022-11-14T03:40:50Z,OWNER,"Tim Sherratt on Twitter: https://twitter.com/wragge/status/1591930345469153282 > Where do I start? The [#GLAMWorkbench](https://twitter.com/hashtag/GLAMWorkbench?src=hashtag_click) now includes a number of examples where GLAM data is harvested, processed, and then made available for exploration via Datasette. > > https://glam-workbench.net/ > > For example the GLAM Name Index Search brings together 10+ million entries from 240 indexes and provides an aggregated search using the Datasette search-all plugin: > > https://glam-workbench.net/name-search/ > > Most recently I converted PDFs of the Tasmanian Postal Directories to a big Datasette instance: https://updates.timsherratt.org/2022/09/15/from-pdfs-to.html the process is documented and reusable.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1447050738,"Call for birthday presents: if you're using Datasette, let us know how you're using it here", https://github.com/simonw/datasette/issues/526#issuecomment-1259693536,https://api.github.com/repos/simonw/datasette/issues/526,1259693536,IC_kwDOBm6k_c5LFWXg,9599,simonw,2022-09-27T15:42:55Z,2022-09-27T15:42:55Z,OWNER,"It's interesting to note WHY the time limit works against this so well. The time limit as-implemented looks like this: https://github.com/simonw/datasette/blob/5f9f567acbc58c9fcd88af440e68034510fb5d2b/datasette/utils/__init__.py#L181-L201 The key here is `conn.set_progress_handler(handler, n)` - which specifies that the handler function should be called every `n` SQLite operations. The handler function then checks to see if too much time has transpired and conditionally cancels the query. This also doubles up as a ""maximum number of operations"" guard, which is what's happening when you attempt to fetch an infinite number of rows from an infinite table. That limit code could even be extended to say ""exit the query after either 5s or 50,000,000 operations"". I don't think that's necessary though. To be honest I'm having trouble with the idea of dropping `max_returned_rows` mainly because what Datasette does (allow arbitrary untrusted SQL queries) is dangerous, so I've designed in multiple redundant defence-in-depth mechanisms right from the start.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",459882902,Stream all results for arbitrary SQL and canned queries, https://github.com/simonw/datasette/pull/1759#issuecomment-1160712911,https://api.github.com/repos/simonw/datasette/issues/1759,1160712911,IC_kwDOBm6k_c5FLxLP,9599,simonw,2022-06-20T17:58:37Z,2022-06-20T17:58:37Z,OWNER,This is a great idea.,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1275523220,Extract facet portions of table.html out into included templates, https://github.com/simonw/datasette/issues/1546#issuecomment-997124280,https://api.github.com/repos/simonw/datasette/issues/1546,997124280,IC_kwDOBm6k_c47bui4,9599,simonw,2021-12-18T02:05:16Z,2021-12-18T02:05:16Z,OWNER,"Sure - there are actually several levels to this. The code that creates connections to the database is this: https://github.com/simonw/datasette/blob/83bacfa9452babe7bd66e3579e23af988d00f6ac/datasette/database.py#L72-L95 For files on disk, it does this: ```python # For read-only connections conn = sqlite3.connect( ""file:my.db?mode=ro"", uri=True, check_same_thread=False) # For connections that should be treated as immutable: conn = sqlite3.connect( ""file:my.db?immutable=1"", uri=True, check_same_thread=False) ``` For in-memory databases it runs this after the connection has been created: ```python conn.execute(""PRAGMA query_only=1"") ``` SQLite `PRAGMA` queries are treated as dangerous: someone could run `PRAGMA query_only=0` to turn that previous option off for example. So this function runs against any incoming SQL to verify that it looks like a `SELECT ...` and doesn't have anything like that in it. https://github.com/simonw/datasette/blob/83bacfa9452babe7bd66e3579e23af988d00f6ac/datasette/utils/__init__.py#L195-L204 You can see the tests for that here: https://github.com/simonw/datasette/blob/b1fed48a95516ae84c0f020582303ab50ab817e2/tests/test_utils.py#L136-L170","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1076057610,validating the sql, https://github.com/simonw/datasette/issues/1553#issuecomment-996103956,https://api.github.com/repos/simonw/datasette/issues/1553,996103956,IC_kwDOBm6k_c47X1cU,9599,simonw,2021-12-16T19:14:38Z,2021-12-16T19:14:38Z,OWNER,This is a really interesting idea - kind of similar to how many APIs include custom HTTP headers informing of rate-limits.,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1079111498,if csv export is truncated in non streaming mode set informative response header, https://github.com/simonw/datasette/issues/1528#issuecomment-988468238,https://api.github.com/repos/simonw/datasette/issues/1528,988468238,IC_kwDOBm6k_c466tQO,30934,20after4,2021-12-08T03:35:45Z,2021-12-08T03:35:45Z,NONE,"FWIW I implemented something similar with a bit of plugin code: ```python @hookimpl def canned_queries(datasette: Datasette, database: str) -> Mapping[str, str]: # load ""canned queries"" from the filesystem under # www/sql/db/query_name.sql queries = {} sqldir = Path(__file__).parent.parent / ""sql"" if database: sqldir = sqldir / database if not sqldir.is_dir(): return queries for f in sqldir.glob('*.sql'): try: sql = f.read_text('utf8').strip() if not len(sql): log(f""Skipping empty canned query file: {f}"") continue queries[f.stem] = { ""sql"": sql } except OSError as err: log(err) return queries ```","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1060631257,"Add new `""sql_file""` key to Canned Queries in metadata?", https://github.com/simonw/datasette/pull/1467#issuecomment-943632697,https://api.github.com/repos/simonw/datasette/issues/1467,943632697,IC_kwDOBm6k_c44PrE5,9599,simonw,2021-10-14T18:54:18Z,2021-10-14T18:54:18Z,OWNER,The test there failed because it turns out there's a whole bunch of places that set the `Access-Control-Allow-Origin` header. I'm going to close this PR and ship a fix that refactors those places to use the same code.,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",991575770,Add Authorization header when CORS flag is set, https://github.com/simonw/datasette/issues/1402#issuecomment-886969541,https://api.github.com/repos/simonw/datasette/issues/1402,886969541,IC_kwDOBm6k_c403hTF,9599,simonw,2021-07-26T19:31:40Z,2021-07-26T19:31:40Z,OWNER,"Datasette could do a pretty good job of this by default, using `twitter:card` and `og:url` tags - like on https://til.simonwillison.net/jq/extracting-objects-recursively I could also provide a mechanism to customize these - in particular to add images of some sort. It feels like something that should tie in to the metadata mechanism.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",951185411,feature request: social meta tags,