issues
2,892 rows sorted by title
This data as json, CSV (advanced)
id | node_id | number | title ▼ | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
732634375 | MDExOlB1bGxSZXF1ZXN0NTEyNTQ1MzY0 | 1061 | .blob output renderer | simonw 9599 | closed | 0 | 0.51 6026070 | 4 | 2020-10-29T20:25:08Z | 2020-10-29T22:01:40Z | 2020-10-29T22:01:39Z | OWNER | simonw/datasette/pulls/1061 | - [x] Remove the `/-/...blob/...` route I added in #1040 in place of the new `.blob` renderer URLs - [x] Link to new `.blob` download links on the arbitrary query page (using `_blob_hash=...`) - plus tests for this Closes #1050, Closes #1051 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/1061/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||
581339961 | MDU6SXNzdWU1ODEzMzk5NjE= | 92 | .columns_dict doesn't work for all possible column types | simonw 9599 | closed | 0 | 7 | 2020-03-14T19:30:35Z | 2020-03-15T18:37:43Z | 2020-03-14T20:04:14Z | OWNER | Got this error: ``` File ".../python3.7/site-packages/sqlite_utils/db.py", line 462, in <dictcomp> for column in self.columns KeyError: 'REAL' ``` `.columns_dict` uses `REVERSE_COLUMN_TYPE_MAPPING`: https://github.com/simonw/sqlite-utils/blob/43f1c6ab4e3a6b76531fb6f5447adb83d26f3971/sqlite_utils/db.py#L457-L463 `REVERSE_COLUMN_TYPE_MAPPING` defines `FLOAT` not `REAL`A https://github.com/simonw/sqlite-utils/blob/43f1c6ab4e3a6b76531fb6f5447adb83d26f3971/sqlite_utils/db.py#L68-L74 | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/92/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
732685643 | MDU6SXNzdWU3MzI2ODU2NDM= | 1063 | .csv should link to .blob downloads | simonw 9599 | closed | 0 | 0.51 6026070 | 3 | 2020-10-29T21:45:58Z | 2021-06-17T18:12:30Z | 2020-10-29T22:47:45Z | OWNER | - [x] Update `.csv` output to link to these things (and get that `xfail` test to pass) - ~~Add a `.csv?_blob_base64=1` argument that causes them to be output in base64 in the CSV~~ > Moving the CSV work to a separate ticket. _Originally posted by @simonw in https://github.com/simonw/datasette/pull/1061#issuecomment-719042601_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1063/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1223699280 | I_kwDOBm6k_c5I8CtQ | 1739 | .db downloads should be served with an ETag | simonw 9599 | closed | 0 | 6 | 2022-05-03T05:11:21Z | 2022-05-04T18:21:18Z | 2022-05-03T14:59:51Z | OWNER | I noticed that my Pyodide Datasette prototype is downloading the same database file every single time rather than browser caching it: ![image](https://user-images.githubusercontent.com/9599/166407074-dee19587-0667-4424-9e88-d3b5b90fd819.png) | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1739/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
702386948 | MDU6SXNzdWU3MDIzODY5NDg= | 159 | .delete_where() does not auto-commit (unlike .insert() or .upsert()) | spdkils 11712349 | open | 0 | 9 | 2020-09-16T01:55:52Z | 2023-04-01T17:21:05Z | NONE | When you use the delete_where() function on a table, it never commits.... Is that intentional? | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/159/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
570101428 | MDExOlB1bGxSZXF1ZXN0Mzc5MTkyMjU4 | 683 | .execute_write() and .execute_write_fn() methods on Database | simonw 9599 | closed | 0 | Datasette 1.0 3268330 | 14 | 2020-02-24T19:51:58Z | 2020-05-30T18:40:20Z | 2020-02-25T04:45:08Z | OWNER | simonw/datasette/pulls/683 | See #682 - [x] Come up with design for `.execute_write()` and `.execute_write_fn()` - [x] Build some quick demo plugins to exercise the design - [x] Write some unit tests - [x] Write the documentation | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/683/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||
1199158210 | I_kwDOCGYnMM5HebPC | 423 | .extract() doesn't set foreign key when extracted columns contain NULL value | jlieth 37447552 | closed | 0 | 1 | 2022-04-10T20:05:30Z | 2022-08-27T14:45:04Z | 2022-08-27T14:45:04Z | NONE | I've run into an issue with `extract` and I don't believe this is the intended behaviour. I'm working with a database with music listening information. Currently it has one large table `listens` that contains all information. I'm trying to normalize the database by extracting relevant columns to separate tables (`artists`, `tracks`, `albums`). Not every track has an album. A simplified demonstration with just `track_title` and `album_title` columns: ```ipython In [1]: import sqlite_utils In [2]: db = sqlite_utils.Database(memory=True) In [3]: db["listens"].insert_all([ ...: {"id": 1, "track_title": "foo", "album_title": "bar"}, ...: {"id": 2, "track_title": "baz", "album_title": None} ...: ], pk="id") Out[3]: <Table listens (id, track_title, album_title)> ``` The track in the first row has an album, the second track doesn't. Now I extract album information into a separate column: ```ipython In [4]: db["listens"].extract(columns=["album_title"], table="albums", fk_column="album_id") Out[4]: <Table listens (id, track_title, album_id)> In [5]: list(db["albums"].rows) Out[5]: [{'id': 1, 'album_title': 'bar'}, {'id': 2, 'album_title': None}] In [6]: list(db["listens"].rows) Out[6]: [{'id': 1, 'track_title': 'foo', 'album_id': 1}, {'id': 2, 'track_title': 'baz', 'album_id': None}] ``` This behaves as expected -- the `album` table contains entries for both the existing album and the NULL album. The `listens` table has a foreign key only for the first row (since the album in the second row was empty). Now I want to extract the track information as well. Album information belongs to the track so I want to extract both columns to a new table. ```ipython In [7]: db["listens"].extract(columns=["track_title", "album_id"], table="tracks", fk_column="track_id") Out[7]: <Table listens (id, track_id)> In [8]: list(db["tracks"].rows) Out[8]: [{'id': 1, 'track_title': 'foo', 'album_id': 1}, {'id': 2, 'track_title': 'baz', 'album_id': None}] In [9]: list(db["… | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/423/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
722816436 | MDU6SXNzdWU3MjI4MTY0MzY= | 186 | .extract() shouldn't extract null values | simonw 9599 | open | 0 | 7 | 2020-10-16T02:41:08Z | 2021-08-12T12:32:14Z | OWNER | This almost works, but it creates a rogue `type` record with a value of None. ``` In [1]: import sqlite_utils In [2]: db = sqlite_utils.Database(memory=True) In [5]: db["creatures"].insert_all([ {"id": 1, "name": "Simon", "type": None}, {"id": 2, "name": "Natalie", "type": None}, {"id": 3, "name": "Cleo", "type": "dog"}], pk="id") Out[5]: <Table creatures (id, name, type)> In [7]: db["creatures"].extract("type") Out[7]: <Table creatures (id, name, type_id)> In [8]: list(db["creatures"].rows) Out[8]: [{'id': 1, 'name': 'Simon', 'type_id': None}, {'id': 2, 'name': 'Natalie', 'type_id': None}, {'id': 3, 'name': 'Cleo', 'type_id': 2}] In [9]: db["type"] Out[9]: <Table type (id, type)> In [10]: list(db["type"].rows) Out[10]: [{'id': 1, 'type': None}, {'id': 2, 'type': 'dog'}] ``` | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/186/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
467928674 | MDExOlB1bGxSZXF1ZXN0Mjk3NDU5Nzk3 | 40 | .get() method plus support for compound primary keys | simonw 9599 | closed | 0 | 1 | 2019-07-15T03:43:13Z | 2019-07-15T04:28:57Z | 2019-07-15T04:28:52Z | OWNER | simonw/sqlite-utils/pulls/40 | - [x] Tests for the `NotFoundError` exception - [x] Documentation for `.get()` method - [x] Support `--pk` multiple times to define CLI compound primary keys - [x] Documentation for compound primary keys | sqlite-utils 140912432 | pull | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/40/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
413871266 | MDU6SXNzdWU0MTM4NzEyNjY= | 18 | .insert/.upsert/.insert_all/.upsert_all should add missing columns | simonw 9599 | closed | 0 | 1.0 4348046 | 2 | 2019-02-24T21:36:11Z | 2019-05-25T00:42:11Z | 2019-05-25T00:42:11Z | OWNER | This is a larger change, but it would be incredibly useful: if you attempt to insert or update a document with a field that does not currently exist in the underlying table, sqlite-utils should add the appropriate column for you. | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
808046597 | MDU6SXNzdWU4MDgwNDY1OTc= | 234 | .insert_all() fails if subsequent chunks contain additional columns | simonw 9599 | closed | 0 | 1 | 2021-02-14T21:01:51Z | 2021-02-14T21:03:40Z | 2021-02-14T21:03:40Z | OWNER | Reported by @nieuwenhoven in #225 along with a proposed fix. | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/234/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
403625674 | MDU6SXNzdWU0MDM2MjU2NzQ= | 7 | .insert_all() should accept a generator and process it efficiently | simonw 9599 | closed | 0 | 3 | 2019-01-28T02:11:58Z | 2019-01-28T06:26:53Z | 2019-01-28T06:26:53Z | OWNER | Right now you have to load every record into memory before passing the list to `.insert_all()` and friends. If you want to process millions of rows, this is inefficient. Python has generators - we should use them! The only catch here is that part of the magic of `sqlite-utils` is that it guesses the column types and creates the table for you. This code will need to be updated to notice if the table needs creating and, if it does, create it using the first X (where x=1,000 but can be customized) records. If a record outside of those first 1,000 has a rogue column, we can crash with an error. This will free us up to make the `--nl` option added in #6 much more efficient. | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
742011049 | MDU6SXNzdWU3NDIwMTEwNDk= | 1091 | .json and .csv exports fail to apply base_url | simonw 9599 | closed | 0 | Datasette 0.54 6346396 | 22 | 2020-11-12T23:45:16Z | 2021-01-24T21:20:24Z | 2021-01-09T22:19:29Z | OWNER | > Just tested with the latest Docker image, and it works pretty much everywhere! THANK YOU! > > I did notice that if I try to export json or csv, the base is not applied. Not sure if I should reopen this issue or open a new one. > > To see this, go here: https://corpora.tika.apache.org/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES > > Click/hover over json or CSV and you'll see that the 'datasette' base is not included. _Originally posted by @tballison in https://github.com/simonw/datasette/issues/865#issuecomment-726385422_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1091/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
403617881 | MDU6SXNzdWU0MDM2MTc4ODE= | 405 | .json?_nl=on option for exporting newline-delimited JSON | simonw 9599 | closed | 0 | 2 | 2019-01-28T01:10:45Z | 2019-01-28T01:49:00Z | 2019-01-28T01:48:37Z | OWNER | The neat thing about newline-delimited JSON is that you don't have to read an entire array (of potentially thousands of objects) into memory in order to parse it - you can parse things a line at a time instead. It will look like this: `https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on` ``` {"pk": 1, "planet_int": 1, "on_earth": 1, "state": "CA", "city_id": 1, "neighborhood": "Mission"} {"pk": 2, "planet_int": 1, "on_earth": 1, "state": "CA", "city_id": 1, "neighborhood": "Dogpatch"} {"pk": 3, "planet_int": 1, "on_earth": 1, "state": "CA", "city_id": 1, "neighborhood": "SOMA"} {"pk": 4, "planet_int": 1, "on_earth": 1, "state": "CA", "city_id": 1, "neighborhood": "Tenderloin"} {"pk": 5, "planet_int": 1, "on_earth": 1, "state": "CA", "city_id": 1, "neighborhood": "Bernal Heights"} ``` I added this as part of the `sqlite-utils json` CLI command is this commit - I think Datasette should offer it as well: https://github.com/simonw/sqlite-utils/commit/5466c9745dfef858286146ea158ffd5a71391d10 It can be offered alongside `_stream=on` (which currently only works for CSV, but it could work for JSON as well thanks to this trick). | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/405/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
787900412 | MDU6SXNzdWU3ODc5MDA0MTI= | 222 | .m2m() should accept alter=True parameter | simonw 9599 | closed | 0 | 0 | 2021-01-18T04:15:43Z | 2021-01-18T04:26:10Z | 2021-01-18T04:26:10Z | OWNER | Needed by https://github.com/dogsheep/swarm-to-sqlite/issues/11 | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/222/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
738128913 | MDU6SXNzdWU3MzgxMjg5MTM= | 201 | .search(columns=) and sqlite-utils search -c ... bug | simonw 9599 | closed | 0 | 3.0 6079500 | 1 | 2020-11-07T01:27:26Z | 2020-11-08T16:54:15Z | 2020-11-08T16:54:15Z | OWNER | Both `table.search(columns=)` and the `sqlite-utils search -c` option do not work as expected - they always return both the `rowid` and the `rank` columns even if those have not been requested. This should be fixed before the 3.0 non-alpha release. | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/201/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
925320167 | MDU6SXNzdWU5MjUzMjAxNjc= | 284 | .transform(types=) turns rowid into a concrete column | simonw 9599 | closed | 0 | 5 | 2021-06-19T05:25:27Z | 2021-06-19T15:28:30Z | 2021-06-19T15:28:30Z | OWNER | Noticed this in the tests for `sqlite-utils memory` in #282 - is it possible to fix this? https://github.com/simonw/sqlite-utils/commit/ec5174ed40fa283cb06f25ee0c0136297ec313ae | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/284/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
561460274 | MDU6SXNzdWU1NjE0NjAyNzQ= | 84 | .upsert() with hash_id throws error | simonw 9599 | closed | 0 | 0 | 2020-02-07T07:08:19Z | 2020-02-07T07:17:11Z | 2020-02-07T07:17:11Z | OWNER | ```python db[table_name].upsert_all(rows, hash_id="pk") ``` This throws an error: `PrimaryKeyRequired('upsert() requires a pk')` The problem is, if you try this: ```python db[table_name].upsert_all(rows, hash_id="pk", pk="pk") ``` You get this error: `AssertionError('Use either pk= or hash_id=')` `hash_id=` should imply that `pk=` that column. | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/84/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
598640234 | MDU6SXNzdWU1OTg2NDAyMzQ= | 99 | .upsert_all() should maybe error if dictionaries passed to it do not have the same keys | simonw 9599 | closed | 0 | 2 | 2020-04-13T03:02:25Z | 2020-04-13T03:05:20Z | 2020-04-13T03:05:04Z | OWNER | While investigating #98 I stumbled across this: ``` def test_upsert_compound_primary_key(fresh_db): table = fresh_db["table"] table.upsert_all( [ {"species": "dog", "id": 1, "name": "Cleo", "age": 4}, {"species": "cat", "id": 1, "name": "Catbag"}, ], pk=("species", "id"), ) table.upsert_all( [ {"species": "dog", "id": 1, "age": 5}, {"species": "dog", "id": 2, "name": "New Dog", "age": 1}, ], pk=("species", "id"), ) > assert [ {"species": "dog", "id": 1, "name": "Cleo", "age": 5}, {"species": "cat", "id": 1, "name": "Catbag", "age": None}, {"species": "dog", "id": 2, "name": "New Dog", "age": 1}, ] == list(table.rows) E AssertionError: assert [{'age': 5, '...cies': 'dog'}] == [{'age': 5, '...cies': 'dog'}] E At index 0 diff: {'species': 'dog', 'id': 1, 'name': 'Cleo', 'age': 5} != {'species': 'dog', 'id': 1, 'name': None, 'age': 5} E Full diff: E - [{'age': 5, 'id': 1, 'name': 'Cleo', 'species': 'dog'}, E ? ^^^ -- E + [{'age': 5, 'id': 1, 'name': None, 'species': 'dog'}, E ? ^^^ E {'age': None, 'id': 1, 'name': 'Catbag', 'species': 'cat'}, E {'age': 1, 'id': 2, 'name': 'New Dog', 'species': 'dog'}] ``` If you run `.upsert_all()` with multiple dictionaries it doesn't quite have the effect you might expect. | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/99/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
315548495 | MDU6SXNzdWUzMTU1NDg0OTU= | 225 | /-/(inspect|metadata|plugins)(.json)? introspection | simonw 9599 | closed | 0 | 0 | 2018-04-18T16:14:58Z | 2018-04-19T05:25:33Z | 2018-04-19T05:25:33Z | OWNER | 3 pages (and accompanying .json endpoints) for viewing: * the metadata.json that datasette was loaded with * the output of ds.inspect() * a list of installed plugins, detected by pluggy | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/225/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
800669347 | MDU6SXNzdWU4MDA2NjkzNDc= | 1216 | /-/databases should reflect connection order, not alphabetical order | simonw 9599 | closed | 0 | 1 | 2021-02-03T20:20:23Z | 2021-06-02T03:10:19Z | 2021-06-02T03:10:19Z | OWNER | The order in which databases are attached to Datasette matters - it affects the homepage, and it's beginning to influence how certain plugins work (see https://github.com/simonw/datasette-tiles/issues/8). Two years ago in cccea85be6aaaeadb31f3b588ec7f732628815f5 I made `/-/databases` return things in alphabetical order, to fix a test failure in Python 3.5. Python 3.5 is no longer supported, so this is no longer necessary - and this behaviour should now be treated as a bug. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1216/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
444997937 | MDU6SXNzdWU0NDQ5OTc5Mzc= | 470 | /-/databases showing currently attached database details | simonw 9599 | closed | 0 | 0.28 4305096 | 1 | 2019-05-16T14:45:18Z | 2019-05-19T19:28:44Z | 2019-05-16T14:50:26Z | OWNER | Split from #419. Mainly useful to see what is connected as mutable v.s. immutable. Also helps fill the gap left by `/-/inspect` until #465 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/470/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
637253789 | MDU6SXNzdWU2MzcyNTM3ODk= | 833 | /-/metadata and so on should respect view-instance permission | simonw 9599 | closed | 0 | Datasette 0.44 5512395 | 4 | 2020-06-11T19:07:21Z | 2020-06-11T22:15:32Z | 2020-06-11T22:14:59Z | OWNER | The only URLs that should be available without authentication at all times are the `/-/static/` prefix, to allow for HTTP caching. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/833/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1122416919 | I_kwDOBm6k_c5C5rkX | 1623 | /-/patterns returns link: alternate JSON header to 404 | simonw 9599 | closed | 0 | Datasette 1.0 3268330 | 2 | 2022-02-02T21:42:49Z | 2022-03-19T04:04:49Z | 2022-02-02T21:48:56Z | OWNER | Bug from: - #1620 ``` % curl -s -I 'https://latest.datasette.io/-/patterns' | grep link link: https://latest.datasette.io/-/patterns.json; rel="alternate"; type="application/json+datasette" ``` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1623/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
319954545 | MDU6SXNzdWUzMTk5NTQ1NDU= | 248 | /-/plugins should show version of each installed plugin | simonw 9599 | closed | 0 | 2 | 2018-05-03T14:50:45Z | 2018-05-04T18:25:40Z | 2018-05-04T18:05:04Z | OWNER | Refs #244 https://stackoverflow.com/questions/20180543/how-to-check-version-of-python-modules ``` >>> import pkg_resources >>> pkg_resources.get_distribution('datasette_cluster_map').version '0.4' ``` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/248/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
512218858 | MDU6SXNzdWU1MTIyMTg4NTg= | 606 | /-/plugins shows incorrect name for plugins | simonw 9599 | closed | 0 | 3 | 2019-10-24T22:53:25Z | 2019-11-01T05:41:04Z | 2019-11-01T05:40:07Z | OWNER | https://fivethirtyeight.datasettes.com/-/plugins ```json [ { "name": "datasette_jellyfish", "static": false, "templates": false, "version": "0.3" }, { "name": "datasette_vega", "static": true, "templates": false, "version": "0.6.2" } ] ``` These should be shown as `datasette-jellyfish` and `datasette-vega` since those are the names on PyPI. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/606/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
778530523 | MDU6SXNzdWU3Nzg1MzA1MjM= | 1172 | /-/static should be excluded from auth and permission checks | simonw 9599 | open | 0 | 0 | 2021-01-05T02:53:41Z | 2021-01-05T02:53:41Z | OWNER | I want to set far future / immutable cache headers on everything served from `/-/static` and `/-/static-plugins` This has security implications since it will be possible to see what plugins are installed by checking for known static URLs. I'm fine with that - performance is more important here. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1172/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
398089089 | MDU6SXNzdWUzOTgwODkwODk= | 399 | /-/versions for official Docker image returns wrong Datasette version | simonw 9599 | closed | 0 | 2 | 2019-01-11T01:19:58Z | 2019-01-13T23:31:59Z | 2019-01-13T23:10:45Z | OWNER | ``` docker run -p 8001:8001 datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 ``` http://0.0.0.0:8001/-/versions returns this: ``` { "datasette": { "version": "0+unknown" }, ... ``` This is because the Docker image is built by copying in the Datasette source code, which confuses versioneer. Maybe the Docker image should install the code using a wheel or similar? | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/399/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
318738000 | MDU6SXNzdWUzMTg3MzgwMDA= | 244 | /-/versions page | simonw 9599 | closed | 0 | 1 | 2018-04-29T18:22:15Z | 2018-05-03T14:13:49Z | 2018-05-03T14:09:53Z | OWNER | Displays the current version of: * datasette * Python * SQLite * Spatialite (if available) Installed plugin versions should be shown on /-/plugins | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/244/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
321624016 | MDU6SXNzdWUzMjE2MjQwMTY= | 252 | /-/versions should report the FTS version supported by SQLite | simonw 9599 | closed | 0 | 0 | 2018-05-09T15:43:47Z | 2018-05-11T13:19:52Z | 2018-05-11T13:19:52Z | OWNER | I can copy this function from `csvs-to-sqlite`: https://github.com/simonw/csvs-to-sqlite/blob/dccbf65b37bc9eed50e9edb80a42f257e93edb1f/csvs_to_sqlite/utils.py#L283-L293 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/252/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
663317875 | MDU6SXNzdWU2NjMzMTc4NzU= | 905 | /database.db download should include content-length header | simonw 9599 | closed | 0 | 2 | 2020-07-21T21:23:48Z | 2020-07-22T04:59:46Z | 2020-07-22T04:52:45Z | OWNER | I can do this by modifying this function: https://github.com/simonw/datasette/blob/02dc6298bdbfb1d63e0d2a39ff597b5fcc60e06b/datasette/utils/asgi.py#L248-L270 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/905/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
462117311 | MDU6SXNzdWU0NjIxMTczMTE= | 531 | /database/-/inspect | simonw 9599 | open | 0 | 1 | 2019-06-28T16:33:41Z | 2019-07-08T15:43:57Z | OWNER | Build `/database/-/inspect` which shows tables, columns, column types and foreign keys It won't show table counts. Or maybe it will include them optionally but only for `-i` databases, in a special area of the JSON reserved for immutable-only inspect details. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/465#issuecomment-506797086_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/531/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
267886865 | MDU6SXNzdWUyNjc4ODY4NjU= | 28 | /database?sql= should redirect correctly | simonw 9599 | closed | 0 | Ship first public release 2857392 | 0 | 2017-10-24T03:38:44Z | 2017-10-24T23:54:30Z | 2017-10-24T23:54:30Z | OWNER | Needs to redirect to the location with the hash while retaining the query string. This should also work with the .json extension. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/28/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1483320357 | I_kwDOBm6k_c5Yaawl | 1937 | /db/-/create API should require insert-rows permission to use row: or rows: option | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 2 | 2022-12-08T01:33:09Z | 2022-12-14T20:21:26Z | 2022-12-14T20:21:26Z | OWNER | Otherwise someone with `create-table` but no` insert-rows` permission could abuse it to insert data. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1937/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
726910999 | MDExOlB1bGxSZXF1ZXN0NTA3OTAzMzky | 1040 | /db/table/-/blob/pk/column.blob download URL | simonw 9599 | closed | 0 | 0.51 6026070 | 3 | 2020-10-21T22:39:15Z | 2020-10-24T23:09:20Z | 2020-10-24T23:09:19Z | OWNER | simonw/datasette/pulls/1040 | Refs #1036. Still needs: - [x] Comprehensive tests across all of the code branches, plus permissions - [x] A bit more refactoring to share logic cleanly with `RowView` - ~~A configuration option to disable this feature (probably)~~ | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/1040/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||
1426080014 | I_kwDOBm6k_c5VAEEO | 1867 | /db/table/-/rename API (also allows atomic replace) | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 1 | 2022-10-27T18:13:23Z | 2023-01-09T15:34:12Z | OWNER | > There's one catch with batched inserts: if your CLI tool fails half way through you could end up with a partially populated table - since a bunch of batches will have succeeded first. > > ... > > If people care about that kind of thing they could always push all of their inserts to a table called `_tablename` and then atomically rename that once they've uploaded all of the data (assuming I provide an atomic-rename-this-table mechanism). _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1866#issuecomment-1293893789_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1867/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1473814539 | PR_kwDOBm6k_c5EMVug | 1931 | /db/table/-/upsert | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 8 | 2022-12-03T07:01:44Z | 2022-12-08T01:12:17Z | 2022-12-08T01:12:16Z | OWNER | simonw/datasette/pulls/1931 | Refs #1878 Still todo: - [x] Support `"return": true` properly for upserts (with tests) - [x] Require both `insert-row` and `update-row` permissions - [x] Tests are going to need to cover both rowid-only and compound primary key tables, including all of the error states - [x] Documentation <!-- readthedocs-preview datasette start --> ---- :books: Documentation preview :books:: https://datasette--1931.org.readthedocs.build/en/1931/ <!-- readthedocs-preview datasette end --> | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/1931/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||
1432013704 | I_kwDOBm6k_c5VWsuI | 1878 | /db/table/-/upsert API | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 8 | 2022-11-01T20:01:18Z | 2022-12-08T01:12:18Z | 2022-12-08T01:12:17Z | OWNER | Equivalent to `sqlite-utils upsert`: https://sqlite-utils.datasette.io/en/stable/python-api.html#upserting-data | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1878/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
647095808 | MDU6SXNzdWU2NDcwOTU4MDg= | 874 | /favicon.ico 500 error | simonw 9599 | closed | 0 | Datasette 0.45 5533512 | 0 | 2020-06-29T04:04:22Z | 2020-06-29T04:27:18Z | 2020-06-29T04:27:18Z | OWNER | ``` Traceback (most recent call last): File "...datasette/datasette/app.py", line 969, in route_path response = await view(request, send) TypeError: favicon() missing 1 required positional argument: 'send' ``` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/874/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1501713288 | I_kwDOBm6k_c5ZglOI | 1963 | 0.63.3 bugfix release | simonw 9599 | closed | 0 | 2 | 2022-12-18T02:48:15Z | 2022-12-18T03:26:55Z | 2022-12-18T03:26:55Z | OWNER | I'm going to ship a release which back-ports these two fixes: - https://github.com/simonw/datasette/issues/1958 - https://github.com/simonw/datasette/issues/1955 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1963/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
1450303205 | I_kwDOBm6k_c5Wcd7l | 1891 | 1.0a0 release notes | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 4 | 2022-11-15T19:58:20Z | 2022-11-29T19:23:41Z | 2022-11-29T19:23:41Z | OWNER | This release will mainly help preview the new Datasette write API: - #1850 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1891/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
735650864 | MDU6SXNzdWU3MzU2NTA4NjQ= | 194 | 3.0 release with some minor breaking changes | simonw 9599 | closed | 0 | 3.0 6079500 | 3 | 2020-11-03T21:36:31Z | 2020-11-08T17:19:35Z | 2020-11-08T17:19:34Z | OWNER | While working on search (#192) I've spotted a few small changes I would like to make that would break backwards compatibility in minor ways, hence requiring a 3.x release. `db[table].search()` - I would like this to default to sorting by rank Also I'd like to free up the `-c` and `-f` options for other purposes from the standard output formats here: https://github.com/simonw/sqlite-utils/blob/43eae8b193d362f2b292df73e087ed6f10838144/sqlite_utils/cli.py#L48-L58 I'd like `-f` to be used to indicate a full-text search column during an insert and `-c` to indicate a column (so you can specify which columns you want to output). | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/194/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
802583450 | MDU6SXNzdWU4MDI1ODM0NTA= | 226 | 3.4 release is broken - includes a rogue line | simonw 9599 | closed | 0 | 0 | 2021-02-06T02:08:01Z | 2021-02-06T02:10:26Z | 2021-02-06T02:10:26Z | OWNER | I started seeing weird errors, caused by this line: https://github.com/simonw/sqlite-utils/blob/f8010ca78fed8c5fca6cde19658ec09fdd468420/sqlite_utils/cli.py#L1-L3 That was added by accident in 1b666f9315d4ea6bb332b2e75e48480c26100199 I'm surprised the tests didn't catch this! | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/226/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
435819321 | MDU6SXNzdWU0MzU4MTkzMjE= | 436 | 400 Error when trying to register new user via https://publish.datasettes.com/ | nniiicc 317694 | closed | 0 | 1 | 2019-04-22T17:55:00Z | 2021-01-04T20:15:42Z | 2021-01-04T20:15:41Z | NONE | Behavior: When registering a new user via Zeit - confirmation is sent and screen acknowledges registered user... When clicking grant access the next screen is a white 400 error message. Replicated: Chrome and Firefox; 2 different email accounts | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/436/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
951817328 | MDU6SXNzdWU5NTE4MTczMjg= | 12 | 403 when getting token | treyhunner 285352 | open | 0 | 1 | 2021-07-23T18:43:26Z | 2021-10-12T18:31:57Z | NONE | I tried to use https://your-foursquare-oauth-token.glitch.me/ to get my Swarm auth token and got a 403 after I clicked the Allow button: ![image](https://user-images.githubusercontent.com/285352/126826478-60e53614-263d-40bb-9f1d-c1a676644eb0.png) I'm not sure if this is the right repo to report this in | swarm-to-sqlite 205429375 | issue | {"url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
1663399821 | I_kwDOBm6k_c5jJXeN | 2058 | 500 "attempt to write a readonly database" error caused by "PRAGMA schema_version" | simonw 9599 | open | 0 | 9 | 2023-04-11T23:57:50Z | 2023-04-13T16:35:21Z | OWNER | I've not been able to replicate this myself yet, but I've seen log files from a user affected by it. ``` File "/usr/local/lib/python3.11/site-packages/datasette/views/base.py", line 89, in dispatch_request await self.ds.refresh_schemas() File "/usr/local/lib/python3.11/site-packages/datasette/app.py", line 371, in refresh_schemas await self._refresh_schemas() File "/usr/local/lib/python3.11/site-packages/datasette/app.py", line 386, in _refresh_schemas schema_version = (await db.execute("PRAGMA schema_version")).first()[0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/datasette/database.py", line 267, in execute results = await self.execute_fn(sql_operation_in_thread) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/datasette/database.py", line 213, in execute_fn return await asyncio.get_event_loop().run_in_executor( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/datasette/database.py", line 211, in in_thread return fn(conn) ^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/datasette/database.py", line 237, in sql_operation_in_thread cursor.execute(sql, params if params is not None else {}) sqlite3.OperationalError: attempt to write a readonly database ``` That's running the official Datasette Docker image on https://fly.io/ - it's causing 500 errors on every page of their site. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/2058/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
810397025 | MDU6SXNzdWU4MTAzOTcwMjU= | 1228 | 500 error caused by faceting if a column called `n` exists | Kabouik 7107523 | closed | 0 | 5 | 2021-02-17T17:41:20Z | 2022-03-19T06:44:40Z | 2022-03-19T01:38:04Z | NONE | I recently discovered `datasette` thanks to your great talk at FOSDEM and would like to use it for some projects. However, when trying to use it on databases created from some csv ot tsv files, I am sometimes getting this issue when going to http://127.0.0.1:8001/databasetest/databasetest and I don't exactly understand what it refers to. So far, I couldn't find anything relevant when reviewing the raw text files that could explain this issue, nor could I find something obvious between the files that generate this issue and those that don't. Does the error ring a bell and, if so, could you please point me to the right direction? ``` $ datasette databasetest.db INFO: Started server process [1408482] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit) INFO: 127.0.0.1:56394 - "GET / HTTP/1.1" 200 OK INFO: 127.0.0.1:56394 - "GET /-/static/app.css?4e362c HTTP/1.1" 200 OK INFO: 127.0.0.1:56396 - "GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1" 200 OK INFO: 127.0.0.1:56398 - "GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1" 200 OK Traceback (most recent call last): File "/home/kabouik/.local/lib/python3.7/site-packages/datasette/app.py", line 1099, in route_path response = await view(request, send) File "/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py", line 147, in view request, **request.scope["url_route"]["kwargs"] File "/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py", line 121, in dispatch_request return await handler(request, *args, **kwargs) File "/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py", line 260, in get request, database, hash, correct_hash_provided, **kwargs File "/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py", line 434, in view_get request, database, hash, **kwargs File "/home/kabouik/.loc… | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1228/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
1318907685 | I_kwDOBm6k_c5OnO8l | 1773 | 500 error if sorted by a column not in the ?_col= list | simonw 9599 | closed | 0 | Datasette 0.62 8303187 | 4 | 2022-07-27T01:20:27Z | 2022-08-14T16:06:25Z | 2022-08-14T15:44:05Z | OWNER | For example: https://latest.datasette.io/fixtures/sortable?_sort_desc=sortable&_col=sortable_with_nulls That's `?_sort_desc=sortable&_col=sortable_with_nulls` <img width="494" alt="image" src="https://user-images.githubusercontent.com/9599/181139719-06e3bf66-3fde-4f9a-bfd1-62b326121601.png"> | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1773/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1308461063 | I_kwDODFdgUs5N_YgH | 74 | 500 error in github-to-sqlite demo | simonw 9599 | closed | 0 | 5 | 2022-07-18T19:39:32Z | 2022-07-18T21:16:18Z | 2022-07-18T21:14:22Z | MEMBER | https://github-to-sqlite.dogsheep.net/github/issue_comments throws a 500: > `cannot import name 'etree' from 'markdown.util' (/usr/local/lib/python3.8/site-packages/markdown/util.py)` https://console.cloud.google.com/run/detail/us-central1/github-to-sqlite/metrics?project=datasette-222320 suggests this started happening 3 days ago. | github-to-sqlite 207052882 | issue | {"url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
463492815 | MDU6SXNzdWU0NjM0OTI4MTU= | 534 | 500 error on m2m facet detection | simonw 9599 | open | 0 | 1 | 2019-07-03T00:42:42Z | 2020-12-17T05:08:22Z | OWNER | This may help debug: ``` diff --git a/datasette/facets.py b/datasette/facets.py index 76d73e5..07a4034 100644 --- a/datasette/facets.py +++ b/datasette/facets.py @@ -499,11 +499,14 @@ class ManyToManyFacet(Facet): "outgoing" ] if len(other_table_outgoing_foreign_keys) == 2: - destination_table = [ - t - for t in other_table_outgoing_foreign_keys - if t["other_table"] != self.table - ][0]["other_table"] + try: + destination_table = [ + t + for t in other_table_outgoing_foreign_keys + if t["other_table"] != self.table + ][0]["other_table"] + except IndexError: + import pdb; pdb.pm() # Only suggest if it's not selected already if ("_facet_m2m", destination_table) in args: continue ``` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/534/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
1493404423 | I_kwDOBm6k_c5ZA4sH | 1948 | 500 error on permission debug page when testing actors with _r | simonw 9599 | open | 0 | 1 | 2022-12-13T05:22:03Z | 2022-12-13T05:22:19Z | OWNER | <img width="1272" alt="image" src="https://user-images.githubusercontent.com/9599/207233390-ecb1b81b-c9f4-40a3-8131-5537ab23e9fc.png"> The 500 error is silent unless you are looking at the DevTools network pane. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1948/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
752789159 | MDU6SXNzdWU3NTI3ODkxNTk= | 1113 | 500 error on row page if query against foreign keys hits time limit | simonw 9599 | closed | 0 | 1 | 2020-11-28T23:20:08Z | 2020-11-29T02:40:01Z | 2020-11-28T23:23:31Z | OWNER | This page exhibited the following error: https://data.catalyst.coop/ferc1/f1_respondent_id/145 `(OperationalError('interrupted'), 'select (select count(*) from f1_acb_epda where respondent_id=:id), (select count(*) from f1_accumdepr_prvsn where respondent_id=:id), (select count(*) from f1_accumdfrrdtaxcr where respondent_id=:id), (select count(*) from f1_adit_190_detail where respondent_id=:id), (select count(*) from f1_adit_190_notes where respondent_id=:id), (select count(*) from f1_adit_amrt_prop where respondent_id=:id), (select count(*) from f1_adit_other where respondent_id=:id), (select count(*) from f1_adit_other_prop where respondent_id=:id), (select count(*) from f1_allowances where respondent_id=:id), (select count(*) from f1_bal_sheet_cr where respondent_id=:id), (select count(*) from f1_capital_stock where respondent_id=:id), (select count(*) from f1_cash_flow where respondent_id=:id), (select count(*) from f1_cmmn_utlty_p_e where respondent_id=:id), (select count(*) from f1_comp_balance_db where respondent_id=:id), (select count(*) from f1_construction where respondent_id=:id), (select count(*) from f1_control_respdnt where respondent_id=:id), (select count(*) from f1_co_directors where respondent_id=:id), (select count(*) from f1_cptl_stk_expns where respondent_id=:id), (select count(*) from f1_csscslc_pcsircs where respondent_id=:id), (select count(*) from f1_dacs_epda where respondent_id=:id), (select count(*) from f1_dscnt_cptl_stk where respondent_id=:id), (select count(*) from f1_edcfu_epda where respondent_id=:id), (select count(*) from f1_elctrc_erg_acct where respondent_id=:id), (select count(*) from f1_elctrc_oper_rev where respondent_id=:id), (select count(*) from f1_elc_oper_rev_nb where respondent_id=:id), (select count(*) from f1_elc_op_mnt_expn where respondent_id=:id), (select count(*) from f1_electric where respondent_id=:id), (select count(*) from f1_envrnmntl_expns where respondent_id=:id), (select count(*) from f1_envrnmntl_fclty where respondent_id=:id), (select count(*) from… | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1113/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
292011379 | MDU6SXNzdWUyOTIwMTEzNzk= | 184 | 500 from missing table name | carlmjohnson 222245 | closed | 0 | 4 | 2018-01-26T19:46:45Z | 2019-05-21T16:17:29Z | 2018-04-13T18:18:59Z | NONE | https://github.com/simonw/datasette/blob/56623e48da5412b25fb39cc26b9c743b684dd968/datasette/app.py#L517-L519 throws an error if it gets an empty list back. Simplest solution is to write a helper func that just says ```python result = list(await self.execute(name, sql, params) if result: return result[0][0] ``` and use it anywhere `[0][0]` is now. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/184/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
273595473 | MDExOlB1bGxSZXF1ZXN0MTUyMzYwNzQw | 81 | :fire: Removes DS_Store | jefftriplett 50527 | closed | 0 | 2 | 2017-11-13T22:07:52Z | 2017-11-14T02:24:54Z | 2017-11-13T22:16:55Z | CONTRIBUTOR | simonw/datasette/pulls/81 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/81/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||||
405801771 | MDExOlB1bGxSZXF1ZXN0MjQ5NjgwOTQ0 | 9 | :pencil: Updates my_database.py to my_database.db | jefftriplett 50527 | closed | 0 | 0 | 2019-02-01T17:35:43Z | 2019-02-24T03:55:04Z | 2019-02-24T03:55:04Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/9 | I noticed that both `.py` and `.db` were used in the docs and assumed you'd prefer `.db`. | sqlite-utils 140912432 | pull | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/9/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
517451234 | MDU6SXNzdWU1MTc0NTEyMzQ= | 615 | ?_col= and ?_nocol= support for toggling columns on table view | simonw 9599 | closed | 0 | 16 | 2019-11-04T22:55:41Z | 2021-05-27T04:26:10Z | 2021-05-27T04:17:44Z | OWNER | Split off from #292 (I guess this is a re-opening of #312). | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/615/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
899851083 | MDExOlB1bGxSZXF1ZXN0NjUxNDkyODg4 | 1339 | ?_col=/?_nocol= to show/hide columns on the table page | simonw 9599 | closed | 0 | 1 | 2021-05-24T17:15:20Z | 2021-05-27T04:17:44Z | 2021-05-27T04:17:43Z | OWNER | simonw/datasette/pulls/1339 | See #615. Still to do: - [x] Allow combination of `?_col=` and `?_nocol=` (`_nocol` wins) - [x] Deduplicate same column if passed in `?_col=` multiple times - [x] Validate that user did not try to remove a primary key - [x] Add tests - [x] Ensure this works correctly for SQL views - [x] Add documentation | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/1339/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
1551694938 | PR_kwDOBm6k_c5IQeKz | 1999 | ?_extra= support (draft) | simonw 9599 | closed | 0 | 49 | 2023-01-21T04:55:18Z | 2023-03-22T22:49:41Z | 2023-03-22T22:49:40Z | OWNER | simonw/datasette/pulls/1999 | Refs: - #262 <!-- readthedocs-preview datasette start --> ---- :books: Documentation preview :books:: https://datasette--1999.org.readthedocs.build/en/1999/ <!-- readthedocs-preview datasette end --> | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/1999/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
893890496 | MDU6SXNzdWU4OTM4OTA0OTY= | 1332 | ?_facet_size=X to increase number of facets results on the page | mroswell 192568 | closed | 0 | 5 | 2021-05-18T02:40:16Z | 2021-05-27T16:13:07Z | 2021-05-23T00:34:37Z | CONTRIBUTOR | Is there a way to add a parameter to the URL to modify default_facet_size? LIkewise, a way to produce a link on the three dots to expand to all items (or match previous number of items, or add x more)? | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1332/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
269731374 | MDU6SXNzdWUyNjk3MzEzNzQ= | 44 | ?_group_count=country - return counts by specific column(s) | simonw 9599 | closed | 0 | 7 | 2017-10-30T19:50:32Z | 2018-04-26T15:09:58Z | 2018-04-26T15:09:58Z | OWNER | Imagine if this: https://stateless-datasets-jykibytogk.now.sh/flights-07d1283/airports.jsono?country__contains=gu&_group_count=country Turned into this: https://stateless-datasets-jykibytogk.now.sh/flights-07d1283?sql=select%20country,%20count(*)%20as%20group_count_country%20from%20airports%20where%20country%20like%20%27%gu%%27%20group%20by%20country%20order%20by%20group_count_country%20desc This would involve introducing a new precedent of query string arguments that start with an _ having special meanings. While we're at it, could try adding _fields=x,y,z Tasks: - [x] Get initial version working - [ ] Refactor code to not just "pretend to be a view" - [ ] Get foreign key relationships expanded | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/44/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
445003029 | MDU6SXNzdWU0NDUwMDMwMjk= | 471 | ?_hash=1 and --config hash_urls:1 should only work for immutable databases | simonw 9599 | closed | 0 | 0.28 4305096 | 1 | 2019-05-16T14:54:25Z | 2019-05-16T15:11:03Z | 2019-05-16T15:11:03Z | OWNER | Split from #419. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/471/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
275089535 | MDU6SXNzdWUyNzUwODk1MzU= | 121 | ?_json=foo&_json=bar query string argument | simonw 9599 | closed | 0 | 4 | 2017-11-18T16:09:55Z | 2018-05-31T13:48:12Z | 2018-05-28T18:11:51Z | OWNER | Causes the specified columns in the output to be treated as JSON, and returned deserialized in the .json or .jsono response. This will be particularly powerful when combined with https://sqlite.org/json1.html | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/121/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
332998752 | MDExOlB1bGxSZXF1ZXN0MTk1MzM5MTEx | 311 | ?_labels=1 to expand foreign keys (in csv and json), refs #233 | simonw 9599 | closed | 0 | 2 | 2018-06-16T16:31:12Z | 2018-06-16T22:20:31Z | 2018-06-16T22:20:31Z | OWNER | simonw/datasette/pulls/311 | Output looks something like this: { "rowid": 233, "TreeID": 121240, "qLegalStatus": { "value" 2, "label": "Private" } "qSpecies": { "value": 16, "label": "Sycamore" } "qAddress": "91 Commonwealth Ave", ... } | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/311/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
904071938 | MDU6SXNzdWU5MDQwNzE5Mzg= | 1345 | ?_nocol= does not interact well with default facets | simonw 9599 | closed | 0 | 7 | 2021-05-27T18:39:55Z | 2021-05-31T02:40:44Z | 2021-05-31T02:31:21Z | OWNER | Clicking "Hide this column" on `fips` on https://covid-19.datasettes.com/covid/ny_times_us_counties shows this error: https://covid-19.datasettes.com/covid/ny_times_us_counties?_nocol=fips > ## Invalid SQL > no such column: fips The reason is that https://covid-19.datasettes.com/-/metadata sets up the following: ```json "ny_times_us_counties": { "sort_desc": "date", "facets": [ "state", "county", "fips" ], ``` It's setting `fips` as a default facet, which breaks if you attempt to remove the column using `?_nocol`. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1345/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
908446997 | MDU6SXNzdWU5MDg0NDY5OTc= | 1353 | ?_nocount=1 for opting out of table counts | simonw 9599 | closed | 0 | 2 | 2021-06-01T15:53:27Z | 2021-06-01T16:18:54Z | 2021-06-01T16:17:04Z | OWNER | Running a trace against a CSV streaming export with the new `_trace=1` feature from #1351 shows that the following code is executing a `select count(*) from table` for every page of results returned: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/table.py#L700-L705 This is inefficient - a new `?_nocount=1` option would let us disable this count in the same way as #1349: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/base.py#L264-L276 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1353/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
906977719 | MDU6SXNzdWU5MDY5Nzc3MTk= | 1350 | ?_nofacets=1 query string argument for disabling facets and suggested facets | simonw 9599 | closed | 0 | 2 | 2021-05-31T02:22:29Z | 2021-06-01T16:19:38Z | 2021-05-31T02:39:18Z | OWNER | This is needed as an internal option for #1349. `datasette-graphql` can benefit from this too - maybe can even use it so that if you pass `?_shape=array` it gets automatically added, fixing #263. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1350/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
812704869 | MDU6SXNzdWU4MTI3MDQ4Njk= | 1237 | ?_pretty=1 option for pretty-printing JSON output | simonw 9599 | open | 0 | Datasette 1.0 3268330 | 1 | 2021-02-20T20:54:40Z | 2021-11-16T18:28:33Z | OWNER | Suggested by @frankieroberto in https://github.com/simonw/datasette/issues/782#issuecomment-782746755 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1237/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
275179724 | MDU6SXNzdWUyNzUxNzk3MjQ= | 135 | ?_search=x should work if used directly against a FTS virtual table | simonw 9599 | closed | 0 | Custom templates edition 2949431 | 3 | 2017-11-19T18:17:53Z | 2017-12-07T04:54:41Z | 2017-12-07T04:54:41Z | OWNER | e.g. https://sf-trees.now.sh/sf-trees-ebc2ad9/Street_Tree_List_fts?_search=grove should work | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/135/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
426722204 | MDU6SXNzdWU0MjY3MjIyMDQ= | 423 | ?_search_col=X not reflected correctly in the UI | simonw 9599 | open | 0 | 0 | 2019-03-28T21:48:19Z | 2020-11-03T19:01:59Z | OWNER | e.g. https://latest.datasette.io/fixtures/searchable?_search_text1=barry ![2019-03-28 at 2 47 PM](https://user-images.githubusercontent.com/9599/55195035-84ebb800-5168-11e9-910b-fc9868bcd93e.png) | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/423/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
570327466 | MDExOlB1bGxSZXF1ZXN0Mzc5Mzc4Nzgw | 686 | ?_searchmode=raw option | simonw 9599 | closed | 0 | 0 | 2020-02-25T05:45:50Z | 2020-02-25T05:56:09Z | 2020-02-25T05:56:04Z | OWNER | simonw/datasette/pulls/686 | Closes #676 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/686/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
568091133 | MDU6SXNzdWU1NjgwOTExMzM= | 676 | ?_searchmode=raw option for running FTS searches without escaping characters | tunguyenatwork 58088336 | closed | 0 | 9 | 2020-02-20T06:56:57Z | 2020-02-25T05:57:24Z | 2020-02-25T05:56:04Z | NONE | After the version 0.34. I am not able to use the wildchar in the _search option( or the full text search). It will not return any result unless I specify the whole word for text search. If I use 'match :search || "*" ' in the sql statement then it will work as expected. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/676/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
610342575 | MDU6SXNzdWU2MTAzNDI1NzU= | 748 | ?_searchmode=raw should be documented on full-text search page | simonw 9599 | closed | 0 | 0 | 2020-04-30T19:50:06Z | 2020-04-30T21:06:12Z | 2020-04-30T21:06:12Z | OWNER | It's currently documented here: https://datasette.readthedocs.io/en/stable/json_api.html#special-table-arguments But it should also be described here: https://datasette.readthedocs.io/en/stable/full_text_search.html#the-table-view-api | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/748/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
319371036 | MDExOlB1bGxSZXF1ZXN0MTg1MzA3NDA3 | 246 | ?_shape=array and _timelimit= | simonw 9599 | closed | 0 | 0 | 2018-05-02T00:18:54Z | 2018-05-02T00:20:41Z | 2018-05-02T00:20:40Z | OWNER | simonw/datasette/pulls/246 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/246/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||||
319358200 | MDU6SXNzdWUzMTkzNTgyMDA= | 245 | ?_shape=array option | simonw 9599 | closed | 0 | 1 | 2018-05-01T23:11:07Z | 2018-05-03T14:14:33Z | 2018-05-02T00:12:20Z | OWNER | Some tools (`pandas.DataFrame(...)` for example) are happiest when you give them a raw array of JSON objects. `?_shape=array` should do just that While I'm at it, rename the default `?_shape=lists` to instead be called `?shape=arrays` And validate that `_shape` is a valid option And have `?_shape=object` return the object at the root level rather than nested in `.rows` to better match the behavior of `?_shape=array` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/245/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
326617744 | MDU6SXNzdWUzMjY2MTc3NDQ= | 287 | ?_shape=arrayfirst | simonw 9599 | closed | 0 | 1 | 2018-05-25T18:11:03Z | 2018-05-27T00:32:53Z | 2018-05-27T00:32:29Z | OWNER | Return an array of single items (the first item in each row returned from the SQL query). | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/287/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
788447787 | MDU6SXNzdWU3ODg0NDc3ODc= | 1194 | ?_size= argument is not persisted by hidden form fields in the table filters | simonw 9599 | closed | 0 | Datasette 0.54 6346396 | 3 | 2021-01-18T17:41:52Z | 2021-01-25T03:10:23Z | 2021-01-25T03:10:23Z | OWNER | Click "Apply" on https://covid-19.datasettes.com/covid/ny_times_us_counties?_size=1000&county__exact=San+Francisco&state__exact=California&_sort_desc=date#g.mark=line&g.x_column=date&g.x_type=temporal&g.y_column=cases&g.y_type=quantitative and the `?_size=1000` parameter from the URL will no longer apply on the reloaded page. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1194/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
792890765 | MDU6SXNzdWU3OTI4OTA3NjU= | 1200 | ?_size=10 option for the arbitrary query page would be useful | simonw 9599 | open | 0 | 2 | 2021-01-24T20:55:35Z | 2021-02-11T03:13:59Z | OWNER | https://latest.datasette.io/fixtures?sql=select+*+from+compound_three_primary_keys&_size=10 - `_size=10` does not do anything at the moment. It would be useful if it did. Would also be good if it persisted in a hidden form field. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1200/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
320090329 | MDU6SXNzdWUzMjAwOTAzMjk= | 249 | ?_size=max argument | simonw 9599 | closed | 0 | 1 | 2018-05-03T21:42:04Z | 2018-05-04T18:26:30Z | 2018-05-04T18:05:04Z | OWNER | For plugins that want to load the most data allowable, having `?_size=max` would be useful. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/249/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
995098231 | MDU6SXNzdWU5OTUwOTgyMzE= | 1470 | ?_sort=rowid with _next= returns error | eigenfoo 19851673 | closed | 0 | 4 | 2021-09-13T16:36:15Z | 2021-10-18T19:30:15Z | 2021-10-10T01:15:03Z | NONE | For example: - Go to https://cryptics.eigenfoo.xyz/clues/clues?_next=100 (this is the second page of results in a Datasette site) - Search anything using the FTS search bar. For example, searching for `hello` will take you to https://cryptics.eigenfoo.xyz/clues/clues?_search=hello&_sort=rowid&_next=100 - A `500 Error: list index out of range` is raised. This is because the search URL includes the `&_next=100` UTM parameter, carried over from where the FTS search was run. However, there isn't a second page in the search results, so a `list index out of range` error is raised. You can confirm that removing this UTM parameter from the URL returns the appropriate search results. The FTS search request should strip any `_next` UTM parameter. --- ```bash datasette, version 0.58.1 sqlite-utils, version 3.17 ``` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1470/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
1216619276 | I_kwDOBm6k_c5IhCMM | 1724 | ?_trace=1 doesn't work on Global Power Plants demo | simonw 9599 | closed | 0 | 3 | 2022-04-27T00:15:02Z | 2022-04-27T06:15:14Z | 2022-04-27T00:18:30Z | OWNER | https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_trace=1 is not showing the trace JSON at the bottom of the page. Confirmed that `trace_debug` is `true` on https://global-power-plants.datasettes.com/-/settings Possibly related: - https://github.com/simonw/datasette-total-page-time/issues/1 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1724/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
1237586379 | I_kwDOBm6k_c5JxBHL | 1742 | ?_trace=1 fails with datasette-geojson for some reason | simonw 9599 | open | 0 | 4 | 2022-05-16T19:06:05Z | 2022-05-16T19:42:13Z | OWNER | view-source:https://calands.datasettes.com/calands/CPAD_2020a_SuperUnits.geojson?_sort=id&id__exact=4&_labels=on&_trace=1 is showing me a blank page. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1742/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
326768188 | MDU6SXNzdWUzMjY3NjgxODg= | 289 | ?_ttl= parameter to control caching | simonw 9599 | closed | 0 | 3 | 2018-05-26T21:22:55Z | 2018-05-26T22:22:47Z | 2018-05-26T22:17:48Z | OWNER | This would allow clients to specify the max-age caching header that should be returned with the query. Most important this will allow caching to be completely urned off for specific queries using `?_ttl=0`. Sending 0 should cause a `Cache-Control: no-cache` header to be returned. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/289/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
432792459 | MDExOlB1bGxSZXF1ZXN0MjcwMTkxMDg0 | 430 | ?_where= parameter on table views, closes #429 | simonw 9599 | closed | 0 | 0 | 2019-04-13T01:15:09Z | 2019-04-13T01:37:23Z | 2019-04-13T01:37:23Z | OWNER | simonw/datasette/pulls/430 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/430/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||||
432636432 | MDU6SXNzdWU0MzI2MzY0MzI= | 429 | ?_where=sql-fragment parameter for table views | simonw 9599 | closed | 0 | 7 | 2019-04-12T15:58:51Z | 2019-04-15T10:48:01Z | 2019-04-13T01:37:25Z | OWNER | Only available if arbitrary SQL is enabled (the default). `?_where=id in (1,2,3)&_where=id in (select tag_id from tags)` Allows any table (or view) page to have arbitrary additional `extra_where` clauses defined using the URL! This would be extremely useful for building JavaScript applications against the Datasette API that only need on extra tiny bit of SQL but still want to benefit from other table view features like faceting. Would be nice if this could take `:named` parameters and have them filled in via querystring as well. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/429/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
433297989 | MDU6SXNzdWU0MzMyOTc5ODk= | 433 | ?column__in=value1,value2,value3 filter | simonw 9599 | closed | 0 | 0 | 2019-04-15T13:58:24Z | 2019-04-15T23:00:20Z | 2019-04-15T23:00:20Z | OWNER | Support for the SQL `where column in (...)` construct, inspired by the new design for facet configuration in #427 `?column__in=value1,value2,value3` will map to `where column in ("value1", "value2", "value3")` If comma separation won't work (because the values themselves contain commas) you can do this instead: `?column__in=["value1","value2","value3,with-comma"]` See also #288 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/433/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
657572753 | MDU6SXNzdWU2NTc1NzI3NTM= | 894 | ?sort=colname~numeric to sort by by column cast to real | simonw 9599 | open | 0 | 21 | 2020-07-15T18:47:48Z | 2021-08-20T02:07:53Z | OWNER | If a text column actually contains numbers, being able to "sort by column, treated as numeric" would be really useful. Probably depends on column actions enabled by #690 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/894/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
737855731 | MDU6SXNzdWU3Mzc4NTU3MzE= | 199 | @db.register_function(..., replace=False) to avoid double-registering custom functions | simonw 9599 | closed | 0 | 1 | 2020-11-06T15:39:21Z | 2020-11-06T18:30:44Z | 2020-11-06T18:30:44Z | OWNER | I'd like a mechanism to optionally avoid registering a custom function if it has already been registered. SQLite doesn't seem to offer a way to introspect registered custom functions so I'll need to track what has already been registered in `sqlite-utils` instead. > Should I register the custom `rank_bm25` SQLite function for every connection, or should I register it against the connection just the first time the user attempts an FTS4 search? I think I'd rather register it only if it is needed. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/198#issuecomment-723145383_ | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/199/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
1345561209 | I_kwDOBm6k_c5QM6J5 | 1790 | A better HTML title for canned query pages | simonw 9599 | open | 0 | 0 | 2022-08-21T18:27:46Z | 2022-08-21T18:27:46Z | OWNER | https://scotrail.datasette.io/scotrail/assemble_sentence?terms=This+train+is+formed+of%2Cbomb+which Current title is: `scotrail: with phrases as ( select key, value from json_each('["' || replace(:terms, ',', '","') || '"]')),matches as (select phrases.key, phrases.value, ( select File from announcements where announcements.Transcription like '%' || trim(phrases.value) || '%' order by length(announcements.Transcription) limit 1 ) as Filefrom phrases),results as ( select key, announcements.Transcription, announcements.mp3 from announcements join matches on announcements.File = matches.File order by key)select 'Combined sentence:' as mp3, group_concat(Transcription, ' ') as Transcription, -1 as keyfrom results unionselect mp3, Transcription, keyfrom resultsorder by key` I think a better title would be: `scotrail: assemble_sentence, terms = This train is formed of,bomb which` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1790/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
705995722 | MDU6SXNzdWU3MDU5OTU3MjI= | 162 | A decorator for registering custom SQL functions | simonw 9599 | closed | 0 | 2 | 2020-09-22T00:18:32Z | 2020-09-22T00:40:44Z | 2020-09-22T00:32:17Z | OWNER | Syntactic sugar for `db.conn.create_function` - it would work something like this: ```python db = sqlite_utils.Database("mydb.db") @db.register_function def scramble(text): chars = list(text) random.shuffle(chars) return "".join(chars) ``` The decorator would inspect the function to find its name and arity (number of arguments). Having run the above you could then do: ```python db.execute("select scramble('hello')").fetchall() ``` | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/162/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
794554881 | MDU6SXNzdWU3OTQ1NTQ4ODE= | 1208 | A lot of open(file) functions are used without a context manager thus producing ResourceWarning: unclosed file <_io.TextIOWrapper | kbaikov 4488943 | closed | 0 | 2 | 2021-01-26T20:56:28Z | 2021-03-11T16:15:49Z | 2021-03-11T16:15:49Z | CONTRIBUTOR | Your code is full of open files that are never closed, especially when you deal with reading/writing json/yaml files. If you run python with warnings enabled this problem becomes evident. This probably contributes to some memory leaks in long running datasettes if the GC will not 'collect' those resources properly. This is easily fixed by using a context manager instead of just using open: ```python with open('some_file', 'w') as opened_file: opened_file.write('string') ``` In some newer parts of the code you use Path objects 'read_text' and 'write_text' functions which close the file properly and are prefered in some cases. If you want I can create a PR for all places i found this pattern in. Bellow is a fraction of places where i found a ResourceWarning: ```python update-docs-help.py: 20 actual = actual.replace("Usage: cli ", "Usage: datasette ") 21: open(docs_path / filename, "w").write(actual) 22 datasette\app.py: 210 ): 211: inspect_data = json.load((config_dir / "inspect-data.json").open()) 212 if immutables is None: 266 if config_dir and (config_dir / "settings.json").exists() and not config: 267: config = json.load((config_dir / "settings.json").open()) 268 self._settings = dict(DEFAULT_SETTINGS, **(config or {})) 445 self._app_css_hash = hashlib.sha1( 446: open(os.path.join(str(app_root), "datasette/static/app.css")) 447 .read() datasette\cli.py: 130 else: 131: out = open(inspect_file, "w") 132 loop = asyncio.get_event_loop() 459 if inspect_file: 460: inspect_data = json.load(open(inspect_file)) 461 ``` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1208/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
728895233 | MDU6SXNzdWU3Mjg4OTUyMzM= | 1047 | A new section in the docs about how Datasette handles BLOB columns | simonw 9599 | closed | 0 | 0.51 6026070 | 1 | 2020-10-24T23:01:02Z | 2020-10-31T22:11:25Z | 2020-10-31T21:38:05Z | OWNER | Split from #1040, refs #1036. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1047/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
277589569 | MDU6SXNzdWUyNzc1ODk1Njk= | 155 | A primary key column that has foreign key restriction associated won't rendering label column | wsxiaoys 388154 | closed | 0 | Custom templates edition 2949431 | 4 | 2017-11-29T00:40:02Z | 2017-12-07T05:39:53Z | 2017-12-07T05:39:53Z | NONE | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/155/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
1108235694 | I_kwDOBm6k_c5CDlWu | 1603 | A proper favicon | simonw 9599 | closed | 0 | Datasette 1.0 3268330 | 19 | 2022-01-19T15:24:55Z | 2022-03-19T04:04:49Z | 2022-01-20T06:07:31Z | OWNER | Tips here: https://adamj.eu/tech/2022/01/18/how-to-add-a-favicon-to-your-django-site/ - I think a PNG served at `/favicon.ico` is the best option, since safari doesn't support SVG yet. Relevant code: https://github.com/simonw/datasette/blob/cb29119db9115b1f40de2fb45263ed77e3bfbb3e/datasette/app.py#L182-L183 I can reuse the icon for https://datasette.io/desktop | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1603/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1071531082 | I_kwDOCGYnMM4_3kRK | 349 | A way of creating indexes on newly created tables | simonw 9599 | open | 0 | 3 | 2021-12-05T18:56:12Z | 2021-12-07T01:04:37Z | OWNER | I'm writing code for https://github.com/simonw/git-history/issues/33 that creates a table inside a loop: ```python item_pk = db[item_table].lookup( {"_item_id": item_id}, item_to_insert, column_order=("_id", "_item_id"), pk="_id", ) ``` I need to look things up by `_item_id` on this table, which means I need an index on that column (the table can get very big). But there's no mechanism in SQLite utils to detect if the table was created for the first time and add an index to it. And I don't want to run `CREATE INDEX IF NOT EXISTS` every time through the loop. This should work like the `foreign_keys=` mechanism. | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/349/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
1469044738 | I_kwDOBm6k_c5Xj9gC | 1918 | API explorer should list mutable databases first | simonw 9599 | closed | 0 | Datasette 1.0a1 7867486 | 1 | 2022-11-30T04:53:33Z | 2022-11-30T05:22:07Z | 2022-11-30T05:07:56Z | OWNER | https://latest.datasette.io/-/api hides `ephemeral` down at the bottom, would be more interesting if it was at the top. Related: - #1915 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1918/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1447439985 | I_kwDOBm6k_c5WRi5x | 1888 | API explorer should take immutability into account | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 1 | 2022-11-14T06:00:14Z | 2022-11-15T19:59:10Z | 2022-11-14T06:04:48Z | OWNER | Refs: - #1871 I noticed the API explorer doesn't show any links on https://latest-1-0-dev.datasette.io/-/api because the `fixtures` database is immutable. It should still show read examples there. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1888/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1427293909 | I_kwDOBm6k_c5VEsbV | 1871 | API explorer tool | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 24 | 2022-10-28T13:49:11Z | 2022-11-15T19:59:05Z | 2022-11-14T04:59:59Z | OWNER | The API will be much easier to develop if there's a page that helps you execute JSON POSTs against it. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1871/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1426001541 | I_kwDOBm6k_c5U_w6F | 1866 | API for bulk inserting records into a table | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 12 | 2022-10-27T17:19:25Z | 2022-11-15T19:59:34Z | 2022-10-30T06:04:07Z | OWNER | Similar to https://github.com/simonw/datasette-insert/blob/0.8/README.md#inserting-data-and-creating-tables I expect this to become by far the most common way that data gets into a Datasette instance - more so than the individual row API in: - #1851 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1866/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1429030341 | I_kwDOBm6k_c5VLUXF | 1874 | API to drop a table | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 4 | 2022-10-30T21:55:11Z | 2022-11-15T19:59:53Z | 2022-11-14T05:45:06Z | OWNER | `POST /db/table/-/drop` Require `drop-table` permission. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1874/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1421544654 | I_kwDOBm6k_c5UuwzO | 1851 | API to insert a single record into an existing table | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 22 | 2022-10-24T22:24:21Z | 2022-11-15T19:59:18Z | 2022-10-28T00:59:25Z | OWNER | Controlled by a new `insert-row` permission. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1851/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issues] ( [id] INTEGER PRIMARY KEY, [node_id] TEXT, [number] INTEGER, [title] TEXT, [user] INTEGER REFERENCES [users]([id]), [state] TEXT, [locked] INTEGER, [assignee] INTEGER REFERENCES [users]([id]), [milestone] INTEGER REFERENCES [milestones]([id]), [comments] INTEGER, [created_at] TEXT, [updated_at] TEXT, [closed_at] TEXT, [author_association] TEXT, [pull_request] TEXT, [body] TEXT, [repo] INTEGER REFERENCES [repos]([id]), [type] TEXT , [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT); CREATE INDEX [idx_issues_repo] ON [issues] ([repo]); CREATE INDEX [idx_issues_milestone] ON [issues] ([milestone]); CREATE INDEX [idx_issues_assignee] ON [issues] ([assignee]); CREATE INDEX [idx_issues_user] ON [issues] ([user]);