issues
2,892 rows sorted by milestone descending
This data as json, CSV (advanced)
Suggested facets: state, assignee, author_association, repo, type, draft, state_reason, created_at (date), updated_at (date), closed_at (date)
id | node_id | number | title | user | state | locked | assignee | milestone ▲ | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1700936245 | I_kwDOCGYnMM5lYjo1 | 542 | Remove `skip_false=True` and `--no-skip-false` in `sqlite-utils` 4.0 | simonw 9599 | open | 0 | 4.0 backwards incomatible changes 9374594 | 1 | 2023-05-08T21:04:28Z | 2023-05-08T21:07:41Z | OWNER | Following: - #527 The only reason I didn't remove fix this mis-feature entirely is that it represents a backwards incompatible change. I'll make that change in 4.0. | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/542/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1649791661 | I_kwDOBm6k_c5iVdKt | 2050 | Row page JSON should use new ?_extra= format | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 1 | 2023-03-31T17:56:53Z | 2023-03-31T17:59:49Z | OWNER | https://latest.datasette.io/fixtures/facetable/2.json Related: - #2049 - #1709 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/2050/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1646734246 | I_kwDOBm6k_c5iJyum | 2049 | Custom SQL queries should use new JSON ?_extra= format | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 4 | 2023-03-30T00:42:53Z | 2023-04-05T23:29:27Z | OWNER | Related: - #262 I've made the change to the table view, now I need the new format to work for arbitrary SQL queries too. Note that this incorporates both arbitrary SQL queries and canned queries. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/2049/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1633077183 | I_kwDOBm6k_c5hVse_ | 2041 | Remove obsolete table POST code | simonw 9599 | closed | 0 | Datasette 1.0a3 8755003 | 2 | 2023-03-21T01:01:40Z | 2023-03-21T01:17:44Z | 2023-03-21T01:17:43Z | OWNER | Spotted this in: - #1999 `POST /db/table` currently executes obsolete code for inserting a row - I replaced that with `/db/table/-/insert` in https://github.com/simonw/datasette/commit/6e788b49edf4f842c0817f006eb9d865778eea5e but forgot to remove the old code. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/2041/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1495716243 | I_kwDOBm6k_c5ZJtGT | 1952 | Improvements to /-/create-token restrictions interface | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 1 | 2022-12-14T05:22:39Z | 2022-12-14T05:23:13Z | OWNER | > It would be neat not to show write permissions against immutable databases too - and not hard from a performance perspective since it doesn't involve hundreds more permission checks. > > That will need permissions to grow a flag for if they need a mutable database though, which is a bigger job. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1947#issuecomment-1350414402_ Also, DO show the `_memory` database there if Datasette was started in `--crossdb` mode. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1952/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1493471221 | I_kwDOBm6k_c5ZBI_1 | 1949 | `.json` errors should be returned as JSON | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 10 | 2022-12-13T06:14:12Z | 2022-12-15T00:46:27Z | OWNER | Eg the error in this issue: - #1945 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1949/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1490576818 | I_kwDOBm6k_c5Y2GWy | 1943 | `/-/permissions` should list available permissions | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 1 | 2022-12-11T23:38:03Z | 2022-12-15T00:41:37Z | OWNER | > Idea: a `/-/permissions` introspection endpoint for listing registered permissions _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1939#issuecomment-1345691103_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1943/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1468689139 | I_kwDOBm6k_c5Ximrz | 1914 | Finalize design of JSON for Datasette 1.0 | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 1 | 2022-11-29T20:59:10Z | 2022-12-13T06:15:54Z | OWNER | Tracking issue. - [ ] #1709 - [ ] #1729 - [ ] #1875 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1914/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1468495358 | I_kwDOBm6k_c5Xh3X- | 1910 | Check incoming column types on various write APIs | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 0 | 2022-11-29T18:09:10Z | 2022-12-13T05:29:09Z | OWNER | > I do think this needs type checking - I just tried and you really can send a string to an integer column and have it work, which feels bad. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1863#issuecomment-1331089156_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1910/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1447465004 | I_kwDOBm6k_c5WRpAs | 1889 | Ability to create new tokens via the API | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 0 | 2022-11-14T06:21:36Z | 2022-12-13T05:29:08Z | OWNER | Refs: - #1850 Initially I decided that the API shouldn't be able to create new tokens at all - I don't like the idea of an API token holder creating themselves additional tokens. Then I realized that two of the API features are specifically more useful if you can generate fresh tokens via the API: - Tokes that expire after a time limit are MUCH more useful if they can be automatically generated - Likewise, tokens that are restricted to a subset of permissions (see #1855) make more sense to be generated like this, especially in conjunction with expiry times | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1889/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1430797211 | I_kwDOBm6k_c5VSDub | 1875 | Figure out design for JSON errors (consider RFC 7807) | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 7 | 2022-11-01T03:14:15Z | 2022-12-13T05:29:08Z | OWNER | https://datatracker.ietf.org/doc/draft-ietf-httpapi-rfc7807bis/ is a brand new standard. Since I need a neat, predictable format for my JSON errors, maybe I should use this one? | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1875/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1428630253 | I_kwDOBm6k_c5VJyrt | 1873 | Ensure insert API has good tests for rowid and compound primark key tables | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 11 | 2022-10-30T06:22:17Z | 2022-12-13T05:29:08Z | OWNER | Following: - #1866 I need to design and implement various edge-cases or primary keys: - Table without an auto-incrementing primary key - Table with compound primary keys - Table with just a `rowid` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1873/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | reopened | ||||||
1426080014 | I_kwDOBm6k_c5VAEEO | 1867 | /db/table/-/rename API (also allows atomic replace) | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 1 | 2022-10-27T18:13:23Z | 2023-01-09T15:34:12Z | OWNER | > There's one catch with batched inserts: if your CLI tool fails half way through you could end up with a partially populated table - since a bunch of batches will have succeeded first. > > ... > > If people care about that kind of thing they could always push all of their inserts to a table called `_tablename` and then atomically rename that once they've uploaded all of the data (assuming I provide an atomic-rename-this-table mechanism). _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1866#issuecomment-1293893789_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1867/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1219385669 | I_kwDOBm6k_c5IrllF | 1729 | Implement ?_extra and new API design for TableView | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 12 | 2022-04-28T22:28:14Z | 2022-12-13T05:29:07Z | OWNER | Part of: - #262 - #1518 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1729/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1216436131 | I_kwDOBm6k_c5IgVej | 1721 | Implement plugin hooks: `register_table_extras`, `register_row_extras`, `register_query_extras` | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 0 | 2022-04-26T20:21:49Z | 2022-12-13T05:29:07Z | OWNER | Designed in: - #1720 Part of: - #262 - #1709 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1721/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1200650491 | I_kwDOBm6k_c5HkHj7 | 1711 | Template context powered entirely by the JSON API format | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 1 | 2022-04-11T22:59:27Z | 2022-12-13T05:29:06Z | OWNER | Datasette 1.0 will have a stable template context. I'm going to achieve this by refactoring the templates to work only with keys returned by the API (or some of its extras) - then the API documentation will double up as template documentation. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1711/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1200649502 | I_kwDOBm6k_c5HkHUe | 1709 | Redesigned JSON API with ?_extra= parameters | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 1 | 2022-04-11T22:57:49Z | 2022-12-13T05:29:06Z | OWNER | This will be the single biggest breaking change for the 1.0 release. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1709/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1200649124 | I_kwDOBm6k_c5HkHOk | 1708 | Datasette 1.0 alpha upcoming release notes | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 2 | 2022-04-11T22:57:12Z | 2022-12-13T05:29:06Z | OWNER | I'm going to try writing the release notes first, to see if that helps unblock me. # ⚠️ Any release notes in this issue are a draft, and should not be treated as the real thing ⚠️ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1708/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1197926598 | I_kwDOBm6k_c5HZujG | 1705 | How to upgrade your plugin for 1.0 documentation | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 1 | 2022-04-08T23:16:47Z | 2022-12-13T05:29:05Z | OWNER | Among other things, needed by: - #1704 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1705/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
627794879 | MDU6SXNzdWU2Mjc3OTQ4Nzk= | 782 | Redesign default .json format | simonw 9599 | open | 0 | Datasette 1.0a3 8755003 | 54 | 2020-05-30T18:47:07Z | 2023-01-17T02:05:45Z | OWNER | The default JSON just isn't right. I find myself using `?_shape=array` for almost everything I build against the API. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/782/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||
1497288666 | I_kwDOBm6k_c5ZPs_a | 1956 | Handle abbreviations properly in permission_allowed_actor_restrictions | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 2 | 2022-12-14T19:54:21Z | 2022-12-14T20:04:29Z | 2022-12-14T20:04:28Z | OWNER | This code currently assumes abbreviations are: ```pyton action_initials = "".join([word[0] for word in action.split("-")]) ``` https://github.com/simonw/datasette/blob/1a3dcf494376e32f7cff110c86a88e5b0a3f3924/datasette/default_permissions.py#L182-L208 That's no longer correct, they are now registered by the new plugin hook: - #1939 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1956/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1495821607 | I_kwDOBm6k_c5ZKG0n | 1953 | Release notes for Datasette 1.0a2 | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 2 | 2022-12-14T06:26:40Z | 2022-12-15T02:02:15Z | 2022-12-15T02:01:08Z | OWNER | https://github.com/simonw/datasette/milestone/27?closed=1 https://github.com/simonw/datasette/compare/1.0a1...9ad76d279e2c3874ca5070626a25458ce129f126 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1953/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1495431932 | I_kwDOBm6k_c5ZInr8 | 1951 | `datasette.create_token(...)` method for creating signed API tokens | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 6 | 2022-12-14T01:25:34Z | 2022-12-14T02:43:45Z | 2022-12-14T02:42:05Z | OWNER | I need this for: - #1947 And I can refactor this to use it too: - #1855 By making this a documented internal API it can be used by other plugins too. It's also going to be really useful for writing tests. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1951/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1493390939 | I_kwDOBm6k_c5ZA1Zb | 1947 | UI to create reduced scope tokens from the `/-/create-token` page | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 22 | 2022-12-13T05:10:48Z | 2022-12-14T05:22:00Z | 2022-12-14T05:13:24Z | OWNER | Split from: - #1855 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1947/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1493339206 | I_kwDOBm6k_c5ZAoxG | 1946 | `datasette --get` mechanism for sending tokens | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 2 | 2022-12-13T04:25:05Z | 2022-12-13T04:36:57Z | 2022-12-13T04:36:57Z | OWNER | > For the tests for `datasette create-token` it would be useful if `datasette --get` had a mechanism for sending an `Authorization: Bearer X` header. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1855#issuecomment-1347731288_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1946/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1493306655 | I_kwDOBm6k_c5ZAg0f | 1945 | `view-instance` should not be checked for /-/actor.json | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 0 | 2022-12-13T04:01:46Z | 2022-12-13T04:11:56Z | 2022-12-13T04:11:56Z | OWNER | Spotted this while testing: - #1855 ``` export TOKEN=$(datasette create-token root --secret s -a foo) curl -H "Authorization: Bearer $TOKEN" http://localhost:8002/-/actor.json ``` Returned a Forbidden error (and not in JSON either). | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1945/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1486011362 | PR_kwDOBm6k_c5E3XqB | 1940 | register_permissions() plugin hook | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 6 | 2022-12-09T05:09:28Z | 2022-12-13T02:05:55Z | 2022-12-13T02:05:54Z | OWNER | simonw/datasette/pulls/1940 | Refs #1939 From this comment: https://github.com/simonw/datasette/issues/1939#issuecomment-1343872168 - [x] Unit test for the registration plugin hook itself - [x] Use them in `check_permission_actions_are_documented` test in `conftest.py` - [x] Add description field to `Permissions` (and update tests and docs) - [x] Documentation for `datasette.permissions` dictionary - [x] If no `default=` provided in call to `permission_allowed()` then use default from `datasette.permissions` list - [x] Remove `default=` from a bunch of places - [x] Throw an error if two permissions are registered with the same name or abbreviation (but other attributes differ) - [x] Update authentication and permissions documentation to explain that permissions are now registered and have a registered default <!-- readthedocs-preview datasette start --> ---- :books: Documentation preview :books:: https://datasette--1940.org.readthedocs.build/en/1940/ <!-- readthedocs-preview datasette end --> | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/1940/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||
1485757511 | I_kwDOBm6k_c5YjtxH | 1939 | register_permissions(datasette) plugin hook | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 20 | 2022-12-09T01:33:25Z | 2022-12-13T02:07:50Z | 2022-12-13T02:05:56Z | OWNER | A plugin hook that adds more named permissions to the list which is initially populated here: https://github.com/simonw/datasette/blob/e539c1c024bc62d88df91d9107cbe37e7f0fe55f/datasette/permissions.py#L1-L19 Originally imagined this hook in this comment: - https://github.com/simonw/datasette/issues/1881#issuecomment-1301639370 I need this for a few reasons: - https://github.com/simonw/datasette/issues/1636 - Needs it in order to validate that permissions defined in `metadata.json` are set in the right place (don't set an instance permissions at table level for example) - https://github.com/simonw/datasette/issues/1855 - Needs it to be able to register additional abbreviations for use in signed cookies - And for validation when you use `datasette create-token` and pass in extra permissions - The https://latest.datasette.io/-/permissions debug interface needs it to add extra debug options to the `<select>` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1939/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1485488236 | PR_kwDOBm6k_c5E1iJG | 1938 | "permissions" blocks in metadata.json/yaml | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 3 | 2022-12-08T22:07:36Z | 2022-12-13T05:23:18Z | 2022-12-13T05:23:18Z | OWNER | simonw/datasette/pulls/1938 | Refs #1636 - [x] Documentation - [ ] Implementation - [ ] Validate metadata to check there are no nonsensical permissions (like `debug-menu` set at the table level) - [ ] Tests <!-- readthedocs-preview datasette start --> ---- :books: Documentation preview :books:: https://datasette--1938.org.readthedocs.build/en/1938/ <!-- readthedocs-preview datasette end --> | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/1938/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||
1483320357 | I_kwDOBm6k_c5Yaawl | 1937 | /db/-/create API should require insert-rows permission to use row: or rows: option | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 2 | 2022-12-08T01:33:09Z | 2022-12-14T20:21:26Z | 2022-12-14T20:21:26Z | OWNER | Otherwise someone with `create-table` but no` insert-rows` permission could abuse it to insert data. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1937/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1473814539 | PR_kwDOBm6k_c5EMVug | 1931 | /db/table/-/upsert | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 8 | 2022-12-03T07:01:44Z | 2022-12-08T01:12:17Z | 2022-12-08T01:12:16Z | OWNER | simonw/datasette/pulls/1931 | Refs #1878 Still todo: - [x] Support `"return": true` properly for upserts (with tests) - [x] Require both `insert-row` and `update-row` permissions - [x] Tests are going to need to cover both rowid-only and compound primary key tables, including all of the error states - [x] Documentation <!-- readthedocs-preview datasette start --> ---- :books: Documentation preview :books:: https://datasette--1931.org.readthedocs.build/en/1931/ <!-- readthedocs-preview datasette end --> | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/1931/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||
1473411197 | I_kwDOBm6k_c5X0nh9 | 1927 | ignore:true/replace:true options for /db/-/create API | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 5 | 2022-12-02T20:32:30Z | 2022-12-15T01:47:01Z | 2022-12-08T01:43:01Z | OWNER | See also: - #1924 It turns out I want to be able to call `/db/-/create` multiple times with the `rows` argument, so that I don't have to worry about creating the table first. As such I find myself wanting support for the `"insert": true` and `"replace": true` options as well. Still TODO: - [x] A test for the case where you call `/-/create` twice with `rows` without using these options - [x] `pk` should be required if you are using these options - [x] Error if you pass `pk` and the table exists already but has a different `pk` - [x] Documentation for `insert` and `replace` - and what happens if you repeat a `/-/create` with rows generally - [x] Documentation should explain that you are allowed to call `/-/create` more than once using `rows`. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1927/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1432013704 | I_kwDOBm6k_c5VWsuI | 1878 | /db/table/-/upsert API | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 8 | 2022-11-01T20:01:18Z | 2022-12-08T01:12:18Z | 2022-12-08T01:12:17Z | OWNER | Equivalent to `sqlite-utils upsert`: https://sqlite-utils.datasette.io/en/stable/python-api.html#upserting-data | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1878/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1423336089 | I_kwDOBm6k_c5U1mKZ | 1855 | `datasette create-token` ability to create tokens with a reduced set of permissions | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 19 | 2022-10-26T02:20:52Z | 2022-12-14T01:24:49Z | 2022-12-13T05:20:24Z | OWNER | Initial design ideas: https://github.com/simonw/datasette/issues/1852#issuecomment-1289733483 > Token design concept: > > ```json > { > "t": { > "a": ["ir", "ur", "dr"], > "d": { > "fixtures": ["ir", "ur", "dr"] > }, > "t": { > "fixtures": { > "searchable": ["ir"] > } > } > } > } > ``` > > That JSON would be minified and signed. > > Minified version of the above looks like this (101 characters): > > `{"t":{"a":["ir","ur","dr"],"d":{"fixtures":["ir","ur","dr"]},"t":{"fixtures":{"searchable":["ir"]}}}}` > > The `"t"` key shows this is a token that as a default API key. > > `"a"` means "all" - these are permissions that have been granted on all tables and databases. > > `"d"` means "databases" - this is a way to set permissions for all tables in a specific database. > > `"t"` means "tables" - this lets you set permissions at a finely grained table level. > > Then the permissions themselves are two character codes which are shortened versions - so: > > * `ir` = `insert-row` > * `ur` = `update-row` > * `dr` = `delete-row` ## Remaining tasks - [x] Add these options to the `datasette create-token` command - [x] Tests for `datasette create-token` options - [x] Documentation for those options at https://docs.datasette.io/en/latest/authentication.html#datasette-create-token - [x] A way to handle permissions that don't have known abbreviations (permissions added by plugins). Probably need to solve the plugin permission registration problem as part of that - [x] Stop hard-coding names of actions in the `permission_allowed_actor_restrictions` function | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1855/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1138008042 | I_kwDOBm6k_c5D1J_q | 1636 | "permissions" propery in metadata for configuring arbitrary permissions | simonw 9599 | closed | 0 | Datasette 1.0a2 8711695 | 14 | 2022-02-15T00:25:59Z | 2022-12-13T02:40:50Z | 2022-12-13T02:40:50Z | OWNER | The `"allow"` block mechanism can already be used to configure various default permissions. When adding permissions to `datasette-tiddlywiki` I realized it would be good to be able to configure arbitrary permissions such as `edit-tiddlywiki` there too. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1636/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1468603401 | I_kwDOBm6k_c5XiRwJ | 1913 | Release Datasette 1.0a0 | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 9 | 2022-11-29T19:41:42Z | 2022-11-29T20:10:35Z | 2022-11-29T20:10:35Z | OWNER | I attempted the release just now - https://github.com/simonw/datasette/releases/tag/1.0a0 - and got an unexpected test failure: https://github.com/simonw/datasette/actions/runs/3577355358/attempts/1 ``` > assert delete_response.status_code == 200 E assert 404 == 200 E + where 404 = <Response [404 Not Found]>.status_code /home/runner/work/datasette/datasette/tests/test_api_write.py:396: AssertionError =========================== short test summary info ============================ FAILED tests/test_api_write.py::test_delete_row[compound_pk_table-row_for_create2-pks2-article,k] - assert 404 == 200 + where 404 = <Response [404 Not Found]>.status_code ``` I hit "retry" on that test but I expect it to fail again. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1913/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1468519699 | I_kwDOBm6k_c5Xh9UT | 1911 | `/db/-/create` should support creating tables with compound primary keys | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 2 | 2022-11-29T18:30:47Z | 2022-11-29T18:50:58Z | 2022-11-29T18:48:05Z | OWNER | Found myself needing this to write the tests for: - #1864 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1911/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1455932972 | I_kwDOBm6k_c5Wx8Ys | 1904 | Datasette Lite tests failing due to httpx upgrade | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 0 | 2022-11-18T22:49:31Z | 2022-11-18T22:57:48Z | 2022-11-18T22:52:22Z | OWNER | Same problem as this one: - https://github.com/simonw/datasette-lite/issues/56 Caused this failure: https://github.com/simonw/datasette/actions/runs/3500765964 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1904/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1452364777 | I_kwDOBm6k_c5WkVPp | 1896 | Extract logic for resolving a URL to a database / table / row | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 4 | 2022-11-16T22:25:20Z | 2022-11-18T22:57:47Z | 2022-11-18T22:56:55Z | OWNER | > In trying to write this I realize that there's a lot of duplicated code with delete row, specifically around resolving the incoming URL into a row (or a database or a table). > > Since this is so common, I think it's worth extracting the logic out first. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1863#issuecomment-1317755263_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1896/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1450312343 | I_kwDOBm6k_c5WcgKX | 1892 | Merge 1.0-dev branch back to main | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 3 | 2022-11-15T20:04:25Z | 2022-11-29T19:40:23Z | 2022-11-29T19:40:23Z | OWNER | I'm committed enough to the 1.0 work now that I'm ready for the `main` branch to reflect that instead. If I need to make any dot-releases against 0.63 I can do those from a branch. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1892/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1450303205 | I_kwDOBm6k_c5Wcd7l | 1891 | 1.0a0 release notes | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 4 | 2022-11-15T19:58:20Z | 2022-11-29T19:23:41Z | 2022-11-29T19:23:41Z | OWNER | This release will mainly help preview the new Datasette write API: - #1850 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1891/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1447439985 | I_kwDOBm6k_c5WRi5x | 1888 | API explorer should take immutability into account | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 1 | 2022-11-14T06:00:14Z | 2022-11-15T19:59:10Z | 2022-11-14T06:04:48Z | OWNER | Refs: - #1871 I noticed the API explorer doesn't show any links on https://latest-1-0-dev.datasette.io/-/api because the `fixtures` database is immutable. It should still show read examples there. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1888/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1447388809 | I_kwDOBm6k_c5WRWaJ | 1887 | Add a confirm step to the drop table API | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 2 | 2022-11-14T04:59:53Z | 2022-11-15T19:59:59Z | 2022-11-14T05:18:51Z | OWNER | > In playing with the API explorer just now I realized it's way too easy to accidentally drop a table using it. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1871#issuecomment-1313097057_ Added drop table API in: - #1874 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1887/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1435294468 | I_kwDOBm6k_c5VjNsE | 1882 | `/db/-/create` API for creating tables | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 12 | 2022-11-03T21:44:32Z | 2022-11-15T19:59:43Z | 2022-11-15T06:00:41Z | OWNER | > It really feels like this should be accompanied by a `/db/-/create` API for creating tables. I had to add that to `sqlite-utils` eventually (initially it only supported creating by passing in an example document): > > https://sqlite-utils.datasette.io/en/stable/cli.html#cli-create-table _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1862#issuecomment-1299073433_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1882/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1429030341 | I_kwDOBm6k_c5VLUXF | 1874 | API to drop a table | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 4 | 2022-10-30T21:55:11Z | 2022-11-15T19:59:53Z | 2022-11-14T05:45:06Z | OWNER | `POST /db/table/-/drop` Require `drop-table` permission. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1874/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1427293909 | I_kwDOBm6k_c5VEsbV | 1871 | API explorer tool | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 24 | 2022-10-28T13:49:11Z | 2022-11-15T19:59:05Z | 2022-11-14T04:59:59Z | OWNER | The API will be much easier to develop if there's a page that helps you execute JSON POSTs against it. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1871/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1426195437 | I_kwDOBm6k_c5VAgPt | 1868 | Design URLs for the write API | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 5 | 2022-10-27T19:55:30Z | 2022-11-15T19:59:14Z | 2022-10-27T20:07:01Z | OWNER | My original design for this issue: - #1851 Was `POST /db/table` with JSON of `{"insert": {...}}`. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1868/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1426001541 | I_kwDOBm6k_c5U_w6F | 1866 | API for bulk inserting records into a table | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 12 | 2022-10-27T17:19:25Z | 2022-11-15T19:59:34Z | 2022-10-30T06:04:07Z | OWNER | Similar to https://github.com/simonw/datasette-insert/blob/0.8/README.md#inserting-data-and-creating-tables I expect this to become by far the most common way that data gets into a Datasette instance - more so than the individual row API in: - #1851 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1866/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1425029275 | I_kwDOBm6k_c5U8Dib | 1864 | Delete a single record from an existing table | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 4 | 2022-10-27T04:53:22Z | 2022-11-29T18:54:04Z | 2022-11-29T18:54:04Z | OWNER | API design: ``` POST /db/table/row-pks/-/delete Or... DELETE /db/table/row-pks/-/delete ``` I'm just going to do `POST` for the moment, like I did here: - #1874 Permission: `delete-row` Still needed: - [ ] Tests for rowid tables - [ ] Tests for compound primary keys | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1864/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1425029242 | I_kwDOBm6k_c5U8Dh6 | 1863 | Update a single record in an existing table | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 16 | 2022-10-27T04:53:17Z | 2022-11-29T18:08:53Z | 2022-11-29T18:06:37Z | OWNER | API design: ``` POST /db/table/row-pks/-/update { "field": "updated_value" } ``` Only the fields that you pass will be updated. Maybe this is the wrong design though? The design for insert currently looks like this: - https://github.com/simonw/datasette/issues/1851#issuecomment-1294224185 ``` POST /db/table/-/insert Authorization: Bearer xxx Content-Type: application/json { "row": { "id": 1, "name": "New name" } } ``` I could use the same format for `/-/update`, but in this case the API doesn't require you to pass every field so `"row"` doesn't seem like the right key. I think I'll go with this: ``` POST /db/table/1/-/update Authorization: Bearer xxx Content-Type: application/json { "update": { "name": "New name" } } ``` The benefit of having an `"update"` key is that it allows me to use other keys in the future. Maybe a `"alter": true` key to indicate that new columns should be added if they are missing. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1863/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1425011030 | I_kwDOBm6k_c5U7_FW | 1862 | Create a new table from one or more records, `sqlite-utils` style | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 5 | 2022-10-27T04:25:02Z | 2022-11-15T19:59:47Z | 2022-11-15T06:42:09Z | OWNER | It's interesting to also think about what the form-based UI for this could look like - since that would involve users creating new columns of different types on the fly. Will need the `create-table` permission. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1862/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1423369494 | I_kwDOBm6k_c5U1uUW | 1859 | datasette create-token CLI command | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 3 | 2022-10-26T03:12:59Z | 2022-11-15T19:59:00Z | 2022-10-26T04:31:39Z | OWNER | The CLI equivalent of the `/-/create-token` page. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1859/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1423364990 | I_kwDOBm6k_c5U1tN- | 1858 | `max_signed_tokens_ttl` setting for a maximum duration on API tokens | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 4 | 2022-10-26T03:05:53Z | 2022-11-15T19:58:52Z | 2022-10-27T03:15:05Z | OWNER | It's currently possible to use `/-/create-token` to create a token that lasts forever. Some administrators may wish to have a maximum expiry instead. I should support that with a setting. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1858/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1423347412 | I_kwDOBm6k_c5U1o7U | 1857 | Prevent API tokens from using /-/create-token to create more tokens | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 1 | 2022-10-26T02:38:09Z | 2022-11-15T19:57:11Z | 2022-10-26T02:57:26Z | OWNER | > It strikes me that users should NOT be able to use a token to create additional tokens. > > The current design actually does allow that, since the `dstok_` Bearer token can be used to authenticate calls to the `/-/create-token` page. > > So I think I need a mechanism whereby that page can only allow access to users authenticated by cookie. > > Not obvious how to do that though, since Datasette's authentication actor system is designed to abstract that detail away! _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1850#issuecomment-1291417100_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1857/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1423336122 | I_kwDOBm6k_c5U1mK6 | 1856 | allow_signed_tokens setting for disabling API signed token mechanism | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 3 | 2022-10-26T02:20:55Z | 2022-11-15T19:57:05Z | 2022-10-26T02:58:35Z | OWNER | Had some design thoughts here: https://github.com/simonw/datasette/issues/1852#issuecomment-1291272280 I liked this option the most: --setting allow_create_tokens off | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1856/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1421552095 | I_kwDOBm6k_c5Uuynf | 1852 | Default API token authentication mechanism | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 30 | 2022-10-24T22:31:07Z | 2022-11-15T19:57:00Z | 2022-10-26T02:19:54Z | OWNER | API authentication will be via `Authorization: Bearer XXX` request headers. I'm inclined to add a default token mechanism to Datasette based on tokens that are signed with the `DATASETTE_SECRET`. Maybe the root user can access `/-/create-token` which provides a UI for generating a time-limited signed token? Could also have a `datasette token` command for creating such tokens at the command-line. Plugins can then define alternative ways of creating tokens, such as the existing https://datasette.io/plugins/datasette-auth-tokens plugin. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1850#issuecomment-1289706439_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1852/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1421544654 | I_kwDOBm6k_c5UuwzO | 1851 | API to insert a single record into an existing table | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 22 | 2022-10-24T22:24:21Z | 2022-11-15T19:59:18Z | 2022-10-28T00:59:25Z | OWNER | Controlled by a new `insert-row` permission. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1851/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1421529723 | I_kwDOBm6k_c5UutJ7 | 1850 | Write API in Datasette core | simonw 9599 | closed | 0 | Datasette 1.0a0 8658075 | 13 | 2022-10-24T22:13:24Z | 2022-11-29T20:11:20Z | 2022-11-29T20:11:20Z | OWNER | I need this for Datasette Cloud, and in thinking it through I realized that it's really time Datasette grew a default write API as well. I'm going to mostly model this off `sqlite-utils`, since I've spent a bunch of time iterating on a pseudo-JSON API for that over the past few years (piping JSON to stdin etc). I want this for Datasette 1.0. I'm going to be building it in the new [1.0-dev](https://github.com/simonw/datasette/tree/1.0-dev) branch, which is automatically deployed to https://latest-1-0-dev.datasette.io/ running on Cloud Run. API features to build: - [x] #1852 - [x] #1856 - [x] #1857 - [x] #1858 - [x] #1859 - [x] #1871 - [x] #1888 - [x] #1868 - [x] #1851 - [x] #1863 - [x] #1864 - [x] #1866 - [x] https://github.com/simonw/datasette/issues/1882 - [x] #1862 - [x] #1874 - [x] https://github.com/simonw/datasette/issues/1887 - [x] #1877 Bumped to later on: - #1855 - #1878 - #1873 - #1875 - Make sure CORS works - https://github.com/simonw/datasette/issues/1889 - Alter a table - `sqlite-utils transform` style (more powerful than straight ALTER) - Execute SQL against a write connection - Maybe even multiple write SQL statements bundled in a single transaction - https://github.com/simonw/datasette/issues/1867 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1850/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1353196970 | I_kwDOCGYnMM5QqCWq | 476 | Release notes for 3.29 | simonw 9599 | closed | 0 | 3.29 8355157 | 2 | 2022-08-27T23:21:21Z | 2022-08-28T04:07:15Z | 2022-08-28T04:07:03Z | OWNER | https://github.com/simonw/sqlite-utils/compare/3.28...104f37fa4d2e7e5999c1d829267b62c737f74d3e | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/476/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1353189941 | I_kwDOCGYnMM5QqAo1 | 475 | table.default_values introspection property | simonw 9599 | closed | 0 | 3.29 8355157 | 1 | 2022-08-27T22:33:31Z | 2022-08-27T22:44:46Z | 2022-08-27T22:43:02Z | OWNER | > Interesting challenge with `default_value`: I need to be able to tell if the default values passed to `.create()` differ from those in the database already. > > Introspecting that is a bit tricky: > > ```pycon > >>> import sqlite_utils > >>> db = sqlite_utils.Database(memory=True) > >>> db["blah"].create({"id": int, "name": str}, not_null=("name",), defaults={"name": "bob"}) > <Table blah (id, name)> > >>> db["blah"].columns > [Column(cid=0, name='id', type='INTEGER', notnull=0, default_value=None, is_pk=0), Column(cid=1, name='name', type='TEXT', notnull=1, default_value="'bob'", is_pk=0)] > ``` > Note how a default value of the Python string `bob` is represented in the results of `PRAGMA table_info()` as `default_value="'bob'"` - it's got single quotes added to it! > > So comparing default values from introspecting the database needs me to first parse that syntax. This may require a new table introspection method. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/468#issuecomment-1229279539_ | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/475/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1352946135 | I_kwDOCGYnMM5QpFHX | 472 | Reuse the locals/globals fix from --functions for other code accepting options | simonw 9599 | closed | 0 | 3.29 8355157 | 2 | 2022-08-27T05:12:05Z | 2022-08-27T05:20:12Z | 2022-08-27T05:20:12Z | OWNER | I figured out a workaround for the ugly `global x` hack here: - https://github.com/simonw/sqlite-utils/issues/471#issuecomment-1229120653 | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/472/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1352932716 | I_kwDOCGYnMM5QpB1s | 471 | sqlite-utils query --functions mechanism for registering extra functions | simonw 9599 | closed | 0 | 3.29 8355157 | 12 | 2022-08-27T03:57:53Z | 2022-09-07T03:46:26Z | 2022-08-27T05:10:57Z | OWNER | It would be really cool if you could register additional custom SQL functions for use with the `sqlite-utils query` command - something like this: ``` sqlite-utils data.db 'update images set domain = extract_domain(url)' --functions ' from urllib.parse import urlparse def extract_domain(url): return urlparse(url).netloc ' ``` Every function defined in that code block would be registered with the connection, unless the name began with an underscore. | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/471/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1352932038 | I_kwDOCGYnMM5QpBrG | 470 | Upgrade `--load-extension` to accept entrypoints like Datasette | simonw 9599 | closed | 0 | 3.29 8355157 | 6 | 2022-08-27T03:53:20Z | 2022-08-27T05:55:49Z | 2022-08-27T05:55:48Z | OWNER | Imitate: - https://github.com/simonw/datasette/pull/1789 ``` # would load default entrypoint like before datasette data.db --load-extension ext # loads the extensions with the "sqlite3_foo_init" entrpoint datasette data.db --load-extension ext:sqlite3_foo_init # loads the extensions with the "sqlite3_bar_init" entrpoint datasette data.db --load-extension ext:sqlite3_bar_init ``` | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/470/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1352931464 | I_kwDOCGYnMM5QpBiI | 469 | sqlite-utils rows --order option | simonw 9599 | closed | 0 | 3.29 8355157 | 1 | 2022-08-27T03:49:51Z | 2022-08-27T04:30:49Z | 2022-08-27T04:10:32Z | OWNER | For consistency with `search`: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#search ``` -o, --order TEXT Order by ('column' or 'column desc') ``` I wanted to run `sqlite-utils rows db.db mytable --order 'rowid desc'` to see the most recently imported rows. | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/469/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1348294436 | PR_kwDOCGYnMM49qP2V | 468 | db[table].create(..., transform=True) and create-table --transform | simonw 9599 | closed | 0 | 3.29 8355157 | 6 | 2022-08-23T17:27:58Z | 2022-08-27T23:17:55Z | 2022-08-27T23:17:55Z | OWNER | simonw/sqlite-utils/pulls/468 | Work in progress. Still needs documentation and tests (and to cover more cases of things that might have changed). Refs: - #467 <!-- readthedocs-preview sqlite-utils start --> ---- :books: Documentation preview :books:: https://sqlite-utils--468.org.readthedocs.build/en/468/ <!-- readthedocs-preview sqlite-utils end --> | sqlite-utils 140912432 | pull | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/468/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||
1348169997 | I_kwDOCGYnMM5QW3EN | 467 | Mechanism for ensuring a table has all the columns | simonw 9599 | closed | 0 | 3.29 8355157 | 13 | 2022-08-23T15:50:23Z | 2022-08-27T23:19:41Z | 2022-08-27T23:17:56Z | OWNER | Suggested by @jefftriplett on Discord: https://discord.com/channels/823971286308356157/997738192360964156/1011655389063958600 | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/467/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1320243134 | I_kwDOCGYnMM5OsU-- | 458 | Support custom names for registered functions | simonw 9599 | closed | 0 | 3.29 8355157 | 1 | 2022-07-28T00:13:00Z | 2022-08-27T03:56:01Z | 2022-07-28T00:13:57Z | OWNER | In this example: ```python @db.register_function def reverse_string(s): return "".join(reversed(list(s))) print(db.execute('select reverse_string("hello")').fetchone()[0]) ``` There's currently no way to over-ride the automatically selected name for the SQL function. | sqlite-utils 140912432 | issue | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/458/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1319881016 | PR_kwDOCGYnMM48Mmde | 457 | Link to installation instructions | simonw 9599 | closed | 0 | 3.29 8355157 | 2 | 2022-07-27T17:38:36Z | 2022-08-27T03:55:52Z | 2022-07-27T17:57:50Z | OWNER | simonw/sqlite-utils/pulls/457 | Also testing https://docs.readthedocs.io/en/stable/pull-requests.html | sqlite-utils 140912432 | pull | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/457/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||
1338278056 | I_kwDOBm6k_c5PxICo | 1782 | Release notes for Datasette 0.62 | simonw 9599 | closed | 0 | Datasette 0.62 8303187 | 2 | 2022-08-14T15:26:45Z | 2022-08-14T17:40:45Z | 2022-08-14T17:32:54Z | OWNER | I've written a lot of these already for the alphas: - https://github.com/simonw/datasette/releases/tag/0.62a0 - https://github.com/simonw/datasette/releases/tag/0.62a1 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1782/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1338137350 | I_kwDOBm6k_c5PwlsG | 1781 | Ensure Datasette Lite is promoted in docs and README | simonw 9599 | closed | 0 | Datasette 0.62 8303187 | 1 | 2022-08-14T05:12:35Z | 2022-08-14T15:24:40Z | 2022-08-14T15:24:40Z | OWNER | As of 0.62 https://lite.datasette.io is a supported piece of the overall Datasette ecosystem. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1781/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1334628400 | I_kwDOBm6k_c5PjNAw | 1779 | google cloudrun updated their limits on maxscale based on memory and cpu count | fgregg 536941 | closed | 0 | Datasette 0.62 8303187 | 13 | 2022-08-10T13:27:21Z | 2022-08-14T19:42:59Z | 2022-08-14T17:07:34Z | CONTRIBUTOR | if you don't set an explicit limit on container scaling, then [google defaults to 100](https://cloud.google.com/run/docs/configuring/max-instances#limits) google recently updated the [limits on container scaling](https://cloud.google.com/run/docs/configuring/max-instances#limits), such that if you set up datasette to use more memory or cpu, then you need to set the maxScale argument much smaller than 100. would be nice if `datasette publish` could do this math for you and set the right maxScale. [Log of an failing publish run](https://github.com/labordata/warehouse/runs/7764725972?check_suite_focus=true#step:8:332). ``` ERROR: (gcloud.run.deploy) spec.template.spec.containers[0].resources.limits.cpu: Invalid value specified for cpu. For the specified value, maxScale may not exceed 15. Consider running your workload in a region with greater capacity, decreasing your requested cpu-per-instance, or requesting an increase in quota for this region if you are seeing sustained usage near this limit, see https://cloud.google.com/run/quotas. Your project may gain access to further scaling by adding billing information to your account. Traceback (most recent call last): File "/home/runner/.local/bin/datasette", line 8, in <module> sys.exit(cli()) File "/home/runner/.local/lib/python3.8/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/home/runner/.local/lib/python3.8/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/home/runner/.local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/runner/.local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/runner/.local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/runner/.local/lib/python3.8/site-packages/… | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1779/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1318907685 | I_kwDOBm6k_c5OnO8l | 1773 | 500 error if sorted by a column not in the ?_col= list | simonw 9599 | closed | 0 | Datasette 0.62 8303187 | 4 | 2022-07-27T01:20:27Z | 2022-08-14T16:06:25Z | 2022-08-14T15:44:05Z | OWNER | For example: https://latest.datasette.io/fixtures/sortable?_sort_desc=sortable&_col=sortable_with_nulls That's `?_sort_desc=sortable&_col=sortable_with_nulls` <img width="494" alt="image" src="https://user-images.githubusercontent.com/9599/181139719-06e3bf66-3fde-4f9a-bfd1-62b326121601.png"> | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1773/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1306492437 | I_kwDOBm6k_c5N334V | 1770 | `handle_exception` plugin hook for custom error handling | simonw 9599 | closed | 0 | Datasette 0.62 8303187 | 14 | 2022-07-15T20:52:49Z | 2022-08-14T15:25:51Z | 2022-08-14T15:25:51Z | OWNER | I need this for a couple of plugins, both of which are broken at the moment: - https://github.com/simonw/datasette-sentry/issues/1 - https://github.com/simonw/datasette-show-errors/issues/2 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1770/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1296222572 | I_kwDOBm6k_c5NQsls | 1768 | Upgrade to 3.10.6-slim-bullseye Docker base image | simonw 9599 | closed | 0 | Datasette 0.62 8303187 | 5 | 2022-07-06T18:37:49Z | 2022-08-14T15:54:36Z | 2022-08-14T15:54:11Z | OWNER | For the package published to Docker Hub and also the containers used by `datasette package` and `datasette publish cloudrun`. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1768/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1223527226 | I_kwDOBm6k_c5I7Ys6 | 1738 | "Cannot use _sort and _sort_desc at the same time" | simonw 9599 | closed | 0 | Datasette 0.62 8303187 | 2 | 2022-05-03T01:06:24Z | 2022-08-14T16:13:55Z | 2022-08-14T16:13:55Z | OWNER | Triggered this error while playing with the sort desc checkbox and the apply button that are only visible on this page at mobile screen width: https://latest.datasette.io/fixtures/compound_three_primary_keys?_sort_desc=pk1 Navigate to that page (with the browser narrow enough to show the box), un-check the box and click Apply: ![sort-bug](https://user-images.githubusercontent.com/9599/166390804-cb289b29-63dc-4986-b7f9-81cf2ae04914.gif) Also notable: I managed to get to a page with `?_sort_desk=pk1` in the URL three times by clicking around with that button. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1738/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1218133366 | I_kwDOBm6k_c5Imz12 | 1728 | Writable canned queries fail with useless non-error against immutable databases | wragge 127565 | closed | 0 | Datasette 0.62 8303187 | 13 | 2022-04-28T03:10:34Z | 2022-08-14T16:34:40Z | 2022-08-14T16:34:40Z | CONTRIBUTOR | I've been banging my head against a wall for a while and would appreciate any pointers... - I have a writeable canned query to update rows in the db. - I'm using the github-oauth plugin for authentication. - I have `allow` set on the query to accept my GitHub id and a GH organisation. - Authentication seems to work as expected both locally and on Cloudrun -- viewing `/-/actor` gives the same result in both environments - I can access the 'padlocked' canned query in both environments. Everything seems to be the same, but the canned query works perfectly when run locally, and fails when I try it on Cloudrun. I'm redirected back to the canned query page and the db is not changed. There's nothing in the Cloudstor logs to indicate an error. Any clues as to where I should be looking for the problem? | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1728/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1471969984 | I_kwDOBm6k_c5XvHrA | 1926 | Release notes for 1.0a1 (and release it) | simonw 9599 | closed | 0 | Datasette 1.0a1 7867486 | 1 | 2022-12-01T21:18:12Z | 2022-12-01T22:06:13Z | 2022-12-01T22:06:12Z | OWNER | Mainly CORS support and a few small bug fixes. Changes: https://github.com/simonw/datasette/compare/1.0a0...99da46f7258225fc6fd8e94ddc20859ccccc4109 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1926/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1470509936 | I_kwDOBm6k_c5XpjNw | 1924 | Docs for replace:true and ignore:true options for insert API | simonw 9599 | closed | 0 | Datasette 1.0a1 7867486 | 4 | 2022-12-01T01:33:25Z | 2022-12-01T18:15:15Z | 2022-12-01T02:08:02Z | OWNER | Equivalent to https://sqlite-utils.datasette.io/en/stable/cli.html#insert-replacing-data | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1924/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1469973742 | I_kwDOBm6k_c5XngTu | 1922 | Make sure CORS works for write APIs | simonw 9599 | closed | 0 | Datasette 1.0a1 7867486 | 13 | 2022-11-30T17:15:55Z | 2022-12-01T18:47:00Z | 2022-12-01T18:47:00Z | OWNER | Split from: - #1850 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1922/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1469044738 | I_kwDOBm6k_c5Xj9gC | 1918 | API explorer should list mutable databases first | simonw 9599 | closed | 0 | Datasette 1.0a1 7867486 | 1 | 2022-11-30T04:53:33Z | 2022-11-30T05:22:07Z | 2022-11-30T05:07:56Z | OWNER | https://latest.datasette.io/-/api hides `ephemeral` down at the bottom, would be more interesting if it was at the top. Related: - #1915 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1918/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1469043836 | I_kwDOBm6k_c5Xj9R8 | 1917 | Don't allow writable API to edit the `_memory` database | simonw 9599 | closed | 0 | Datasette 1.0a1 7867486 | 2 | 2022-11-30T04:51:59Z | 2022-11-30T05:07:56Z | 2022-11-30T05:07:55Z | OWNER | It shows up on https://latest.datasette.io/-/api (once you are signed in as root) - but there's no point in creating tables in it because they likely won't persist from one request to the next, as it's not a shared named database. <img width="1122" alt="image" src="https://user-images.githubusercontent.com/9599/204710529-bfab8442-d89b-4928-9b9a-e39bc53c1673.png"> | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1917/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1469015001 | I_kwDOBm6k_c5Xj2PZ | 1916 | GET requests against POST endpoints should not 500 error | simonw 9599 | closed | 0 | Datasette 1.0a1 7867486 | 1 | 2022-11-30T04:04:43Z | 2022-11-30T05:15:19Z | 2022-11-30T05:15:19Z | OWNER | ![CF37BA4D-0677-4DDD-A339-EAF163BB63B7](https://user-images.githubusercontent.com/9599/204705025-6f88e9f7-757d-45e8-a89c-ab97e84781e8.png) | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1916/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1102484126 | I_kwDOBm6k_c5BtpKe | 1595 | Release notes for 0.60 | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 4 | 2022-01-13T22:23:14Z | 2022-01-14T01:37:39Z | 2022-01-14T01:37:39Z | OWNER | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1595/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
1102359726 | I_kwDOBm6k_c5BtKyu | 1594 | Add a CLI reference page to the docs, inspired by sqlite-utils | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 3 | 2022-01-13T20:55:08Z | 2022-01-13T22:28:22Z | 2022-01-13T21:38:48Z | OWNER | Thought of this while posting this comment: https://github.com/simonw/datasette/issues/1591#issuecomment-1012506595 I added https://sqlite-utils.datasette.io/en/stable/cli-reference.html to `sqlite-utils` in https://github.com/simonw/sqlite-utils/issues/383 and I _really_ like it - it's a page showing the `--help` output of every CLI command for that tool. It's maintained using `cog`. One of the benefits is that I get a free commit history of changes to `--help` at https://github.com/simonw/sqlite-utils/commits/main/docs/cli-reference.rst | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1594/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1099723916 | I_kwDOBm6k_c5BjHSM | 1590 | Table+query JSON and CSV links broken when using `base_url` setting | eelkevdbos 1001306 | closed | 0 | Datasette 0.60 7571612 | 11 | 2022-01-11T23:46:39Z | 2022-01-14T01:16:34Z | 2022-01-14T01:16:08Z | NONE | Datasette appends the prefix found in the `base_url` setting twice if a `base_url` is set. In the follow asgi example, I'm hosting a custom Datasette instance: ```python # asgi.py import pathlib from asgi_cors import asgi_cors from channels.routing import URLRouter from django.urls import re_path from datasette.app import Datasette datasette_ = Datasette( files=[], settings={ "base_url": "/datasettes/", "plugins": {} }, config_dir=pathlib.Path('.'), ) application = URLRouter([ re_path(r"^datasettes/.*", asgi_cors(datasette_.app(), allow_all=True)), ]) ``` Running it with: ```shell $ daphne -p 8002 asgi:application ``` Using a simple query on the `_memory` table: ```sql select sqlite_version() ``` http://localhost:8002/datasettes/_memory?sql=select+sqlite_version%28%29 It renders the following upon inspection: ![image](https://user-images.githubusercontent.com/1001306/149038851-aa842950-126a-467c-9a86-fae13bce6221.png) I am using datasette version `0.59.4` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1590/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1097101917 | I_kwDOBm6k_c5BZHJd | 1588 | `explain query plan select` is too strict about whitespace | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 3 | 2022-01-09T04:22:42Z | 2022-01-13T22:28:19Z | 2022-01-13T20:35:05Z | OWNER | `explain query plan select * from facetable` is allowed: https://latest.datasette.io/fixtures?sql=explain+query+plan+select+*+from+facetable But... `explain query plan select * from facetable` (with two spaces before the `select`) returns a "Statement must be a SELECT" error: https://latest.datasette.io/fixtures?sql=explain+query+plan++select+*+from+facetable | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1588/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1087931918 | I_kwDOBm6k_c5A2IYO | 1579 | `.execute_write(... block=True)` should be the default behaviour | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 7 | 2021-12-23T18:54:28Z | 2022-01-13T22:28:08Z | 2021-12-23T19:18:26Z | OWNER | Every single piece of code I've written against the write APIs has used the `block=True` option to wait for the result. Without that, it instead fires the write into the queue but then continues even before it has finished executing. `block=True` should clearly be the default behaviour here! | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1579/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1084007781 | I_kwDOBm6k_c5AnKVl | 1572 | "Query took" should be "Queries took" | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 0 | 2021-12-19T04:03:00Z | 2022-01-13T22:27:43Z | 2021-12-19T04:03:24Z | OWNER | This is misleading, since usually there have been more than one query executed: ![CleanShot 2021-12-18 at 20 02 35@2x](https://user-images.githubusercontent.com/9599/146663457-9c4c2900-5cc0-4650-a565-bb1ff0b8a725.png) | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1572/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1083927147 | I_kwDOBm6k_c5Am2pr | 1571 | Track number of executions for execute_write_many() in traces | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 0 | 2021-12-18T19:16:17Z | 2022-01-13T22:27:49Z | 2021-12-19T20:30:40Z | OWNER | Spotted while working on #1555 <img width="1171" alt="image" src="https://user-images.githubusercontent.com/9599/146653166-ce5358bc-b822-4961-a2dd-ed3efaa8dd13.png"> There's no indication there of how many times `execute_write_many()` executed the SQL. Solving this is a tiny bit tricky because `params_seq` is an iterator that we don't want to exhaust before passing it to `conn.executemany()` - so we need to instead wrap it in something that counts how many times it was called. But then we need a way to attach that to the trace here: https://github.com/simonw/datasette/blob/d637ed46762fdbbd8e32b86f258cd9a53c1cfdc7/datasette/database.py#L115-L122 So probably need to redesign the `trace()` decorator to allow extra pairs to be attached to it within the `with` statement. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1571/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1083921371 | I_kwDOBm6k_c5Am1Pb | 1570 | Separate db.execute_write() into three methods | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 2 | 2021-12-18T18:45:54Z | 2022-01-13T22:27:38Z | 2021-12-18T18:57:25Z | OWNER | > Rather than adding a `executemany=True` parameter, I'm now thinking a better design might be to have three methods: > > - `db.execute_write(sql, params=None, block=False)` > - `db.execute_write_script(sql, block=False)` > - `db.execute_write_many(sql, params_seq, block=False)` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997267416_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1570/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1083895395 | I_kwDOBm6k_c5Amu5j | 1569 | db.execute_write(..., executescript=True) parameter | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 2 | 2021-12-18T18:20:47Z | 2022-01-13T22:27:27Z | 2021-12-18T18:34:18Z | OWNER | > Idea: teach `execute_write` to accept an optional `executescript=True` parameter, like this: ```diff diff --git a/datasette/database.py b/datasette/database.py index 468e936..1a424f5 100644 --- a/datasette/database.py +++ b/datasette/database.py @@ -94,10 +94,14 @@ class Database: f"file:{self.path}{qs}", uri=True, check_same_thread=False ) - async def execute_write(self, sql, params=None, block=False): + async def execute_write(self, sql, params=None, executescript=False, block=False): + assert not executescript and params, "Cannot use params with executescript=True" def _inner(conn): with conn: - return conn.execute(sql, params or []) + if executescript: + return conn.executescript(sql) + else: + return conn.execute(sql, params or []) with trace("sql", database=self.name, sql=sql.strip(), params=params): results = await self.execute_write_fn(_inner, block=block) ``` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997248364_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1569/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1083726550 | I_kwDOBm6k_c5AmFrW | 1568 | Trace should show queries on the write connection too | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 2 | 2021-12-18T02:34:12Z | 2022-01-13T22:27:23Z | 2021-12-18T02:42:34Z | OWNER | > Here's why - `trace` only applies to read, not write SQL operations: https://github.com/simonw/datasette/blob/7c8f8aa209e4ba7bf83976f8495d67c28fbfca24/datasette/database.py#L209-L211 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997128508_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1568/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1083718998 | I_kwDOBm6k_c5AmD1W | 1567 | Remove undocumented sqlite_functions mechanism | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 0 | 2021-12-18T01:51:10Z | 2022-01-13T22:27:04Z | 2021-12-18T01:54:46Z | OWNER | I added this in 0b8c1b0a6da9cb8ac0d28cc90dd783de87554036 but it's never been documented and the same thing can now be achieved using the `prepare_connection` plugin hook. https://github.com/simonw/datasette/blob/0c91e59d2bbfc08884cfcf5d1b902a2f4968b7ff/datasette/app.py#L262 https://github.com/simonw/datasette/blob/0c91e59d2bbfc08884cfcf5d1b902a2f4968b7ff/datasette/app.py#L551-L552 It's used here in the tests: https://github.com/simonw/datasette/blob/69244a617b1118dcbd04a8f102173f04680cf08c/tests/fixtures.py#L156 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1567/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1083669410 | I_kwDOBm6k_c5Al3ui | 1566 | Release Datasette 0.60 | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 6 | 2021-12-17T22:58:12Z | 2022-01-14T01:59:55Z | 2022-01-14T01:59:55Z | OWNER | Using this as a tracking issue. I'm hoping to get the bulk of the JSON redesign work from the refactor in #1554 in for this release. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1566/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1083581011 | I_kwDOBm6k_c5AliJT | 1564 | _prepare_connection not called on write connections | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 1 | 2021-12-17T20:06:47Z | 2022-01-20T21:29:43Z | 2021-12-18T01:58:44Z | OWNER | I was trying to initalize SpatiaLite in a write connection: ```pycon >>> from datasette.app import Datasette >>> ds = Datasette(memory=True, files=[], sqlite_extensions=["spatialite"]) >>> db = ds.add_memory_database('geo') >>> await db.execute_write("select InitSpatialMetadata(1)") UUID('3f143baa-4e3d-5842-a36f-4fa2f683b72f') no such function: InitSpatialMetadata ``` It looks like the code that loads additional modules only works on read-only connections, not on write connections: https://github.com/simonw/datasette/blob/92a5280d2e75c39424a75ad6226fc74400ae984f/datasette/database.py#L146-L153 Compared to: https://github.com/simonw/datasette/blob/92a5280d2e75c39424a75ad6226fc74400ae984f/datasette/database.py#L124-L132 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1564/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1083573206 | I_kwDOBm6k_c5AlgPW | 1563 | Datasette(... files=) should not be a required argument | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 2 | 2021-12-17T19:54:18Z | 2022-01-13T22:27:18Z | 2021-12-18T02:19:40Z | OWNER | ```pycon >>> ds = Datasette(memory=True) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: __init__() missing 1 required positional argument: 'files' >>> ds = Datasette(memory=True, files=[]) ``` I wanted to create an in-memory Datasette for running some tests, no point in forcing me to pass `files=[]` to do that. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1563/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1082746149 | I_kwDOBm6k_c5AiWUl | 1560 | Table page title has "where where" in it | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 0 | 2021-12-17T00:05:48Z | 2022-01-13T22:28:35Z | 2022-01-13T22:20:15Z | OWNER | Just noticed this while working on #1518. ``` % curl -s 'https://latest.datasette.io/fixtures/facetable?_sort=pk&on_earth__exact=1' | grep -C 1 '<title>' <head> <title>fixtures: facetable: 14 rows where where on_earth = 1 sorted by pk</title> ``` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1560/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1082564912 | I_kwDOBm6k_c5AhqEw | 1557 | `?_nosuggest=1` parameter for disabling facet suggestions on table view | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 1 | 2021-12-16T19:21:42Z | 2022-01-13T22:26:48Z | 2021-12-16T19:24:59Z | OWNER | Found I wanted this while I was debugging #625 just to clean up the debug traces, but it makes sense as a partner to `?_nofacet=1` and `?_nocount=1` from #1350 and #1353. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1557/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||
1081318247 | I_kwDOBm6k_c5Ac5tn | 1556 | Show count of facet values always, not just for `?_facet_size=max` | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 1 | 2021-12-15T17:49:01Z | 2022-01-13T22:26:07Z | 2021-12-15T17:58:06Z | OWNER | > You've caused me to rethink this feature - I no longer think there's value in only showing these numbers if `?_facet_size=max` as opposed to all of the time. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1423#issuecomment-995023410_ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1556/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0} | completed | |||||
1079149656 | I_kwDOBm6k_c5AUoRY | 1555 | Optimize all those calls to index_list and foreign_key_list | simonw 9599 | closed | 0 | Datasette 0.60 7571612 | 27 | 2021-12-13T23:50:56Z | 2022-01-13T22:27:32Z | 2021-12-19T20:55:59Z | OWNER | On the first hit to a restarted index I'm seeing this in the SQL traces: https://latest-with-plugins.datasette.io/github/commits?_trace=1 <img width="826" alt="image" src="https://user-images.githubusercontent.com/9599/145907011-e4abc93c-8dbf-468d-afef-13d5e7463f92.png"> I imagine this could be sped up a lot using tricks like this one from the SQLite documentation: https://sqlite.org/pragma.html#pragfunc ```sql SELECT DISTINCT m.name || '.' || ii.name AS 'indexed-columns' FROM sqlite_schema AS m, pragma_index_list(m.name) AS il, pragma_index_info(il.name) AS ii WHERE m.type='table' ORDER BY 1; ``` https://latest-with-plugins.datasette.io/fixtures?sql=SELECT+DISTINCT+m.name+%7C%7C+%27.%27+%7C%7C+ii.name+AS+%27indexed-columns%27%0D%0A++FROM+sqlite_schema+AS+m%2C%0D%0A+++++++pragma_index_list%28m.name%29+AS+il%2C%0D%0A+++++++pragma_index_info%28il.name%29+AS+ii%0D%0A+WHERE+m.type%3D%27table%27%0D%0A+ORDER+BY+1%3B | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/1555/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issues] ( [id] INTEGER PRIMARY KEY, [node_id] TEXT, [number] INTEGER, [title] TEXT, [user] INTEGER REFERENCES [users]([id]), [state] TEXT, [locked] INTEGER, [assignee] INTEGER REFERENCES [users]([id]), [milestone] INTEGER REFERENCES [milestones]([id]), [comments] INTEGER, [created_at] TEXT, [updated_at] TEXT, [closed_at] TEXT, [author_association] TEXT, [pull_request] TEXT, [body] TEXT, [repo] INTEGER REFERENCES [repos]([id]), [type] TEXT , [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT); CREATE INDEX [idx_issues_repo] ON [issues] ([repo]); CREATE INDEX [idx_issues_milestone] ON [issues] ([milestone]); CREATE INDEX [idx_issues_assignee] ON [issues] ([assignee]); CREATE INDEX [idx_issues_user] ON [issues] ([user]);