issues
1,991 rows where user = 9599 sorted by milestone
This data as json, CSV (advanced)
id | node_id | number | title | user | state | locked | assignee | milestone ▼ | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
315960272 | MDU6SXNzdWUzMTU5NjAyNzI= | 227 | prepare_context() plugin hook | simonw 9599 | closed | 0 | 8 | 2018-04-19T16:55:26Z | 2020-03-24T22:19:54Z | 2020-03-24T22:19:54Z | OWNER | This would be called with the context dictionary before each template is rendered. It would have the opportunity to modify that context. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/227/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
316031566 | MDU6SXNzdWUzMTYwMzE1NjY= | 228 | If spatialite detected, mark idx_XXX_Geometry tables as hidden | simonw 9599 | closed | 0 | 1 | 2018-04-19T20:37:24Z | 2018-04-26T03:25:39Z | 2018-04-26T03:25:39Z | OWNER | https://timezones-api.now.sh/timezones-faf26d0 ![2018-04-19 at 1 36 pm](https://user-images.githubusercontent.com/9599/39016906-a5acbb3e-43d6-11e8-9a31-814ff1d0022e.png) Need to update this logic: https://github.com/simonw/datasette/blob/e2750c7cc0585adaa8c866be611089e62961ee35/datasette/app.py#L1276-L1288 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/228/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
316123256 | MDU6SXNzdWUzMTYxMjMyNTY= | 229 | Table view should support ?_size=400 parameter | simonw 9599 | closed | 0 | 1 | 2018-04-20T04:23:18Z | 2018-04-26T04:49:46Z | 2018-04-26T04:48:32Z | OWNER | Allows callers to request more rows at once. The limit will still be `max_returned_rows` (defaults to 1000). | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/229/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
316128955 | MDU6SXNzdWUzMTYxMjg5NTU= | 230 | Setting page size AND max returned rows to 1000 doesn't seem to work | simonw 9599 | closed | 0 | 1 | 2018-04-20T05:05:11Z | 2018-04-26T04:04:25Z | 2018-04-26T04:04:25Z | OWNER | It appears that if the two settings are the same Datasette fails to return any results, probably because of the trick where we try to fetch 1001 rows so we know if there's a next page. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/230/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
316323336 | MDU6SXNzdWUzMTYzMjMzMzY= | 231 | metadata.json support for plugin configuration options | simonw 9599 | closed | 0 | 4 | 2018-04-20T15:58:47Z | 2019-05-13T18:56:21Z | 2019-05-13T18:56:21Z | OWNER | My [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map) plugin currently works by detecting `latitude` and `longitude` columns. I'd like to be able to configure it to look for different column names. One way to do this could be to support optional plugin configuration as part of `metadata.json`. Something like this: { "title": "Polar Bear Ear Tags, 2009-2011", "source": "USGS Alaska Science Center, Polar Bear Research Program", "source_url": "https://alaska.usgs.gov/products/data.php?dataid=130", "plugins": { "datasette_cluster_map": { "latitude_columns": [ "latitude", "Capture Latitude" ], "longitude_columns": [ "longitude", "Capture Longitude" ] } } } These settings should be supported at the root level or at the individual database or table level. They could also be exposed in the https://datasette-cluster-map-demo.now.sh/-/plugins debug tool. Refs #14 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/231/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
316444720 | MDU6SXNzdWUzMTY0NDQ3MjA= | 233 | Option to expose expanded foreign keys in JSON/CSV | simonw 9599 | closed | 0 | 11 | 2018-04-21T00:18:25Z | 2018-06-16T22:26:21Z | 2018-06-16T22:20:14Z | OWNER | https://datasette-cluster-map-demo.datasettes.com/sf-trees-02c8ef1/Street_Tree_List?qCareAssistant=1 ![f36b87c0-478e-4d55-9a5f-ad37df0b47cb](https://user-images.githubusercontent.com/9599/39078411-bb3e4f88-44be-11e8-9d0c-d22324793c77.png) It would be nice if the info bubbles there could expose more than just the IDs, and if the title showed the expanded name of the selected qCareAssistant. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/233/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
316526433 | MDU6SXNzdWUzMTY1MjY0MzM= | 234 | label_column option in metadata.json | simonw 9599 | closed | 0 | 3 | 2018-04-21T21:19:08Z | 2018-04-22T20:47:12Z | 2018-04-22T20:47:12Z | OWNER | Currently the column used for displaying a foreign key relationship is automatically detected by `inspect()` by looking for tables that have a primary key column and one other column. This doesn't work for tables with more than two columns. Let's allow the table section in `metadata.json` to optionally define a `label_column` which, if present, will be used for those displays. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/234/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
316621102 | MDU6SXNzdWUzMTY2MjExMDI= | 235 | Add limit on the size in KB of data returned from a single query | simonw 9599 | open | 0 | 2 | 2018-04-22T23:01:15Z | 2018-04-24T00:30:02Z | OWNER | Datasette limits the number of rows returned to 1,000 and limits the time spent executing a SQL query to 1000ms - and both of these limits can be customized. It does not have a limit on the size of the response returned. It's possible to compose maliciously large SQL responses in a small number of rows using mechanisms like the `group_concat()` aggregate function. It would be good to avoid malicious SQL creating 100MB+ responses and potentially crashing the server. I think the easiest place to implement that is here: https://github.com/simonw/datasette/blob/f3f42957128c1e7ece584d45d9167f2ac003a3b8/datasette/app.py#L175-L190 Currently we use `cursor.fetchmany()` to fetch up to 1,001 rows at once. Instead, we could switch to iterating through `cursor.fetchone()` (or just using `for row in cursor`) and keeping a running tally of the size of the response as we go - maybe just using `rough_response_size += len(str(row))`. If that goes above a certain threshold we can terminate the response with an error, like we do with timelimits. The bigger challenge here is understanding how well this approach works and what impact it will have on overall Datasette performance. I think I need #33 for this. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/235/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
317001500 | MDU6SXNzdWUzMTcwMDE1MDA= | 236 | datasette publish lambda plugin | simonw 9599 | open | 0 | 11 | 2018-04-23T22:10:30Z | 2023-03-12T14:04:15Z | OWNER | Refs #217 - create a publish plugin that can deploy to AWS Lambda. https://docs.aws.amazon.com/lambda/latest/dg/limits.html says lambda packages can be up to 50 MB, so this would only work with smaller databases (the command can check the filesize before attempting to package and deploy it). Lambdas do get a 512 MB `/tmp` directory too, so for larger databases the function could start and then download up to 512MB from an S3 bucket - so the plugin could take an optional S3 bucket to write to and know how to upload the `.db` file there and then have the lambda download it on startup. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/236/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
317475156 | MDU6SXNzdWUzMTc0NzUxNTY= | 237 | Support for ?_search_colname=blah searches | simonw 9599 | closed | 0 | 2 | 2018-04-25T04:29:53Z | 2018-05-05T22:56:42Z | 2018-05-05T22:33:23Z | OWNER | Right now the `_search=` argument searches across all fields in a full-text index, for example: https://san-francisco.datasettes.com/sf-film-locations-84594a7/Film_Locations_in_San_Francisco?_search=justin SQLite FTS also supports searches within a specified field, for example: https://san-francisco.datasettes.com/sf-film-locations-84594a7?sql=select+rowid%2C+*+from+Film_Locations_in_San_Francisco+where+rowid+in+%28select+rowid+from+%5BFilm_Locations_in_San_Francisco_fts%5D+where+%5BLocations%5D+match+%3Asearch%29+order+by+rowid+limit+101&search=justin ``` select rowid, * from Film_Locations_in_San_Francisco where rowid in ( select rowid from [Film_Locations_in_San_Francisco_fts] where [Locations] match :search ) order by rowid limit 101 ``` The `_search=` parameter could be extended to support this using `_search_colname=`. This should also be able to support columns with spaces and special characters in their names, something like this: `_search_Column%20With%20Spaces=foo` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/237/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
317714268 | MDU6SXNzdWUzMTc3MTQyNjg= | 238 | External metadata.json | simonw 9599 | closed | 0 | 3 | 2018-04-25T17:02:30Z | 2019-06-24T06:52:55Z | 2019-06-24T06:52:45Z | OWNER | A frustration I'm having with https://register-of-members-interests.datasettes.com/ is that I keep coming up with new canned queries but I don't want to redeploy the whole thing just to add them to `metadata.json` Maybe Datasette could optionally take a `--metadata-url` option which causes it to load from a URL instead and occasionally check for updates. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/238/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
317760361 | MDU6SXNzdWUzMTc3NjAzNjE= | 239 | Support for hidden tables in metadata.json | simonw 9599 | closed | 0 | 2 | 2018-04-25T19:21:17Z | 2018-04-26T03:45:12Z | 2018-04-26T03:43:10Z | OWNER | Since we already have a hidden feature, let's expose it more to our users | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/239/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
317900587 | MDU6SXNzdWUzMTc5MDA1ODc= | 240 | FTS table detection should be part of .inspect() | simonw 9599 | closed | 0 | 0 | 2018-04-26T06:58:10Z | 2018-04-29T00:04:44Z | 2018-04-29T00:04:44Z | OWNER | The code that detects if specific tables have a corresponding FTS column is currently called from TableView - it should instead be handled as part of `.inspect()`. This will make it easier to build other features that need to behave differently depending on whether a table can be searched, e.g. an autocomplete widget for selecting filters from foreign key tables. Current code: https://github.com/simonw/datasette/blob/f188ceaa2a3a5b2eab83425ad0f00cb0d364e24a/datasette/app.py#L728-L733 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/240/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
318490133 | MDU6SXNzdWUzMTg0OTAxMzM= | 241 | Default datasette logging format should be JSON | simonw 9599 | open | 0 | 0 | 2018-04-27T17:32:48Z | 2018-07-10T17:45:40Z | OWNER | Structured logs are better. Datasette should default to outputting it's HTTP access log lines as newline delimited JSON instead of the Sanic default format it uses at the moment. For improved greppability these logs should have keys ordered in a consistent way. Python's JSON module can do this with ordered dictionaries. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/241/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
318692953 | MDU6SXNzdWUzMTg2OTI5NTM= | 242 | Rename ?_sql_time_limit_ms= to ?_timelimit= | simonw 9599 | closed | 0 | 0 | 2018-04-29T06:11:35Z | 2018-05-02T00:20:42Z | 2018-05-02T00:20:42Z | OWNER | It's a bit of a mouthful at the moment. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/242/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
318737808 | MDU6SXNzdWUzMTg3Mzc4MDg= | 243 | --spatialite option for datasette publish commands | simonw 9599 | closed | 0 | 2 | 2018-04-29T18:19:32Z | 2018-05-31T14:17:53Z | 2018-05-31T14:17:53Z | OWNER | Performs the necessary incantations to install Spatialite on Zeit Now or Heroku and sets the corresponding environment variable to ensure the module is correctly loaded by datasette serve. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/243/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
318738000 | MDU6SXNzdWUzMTg3MzgwMDA= | 244 | /-/versions page | simonw 9599 | closed | 0 | 1 | 2018-04-29T18:22:15Z | 2018-05-03T14:13:49Z | 2018-05-03T14:09:53Z | OWNER | Displays the current version of: * datasette * Python * SQLite * Spatialite (if available) Installed plugin versions should be shown on /-/plugins | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/244/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
319358200 | MDU6SXNzdWUzMTkzNTgyMDA= | 245 | ?_shape=array option | simonw 9599 | closed | 0 | 1 | 2018-05-01T23:11:07Z | 2018-05-03T14:14:33Z | 2018-05-02T00:12:20Z | OWNER | Some tools (`pandas.DataFrame(...)` for example) are happiest when you give them a raw array of JSON objects. `?_shape=array` should do just that While I'm at it, rename the default `?_shape=lists` to instead be called `?shape=arrays` And validate that `_shape` is a valid option And have `?_shape=object` return the object at the root level rather than nested in `.rows` to better match the behavior of `?_shape=array` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/245/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
319371036 | MDExOlB1bGxSZXF1ZXN0MTg1MzA3NDA3 | 246 | ?_shape=array and _timelimit= | simonw 9599 | closed | 0 | 0 | 2018-05-02T00:18:54Z | 2018-05-02T00:20:41Z | 2018-05-02T00:20:40Z | OWNER | simonw/datasette/pulls/246 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/246/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||||
319954545 | MDU6SXNzdWUzMTk5NTQ1NDU= | 248 | /-/plugins should show version of each installed plugin | simonw 9599 | closed | 0 | 2 | 2018-05-03T14:50:45Z | 2018-05-04T18:25:40Z | 2018-05-04T18:05:04Z | OWNER | Refs #244 https://stackoverflow.com/questions/20180543/how-to-check-version-of-python-modules ``` >>> import pkg_resources >>> pkg_resources.get_distribution('datasette_cluster_map').version '0.4' ``` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/248/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
320090329 | MDU6SXNzdWUzMjAwOTAzMjk= | 249 | ?_size=max argument | simonw 9599 | closed | 0 | 1 | 2018-05-03T21:42:04Z | 2018-05-04T18:26:30Z | 2018-05-04T18:05:04Z | OWNER | For plugins that want to load the most data allowable, having `?_size=max` would be useful. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/249/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
320132682 | MDU6SXNzdWUzMjAxMzI2ODI= | 250 | Setup some issue templates | simonw 9599 | open | 0 | 0 | 2018-05-04T01:49:07Z | 2018-05-04T01:49:07Z | OWNER | https://twitter.com/left_pad/status/99216385740464537 I like the idea of using these to help people understand some of the ways I want to use issues. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/250/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
320592643 | MDU6SXNzdWUzMjA1OTI2NDM= | 251 | Explore "distinct values for column" in inspect() | simonw 9599 | closed | 0 | 4 | 2018-05-06T13:27:24Z | 2018-05-14T22:47:55Z | 2018-05-14T22:47:55Z | OWNER | A lot of datasets have columns which have a small number of possible values in them - this one for example: https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+distinct+category+from+%5Binconvenient-sequel%2Fratings%5D%3B Detecting these could be interesting as part of `.inspect()`, since it would allow for various UI enhancements like autocomplete / select box filters for those columns. The problem is detecting them efficiently. `.inspect()` shouldn't spend 5 minutes churning through columns on giant tables trying to determine if they have a small collection of unique values. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/251/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
321624016 | MDU6SXNzdWUzMjE2MjQwMTY= | 252 | /-/versions should report the FTS version supported by SQLite | simonw 9599 | closed | 0 | 0 | 2018-05-09T15:43:47Z | 2018-05-11T13:19:52Z | 2018-05-11T13:19:52Z | OWNER | I can copy this function from `csvs-to-sqlite`: https://github.com/simonw/csvs-to-sqlite/blob/dccbf65b37bc9eed50e9edb80a42f257e93edb1f/csvs_to_sqlite/utils.py#L283-L293 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/252/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
321631020 | MDU6SXNzdWUzMjE2MzEwMjA= | 253 | Documentation explaining how to use SQLite FTS with Datasette | simonw 9599 | closed | 0 | 1 | 2018-05-09T16:02:08Z | 2018-05-12T12:09:02Z | 2018-05-12T12:06:51Z | OWNER | In particular how to work with https://www.sqlite.org/fts3.html#_external_content_fts4_tables_ - which Datasette can automatically detect and use to add a search UI to your page. Examples of basic search setup like this: ``` CREATE VIRTUAL TABLE "interests_fts" USING FTS4 (name, content="interests"); INSERT INTO "interests_fts" (rowid, name) SELECT rowid, name FROM interests; ``` And complex join-based search setup like this: ``` CREATE VIRTUAL TABLE "interests_fts" USING FTS4 (name, category, member, content="interests"); INSERT INTO "interests_fts" (rowid, name, category, member) SELECT interests.rowid, interests.name, interest_categories.name, members.name FROM interests JOIN interest_categories ON interests.category_id = interest_categories.id JOIN members ON interests.member_id = members.id; ``` Also mention how `csvs-to-sqlite` can be used to do this easily. This will benefit from #252 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/253/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
322477187 | MDU6SXNzdWUzMjI0NzcxODc= | 255 | Facets | simonw 9599 | closed | 0 | 16 | 2018-05-12T03:00:07Z | 2019-05-29T21:39:12Z | 2018-05-16T15:32:12Z | OWNER | Ability to display facets and facet counts on the table view. Facets can be specified in the URL with `?_facet=column&_facet=othercolumn` or the default facets for a table can be set using a new `"facets": [...]` property in `metadata.json` - [x] Implement `?_facet=` - [x] Implement `metadata.json` `facets` key - [x] Design for how facets should be presented - [x] Facets should be able to toggle off as well as on - [x] Expand labels for facets that are foreign keys - [x] Suggest potential facets (if we can do so within a tight time limit) - [x] Documentation | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/255/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
322551723 | MDU6SXNzdWUzMjI1NTE3MjM= | 256 | Break up app.py into separate view modules | simonw 9599 | closed | 0 | 1 | 2018-05-12T23:56:33Z | 2018-05-14T03:05:37Z | 2018-05-14T03:05:37Z | OWNER | `views/table.py` and `views/database.py` and `views/utils.py` as a starting point. Likewise, create `tests/test_views_table.py` and `tests/test_views_database.py` - these will contain both HTML and API test for those views. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/256/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
322591993 | MDExOlB1bGxSZXF1ZXN0MTg3NjY4ODkw | 257 | Refactor views | simonw 9599 | closed | 0 | 5 | 2018-05-13T13:00:50Z | 2018-05-14T03:04:25Z | 2018-05-14T03:04:24Z | OWNER | simonw/datasette/pulls/257 | * Split out view classes from main `app.py` * Run [black](https://github.com/ambv/black) against resulting code to apply opinionated source code formatting * Run [isort](https://github.com/timothycrosley/isort) to re-order my imports Refs #256 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/257/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
322787470 | MDU6SXNzdWUzMjI3ODc0NzA= | 259 | inspect() should detect many-to-many relationships | simonw 9599 | closed | 0 | 6 | 2018-05-14T12:03:58Z | 2019-05-23T03:55:37Z | 2019-05-23T03:55:37Z | OWNER | Relates to #255 - in particular supporting facets across M2M relationships. It should be possible for `.inspect()` to notice when a table has two foreign keys to two different tables, and assume that this means there is a M2M relationship between those tables. When rendering a table with a m2m relationship we could display the first X associated records as a comma separated list of hyperlinks in a new column on the table view, with a column name derived from the table on the other side. Since SQLite doesn't have RANK or an equivalent of https://www.xaprb.com/blog/2006/12/02/how-to-number-rows-in-mysql/ this would be implemented as N+1 queries (one query per cell that we want to display an m2m summary). This should be OK in SQLite: https://sqlite.org/np1queryprob.html | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/259/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
323223872 | MDU6SXNzdWUzMjMyMjM4NzI= | 260 | Validate metadata.json on startup | simonw 9599 | open | 0 | 7 | 2018-05-15T13:42:56Z | 2023-06-21T12:51:22Z | OWNER | It's easy to misspell the name of a database or table and then be puzzled when the metadata settings silently fail. To avoid this, let's sanity check the provided metadata.json on startup and quit with a useful error message if we find any obvious mistakes. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/260/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
323459939 | MDExOlB1bGxSZXF1ZXN0MTg4MzEyNDEx | 261 | Facets improvements plus suggested facets | simonw 9599 | closed | 0 | 0 | 2018-05-16T03:52:39Z | 2018-05-16T15:27:26Z | 2018-05-16T15:27:25Z | OWNER | simonw/datasette/pulls/261 | Refs #255 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/261/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
323671577 | MDU6SXNzdWUzMjM2NzE1Nzc= | 263 | Facets should not execute for ?shape=array|object | simonw 9599 | closed | 0 | 3 | 2018-05-16T15:26:13Z | 2021-06-02T02:54:34Z | 2021-06-02T02:54:34Z | OWNER | Split off from #255 - there's no point executing the facet SQL for the `?_shape=array` and `?_shape=object` API responses. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/263/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
323673899 | MDU6SXNzdWUzMjM2NzM4OTk= | 264 | Make it possible to customize various facet settings | simonw 9599 | closed | 0 | 1 | 2018-05-16T15:31:34Z | 2018-05-18T06:18:00Z | 2018-05-18T05:11:52Z | OWNER | The new Facets implementation from #255 includes several hard-coded settings which should be made configurable somehow: Number of rows to return in a facet (maybe this should also be an option that can be set via quersytring argument, e.g. `?_facet=qSpecies:40`): https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L539 Time limit for executing a facet: https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L559-L562 Maximum unique values returned in order for a column to be suggested as a facet: https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L646-L647 Time limit for calculating if a column should be a suggested facet: https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L664-L667 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/264/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
323677499 | MDU6SXNzdWUzMjM2Nzc0OTk= | 265 | Add links to example Datasette instances to appropiate places in docs | simonw 9599 | closed | 0 | 5 | 2018-05-16T15:40:20Z | 2018-06-18T15:52:15Z | 2018-06-18T15:52:15Z | OWNER | Links to working examples would really help, especially on these pages: * http://datasette.readthedocs.io/en/latest/json_api.html * http://datasette.readthedocs.io/en/latest/sql_queries.html * http://datasette.readthedocs.io/en/latest/facets.html * http://datasette.readthedocs.io/en/latest/full_text_search.html | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/265/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
323681589 | MDU6SXNzdWUzMjM2ODE1ODk= | 266 | Export to CSV | simonw 9599 | closed | 0 | 27 | 2018-05-16T15:50:24Z | 2021-06-17T18:14:24Z | 2018-06-18T06:05:25Z | OWNER | Datasette needs to be able to export data to CSV. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/266/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
323716411 | MDU6SXNzdWUzMjM3MTY0MTE= | 267 | Documentation for URL hashing, redirects and cache policy | simonw 9599 | closed | 0 | 3 | 2018-05-16T17:29:01Z | 2019-06-24T06:41:02Z | 2019-06-24T06:41:02Z | OWNER | See my comments on #258 for a starting point | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/267/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
323718842 | MDU6SXNzdWUzMjM3MTg4NDI= | 268 | Mechanism for ranking results from SQLite full-text search | simonw 9599 | open | 0 | 12 | 2018-05-16T17:36:40Z | 2022-01-13T22:21:28Z | OWNER | This isn't particularly straight-forward - all the more reason for Datasette to implement it for you. This article is helpful: http://charlesleifer.com/blog/using-sqlite-full-text-search-with-python/ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/268/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
323726888 | MDU6SXNzdWUzMjM3MjY4ODg= | 269 | If a facet fails due to timing out, let the user know somehow | simonw 9599 | closed | 0 | 0 | 2018-05-16T18:01:47Z | 2018-05-18T06:11:46Z | 2018-05-18T06:11:46Z | OWNER | Refs #255 - right now facets fail silently if the user requested them but they take longer than 200ms to calculate - see also #264 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/269/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
323830051 | MDU6SXNzdWUzMjM4MzAwNTE= | 270 | --limit= CLI option for setting limits | simonw 9599 | closed | 0 | 1 | 2018-05-17T00:14:24Z | 2018-05-18T06:19:31Z | 2018-05-18T06:16:39Z | OWNER | #264 calls for four new datasette limit options, on top of the two existing ones: * `--max_returned_rows` * `--sql_time_limit_ms` These are already clogging up `datasette serve --help` a bit. How about this syntax instead? datasette --limit max_returned_rows:100 \ --limit facet_timeout_ms:500 demo.db Then we can add as many new user over-rideable limits as we like without clogging up `--help` too much - though it would be good to have a way of optionally listings their documentation as well. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/270/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
324162476 | MDU6SXNzdWUzMjQxNjI0NzY= | 271 | Mechanism for automatically picking up changes when on-disk .db file changes | simonw 9599 | closed | 0 | 4 | 2018-05-17T19:53:15Z | 2019-01-10T21:35:18Z | 2019-01-10T21:35:18Z | OWNER | It would be useful if Datasette could spot when a SQLite database file changes on disk and restart itself (hence re-running .inspect() and picking up the new content hash). Ideally this could happen in an atomic way so no requests get dropped during the switch-over. This may not play well with SQLite opening databases in immutable mode. Research required. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/271/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
324451322 | MDU6SXNzdWUzMjQ0NTEzMjI= | 273 | Figure out a way to have /-/version return current git commit hash | simonw 9599 | closed | 0 | 2 | 2018-05-18T15:16:56Z | 2018-05-22T19:35:22Z | 2018-05-22T19:35:22Z | OWNER | https://fivethirtyeight.datasettes.com/-/versions reports Datasette version `0.21` This isn't actually correct. The deploy script for that site actually deploys current master using `https://github.com/simonw/datasette/archive/master.zip`: https://github.com/simonw/fivethirtyeight-datasette/blob/66b4b0dfedd7237bc8c02d3e26d905bca7b84069/Dockerfile#L9 Ideally this would show the current commit hash, but I'm not at all sure if it's possible to derive that from `pip install https://github.com/simonw/datasette/archive/master.zip`. Is there another mechanism that could be used to reliably `pip install` current master but still provide access to the most recent commit hash? | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/273/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
324652142 | MDU6SXNzdWUzMjQ2NTIxNDI= | 274 | Rename --limit to --config, add --help-config | simonw 9599 | closed | 0 | 2 | 2018-05-19T18:57:42Z | 2018-05-20T17:04:55Z | 2018-05-20T17:04:11Z | OWNER | #270 introduced `--limit` but on further thought it should be called `--config` instead. `--page_size` should becomes `--config default_page_size:1000` Add `--help-config` to show full help showing all config settings. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/274/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
324720095 | MDU6SXNzdWUzMjQ3MjAwOTU= | 275 | "config" section in metadata.json (root, database and table level) | simonw 9599 | open | 0 | 2 | 2018-05-20T16:02:28Z | 2019-05-13T18:33:31Z | OWNER | Split off from #274 Metadata should an optional `"config"` section at root, table or database level. The TableView and RowView and DatabaseView and BaseView classes could all have a `.config("key")` method which knows how to resolve the hierarchy of configs. This will allow individual tables (or databases) to set their own config settings for things like `sql_time_limit_ms` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/275/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
325294102 | MDU6SXNzdWUzMjUyOTQxMDI= | 278 | Build smallest possible Docker image with Datasette plus recent SQLite (with json1) plus Spatialite 4.4.0 | simonw 9599 | closed | 0 | 3 | 2018-05-22T13:28:40Z | 2018-05-23T17:43:36Z | 2018-05-23T17:43:36Z | OWNER | A Dockerfile that does the following: * Bundles Datasette master * Python 3.6 most recent version (or 3.7 if it has been released) * SQLite 3.23.1 (or most recent release) such that "import sqite3" in Python gets that version. Ideally with the json1 module baked in by default, but having it loadable as an optional module is fine too * SpatiaLite 4.4.0-RC0 (or most recent version) such that it can be loaded as an optional module * Uses multi-stage builds to stay as small as possible Note that the current "release" of SpatiaLite is 4.3.0 which is missing key features like https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN - 4.4.0 probably needs to be compiled from source. I don't know the best way to get a current SQLite version bundled for Python 3. Maybe https://github.com/coleifer/pysqlite3 ? | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/278/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
325705981 | MDU6SXNzdWUzMjU3MDU5ODE= | 282 | Faceting breaks pagination | simonw 9599 | closed | 0 | 1 | 2018-05-23T13:29:47Z | 2018-05-23T13:53:39Z | 2018-05-23T13:42:07Z | OWNER | e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/nba-elo%2Fnbaallelo?_facet=lg_id#facet-lg_id - click the "next page" link: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/nba-elo%2Fnbaallelo?_facet=lg_id&_next=100 Invalid SQL: near "and": syntax error | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/282/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
325958506 | MDU6SXNzdWUzMjU5NTg1MDY= | 283 | Support cross-database joins | simonw 9599 | closed | 0 | 26 | 2018-05-24T04:18:39Z | 2021-06-06T09:40:18Z | 2021-02-18T22:16:46Z | OWNER | SQLite has the ability to attach multiple databases to a single connection and then run joins across multiple databases. Since Datasette supports more than one database, this would make a pretty neat feature. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/283/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
326182814 | MDU6SXNzdWUzMjYxODI4MTQ= | 284 | Ability to enable/disable specific features via --config | simonw 9599 | closed | 0 | 5 | 2018-05-24T15:47:56Z | 2018-05-25T06:05:02Z | 2018-05-25T05:51:09Z | OWNER | `--config` settings from #274 can currently only be integers. I'd like them to be available as boooeans too. Then we can use them to have that are turned on by default but can be turned off. First features to get this treatment: - [x] `allow_sql` - whether or not the `?sql=` parameter is allowed and form is displayed - [X] `allow_facet` - is `?_facet=` allowed or do we only run facets defined in `metadata.json` - [X] `allow_download` - do we let users download the full SQLite database file? - [X] `suggest_facets` - do we attempt to calculate suggested facets? Refs #275 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/284/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
326189744 | MDU6SXNzdWUzMjYxODk3NDQ= | 285 | num_threads and cache_max_age should be --config options | simonw 9599 | closed | 0 | 2 | 2018-05-24T16:04:51Z | 2018-05-27T00:53:35Z | 2018-05-27T00:43:33Z | OWNER | https://github.com/simonw/datasette/blob/58b5a37dbbf13868a46bcbb284509434e66eca25/datasette/app.py#L106 And https://github.com/simonw/datasette/blob/58b5a37dbbf13868a46bcbb284509434e66eca25/datasette/views/base.py#L325 Refs #275 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/285/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
326599525 | MDU6SXNzdWUzMjY1OTk1MjU= | 286 | Database hash should include current datasette version | simonw 9599 | open | 0 | 2 | 2018-05-25T17:03:42Z | 2018-05-25T17:07:36Z | OWNER | Right now deploying a new version of datasette doesn't invalidate existing URLs, so users may still see a cached copy of the old templates. We can fix this by including the current datasette version in the input to the hash function (which currently just the database file contents). | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/286/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
326617744 | MDU6SXNzdWUzMjY2MTc3NDQ= | 287 | ?_shape=arrayfirst | simonw 9599 | closed | 0 | 1 | 2018-05-25T18:11:03Z | 2018-05-27T00:32:53Z | 2018-05-27T00:32:29Z | OWNER | Return an array of single items (the first item in each row returned from the SQL query). | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/287/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
326767626 | MDU6SXNzdWUzMjY3Njc2MjY= | 288 | Support multiple filters of the same type | simonw 9599 | closed | 0 | 3 | 2018-05-26T21:13:12Z | 2019-04-15T23:45:04Z | 2019-04-15T23:44:26Z | OWNER | This should work for example: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/biopics%2Fbiopics?year_release__not=2014&year_release__not=2015 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/288/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
326768188 | MDU6SXNzdWUzMjY3NjgxODg= | 289 | ?_ttl= parameter to control caching | simonw 9599 | closed | 0 | 3 | 2018-05-26T21:22:55Z | 2018-05-26T22:22:47Z | 2018-05-26T22:17:48Z | OWNER | This would allow clients to specify the max-age caching header that should be returned with the query. Most important this will allow caching to be completely urned off for specific queries using `?_ttl=0`. Sending 0 should cause a `Cache-Control: no-cache` header to be returned. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/289/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
326778161 | MDU6SXNzdWUzMjY3NzgxNjE= | 290 | Consider increasing the default for num_sql_threads (currently 3) | simonw 9599 | open | 0 | 0 | 2018-05-27T00:52:41Z | 2018-05-27T00:52:41Z | OWNER | I ran a very rough micro-benchmark on the new `num_sql_threads` config option (added in #285) datasette --config num_sql_threads:1 fivethirtyeight.db Then ab -n 100 -c 10 'http://127.0.0.1:8011/fivethirtyeight-2628db9/twitter-ratio%2Fsenators' | Number of threads | Requests/second | |---|---| | 1 | 4.57 | | 3 | 9.77 | | 10 | 13.53 | | 20 | 15.24 | 50 | 8.21 | This was on my early 2018 OS X laptop. Need to benchmark in other common environments before making a decision on changing the default. That said, the default of 3 was a number I plucked out of thin air. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/290/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
326783670 | MDU6SXNzdWUzMjY3ODM2NzA= | 291 | Avoid plugins accidentally loading dependencies twice | simonw 9599 | closed | 0 | 3 | 2018-05-27T03:15:21Z | 2020-09-30T20:36:12Z | 2018-05-28T20:42:02Z | OWNER | Plugins that include JavaScript files risk loading the same code twice. In particular: I want to build a second plugin that uses the Leaflet mapping library (the first was [datasette-cluster-map](https://pypi.org/project/datasette-cluster-map/)). But I don't want the two plugins to load duplicate copies of Leaflet. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/291/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
326800219 | MDU6SXNzdWUzMjY4MDAyMTk= | 292 | Mechanism for customizing the SQL used to select specific columns in the table view | simonw 9599 | closed | 0 | 15 | 2018-05-27T09:05:52Z | 2021-05-27T04:25:01Z | 2021-05-27T04:25:01Z | OWNER | Some columns don't make a lot of sense in their default representation - binary blobs such as SpatiaLite geometries for example, or lengthy columns that really should be truncated somehow. We may also find that there are tables where we don't want to show all of the columns - so a mechanism to select a subset of columns would be nice. I think there are two features here: * the ability to request a subset of columns on the table view * the ability to override the SQL for a specific column and/or add extra columns - `AsGeoJSON(Geometry)` for example Both features should be available via both querystring arguments and in `metadata.json` The querystring argument for custom SQL should only work if `allow_sql` config is turned on. Refs #276 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/292/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
327365110 | MDU6SXNzdWUzMjczNjUxMTA= | 294 | inspect should record column types | simonw 9599 | open | 0 | 7 | 2018-05-29T15:10:41Z | 2019-06-28T16:45:28Z | OWNER | For each table we want to know the columns, their order and what type they are. I'm going to break with SQLite defaults a little on this one and allow datasette to define additional types - to start with just a `geometry` type for columns that are detected as SpatiaLite geometries. Possible JSON design: "columns": [{ "name": "title", "type": "text" }, ...] Refs #276 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/294/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
327383759 | MDU6SXNzdWUzMjczODM3NTk= | 295 | Extract unit tests for inspect out to test_inspect.py | simonw 9599 | closed | 0 | 2 | 2018-05-29T15:55:04Z | 2019-05-11T21:40:32Z | 2019-05-11T21:40:32Z | OWNER | Right now they are bundled up as API unit tests for a relatively unimportant endpoint. They should be their own thing. Blocks #294 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/295/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
327395270 | MDU6SXNzdWUzMjczOTUyNzA= | 296 | Per-database and per-table /-/ URL namespace | simonw 9599 | open | 0 | 3 | 2018-05-29T16:23:13Z | 2019-06-28T16:46:34Z | OWNER | Initially this will be for subsets of `/-/inspect` and `/-/metadata` but it will also give us a URL namespace for future features like `/-/facet` (expanded list of a specific facet, linked to from `...`) and `/-/graph` To start: * `/dbname/-/inspect` * `/dbname/-/metadata` * `/dbname/tablename/-/inspect` * `/dbname/tablename/-/metadata` This means we will no longer allow databases or tables to have the name `"-"` - I think that's OK We will continue to support rows with a primary key of `"-"` at the following URL: * `/dbname/tablename/-` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/296/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
327420945 | MDU6SXNzdWUzMjc0MjA5NDU= | 297 | datasette publish Dockerfile should use python:3.6-slim-stretch | simonw 9599 | closed | 0 | 1 | 2018-05-29T17:40:08Z | 2018-05-31T14:44:37Z | 2018-05-31T14:44:37Z | OWNER | Right now the Dockerfile generated by `datasette package` and `datasette publish` uses this: https://github.com/simonw/datasette/blob/b0a95da96386ddf99816911e08df86178ffa9a89/datasette/utils.py#L269 This appears to result in a SQLite version of `3.8.7.1` - https://parlgov.datasettes.com/-/versions ``` "sqlite": { "extensions": {}, "fts_versions": [ "FTS4", "FTS3" ], "version": "3.8.7.1" } ``` Meanwhile, https://fivethirtyeight.datasettes.com/-/versions is deployed with this Dockerfile https://github.com/simonw/fivethirtyeight-datasette/blob/0849901cae06e957fe04892cd4033bdcd1fcf966/Dockerfile which uses `FROM python:3.6-slim-stretch` and results in the following version report: ``` "sqlite": { "extensions": { "json1": null }, "fts_versions": [ "FTS5", "FTS4", "FTS3" ], "version": "3.16.2" } ``` So not only do we get a more recent SQLite (including https://www.sqlite.org/rowvalue.html added in 3.15) but we also get `FTS5` and `json1` as well. Refs #191 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/297/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
327459829 | MDU6SXNzdWUzMjc0NTk4Mjk= | 298 | URLify URLs in results from custom SQL statements / views | simonw 9599 | closed | 0 | 2 | 2018-05-29T19:41:07Z | 2018-07-24T04:53:20Z | 2018-07-24T03:56:50Z | OWNER | Consider this custom query: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3?sql=select+user%2C+%28%27https%3A%2F%2Ftwitter.com%2F%27+%7C%7C+user%29+as+user_url%2C+created_at%2C+text%2C+url+from+%5Btwitter-ratio%2Fsenators%5D+limit+10%3B ```select user, ('https://twitter.com/' || user) as user_url, created_at, text, url from [twitter-ratio/senators] limit 10;``` ![2018-05-29 at 12 38 pm](https://user-images.githubusercontent.com/9599/40681177-44a36d5c-633d-11e8-935b-c49dad7ac682.png) It would be nice if these URLs were turned into links, as happens on the table view page: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/twitter-ratio%2Fsenators ![2018-05-29 at 12 39 pm](https://user-images.githubusercontent.com/9599/40681206-5c69c47c-633d-11e8-9f3a-08899f8659b8.png) This currently does not happen because the table view render logic takes a different path through `display_columns_and_rows()` which includes this bit: https://github.com/simonw/datasette/blob/b0a95da96386ddf99816911e08df86178ffa9a89/datasette/views/table.py#L195-L202 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/298/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
327461381 | MDU6SXNzdWUzMjc0NjEzODE= | 299 | Documentation covering ALL datasette URLs | simonw 9599 | closed | 0 | 1 | 2018-05-29T19:46:15Z | 2018-07-28T04:24:05Z | 2018-07-28T04:22:30Z | OWNER | Relates to #296. We need a single page of the docs listing all of the URL patterns Datasette responds to, also detailing which templates are used to render them and linking to examples of the JSON they output when called with `.json`. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/299/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
327541975 | MDU6SXNzdWUzMjc1NDE5NzU= | 300 | Hide sort select box on larger screens | simonw 9599 | closed | 0 | 0 | 2018-05-30T01:34:59Z | 2018-05-31T14:43:13Z | 2018-05-31T14:43:13Z | OWNER | I'm larger screens you can sort by clicking column headers, so no need to show the select box (which was added for the small screen layout that doesn't show headers) | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/300/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
328155946 | MDU6SXNzdWUzMjgxNTU5NDY= | 301 | --spatialite option for "datasette publish heroku" | simonw 9599 | open | 0 | 1 | 2018-05-31T14:13:09Z | 2022-01-20T21:28:50Z | OWNER | Split off from #243. Need to figure out how to install and configure SpatiaLite on Heroku. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/301/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
328172521 | MDU6SXNzdWUzMjgxNzI1MjE= | 303 | Support table names ending with .json or .csv | simonw 9599 | closed | 0 | 4 | 2018-05-31T14:53:23Z | 2018-06-15T06:55:50Z | 2018-06-15T06:55:50Z | OWNER | This is needed for #266 - if a table name ends with `.json` or `.csv` right now our URL pattern matching will do the wrong thing. We should be smarter about this. This does mean we will have some URLs that look like this: http://localhost:8001/dbname/weird.json - returning HTML, not JSON http://localhost:8001/dbname/weird.json.json - returning JSON http://localhost:8001/dbname/weird.json.csv - returning CSV | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/303/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
328229224 | MDU6SXNzdWUzMjgyMjkyMjQ= | 304 | Ability to configure SQLite cache_size | simonw 9599 | closed | 0 | 3 | 2018-05-31T17:28:07Z | 2018-06-04T16:13:32Z | 2018-06-04T16:03:19Z | OWNER | See https://www.sqlite.org/pragma.html#pragma_cache_size Let's call the config setting `cache_size_kb` to emphasize that we're using the negative option. Note this warning: perhaps we should raise an error if you try to use this setting against a SQLite version prior to 3.7.10 > If the argument N is positive then the suggested cache size is set to N. If the argument N is negative, then the number of cache pages is adjusted to use approximately abs(N*1024) bytes of memory. Backwards compatibility note: The behavior of cache_size with a negative N was different in prior to version 3.7.10 (2012-01-16). In version 3.7.9 and earlier, the number of pages in the cache was set to the absolute value of N. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/304/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
329147284 | MDU6SXNzdWUzMjkxNDcyODQ= | 305 | Add contributor guidelines to docs | simonw 9599 | closed | 0 | 2 | 2018-06-04T17:25:30Z | 2019-06-24T06:40:19Z | 2019-06-24T06:40:19Z | OWNER | https://channels.readthedocs.io/en/latest/contributing.html is a nice example of this done well. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/305/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
329661905 | MDU6SXNzdWUzMjk2NjE5MDU= | 306 | Custom URL routing with independent tests | simonw 9599 | closed | 0 | 5 | 2018-06-05T23:40:08Z | 2018-06-07T15:29:28Z | 2018-06-07T15:29:28Z | OWNER | The more I think about #303 the more I feel that Datasette's URL routing needs go beyond Django-style regex matching. If we go custom, tests should live in `test_routing.py` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/306/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
330323860 | MDExOlB1bGxSZXF1ZXN0MTkzMzYxMzQx | 307 | Initial sketch of custom URL routing, refs #306 | simonw 9599 | closed | 0 | 1 | 2018-06-07T15:26:48Z | 2018-06-07T15:29:54Z | 2018-06-07T15:29:41Z | OWNER | simonw/datasette/pulls/307 | See #306 for background on this. | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/307/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
332830309 | MDU6SXNzdWUzMzI4MzAzMDk= | 310 | datasette publish now is broken in master | simonw 9599 | closed | 0 | 0 | 2018-06-15T16:01:14Z | 2018-06-16T16:29:50Z | 2018-06-16T16:29:50Z | OWNER | ``` > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2 > unable to execute 'gcc': No such file or directory > error: command 'gcc' failed with exit status 1 > > ---------------------------------------- > Command "/usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-s73273rj/httptools/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-yha7dxqq/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-s73273rj/httptools/ ``` Turns out the `python-slim` base image I introduced in b18e4515855c3f1eeca3dfcccdbb6df05869084a doesn't include gcc: https://github.com/docker-library/python/issues/60#issuecomment-134322383 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/310/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
332998752 | MDExOlB1bGxSZXF1ZXN0MTk1MzM5MTEx | 311 | ?_labels=1 to expand foreign keys (in csv and json), refs #233 | simonw 9599 | closed | 0 | 2 | 2018-06-16T16:31:12Z | 2018-06-16T22:20:31Z | 2018-06-16T22:20:31Z | OWNER | simonw/datasette/pulls/311 | Output looks something like this: { "rowid": 233, "TreeID": 121240, "qLegalStatus": { "value" 2, "label": "Private" } "qSpecies": { "value": 16, "label": "Sycamore" } "qAddress": "91 Commonwealth Ave", ... } | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/311/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
333000163 | MDU6SXNzdWUzMzMwMDAxNjM= | 312 | HTML, CSV and JSON views should support ?_col=&_col= | simonw 9599 | closed | 0 | 1 | 2018-06-16T16:53:35Z | 2021-06-17T18:14:24Z | 2018-06-16T17:00:12Z | OWNER | To support whitelisting columns to display. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/312/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
333086005 | MDU6SXNzdWUzMzMwODYwMDU= | 313 | Deploy demo of Datasette on every commit that passes tests | simonw 9599 | closed | 0 | 6 | 2018-06-17T19:19:12Z | 2018-06-17T21:52:58Z | 2018-06-17T21:52:58Z | OWNER | We can use Travis CI and Zeit Now to ensure there is always a live demo of current master. We can ship archived demos for releases as well. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/313/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
333120982 | MDExOlB1bGxSZXF1ZXN0MTk1NDEzMjQx | 315 | Streaming mode for downloading all rows as a CSV | simonw 9599 | closed | 0 | 0 | 2018-06-18T03:06:59Z | 2018-06-18T03:29:13Z | 2018-06-18T03:21:02Z | OWNER | simonw/datasette/pulls/315 | Refs #266 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/315/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
334169932 | MDU6SXNzdWUzMzQxNjk5MzI= | 320 | Need unit tests covering the different states for the advanced export box | simonw 9599 | closed | 0 | 1 | 2018-06-20T17:03:40Z | 2018-07-24T04:53:20Z | 2018-07-24T03:38:40Z | OWNER | There are quite a few variants of this box: ![2018-06-20 at 10 02 am](https://user-images.githubusercontent.com/9599/41673229-1d423adc-7471-11e8-99d4-4251f7d03aa5.png) Test coverage should exercise all of them, since the logic is a little unclear. https://github.com/simonw/datasette/blob/fdfbbbb9ee0d02fd4d43dfc42382252fa2287d6d/datasette/templates/table.html#L140-L159 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/320/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
334698969 | MDU6SXNzdWUzMzQ2OTg5Njk= | 323 | Speed up Travis CI builds | simonw 9599 | closed | 0 | 1 | 2018-06-21T23:55:27Z | 2018-07-10T15:03:37Z | 2018-07-10T15:03:36Z | OWNER | They've got a bit slow. Part of this is the Zeit Now deploy, but the build-and-test cycle is taking at least a couple of minutes. ![2018-06-21 at 4 54 pm](https://user-images.githubusercontent.com/9599/41751010-e48c823e-7573-11e8-88f3-7aa8a7e53917.png) | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/323/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
334731076 | MDExOlB1bGxSZXF1ZXN0MTk2NjI4MzA0 | 324 | Speed up Travis by reusing pip wheel cache across builds | simonw 9599 | closed | 0 | 0 | 2018-06-22T03:20:08Z | 2018-06-24T01:03:47Z | 2018-06-24T01:03:47Z | OWNER | simonw/datasette/pulls/324 | From https://atchai.com/blog/faster-ci/ - refs #323 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/324/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
335064777 | MDU6SXNzdWUzMzUwNjQ3Nzc= | 325 | Error on row page if table has slashes in the name and ends in .csv | simonw 9599 | closed | 0 | 1 | 2018-06-23T03:43:42Z | 2018-07-09T17:28:27Z | 2018-07-08T05:21:59Z | OWNER | https://v0-23-1.datasette.io/fixtures-e14e080/table%252Fwith%252Fslashes.csv/3 > no such table: table%252Fwith%252Fslashes.csv From clicking the row link on https://v0-23-1.datasette.io/fixtures-e14e080/table%2Fwith%2Fslashes.csv | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/325/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
335141434 | MDU6SXNzdWUzMzUxNDE0MzQ= | 326 | CSV should respect --cors and return cors headers | simonw 9599 | closed | 0 | 1 | 2018-06-24T00:44:07Z | 2021-06-17T18:14:24Z | 2018-06-24T00:59:45Z | OWNER | Otherwise tools like Vega can't load data via CSV. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/326/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
335200136 | MDU6SXNzdWUzMzUyMDAxMzY= | 327 | Explore if SquashFS can be used to shrink size of packaged Docker containers | simonw 9599 | open | 0 | 4 | 2018-06-24T18:15:16Z | 2022-02-17T23:37:24Z | OWNER | Inspired by this article: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html#sqlite-database-indexed--squashed https://en.wikipedia.org/wiki/SquashFS is "a compressed read-only file system for Linux" - which means it could be a really nice fit for Datasette and its read-only SQLite databases. It would be interesting to explore a Dockerfile recipe that used SquashFS to compress the SQLite database file that was bundled up by `datasette package` and friends. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/327/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
336464733 | MDU6SXNzdWUzMzY0NjQ3MzM= | 328 | Installation instructions, including how to use the docker image | simonw 9599 | closed | 0 | 3 | 2018-06-28T03:59:33Z | 2018-10-05T06:37:07Z | 2018-06-28T04:02:10Z | OWNER | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/328/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||||
336465018 | MDU6SXNzdWUzMzY0NjUwMTg= | 329 | Travis should push tagged images to Docker Hub for each release | simonw 9599 | closed | 0 | 7 | 2018-06-28T04:01:31Z | 2018-11-05T06:54:10Z | 2018-11-05T06:53:28Z | OWNER | https://sebest.github.io/post/using-travis-ci-to-build-docker-images/ | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/329/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
337141108 | MDU6SXNzdWUzMzcxNDExMDg= | 332 | Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1 | simonw 9599 | closed | 0 | 12 | 2018-06-29T21:21:27Z | 2018-07-24T04:53:20Z | 2018-07-24T03:08:30Z | OWNER | It turns out if you load this CSV using `csvs-to-sqlite` you get an Infinity value in SQLite: ``` name,num sasha,10 terry,Inf cathy,0.5 ``` `csvs-to-sqlite infinity-bug.csv infinity-bug.db` I deployed this using: ``` datasette publish now infinity-bug.db --name=datasette-infinity-bug --install=datasette-vega ``` Datasette outputs that as `Infinity` in the JSON format, which causes JavaScript errors. Demo * https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug - HTML view works * https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug.json?_shape=array - this outputs the following: ``` [ { "rowid": 1, "name": "sasha", "num": 10.0 }, { "rowid": 2, "name": "terry", "num": Infinity }, { "rowid": 3, "name": "cathy", "num": 0.5 } ] ``` But... in Firefox that gets rendered like this: ![2018-06-29 at 4 20 pm](https://user-images.githubusercontent.com/9599/42115408-5d30f630-7bb8-11e8-8370-c8484801c49b.png) And if you click the "Show charting options" button you get this error in the console: ``` SyntaxError: JSON.parse: unexpected character at line 1 column 83 of the JSON data ``` | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/332/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
338768551 | MDU6SXNzdWUzMzg3Njg1NTE= | 333 | Datasette on Zeit Now returns http URLs for facet and next links | simonw 9599 | closed | 0 | 4 | 2018-07-06T00:40:49Z | 2018-07-24T04:53:20Z | 2018-07-24T01:51:53Z | OWNER | e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=0 ``` { "facet_results": { "lg_id": { "name": "lg_id", "results": [ { "value": "NBA", "label": "NBA", "count": 118016, "toggle_url": "http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=NBA", "selected": false }, { "value": "ABA", "label": "ABA", "count": 8298, "toggle_url": "http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=ABA", "selected": false } ], "truncated": false } }, "suggested_facets": [ { "name": "_iscopy", "toggle_url": "/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_facet=_iscopy" } ], "next_url": "http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_next=1", } ``` `next_url` and `facet_results` both link to `http://` when they should link to `https://`. Note that suggested facets doesn't include the full URL at all, which is a consistency bug. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/333/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
339505204 | MDU6SXNzdWUzMzk1MDUyMDQ= | 335 | Package datasette for installation using homebrew | simonw 9599 | closed | 0 | 12 | 2018-07-09T15:45:03Z | 2020-08-11T16:54:06Z | 2020-08-11T16:54:06Z | OWNER | https://docs.brew.sh/Python-for-Formula-Authors describes how. > Applications should be installed into a Python virtualenv environment rooted in libexec. This prevents the app’s Python modules from contaminating the system site-packages and vice versa. It recommends using https://github.com/tdsmith/homebrew-pypi-poet | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/335/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
340039409 | MDU6SXNzdWUzNDAwMzk0MDk= | 336 | Ensure --help examples in docs are always up to date | simonw 9599 | closed | 0 | 3 | 2018-07-10T23:20:01Z | 2018-07-24T16:01:29Z | 2018-07-24T16:01:29Z | OWNER | Ideally I would automatically generate the --help output shown in our docs, but I don't think I can get that working with readthedocs. Instead, I'm going to add a unit test that checks that those extracts in the documentation match the current output of the --help command. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/336/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
340065374 | MDU6SXNzdWUzNDAwNjUzNzQ= | 337 | Documentation for datasette publish and datasette package | simonw 9599 | closed | 0 | 1 | 2018-07-11T02:04:06Z | 2018-07-11T02:07:32Z | 2018-07-11T02:05:56Z | OWNER | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/337/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | |||||||
340282796 | MDU6SXNzdWUzNDAyODI3OTY= | 338 | Only load vegaEmbed if charting tools are enabled | simonw 9599 | closed | 0 | 1 | 2018-07-11T15:02:14Z | 2018-07-11T15:21:47Z | 2018-07-11T15:21:47Z | OWNER | vegaEmbed is a LOT of code (it bundles d3) Inspired by this tweet: https://twitter.com/thelarkinn/status/1017053567641948162 - it would be great if we loaded that code on demand the first time the "Show chart options" button was clicked, or when the page loads with #g. options in the URL. Even better: avoid the overhead if loading React unless the chart options need to be displayed. This would be a pretty major refactoring though. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/338/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
340730961 | MDU6SXNzdWUzNDA3MzA5NjE= | 340 | Embrace black | simonw 9599 | closed | 0 | 1 | 2018-07-12T17:32:29Z | 2019-06-24T06:50:27Z | 2019-06-24T06:50:26Z | OWNER | Run [black](https://github.com/ambv/black) against everything. Then set up CI to fail if code doesn't conform to black's style. Here's how Starlette does this: * https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/.travis.yml#L14 * https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/scripts/lint - essentially runs `black starlette tests --check` And here's an example of a test run that failed: https://travis-ci.org/encode/starlette/jobs/403172478 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/340/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
340733753 | MDExOlB1bGxSZXF1ZXN0MjAxMDc1NTMy | 341 | Bump aiohttp to fix compatibility with Python 3.7 | simonw 9599 | closed | 0 | 0 | 2018-07-12T17:41:24Z | 2018-07-12T18:07:38Z | 2018-07-12T18:07:38Z | OWNER | simonw/datasette/pulls/341 | Tests failed here: https://travis-ci.org/simonw/datasette/jobs/403223333 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/341/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
344654623 | MDU6SXNzdWUzNDQ2NTQ2MjM= | 347 | Rename "datasette package" to "datasette publish docker" | simonw 9599 | open | 0 | 0 | 2018-07-26T00:42:46Z | 2018-07-26T00:42:46Z | OWNER | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/347/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | |||||||||
344656114 | MDU6SXNzdWUzNDQ2NTYxMTQ= | 348 | Unit tests for "datasette publish" | simonw 9599 | closed | 0 | 1 | 2018-07-26T00:52:23Z | 2018-07-26T05:46:10Z | 2018-07-26T05:46:10Z | OWNER | The datasette publish family of commands all work by shelling out to heroku/now/docker from subprocess import call, check_output So in tests I should be able to mock those calls: @mock.patch('subprocess.call') | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/348/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
344695978 | MDExOlB1bGxSZXF1ZXN0MjA0MDI5MTQy | 349 | publish_subcommand hook + default plugins mechanism, used for publish heroku/now | simonw 9599 | closed | 0 | 1 | 2018-07-26T05:03:22Z | 2018-07-26T05:28:54Z | 2018-07-26T05:16:00Z | OWNER | simonw/datasette/pulls/349 | This change introduces a new plugin hook, publish_subcommand, which can be used to implement new subcommands for the "datasette publish" command family. I've used this new hook to refactor out the "publish now" and "publish heroku" implementations into separate modules. I've also added unit tests for these two publishers, mocking the subprocess.call and subprocess.check_output functions. As part of this, I introduced a mechanism for loading default plugins. These are defined in the new "default_plugins" list inside datasette/app.py Closes #217 (Plugin support for "datasette publish") Closes #348 (Unit tests for "datasette publish") Refs #14, #59, #102, #103, #146, #236, #347 | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/349/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
344701755 | MDU6SXNzdWUzNDQ3MDE3NTU= | 350 | Don't list default plugins on /-/plugins | simonw 9599 | closed | 0 | 2 | 2018-07-26T05:38:00Z | 2018-08-28T17:13:50Z | 2018-08-28T16:48:19Z | OWNER | https://dbbe707.datasette.io/-/plugins is showing "datasette.publish.now" and "datasette.publish.heroku" | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/350/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
345469355 | MDU6SXNzdWUzNDU0NjkzNTU= | 351 | Automatically create a GitHub release linking to release notes for every tagged release | simonw 9599 | closed | 0 | 1 | 2018-07-28T18:31:12Z | 2020-05-28T18:56:16Z | 2020-05-28T18:56:15Z | OWNER | Can use this API called from Travis: https://developer.github.com/v3/repos/releases/#create-a-release The release it generates should look like this one: https://github.com/simonw/datasette/releases/tag/0.24 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/351/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
345821500 | MDU6SXNzdWUzNDU4MjE1MDA= | 352 | render_cell(value) plugin hook | simonw 9599 | closed | 0 | 4 | 2018-07-30T15:56:20Z | 2020-02-10T16:18:58Z | 2018-08-05T00:14:57Z | OWNER | To allow plugins to customize how values matching a specific pattern are displayed in the HTML table view. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/352/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
345821778 | MDExOlB1bGxSZXF1ZXN0MjA0ODUxNTEx | 353 | render_cell(value) plugin hook | simonw 9599 | closed | 0 | 0 | 2018-07-30T15:57:08Z | 2018-08-05T00:14:57Z | 2018-08-05T00:14:57Z | OWNER | simonw/datasette/pulls/353 | Closes #352. | datasette 107914493 | pull | {"url": "https://api.github.com/repos/simonw/datasette/issues/353/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
346026869 | MDU6SXNzdWUzNDYwMjY4Njk= | 354 | Handle many-to-many relationships | simonw 9599 | open | 0 | 0 | 2018-07-31T04:03:13Z | 2020-11-24T19:51:18Z | OWNER | This is a master tracking ticket for various many-2-many features. | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/354/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
346027040 | MDU6SXNzdWUzNDYwMjcwNDA= | 355 | Table view should support filtering via many-to-many relationships | simonw 9599 | open | 0 | 10 | 2018-07-31T04:04:16Z | 2019-05-23T06:04:03Z | OWNER | Parent: #354 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/355/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
346028655 | MDU6SXNzdWUzNDYwMjg2NTU= | 356 | Ability to display facet counts for many-to-many relationships | simonw 9599 | closed | 0 | 4 | 2018-07-31T04:14:26Z | 2019-05-29T21:39:12Z | 2019-05-25T16:30:09Z | OWNER | Parent: #354 | datasette 107914493 | issue | {"url": "https://api.github.com/repos/simonw/datasette/issues/356/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
347058326 | MDExOlB1bGxSZXF1ZXN0MjA1NzcwOTk2 | 1 | Make .indexes compatible with older SQLite versions | simonw 9599 | closed | 0 | 0 | 2018-08-02T15:17:05Z | 2018-08-02T15:17:30Z | 2018-08-02T15:17:30Z | OWNER | simonw/sqlite-utils/pulls/1 | Older SQLite versions return a different set of columns from the PRAGMA we are using. | sqlite-utils 140912432 | pull | {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issues] ( [id] INTEGER PRIMARY KEY, [node_id] TEXT, [number] INTEGER, [title] TEXT, [user] INTEGER REFERENCES [users]([id]), [state] TEXT, [locked] INTEGER, [assignee] INTEGER REFERENCES [users]([id]), [milestone] INTEGER REFERENCES [milestones]([id]), [comments] INTEGER, [created_at] TEXT, [updated_at] TEXT, [closed_at] TEXT, [author_association] TEXT, [pull_request] TEXT, [body] TEXT, [repo] INTEGER REFERENCES [repos]([id]), [type] TEXT , [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT); CREATE INDEX [idx_issues_repo] ON [issues] ([repo]); CREATE INDEX [idx_issues_milestone] ON [issues] ([milestone]); CREATE INDEX [idx_issues_assignee] ON [issues] ([assignee]); CREATE INDEX [idx_issues_user] ON [issues] ([user]);