id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason
315960272,MDU6SXNzdWUzMTU5NjAyNzI=,227,prepare_context() plugin hook,9599,closed,0,,,8,2018-04-19T16:55:26Z,2020-03-24T22:19:54Z,2020-03-24T22:19:54Z,OWNER,,This would be called with the context dictionary before each template is rendered. It would have the opportunity to modify that context.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/227/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
316031566,MDU6SXNzdWUzMTYwMzE1NjY=,228,"If spatialite detected, mark idx_XXX_Geometry tables as hidden",9599,closed,0,,,1,2018-04-19T20:37:24Z,2018-04-26T03:25:39Z,2018-04-26T03:25:39Z,OWNER,,"https://timezones-api.now.sh/timezones-faf26d0

![2018-04-19 at 1 36 pm](https://user-images.githubusercontent.com/9599/39016906-a5acbb3e-43d6-11e8-9a31-814ff1d0022e.png)

Need to update this logic:

https://github.com/simonw/datasette/blob/e2750c7cc0585adaa8c866be611089e62961ee35/datasette/app.py#L1276-L1288",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/228/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
316123256,MDU6SXNzdWUzMTYxMjMyNTY=,229,Table view should support ?_size=400 parameter,9599,closed,0,,,1,2018-04-20T04:23:18Z,2018-04-26T04:49:46Z,2018-04-26T04:48:32Z,OWNER,,Allows callers to request more rows at once. The limit will still be `max_returned_rows` (defaults to 1000).,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/229/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
316128955,MDU6SXNzdWUzMTYxMjg5NTU=,230,Setting page size AND max returned rows to 1000 doesn't seem to work,9599,closed,0,,,1,2018-04-20T05:05:11Z,2018-04-26T04:04:25Z,2018-04-26T04:04:25Z,OWNER,,"It appears that if the two settings are the same Datasette fails to return any results, probably because of the trick where we try to fetch 1001 rows so we know if there's a next page.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/230/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
316323336,MDU6SXNzdWUzMTYzMjMzMzY=,231,metadata.json support for plugin configuration options,9599,closed,0,,,4,2018-04-20T15:58:47Z,2019-05-13T18:56:21Z,2019-05-13T18:56:21Z,OWNER,,"My [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map) plugin currently works by detecting `latitude` and `longitude` columns. I'd like to be able to configure it to look for different column names.

One way to do this could be to support optional plugin configuration as part of `metadata.json`. Something like this:

    {
        ""title"": ""Polar Bear Ear Tags, 2009-2011"",
        ""source"": ""USGS Alaska Science Center, Polar Bear Research Program"",
        ""source_url"": ""https://alaska.usgs.gov/products/data.php?dataid=130"",
        ""plugins"": {
            ""datasette_cluster_map"": {
                ""latitude_columns"": [
                    ""latitude"",
                    ""Capture Latitude""
                ],
                ""longitude_columns"": [
                    ""longitude"",
                    ""Capture Longitude""
                ]
            }
        }
    }

These settings should be supported at the root level or at the individual database or table level.

They could also be exposed in the https://datasette-cluster-map-demo.now.sh/-/plugins debug tool.

Refs #14",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/231/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
316444720,MDU6SXNzdWUzMTY0NDQ3MjA=,233,Option to expose expanded foreign keys in JSON/CSV,9599,closed,0,,,11,2018-04-21T00:18:25Z,2018-06-16T22:26:21Z,2018-06-16T22:20:14Z,OWNER,,"https://datasette-cluster-map-demo.datasettes.com/sf-trees-02c8ef1/Street_Tree_List?qCareAssistant=1

![f36b87c0-478e-4d55-9a5f-ad37df0b47cb](https://user-images.githubusercontent.com/9599/39078411-bb3e4f88-44be-11e8-9d0c-d22324793c77.png)

It would be nice if the info bubbles there could expose more than just the IDs, and if the title showed the expanded name of the selected qCareAssistant.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/233/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
316526433,MDU6SXNzdWUzMTY1MjY0MzM=,234,label_column option in metadata.json,9599,closed,0,,,3,2018-04-21T21:19:08Z,2018-04-22T20:47:12Z,2018-04-22T20:47:12Z,OWNER,,"Currently the column used for displaying a foreign key relationship is automatically detected by `inspect()` by looking for tables that have a primary key column and one other column.

This doesn't work for tables with more than two columns.

Let's allow the table section in `metadata.json` to optionally define a `label_column` which, if present, will be used for those displays.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/234/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
316621102,MDU6SXNzdWUzMTY2MjExMDI=,235,Add limit on the size in KB of data returned from a single query,9599,open,0,,,2,2018-04-22T23:01:15Z,2018-04-24T00:30:02Z,,OWNER,,"Datasette limits the number of rows returned to 1,000 and limits the time spent executing a SQL query to 1000ms - and both of these limits can be customized.

It does not have a limit on the size of the response returned. It's possible to compose maliciously large SQL responses in a small number of rows using mechanisms like the `group_concat()` aggregate function. It would be good to avoid malicious SQL creating 100MB+ responses and potentially crashing the server.

I think the easiest place to implement that is here:

https://github.com/simonw/datasette/blob/f3f42957128c1e7ece584d45d9167f2ac003a3b8/datasette/app.py#L175-L190

Currently we use `cursor.fetchmany()` to fetch up to 1,001 rows at once. Instead, we could switch to iterating through `cursor.fetchone()` (or just using `for row in cursor`) and keeping a running tally of the size of the response as we go - maybe just using `rough_response_size += len(str(row))`. If that goes above a certain threshold we can terminate the response with an error, like we do with timelimits.

The bigger challenge here is understanding how well this approach works and what impact it will have on overall Datasette performance. I think I need #33 for this.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/235/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
317001500,MDU6SXNzdWUzMTcwMDE1MDA=,236,datasette publish lambda plugin,9599,open,0,,,11,2018-04-23T22:10:30Z,2023-03-12T14:04:15Z,,OWNER,,"Refs #217 - create a publish plugin that can deploy to AWS Lambda.

https://docs.aws.amazon.com/lambda/latest/dg/limits.html says lambda packages can be up to 50 MB, so this would only work with smaller databases (the command can check the filesize before attempting to package and deploy it).

Lambdas do get a 512 MB `/tmp` directory too, so for larger databases the function could start and then download up to 512MB from an S3 bucket - so the plugin could take an optional S3 bucket to write to and know how to upload the `.db` file there and then have the lambda download it on startup.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/236/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
317475156,MDU6SXNzdWUzMTc0NzUxNTY=,237,Support for ?_search_colname=blah searches,9599,closed,0,,,2,2018-04-25T04:29:53Z,2018-05-05T22:56:42Z,2018-05-05T22:33:23Z,OWNER,,"Right now the `_search=` argument searches across all fields in a full-text index, for example:

https://san-francisco.datasettes.com/sf-film-locations-84594a7/Film_Locations_in_San_Francisco?_search=justin

SQLite FTS also supports searches within a specified field, for example:

https://san-francisco.datasettes.com/sf-film-locations-84594a7?sql=select+rowid%2C+*+from+Film_Locations_in_San_Francisco+where+rowid+in+%28select+rowid+from+%5BFilm_Locations_in_San_Francisco_fts%5D+where+%5BLocations%5D+match+%3Asearch%29+order+by+rowid+limit+101&search=justin

```
select rowid, * from Film_Locations_in_San_Francisco
where rowid in (
  select rowid from [Film_Locations_in_San_Francisco_fts]
  where [Locations] match :search
) order by rowid limit 101
```

The `_search=` parameter could be extended to support this using `_search_colname=`.

This should also be able to support columns with spaces and special characters in their names, something like this:

`_search_Column%20With%20Spaces=foo`
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/237/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
317714268,MDU6SXNzdWUzMTc3MTQyNjg=,238,External metadata.json,9599,closed,0,,,3,2018-04-25T17:02:30Z,2019-06-24T06:52:55Z,2019-06-24T06:52:45Z,OWNER,,"A frustration I'm having with https://register-of-members-interests.datasettes.com/ is that I keep coming up with new canned queries but I don't want to redeploy the whole thing just to add them to `metadata.json`

Maybe Datasette could optionally take a `--metadata-url` option which causes it to load from a URL instead and occasionally check for updates.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/238/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
317760361,MDU6SXNzdWUzMTc3NjAzNjE=,239,Support for hidden tables in metadata.json,9599,closed,0,,,2,2018-04-25T19:21:17Z,2018-04-26T03:45:12Z,2018-04-26T03:43:10Z,OWNER,,"Since we already have a hidden feature, let's expose it more to our users ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/239/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
317900587,MDU6SXNzdWUzMTc5MDA1ODc=,240,FTS table detection should be part of .inspect(),9599,closed,0,,,0,2018-04-26T06:58:10Z,2018-04-29T00:04:44Z,2018-04-29T00:04:44Z,OWNER,,"The code that detects if specific tables have a corresponding FTS column is currently called from TableView - it should instead be handled as part of `.inspect()`. This will make it easier to build other features that need to behave differently depending on whether a table can be searched, e.g. an autocomplete widget for selecting filters from foreign key tables.

Current code:

https://github.com/simonw/datasette/blob/f188ceaa2a3a5b2eab83425ad0f00cb0d364e24a/datasette/app.py#L728-L733",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/240/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
318490133,MDU6SXNzdWUzMTg0OTAxMzM=,241,Default datasette logging format should be JSON,9599,open,0,,,0,2018-04-27T17:32:48Z,2018-07-10T17:45:40Z,,OWNER,,"Structured logs are better. Datasette should default to outputting it's HTTP access log lines as newline delimited JSON instead of the Sanic default format it uses at the moment.

For improved greppability these logs should have keys ordered in a consistent way. Python's JSON module can do this with ordered dictionaries.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/241/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
318692953,MDU6SXNzdWUzMTg2OTI5NTM=,242,Rename ?_sql_time_limit_ms= to ?_timelimit=,9599,closed,0,,,0,2018-04-29T06:11:35Z,2018-05-02T00:20:42Z,2018-05-02T00:20:42Z,OWNER,,It's a bit of a mouthful at the moment.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/242/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
318737808,MDU6SXNzdWUzMTg3Mzc4MDg=,243,--spatialite option for datasette publish commands,9599,closed,0,,,2,2018-04-29T18:19:32Z,2018-05-31T14:17:53Z,2018-05-31T14:17:53Z,OWNER,,Performs the necessary incantations to install Spatialite on Zeit Now or Heroku and sets the corresponding environment variable to ensure the module is correctly loaded by datasette serve.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/243/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
318738000,MDU6SXNzdWUzMTg3MzgwMDA=,244,/-/versions page,9599,closed,0,,,1,2018-04-29T18:22:15Z,2018-05-03T14:13:49Z,2018-05-03T14:09:53Z,OWNER,,"Displays the current version of:

* datasette
* Python
* SQLite
* Spatialite (if available)

Installed plugin versions should be shown on /-/plugins",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/244/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
319358200,MDU6SXNzdWUzMTkzNTgyMDA=,245,?_shape=array option,9599,closed,0,,,1,2018-05-01T23:11:07Z,2018-05-03T14:14:33Z,2018-05-02T00:12:20Z,OWNER,,"Some tools (`pandas.DataFrame(...)` for example) are happiest when you give them a raw array of JSON objects.

`?_shape=array` should do just that

While I'm at it, rename the default `?_shape=lists` to instead be called `?shape=arrays`

And validate that `_shape` is a valid option

And have `?_shape=object` return the object at the root level rather than nested in `.rows` to better match the behavior of `?_shape=array`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/245/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
319371036,MDExOlB1bGxSZXF1ZXN0MTg1MzA3NDA3,246,?_shape=array and _timelimit=,9599,closed,0,,,0,2018-05-02T00:18:54Z,2018-05-02T00:20:41Z,2018-05-02T00:20:40Z,OWNER,simonw/datasette/pulls/246,,107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/246/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
319954545,MDU6SXNzdWUzMTk5NTQ1NDU=,248,/-/plugins should show version of each installed plugin,9599,closed,0,,,2,2018-05-03T14:50:45Z,2018-05-04T18:25:40Z,2018-05-04T18:05:04Z,OWNER,,"Refs #244 

https://stackoverflow.com/questions/20180543/how-to-check-version-of-python-modules

```
>>> import pkg_resources
>>> pkg_resources.get_distribution('datasette_cluster_map').version
'0.4'
```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/248/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
320090329,MDU6SXNzdWUzMjAwOTAzMjk=,249,?_size=max argument ,9599,closed,0,,,1,2018-05-03T21:42:04Z,2018-05-04T18:26:30Z,2018-05-04T18:05:04Z,OWNER,,"For plugins that want to load the most data allowable, having `?_size=max` would be useful.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/249/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
320132682,MDU6SXNzdWUzMjAxMzI2ODI=,250,Setup some issue templates,9599,open,0,,,0,2018-05-04T01:49:07Z,2018-05-04T01:49:07Z,,OWNER,,"https://twitter.com/left_pad/status/99216385740464537

I like the idea of using these to help people understand some of the ways I want to use issues.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/250/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
320592643,MDU6SXNzdWUzMjA1OTI2NDM=,251,"Explore ""distinct values for column"" in inspect()",9599,closed,0,,,4,2018-05-06T13:27:24Z,2018-05-14T22:47:55Z,2018-05-14T22:47:55Z,OWNER,,"A lot of datasets have columns which have a small number of possible values in them - this one for example: https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+distinct+category+from+%5Binconvenient-sequel%2Fratings%5D%3B

Detecting these could be interesting as part of `.inspect()`, since it would allow for various UI enhancements like autocomplete / select box filters for those columns.

The problem is detecting them efficiently. `.inspect()` shouldn't spend 5 minutes churning through columns on giant tables trying to determine if they have a small collection of unique values.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/251/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
321624016,MDU6SXNzdWUzMjE2MjQwMTY=,252,/-/versions should report the FTS version supported by SQLite,9599,closed,0,,,0,2018-05-09T15:43:47Z,2018-05-11T13:19:52Z,2018-05-11T13:19:52Z,OWNER,,I can copy this function from `csvs-to-sqlite`: https://github.com/simonw/csvs-to-sqlite/blob/dccbf65b37bc9eed50e9edb80a42f257e93edb1f/csvs_to_sqlite/utils.py#L283-L293,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/252/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
321631020,MDU6SXNzdWUzMjE2MzEwMjA=,253,Documentation explaining how to use SQLite FTS with Datasette,9599,closed,0,,,1,2018-05-09T16:02:08Z,2018-05-12T12:09:02Z,2018-05-12T12:06:51Z,OWNER,,"In particular how to work with https://www.sqlite.org/fts3.html#_external_content_fts4_tables_ - which Datasette can automatically detect and use to add a search UI to your page.

Examples of basic search setup like this:

```
CREATE VIRTUAL TABLE ""interests_fts""
    USING FTS4 (name, content=""interests"");
INSERT INTO ""interests_fts"" (rowid, name)
    SELECT rowid, name FROM interests;
```

And complex join-based search setup like this:

```
CREATE VIRTUAL TABLE ""interests_fts""
    USING FTS4 (name, category, member, content=""interests"");
INSERT INTO ""interests_fts"" (rowid, name, category, member)
    SELECT interests.rowid, interests.name, interest_categories.name, members.name
    FROM interests
        JOIN interest_categories ON interests.category_id = interest_categories.id
        JOIN members ON interests.member_id = members.id;
```

Also mention how `csvs-to-sqlite` can be used to do this easily.

This will benefit from #252 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/253/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
322477187,MDU6SXNzdWUzMjI0NzcxODc=,255,Facets,9599,closed,0,,,16,2018-05-12T03:00:07Z,2019-05-29T21:39:12Z,2018-05-16T15:32:12Z,OWNER,,"Ability to display facets and facet counts on the table view.

Facets can be specified in the URL with `?_facet=column&_facet=othercolumn` or the default facets for a table can be set using a new `""facets"": [...]` property in `metadata.json`

- [x] Implement `?_facet=`
- [x] Implement `metadata.json` `facets` key
- [x] Design for how facets should be presented
- [x] Facets should be able to toggle off as well as on
- [x] Expand labels for facets that are foreign keys
- [x] Suggest potential facets (if we can do so within a tight time limit)
- [x] Documentation",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/255/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
322551723,MDU6SXNzdWUzMjI1NTE3MjM=,256,Break up app.py into separate view modules,9599,closed,0,,,1,2018-05-12T23:56:33Z,2018-05-14T03:05:37Z,2018-05-14T03:05:37Z,OWNER,,"`views/table.py` and `views/database.py` and `views/utils.py` as a starting point.

Likewise, create `tests/test_views_table.py` and `tests/test_views_database.py` - these will contain both HTML and API test for those views.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/256/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
322591993,MDExOlB1bGxSZXF1ZXN0MTg3NjY4ODkw,257,Refactor views,9599,closed,0,,,5,2018-05-13T13:00:50Z,2018-05-14T03:04:25Z,2018-05-14T03:04:24Z,OWNER,simonw/datasette/pulls/257,"* Split out view classes from main `app.py`
* Run [black](https://github.com/ambv/black) against resulting code to apply opinionated source code formatting
* Run [isort](https://github.com/timothycrosley/isort) to re-order my imports

Refs #256 ",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/257/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
322787470,MDU6SXNzdWUzMjI3ODc0NzA=,259,inspect() should detect many-to-many relationships,9599,closed,0,,,6,2018-05-14T12:03:58Z,2019-05-23T03:55:37Z,2019-05-23T03:55:37Z,OWNER,,"Relates to #255 - in particular supporting facets across M2M relationships.

It should be possible for `.inspect()` to notice when a table has two foreign keys to two different tables, and assume that this means there is a M2M relationship between those tables.

When rendering a table with a m2m relationship we could display the first X associated records as a comma separated list of hyperlinks in a new column on the table view, with a column name derived from the table on the other side.

Since SQLite doesn't have RANK or an equivalent of https://www.xaprb.com/blog/2006/12/02/how-to-number-rows-in-mysql/ this would be implemented as N+1 queries (one query per cell that we want to display an m2m summary). This should be OK in SQLite: https://sqlite.org/np1queryprob.html",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/259/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
323223872,MDU6SXNzdWUzMjMyMjM4NzI=,260,Validate metadata.json on startup,9599,open,0,,,7,2018-05-15T13:42:56Z,2023-06-21T12:51:22Z,,OWNER,,"It's easy to misspell the name of a database or table and then be puzzled when the metadata settings silently fail.

To avoid this, let's sanity check the provided metadata.json on startup and quit with a useful error message if we find any obvious mistakes.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/260/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
323459939,MDExOlB1bGxSZXF1ZXN0MTg4MzEyNDEx,261,Facets improvements plus suggested facets,9599,closed,0,,,0,2018-05-16T03:52:39Z,2018-05-16T15:27:26Z,2018-05-16T15:27:25Z,OWNER,simonw/datasette/pulls/261,Refs #255,107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/261/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
323671577,MDU6SXNzdWUzMjM2NzE1Nzc=,263,Facets should not execute for ?shape=array|object,9599,closed,0,,,3,2018-05-16T15:26:13Z,2021-06-02T02:54:34Z,2021-06-02T02:54:34Z,OWNER,,Split off from #255 - there's no point executing the facet SQL for the `?_shape=array` and `?_shape=object` API responses.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/263/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
323673899,MDU6SXNzdWUzMjM2NzM4OTk=,264,Make it possible to customize various facet settings,9599,closed,0,,,1,2018-05-16T15:31:34Z,2018-05-18T06:18:00Z,2018-05-18T05:11:52Z,OWNER,,"The new Facets implementation from #255 includes several hard-coded settings which should be made configurable somehow:

Number of rows to return in a facet (maybe this should also be an option that can be set via quersytring argument, e.g. `?_facet=qSpecies:40`):

https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L539

Time limit for executing a facet:

https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L559-L562

Maximum unique values returned in order for a column to be suggested as a facet:

https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L646-L647

Time limit for calculating if a column should be a suggested facet:

https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L664-L667
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/264/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
323677499,MDU6SXNzdWUzMjM2Nzc0OTk=,265,Add links to example Datasette instances to appropiate places in docs,9599,closed,0,,,5,2018-05-16T15:40:20Z,2018-06-18T15:52:15Z,2018-06-18T15:52:15Z,OWNER,,"Links to working examples would really help, especially on these pages:

* http://datasette.readthedocs.io/en/latest/json_api.html
* http://datasette.readthedocs.io/en/latest/sql_queries.html
* http://datasette.readthedocs.io/en/latest/facets.html
* http://datasette.readthedocs.io/en/latest/full_text_search.html",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/265/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
323681589,MDU6SXNzdWUzMjM2ODE1ODk=,266,Export to CSV,9599,closed,0,,,27,2018-05-16T15:50:24Z,2021-06-17T18:14:24Z,2018-06-18T06:05:25Z,OWNER,,Datasette needs to be able to export data to CSV.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/266/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
323716411,MDU6SXNzdWUzMjM3MTY0MTE=,267,"Documentation for URL hashing, redirects and cache policy",9599,closed,0,,,3,2018-05-16T17:29:01Z,2019-06-24T06:41:02Z,2019-06-24T06:41:02Z,OWNER,,See my comments on #258 for a starting point,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/267/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
323718842,MDU6SXNzdWUzMjM3MTg4NDI=,268,Mechanism for ranking results from SQLite full-text search,9599,open,0,,,12,2018-05-16T17:36:40Z,2022-01-13T22:21:28Z,,OWNER,,This isn't particularly straight-forward - all the more reason for Datasette to implement it for you. This article is helpful: http://charlesleifer.com/blog/using-sqlite-full-text-search-with-python/,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/268/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
323726888,MDU6SXNzdWUzMjM3MjY4ODg=,269,"If a facet fails due to timing out, let the user know somehow",9599,closed,0,,,0,2018-05-16T18:01:47Z,2018-05-18T06:11:46Z,2018-05-18T06:11:46Z,OWNER,,Refs #255 - right now facets fail silently if the user requested them but they take longer than 200ms to calculate - see also #264,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/269/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
323830051,MDU6SXNzdWUzMjM4MzAwNTE=,270,--limit= CLI option for setting limits,9599,closed,0,,,1,2018-05-17T00:14:24Z,2018-05-18T06:19:31Z,2018-05-18T06:16:39Z,OWNER,,"#264 calls for four new datasette limit options, on top of the two existing ones:

* `--max_returned_rows`
* `--sql_time_limit_ms`

These are already clogging up `datasette serve --help` a bit.

How about this syntax instead?

    datasette --limit max_returned_rows:100 \
        --limit facet_timeout_ms:500 demo.db

Then we can add as many new user over-rideable limits as we like without clogging up `--help` too much - though it would be good to have a way of optionally listings their documentation as well.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/270/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
324162476,MDU6SXNzdWUzMjQxNjI0NzY=,271,Mechanism for automatically picking up changes when on-disk .db file changes,9599,closed,0,,,4,2018-05-17T19:53:15Z,2019-01-10T21:35:18Z,2019-01-10T21:35:18Z,OWNER,,"It would be useful if Datasette could spot when a SQLite database file changes on disk and restart itself (hence re-running .inspect() and picking up the new content hash). Ideally this could happen in an atomic way so no requests get dropped during the switch-over.

This may not play well with SQLite opening databases in immutable mode. Research required.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/271/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
324451322,MDU6SXNzdWUzMjQ0NTEzMjI=,273,Figure out a way to have /-/version return current git commit hash,9599,closed,0,,,2,2018-05-18T15:16:56Z,2018-05-22T19:35:22Z,2018-05-22T19:35:22Z,OWNER,,"https://fivethirtyeight.datasettes.com/-/versions reports Datasette version `0.21`

This isn't actually correct. The deploy script for that site actually deploys current master using `https://github.com/simonw/datasette/archive/master.zip`: https://github.com/simonw/fivethirtyeight-datasette/blob/66b4b0dfedd7237bc8c02d3e26d905bca7b84069/Dockerfile#L9

Ideally this would show the current commit hash, but I'm not at all sure if it's possible to derive that from `pip install https://github.com/simonw/datasette/archive/master.zip`. Is there another mechanism that could be used to reliably `pip install` current master but still provide access to the most recent commit hash?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/273/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
324652142,MDU6SXNzdWUzMjQ2NTIxNDI=,274,"Rename --limit to --config, add --help-config",9599,closed,0,,,2,2018-05-19T18:57:42Z,2018-05-20T17:04:55Z,2018-05-20T17:04:11Z,OWNER,,"#270 introduced `--limit` but on further thought it should be called `--config` instead.

`--page_size` should becomes `--config default_page_size:1000`

Add `--help-config` to show full help showing all config settings.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/274/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
324720095,MDU6SXNzdWUzMjQ3MjAwOTU=,275,"""config"" section in metadata.json (root, database and table level)",9599,open,0,,,2,2018-05-20T16:02:28Z,2019-05-13T18:33:31Z,,OWNER,,"Split off from #274 

Metadata should an optional `""config""` section at root, table or database level.

The TableView and RowView and DatabaseView and BaseView classes could all have a `.config(""key"")` method which knows how to resolve the hierarchy of configs.

This will allow individual tables (or databases) to set their own config settings for things like `sql_time_limit_ms`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/275/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
325294102,MDU6SXNzdWUzMjUyOTQxMDI=,278,Build smallest possible Docker image with Datasette plus recent SQLite (with json1) plus Spatialite 4.4.0,9599,closed,0,,,3,2018-05-22T13:28:40Z,2018-05-23T17:43:36Z,2018-05-23T17:43:36Z,OWNER,,"A Dockerfile that does the following:

* Bundles Datasette master
* Python 3.6 most recent version (or 3.7 if it has been released)
* SQLite 3.23.1 (or most recent release) such that ""import sqite3"" in Python gets that version. Ideally with the json1 module baked in by default, but having it loadable as an optional module is fine too
* SpatiaLite 4.4.0-RC0 (or most recent version) such that it can be loaded as an optional module
* Uses multi-stage builds to stay as small as possible

Note that the current ""release"" of SpatiaLite is 4.3.0 which is missing key features like https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN - 4.4.0 probably needs to be compiled from source.

I don't know the best way to get a current SQLite version bundled for Python 3. Maybe https://github.com/coleifer/pysqlite3 ?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/278/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
325705981,MDU6SXNzdWUzMjU3MDU5ODE=,282,Faceting breaks pagination,9599,closed,0,,,1,2018-05-23T13:29:47Z,2018-05-23T13:53:39Z,2018-05-23T13:42:07Z,OWNER,,"e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/nba-elo%2Fnbaallelo?_facet=lg_id#facet-lg_id - click the ""next page"" link: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/nba-elo%2Fnbaallelo?_facet=lg_id&_next=100

    Invalid SQL: near ""and"": syntax error",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/282/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
325958506,MDU6SXNzdWUzMjU5NTg1MDY=,283,Support cross-database joins,9599,closed,0,,,26,2018-05-24T04:18:39Z,2021-06-06T09:40:18Z,2021-02-18T22:16:46Z,OWNER,,"SQLite has the ability to attach multiple databases to a single connection and then run joins across multiple databases.

Since Datasette supports more than one database, this would make a pretty neat feature.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/283/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
326182814,MDU6SXNzdWUzMjYxODI4MTQ=,284,Ability to enable/disable specific features via --config,9599,closed,0,,,5,2018-05-24T15:47:56Z,2018-05-25T06:05:02Z,2018-05-25T05:51:09Z,OWNER,,"`--config` settings from #274 can currently only be integers.

I'd like them to be available as boooeans too. Then we can use them to have that are turned on by default but can be turned off.

First features to get this treatment:

- [x] `allow_sql` - whether or not the `?sql=` parameter is allowed and form is displayed
- [X] `allow_facet` - is `?_facet=` allowed or do we only run facets defined in `metadata.json`
- [X] `allow_download` - do we let users download the full SQLite database file?
- [X] `suggest_facets` - do we attempt to calculate suggested facets?

Refs #275 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/284/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
326189744,MDU6SXNzdWUzMjYxODk3NDQ=,285,num_threads and cache_max_age should be --config options,9599,closed,0,,,2,2018-05-24T16:04:51Z,2018-05-27T00:53:35Z,2018-05-27T00:43:33Z,OWNER,,"https://github.com/simonw/datasette/blob/58b5a37dbbf13868a46bcbb284509434e66eca25/datasette/app.py#L106

And

https://github.com/simonw/datasette/blob/58b5a37dbbf13868a46bcbb284509434e66eca25/datasette/views/base.py#L325

Refs #275 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/285/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
326599525,MDU6SXNzdWUzMjY1OTk1MjU=,286,Database hash should include current datasette version,9599,open,0,,,2,2018-05-25T17:03:42Z,2018-05-25T17:07:36Z,,OWNER,,"Right now deploying a new version of datasette doesn't invalidate existing URLs, so users may still see a cached copy of the old templates.

We can fix this by including the current datasette version in the input to the hash function (which currently just the database file contents).",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/286/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
326617744,MDU6SXNzdWUzMjY2MTc3NDQ=,287,?_shape=arrayfirst,9599,closed,0,,,1,2018-05-25T18:11:03Z,2018-05-27T00:32:53Z,2018-05-27T00:32:29Z,OWNER,,Return an array of single items (the first item in each row returned from the SQL query).,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/287/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
326767626,MDU6SXNzdWUzMjY3Njc2MjY=,288,Support multiple filters of the same type,9599,closed,0,,,3,2018-05-26T21:13:12Z,2019-04-15T23:45:04Z,2019-04-15T23:44:26Z,OWNER,,"This should work for example:

https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/biopics%2Fbiopics?year_release__not=2014&year_release__not=2015",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/288/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
326768188,MDU6SXNzdWUzMjY3NjgxODg=,289,?_ttl= parameter to control caching,9599,closed,0,,,3,2018-05-26T21:22:55Z,2018-05-26T22:22:47Z,2018-05-26T22:17:48Z,OWNER,,"This would allow clients to specify the max-age caching header that should be returned with the query.

Most important this will allow caching to be completely urned off for specific queries using `?_ttl=0`. Sending 0 should cause a `Cache-Control: no-cache` header to be returned.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/289/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
326778161,MDU6SXNzdWUzMjY3NzgxNjE=,290,Consider increasing the default for num_sql_threads (currently 3),9599,open,0,,,0,2018-05-27T00:52:41Z,2018-05-27T00:52:41Z,,OWNER,,"I ran a very rough micro-benchmark on the new `num_sql_threads` config option (added in #285)

    datasette --config num_sql_threads:1 fivethirtyeight.db

Then

    ab -n 100 -c 10 'http://127.0.0.1:8011/fivethirtyeight-2628db9/twitter-ratio%2Fsenators'

| Number of threads | Requests/second |
|---|---|
| 1 | 4.57 |
| 3 | 9.77 |
| 10 | 13.53 |
| 20 | 15.24 
| 50 |  8.21 | 

This was on my early 2018 OS X laptop. Need to benchmark in other common environments before making a decision on changing the default. That said, the default of 3 was a number I plucked out of thin air.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/290/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
326783670,MDU6SXNzdWUzMjY3ODM2NzA=,291,Avoid plugins accidentally loading dependencies twice,9599,closed,0,,,3,2018-05-27T03:15:21Z,2020-09-30T20:36:12Z,2018-05-28T20:42:02Z,OWNER,,Plugins that include JavaScript files risk loading the same code twice. In particular: I want to build a second plugin that uses the Leaflet mapping library (the first was [datasette-cluster-map](https://pypi.org/project/datasette-cluster-map/)). But I don't want the two plugins to load duplicate copies of Leaflet.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/291/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
326800219,MDU6SXNzdWUzMjY4MDAyMTk=,292,Mechanism for customizing the SQL used to select specific columns in the table view,9599,closed,0,,,15,2018-05-27T09:05:52Z,2021-05-27T04:25:01Z,2021-05-27T04:25:01Z,OWNER,,"Some columns don't make a lot of sense in their default representation - binary blobs such as SpatiaLite geometries for example, or lengthy columns that really should be truncated somehow.

We may also find that there are tables where we don't want to show all of the columns - so a mechanism to select a subset of columns would be nice.

I think there are two features here:

* the ability to request a subset of columns on the table view
* the ability to override the SQL for a specific column and/or add extra columns - `AsGeoJSON(Geometry)` for example

Both features should be available via both querystring arguments and in `metadata.json`

The querystring argument for custom SQL should only work if `allow_sql` config is turned on.

Refs #276",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/292/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
327365110,MDU6SXNzdWUzMjczNjUxMTA=,294,inspect should record column types,9599,open,0,,,7,2018-05-29T15:10:41Z,2019-06-28T16:45:28Z,,OWNER,,"For each table we want to know the columns, their order and what type they are.

I'm going to break with SQLite defaults a little on this one and allow datasette to define additional types - to start with just a `geometry` type for columns that are detected as SpatiaLite geometries.

Possible JSON design:

    ""columns"": [{
        ""name"": ""title"",
        ""type"": ""text""
    }, ...]

Refs #276",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/294/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
327383759,MDU6SXNzdWUzMjczODM3NTk=,295,Extract unit tests for inspect out to test_inspect.py,9599,closed,0,,,2,2018-05-29T15:55:04Z,2019-05-11T21:40:32Z,2019-05-11T21:40:32Z,OWNER,,"Right now they are bundled up as API unit tests for a relatively unimportant endpoint. They should be their own thing.

Blocks #294",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/295/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
327395270,MDU6SXNzdWUzMjczOTUyNzA=,296,Per-database and per-table /-/ URL namespace,9599,open,0,,,3,2018-05-29T16:23:13Z,2019-06-28T16:46:34Z,,OWNER,,"Initially this will be for subsets of `/-/inspect` and `/-/metadata` but it will also give us a URL namespace for future features like `/-/facet` (expanded list of a specific facet, linked to from `...`) and `/-/graph`

To start:

* `/dbname/-/inspect`
* `/dbname/-/metadata`
* `/dbname/tablename/-/inspect`
* `/dbname/tablename/-/metadata`

This means we will no longer allow databases or tables to have the name `""-""` - I think that's OK

We will continue to support rows with a primary key of `""-""` at the following URL:

* `/dbname/tablename/-`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/296/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
327420945,MDU6SXNzdWUzMjc0MjA5NDU=,297,datasette publish Dockerfile should use python:3.6-slim-stretch,9599,closed,0,,,1,2018-05-29T17:40:08Z,2018-05-31T14:44:37Z,2018-05-31T14:44:37Z,OWNER,,"Right now the Dockerfile generated by `datasette package` and `datasette publish` uses this:

https://github.com/simonw/datasette/blob/b0a95da96386ddf99816911e08df86178ffa9a89/datasette/utils.py#L269

This appears to result in a SQLite version of `3.8.7.1` - https://parlgov.datasettes.com/-/versions

```
    ""sqlite"": {
        ""extensions"": {},
        ""fts_versions"": [
            ""FTS4"",
            ""FTS3""
        ],
        ""version"": ""3.8.7.1""
    }
```

Meanwhile, https://fivethirtyeight.datasettes.com/-/versions is deployed with this Dockerfile https://github.com/simonw/fivethirtyeight-datasette/blob/0849901cae06e957fe04892cd4033bdcd1fcf966/Dockerfile which uses `FROM python:3.6-slim-stretch` and results in the following version report:

```
    ""sqlite"": {
        ""extensions"": {
            ""json1"": null
        },
        ""fts_versions"": [
            ""FTS5"",
            ""FTS4"",
            ""FTS3""
        ],
        ""version"": ""3.16.2""
    }
```

So not only do we get a more recent SQLite (including https://www.sqlite.org/rowvalue.html added in 3.15) but we also get `FTS5` and `json1` as well.

Refs #191 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/297/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
327459829,MDU6SXNzdWUzMjc0NTk4Mjk=,298,URLify URLs in results from custom SQL statements / views,9599,closed,0,,,2,2018-05-29T19:41:07Z,2018-07-24T04:53:20Z,2018-07-24T03:56:50Z,OWNER,,"Consider this custom query:

https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3?sql=select+user%2C+%28%27https%3A%2F%2Ftwitter.com%2F%27+%7C%7C+user%29+as+user_url%2C+created_at%2C+text%2C+url+from+%5Btwitter-ratio%2Fsenators%5D+limit+10%3B

```select user, ('https://twitter.com/' || user) as user_url, created_at, text, url from [twitter-ratio/senators] limit 10;```

![2018-05-29 at 12 38 pm](https://user-images.githubusercontent.com/9599/40681177-44a36d5c-633d-11e8-935b-c49dad7ac682.png)

It would be nice if these URLs were turned into links, as happens on the table view page: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/twitter-ratio%2Fsenators

![2018-05-29 at 12 39 pm](https://user-images.githubusercontent.com/9599/40681206-5c69c47c-633d-11e8-9f3a-08899f8659b8.png)

This currently does not happen because the table view render logic takes a different path through `display_columns_and_rows()` which includes this bit:

https://github.com/simonw/datasette/blob/b0a95da96386ddf99816911e08df86178ffa9a89/datasette/views/table.py#L195-L202",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/298/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
327461381,MDU6SXNzdWUzMjc0NjEzODE=,299,Documentation covering ALL datasette URLs,9599,closed,0,,,1,2018-05-29T19:46:15Z,2018-07-28T04:24:05Z,2018-07-28T04:22:30Z,OWNER,,"Relates to #296. We need a single page of the docs listing all of the URL patterns Datasette responds to, also detailing which templates are used to render them and linking to examples of the JSON they output when called with `.json`.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/299/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
327541975,MDU6SXNzdWUzMjc1NDE5NzU=,300,Hide sort select box on larger screens,9599,closed,0,,,0,2018-05-30T01:34:59Z,2018-05-31T14:43:13Z,2018-05-31T14:43:13Z,OWNER,,"I'm larger screens you can sort by clicking column headers, so no need to show the select box (which was added for the small screen layout that doesn't show headers)",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/300/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
328155946,MDU6SXNzdWUzMjgxNTU5NDY=,301,"--spatialite option for ""datasette publish heroku""",9599,open,0,,,1,2018-05-31T14:13:09Z,2022-01-20T21:28:50Z,,OWNER,,Split off from #243. Need to figure out how to install and configure SpatiaLite on Heroku.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/301/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
328172521,MDU6SXNzdWUzMjgxNzI1MjE=,303,Support table names ending with .json or .csv,9599,closed,0,,,4,2018-05-31T14:53:23Z,2018-06-15T06:55:50Z,2018-06-15T06:55:50Z,OWNER,,"This is needed for #266 - if a table name ends with `.json` or `.csv` right now our URL pattern matching will do the wrong thing.

We should be smarter about this. This does mean we will have some URLs that look like this:

    http://localhost:8001/dbname/weird.json - returning HTML, not JSON
    http://localhost:8001/dbname/weird.json.json - returning JSON
    http://localhost:8001/dbname/weird.json.csv - returning CSV
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/303/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
328229224,MDU6SXNzdWUzMjgyMjkyMjQ=,304,Ability to configure SQLite cache_size,9599,closed,0,,,3,2018-05-31T17:28:07Z,2018-06-04T16:13:32Z,2018-06-04T16:03:19Z,OWNER,,"See https://www.sqlite.org/pragma.html#pragma_cache_size

Let's call the config setting `cache_size_kb` to emphasize that we're using the negative option.

Note this warning: perhaps we should raise an error if you try to use this setting against a SQLite version prior to 3.7.10
> If the argument N is positive then the suggested cache size is set to N. If the argument N is negative, then the number of cache pages is adjusted to use approximately abs(N*1024) bytes of memory. Backwards compatibility note: The behavior of cache_size with a negative N was different in prior to version 3.7.10 (2012-01-16). In version 3.7.9 and earlier, the number of pages in the cache was set to the absolute value of N.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/304/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
329147284,MDU6SXNzdWUzMjkxNDcyODQ=,305,Add contributor guidelines to docs,9599,closed,0,,,2,2018-06-04T17:25:30Z,2019-06-24T06:40:19Z,2019-06-24T06:40:19Z,OWNER,,https://channels.readthedocs.io/en/latest/contributing.html is a nice example of this done well.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/305/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
329661905,MDU6SXNzdWUzMjk2NjE5MDU=,306,Custom URL routing with independent tests,9599,closed,0,,,5,2018-06-05T23:40:08Z,2018-06-07T15:29:28Z,2018-06-07T15:29:28Z,OWNER,,"The more I think about #303 the more I feel that Datasette's URL routing needs go beyond Django-style regex matching.

If we go custom, tests should live in `test_routing.py`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/306/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
330323860,MDExOlB1bGxSZXF1ZXN0MTkzMzYxMzQx,307,"Initial sketch of custom URL routing, refs #306",9599,closed,0,,,1,2018-06-07T15:26:48Z,2018-06-07T15:29:54Z,2018-06-07T15:29:41Z,OWNER,simonw/datasette/pulls/307,See #306 for background on this.,107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/307/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
332830309,MDU6SXNzdWUzMzI4MzAzMDk=,310,datasette publish now is broken in master,9599,closed,0,,,0,2018-06-15T16:01:14Z,2018-06-16T16:29:50Z,2018-06-16T16:29:50Z,OWNER,,"```
>     gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2
>     unable to execute 'gcc': No such file or directory
>     error: command 'gcc' failed with exit status 1
>     
>     ----------------------------------------
> Command ""/usr/local/bin/python -u -c ""import setuptools, tokenize;__file__='/tmp/pip-install-s73273rj/httptools/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))"" install --record /tmp/pip-record-yha7dxqq/install-record.txt --single-version-externally-managed --compile"" failed with error code 1 in /tmp/pip-install-s73273rj/httptools/
```

Turns out the `python-slim` base image I introduced in b18e4515855c3f1eeca3dfcccdbb6df05869084a doesn't include gcc: https://github.com/docker-library/python/issues/60#issuecomment-134322383",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/310/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
332998752,MDExOlB1bGxSZXF1ZXN0MTk1MzM5MTEx,311,"?_labels=1 to expand foreign keys (in csv and json), refs #233",9599,closed,0,,,2,2018-06-16T16:31:12Z,2018-06-16T22:20:31Z,2018-06-16T22:20:31Z,OWNER,simonw/datasette/pulls/311,"Output looks something like this:

    {
        ""rowid"": 233,
        ""TreeID"": 121240,
        ""qLegalStatus"": {
            ""value"" 2,
            ""label"": ""Private""
        }
        ""qSpecies"": {
            ""value"": 16,
            ""label"": ""Sycamore""
        }
        ""qAddress"": ""91 Commonwealth Ave"",
        ...
    }",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/311/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
333000163,MDU6SXNzdWUzMzMwMDAxNjM=,312,"HTML, CSV and JSON views should support ?_col=&_col=",9599,closed,0,,,1,2018-06-16T16:53:35Z,2021-06-17T18:14:24Z,2018-06-16T17:00:12Z,OWNER,,To support whitelisting columns to display.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/312/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
333086005,MDU6SXNzdWUzMzMwODYwMDU=,313,Deploy demo of Datasette on every commit that passes tests,9599,closed,0,,,6,2018-06-17T19:19:12Z,2018-06-17T21:52:58Z,2018-06-17T21:52:58Z,OWNER,,We can use Travis CI and Zeit Now to ensure there is always a live demo of current master. We can ship archived demos for releases as well.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/313/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
333120982,MDExOlB1bGxSZXF1ZXN0MTk1NDEzMjQx,315,Streaming mode for downloading all rows as a CSV,9599,closed,0,,,0,2018-06-18T03:06:59Z,2018-06-18T03:29:13Z,2018-06-18T03:21:02Z,OWNER,simonw/datasette/pulls/315,Refs #266,107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/315/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
334169932,MDU6SXNzdWUzMzQxNjk5MzI=,320,Need unit tests covering the different states for the advanced export box,9599,closed,0,,,1,2018-06-20T17:03:40Z,2018-07-24T04:53:20Z,2018-07-24T03:38:40Z,OWNER,,"There are quite a few variants of this box:

![2018-06-20 at 10 02 am](https://user-images.githubusercontent.com/9599/41673229-1d423adc-7471-11e8-99d4-4251f7d03aa5.png)

Test coverage should exercise all of them, since the logic is a little unclear.

https://github.com/simonw/datasette/blob/fdfbbbb9ee0d02fd4d43dfc42382252fa2287d6d/datasette/templates/table.html#L140-L159",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/320/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
334698969,MDU6SXNzdWUzMzQ2OTg5Njk=,323,Speed up Travis CI builds,9599,closed,0,,,1,2018-06-21T23:55:27Z,2018-07-10T15:03:37Z,2018-07-10T15:03:36Z,OWNER,,"They've got a bit slow. Part of this is the Zeit Now deploy, but the build-and-test cycle is taking at least a couple of minutes.

![2018-06-21 at 4 54 pm](https://user-images.githubusercontent.com/9599/41751010-e48c823e-7573-11e8-88f3-7aa8a7e53917.png)
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/323/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
334731076,MDExOlB1bGxSZXF1ZXN0MTk2NjI4MzA0,324,Speed up Travis by reusing pip wheel cache across builds,9599,closed,0,,,0,2018-06-22T03:20:08Z,2018-06-24T01:03:47Z,2018-06-24T01:03:47Z,OWNER,simonw/datasette/pulls/324,From https://atchai.com/blog/faster-ci/ - refs #323 ,107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/324/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
335064777,MDU6SXNzdWUzMzUwNjQ3Nzc=,325,Error on row page if table has slashes in the name and ends in .csv,9599,closed,0,,,1,2018-06-23T03:43:42Z,2018-07-09T17:28:27Z,2018-07-08T05:21:59Z,OWNER,,"https://v0-23-1.datasette.io/fixtures-e14e080/table%252Fwith%252Fslashes.csv/3

> no such table: table%252Fwith%252Fslashes.csv

From clicking the row link on https://v0-23-1.datasette.io/fixtures-e14e080/table%2Fwith%2Fslashes.csv",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/325/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
335141434,MDU6SXNzdWUzMzUxNDE0MzQ=,326,CSV should respect --cors and return cors headers,9599,closed,0,,,1,2018-06-24T00:44:07Z,2021-06-17T18:14:24Z,2018-06-24T00:59:45Z,OWNER,,Otherwise tools like Vega can't load data via CSV.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/326/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
335200136,MDU6SXNzdWUzMzUyMDAxMzY=,327,Explore if SquashFS can be used to shrink size of packaged Docker containers,9599,open,0,,,4,2018-06-24T18:15:16Z,2022-02-17T23:37:24Z,,OWNER,,"Inspired by this article: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html#sqlite-database-indexed--squashed

https://en.wikipedia.org/wiki/SquashFS is ""a compressed read-only file system for Linux"" - which means it could be a really nice fit for Datasette and its read-only SQLite databases.

It would be interesting to explore a Dockerfile recipe that used SquashFS to compress the SQLite database file that was bundled up by `datasette package` and friends.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/327/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
336464733,MDU6SXNzdWUzMzY0NjQ3MzM=,328,"Installation instructions, including how to use the docker image",9599,closed,0,,,3,2018-06-28T03:59:33Z,2018-10-05T06:37:07Z,2018-06-28T04:02:10Z,OWNER,,,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/328/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
336465018,MDU6SXNzdWUzMzY0NjUwMTg=,329,Travis should push tagged images to Docker Hub for each release,9599,closed,0,,,7,2018-06-28T04:01:31Z,2018-11-05T06:54:10Z,2018-11-05T06:53:28Z,OWNER,,https://sebest.github.io/post/using-travis-ci-to-build-docker-images/,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/329/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
337141108,MDU6SXNzdWUzMzcxNDExMDg=,332,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,9599,closed,0,,,12,2018-06-29T21:21:27Z,2018-07-24T04:53:20Z,2018-07-24T03:08:30Z,OWNER,,"It turns out if you load this CSV using `csvs-to-sqlite` you get an Infinity value in SQLite:

```
name,num
sasha,10
terry,Inf
cathy,0.5
```

`csvs-to-sqlite infinity-bug.csv infinity-bug.db`

I deployed this using:

```
datasette publish now infinity-bug.db --name=datasette-infinity-bug --install=datasette-vega
```

Datasette outputs that as `Infinity` in the JSON format, which causes JavaScript errors.

Demo
* https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug - HTML view works
* https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug.json?_shape=array - this outputs the following:

```
[
    {
        ""rowid"": 1,
        ""name"": ""sasha"",
        ""num"": 10.0
    },
    {
        ""rowid"": 2,
        ""name"": ""terry"",
        ""num"": Infinity
    },
    {
        ""rowid"": 3,
        ""name"": ""cathy"",
        ""num"": 0.5
    }
]
```

But... in Firefox that gets rendered like this:

![2018-06-29 at 4 20 pm](https://user-images.githubusercontent.com/9599/42115408-5d30f630-7bb8-11e8-8370-c8484801c49b.png)

And if you click the ""Show charting options"" button you get this error in the console:

```
SyntaxError: JSON.parse: unexpected character at line 1 column 83 of the JSON data
```
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/332/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
338768551,MDU6SXNzdWUzMzg3Njg1NTE=,333,Datasette on Zeit Now returns http URLs for facet and next links,9599,closed,0,,,4,2018-07-06T00:40:49Z,2018-07-24T04:53:20Z,2018-07-24T01:51:53Z,OWNER,,"e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=0
```
{
  ""facet_results"": {
    ""lg_id"": {
      ""name"": ""lg_id"",
      ""results"": [
        {
          ""value"": ""NBA"",
          ""label"": ""NBA"",
          ""count"": 118016,
          ""toggle_url"": ""http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=NBA"",
          ""selected"": false
        },
        {
          ""value"": ""ABA"",
          ""label"": ""ABA"",
          ""count"": 8298,
          ""toggle_url"": ""http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=ABA"",
          ""selected"": false
        }
      ],
      ""truncated"": false
    }
  },
  ""suggested_facets"": [
    {
      ""name"": ""_iscopy"",
      ""toggle_url"": ""/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_facet=_iscopy""
    }
  ],
  ""next_url"": ""http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_next=1"",
}
```
`next_url` and `facet_results` both link to `http://` when they should link to `https://`.

Note that suggested facets doesn't include the full URL at all, which is a consistency bug.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/333/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
339505204,MDU6SXNzdWUzMzk1MDUyMDQ=,335,Package datasette for installation using homebrew,9599,closed,0,,,12,2018-07-09T15:45:03Z,2020-08-11T16:54:06Z,2020-08-11T16:54:06Z,OWNER,,"https://docs.brew.sh/Python-for-Formula-Authors describes how.

> Applications should be installed into a Python virtualenv environment rooted in libexec. This prevents the app’s Python modules from contaminating the system site-packages and vice versa.

It recommends using https://github.com/tdsmith/homebrew-pypi-poet",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/335/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
340039409,MDU6SXNzdWUzNDAwMzk0MDk=,336,Ensure --help examples in docs are always up to date,9599,closed,0,,,3,2018-07-10T23:20:01Z,2018-07-24T16:01:29Z,2018-07-24T16:01:29Z,OWNER,,"Ideally I would automatically generate the --help output shown in our docs, but I don't think I can get that working with readthedocs.

Instead, I'm going to add a unit test that checks that those extracts in the documentation match the current output of the --help command.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/336/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
340065374,MDU6SXNzdWUzNDAwNjUzNzQ=,337,Documentation for datasette publish and datasette package,9599,closed,0,,,1,2018-07-11T02:04:06Z,2018-07-11T02:07:32Z,2018-07-11T02:05:56Z,OWNER,,,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/337/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
340282796,MDU6SXNzdWUzNDAyODI3OTY=,338,Only load vegaEmbed if charting tools are enabled,9599,closed,0,,,1,2018-07-11T15:02:14Z,2018-07-11T15:21:47Z,2018-07-11T15:21:47Z,OWNER,,"vegaEmbed is a LOT of code (it bundles d3)

Inspired by this tweet: https://twitter.com/thelarkinn/status/1017053567641948162 - it would be great if we loaded that code on demand the first time the ""Show chart options"" button was clicked, or when the page loads with #g. options in the URL.

Even better: avoid the overhead if loading React unless the chart options need to be displayed. This would be a pretty major refactoring though.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/338/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
340730961,MDU6SXNzdWUzNDA3MzA5NjE=,340,Embrace black,9599,closed,0,,,1,2018-07-12T17:32:29Z,2019-06-24T06:50:27Z,2019-06-24T06:50:26Z,OWNER,,"Run [black](https://github.com/ambv/black) against everything. Then set up CI to fail if code doesn't conform to black's style.

Here's how Starlette does this:

* https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/.travis.yml#L14
* https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/scripts/lint - essentially runs `black starlette tests --check`

And here's an example of a test run that failed:  https://travis-ci.org/encode/starlette/jobs/403172478",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/340/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
340733753,MDExOlB1bGxSZXF1ZXN0MjAxMDc1NTMy,341,Bump aiohttp to fix compatibility with Python 3.7,9599,closed,0,,,0,2018-07-12T17:41:24Z,2018-07-12T18:07:38Z,2018-07-12T18:07:38Z,OWNER,simonw/datasette/pulls/341,Tests failed here: https://travis-ci.org/simonw/datasette/jobs/403223333,107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/341/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
344654623,MDU6SXNzdWUzNDQ2NTQ2MjM=,347,"Rename ""datasette package"" to ""datasette publish docker""",9599,open,0,,,0,2018-07-26T00:42:46Z,2018-07-26T00:42:46Z,,OWNER,,,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/347/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
344656114,MDU6SXNzdWUzNDQ2NTYxMTQ=,348,"Unit tests for ""datasette publish""",9599,closed,0,,,1,2018-07-26T00:52:23Z,2018-07-26T05:46:10Z,2018-07-26T05:46:10Z,OWNER,,"The datasette publish family of commands all work by shelling out to heroku/now/docker

    from subprocess import call, check_output

So in tests I should be able to mock those calls:

    @mock.patch('subprocess.call')
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/348/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
344695978,MDExOlB1bGxSZXF1ZXN0MjA0MDI5MTQy,349,"publish_subcommand hook + default plugins mechanism, used for publish heroku/now",9599,closed,0,,,1,2018-07-26T05:03:22Z,2018-07-26T05:28:54Z,2018-07-26T05:16:00Z,OWNER,simonw/datasette/pulls/349,"This change introduces a new plugin hook, publish_subcommand, which can be
used to implement new subcommands for the ""datasette publish"" command family.

I've used this new hook to refactor out the ""publish now"" and ""publish heroku""
implementations into separate modules. I've also added unit tests for these
two publishers, mocking the subprocess.call and subprocess.check_output
functions.

As part of this, I introduced a mechanism for loading default plugins. These
are defined in the new ""default_plugins"" list inside datasette/app.py

Closes #217 (Plugin support for ""datasette publish"")
Closes #348 (Unit tests for ""datasette publish"")
Refs #14, #59, #102, #103, #146, #236, #347",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/349/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
344701755,MDU6SXNzdWUzNDQ3MDE3NTU=,350,Don't list default plugins on /-/plugins,9599,closed,0,,,2,2018-07-26T05:38:00Z,2018-08-28T17:13:50Z,2018-08-28T16:48:19Z,OWNER,,"https://dbbe707.datasette.io/-/plugins is showing ""datasette.publish.now"" and ""datasette.publish.heroku""",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/350/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
345469355,MDU6SXNzdWUzNDU0NjkzNTU=,351,Automatically create a GitHub release linking to release notes for every tagged release,9599,closed,0,,,1,2018-07-28T18:31:12Z,2020-05-28T18:56:16Z,2020-05-28T18:56:15Z,OWNER,,"Can use this API called from Travis: https://developer.github.com/v3/repos/releases/#create-a-release

The release it generates should look like this one: https://github.com/simonw/datasette/releases/tag/0.24",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/351/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
345821500,MDU6SXNzdWUzNDU4MjE1MDA=,352,render_cell(value) plugin hook,9599,closed,0,,,4,2018-07-30T15:56:20Z,2020-02-10T16:18:58Z,2018-08-05T00:14:57Z,OWNER,,To allow plugins to customize how values matching a specific pattern are displayed in the HTML table view.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/352/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
345821778,MDExOlB1bGxSZXF1ZXN0MjA0ODUxNTEx,353,render_cell(value) plugin hook,9599,closed,0,,,0,2018-07-30T15:57:08Z,2018-08-05T00:14:57Z,2018-08-05T00:14:57Z,OWNER,simonw/datasette/pulls/353,Closes #352.,107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/353/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
346026869,MDU6SXNzdWUzNDYwMjY4Njk=,354,Handle many-to-many relationships,9599,open,0,,,0,2018-07-31T04:03:13Z,2020-11-24T19:51:18Z,,OWNER,,This is a master tracking ticket for various many-2-many features.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/354/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
346027040,MDU6SXNzdWUzNDYwMjcwNDA=,355,Table view should support filtering via many-to-many relationships,9599,open,0,,,10,2018-07-31T04:04:16Z,2019-05-23T06:04:03Z,,OWNER,,Parent: #354 ,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/355/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
346028655,MDU6SXNzdWUzNDYwMjg2NTU=,356,Ability to display facet counts for many-to-many relationships,9599,closed,0,,,4,2018-07-31T04:14:26Z,2019-05-29T21:39:12Z,2019-05-25T16:30:09Z,OWNER,,Parent: #354,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/356/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
347058326,MDExOlB1bGxSZXF1ZXN0MjA1NzcwOTk2,1,Make .indexes compatible with older SQLite versions,9599,closed,0,,,0,2018-08-02T15:17:05Z,2018-08-02T15:17:30Z,2018-08-02T15:17:30Z,OWNER,simonw/sqlite-utils/pulls/1,Older SQLite versions return a different set of columns from the PRAGMA we are using.,140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
348043884,MDU6SXNzdWUzNDgwNDM4ODQ=,357,Plugin hook for loading metadata.json,9599,open,0,,,6,2018-08-06T19:00:01Z,2020-06-21T22:19:58Z,,OWNER,,"For https://github.com/simonw/russian-ira-facebook-ads-datasette/tree/af6d956995e14afd585c35a6a06bb01da32043ba I wrote a script to convert YAML to JSON because YAML is a better format for embedding multi-line HTML descriptions and canned SQL statements.

Example yaml metadata file: https://github.com/simonw/russian-ira-facebook-ads-datasette/blob/af6d956995e14afd585c35a6a06bb01da32043ba/russian-ads-metadata.yaml

It would be useful if Datasette could be fed a YAML file directly:

    datasette -m metadata.yaml

Question is... should this be a native feature (hence adding a YAML dependency) or should it be handled by a `datasette-metadata-yaml` plugin, using a new plugin hook for loading metadata? If so, what would other use-cases for that plugin hook be?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/357/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,