from pluggy import HookimplMarker
ModuleNotFoundError: No module named 'pluggy'
```
Looks like I've run into point 6 on https://packaging.python.org/guides/single-sourcing-package-version/ :
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",267707940,Datasette Plugins,
https://github.com/simonw/datasette/issues/139#issuecomment-381455054,https://api.github.com/repos/simonw/datasette/issues/139,381455054,MDEyOklzc3VlQ29tbWVudDM4MTQ1NTA1NA==,9599,simonw,2018-04-16T01:24:13Z,2018-04-16T01:24:13Z,OWNER,"I think Vega-Lite is the way to go here: https://vega.github.io/vega-lite/
I've been playing around with it and Datasette with some really positive initial results:
https://vega.github.io/editor/#/gist/vega-lite/simonw/89100ce80573d062d70f780d10e5e609/decada131575825875c0a076e418c661c2adb014/vice-shootings-gender-race-by-department.vl.json
https://vega.github.io/editor/#/gist/vega-lite/simonw/5f69fbe29380b0d5d95f31a385f49ee4/7087b64df03cf9dba44a5258a606f29182cb8619/trees-san-francisco.vl.json","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275493851,Build a visualization plugin for Vega,
https://github.com/simonw/datasette/issues/211#issuecomment-381456434,https://api.github.com/repos/simonw/datasette/issues/211,381456434,MDEyOklzc3VlQ29tbWVudDM4MTQ1NjQzNA==,9599,simonw,2018-04-16T01:36:16Z,2018-04-16T01:37:44Z,OWNER,"The easiest way to implement this in Python 2 would be `execfile(...)` - but that was removed in Python 3. According to https://stackoverflow.com/a/437857/6083 `2to3` replaces that with this, which ensures the filename is associated with the code for debugging purposes:
```
with open(""somefile.py"") as f:
code = compile(f.read(), ""somefile.py"", 'exec')
exec(code, global_vars, local_vars)
```
Implementing it this way would force this kind of plugin to be self-contained in a single file. I think that's OK: if you want a more complex plugin you can use the standard pluggy-powered setuptools mechanism to build it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314471743,Load plugins from a `--plugins-dir=plugins/` directory,
https://github.com/simonw/datasette/issues/211#issuecomment-381462005,https://api.github.com/repos/simonw/datasette/issues/211,381462005,MDEyOklzc3VlQ29tbWVudDM4MTQ2MjAwNQ==,9599,simonw,2018-04-16T02:23:07Z,2018-04-16T02:23:07Z,OWNER,This needs unit tests. I also need to manually test the `datasette package` and `datesette publish` commands.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314471743,Load plugins from a `--plugins-dir=plugins/` directory,
https://github.com/simonw/datasette/issues/211#issuecomment-381478217,https://api.github.com/repos/simonw/datasette/issues/211,381478217,MDEyOklzc3VlQ29tbWVudDM4MTQ3ODIxNw==,9599,simonw,2018-04-16T04:41:38Z,2018-04-16T04:41:38Z,OWNER,"Here's the result of running:
datasette publish now fivethirtyeight.db \
--plugins-dir=plugins/ --title=""FiveThirtyEight"" --branch=plugins-dir
https://datasette-phjtvzwwzl.now.sh/fivethirtyeight-2628db9?sql=select+convert_units%28100%2C+%27m%27%2C+%27ft%27%29
Where `plugins/pint_plugin.py` contains the following:
```
from datasette import hookimpl
import pint
ureg = pint.UnitRegistry()
@hookimpl
def prepare_connection(conn):
def convert_units(amount, from_, to_):
""select convert_units(100, 'm', 'ft');""
return (amount * ureg(from_)).to(to_).to_tuple()[0]
conn.create_function('convert_units', 3, convert_units)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314471743,Load plugins from a `--plugins-dir=plugins/` directory,
https://github.com/simonw/datasette/issues/211#issuecomment-381478253,https://api.github.com/repos/simonw/datasette/issues/211,381478253,MDEyOklzc3VlQ29tbWVudDM4MTQ3ODI1Mw==,9599,simonw,2018-04-16T04:42:02Z,2018-04-16T04:42:02Z,OWNER,"This worked as well:
datasette package fivethirtyeight.db \
--plugins-dir=plugins/ --title=""FiveThirtyEight"" --branch=plugins-dir
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314471743,Load plugins from a `--plugins-dir=plugins/` directory,
https://github.com/simonw/datasette/issues/211#issuecomment-381481990,https://api.github.com/repos/simonw/datasette/issues/211,381481990,MDEyOklzc3VlQ29tbWVudDM4MTQ4MTk5MA==,9599,simonw,2018-04-16T05:14:57Z,2018-04-16T05:14:57Z,OWNER,Added unit tests in 33c6bcadb962457be6b0c7f369826b404e2bcef5,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314471743,Load plugins from a `--plugins-dir=plugins/` directory,
https://github.com/simonw/datasette/issues/211#issuecomment-381482407,https://api.github.com/repos/simonw/datasette/issues/211,381482407,MDEyOklzc3VlQ29tbWVudDM4MTQ4MjQwNw==,9599,simonw,2018-04-16T05:18:29Z,2018-04-16T05:18:29Z,OWNER,"Here's the result of running this:
datasette publish heroku fivethirtyeight.db \
--plugins-dir=plugins/ --title=""FiveThirtyEight"" --branch=plugins-dir
https://intense-river-24599.herokuapp.com/fivethirtyeight-2628db9?sql=select+convert_units%28100%2C+%27m%27%2C+%27ft%27%29","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314471743,Load plugins from a `--plugins-dir=plugins/` directory,
https://github.com/simonw/datasette/pull/209#issuecomment-381483301,https://api.github.com/repos/simonw/datasette/issues/209,381483301,MDEyOklzc3VlQ29tbWVudDM4MTQ4MzMwMQ==,9599,simonw,2018-04-16T05:25:08Z,2018-04-16T05:25:08Z,OWNER,I think this is a good improvement. If you fix the tests I'll merge it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314455877, Don't duplicate simple primary keys in the link column,
https://github.com/simonw/datasette/issues/14#issuecomment-381446392,https://api.github.com/repos/simonw/datasette/issues/14,381446392,MDEyOklzc3VlQ29tbWVudDM4MTQ0NjM5Mg==,9599,simonw,2018-04-15T23:22:40Z,2018-04-16T05:25:57Z,OWNER,"OK, from that prototype in f2720b0c6b7172ebe8820 it looks like pluggy provides a solid path forward.
Next steps:
- [x] Build a demo plugin that uses setuptools entrypoints to register with the `datasette` plugin manager via pluggy
- [x] Figure out a mechanism for registering plugins without first needing to publish them to PyPI. Can I load plugins from a special `plugins/` directory similar to the `--template-dir=templates/` option already supported by Datasette? #211","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",267707940,Datasette Plugins,
https://github.com/simonw/datasette/issues/191#issuecomment-381488049,https://api.github.com/repos/simonw/datasette/issues/191,381488049,MDEyOklzc3VlQ29tbWVudDM4MTQ4ODA0OQ==,9599,simonw,2018-04-16T05:58:15Z,2018-04-16T05:58:15Z,OWNER,"I think this is pretty hard. @coleifer has done some work in this direction, including https://github.com/coleifer/pysqlite3 which ports the standalone pysqlite module to Python 3. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",310533258,Figure out how to bundle a more up-to-date SQLite,
https://github.com/simonw/datasette/issues/214#issuecomment-381490361,https://api.github.com/repos/simonw/datasette/issues/214,381490361,MDEyOklzc3VlQ29tbWVudDM4MTQ5MDM2MQ==,9599,simonw,2018-04-16T06:13:02Z,2018-04-16T06:13:02Z,OWNER,"Packaging JS and CSS in a pip installable wheel is fiddly but possible. http://peak.telecommunity.com/DevCenter/PythonEggs#accessing-package-resources
from pkg_resources import resource_string
foo_config = resource_string(__name__, 'foo.conf')","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314506446,Ability for plugins to define extra JavaScript and CSS,
https://github.com/simonw/datasette/issues/214#issuecomment-381491707,https://api.github.com/repos/simonw/datasette/issues/214,381491707,MDEyOklzc3VlQ29tbWVudDM4MTQ5MTcwNw==,9599,simonw,2018-04-16T06:21:23Z,2018-04-16T06:21:23Z,OWNER,This looks like a good example: https://github.com/funkey/nyroglancer/commit/d4438ab42171360b2b8e9020f672846dd70c8d80,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314506446,Ability for plugins to define extra JavaScript and CSS,
https://github.com/simonw/datasette/issues/191#issuecomment-381602005,https://api.github.com/repos/simonw/datasette/issues/191,381602005,MDEyOklzc3VlQ29tbWVudDM4MTYwMjAwNQ==,119974,coleifer,2018-04-16T13:37:32Z,2018-04-16T13:37:32Z,NONE,I don't think it should be too difficult... you can look at what @ghaering did with pysqlite (and similarly what I copied for pysqlite3). You would theoretically take an amalgamation build of Sqlite (all code in a single .c and .h file). The `AmalgamationLibSqliteBuilder` class detects the presence of this amalgamated source file and builds a statically-linked pysqlite.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",310533258,Figure out how to bundle a more up-to-date SQLite,
https://github.com/simonw/datasette/issues/14#issuecomment-381611738,https://api.github.com/repos/simonw/datasette/issues/14,381611738,MDEyOklzc3VlQ29tbWVudDM4MTYxMTczOA==,9599,simonw,2018-04-16T14:07:30Z,2018-04-16T14:07:30Z,OWNER,I should check if it's possible to have two template registration function plugins in a single plugin module. If it isn't maybe I should use class plugins instead of module plugins.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",267707940,Datasette Plugins,
https://github.com/simonw/datasette/issues/214#issuecomment-381612585,https://api.github.com/repos/simonw/datasette/issues/214,381612585,MDEyOklzc3VlQ29tbWVudDM4MTYxMjU4NQ==,9599,simonw,2018-04-16T14:10:16Z,2018-04-16T14:10:16Z,OWNER,`resource_stream` returns a file-like object which may be better for serving from Sanic.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314506446,Ability for plugins to define extra JavaScript and CSS,
https://github.com/simonw/datasette/issues/14#issuecomment-381621338,https://api.github.com/repos/simonw/datasette/issues/14,381621338,MDEyOklzc3VlQ29tbWVudDM4MTYyMTMzOA==,9599,simonw,2018-04-16T14:36:27Z,2018-04-16T14:36:27Z,OWNER,"Annoyingly, the following only results in the last of the two `prepare_connection` hooks being registered:
```
from datasette import hookimpl
import pint
import random
ureg = pint.UnitRegistry()
@hookimpl
def prepare_connection(conn):
def convert_units(amount, from_, to_):
""select convert_units(100, 'm', 'ft');""
return (amount * ureg(from_)).to(to_).to_tuple()[0]
conn.create_function('convert_units', 3, convert_units)
@hookimpl
def prepare_connection(conn):
conn.create_function('random_integer', 2, random.randint)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",267707940,Datasette Plugins,
https://github.com/simonw/datasette/issues/216#issuecomment-381643173,https://api.github.com/repos/simonw/datasette/issues/216,381643173,MDEyOklzc3VlQ29tbWVudDM4MTY0MzE3Mw==,9599,simonw,2018-04-16T15:21:17Z,2018-04-16T15:21:17Z,OWNER,"Yikes, definitely a bug.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381644355,https://api.github.com/repos/simonw/datasette/issues/216,381644355,MDEyOklzc3VlQ29tbWVudDM4MTY0NDM1NQ==,9599,simonw,2018-04-16T15:24:38Z,2018-04-16T15:24:38Z,OWNER,"So there are two tricky problems to solve here:
* I need a way of encoding `null` into that `_next=` that is unambiguous from the string `None` or `null`. This means introducing some kind of escaping mechanism in those strings. I already use URL encoding as part of the construction of those components here, maybe that can help here?
* I need to figure out what the SQL should be for the ""next"" set of results if the previous value was null. Thankfully we use the primary key as a tie-breaker so this shouldn't be impossible.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381645274,https://api.github.com/repos/simonw/datasette/issues/216,381645274,MDEyOklzc3VlQ29tbWVudDM4MTY0NTI3NA==,9599,simonw,2018-04-16T15:27:16Z,2018-04-16T15:27:16Z,OWNER,"Relevant code:
https://github.com/simonw/datasette/blob/904f1c75a3c17671d25c53b91e177c249d14ab3b/datasette/app.py#L828-L832","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381645973,https://api.github.com/repos/simonw/datasette/issues/216,381645973,MDEyOklzc3VlQ29tbWVudDM4MTY0NTk3Mw==,9599,simonw,2018-04-16T15:29:11Z,2018-04-16T15:29:11Z,OWNER,"I could use `$null` as a magic value that means None. Since I'm applying `quote_plus()` to actual values, any legit strings that look like this will be encoded as `%24null`:
```
>>> urllib.parse.quote_plus('$null')
'%24null'
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381648053,https://api.github.com/repos/simonw/datasette/issues/216,381648053,MDEyOklzc3VlQ29tbWVudDM4MTY0ODA1Mw==,9599,simonw,2018-04-16T15:35:17Z,2018-04-16T15:35:17Z,OWNER,"I think the correct SQL is this: https://datasette-issue-189-demo-3.now.sh/salaries-7859114-7859114?sql=select+rowid%2C+*+from+%5B2017+Maryland+state+salaries%5D%0D%0Awhere+%28middle_initial+is+not+null+or+%28middle_initial+is+null+and+rowid+%3E+%3Ap0%29%29%0D%0Aorder+by+middle_initial+limit+101&p0=391
```
select rowid, * from [2017 Maryland state salaries]
where (middle_initial is not null or (middle_initial is null and rowid > :p0))
order by middle_initial limit 101
```
Though this will also need to be taken into account for #198 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381649140,https://api.github.com/repos/simonw/datasette/issues/216,381649140,MDEyOklzc3VlQ29tbWVudDM4MTY0OTE0MA==,9599,simonw,2018-04-16T15:38:29Z,2018-04-16T15:38:29Z,OWNER,But what would that SQL look like for `_sort_desc`?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381649437,https://api.github.com/repos/simonw/datasette/issues/216,381649437,MDEyOklzc3VlQ29tbWVudDM4MTY0OTQzNw==,9599,simonw,2018-04-16T15:39:21Z,2018-04-16T15:39:21Z,OWNER,"Here's where that SQL gets constructed at the moment:
https://github.com/simonw/datasette/blob/10a34f995c70daa37a8a2aa02c3135a4b023a24c/datasette/app.py#L761-L771","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/pull/209#issuecomment-381738137,https://api.github.com/repos/simonw/datasette/issues/209,381738137,MDEyOklzc3VlQ29tbWVudDM4MTczODEzNw==,45057,russss,2018-04-16T20:27:43Z,2018-04-16T20:27:43Z,CONTRIBUTOR,"Tests now fixed, honest. The failing test on Travis looks like an intermittent sqlite failure which should resolve itself on a retry...","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314455877, Don't duplicate simple primary keys in the link column,
https://github.com/simonw/datasette/issues/203#issuecomment-381763651,https://api.github.com/repos/simonw/datasette/issues/203,381763651,MDEyOklzc3VlQ29tbWVudDM4MTc2MzY1MQ==,45057,russss,2018-04-16T21:59:17Z,2018-04-16T21:59:17Z,CONTRIBUTOR,"Ah, I had no idea you could bind python functions into sqlite!
I think the primary purpose of this issue has been served now - I'm going to close this and create a new issue for the only bit of this that hasn't been touched yet, which is (optionally) exposing units in the JSON API.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",313837303,Support for units,
https://github.com/simonw/datasette/issues/220#issuecomment-381777108,https://api.github.com/repos/simonw/datasette/issues/220,381777108,MDEyOklzc3VlQ29tbWVudDM4MTc3NzEwOA==,9599,simonw,2018-04-16T23:04:04Z,2018-04-16T23:04:04Z,OWNER,This could also help workaround the current predicament that a single plugin can only define one prepare_connection hook.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314847571,Investigate syntactic sugar for plugins,
https://github.com/simonw/datasette/issues/216#issuecomment-381786522,https://api.github.com/repos/simonw/datasette/issues/216,381786522,MDEyOklzc3VlQ29tbWVudDM4MTc4NjUyMg==,9599,simonw,2018-04-16T23:58:45Z,2018-04-16T23:59:13Z,OWNER,"Weird... tests are failing in Travis, despite passing on my local machine. https://travis-ci.org/simonw/datasette/builds/367423706","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381788051,https://api.github.com/repos/simonw/datasette/issues/216,381788051,MDEyOklzc3VlQ29tbWVudDM4MTc4ODA1MQ==,9599,simonw,2018-04-17T00:07:48Z,2018-04-17T00:07:48Z,OWNER,Still failing. This is very odd.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381794744,https://api.github.com/repos/simonw/datasette/issues/216,381794744,MDEyOklzc3VlQ29tbWVudDM4MTc5NDc0NA==,9599,simonw,2018-04-17T00:51:41Z,2018-04-17T00:51:41Z,OWNER,I'm reverting this out of master until I can figure out why the tests are failing.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381798786,https://api.github.com/repos/simonw/datasette/issues/216,381798786,MDEyOklzc3VlQ29tbWVudDM4MTc5ODc4Ng==,9599,simonw,2018-04-17T01:18:25Z,2018-04-17T01:18:25Z,OWNER,"Here's the test that's failing:
https://github.com/simonw/datasette/blob/59a3aa859c0e782aeda9a515b1b52c358e8458a2/tests/test_api.py#L437-L470
I got Travis to spit out the `fetched` and `expected` variables.
`expected` has 201 items in it and is identical to what I get on my local laptop.
`fetched` has 250 items in it, so it's clearly different from my local environment.
I've managed to replicate the bug in production! I created a test database like this:
python tests/fixtures.py sortable.db
Then deployed that database like so:
datasette publish now sortable.db \
--extra-options=""--page_size=50"" --branch=debug-travis-issue-216
And... if you click ""next"" on this page https://datasette-issue-216-pagination.now.sh/sortable-5679797/sortable?_sort_desc=sortable_with_nulls five times you get back 250 results, when you should only get back 201.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381799267,https://api.github.com/repos/simonw/datasette/issues/216,381799267,MDEyOklzc3VlQ29tbWVudDM4MTc5OTI2Nw==,9599,simonw,2018-04-17T01:21:35Z,2018-04-17T01:21:35Z,OWNER,"The version that I deployed which exhibits the bug is running SQLite `3.8.7.1` - https://datasette-issue-216-pagination.now.sh/sortable-5679797?sql=select+sqlite_version%28%29
The version that I have running locally which does NOT exhibit the bug is running SQLite `3.23.0`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381799408,https://api.github.com/repos/simonw/datasette/issues/216,381799408,MDEyOklzc3VlQ29tbWVudDM4MTc5OTQwOA==,9599,simonw,2018-04-17T01:22:30Z,2018-04-17T01:22:30Z,OWNER,"... which is VERY surprising, because `3.23.0` only came out on 2nd April this year: https://www.sqlite.org/changes.html - I have no idea how I came to be running that version on my laptop.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381801302,https://api.github.com/repos/simonw/datasette/issues/216,381801302,MDEyOklzc3VlQ29tbWVudDM4MTgwMTMwMg==,9599,simonw,2018-04-17T01:33:43Z,2018-04-17T01:33:43Z,OWNER,"This is the SQL that returns differing results in production and on my laptop: https://datasette-issue-216-pagination.now.sh/sortable-5679797?sql=select+%2A+from+sortable+where+%28sortable_with_nulls+is+null+and+%28%28pk1+%3E+%3Ap0%29%0A++or%0A%28pk1+%3D+%3Ap0+and+pk2+%3E+%3Ap1%29%29%29+order+by+sortable_with_nulls+desc+limit+51&p0=b&p1=t
```
select * from sortable where (sortable_with_nulls is null and ((pk1 > :p0)
or
(pk1 = :p0 and pk2 > :p1))) order by sortable_with_nulls desc limit 51
```
I think that `order by sortable_with_nulls desc` bit is at fault - the primary keys should be included in that order by as well.
Sure enough, changing the query to this one returns the same results across both environments:
```
select * from sortable where (sortable_with_nulls is null and ((pk1 > :p0)
or
(pk1 = :p0 and pk2 > :p1))) order by sortable_with_nulls desc, pk1, pk2 limit 51
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/216#issuecomment-381803157,https://api.github.com/repos/simonw/datasette/issues/216,381803157,MDEyOklzc3VlQ29tbWVudDM4MTgwMzE1Nw==,9599,simonw,2018-04-17T01:45:24Z,2018-04-17T01:45:24Z,OWNER,Fixed!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314665147,Bug: Sort by column with NULL in next_page URL,
https://github.com/simonw/datasette/issues/14#issuecomment-381622793,https://api.github.com/repos/simonw/datasette/issues/14,381622793,MDEyOklzc3VlQ29tbWVudDM4MTYyMjc5Mw==,9599,simonw,2018-04-16T14:40:39Z,2018-04-17T01:47:15Z,OWNER,"I think that's OK. The two plugins I've implemented so far (`prepare_connection` and `prepare_jinja2_environment`) both make sense if they can only be defined once-per-plugin. For the moment I'll assume I can define future hooks to work well with the same limitation.
The syntactic sugar idea in #220 can help here too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",267707940,Datasette Plugins,
https://github.com/simonw/datasette/issues/14#issuecomment-381809998,https://api.github.com/repos/simonw/datasette/issues/14,381809998,MDEyOklzc3VlQ29tbWVudDM4MTgwOTk5OA==,9599,simonw,2018-04-17T02:23:39Z,2018-04-17T02:23:39Z,OWNER,I just shipped Datasette 0.19 with where I'm at so far: https://github.com/simonw/datasette/releases/tag/0.19,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",267707940,Datasette Plugins,
https://github.com/simonw/datasette/pull/209#issuecomment-381905593,https://api.github.com/repos/simonw/datasette/issues/209,381905593,MDEyOklzc3VlQ29tbWVudDM4MTkwNTU5Mw==,45057,russss,2018-04-17T08:50:28Z,2018-04-17T08:50:28Z,CONTRIBUTOR,"I've added another commit which puts classes a class on each `` by default with its column name, and I've also made the PK column bold.
Unfortunately the tests are still failing on 3.6, which is weird. I can't reproduce locally...","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314455877, Don't duplicate simple primary keys in the link column,
https://github.com/simonw/datasette/issues/214#issuecomment-382038613,https://api.github.com/repos/simonw/datasette/issues/214,382038613,MDEyOklzc3VlQ29tbWVudDM4MjAzODYxMw==,9599,simonw,2018-04-17T15:38:23Z,2018-04-17T15:38:23Z,OWNER,"I figured out the recipe for bundling static assets in a plugin: https://github.com/simonw/datasette-plugin-demos/commit/26c5548f4ab7c6cc6d398df17767950be50d0edf (and then `python3 setup.py bdist_wheel`)
Having done that, I ran `pip install ../datasette-plugin-demos/dist/datasette_plugin_demos-0.2-py3-none-any.whl` from my Datasette virtual environment and then did the following:
```
>>> import pkg_resources
>>> pkg_resources.resource_stream(
... 'datasette_plugin_demos', 'static/plugin.js'
... ).read()
b""alert('hello');\n""
>>> pkg_resources.resource_filename(
... 'datasette_plugin_demos', 'static/plugin.js'
... )
'..../venv/lib/python3.6/site-packages/datasette_plugin_demos/static/plugin.js'
>>> pkg_resources.resource_string(
... 'datasette_plugin_demos', 'static/plugin.js'
... )
b""alert('hello');\n""
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314506446,Ability for plugins to define extra JavaScript and CSS,
https://github.com/simonw/datasette/issues/214#issuecomment-382069980,https://api.github.com/repos/simonw/datasette/issues/214,382069980,MDEyOklzc3VlQ29tbWVudDM4MjA2OTk4MA==,9599,simonw,2018-04-17T17:08:28Z,2018-04-17T17:08:28Z,OWNER,"Even if we automatically serve ALL `static/` content from installed plugins, we'll still need them to register which files need to be linked to from `extra_css_urls` and `extra_js_urls`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314506446,Ability for plugins to define extra JavaScript and CSS,
https://github.com/simonw/datasette/pull/209#issuecomment-382205189,https://api.github.com/repos/simonw/datasette/issues/209,382205189,MDEyOklzc3VlQ29tbWVudDM4MjIwNTE4OQ==,9599,simonw,2018-04-18T00:42:44Z,2018-04-18T00:43:02Z,OWNER,"I managed to get a better error message out of that test. The server is returning this (but only on Python 3.6, not on Python 3.5 - and only in Travis, not in my local environment):
```{'error': 'interrupted', 'ok': False, 'status': 400, 'title': 'Invalid SQL'}```
https://travis-ci.org/simonw/datasette/jobs/367929134","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314455877, Don't duplicate simple primary keys in the link column,
https://github.com/simonw/datasette/pull/209#issuecomment-382210976,https://api.github.com/repos/simonw/datasette/issues/209,382210976,MDEyOklzc3VlQ29tbWVudDM4MjIxMDk3Ng==,9599,simonw,2018-04-18T01:12:26Z,2018-04-18T01:12:26Z,OWNER,"OK, aaf59db570ab7688af72c08bb5bc1edc145e3e07 should mean that the tests pass when I merge that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314455877, Don't duplicate simple primary keys in the link column,
https://github.com/simonw/datasette/issues/214#issuecomment-382048582,https://api.github.com/repos/simonw/datasette/issues/214,382048582,MDEyOklzc3VlQ29tbWVudDM4MjA0ODU4Mg==,9599,simonw,2018-04-17T16:04:42Z,2018-04-18T02:24:46Z,OWNER,"One possible option: let plugins bundle their own `static/` directory and then register themselves with Datasette, then have `/-/static-plugins/name-of-plugin/...` serve files from that directory.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314506446,Ability for plugins to define extra JavaScript and CSS,
https://github.com/simonw/datasette/issues/14#issuecomment-382256729,https://api.github.com/repos/simonw/datasette/issues/14,382256729,MDEyOklzc3VlQ29tbWVudDM4MjI1NjcyOQ==,9599,simonw,2018-04-18T04:29:29Z,2018-04-18T04:30:14Z,OWNER,I added a mechanism for plugins to serve static files and define custom CSS and JS URLs in #214 - see new documentation on http://datasette.readthedocs.io/en/latest/plugins.html#static-assets and http://datasette.readthedocs.io/en/latest/plugins.html#extra-css-urls,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",267707940,Datasette Plugins,
https://github.com/simonw/datasette/issues/223#issuecomment-382408128,https://api.github.com/repos/simonw/datasette/issues/223,382408128,MDEyOklzc3VlQ29tbWVudDM4MjQwODEyOA==,9599,simonw,2018-04-18T14:33:09Z,2018-04-18T14:33:09Z,OWNER,"Demo:
datasette publish now sortable.db --install datasette-plugin-demos --branch=master
Produced this deployment, with both the `random_integer()` function and the static file from https://github.com/simonw/datasette-plugin-demos/tree/0.2
https://datasette-issue-223.now.sh/-/static-plugins/datasette_plugin_demos/plugin.js
https://datasette-issue-223.now.sh/sortable-4bbaa6f?sql=select+random_integer%280%2C+10%29
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",315327860,datasette publish --install=name-of-plugin,
https://github.com/simonw/datasette/issues/223#issuecomment-382409989,https://api.github.com/repos/simonw/datasette/issues/223,382409989,MDEyOklzc3VlQ29tbWVudDM4MjQwOTk4OQ==,9599,simonw,2018-04-18T14:38:08Z,2018-04-18T14:38:08Z,OWNER,"Tested on Heroku as well.
datasette publish heroku sortable.db --install datasette-plugin-demos --branch=master
https://morning-tor-45944.herokuapp.com/-/static-plugins/datasette_plugin_demos/plugin.js
https://morning-tor-45944.herokuapp.com/sortable-4bbaa6f?sql=select+random_integer%280%2C+10%29","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",315327860,datasette publish --install=name-of-plugin,
https://github.com/simonw/datasette/issues/223#issuecomment-382413121,https://api.github.com/repos/simonw/datasette/issues/223,382413121,MDEyOklzc3VlQ29tbWVudDM4MjQxMzEyMQ==,9599,simonw,2018-04-18T14:47:18Z,2018-04-18T14:47:18Z,OWNER,"And tested `datasette package` - this time exercising the ability to pass more than one `--install` option:
```
$ datasette package sortable.db --branch=master --install requests --install datasette-plugin-demos
Sending build context to Docker daemon 125.4kB
Step 1/7 : FROM python:3
---> 79e1dc9af1c1
Step 2/7 : COPY . /app
---> 6e8e40bce378
Step 3/7 : WORKDIR /app
Removing intermediate container 7cdc9ab20d09
---> f42258c2211f
Step 4/7 : RUN pip install https://github.com/simonw/datasette/archive/master.zip requests datasette-plugin-demos
---> Running in a0f17cec08a4
Collecting ...
Removing intermediate container a0f17cec08a4
---> beea84e73271
Step 5/7 : RUN datasette inspect sortable.db --inspect-file inspect-data.json
---> Running in 4daa28792348
Removing intermediate container 4daa28792348
---> c60312d21b99
Step 6/7 : EXPOSE 8001
---> Running in fa728468482d
Removing intermediate container fa728468482d
---> 8f219a61fddc
Step 7/7 : CMD [""datasette"", ""serve"", ""--host"", ""0.0.0.0"", ""sortable.db"", ""--cors"", ""--port"", ""8001"", ""--inspect-file"", ""inspect-data.json""]
---> Running in cd4eaeb2ce9e
Removing intermediate container cd4eaeb2ce9e
---> 066e257c7c44
Successfully built 066e257c7c44
(venv) datasette $ docker run -p 8081:8001 066e257c7c44
Serve! files=('sortable.db',) on port 8001
[2018-04-18 14:40:18 +0000] [1] [INFO] Goin' Fast @ http://0.0.0.0:8001
[2018-04-18 14:40:18 +0000] [1] [INFO] Starting worker [1]
[2018-04-18 14:46:01 +0000] - (sanic.access)[INFO][1:7]: GET http://localhost:8081/-/static-plugins/datasette_plugin_demos/plugin.js 200 16
``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",315327860,datasette publish --install=name-of-plugin,
https://github.com/simonw/datasette/issues/224#issuecomment-382616527,https://api.github.com/repos/simonw/datasette/issues/224,382616527,MDEyOklzc3VlQ29tbWVudDM4MjYxNjUyNw==,9599,simonw,2018-04-19T05:40:28Z,2018-04-19T05:40:28Z,OWNER,"No need to use `PackageLoader` after all, we can use the same mechanism we used for the static path:
https://github.com/simonw/datasette/blob/b55809a1e20986bb2e638b698815a77902e8708d/datasette/utils.py#L694-L695","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",315517578,Ability for plugins to bundle templates,
https://github.com/simonw/datasette/issues/227#issuecomment-382808266,https://api.github.com/repos/simonw/datasette/issues/227,382808266,MDEyOklzc3VlQ29tbWVudDM4MjgwODI2Ng==,9599,simonw,2018-04-19T16:59:23Z,2018-04-19T16:59:23Z,OWNER,"Maybe this should have a second argument indicating which codepath was being handled. That way plugins could say ""only inject this extra context variable on the row page"".","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",315960272,prepare_context() plugin hook,
https://github.com/simonw/datasette/issues/228#issuecomment-382924910,https://api.github.com/repos/simonw/datasette/issues/228,382924910,MDEyOklzc3VlQ29tbWVudDM4MjkyNDkxMA==,9599,simonw,2018-04-20T00:35:48Z,2018-04-20T00:35:48Z,OWNER,"Hiding tables with the `idx_` prefix should be good enough here, since false positives aren't very harmful.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316031566,"If spatialite detected, mark idx_XXX_Geometry tables as hidden",
https://github.com/simonw/datasette/issues/227#issuecomment-382958693,https://api.github.com/repos/simonw/datasette/issues/227,382958693,MDEyOklzc3VlQ29tbWVudDM4Mjk1ODY5Mw==,9599,simonw,2018-04-20T03:15:52Z,2018-04-20T03:15:52Z,OWNER,"A better way to do this would be with many different plugin hooks, one for each view.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",315960272,prepare_context() plugin hook,
https://github.com/simonw/datasette/issues/227#issuecomment-382959857,https://api.github.com/repos/simonw/datasette/issues/227,382959857,MDEyOklzc3VlQ29tbWVudDM4Mjk1OTg1Nw==,9599,simonw,2018-04-20T03:21:43Z,2018-04-20T03:21:43Z,OWNER,"Plus a generic prepare_context() hook called in the common render method.
prepare_context_table(), prepare_context_row() etc
Arguments are context, request, self (hence can access self.ds)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",315960272,prepare_context() plugin hook,
https://github.com/simonw/datasette/issues/227#issuecomment-382964794,https://api.github.com/repos/simonw/datasette/issues/227,382964794,MDEyOklzc3VlQ29tbWVudDM4Mjk2NDc5NA==,9599,simonw,2018-04-20T03:45:18Z,2018-04-20T03:45:18Z,OWNER,"What if the context needs to make await calls?
One possible option: plugins can either manipulate the context in place OR they can return an awaitable. If they do that, the caller will await it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",315960272,prepare_context() plugin hook,
https://github.com/simonw/datasette/issues/227#issuecomment-382966604,https://api.github.com/repos/simonw/datasette/issues/227,382966604,MDEyOklzc3VlQ29tbWVudDM4Mjk2NjYwNA==,9599,simonw,2018-04-20T03:54:56Z,2018-04-20T03:54:56Z,OWNER,Should this differentiate between preparing the data to be sent back as JSON and preparing the context for the template?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",315960272,prepare_context() plugin hook,
https://github.com/simonw/datasette/issues/227#issuecomment-382967238,https://api.github.com/repos/simonw/datasette/issues/227,382967238,MDEyOklzc3VlQ29tbWVudDM4Mjk2NzIzOA==,9599,simonw,2018-04-20T03:58:09Z,2018-04-20T03:58:09Z,OWNER,Maybe prepare_table_data() vs prepare_table_context(),"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",315960272,prepare_context() plugin hook,
https://github.com/simonw/datasette/issues/230#issuecomment-383109984,https://api.github.com/repos/simonw/datasette/issues/230,383109984,MDEyOklzc3VlQ29tbWVudDM4MzEwOTk4NA==,9599,simonw,2018-04-20T14:15:39Z,2018-04-20T14:15:39Z,OWNER,Refs #229,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316128955,Setting page size AND max returned rows to 1000 doesn't seem to work,
https://github.com/simonw/datasette/issues/14#issuecomment-383139889,https://api.github.com/repos/simonw/datasette/issues/14,383139889,MDEyOklzc3VlQ29tbWVudDM4MzEzOTg4OQ==,9599,simonw,2018-04-20T15:51:47Z,2018-04-20T15:51:47Z,OWNER,"I released everything we have so far in [Datasette 0.20](https://github.com/simonw/datasette/releases/tag/0.20) and built and released an example plugin, [datasette-cluster-map](https://pypi.org/project/datasette-cluster-map/). Here's my blog entry about it: https://simonwillison.net/2018/Apr/20/datasette-plugins/","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",267707940,Datasette Plugins,
https://github.com/simonw/datasette/issues/14#issuecomment-383140111,https://api.github.com/repos/simonw/datasette/issues/14,383140111,MDEyOklzc3VlQ29tbWVudDM4MzE0MDExMQ==,9599,simonw,2018-04-20T15:52:33Z,2018-04-20T15:52:33Z,OWNER,Here's a link demonstrating my new plugin: https://datasette-cluster-map-demo.now.sh/polar-bears-455fe3a/USGS_WC_eartags_output_files_2009-2011-Status,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",267707940,Datasette Plugins,
https://github.com/simonw/datasette/pull/232#issuecomment-383252624,https://api.github.com/repos/simonw/datasette/issues/232,383252624,MDEyOklzc3VlQ29tbWVudDM4MzI1MjYyNA==,9599,simonw,2018-04-21T00:19:00Z,2018-04-21T00:19:00Z,OWNER,Thanks!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316365426,Fix a typo,
https://github.com/simonw/datasette/issues/234#issuecomment-383398182,https://api.github.com/repos/simonw/datasette/issues/234,383398182,MDEyOklzc3VlQ29tbWVudDM4MzM5ODE4Mg==,9599,simonw,2018-04-22T17:31:12Z,2018-04-22T17:31:12Z,OWNER,"```{
""databases"": {
""database1"": {
""tables"": {
""example_table"": {
""label_column"": ""name""
}
}
}
}
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316526433,label_column option in metadata.json,
https://github.com/simonw/datasette/issues/234#issuecomment-383399762,https://api.github.com/repos/simonw/datasette/issues/234,383399762,MDEyOklzc3VlQ29tbWVudDM4MzM5OTc2Mg==,9599,simonw,2018-04-22T17:54:39Z,2018-04-22T17:54:39Z,OWNER,Docs here: http://datasette.readthedocs.io/en/latest/metadata.html#specifying-the-label-column-for-a-table,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316526433,label_column option in metadata.json,
https://github.com/simonw/datasette/issues/234#issuecomment-383410146,https://api.github.com/repos/simonw/datasette/issues/234,383410146,MDEyOklzc3VlQ29tbWVudDM4MzQxMDE0Ng==,9599,simonw,2018-04-22T20:32:30Z,2018-04-22T20:47:02Z,OWNER,"I built this wrong: my implementation is looking for the `label_column` on the table-being-displayed, but it should be looking for it on the table-the-foreign-key-links-to.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316526433,label_column option in metadata.json,
https://github.com/simonw/datasette/issues/231#issuecomment-383315348,https://api.github.com/repos/simonw/datasette/issues/231,383315348,MDEyOklzc3VlQ29tbWVudDM4MzMxNTM0OA==,9599,simonw,2018-04-21T17:37:50Z,2018-04-22T23:06:04Z,OWNER,"I could also have an `""autodetect"": false` option for that plugin to turn off autodetecting entirely.
Would be useful if the plugin didn't append its JavaScript in pages that it wasn't used for - that might require making the `extra_js_urls()` hook optionally aware of the columns and table and metadata.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316323336,metadata.json support for plugin configuration options,
https://github.com/simonw/datasette/issues/235#issuecomment-383727973,https://api.github.com/repos/simonw/datasette/issues/235,383727973,MDEyOklzc3VlQ29tbWVudDM4MzcyNzk3Mw==,9599,simonw,2018-04-23T21:23:59Z,2018-04-23T21:23:59Z,OWNER,"There might also be something clever we can do here with PRAGMA statements: https://stackoverflow.com/questions/14146881/limit-the-maximum-amount-of-memory-sqlite3-uses
And https://www.sqlite.org/pragma.html","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316621102,Add limit on the size in KB of data returned from a single query,
https://github.com/simonw/datasette/issues/235#issuecomment-383764533,https://api.github.com/repos/simonw/datasette/issues/235,383764533,MDEyOklzc3VlQ29tbWVudDM4Mzc2NDUzMw==,9599,simonw,2018-04-24T00:30:02Z,2018-04-24T00:30:02Z,OWNER,The `resource` module in he standard library has the ability to set limits on memory usage for the current process: https://pymotw.com/2/resource/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316621102,Add limit on the size in KB of data returned from a single query,
https://github.com/simonw/datasette/issues/238#issuecomment-384362028,https://api.github.com/repos/simonw/datasette/issues/238,384362028,MDEyOklzc3VlQ29tbWVudDM4NDM2MjAyOA==,9599,simonw,2018-04-25T17:07:11Z,2018-04-25T17:07:11Z,OWNER,"On further thought: this is actually only an issue for immutable deployments to platforms like Zeit Now and Heroku.
As such, adding it to `datasette serve` feels clumsy. Maybe `datasette publish` should instead gain the ability to optionally install an extra mechanism that periodically pulls a fresh copy of `metadata.json` from a URL.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",317714268,External metadata.json,
https://github.com/simonw/datasette/issues/239#issuecomment-384500327,https://api.github.com/repos/simonw/datasette/issues/239,384500327,MDEyOklzc3VlQ29tbWVudDM4NDUwMDMyNw==,9599,simonw,2018-04-26T03:18:12Z,2018-04-26T03:18:20Z,OWNER,"```
{
""databases"": {
""database1"": {
""tables"": {
""example_table"": {
""hidden"": true
}
}
}
}
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",317760361,Support for hidden tables in metadata.json,
https://github.com/simonw/datasette/issues/239#issuecomment-384503873,https://api.github.com/repos/simonw/datasette/issues/239,384503873,MDEyOklzc3VlQ29tbWVudDM4NDUwMzg3Mw==,9599,simonw,2018-04-26T03:45:11Z,2018-04-26T03:45:11Z,OWNER,Documentation: http://datasette.readthedocs.io/en/latest/metadata.html#hiding-tables,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",317760361,Support for hidden tables in metadata.json,
https://github.com/simonw/datasette/issues/229#issuecomment-384512192,https://api.github.com/repos/simonw/datasette/issues/229,384512192,MDEyOklzc3VlQ29tbWVudDM4NDUxMjE5Mg==,9599,simonw,2018-04-26T04:49:46Z,2018-04-26T04:49:46Z,OWNER,Documentation: http://datasette.readthedocs.io/en/latest/json_api.html#special-table-arguments,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316123256,Table view should support ?_size=400 parameter,
https://github.com/simonw/datasette/issues/79#issuecomment-384675792,https://api.github.com/repos/simonw/datasette/issues/79,384675792,MDEyOklzc3VlQ29tbWVudDM4NDY3NTc5Mg==,9599,simonw,2018-04-26T15:08:13Z,2018-04-26T15:08:13Z,OWNER,"Docs now live at http://datasette.readthedocs.io/
I still need to document a few more parts of the API before closing this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",273569068,Add more detailed API documentation to the README,
https://github.com/simonw/datasette/issues/44#issuecomment-384676488,https://api.github.com/repos/simonw/datasette/issues/44,384676488,MDEyOklzc3VlQ29tbWVudDM4NDY3NjQ4OA==,9599,simonw,2018-04-26T15:09:57Z,2018-04-26T15:09:57Z,OWNER,Remaining work for this is tracked in #150,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",269731374,?_group_count=country - return counts by specific column(s),
https://github.com/simonw/datasette/issues/125#issuecomment-384678319,https://api.github.com/repos/simonw/datasette/issues/125,384678319,MDEyOklzc3VlQ29tbWVudDM4NDY3ODMxOQ==,9599,simonw,2018-04-26T15:14:31Z,2018-04-26T15:14:31Z,OWNER,"I shipped this last week as the first plugin: https://simonwillison.net/2018/Apr/20/datasette-plugins/
Demo: https://datasette-cluster-map-demo.datasettes.com/polar-bears-455fe3a/USGS_WC_eartags_output_files_2009-2011-Status
Plugin: https://github.com/simonw/datasette-cluster-map","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275135393,Plot rows on a map with Leaflet and Leaflet.markercluster,
https://github.com/simonw/datasette/issues/244#issuecomment-386309928,https://api.github.com/repos/simonw/datasette/issues/244,386309928,MDEyOklzc3VlQ29tbWVudDM4NjMwOTkyOA==,9599,simonw,2018-05-03T14:13:49Z,2018-05-03T14:13:49Z,OWNER,Demo: https://datasette-versions-and-shape-demo.now.sh/-/versions,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",318738000,/-/versions page,
https://github.com/simonw/datasette/issues/245#issuecomment-386310149,https://api.github.com/repos/simonw/datasette/issues/245,386310149,MDEyOklzc3VlQ29tbWVudDM4NjMxMDE0OQ==,9599,simonw,2018-05-03T14:14:33Z,2018-05-03T14:14:33Z,OWNER,"Demos:
* https://datasette-versions-and-shape-demo.now.sh/sf-trees-02c8ef1/qSpecies.json?_shape=array
* https://datasette-versions-and-shape-demo.now.sh/sf-trees-02c8ef1/qSpecies.json?_shape=object
* https://datasette-versions-and-shape-demo.now.sh/sf-trees-02c8ef1/qSpecies.json?_shape=arrays
* https://datasette-versions-and-shape-demo.now.sh/sf-trees-02c8ef1/qSpecies.json?_shape=objects","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",319358200,?_shape=array option,
https://github.com/simonw/datasette/issues/248#issuecomment-386357645,https://api.github.com/repos/simonw/datasette/issues/248,386357645,MDEyOklzc3VlQ29tbWVudDM4NjM1NzY0NQ==,9599,simonw,2018-05-03T16:36:59Z,2018-05-03T16:36:59Z,OWNER,"Even better: use `plugin_manager.list_plugin_distinfo()` from pluggy to get back a list of tuples, the second item in each tuple is a `pkg_resources.DistInfoDistribution` with a `.version` attribute.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",319954545,/-/plugins should show version of each installed plugin,
https://github.com/simonw/datasette/issues/248#issuecomment-386692333,https://api.github.com/repos/simonw/datasette/issues/248,386692333,MDEyOklzc3VlQ29tbWVudDM4NjY5MjMzMw==,9599,simonw,2018-05-04T18:25:40Z,2018-05-04T18:25:40Z,OWNER,Demo: https://datasette-plugins-and-max-size-demo.now.sh/-/plugins,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",319954545,/-/plugins should show version of each installed plugin,
https://github.com/simonw/datasette/issues/249#issuecomment-386692534,https://api.github.com/repos/simonw/datasette/issues/249,386692534,MDEyOklzc3VlQ29tbWVudDM4NjY5MjUzNA==,9599,simonw,2018-05-04T18:26:30Z,2018-05-04T18:26:30Z,OWNER,Demo: https://datasette-plugins-and-max-size-demo.now.sh/sf-trees/Street_Tree_List.json?_size=max,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",320090329,?_size=max argument ,
https://github.com/simonw/datasette/issues/237#issuecomment-386840307,https://api.github.com/repos/simonw/datasette/issues/237,386840307,MDEyOklzc3VlQ29tbWVudDM4Njg0MDMwNw==,9599,simonw,2018-05-05T22:45:45Z,2018-05-05T22:45:45Z,OWNER,Documented here: http://datasette.readthedocs.io/en/latest/json_api.html#special-table-arguments,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",317475156,Support for ?_search_colname=blah searches,
https://github.com/simonw/datasette/issues/237#issuecomment-386840806,https://api.github.com/repos/simonw/datasette/issues/237,386840806,MDEyOklzc3VlQ29tbWVudDM4Njg0MDgwNg==,9599,simonw,2018-05-05T22:56:42Z,2018-05-05T22:56:42Z,OWNER,"Demo:
datasette publish now ../datasettes/san-francisco/sf-film-locations.db --branch=master --name datasette-column-search-demo
https://datasette-column-search-demo.now.sh/sf-film-locations/Film_Locations_in_San_Francisco?_search_Locations=justin","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",317475156,Support for ?_search_colname=blah searches,
https://github.com/simonw/datasette/issues/251#issuecomment-386879509,https://api.github.com/repos/simonw/datasette/issues/251,386879509,MDEyOklzc3VlQ29tbWVudDM4Njg3OTUwOQ==,9599,simonw,2018-05-06T13:29:26Z,2018-05-06T13:29:26Z,OWNER,"We can solve this using the `sqlite_timelimit(conn, 20)` helper, which can tell SQLite to give up after 20ms. We can wrap that around the following SQL:
select distinct COLUMN from TABLE limit 21;
Then we look at the number of rows returned. If it's 21 or more we know that this table had more than 21 distinct values, so we'll treat it as ""unlimited"". Likewise, if the SQL times out before 20ms is up we will skip this introspection.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",320592643,"Explore ""distinct values for column"" in inspect()",
https://github.com/simonw/datasette/issues/251#issuecomment-386879840,https://api.github.com/repos/simonw/datasette/issues/251,386879840,MDEyOklzc3VlQ29tbWVudDM4Njg3OTg0MA==,9599,simonw,2018-05-06T13:34:24Z,2018-05-06T13:34:24Z,OWNER,"Here's a quick demo of that exploration: https://datasette-distinct-column-values.now.sh/-/inspect
Example output:
```
{
""antiquities-act/actions_under_antiquities_act"": {
""columns"": [
""current_name"",
""states"",
""original_name"",
""current_agency"",
""action"",
""date"",
""year"",
""pres_or_congress"",
""acres_affected""
],
""count"": 344,
""distinct_values_by_column"": {
""acres_affected"": null,
""action"": null,
""current_agency"": [
""NPS"",
""State of Montana"",
""BLM"",
""State of Arizona"",
""USFS"",
""State of North Dakota"",
""NPS, BLM"",
""State of South Carolina"",
""State of New York"",
""FWS"",
""FWS, NOAA"",
""NPS, FWS"",
""NOAA"",
""BLM, USFS"",
""NOAA, FWS""
],
""current_name"": null,
""date"": null,
""original_name"": null,
""pres_or_congress"": null,
""states"": null,
""year"": null
},
""foreign_keys"": {
""incoming"": [],
""outgoing"": []
},
""fts_table"": null,
""hidden"": false,
""label_column"": null,
""name"": ""antiquities-act/actions_under_antiquities_act"",
""primary_keys"": []
}
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",320592643,"Explore ""distinct values for column"" in inspect()",
https://github.com/simonw/datasette/issues/251#issuecomment-386879878,https://api.github.com/repos/simonw/datasette/issues/251,386879878,MDEyOklzc3VlQ29tbWVudDM4Njg3OTg3OA==,9599,simonw,2018-05-06T13:34:57Z,2018-05-06T13:34:57Z,OWNER,If I'm going to expand column introspection in this way it would be useful to also capture column type information.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",320592643,"Explore ""distinct values for column"" in inspect()",
https://github.com/simonw/datasette/issues/254#issuecomment-388367027,https://api.github.com/repos/simonw/datasette/issues/254,388367027,MDEyOklzc3VlQ29tbWVudDM4ODM2NzAyNw==,247131,philroche,2018-05-11T13:41:46Z,2018-05-11T13:41:46Z,NONE,"An example deployment @ https://datasette-zkcvlwdrhl.now.sh/simplestreams-270f20c/cloudimage?content_id__exact=com.ubuntu.cloud%3Areleased%3Adownload
It is not causing errors, more of an inconvenience. I have worked around it using a `like` query instead. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322283067,Escaping named parameters in canned queries,
https://github.com/simonw/datasette/issues/254#issuecomment-388497467,https://api.github.com/repos/simonw/datasette/issues/254,388497467,MDEyOklzc3VlQ29tbWVudDM4ODQ5NzQ2Nw==,9599,simonw,2018-05-11T22:06:00Z,2018-05-11T22:06:34Z,OWNER,"Got it, this seems to trigger the problem: https://datasette-zkcvlwdrhl.now.sh/simplestreams-270f20c?sql=select+*+from+cloudimage+where+%22content_id%22+%3D+%22com.ubuntu.cloud%3Areleased%3Adownload%22+order+by+id+limit+10","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322283067,Escaping named parameters in canned queries,
https://github.com/simonw/datasette/issues/254#issuecomment-388360255,https://api.github.com/repos/simonw/datasette/issues/254,388360255,MDEyOklzc3VlQ29tbWVudDM4ODM2MDI1NQ==,9599,simonw,2018-05-11T13:16:09Z,2018-05-11T22:45:31Z,OWNER,"Do you have an example I can look at?
I think I have a possible route for fixing this, but it's pretty tricky (it involves adding a full SQL statement parser, but that's needed for some other potential improvements as well).
In the meantime, is this causing actual errors for you or is it more of an inconvenience (form fields being displayed that don't actually do anything)?
Another potential solution here could be to allow canned queries to optionally declare their parameters in metadata.json","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322283067,Escaping named parameters in canned queries,
https://github.com/simonw/datasette/issues/255#issuecomment-388525357,https://api.github.com/repos/simonw/datasette/issues/255,388525357,MDEyOklzc3VlQ29tbWVudDM4ODUyNTM1Nw==,9599,simonw,2018-05-12T03:01:14Z,2018-05-12T03:01:14Z,OWNER,Facet counts will be generated by extra SQL queries with their own aggressive time limit.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/253#issuecomment-388550742,https://api.github.com/repos/simonw/datasette/issues/253,388550742,MDEyOklzc3VlQ29tbWVudDM4ODU1MDc0Mg==,9599,simonw,2018-05-12T12:09:02Z,2018-05-12T12:09:02Z,OWNER,http://datasette.readthedocs.io/en/latest/full_text_search.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",321631020,Documentation explaining how to use SQLite FTS with Datasette,
https://github.com/simonw/datasette/issues/255#issuecomment-388587855,https://api.github.com/repos/simonw/datasette/issues/255,388587855,MDEyOklzc3VlQ29tbWVudDM4ODU4Nzg1NQ==,9599,simonw,2018-05-12T22:30:23Z,2018-05-12T22:30:23Z,OWNER,Adding some TODOs to the original description (so they show up as a todo progress bar),"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/255#issuecomment-388588011,https://api.github.com/repos/simonw/datasette/issues/255,388588011,MDEyOklzc3VlQ29tbWVudDM4ODU4ODAxMQ==,9599,simonw,2018-05-12T22:33:39Z,2018-05-12T22:33:39Z,OWNER,Initial documentation: http://datasette.readthedocs.io/en/latest/facets.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/255#issuecomment-388589072,https://api.github.com/repos/simonw/datasette/issues/255,388589072,MDEyOklzc3VlQ29tbWVudDM4ODU4OTA3Mg==,9599,simonw,2018-05-12T22:59:07Z,2018-05-12T22:59:07Z,OWNER,"I need to decide how to display these. They currently look like this:
https://datasette-facets-demo.now.sh/fivethirtyeight-2628db9/congress-age%2Fcongress-terms?_facet=chamber&_facet=state&_facet=party&_facet=incumbent&state=MO
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/255#issuecomment-388588998,https://api.github.com/repos/simonw/datasette/issues/255,388588998,MDEyOklzc3VlQ29tbWVudDM4ODU4ODk5OA==,9599,simonw,2018-05-12T22:57:30Z,2018-05-12T23:00:24Z,OWNER,"A few demos:
* https://datasette-facets-demo.now.sh/fivethirtyeight-2628db9/college-majors%2Fall-ages?_facet=Major_category
* https://datasette-facets-demo.now.sh/fivethirtyeight-2628db9/congress-age%2Fcongress-terms?_facet=chamber&_facet=state&_facet=party&_facet=incumbent
* https://datasette-facets-demo.now.sh/fivethirtyeight-2628db9/bechdel%2Fmovies?_facet=binary&_facet=test","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/pull/257#issuecomment-388625703,https://api.github.com/repos/simonw/datasette/issues/257,388625703,MDEyOklzc3VlQ29tbWVudDM4ODYyNTcwMw==,9599,simonw,2018-05-13T13:10:09Z,2018-05-13T13:10:09Z,OWNER,"I'm still seeing intermittent Python 3.5 failures due to dictionary ordering differences.
https://travis-ci.org/simonw/datasette/jobs/378356802
```
> assert expected_facet_results == facet_results
E AssertionError: assert {'city': [{'c...alue': 'MI'}]} == {'city': [{'co...alue': 'MI'}]}
E Omitting 1 identical items, use -vv to show
E Differing items:
E {'city': [{'count': 4, 'toggle_url': '_facet=state&_facet=city&state=MI&city=Detroit', 'value': 'Detroit'}]} != {'city': [{'count': 4, 'toggle_url': 'state=MI&_facet=state&_facet=city&city=Detroit', 'value': 'Detroit'}]}
E Use -v to get the full diff
```
To solve these cleanly I need to be able to run Python 3.5 on my local laptop rather than relying on Travis every time.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322591993,Refactor views,
https://github.com/simonw/datasette/pull/257#issuecomment-388626721,https://api.github.com/repos/simonw/datasette/issues/257,388626721,MDEyOklzc3VlQ29tbWVudDM4ODYyNjcyMQ==,9599,simonw,2018-05-13T13:27:04Z,2018-05-13T13:27:04Z,OWNER,"I managed to get Python 3.5.0 running on my laptop using [pyenv](https://github.com/pyenv/pyenv). Here's the incantation I used:
```
# Install pyenv using homebrew (turns out I already had it)
brew install pyenv
# Check which versions of Python I have installed
pyenv versions
# Install Python 3.5.0
pyenv install 3.5.0
# Figure out where pyenv has been installing things
pyenv root
# Check I can run my newly installed Python 3.5.0
/Users/simonw/.pyenv/versions/3.5.0/bin/python
# Use it to create a new virtualenv
/Users/simonw/.pyenv/versions/3.5.0/bin/python -mvenv venv35
source venv35/bin/activate
# Install datasette into that virtualenv
python setup.py install
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322591993,Refactor views,
https://github.com/simonw/datasette/pull/257#issuecomment-388626804,https://api.github.com/repos/simonw/datasette/issues/257,388626804,MDEyOklzc3VlQ29tbWVudDM4ODYyNjgwNA==,9599,simonw,2018-05-13T13:28:20Z,2018-05-13T13:28:20Z,OWNER,"Unfortunately, running `python setup.py test` on my laptop using Python 3.5.0 in that virtualenv results in a flow of weird Sanic-related errors:
```
File ""/Users/simonw/Dropbox/Development/datasette/venv35/lib/python3.5/site-packages/sanic-0.7.0-py3.5.egg/sanic/testing.py"", line 16, in _local_request
import aiohttp
File ""/Users/simonw/Dropbox/Development/datasette/.eggs/aiohttp-2.3.2-py3.5-macosx-10.13-x86_64.egg/aiohttp/__init__.py"", line 6, in
from .client import * # noqa
File ""/Users/simonw/Dropbox/Development/datasette/.eggs/aiohttp-2.3.2-py3.5-macosx-10.13-x86_64.egg/aiohttp/client.py"", line 13, in
from yarl import URL
File ""/Users/simonw/Dropbox/Development/datasette/.eggs/yarl-1.2.4-py3.5-macosx-10.13-x86_64.egg/yarl/__init__.py"", line 11, in
from .quoting import _Quoter, _Unquoter
File ""/Users/simonw/Dropbox/Development/datasette/.eggs/yarl-1.2.4-py3.5-macosx-10.13-x86_64.egg/yarl/quoting.py"", line 3, in
from typing import Optional, TYPE_CHECKING, cast
ImportError: cannot import name 'TYPE_CHECKING'
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322591993,Refactor views,
https://github.com/simonw/datasette/pull/257#issuecomment-388627281,https://api.github.com/repos/simonw/datasette/issues/257,388627281,MDEyOklzc3VlQ29tbWVudDM4ODYyNzI4MQ==,9599,simonw,2018-05-13T13:36:21Z,2018-05-13T13:36:21Z,OWNER,"https://github.com/rtfd/readthedocs.org/issues/3812#issuecomment-373780860 suggests Python 3.5.2 may have the fix.
Yup, that worked:
```
pyenv install 3.5.2
rm -rf venv35
/Users/simonw/.pyenv/versions/3.5.2/bin/python -mvenv venv35
source venv35/bin/activate
# Not sure why I need this in my local environment but I do:
pip install datasette_plugin_demos
python setup.py test
```
This is now giving me the same test failure locally that I am seeing in Travis.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322591993,Refactor views,
https://github.com/simonw/datasette/pull/257#issuecomment-388628966,https://api.github.com/repos/simonw/datasette/issues/257,388628966,MDEyOklzc3VlQ29tbWVudDM4ODYyODk2Ng==,9599,simonw,2018-05-13T14:00:47Z,2018-05-13T14:06:35Z,OWNER,"Running specific tests:
```
venv35/bin/pip install pytest beautifulsoup4 aiohttp
venv35/bin/pytest tests/test_utils.py
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322591993,Refactor views,
https://github.com/simonw/datasette/issues/255#issuecomment-388645828,https://api.github.com/repos/simonw/datasette/issues/255,388645828,MDEyOklzc3VlQ29tbWVudDM4ODY0NTgyOA==,9599,simonw,2018-05-13T18:18:56Z,2018-05-13T18:20:02Z,OWNER,I may be able to run the SQL for all of the facet counts in one go using a WITH CTE query - will have to microbenchmark this to make sure it is worthwhile: https://datasette-facets-demo.now.sh/fivethirtyeight-2628db9?sql=with+blah+as+%28select+*+from+%5Bcollege-majors%2Fall-ages%5D%29%0D%0Aselect+*+from+%28select+%22Major_category%22%2C+Major_category%2C+count%28*%29+as+n+from%0D%0Ablah+group+by+Major_category+order+by+n+desc+limit+10%29%0D%0Aunion+all%0D%0Aselect+*+from+%28select+%22Major_category2%22%2C+Major_category%2C+count%28*%29+as+n+from%0D%0Ablah+group+by+Major_category+order+by+n+desc+limit+10%29,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/256#issuecomment-388684356,https://api.github.com/repos/simonw/datasette/issues/256,388684356,MDEyOklzc3VlQ29tbWVudDM4ODY4NDM1Ng==,9599,simonw,2018-05-14T03:05:37Z,2018-05-14T03:05:37Z,OWNER,"I just landed pull request #257 - I haven't refactored the tests, I may do that later if it looks worthwhile.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322551723,Break up app.py into separate view modules,
https://github.com/simonw/datasette/issues/255#issuecomment-388686463,https://api.github.com/repos/simonw/datasette/issues/255,388686463,MDEyOklzc3VlQ29tbWVudDM4ODY4NjQ2Mw==,9599,simonw,2018-05-14T03:23:44Z,2018-05-14T03:25:22Z,OWNER,It would be neat if there was a mechanism for calculating aggregates per facet - e.g. calculating the sum() of specific columns against each facet result on https://datasette-facets-demo.now.sh/fivethirtyeight-2628db9/nba-elo%2Fnbaallelo?_facet=lg_id&_facet=fran_id&lg_id=ABA&_facet=team_id,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/255#issuecomment-388784063,https://api.github.com/repos/simonw/datasette/issues/255,388784063,MDEyOklzc3VlQ29tbWVudDM4ODc4NDA2Mw==,9599,simonw,2018-05-14T11:25:00Z,2018-05-14T11:25:15Z,OWNER,"Can I get facets working across many2many relationships?
This would be fiendishly useful, but the querystring and `metadata.json` syntax is non-obvious.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/255#issuecomment-388784787,https://api.github.com/repos/simonw/datasette/issues/255,388784787,MDEyOklzc3VlQ29tbWVudDM4ODc4NDc4Nw==,9599,simonw,2018-05-14T11:28:05Z,2018-05-14T11:28:05Z,OWNER,"To decide which facets to suggest: for each column, is the unique value count less than the number of rows matching the current query or is it less than 20 (if we are showing more than 20 rows)?
Maybe only do this if there are less than ten non-float columns. Or always try for foreign keys and booleans, then if there are none of those try indexed text and integer fields, then finally try non-indexed text and integer fields but only if there are less than ten.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/259#issuecomment-388797919,https://api.github.com/repos/simonw/datasette/issues/259,388797919,MDEyOklzc3VlQ29tbWVudDM4ODc5NzkxOQ==,9599,simonw,2018-05-14T12:23:11Z,2018-05-14T12:23:11Z,OWNER,"For M2M to work we will need a mechanism for applying IN queries to the table view, so you can select multiple M2M filters. Maybe this would work:
?_m2m_category=123&_m2m_category=865","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322787470,inspect() should detect many-to-many relationships,
https://github.com/simonw/datasette/issues/251#issuecomment-388987044,https://api.github.com/repos/simonw/datasette/issues/251,388987044,MDEyOklzc3VlQ29tbWVudDM4ODk4NzA0NA==,9599,simonw,2018-05-14T22:47:55Z,2018-05-14T22:47:55Z,OWNER,This work is now happening in the facets branch. Closing this in favor of #255.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",320592643,"Explore ""distinct values for column"" in inspect()",
https://github.com/simonw/datasette/issues/255#issuecomment-389145872,https://api.github.com/repos/simonw/datasette/issues/255,389145872,MDEyOklzc3VlQ29tbWVudDM4OTE0NTg3Mg==,9599,simonw,2018-05-15T12:17:52Z,2018-05-15T12:17:52Z,OWNER,Activity has now moved to this branch: https://github.com/simonw/datasette/commits/suggested-facets,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/255#issuecomment-389147608,https://api.github.com/repos/simonw/datasette/issues/255,389147608,MDEyOklzc3VlQ29tbWVudDM4OTE0NzYwOA==,9599,simonw,2018-05-15T12:24:46Z,2018-05-15T12:24:46Z,OWNER,"New demo (published with `datasette publish now --branch=suggested-facets fivethirtyeight.db sf-trees.db --name=datastte-suggested-facets-demo`): https://datasette-suggested-facets-demo.now.sh/fivethirtyeight-2628db9/comic-characters%2Fmarvel-wikia-data
After turning on a couple of suggested facets... https://datasette-suggested-facets-demo.now.sh/fivethirtyeight-2628db9/comic-characters%2Fmarvel-wikia-data?_facet=SEX&_facet=ID
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/pull/258#issuecomment-389386142,https://api.github.com/repos/simonw/datasette/issues/258,389386142,MDEyOklzc3VlQ29tbWVudDM4OTM4NjE0Mg==,9599,simonw,2018-05-16T03:51:13Z,2018-05-16T03:51:13Z,OWNER,"The URL does persist across deployments already, in that you can use the URL without the hash and it will redirect to the current location. Here's an example of that: https://san-francisco.datasettes.com/sf-trees/Street_Tree_List.json
This also works if you attempt to hit the incorrect hash, e.g. if you have deployed a new version of the database with an updated hash. The old hash will redirect, e.g. https://san-francisco.datasettes.com/sf-trees-c4b972c/Street_Tree_List.json
If you serve Datasette from a HTTP/2 proxy (I've been using Cloudflare for this) you won't even have to pay the cost of the redirect - Datasette sends a `Link: ; rel=preload` header with those redirects, which causes Cloudflare to push out the redirected source as part of that HTTP/2 request. You can fire up the Chrome DevTools to watch this happen.
https://github.com/simonw/datasette/blob/2b79f2bdeb1efa86e0756e741292d625f91cb93d/datasette/views/base.py#L91
All of that said... I'm not at all opposed to this feature. For consistency with other Datasette options (e.g. `--cors`) I'd prefer to do this as an optional argument to the `datasette serve` command - something like this:
datasette serve mydb.db --no-url-hash","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322741659,Add new metadata key persistent_urls which removes the hash from all database urls,
https://github.com/simonw/datasette/issues/255#issuecomment-389386919,https://api.github.com/repos/simonw/datasette/issues/255,389386919,MDEyOklzc3VlQ29tbWVudDM4OTM4NjkxOQ==,9599,simonw,2018-05-16T03:57:47Z,2018-05-16T03:58:30Z,OWNER,"I updated that demo to demonstrate the new foreign key label expansions: https://datasette-suggested-facets-demo.now.sh/sf-trees-02c8ef1/Street_Tree_List?_facet=qLegalStatus
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/255#issuecomment-389397457,https://api.github.com/repos/simonw/datasette/issues/255,389397457,MDEyOklzc3VlQ29tbWVudDM4OTM5NzQ1Nw==,9599,simonw,2018-05-16T05:20:04Z,2018-05-16T05:20:04Z,OWNER,Maybe `suggested_facets` should only be calculated for the HTML view.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/pull/258#issuecomment-389536870,https://api.github.com/repos/simonw/datasette/issues/258,389536870,MDEyOklzc3VlQ29tbWVudDM4OTUzNjg3MA==,9599,simonw,2018-05-16T14:22:31Z,2018-05-16T14:22:31Z,OWNER,"The principle benefit provided by the hash URLs is that Datasette can set a far-future cache expiry header on every response. This is particularly useful for JavaScript API work as it makes fantastic use of the browser's cache. It also means that if you are serving your API from behind a caching proxy like Cloudflare you get a fantastic cache hit rate.
An option to serve without persistent hashes would also need to turn off the cache headers.
Maybe the option should support both? If you hit a page with the hash in the URL you still get the cache headers, but hits to the URL without the hash serve uncashed content directly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322741659,Add new metadata key persistent_urls which removes the hash from all database urls,
https://github.com/simonw/datasette/issues/255#issuecomment-389546040,https://api.github.com/repos/simonw/datasette/issues/255,389546040,MDEyOklzc3VlQ29tbWVudDM4OTU0NjA0MA==,9599,simonw,2018-05-16T14:47:34Z,2018-05-16T14:47:34Z,OWNER,"Latest demo - now with multiple columns: https://datasette-suggested-facets-demo.now.sh/sf-trees-02c8ef1/Street_Tree_List?_facet=qCaretaker&_facet=qCareAssistant&_facet=qLegalStatus
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/255#issuecomment-389562708,https://api.github.com/repos/simonw/datasette/issues/255,389562708,MDEyOklzc3VlQ29tbWVudDM4OTU2MjcwOA==,9599,simonw,2018-05-16T15:32:12Z,2018-05-16T15:32:12Z,OWNER,"This is now landed in master, ready for the next release.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/263#issuecomment-389563719,https://api.github.com/repos/simonw/datasette/issues/263,389563719,MDEyOklzc3VlQ29tbWVudDM4OTU2MzcxOQ==,9599,simonw,2018-05-16T15:34:46Z,2018-05-16T15:34:46Z,OWNER,The underlying mechanics for the `_extras` mechanism described in #262 may help with this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323671577,Facets should not execute for ?shape=array|object,
https://github.com/simonw/datasette/issues/265#issuecomment-389566147,https://api.github.com/repos/simonw/datasette/issues/265,389566147,MDEyOklzc3VlQ29tbWVudDM4OTU2NjE0Nw==,9599,simonw,2018-05-16T15:41:42Z,2018-05-16T15:41:42Z,OWNER,"An official demo instance of Datasette dedicated to this use-case would be useful, especially if it was automatically deployed by Travis for every commit to master that passes the tests.
Maybe there should be a permanent version of it deployed for each released version too?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323677499,Add links to example Datasette instances to appropiate places in docs,
https://github.com/simonw/datasette/issues/266#issuecomment-389572201,https://api.github.com/repos/simonw/datasette/issues/266,389572201,MDEyOklzc3VlQ29tbWVudDM4OTU3MjIwMQ==,9599,simonw,2018-05-16T15:58:43Z,2018-05-16T16:00:47Z,OWNER,"This will likely be implemented in the `BaseView` class, which needs to know how to spot the `.csv` extension, call the underlying JSON generating function and then return the `columns` and `rows` as correctly formatted CSV.
https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/base.py#L201-L207
This means it will take ALL arguments that are available to the `.json` view. It may ignore some (e.g. `_facet=` makes no sense since CSV tables don't have space to show the facet results).
In streaming mode, things will behave a little bit differently - in particular, if `_stream=1` then `_next=` will be forbidden.
It can't include a length header because we don't know how many bytes it will be
CSV output will throw an error if the endpoint doesn't have rows and columns keys eg `/-/inspect.json`
So the implementation...
- looks for the `.csv` extension
- internally fetches the `.json` data instead
- If no `_stream` it just transposes that JSON to CSV with the correct content type header
- If `_stream=1` - checks for `_next=` and throws an error if it was provided
- Otherwise... fetch first page and emit CSV header and first set of rows
- Then start async looping, emitting more CSV rows and following the `_next=` internal reference until done
I like that this takes advantage of efficient pagination. It may not work so well for views which use offset/limit though.
It won't work at all for custom SQL because custom SQL doesn't support _next= pagination. That's fine.
For views... easiest fix is to cut off after first X000 records. That seems OK. View JSON would need to include a property that the mechanism can identify.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-389579363,https://api.github.com/repos/simonw/datasette/issues/266,389579363,MDEyOklzc3VlQ29tbWVudDM4OTU3OTM2Mw==,9599,simonw,2018-05-16T16:20:06Z,2018-05-16T16:20:06Z,OWNER,I started a thread on Twitter discussing various CSV output dialects: https://twitter.com/simonw/status/996783395504979968 - I want to pick defaults which will work as well as possible for whatever tools people might be using to consume the data.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-389579762,https://api.github.com/repos/simonw/datasette/issues/266,389579762,MDEyOklzc3VlQ29tbWVudDM4OTU3OTc2Mg==,9599,simonw,2018-05-16T16:21:12Z,2018-05-16T16:21:12Z,OWNER,"> I basically want someone to tell me which arguments I can pass to Python's csv.writer() function that will result in the least complaints from people who try to parse the results :)
https://twitter.com/simonw/status/996786815938977792","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-389592566,https://api.github.com/repos/simonw/datasette/issues/266,389592566,MDEyOklzc3VlQ29tbWVudDM4OTU5MjU2Ng==,9599,simonw,2018-05-16T17:01:29Z,2018-05-16T17:02:21Z,OWNER,Let's provide a CSV Dialect definition too: https://frictionlessdata.io/specs/csv-dialect/ - via https://twitter.com/drewdaraabrams/status/996794915680997382,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-389608473,https://api.github.com/repos/simonw/datasette/issues/266,389608473,MDEyOklzc3VlQ29tbWVudDM4OTYwODQ3Mw==,9599,simonw,2018-05-16T17:52:35Z,2018-05-16T17:54:11Z,OWNER,"There are some code examples in this issue which should help with the streaming part: https://github.com/channelcat/sanic/issues/1067
Also https://github.com/channelcat/sanic/blob/master/docs/sanic/streaming.md#response-streaming","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-389626715,https://api.github.com/repos/simonw/datasette/issues/266,389626715,MDEyOklzc3VlQ29tbWVudDM4OTYyNjcxNQ==,9599,simonw,2018-05-16T18:50:46Z,2018-05-16T18:50:46Z,OWNER,"> I’d recommend using the Windows-1252 encoding for maximum compatibility, unless you have any characters not in that set, in which case use UTF8 with a byte order mark. Bit of a pain, but some progams (eg various versions of Excel) don’t read UTF8.
**frankieroberto** https://twitter.com/frankieroberto/status/996823071947460616
> There is software that consumes CSV and doesn't speak UTF8!? Huh. Well I can't just use Windows-1252 because I need to support the full UTF8 range of potential data - maybe I should support an optional ?_encoding=windows-1252 argument
**simonw** https://twitter.com/simonw/status/996824677245857793","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-389893810,https://api.github.com/repos/simonw/datasette/issues/266,389893810,MDEyOklzc3VlQ29tbWVudDM4OTg5MzgxMA==,9599,simonw,2018-05-17T14:49:35Z,2018-05-17T14:49:35Z,OWNER,Idea: add a `supports_csv = False` property to `BaseView` and over-ride it to `True` just on the view classes that should support CSV (Table and Row). Slight subtlety: the `DatabaseView` class only supports CSV in the `custom_sql()` path. Maybe that needs to be refactored a bit.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-389894382,https://api.github.com/repos/simonw/datasette/issues/266,389894382,MDEyOklzc3VlQ29tbWVudDM4OTg5NDM4Mg==,9599,simonw,2018-05-17T14:51:13Z,2018-05-17T14:53:23Z,OWNER,"I should definitely sanity check if the `_next=` route really is the most efficient way to build this. It may turn out that iterating over a SQLite cursor with a million rows in it is super-efficient and would provide much more reliable performance (plus solve the problem for retrieving full custom SQL queries where we can't do keyset pagination).
Problem here is that we run SQL queries in a thread pool. A query that returns millions of rows would presumably tie up a SQL thread until it has finished, which could block the server. This may be a reason to stick with `_next=` keyset pagination - since it ensures each SQL thread yields back again after each 1,000 rows.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/271#issuecomment-389989015,https://api.github.com/repos/simonw/datasette/issues/271,389989015,MDEyOklzc3VlQ29tbWVudDM4OTk4OTAxNQ==,9599,simonw,2018-05-17T19:54:10Z,2018-05-17T19:54:10Z,OWNER,"This is a departure from how Datasette has been designed so far, and it may turn out that it's not feasible or it requires too many philosophical changes to be worthwhile.
If we CAN do it though it would mean Datasette could stay running pointed at a directory on disk and new SQLite databases could be dropped into that directory by another process and served directly as they become available.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324162476,Mechanism for automatically picking up changes when on-disk .db file changes,
https://github.com/simonw/datasette/issues/271#issuecomment-389989615,https://api.github.com/repos/simonw/datasette/issues/271,389989615,MDEyOklzc3VlQ29tbWVudDM4OTk4OTYxNQ==,9599,simonw,2018-05-17T19:56:13Z,2018-05-17T19:56:13Z,OWNER,"From https://www.sqlite.org/c3ref/open.html
> **immutable**: The immutable parameter is a boolean query parameter that indicates that the database file is stored on read-only media. When immutable is set, SQLite assumes that the database file cannot be changed, even by a process with higher privilege, and so the database is opened read-only and all locking and change detection is disabled. Caution: Setting the immutable property on a database file that does in fact change can result in incorrect query results and/or SQLITE_CORRUPT errors. See also: SQLITE_IOCAP_IMMUTABLE.
So this would probably have to be a new mode, `datasette serve --detect-db-changes`, which no longer opens in immutable mode. Or maybe current behavior becomes not-the-default and you opt into it with `datasette serve --immutable`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324162476,Mechanism for automatically picking up changes when on-disk .db file changes,
https://github.com/simonw/datasette/issues/270#issuecomment-390105147,https://api.github.com/repos/simonw/datasette/issues/270,390105147,MDEyOklzc3VlQ29tbWVudDM5MDEwNTE0Nw==,9599,simonw,2018-05-18T06:13:07Z,2018-05-18T06:13:07Z,OWNER,I'm going to add a `/-/limits` page that shows the current limits.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323830051,--limit= CLI option for setting limits,
https://github.com/simonw/datasette/issues/264#issuecomment-390105943,https://api.github.com/repos/simonw/datasette/issues/264,390105943,MDEyOklzc3VlQ29tbWVudDM5MDEwNTk0Mw==,9599,simonw,2018-05-18T06:18:00Z,2018-05-18T06:18:00Z,OWNER,Docs: http://datasette.readthedocs.io/en/latest/limits.html#default-facet-size,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323673899,Make it possible to customize various facet settings,
https://github.com/simonw/datasette/issues/273#issuecomment-390250253,https://api.github.com/repos/simonw/datasette/issues/273,390250253,MDEyOklzc3VlQ29tbWVudDM5MDI1MDI1Mw==,198537,rgieseke,2018-05-18T15:49:52Z,2018-05-18T15:49:52Z,CONTRIBUTOR,"Shouldn't [versioneer](https://github.com/warner/python-versioneer) do that?
E.g. 0.21+2.g1076c97
You'd need to install via `pip install git+https://github.com/simow/datasette.git` though, this does a temp git clone.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324451322,Figure out a way to have /-/version return current git commit hash,
https://github.com/simonw/datasette/issues/274#issuecomment-390433040,https://api.github.com/repos/simonw/datasette/issues/274,390433040,MDEyOklzc3VlQ29tbWVudDM5MDQzMzA0MA==,9599,simonw,2018-05-19T21:12:42Z,2018-05-20T16:01:03Z,OWNER,Could also support these as optional environment variables - `DATASETTE_NAMEOFSETTING`,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324652142,"Rename --limit to --config, add --help-config",
https://github.com/simonw/datasette/issues/274#issuecomment-390496376,https://api.github.com/repos/simonw/datasette/issues/274,390496376,MDEyOklzc3VlQ29tbWVudDM5MDQ5NjM3Ng==,9599,simonw,2018-05-20T17:04:55Z,2018-05-20T17:04:55Z,OWNER,http://datasette.readthedocs.io/en/latest/config.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324652142,"Rename --limit to --config, add --help-config",
https://github.com/simonw/datasette/pull/258#issuecomment-390577711,https://api.github.com/repos/simonw/datasette/issues/258,390577711,MDEyOklzc3VlQ29tbWVudDM5MDU3NzcxMQ==,247131,philroche,2018-05-21T07:38:15Z,2018-05-21T07:38:15Z,NONE,"Excellent, I was not aware of the auto redirect to the new hash. My bad
This solves my use case.
I do agree that your suggested --no-url-hash approach is much neater. I will investigate ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322741659,Add new metadata key persistent_urls which removes the hash from all database urls,
https://github.com/simonw/datasette/issues/247#issuecomment-390689406,https://api.github.com/repos/simonw/datasette/issues/247,390689406,MDEyOklzc3VlQ29tbWVudDM5MDY4OTQwNg==,11912854,jsancho-gpl,2018-05-21T15:29:31Z,2018-05-21T15:29:31Z,NONE,"I've changed my mind about the way to support external connectors aside of SQLite and I'm working in a more simple style that respects the original Datasette, i.e. less refactoring. I present you [a version of Datasette wich supports other database connectors](https://github.com/jsancho-gpl/datasette/tree/external-connectors) and [a Datasette connector for HDF5/PyTables files](https://github.com/jsancho-gpl/datasette-pytables).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",319449852,SQLite code decoupled from Datasette,
https://github.com/simonw/datasette/pull/277#issuecomment-390707183,https://api.github.com/repos/simonw/datasette/issues/277,390707183,MDEyOklzc3VlQ29tbWVudDM5MDcwNzE4Mw==,9599,simonw,2018-05-21T16:28:39Z,2018-05-21T16:28:39Z,OWNER,"This is definitely a big improvement.
I'd like to refactor the unit tests that cover .inspect() too - currently they are a huge ugly blob at the top of test_api.py","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324836533,Refactor inspect logic,
https://github.com/simonw/datasette/issues/276#issuecomment-390707760,https://api.github.com/repos/simonw/datasette/issues/276,390707760,MDEyOklzc3VlQ29tbWVudDM5MDcwNzc2MA==,9599,simonw,2018-05-21T16:30:35Z,2018-05-21T16:30:35Z,OWNER,"This probably needs to be in a plugin simply because getting Spatialite compiled and installed is a bit of a pain.
It's a great opportunity to expand the plugin hooks in useful ways though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/276#issuecomment-390795067,https://api.github.com/repos/simonw/datasette/issues/276,390795067,MDEyOklzc3VlQ29tbWVudDM5MDc5NTA2Nw==,45057,russss,2018-05-21T21:55:57Z,2018-05-21T21:55:57Z,CONTRIBUTOR,"Well, we do have the capability to detect spatialite so my intention certainly wasn't to require it.
I can see the advantage of having it as a plugin but it does touch a number of points in the code. I think I'm going to attack this by refactoring the necessary bits and seeing where that leads (which was my plan anyway).
I think my main concern is - if I add certain plugin hooks for this, is anything else ever going to use them? I'm not sure I have an answer to that question yet, either way.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/pull/277#issuecomment-390804333,https://api.github.com/repos/simonw/datasette/issues/277,390804333,MDEyOklzc3VlQ29tbWVudDM5MDgwNDMzMw==,9599,simonw,2018-05-21T22:40:16Z,2018-05-21T22:43:50Z,OWNER,"We should merge this before refactoring the tests though, because that way we don't couple the new tests to the verification of this change.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324836533,Refactor inspect logic,
https://github.com/simonw/datasette/issues/278#issuecomment-390991640,https://api.github.com/repos/simonw/datasette/issues/278,390991640,MDEyOklzc3VlQ29tbWVudDM5MDk5MTY0MA==,9599,simonw,2018-05-22T13:33:46Z,2018-05-22T13:33:46Z,OWNER,For SpatiaLite this example may be useful - though it's building 4.3.0 and not 4.4.0: https://github.com/terranodo/spatialite-docker/blob/master/Dockerfile,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325294102,Build smallest possible Docker image with Datasette plus recent SQLite (with json1) plus Spatialite 4.4.0,
https://github.com/simonw/datasette/issues/278#issuecomment-390993397,https://api.github.com/repos/simonw/datasette/issues/278,390993397,MDEyOklzc3VlQ29tbWVudDM5MDk5MzM5Nw==,9599,simonw,2018-05-22T13:38:57Z,2018-05-22T13:38:57Z,OWNER,"Useful GitHub code search: https://github.com/search?utf8=✓&q=%22libspatialite-4.4.0%22+%22RC0%22&type=Code
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325294102,Build smallest possible Docker image with Datasette plus recent SQLite (with json1) plus Spatialite 4.4.0,
https://github.com/simonw/datasette/issues/255#issuecomment-390999055,https://api.github.com/repos/simonw/datasette/issues/255,390999055,MDEyOklzc3VlQ29tbWVudDM5MDk5OTA1NQ==,9599,simonw,2018-05-22T13:54:55Z,2018-05-22T13:54:55Z,OWNER,This shipped in Datasette 0.22. Here's my blog post about it: https://simonwillison.net/2018/May/20/datasette-facets/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322477187,Facets,
https://github.com/simonw/datasette/issues/276#issuecomment-391000659,https://api.github.com/repos/simonw/datasette/issues/276,391000659,MDEyOklzc3VlQ29tbWVudDM5MTAwMDY1OQ==,9599,simonw,2018-05-22T13:59:27Z,2018-05-22T13:59:27Z,OWNER,"Right now the plugin stuff is early enough that I'd like to get as many potential plugin hooks as possible crafted out A much easier to judge if they should be added as actual hooks if we have a working branch prototype of them.
Some kind of mechanism for custom column display is already needed - eg there are columns where I want to say ""render this as markdown"" or ""URLify any links in this text"" - or even ""use this date format"" or ""add commas to this integer"".
You can do it with a custom template but a lower-level mechanism would be nicer. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/273#issuecomment-391003285,https://api.github.com/repos/simonw/datasette/issues/273,391003285,MDEyOklzc3VlQ29tbWVudDM5MTAwMzI4NQ==,9599,simonw,2018-05-22T14:06:40Z,2018-05-22T14:06:40Z,OWNER,"That looks great. I don't think it's possible to derive the current commit version from the .zip downloaded directly from GitHub, so needing to pip install via git+https feels reasonable to me.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324451322,Figure out a way to have /-/version return current git commit hash,
https://github.com/simonw/datasette/issues/272#issuecomment-391011268,https://api.github.com/repos/simonw/datasette/issues/272,391011268,MDEyOklzc3VlQ29tbWVudDM5MTAxMTI2OA==,9599,simonw,2018-05-22T14:28:12Z,2018-05-22T14:28:12Z,OWNER,"I think I can do this almost entirely within my existing BaseView class structure.
First, decouple the async data() methods by teaching them to take a querystring object as an argument instead of a Sanic request object. The get() method can then send that new object instead of a request.
Next teach the base class how to obey the ASGI protocol.
I should be able to get support for both Sanic and uvicorn/daphne working in the same codebase, which will make it easy to compare their performance. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/278#issuecomment-390993861,https://api.github.com/repos/simonw/datasette/issues/278,390993861,MDEyOklzc3VlQ29tbWVudDM5MDk5Mzg2MQ==,9599,simonw,2018-05-22T13:40:14Z,2018-05-22T14:38:05Z,OWNER,If we can't get `import sqlite3` to load the latest version but we can get `import pysqlite3` to work that's fine too - I can teach Datasette to import the best available version.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325294102,Build smallest possible Docker image with Datasette plus recent SQLite (with json1) plus Spatialite 4.4.0,
https://github.com/simonw/datasette/issues/276#issuecomment-391025841,https://api.github.com/repos/simonw/datasette/issues/276,391025841,MDEyOklzc3VlQ29tbWVudDM5MTAyNTg0MQ==,9599,simonw,2018-05-22T15:06:36Z,2018-05-22T15:06:36Z,OWNER,"The other reason I mention plugins is that I have an idea to outlaw JavaScript entirely from Datasette core and instead encourage ALL JavaScript functionality to move into plugins.right now that just means CodeMirror. I may set up some of those plugins (like CodeMirror) as default dependencies so you get them from ""pip install datasette"".
I like the neatness of saying that core Datasette is a very simple JSON + HTML application, then encouraging people to go completely wild with JavaScript in the plugins.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/243#issuecomment-391030083,https://api.github.com/repos/simonw/datasette/issues/243,391030083,MDEyOklzc3VlQ29tbWVudDM5MTAzMDA4Mw==,9599,simonw,2018-05-22T15:17:10Z,2018-05-22T15:17:10Z,OWNER,See also #278,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",318737808,--spatialite option for datasette publish commands,
https://github.com/simonw/datasette/issues/276#issuecomment-391050113,https://api.github.com/repos/simonw/datasette/issues/276,391050113,MDEyOklzc3VlQ29tbWVudDM5MTA1MDExMw==,45057,russss,2018-05-22T16:13:00Z,2018-05-22T16:13:00Z,CONTRIBUTOR,"Yup, I'll have a think about it. My current thoughts are for spatialite we'll need to hook into the following places:
* Inspection, so we can detect which columns are geometry columns. (We also currently ignore spatialite tables during inspection, it may be worth moving that to the plugin as well.)
* After data load, so we can convert WKB into the correct intermediate format for display. The alternative here is to alter the select SQL itself and get spatialite to do this conversion, but that strikes me as a bit more complex and possibly not as useful.
* HTML rendering.
* Querying?
The rendering and querying hooks could also potentially be used to move the units support into a plugin.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/pull/279#issuecomment-391055490,https://api.github.com/repos/simonw/datasette/issues/279,391055490,MDEyOklzc3VlQ29tbWVudDM5MTA1NTQ5MA==,9599,simonw,2018-05-22T16:29:30Z,2018-05-22T16:29:30Z,OWNER,"This is fantastic!
I think I prefer the aesthetics of just ""0.22"" for the version string if it's a tagged release with no additional changes - does that work?
I'd like to continue to provide a tuple that can be imported from the version.py module as well, as seen here:
https://github.com/simonw/datasette/blob/558d9d7bfef3dd633eb16389281b67d42c9bdeef/datasette/version.py#L1
Presumably we can generate that from the versioneer string?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325352370,Add version number support with Versioneer,
https://github.com/simonw/datasette/pull/280#issuecomment-391059008,https://api.github.com/repos/simonw/datasette/issues/280,391059008,MDEyOklzc3VlQ29tbWVudDM5MTA1OTAwOA==,565628,r4vi,2018-05-22T16:40:27Z,2018-05-22T16:40:27Z,CONTRIBUTOR,"```python
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.23.1'
>>>
```
running the above in the container seems to show 3.23.1 too so maybe we don't need pysqlite3 at all?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325373747,Build Dockerfile with recent Sqlite + Spatialite,
https://github.com/simonw/datasette/pull/279#issuecomment-391073009,https://api.github.com/repos/simonw/datasette/issues/279,391073009,MDEyOklzc3VlQ29tbWVudDM5MTA3MzAwOQ==,198537,rgieseke,2018-05-22T17:23:26Z,2018-05-22T17:23:26Z,CONTRIBUTOR,"> I think I prefer the aesthetics of just ""0.22"" for the version string if it's a tagged release with no additional changes - does that work?
Yes! That's the default versioneer behaviour.
> I'd like to continue to provide a tuple that can be imported from the version.py module as well, as seen here:
Should work now, it can be a two (for a tagged version), three or four items tuple.
```
In [2]: datasette.__version__
Out[2]: '0.12+292.ga70c2a8.dirty'
In [3]: datasette.__version_info__
Out[3]: ('0', '12+292', 'ga70c2a8', 'dirty')
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325352370,Add version number support with Versioneer,
https://github.com/simonw/datasette/pull/279#issuecomment-391073267,https://api.github.com/repos/simonw/datasette/issues/279,391073267,MDEyOklzc3VlQ29tbWVudDM5MTA3MzI2Nw==,198537,rgieseke,2018-05-22T17:24:16Z,2018-05-22T17:24:16Z,CONTRIBUTOR,"Sorry, just realised you rely on `version` being a module ...","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325352370,Add version number support with Versioneer,
https://github.com/simonw/datasette/pull/280#issuecomment-391076239,https://api.github.com/repos/simonw/datasette/issues/280,391076239,MDEyOklzc3VlQ29tbWVudDM5MTA3NjIzOQ==,9599,simonw,2018-05-22T17:33:33Z,2018-05-22T17:33:33Z,OWNER,This looks amazing! Can't wait to try this out this evening.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325373747,Build Dockerfile with recent Sqlite + Spatialite,
https://github.com/simonw/datasette/pull/280#issuecomment-391076458,https://api.github.com/repos/simonw/datasette/issues/280,391076458,MDEyOklzc3VlQ29tbWVudDM5MTA3NjQ1OA==,9599,simonw,2018-05-22T17:34:13Z,2018-05-22T17:34:13Z,OWNER,Yeah let's try this without pysqlite3 and see if we still get the correct version.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325373747,Build Dockerfile with recent Sqlite + Spatialite,
https://github.com/simonw/datasette/pull/279#issuecomment-391077700,https://api.github.com/repos/simonw/datasette/issues/279,391077700,MDEyOklzc3VlQ29tbWVudDM5MTA3NzcwMA==,198537,rgieseke,2018-05-22T17:38:17Z,2018-05-22T17:38:17Z,CONTRIBUTOR,"Alright, that should work now -- let me know if you would prefer any different behaviour.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325352370,Add version number support with Versioneer,
https://github.com/simonw/datasette/pull/280#issuecomment-391141391,https://api.github.com/repos/simonw/datasette/issues/280,391141391,MDEyOklzc3VlQ29tbWVudDM5MTE0MTM5MQ==,565628,r4vi,2018-05-22T21:08:39Z,2018-05-22T21:08:39Z,CONTRIBUTOR,"I'm going to clean this up for consistency tomorrow morning so hold off
merging until then please
On Tue, May 22, 2018 at 6:34 PM, Simon Willison
wrote:
> Yeah let's try this without pysqlite3 and see if we still get the correct
> version.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> , or mute
> the thread
>
> .
>
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325373747,Build Dockerfile with recent Sqlite + Spatialite,
https://github.com/simonw/datasette/pull/280#issuecomment-391190497,https://api.github.com/repos/simonw/datasette/issues/280,391190497,MDEyOklzc3VlQ29tbWVudDM5MTE5MDQ5Nw==,9599,simonw,2018-05-23T01:22:53Z,2018-05-23T01:22:53Z,OWNER,"I grabbed just your Dockerfile and built it like this:
docker build . -t datasette
Once it had built, I ran it like this:
docker run -p 8001:8001 -v `pwd`:/mnt datasette \
datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db \
--load-extension=/usr/local/lib/mod_spatialite.so
(The fixtures.db file is created by running `python tests/fixtures.py fixtures.db`)
Then I visited http://localhost:8001/-/versions and I got this:
{
""datasette"": {
""version"": ""0+unknown""
},
""python"": {
""full"": ""3.6.3 (default, Dec 12 2017, 06:37:05) \n[GCC 6.3.0 20170516]"",
""version"": ""3.6.3""
},
""sqlite"": {
""extensions"": {
""json1"": null,
""spatialite"": ""4.4.0-RC0""
},
""fts_versions"": [
""FTS4"",
""FTS3""
],
""version"": ""3.23.1""
}
}
Fantastic! I'm getting SQLite `3.23.1` and SpatiaLite `4.4.0-RC0`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325373747,Build Dockerfile with recent Sqlite + Spatialite,
https://github.com/simonw/datasette/pull/280#issuecomment-391290271,https://api.github.com/repos/simonw/datasette/issues/280,391290271,MDEyOklzc3VlQ29tbWVudDM5MTI5MDI3MQ==,565628,r4vi,2018-05-23T09:53:38Z,2018-05-23T09:53:38Z,CONTRIBUTOR,"Running:
```bash
docker run -p 8001:8001 -v `pwd`:/mnt datasette \
datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db \
--load-extension=/usr/local/lib/mod_spatialite.so
```
is now returning FTS5 enabled in the versions output:
```json
{
""datasette"": {
""version"": ""0.22""
},
""python"": {
""full"": ""3.6.5 (default, May 5 2018, 03:07:21) \n[GCC 6.3.0 20170516]"",
""version"": ""3.6.5""
},
""sqlite"": {
""extensions"": {
""json1"": null,
""spatialite"": ""4.4.0-RC0""
},
""fts_versions"": [
""FTS5"",
""FTS4"",
""FTS3""
],
""version"": ""3.23.1""
}
}
```
The old query didn't work because specifying `(t TEXT)` caused an error","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325373747,Build Dockerfile with recent Sqlite + Spatialite,
https://github.com/simonw/datasette/pull/280#issuecomment-391354237,https://api.github.com/repos/simonw/datasette/issues/280,391354237,MDEyOklzc3VlQ29tbWVudDM5MTM1NDIzNw==,9599,simonw,2018-05-23T13:51:22Z,2018-05-23T13:51:22Z,OWNER,@r4vi any objections to me merging this?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325373747,Build Dockerfile with recent Sqlite + Spatialite,
https://github.com/simonw/datasette/issues/282#issuecomment-391355099,https://api.github.com/repos/simonw/datasette/issues/282,391355099,MDEyOklzc3VlQ29tbWVudDM5MTM1NTA5OQ==,9599,simonw,2018-05-23T13:53:39Z,2018-05-23T13:53:39Z,OWNER,Confirmed fixed: https://fivethirtyeight-datasette-mipwbeadvr.now.sh/fivethirtyeight-5de27e3/nba-elo%2Fnbaallelo?_facet=lg_id&_next=100 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325705981,Faceting breaks pagination,
https://github.com/simonw/datasette/pull/280#issuecomment-391355030,https://api.github.com/repos/simonw/datasette/issues/280,391355030,MDEyOklzc3VlQ29tbWVudDM5MTM1NTAzMA==,565628,r4vi,2018-05-23T13:53:27Z,2018-05-23T15:22:45Z,CONTRIBUTOR,"No objections;
It's good to go @simonw
On Wed, 23 May 2018, 14:51 Simon Willison, wrote:
> @r4vi any objections to me merging this?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or mute
> the thread
>
> .
>
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325373747,Build Dockerfile with recent Sqlite + Spatialite,
https://github.com/simonw/datasette/pull/280#issuecomment-391437199,https://api.github.com/repos/simonw/datasette/issues/280,391437199,MDEyOklzc3VlQ29tbWVudDM5MTQzNzE5OQ==,9599,simonw,2018-05-23T17:44:20Z,2018-05-23T17:44:20Z,OWNER,Thank you very much! This is most excellent.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325373747,Build Dockerfile with recent Sqlite + Spatialite,
https://github.com/simonw/datasette/pull/281#issuecomment-391437462,https://api.github.com/repos/simonw/datasette/issues/281,391437462,MDEyOklzc3VlQ29tbWVudDM5MTQzNzQ2Mg==,9599,simonw,2018-05-23T17:45:07Z,2018-05-23T17:45:07Z,OWNER,I'm afraid I just merged #280 which means this no longer applies. You're very welcome to see if you can further optimize the new Dockerfile though.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325553991,Reduces image size using Alpine + Multistage (re: #278),
https://github.com/simonw/datasette/issues/276#issuecomment-391504199,https://api.github.com/repos/simonw/datasette/issues/276,391504199,MDEyOklzc3VlQ29tbWVudDM5MTUwNDE5OQ==,9599,simonw,2018-05-23T21:35:17Z,2018-05-23T21:35:17Z,OWNER,"I'm not keen on anything that modifies the SQLite file itself on startup - part of the datasette contract is that it should work with any SQLite file you throw at it without having any side-effects.
A neat thing about SQLite is that because everything happens in the same process there's very little additional overhead involved in executing extra SQL queries - even if we ran a query-per-row to transform data in one specific column it shouldn't add more than a few ms to the total page load time (whereas with MySQL all of the extra query overhead would kill us).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/276#issuecomment-391504757,https://api.github.com/repos/simonw/datasette/issues/276,391504757,MDEyOklzc3VlQ29tbWVudDM5MTUwNDc1Nw==,9599,simonw,2018-05-23T21:37:07Z,2018-05-23T21:37:18Z,OWNER,"That said, it looks like we may be able to use a library like https://github.com/geomet/geomet to run the conversion from WKB entirely in Python space.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/276#issuecomment-391505930,https://api.github.com/repos/simonw/datasette/issues/276,391505930,MDEyOklzc3VlQ29tbWVudDM5MTUwNTkzMA==,45057,russss,2018-05-23T21:41:37Z,2018-05-23T21:41:37Z,CONTRIBUTOR,"> I'm not keen on anything that modifies the SQLite file itself on startup
Ah I didn't mean that - I meant altering the SELECT query to fetch the data so that it ran a spatialite function to transform that specific column.
I think that's less useful as a general-purpose plugin hook though, and it's not that hard to parse the WKB in Python (my default approach would be to use [shapely](https://github.com/Toblerity/Shapely), which is great, but geomet looks like an interesting pure-python alternative).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/283#issuecomment-391583528,https://api.github.com/repos/simonw/datasette/issues/283,391583528,MDEyOklzc3VlQ29tbWVudDM5MTU4MzUyOA==,9599,simonw,2018-05-24T04:21:49Z,2018-05-24T04:21:49Z,OWNER,"The challenge here is which database should be the ""default"" database. The first database attached to SQLite is treated as the default - if no database is specified in a query, that's the database that queries will be executed against.
Currently, each database URL in Datasette (e.g. https://san-francisco.datasettes.com/sf-film-locations-84594a7 v.s. https://san-francisco.datasettes.com/sf-trees-ebc2ad9 ) gets its own independent connection, and all queries within that base URL run against that database.
If we're going to attach multiple databases to the same connection, how do we set which database gets to be the default?
The easiest thing to do here will be to have a special database (maybe which is turned off by default and can be enabled using `datasette serve --enable-cross-database-joins` or similar) which attaches to ALL the databases. Perhaps it starts as an in-memory database, maybe at `/memory`?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/283#issuecomment-391584366,https://api.github.com/repos/simonw/datasette/issues/283,391584366,MDEyOklzc3VlQ29tbWVudDM5MTU4NDM2Ng==,9599,simonw,2018-05-24T04:28:20Z,2018-05-24T04:28:20Z,OWNER,"I used some pretty ugly hacks, like faking an entire `.inspect()` block for the `:memory:` database just to get past the errors I was seeing. To ship this as a feature it will need quite a bit of code refactoring to make those hacks unnecessary.
https://github.com/simonw/datasette/blob/7a3040f5782375373b2b66e5969bc2c49b3a6f0e/datasette/views/database.py#L18-L26","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/283#issuecomment-391584527,https://api.github.com/repos/simonw/datasette/issues/283,391584527,MDEyOklzc3VlQ29tbWVudDM5MTU4NDUyNw==,9599,simonw,2018-05-24T04:29:40Z,2018-05-24T04:29:40Z,OWNER,Rather than stealing the `/memory` namespace for this it would be nicer if these cross-database joins could be executed at the very top-level URL of the Datasette instance - `https://example.com/?sql=...`,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/283#issuecomment-391584112,https://api.github.com/repos/simonw/datasette/issues/283,391584112,MDEyOklzc3VlQ29tbWVudDM5MTU4NDExMg==,9599,simonw,2018-05-24T04:26:29Z,2018-05-24T04:30:50Z,OWNER,"I built a very rough prototype of this to prove it could work. It's deployed here - and here's an example of a query that joins across two different databases:
https://datasette-cross-database-joins-prototype.now.sh/memory?sql=select+fivethirtyeight.%5Blove-actually%2Flove_actually_adjacencies%5D.rowid%2C%0D%0Afivethirtyeight.%5Blove-actually%2Flove_actually_adjacencies%5D.actors%2C%0D%0A%5Bgoogle-trends%5D.%5B20150430_UKDebate%5D.city%0D%0Afrom+fivethirtyeight.%5Blove-actually%2Flove_actually_adjacencies%5D%0D%0Ajoin+%5Bgoogle-trends%5D.%5B20150430_UKDebate%5D%0D%0A++on+%5Bgoogle-trends%5D.%5B20150430_UKDebate%5D.rowid+%3D+fivethirtyeight.%5Blove-actually%2Flove_actually_adjacencies%5D.rowid
```
select fivethirtyeight.[love-actually/love_actually_adjacencies].rowid,
fivethirtyeight.[love-actually/love_actually_adjacencies].actors,
[google-trends].[20150430_UKDebate].city
from fivethirtyeight.[love-actually/love_actually_adjacencies]
join [google-trends].[20150430_UKDebate]
on [google-trends].[20150430_UKDebate].rowid = fivethirtyeight.[love-actually/love_actually_adjacencies].rowid
```
I deployed it like this:
datasette publish now --branch=cross-database-joins fivethirtyeight.db google-trends.db --name=datasette-cross-database-joins-prototype
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/283#issuecomment-391752218,https://api.github.com/repos/simonw/datasette/issues/283,391752218,MDEyOklzc3VlQ29tbWVudDM5MTc1MjIxOA==,9599,simonw,2018-05-24T15:15:19Z,2018-05-24T15:15:19Z,OWNER,Most of the time Datasette is used with just a single database file. So maybe it makes sense for this option to be turned on by default and to ALWAYS be available on the Datasette instance homepage unless the user has explicitly disabled it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/283#issuecomment-391752425,https://api.github.com/repos/simonw/datasette/issues/283,391752425,MDEyOklzc3VlQ29tbWVudDM5MTc1MjQyNQ==,9599,simonw,2018-05-24T15:15:51Z,2018-05-24T15:15:51Z,OWNER,"This would make Datasett's SQL features a lot more instantly obvious to people who land on a homepage, which is probably a good thing.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/283#issuecomment-391752629,https://api.github.com/repos/simonw/datasette/issues/283,391752629,MDEyOklzc3VlQ29tbWVudDM5MTc1MjYyOQ==,9599,simonw,2018-05-24T15:16:25Z,2018-05-24T15:16:25Z,OWNER,"Should this support canned queries too? I think it should, though that raises interesting questions regarding their URL structure.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/283#issuecomment-391752882,https://api.github.com/repos/simonw/datasette/issues/283,391752882,MDEyOklzc3VlQ29tbWVudDM5MTc1Mjg4Mg==,9599,simonw,2018-05-24T15:17:10Z,2018-05-24T15:17:10Z,OWNER,Another option: give this the `/-/all` URL namespace.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/283#issuecomment-391754506,https://api.github.com/repos/simonw/datasette/issues/283,391754506,MDEyOklzc3VlQ29tbWVudDM5MTc1NDUwNg==,9599,simonw,2018-05-24T15:21:37Z,2018-05-24T15:21:53Z,OWNER,"Giving it `/all/` would be easier since that way the existing URL routes (including canned queries) would all work... but I would have to teach it NOT to expect a database content hash on that URL.
Or maybe it should still have a content hash (to enable far-future cache expiry headers on query results) but the hash should be constructed out of all of the other database hashes concatenated together.
That way the URLs would be `/all-5de27e3` and `/all-5de27e3/canned-query-name`
Only downside: this would make it impossible to have a database file with the name `all.db`. I think that's probably an OK trade-off. You could turn the feature off with a config flag if you really want to use that filename (for whatever reason).
How about `/-all-5de27e3/` instead to avoid collisions?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/283#issuecomment-391755300,https://api.github.com/repos/simonw/datasette/issues/283,391755300,MDEyOklzc3VlQ29tbWVudDM5MTc1NTMwMA==,9599,simonw,2018-05-24T15:23:37Z,2018-05-24T15:23:37Z,OWNER,On the `/-all-5de27e3` page we can show the regular https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3 interface but instead of the list of tables we can show a list of attached databases plus some help text showing how to construct a cross-database join.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/283#issuecomment-391756841,https://api.github.com/repos/simonw/datasette/issues/283,391756841,MDEyOklzc3VlQ29tbWVudDM5MTc1Njg0MQ==,9599,simonw,2018-05-24T15:27:42Z,2018-05-24T15:27:42Z,OWNER,"For an example query that pre-populates that textarea... maybe a UNION that pulls the first 10 rows from the first table of each of the first two databases?
```
select * from (select rowid, actors from fivethirtyeight.[love-actually/love_actually_adjacencies] limit 10)
union all
select * from (select rowid, city from [google-trends].[20150430_UKDebate] limit 10)
```
https://datasette-cross-database-joins-prototype.now.sh/memory?sql=select+*+from+%28select+rowid%2C+actors+from+fivethirtyeight.%5Blove-actually%2Flove_actually_adjacencies%5D+limit+10%29%0D%0A+++union+all%0D%0Aselect+*+from+%28select+rowid%2C+city+from+%5Bgoogle-trends%5D.%5B20150430_UKDebate%5D+limit+10%29","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/284#issuecomment-391765706,https://api.github.com/repos/simonw/datasette/issues/284,391765706,MDEyOklzc3VlQ29tbWVudDM5MTc2NTcwNg==,9599,simonw,2018-05-24T15:52:24Z,2018-05-24T15:52:24Z,OWNER,I'm not crazy about the `enable_` prefix on these.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326182814,Ability to enable/disable specific features via --config,
https://github.com/simonw/datasette/issues/284#issuecomment-391765973,https://api.github.com/repos/simonw/datasette/issues/284,391765973,MDEyOklzc3VlQ29tbWVudDM5MTc2NTk3Mw==,9599,simonw,2018-05-24T15:53:08Z,2018-05-24T15:53:08Z,OWNER,This will also give us a mechanism for turning on and off the cross-database joins feature from #283,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326182814,Ability to enable/disable specific features via --config,
https://github.com/simonw/datasette/issues/284#issuecomment-391766420,https://api.github.com/repos/simonw/datasette/issues/284,391766420,MDEyOklzc3VlQ29tbWVudDM5MTc2NjQyMA==,9599,simonw,2018-05-24T15:54:33Z,2018-05-24T15:54:33Z,OWNER,"Maybe `allow_sql`, `allow_facet` and `allow_download`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326182814,Ability to enable/disable specific features via --config,
https://github.com/simonw/datasette/issues/283#issuecomment-391768302,https://api.github.com/repos/simonw/datasette/issues/283,391768302,MDEyOklzc3VlQ29tbWVudDM5MTc2ODMwMg==,9599,simonw,2018-05-24T16:00:05Z,2018-05-24T16:00:05Z,OWNER,I like `/-/all-5de27e3` for this (with `/-/all` redirecting to the correct hash),"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325958506,Support cross-database joins,
https://github.com/simonw/datasette/issues/275#issuecomment-391771202,https://api.github.com/repos/simonw/datasette/issues/275,391771202,MDEyOklzc3VlQ29tbWVudDM5MTc3MTIwMg==,9599,simonw,2018-05-24T16:08:41Z,2018-05-24T16:08:41Z,OWNER,"So the lookup priority order should be:
* table level in metadata
* database level in metadata
* root level in metadata
* `--config` options passed to `datasette serve`
* `DATASETTE_X` environment variables","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324720095,"""config"" section in metadata.json (root, database and table level)",
https://github.com/simonw/datasette/issues/275#issuecomment-391771658,https://api.github.com/repos/simonw/datasette/issues/275,391771658,MDEyOklzc3VlQ29tbWVudDM5MTc3MTY1OA==,9599,simonw,2018-05-24T16:09:55Z,2018-05-24T16:09:55Z,OWNER,It feels slightly weird continuing to call it `metadata.json` as it starts to grow support for config options (which already started with the `units` and `facets` keys) but I can live with that.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324720095,"""config"" section in metadata.json (root, database and table level)",
https://github.com/simonw/datasette/issues/284#issuecomment-391912392,https://api.github.com/repos/simonw/datasette/issues/284,391912392,MDEyOklzc3VlQ29tbWVudDM5MTkxMjM5Mg==,9599,simonw,2018-05-25T01:16:56Z,2018-05-25T01:17:13Z,OWNER,`allow_sql` should only affect the `?sql=` parameter and whether or not the form is displayed. You should still be able to use and execute canned queries even if this option is turned off.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326182814,Ability to enable/disable specific features via --config,
https://github.com/simonw/datasette/issues/284#issuecomment-391950691,https://api.github.com/repos/simonw/datasette/issues/284,391950691,MDEyOklzc3VlQ29tbWVudDM5MTk1MDY5MQ==,9599,simonw,2018-05-25T06:01:23Z,2018-05-25T06:05:02Z,OWNER,"Demo:
datasette publish now --branch=master fixtures.db \
--source=""#284 Demo"" \
--source_url=""https://github.com/simonw/datasette/issues/284"" \
--extra-options ""--config allow_sql:off --config allow_facet:off --config allow_download:off"" \
--name=datasette-demo-284
now alias https://datasette-demo-284-jogjwngegj.now.sh datasette-demo-284.now.sh
https://datasette-demo-284.now.sh/
Note the following:
* https://datasette-demo-284.now.sh/fixtures-fda0fea has no SQL input textarea
* https://datasette-demo-284.now.sh/fixtures-fda0fea has no database download link
* https://datasette-demo-284.now.sh/fixtures-fda0fea.db returns 403 forbidden
* https://datasette-demo-284.now.sh/fixtures-fda0fea?sql=select%20*%20from%20sqlite_master throws error 400
* https://datasette-demo-284.now.sh/fixtures-fda0fea/facetable shows no suggested facets
* https://datasette-demo-284.now.sh/fixtures-fda0fea/facetable?_facet=city_id throws error 400","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326182814,Ability to enable/disable specific features via --config,
https://github.com/simonw/datasette/issues/286#issuecomment-392121500,https://api.github.com/repos/simonw/datasette/issues/286,392121500,MDEyOklzc3VlQ29tbWVudDM5MjEyMTUwMA==,9599,simonw,2018-05-25T17:06:46Z,2018-05-25T17:06:46Z,OWNER,"A few extra thoughts:
* Some users may want to opt out of this. We could have `--config version_in_hash:false`
* should this affect the filename for the downloadable copy of the SQLite database? Maybe that should stay as just the hash of the contents, but that's a fair bit more complex
* What about users who stick with the same version of datasette but deploy changes to their custom templates - how can we help them cache bust? Maybe with `--config cache_version:2`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326599525,Database hash should include current datasette version,
https://github.com/simonw/datasette/issues/286#issuecomment-392121743,https://api.github.com/repos/simonw/datasette/issues/286,392121743,MDEyOklzc3VlQ29tbWVudDM5MjEyMTc0Mw==,9599,simonw,2018-05-25T17:07:36Z,2018-05-25T17:07:36Z,OWNER,This is also a great excuse to finally write up some detailed documentation on Datasette's caching strategy,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326599525,Database hash should include current datasette version,
https://github.com/simonw/datasette/issues/267#issuecomment-392121905,https://api.github.com/repos/simonw/datasette/issues/267,392121905,MDEyOklzc3VlQ29tbWVudDM5MjEyMTkwNQ==,9599,simonw,2018-05-25T17:08:14Z,2018-05-25T17:08:14Z,OWNER,See also #286,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323716411,"Documentation for URL hashing, redirects and cache policy",
https://github.com/simonw/datasette/issues/259#issuecomment-392212119,https://api.github.com/repos/simonw/datasette/issues/259,392212119,MDEyOklzc3VlQ29tbWVudDM5MjIxMjExOQ==,9599,simonw,2018-05-25T23:22:26Z,2018-05-25T23:22:26Z,OWNER,"This should detect any table which can be linked to the current table via some other table, based on the other table having a foreign key to them both.
These join tables could be arbitrarily complicated. They might have foreign keys to more than two other tables, maybe even multiple foreign keys to the same column.
Ideally M2M defection would catch all of these cases. Maybe the resulting inspect data looks something like this:
```
""artists"": {
...
""m2m"": [{
""other_table"": ""festivals"",
""through"": ""performances"",
""our_fk"": ""artist_id"",
""other_fk"": ""performance_id""
}]
```
Let's ignore compound primary keys: we k it detect m2m relationships where the join table has foreign keys to a single primary key on the other two tables.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322787470,inspect() should detect many-to-many relationships,
https://github.com/simonw/datasette/issues/276#issuecomment-392279508,https://api.github.com/repos/simonw/datasette/issues/276,392279508,MDEyOklzc3VlQ29tbWVudDM5MjI3OTUwOA==,9599,simonw,2018-05-26T18:32:07Z,2018-05-26T18:32:07Z,OWNER,Related: I started the documentation for using SpatiaLite with Datasette here: https://datasette.readthedocs.io/en/latest/spatialite.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/276#issuecomment-392279644,https://api.github.com/repos/simonw/datasette/issues/276,392279644,MDEyOklzc3VlQ29tbWVudDM5MjI3OTY0NA==,9599,simonw,2018-05-26T18:34:21Z,2018-05-26T18:34:21Z,OWNER,"I've been thinking a bit about modifying the SQL select statement used for the table view recently. I've run into a few examples of SQLite database that slow to a crawl when viewed with datasette because the rows are too big, so there's definitely scope for supporting custom select clauses (avoiding some columns, showing length(colname) for others).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/289#issuecomment-392288990,https://api.github.com/repos/simonw/datasette/issues/289,392288990,MDEyOklzc3VlQ29tbWVudDM5MjI4ODk5MA==,9599,simonw,2018-05-26T21:24:10Z,2018-05-26T21:24:10Z,OWNER,An example of a query where you might want to use this option: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3?sql=select+rowid%2C+*+from+%5Balcohol-consumption%2Fdrinks%5D+order+by+random%28%29+limit+1,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326768188,?_ttl= parameter to control caching,
https://github.com/simonw/datasette/issues/289#issuecomment-392291605,https://api.github.com/repos/simonw/datasette/issues/289,392291605,MDEyOklzc3VlQ29tbWVudDM5MjI5MTYwNQ==,9599,simonw,2018-05-26T22:20:02Z,2018-05-26T22:20:02Z,OWNER,Documented here https://datasette.readthedocs.io/en/latest/json_api.html#special-table-arguments and here: https://datasette.readthedocs.io/en/latest/config.html#default-cache-ttl,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326768188,?_ttl= parameter to control caching,
https://github.com/simonw/datasette/issues/289#issuecomment-392291716,https://api.github.com/repos/simonw/datasette/issues/289,392291716,MDEyOklzc3VlQ29tbWVudDM5MjI5MTcxNg==,9599,simonw,2018-05-26T22:22:47Z,2018-05-26T22:22:47Z,OWNER,Demo: hit refresh on https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3?sql=select+rowid%2C+*+from+%5Balcohol-consumption%2Fdrinks%5D+order+by+random%28%29+limit+1&_ttl=0,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326768188,?_ttl= parameter to control caching,
https://github.com/simonw/datasette/issues/287#issuecomment-392296758,https://api.github.com/repos/simonw/datasette/issues/287,392296758,MDEyOklzc3VlQ29tbWVudDM5MjI5Njc1OA==,9599,simonw,2018-05-27T00:32:53Z,2018-05-27T00:32:53Z,OWNER,Docs: https://datasette.readthedocs.io/en/latest/json_api.html#different-shapes,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326617744,?_shape=arrayfirst,
https://github.com/simonw/datasette/issues/285#issuecomment-392297392,https://api.github.com/repos/simonw/datasette/issues/285,392297392,MDEyOklzc3VlQ29tbWVudDM5MjI5NzM5Mg==,9599,simonw,2018-05-27T00:50:27Z,2018-05-27T00:50:27Z,OWNER,"I ran a very rough micro-benchmark on the new `num_sql_threads` config option.
datasette --config num_sql_threads:1 fivethirtyeight.db
Then
ab -n 100 -c 10 'http://127.0.0.1:8011/fivethirtyeight-2628db9/twitter-ratio%2Fsenators'
| Number of threads | Requests/second |
|---|---|
| 1 | 4.57 |
| 3 | 9.77 |
| 10 | 13.53 |
| 20 | 15.24
| 50 | 8.21 |
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326189744,num_threads and cache_max_age should be --config options,
https://github.com/simonw/datasette/issues/285#issuecomment-392297508,https://api.github.com/repos/simonw/datasette/issues/285,392297508,MDEyOklzc3VlQ29tbWVudDM5MjI5NzUwOA==,9599,simonw,2018-05-27T00:53:35Z,2018-05-27T00:53:35Z,OWNER,Documentation: http://datasette.readthedocs.io/en/latest/config.html#num-sql-threads,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326189744,num_threads and cache_max_age should be --config options,
https://github.com/simonw/datasette/issues/291#issuecomment-392302406,https://api.github.com/repos/simonw/datasette/issues/291,392302406,MDEyOklzc3VlQ29tbWVudDM5MjMwMjQwNg==,9599,simonw,2018-05-27T03:18:06Z,2018-05-27T03:18:06Z,OWNER,"My first attempt at this was to have plugins depend on each other - so there would be a `datasette-leaflet` plugin which adds Leaflet to the page, and the `datasette-cluster-map` and `datasette-leaflet-geojson` plugins would depend on that plugin.
I tried this and it didn't work, because it turns out the order in which plugins are loaded isn't predictable. `datasette-cluster-map` ended up adding it's script link before Leaflet had been loaded by `datasette-leaflet`, resulting in JavaScript errors.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326783670,Avoid plugins accidentally loading dependencies twice,
https://github.com/simonw/datasette/issues/291#issuecomment-392302416,https://api.github.com/repos/simonw/datasette/issues/291,392302416,MDEyOklzc3VlQ29tbWVudDM5MjMwMjQxNg==,9599,simonw,2018-05-27T03:18:16Z,2018-05-27T03:18:16Z,OWNER,For the moment then I'm going with a really simple solution: when iterating through `extra_css_urls` and `extra_js_urls` de-dupe by URL and avoid outputting the same link twice.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326783670,Avoid plugins accidentally loading dependencies twice,
https://github.com/simonw/datasette/issues/291#issuecomment-392302456,https://api.github.com/repos/simonw/datasette/issues/291,392302456,MDEyOklzc3VlQ29tbWVudDM5MjMwMjQ1Ng==,9599,simonw,2018-05-27T03:19:24Z,2018-05-27T03:19:24Z,OWNER,The big gap in this solution is conflicting versions: I don't yet have a story for what happens if two plugins attempt to load different versions of Leaflet. ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326783670,Avoid plugins accidentally loading dependencies twice,
https://github.com/simonw/datasette/issues/231#issuecomment-392305776,https://api.github.com/repos/simonw/datasette/issues/231,392305776,MDEyOklzc3VlQ29tbWVudDM5MjMwNTc3Ng==,9599,simonw,2018-05-27T05:10:46Z,2018-05-27T05:10:46Z,OWNER,These plugin config options should be exposed to JavaScript as `datasette.config.plugins`,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316323336,metadata.json support for plugin configuration options,
https://github.com/simonw/datasette/issues/276#issuecomment-392316250,https://api.github.com/repos/simonw/datasette/issues/276,392316250,MDEyOklzc3VlQ29tbWVudDM5MjMxNjI1MA==,9599,simonw,2018-05-27T08:59:46Z,2018-05-27T08:59:46Z,OWNER,It looks like we can use the `geometry_columns` table to introspect which columns are SpatiaLite geometries. It includes a `geometry_type` integer which is documented here: https://www.gaia-gis.it/fossil/libspatialite/wiki?name=switching-to-4.0,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/276#issuecomment-392316306,https://api.github.com/repos/simonw/datasette/issues/276,392316306,MDEyOklzc3VlQ29tbWVudDM5MjMxNjMwNg==,9599,simonw,2018-05-27T09:00:46Z,2018-05-27T09:00:46Z,OWNER,Relevant to this ticket: I've been playing with a plugin that automatically renders any GeoJSON cells as leaflet maps: https://github.com/simonw/datasette-leaflet-geojson,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/292#issuecomment-392316673,https://api.github.com/repos/simonw/datasette/issues/292,392316673,MDEyOklzc3VlQ29tbWVudDM5MjMxNjY3Mw==,9599,simonw,2018-05-27T09:08:06Z,2018-05-27T09:08:06Z,OWNER,Open question: how should this affect the row page? Just because columns were hidden on the table page doesn't necessarily mean they should be hidden on the row page as well. ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/292#issuecomment-392316701,https://api.github.com/repos/simonw/datasette/issues/292,392316701,MDEyOklzc3VlQ29tbWVudDM5MjMxNjcwMQ==,9599,simonw,2018-05-27T09:08:49Z,2018-05-27T09:08:49Z,OWNER,I could certainly see people wanting different custom column selects for the row page compared to the table page.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/292#issuecomment-392338130,https://api.github.com/repos/simonw/datasette/issues/292,392338130,MDEyOklzc3VlQ29tbWVudDM5MjMzODEzMA==,9599,simonw,2018-05-27T15:09:18Z,2018-05-27T15:09:28Z,OWNER,"Here's my first sketch at a metadata format for this:
* `columns`: optional list of columns to include - if missing, shows all
* `column_selects`: dictionary mapping column names to alternative select clauses
`column_selects` can also invent new keys and use them to create derived columns. These new keys will be selected at the end of the list of columns UNLESS they are mentioned in `columns`, in which case that sequence will define the order.
Can you facet by things that are customized using `column_selects`? Yes, and let's try running suggested facets against those columns as well.
```
{
""databases"": {
""databasename"": {
""tables"": {
""tablename"": {
""columns"": [
""id"", ""name"", ""size""
],
""column_selects"": {
""name"": ""upper(name)"",
""geo_json"": ""AsGeoJSON(Geometry)""
}
""row_columns"": [...]
""row_column_selects"": {...}
}
```
The `row_columns` and `row_column_selects` properties work the same as the `column*` ones, except they are applied on the row page instead.
If omitted, the `column*` ones will be used on the row page as well.
If you want the row page to switch back to Datasette's default behaviour you can set `""row_columns"": [], ""row_column_selects"": {}`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/292#issuecomment-392342269,https://api.github.com/repos/simonw/datasette/issues/292,392342269,MDEyOklzc3VlQ29tbWVudDM5MjM0MjI2OQ==,9599,simonw,2018-05-27T15:55:40Z,2018-05-27T16:01:26Z,OWNER,"Here's the metadata I tried against that first working prototype:
```
{
""databases"": {
""timezones"": {
""tables"": {
""timezones"": {
""columns"": [""PK_UID""],
""column_selects"": {
""upper_tzid"": ""upper(tzid)"",
""Geometry"": ""AsGeoJSON(Geometry)""
}
}
}
},
""wtr"": {
""tables"": {
""license_frequency"": {
""columns"": [""id"", ""license"", ""tx_rx"", ""frequency""],
""column_selects"": {
""latitude"": ""Y(Geometry)"",
""longitude"": ""X(Geometry)""
}
}
}
}
}
}
```
Run using this:
datasette timezones.db wtr.db \
--reload --debug --load-extension=/usr/local/lib/mod_spatialite.dylib \
-m column-metadata.json --config sql_time_limit_ms:10000
Usefully, the `--reload` flag detects changes to the `metadata.json` file as well as Datasette's own Python code.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/292#issuecomment-392342947,https://api.github.com/repos/simonw/datasette/issues/292,392342947,MDEyOklzc3VlQ29tbWVudDM5MjM0Mjk0Nw==,9599,simonw,2018-05-27T16:01:43Z,2018-05-27T16:01:43Z,OWNER,I'd still like to be able to over-ride this using querystring arguments.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/292#issuecomment-392343690,https://api.github.com/repos/simonw/datasette/issues/292,392343690,MDEyOklzc3VlQ29tbWVudDM5MjM0MzY5MA==,9599,simonw,2018-05-27T16:08:25Z,2018-05-27T16:08:40Z,OWNER,"Turns out it's actually possible to pull data from other tables using the mechanism in the prototype:
```
{
""databases"": {
""wtr"": {
""tables"": {
""license"": {
""column_selects"": {
""count"": ""(select count(*) from license_frequency where license_frequency.license = license.id)""
}
}
}
}
}
}
```
Performance using this technique is pretty terrible though:
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/292#issuecomment-392345062,https://api.github.com/repos/simonw/datasette/issues/292,392345062,MDEyOklzc3VlQ29tbWVudDM5MjM0NTA2Mg==,9599,simonw,2018-05-27T16:26:53Z,2018-05-27T16:26:53Z,OWNER,There needs to be a way to turn this off and return to Datasette default bahviour. Maybe a `?_raw=1` querystring parameter for the table view.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/292#issuecomment-392350495,https://api.github.com/repos/simonw/datasette/issues/292,392350495,MDEyOklzc3VlQ29tbWVudDM5MjM1MDQ5NQ==,9599,simonw,2018-05-27T17:47:31Z,2018-05-27T17:47:31Z,OWNER,"Querystring design:
* `?_column=a&_column=b` - equivalent of `""columns"": [""a"", ""b""]` in `metadata.json`
* `?_select_nameupper=upper(name)` - equivalent of `""column_selects"": {""nameupper"": ""upper(name)""}`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/292#issuecomment-392350568,https://api.github.com/repos/simonw/datasette/issues/292,392350568,MDEyOklzc3VlQ29tbWVudDM5MjM1MDU2OA==,9599,simonw,2018-05-27T17:48:45Z,2018-05-27T17:54:41Z,OWNER,"If any `?_column=` parameters are provided the metadata version is completely ignored.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/292#issuecomment-392350980,https://api.github.com/repos/simonw/datasette/issues/292,392350980,MDEyOklzc3VlQ29tbWVudDM5MjM1MDk4MA==,9599,simonw,2018-05-27T17:56:30Z,2018-05-27T17:56:50Z,OWNER,"Should `?_raw=1` also turn off foreign key expansions? No, we will eventually provide a separate mechanism for that (or leave it to nerds who care to figure out using JSON or CSV export).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/150#issuecomment-392568047,https://api.github.com/repos/simonw/datasette/issues/150,392568047,MDEyOklzc3VlQ29tbWVudDM5MjU2ODA0Nw==,9599,simonw,2018-05-28T16:41:28Z,2018-05-28T16:41:28Z,OWNER,Closing this as obsolete since we have facets now.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",276704327,_group_count= feature improvements,
https://github.com/simonw/datasette/issues/116#issuecomment-392574208,https://api.github.com/repos/simonw/datasette/issues/116,392574208,MDEyOklzc3VlQ29tbWVudDM5MjU3NDIwOA==,9599,simonw,2018-05-28T17:23:41Z,2018-05-28T17:23:41Z,OWNER,"I'm handling this as separate documentation sections instead, e.g. http://datasette.readthedocs.io/en/latest/spatialite.html","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",274884209,Add documentation section about SQLite extensions,
https://github.com/simonw/datasette/issues/79#issuecomment-392574358,https://api.github.com/repos/simonw/datasette/issues/79,392574358,MDEyOklzc3VlQ29tbWVudDM5MjU3NDM1OA==,9599,simonw,2018-05-28T17:24:48Z,2018-05-28T17:24:48Z,OWNER,Closing this as obsolete in favor of other issues [tagged documentation](https://github.com/simonw/datasette/labels/documentation).,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",273569068,Add more detailed API documentation to the README,
https://github.com/simonw/datasette/issues/73#issuecomment-392574415,https://api.github.com/repos/simonw/datasette/issues/73,392574415,MDEyOklzc3VlQ29tbWVudDM5MjU3NDQxNQ==,9599,simonw,2018-05-28T17:25:14Z,2018-05-28T17:25:14Z,OWNER,I implemented this as `?_ttl=0` in #289 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",273296178,_nocache=1 query string option for use with sort-by-random,
https://github.com/simonw/datasette/issues/36#issuecomment-392575160,https://api.github.com/repos/simonw/datasette/issues/36,392575160,MDEyOklzc3VlQ29tbWVudDM5MjU3NTE2MA==,9599,simonw,2018-05-28T17:30:52Z,2018-05-28T17:30:52Z,OWNER,"I've changed my mind about this.
""Select every record on the 3rd day of the month"" doesn't strike me as an actually useful feature.
""Select every record in 2018 / in May 2018 / on 1st May 2018"", if you are using the SQLite-preferred datestring format, are already supported using LIKE queries (or the startswith filter):
* https://fivethirtyeight.datasettes.com/fivethirtyeight/inconvenient-sequel%2Fratings?timestamp__startswith=2017
* https://fivethirtyeight.datasettes.com/fivethirtyeight/inconvenient-sequel%2Fratings?timestamp__startswith=2017-08
* https://fivethirtyeight.datasettes.com/fivethirtyeight/inconvenient-sequel%2Fratings?timestamp__startswith=2017-08-29
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",268262480,"date, year, month and day querystring lookups",
https://github.com/simonw/datasette/issues/121#issuecomment-392575448,https://api.github.com/repos/simonw/datasette/issues/121,392575448,MDEyOklzc3VlQ29tbWVudDM5MjU3NTQ0OA==,9599,simonw,2018-05-28T17:33:07Z,2018-05-28T17:33:07Z,OWNER,"This shouldn't be a comma-separated list, it should be an argument you can pass multiple times to better match #255 and #292
Maybe `?_json=foo&_json=bar`
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275089535,?_json=foo&_json=bar query string argument ,
https://github.com/simonw/datasette/issues/31#issuecomment-392580715,https://api.github.com/repos/simonw/datasette/issues/31,392580715,MDEyOklzc3VlQ29tbWVudDM5MjU4MDcxNQ==,9599,simonw,2018-05-28T18:10:45Z,2018-05-28T18:10:45Z,OWNER,"Oops, that commit should have referenced #121 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",268087542,Idea: colour scheme based on sha256 of db,
https://github.com/simonw/datasette/issues/121#issuecomment-392580902,https://api.github.com/repos/simonw/datasette/issues/121,392580902,MDEyOklzc3VlQ29tbWVudDM5MjU4MDkwMg==,9599,simonw,2018-05-28T18:11:51Z,2018-05-28T18:11:51Z,OWNER,"Implemented in 76d11eb768e2f05f593c4d37a25280c0fcdf8fd6
Documented here: http://datasette.readthedocs.io/en/latest/json_api.html#special-json-arguments","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275089535,?_json=foo&_json=bar query string argument ,
https://github.com/simonw/datasette/issues/34#issuecomment-392600866,https://api.github.com/repos/simonw/datasette/issues/34,392600866,MDEyOklzc3VlQ29tbWVudDM5MjYwMDg2Ng==,9599,simonw,2018-05-28T20:45:34Z,2018-05-28T20:45:42Z,OWNER,"This is an accidental duplicate, work is now taking place in #266","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",268176505,Support CSV export with a .csv extension,
https://github.com/simonw/datasette/issues/38#issuecomment-392601114,https://api.github.com/repos/simonw/datasette/issues/38,392601114,MDEyOklzc3VlQ29tbWVudDM5MjYwMTExNA==,9599,simonw,2018-05-28T20:47:31Z,2018-05-28T20:47:31Z,OWNER,I think the way Datasette executes SQL queries in a thread pool introduced in #45 is a good solution for this ticket.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",268462768,Experiment with patterns for concurrent long running queries,
https://github.com/simonw/datasette/issues/56#issuecomment-392601478,https://api.github.com/repos/simonw/datasette/issues/56,392601478,MDEyOklzc3VlQ29tbWVudDM5MjYwMTQ3OA==,9599,simonw,2018-05-28T20:50:24Z,2018-05-28T20:50:24Z,OWNER,I'm going to close this as WONTFIX for the moment. Once Plugins #14 grows the ability to add extra URL paths and views someone who needs this could build it as a plugin instead.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",273127443,Easy way to block search engine crawling in robots.txt,
https://github.com/simonw/datasette/issues/97#issuecomment-392602334,https://api.github.com/repos/simonw/datasette/issues/97,392602334,MDEyOklzc3VlQ29tbWVudDM5MjYwMjMzNA==,9599,simonw,2018-05-28T20:57:21Z,2018-05-28T20:57:21Z,OWNER,"The `/.json` endpoint is more of an implementation detail of the homepage at this point. A better, documented ( http://datasette.readthedocs.io/en/stable/introspection.html#inspect ) endpoint for finding all of the databases and tables is https://parlgov.datasettes.com/-/inspect.json","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",274022950,Link to JSON for the list of tables ,
https://github.com/simonw/datasette/issues/142#issuecomment-392602558,https://api.github.com/repos/simonw/datasette/issues/142,392602558,MDEyOklzc3VlQ29tbWVudDM5MjYwMjU1OA==,9599,simonw,2018-05-28T20:58:59Z,2018-05-28T20:58:59Z,OWNER,I'll have the error message display a link to the documentation.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275917760,Show extra instructions with the interrupted,
https://github.com/simonw/datasette/issues/142#issuecomment-392605574,https://api.github.com/repos/simonw/datasette/issues/142,392605574,MDEyOklzc3VlQ29tbWVudDM5MjYwNTU3NA==,9599,simonw,2018-05-28T21:25:05Z,2018-05-28T21:25:05Z,OWNER,"
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275917760,Show extra instructions with the interrupted,
https://github.com/simonw/datasette/issues/144#issuecomment-392606044,https://api.github.com/repos/simonw/datasette/issues/144,392606044,MDEyOklzc3VlQ29tbWVudDM5MjYwNjA0NA==,9599,simonw,2018-05-28T21:29:42Z,2018-05-28T21:29:42Z,OWNER,"The other major limitation of APSW is its treatment of unicode: https://rogerbinns.github.io/apsw/types.html - it tells you that it is your responsibility to ensure that TEXT columns in your SQLite database are correctly encoded.
Since Datasette is designed to work against ANY SQLite database that someone may have already created, I see that as a show-stopping limitation.
Thanks to https://github.com/coleifer/sqlite-vtfunc I now have a working mechanism for virtual tables (I've even built a demo plugin with them - https://github.com/simonw/datasette-sql-scraper ) which was the main thing that interested me about APSW.
I'm going to close this as WONTFIX - I think Python's built-in `sqlite3` is good enough, and is now so firmly embedded in the project that making it pluggable would be more trouble than it's worth.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",276091279,apsw as alternative sqlite3 binding (for full text search),
https://github.com/simonw/datasette/issues/179#issuecomment-392606418,https://api.github.com/repos/simonw/datasette/issues/179,392606418,MDEyOklzc3VlQ29tbWVudDM5MjYwNjQxOA==,9599,simonw,2018-05-28T21:32:37Z,2018-05-28T21:32:37Z,OWNER,"> It could also be useful to allow users to import a python file containing custom functions that can that be loaded into scope and made available to custom templates.
That's now covered by the plugins mechanism - you can create plugins that define custom template functions: http://datasette.readthedocs.io/en/stable/plugins.html#prepare-jinja2-environment-env","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",288438570,More metadata options for template authors ,
https://github.com/simonw/datasette/issues/188#issuecomment-376594727,https://api.github.com/repos/simonw/datasette/issues/188,376594727,MDEyOklzc3VlQ29tbWVudDM3NjU5NDcyNw==,9599,simonw,2018-03-27T16:46:49Z,2018-05-28T21:34:34Z,OWNER,"One point of complexity: datasette can be used to bundle multiple .db files into a single ""app"".
I think that's OK. We could require that the `datasette_files` table is present in the first database file passed on the command-line. Or we could even construct a search path and consult multiple versions of the table spread across multiple files.
That said... any configuration that corresponds to a specific table should live in the same database file as that table.
Ditto for general metadata: if we have license/source information for a specific table or database that information should be able to live in the same .db file as the data.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",309047460,Ability to bundle metadata and templates inside the SQLite file,
https://github.com/simonw/datasette/issues/276#issuecomment-392815673,https://api.github.com/repos/simonw/datasette/issues/276,392815673,MDEyOklzc3VlQ29tbWVudDM5MjgxNTY3Mw==,9599,simonw,2018-05-29T15:17:04Z,2018-05-29T15:17:04Z,OWNER,"I'm coming round to the idea that this should be baked into Datasette core - see above referenced issues for some of the explorations I've been doing around this area.
Datasette should absolutely work without SpatiaLite, but it's such a huge bonus part of the SQLite ecosystem that I'm happy to ship features that take advantage of it without being relegated to plugins.
I'm also becoming aware that there aren't really that many other interesting loadable extensions for SQLite. If SpatiaLite was one of dozens I'd feel that a rule that ""anything dependent on an extension lives in a plugin"" would make sense, but as it stands I think 99% of the time the only loadable extensions people will be using will be SpatiaLite and json1 (and json1 is available in the amalgamation anyway).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/191#issuecomment-392822050,https://api.github.com/repos/simonw/datasette/issues/191,392822050,MDEyOklzc3VlQ29tbWVudDM5MjgyMjA1MA==,9599,simonw,2018-05-29T15:33:25Z,2018-05-29T15:33:25Z,OWNER,"I don't know how it happened, but I've somehow got myself into a state where my local SQLite for Python 3 on OS X is `3.23.1`:
```
~ $ python3
Python 3.6.5 (default, Mar 30 2018, 06:41:53)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type ""help"", ""copyright"", ""credits"" or ""license"" for more information.
>>> import sqlite3
>>> sqlite3.connect(':memory:').execute('select sqlite_version()').fetchall()
[('3.23.1',)]
>>>
```
Maybe I did something in homebrew that changed this? I'd love to understand what exactly I did to get to this state.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",310533258,Figure out how to bundle a more up-to-date SQLite,
https://github.com/simonw/datasette/issues/276#issuecomment-392825746,https://api.github.com/repos/simonw/datasette/issues/276,392825746,MDEyOklzc3VlQ29tbWVudDM5MjgyNTc0Ng==,45057,russss,2018-05-29T15:42:53Z,2018-05-29T15:42:53Z,CONTRIBUTOR,"I haven't had time to look further into this, but if doing this as a plugin results in useful hooks then I think we should do it that way. We could always require the plugin as a standard dependency.
I think this is going to result in quite a bit of refactoring anyway so it's a good time to add hooks regardless.
On the other hand, if we have to add lots of specialist hooks for it then maybe it's worth integrating into the core.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/191#issuecomment-392828475,https://api.github.com/repos/simonw/datasette/issues/191,392828475,MDEyOklzc3VlQ29tbWVudDM5MjgyODQ3NQ==,119974,coleifer,2018-05-29T15:50:18Z,2018-05-29T15:50:18Z,NONE,"Python standard-library SQLite dynamically links against the system sqlite3. So presumably you installed a more up-to-date sqlite3 somewhere on your `LD_LIBRARY_PATH`.
To compile a statically-linked pysqlite you need to include an amalgamation in the project root when building the extension. Read the relevant setup.py.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",310533258,Figure out how to bundle a more up-to-date SQLite,
https://github.com/simonw/datasette/issues/191#issuecomment-392831543,https://api.github.com/repos/simonw/datasette/issues/191,392831543,MDEyOklzc3VlQ29tbWVudDM5MjgzMTU0Mw==,9599,simonw,2018-05-29T15:58:33Z,2018-05-29T15:58:33Z,OWNER,"I ran an informal survey on twitter and most people were on 3.21 - https://twitter.com/simonw/status/1001487546289815553
Maybe this is from upgrading to the latest OS X release.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",310533258,Figure out how to bundle a more up-to-date SQLite,
https://github.com/simonw/datasette/issues/265#issuecomment-392890045,https://api.github.com/repos/simonw/datasette/issues/265,392890045,MDEyOklzc3VlQ29tbWVudDM5Mjg5MDA0NQ==,231923,yschimke,2018-05-29T18:37:49Z,2018-05-29T18:37:49Z,NONE,"Just about to ask for this! Move this page https://github.com/simonw/datasette/wiki/Datasettes
into a datasette, with some concept of versioning as well.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323677499,Add links to example Datasette instances to appropiate places in docs,
https://github.com/simonw/datasette/issues/97#issuecomment-392895733,https://api.github.com/repos/simonw/datasette/issues/97,392895733,MDEyOklzc3VlQ29tbWVudDM5Mjg5NTczMw==,231923,yschimke,2018-05-29T18:51:35Z,2018-05-29T18:51:35Z,NONE,Do you have an existing example with views?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",274022950,Link to JSON for the list of tables ,
https://github.com/simonw/datasette/issues/298#issuecomment-392917380,https://api.github.com/repos/simonw/datasette/issues/298,392917380,MDEyOklzc3VlQ29tbWVudDM5MjkxNzM4MA==,9599,simonw,2018-05-29T19:41:59Z,2018-05-29T19:41:59Z,OWNER,Creating URLs using concatenation as seen in `('https://twitter.com/' || user) as user_url` is likely to have all sorts of useful applications for ad-hoc analysis.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327459829,URLify URLs in results from custom SQL statements / views,
https://github.com/simonw/datasette/issues/296#issuecomment-392840811,https://api.github.com/repos/simonw/datasette/issues/296,392840811,MDEyOklzc3VlQ29tbWVudDM5Mjg0MDgxMQ==,9599,simonw,2018-05-29T16:26:27Z,2018-05-29T19:43:23Z,OWNER,"Since #275 will allow configs to be overridden at the table and database level it also makes sense to expose a completely evaluated list of configs at:
* `/dbname/-/config`
* `/dbname/tablename/-/config`
Similar to https://fivethirtyeight.datasettes.com/-/config","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327395270,Per-database and per-table /-/ URL namespace,
https://github.com/simonw/datasette/issues/296#issuecomment-392918311,https://api.github.com/repos/simonw/datasette/issues/296,392918311,MDEyOklzc3VlQ29tbWVudDM5MjkxODMxMQ==,9599,simonw,2018-05-29T19:44:33Z,2018-05-29T19:44:33Z,OWNER,Should the `tablename` ones also work for views and canned queries? Probably not.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327395270,Per-database and per-table /-/ URL namespace,
https://github.com/simonw/datasette/issues/276#issuecomment-392969173,https://api.github.com/repos/simonw/datasette/issues/276,392969173,MDEyOklzc3VlQ29tbWVudDM5Mjk2OTE3Mw==,9599,simonw,2018-05-29T22:32:08Z,2018-05-29T22:32:08Z,OWNER,The more time I spend with SpatiaLite the more convinced I am that this should be default behavior. There's nothing useful about the binary Geometry representation - it's not even valid WKB. I'm on board with WKT as the default display in HTML and GeoJSON as the default for `.json`,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/265#issuecomment-393003340,https://api.github.com/repos/simonw/datasette/issues/265,393003340,MDEyOklzc3VlQ29tbWVudDM5MzAwMzM0MA==,9599,simonw,2018-05-30T01:44:22Z,2018-05-30T01:44:22Z,OWNER,Funny you should mention that... I'm planning on doing that as part of the official Datasette website at some point soon. A Datasette instance that lists other Datasette instances feels pleasingly appropriate.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323677499,Add links to example Datasette instances to appropiate places in docs,
https://github.com/simonw/datasette/issues/276#issuecomment-393014943,https://api.github.com/repos/simonw/datasette/issues/276,393014943,MDEyOklzc3VlQ29tbWVudDM5MzAxNDk0Mw==,9599,simonw,2018-05-30T02:59:53Z,2018-05-30T02:59:53Z,OWNER,I just realised a problem with GeoJSON is that it assumes that the underlying geometry is WGS 84 latitude/longitude points - but it's very possible for a SpatiaLite geometry to contain geometric data that's nothing to do with geospatial projections.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/266#issuecomment-393020749,https://api.github.com/repos/simonw/datasette/issues/266,393020749,MDEyOklzc3VlQ29tbWVudDM5MzAyMDc0OQ==,9599,simonw,2018-05-30T03:42:54Z,2018-05-30T03:42:54Z,OWNER,"Challenge: how to deal with tables where the name ends in `.csv`? I actually have one of these in the test suite at the moment:
https://github.com/simonw/datasette/blob/d69ebce53385b7c6fafb85fdab3b136dbf3f332c/tests/fixtures.py#L234-L237","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/265#issuecomment-393064224,https://api.github.com/repos/simonw/datasette/issues/265,393064224,MDEyOklzc3VlQ29tbWVudDM5MzA2NDIyNA==,9599,simonw,2018-05-30T07:48:37Z,2018-05-30T07:48:37Z,OWNER,"https://datasette-registry.now.sh Is now live, powered by https://github.com/simonw/datasette-registry - still needs plenty of work but it's an interesting start.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323677499,Add links to example Datasette instances to appropiate places in docs,
https://github.com/simonw/datasette/issues/276#issuecomment-393106520,https://api.github.com/repos/simonw/datasette/issues/276,393106520,MDEyOklzc3VlQ29tbWVudDM5MzEwNjUyMA==,45057,russss,2018-05-30T10:09:25Z,2018-05-30T10:09:25Z,CONTRIBUTOR,"I don't think it's unreasonable to only support spatialite geometries in a coordinate reference system which is at least transformable to WGS84. It would be nice to support different CRSes in the database so conversion to spatialite from the source data is lossless.
I think the working CRS for datasette should be WGS84 though (leaflet requires it, for example) - it's just a case of calling `ST_Transform(geom, 4326)` on the column while we're loading the data.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/295#issuecomment-393534579,https://api.github.com/repos/simonw/datasette/issues/295,393534579,MDEyOklzc3VlQ29tbWVudDM5MzUzNDU3OQ==,9599,simonw,2018-05-31T13:44:15Z,2018-05-31T13:44:15Z,OWNER,I actually started doing this in 45e502aace6cc1198cc5f9a04d61b4a1860a012b,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327383759,Extract unit tests for inspect out to test_inspect.py,
https://github.com/simonw/datasette/issues/243#issuecomment-393544357,https://api.github.com/repos/simonw/datasette/issues/243,393544357,MDEyOklzc3VlQ29tbWVudDM5MzU0NDM1Nw==,9599,simonw,2018-05-31T14:14:49Z,2018-05-31T14:14:49Z,OWNER,"Demo: https://datasette-publish-spatialite-demo.now.sh/spatialite-test-c88bc35?sql=select+AsText(Geometry)+from+HighWays+limit+1%3B
Published using `datasette publish now --spatialite /tmp/spatialite-test.db`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",318737808,--spatialite option for datasette publish commands,
https://github.com/simonw/datasette/issues/294#issuecomment-393547960,https://api.github.com/repos/simonw/datasette/issues/294,393547960,MDEyOklzc3VlQ29tbWVudDM5MzU0Nzk2MA==,9599,simonw,2018-05-31T14:25:43Z,2018-05-31T14:25:43Z,OWNER,"SpatialLite columns are actually quite a bit more interesting than this - they also have a `geometry_type` (point, polygon, linestring etc), a `coord_dimension` (usually 2 but can be higher) and an `srid`.
For example:
https://datasette-publish-spatialite-demo.now.sh/spatialite-test-c88bc35/geometry_columns
The SRID here is particularly interesting, because it helps hint at the fact that the results from these queries won't be latitude/longitude co-ordinates - which means that `AsGeoJSON()` won't return results that can be easily rendered by Leaflet:
https://datasette-publish-spatialite-demo.now.sh/spatialite-test-c88bc35?sql=select+AsGeoJSON(Geometry)+from+HighWays%20limit1
Compare with https://timezones-api.now.sh/timezones-a99b2e3/geometry_columns:
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327365110,inspect should record column types,
https://github.com/simonw/datasette/issues/294#issuecomment-393548602,https://api.github.com/repos/simonw/datasette/issues/294,393548602,MDEyOklzc3VlQ29tbWVudDM5MzU0ODYwMg==,9599,simonw,2018-05-31T14:27:41Z,2018-05-31T14:27:56Z,OWNER,Presumably the difference in primary key structure between those two is caused by the fact that the `spatialite-test` database (actually https://www.gaia-gis.it/spatialite-2.3.1/test-2.3.sqlite.gz downloaded from https://www.gaia-gis.it/spatialite-2.3.1/resources.html ) was created by a much older version of SpatialLite - presumably v2.3.1,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327365110,inspect should record column types,
https://github.com/simonw/datasette/issues/294#issuecomment-393549215,https://api.github.com/repos/simonw/datasette/issues/294,393549215,MDEyOklzc3VlQ29tbWVudDM5MzU0OTIxNQ==,9599,simonw,2018-05-31T14:29:37Z,2018-05-31T14:29:37Z,OWNER,"Also of note: `spatialite-test` uses readable strings in the `type` column, while `timezones` has a `geometry_type` column with integers in it.
Those integers are documented here: https://www.gaia-gis.it/fossil/libspatialite/wiki?name=switching-to-4.0
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327365110,inspect should record column types,
https://github.com/simonw/datasette/issues/297#issuecomment-393554151,https://api.github.com/repos/simonw/datasette/issues/297,393554151,MDEyOklzc3VlQ29tbWVudDM5MzU1NDE1MQ==,9599,simonw,2018-05-31T14:44:37Z,2018-05-31T14:44:37Z,OWNER,I fixed this in https://github.com/simonw/datasette/commit/b18e4515855c3f1eeca3dfcccdbb6df05869084a,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327420945,datasette publish Dockerfile should use python:3.6-slim-stretch,
https://github.com/simonw/datasette/issues/303#issuecomment-393557406,https://api.github.com/repos/simonw/datasette/issues/303,393557406,MDEyOklzc3VlQ29tbWVudDM5MzU1NzQwNg==,9599,simonw,2018-05-31T14:54:03Z,2018-05-31T14:54:03Z,OWNER,"Our test fixtures currently have a table with a name ending in `.csv`:
https://github.com/simonw/datasette/blob/d69ebce53385b7c6fafb85fdab3b136dbf3f332c/tests/fixtures.py#L234-L237","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",328172521,Support table names ending with .json or .csv,
https://github.com/simonw/datasette/issues/294#issuecomment-393557968,https://api.github.com/repos/simonw/datasette/issues/294,393557968,MDEyOklzc3VlQ29tbWVudDM5MzU1Nzk2OA==,9599,simonw,2018-05-31T14:55:46Z,2018-05-31T14:55:46Z,OWNER,"I'm not sure what the best JSON shape for this would be considering the potential complexity of geospatial columns. I do think it's worth exposing these in the inspect JSON though, mainly so Datasette Registry can keep track of all of the openly available geodata out there.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327365110,inspect should record column types,
https://github.com/simonw/datasette/issues/303#issuecomment-393599840,https://api.github.com/repos/simonw/datasette/issues/303,393599840,MDEyOklzc3VlQ29tbWVudDM5MzU5OTg0MA==,9599,simonw,2018-05-31T16:54:22Z,2018-05-31T16:54:32Z,OWNER,The interesting thing about this is that it requires URL routing to become aware of the names of all of the available tables.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",328172521,Support table names ending with .json or .csv,
https://github.com/simonw/datasette/issues/303#issuecomment-393600441,https://api.github.com/repos/simonw/datasette/issues/303,393600441,MDEyOklzc3VlQ29tbWVudDM5MzYwMDQ0MQ==,9599,simonw,2018-05-31T16:56:25Z,2018-05-31T16:57:41Z,OWNER,"Here's a nasty challenge: what happens if a database has the following two tables:
* `blah`
* `blah.json`
What would the URL be for the JSON endpoint for the `blah` table?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",328172521,Support table names ending with .json or .csv,
https://github.com/simonw/datasette/issues/304#issuecomment-393610731,https://api.github.com/repos/simonw/datasette/issues/304,393610731,MDEyOklzc3VlQ29tbWVudDM5MzYxMDczMQ==,9599,simonw,2018-05-31T17:29:31Z,2018-05-31T17:30:05Z,OWNER,I prototyped this a while ago here https://github.com/simonw/datasette/commit/04476ead53758044a5f272ae8696b63d6703115e before we had the ``--config`` mechanism.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",328229224,Ability to configure SQLite cache_size,
https://github.com/simonw/datasette/issues/303#issuecomment-394037368,https://api.github.com/repos/simonw/datasette/issues/303,394037368,MDEyOklzc3VlQ29tbWVudDM5NDAzNzM2OA==,9599,simonw,2018-06-01T23:50:17Z,2018-06-01T23:50:35Z,OWNER,"Solution for he above: support an optional `?_format=json/csv` parameter on the regular table view.
Then if you have tables with the above colliding names you can use `/db/blah.json?_format=json` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",328172521,Support table names ending with .json or .csv,
https://github.com/simonw/datasette/issues/304#issuecomment-394400419,https://api.github.com/repos/simonw/datasette/issues/304,394400419,MDEyOklzc3VlQ29tbWVudDM5NDQwMDQxOQ==,9599,simonw,2018-06-04T15:39:03Z,2018-06-04T15:39:03Z,OWNER,"In the interest of getting this shipped, I'm going to ignore the `3.7.10` issue.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",328229224,Ability to configure SQLite cache_size,
https://github.com/simonw/datasette/issues/304#issuecomment-394412217,https://api.github.com/repos/simonw/datasette/issues/304,394412217,MDEyOklzc3VlQ29tbWVudDM5NDQxMjIxNw==,9599,simonw,2018-06-04T16:13:32Z,2018-06-04T16:13:32Z,OWNER,Docs: http://datasette.readthedocs.io/en/latest/config.html#cache-size-kb,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",328229224,Ability to configure SQLite cache_size,
https://github.com/simonw/datasette/issues/302#issuecomment-394412784,https://api.github.com/repos/simonw/datasette/issues/302,394412784,MDEyOklzc3VlQ29tbWVudDM5NDQxMjc4NA==,9599,simonw,2018-06-04T16:15:22Z,2018-06-04T16:15:22Z,OWNER,I think this is related to #303,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",328171513,test-2.3.sqlite database filename throws a 404,
https://github.com/simonw/datasette/issues/266#issuecomment-394417567,https://api.github.com/repos/simonw/datasette/issues/266,394417567,MDEyOklzc3VlQ29tbWVudDM5NDQxNzU2Nw==,9599,simonw,2018-06-04T16:30:48Z,2018-06-04T16:32:55Z,OWNER,"When serving streaming responses, I need to check that a large CSV file doesn't completely max out the CPU in a way that is harmful to the rest of the instance.
If it does, one option may be to insert an async sleep call in between each chunk that is streamed back. This could be controlled by a `csv_pause_ms` config setting, defaulting to maybe 5 but can be disabled entirely by setting to 0.
That's only if testing proves that this is a necessary mechanism.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/272#issuecomment-394431323,https://api.github.com/repos/simonw/datasette/issues/272,394431323,MDEyOklzc3VlQ29tbWVudDM5NDQzMTMyMw==,9599,simonw,2018-06-04T17:17:37Z,2018-06-04T17:17:37Z,OWNER,I built this ASGI debugging tool to help with this migration: https://asgi-scope.now.sh/fivethirtyeight-34d6604/most-common-name%2Fsurnames.json?foo=bar&bazoeuto=onetuh&a=.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/292#issuecomment-392343839,https://api.github.com/repos/simonw/datasette/issues/292,392343839,MDEyOklzc3VlQ29tbWVudDM5MjM0MzgzOQ==,9599,simonw,2018-05-27T16:10:09Z,2018-06-04T17:38:04Z,OWNER,"The more efficient way of doing this kind of count would be to provide a mechanism which can also add extra fragments to a `GROUP BY` clause used for the `SELECT`.
Or... how about a mechanism similar to Django's `prefetch_related` which lets you define extra queries that will be called with a list of primary keys (or values from other columns) and used to populate a new column? A little unconventional but could be extremely useful and efficient.
Related to that: since the per-query overhead in SQLite is tiny, could even define an extra query to be run once-per-row before returning results.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/272#issuecomment-394503399,https://api.github.com/repos/simonw/datasette/issues/272,394503399,MDEyOklzc3VlQ29tbWVudDM5NDUwMzM5OQ==,9599,simonw,2018-06-04T21:20:14Z,2018-06-04T21:20:14Z,OWNER,Results of an extremely simple micro-benchmark comparing the two shows that uvicorn is at least as fast as Sanic (benchmarks a little faster with a very simple payload): https://gist.github.com/simonw/418950af178c01c416363cc057420851,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/272#issuecomment-394764713,https://api.github.com/repos/simonw/datasette/issues/272,394764713,MDEyOklzc3VlQ29tbWVudDM5NDc2NDcxMw==,9599,simonw,2018-06-05T15:58:54Z,2018-06-05T16:00:40Z,OWNER,"https://github.com/encode/uvicorn/blob/572b5fe6c811b63298d5350a06b664839624c860/uvicorn/run.py#L63 is how you start a Uvicorn server from code as opposed to the `uvicorn` CLI
from uvicorn.run import UvicornServer
UvicornServer().run(app, host=host, port=port)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/272#issuecomment-392118755,https://api.github.com/repos/simonw/datasette/issues/272,392118755,MDEyOklzc3VlQ29tbWVudDM5MjExODc1NQ==,9599,simonw,2018-05-25T16:56:40Z,2018-06-05T16:01:13Z,OWNER,"Thinking about this further, maybe I should embrace ASGI turtles-all-the-way-down and teach each datasette view class to take a scope to the constructor and act entirely as an ASGI component. Would be a nice way of diving deep into ASGI and I can add utility helpers for things like querystring evaluation as I need them.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/306#issuecomment-394894500,https://api.github.com/repos/simonw/datasette/issues/306,394894500,MDEyOklzc3VlQ29tbWVudDM5NDg5NDUwMA==,9599,simonw,2018-06-05T23:40:40Z,2018-06-05T23:40:40Z,OWNER,"Input:
- function that says if a name is a valid database
- Function that says if a table exists
- URL
Output:
- view class
- Arguments
- Redirect (if it should redirect)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",329661905,Custom URL routing with independent tests,
https://github.com/simonw/datasette/issues/306#issuecomment-394895267,https://api.github.com/repos/simonw/datasette/issues/306,394895267,MDEyOklzc3VlQ29tbWVudDM5NDg5NTI2Nw==,9599,simonw,2018-06-05T23:45:26Z,2018-06-05T23:45:26Z,OWNER,To support a future where Datasette is an ASGI app that can be attached to a URL within a larger application the routing function should have the option to accept a path prefix which will then be automatically attached to any resulting redirects.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",329661905,Custom URL routing with independent tests,
https://github.com/simonw/datasette/issues/306#issuecomment-394894910,https://api.github.com/repos/simonw/datasette/issues/306,394894910,MDEyOklzc3VlQ29tbWVudDM5NDg5NDkxMA==,9599,simonw,2018-06-05T23:43:18Z,2018-06-05T23:49:41Z,OWNER,I'm going to use a named tuple for the output. That way I can support either tuple destructing or explicit property access on the returned value.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",329661905,Custom URL routing with independent tests,
https://github.com/simonw/datasette/issues/306#issuecomment-394895750,https://api.github.com/repos/simonw/datasette/issues/306,394895750,MDEyOklzc3VlQ29tbWVudDM5NDg5NTc1MA==,9599,simonw,2018-06-05T23:48:06Z,2018-06-06T23:50:31Z,OWNER,"A neat trick could be that if the router returns a redirect it could then resolve that redirect to see if it will 404 (or redirect itself) before returning that response.
This would need its own counter to guard against infinite redirects.
I'm not going to do this though: any view that results in a chain of redirects like this is a bug that should be fixed at the source.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",329661905,Custom URL routing with independent tests,
https://github.com/simonw/datasette/issues/306#issuecomment-395463497,https://api.github.com/repos/simonw/datasette/issues/306,395463497,MDEyOklzc3VlQ29tbWVudDM5NTQ2MzQ5Nw==,9599,simonw,2018-06-07T15:29:28Z,2018-06-07T15:29:28Z,OWNER,"I started sketching this out in a branch, see pull request #307 - but I've decided I don't like it. I'm going to close this ticket and stick with regular expression URL routing for the moment. If I change my mind in the future the code in #307 lives in separate files (`datasette/routes.py` and `tests/test_routes.py`) so bringing it back into the project will be trivial.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",329661905,Custom URL routing with independent tests,
https://github.com/simonw/datasette/pull/307#issuecomment-395463598,https://api.github.com/repos/simonw/datasette/issues/307,395463598,MDEyOklzc3VlQ29tbWVudDM5NTQ2MzU5OA==,9599,simonw,2018-06-07T15:29:41Z,2018-06-07T15:29:41Z,OWNER,Closing this pull request for reasons outlined here: https://github.com/simonw/datasette/issues/306#issuecomment-395463497,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",330323860,"Initial sketch of custom URL routing, refs #306",
https://github.com/simonw/datasette/issues/305#issuecomment-396048471,https://api.github.com/repos/simonw/datasette/issues/305,396048471,MDEyOklzc3VlQ29tbWVudDM5NjA0ODQ3MQ==,9599,simonw,2018-06-10T13:16:13Z,2018-06-10T13:16:13Z,OWNER,https://github.com/kubernetes/community/blob/master/contributors/devel/help-wanted.md Is worth stealing from too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",329147284,Add contributor guidelines to docs,
https://github.com/simonw/datasette/issues/266#issuecomment-397534196,https://api.github.com/repos/simonw/datasette/issues/266,397534196,MDEyOklzc3VlQ29tbWVudDM5NzUzNDE5Ng==,9599,simonw,2018-06-15T07:12:16Z,2018-06-15T07:12:16Z,OWNER,"The first version of this is now shipped to master. I ended up rewriting most of the experimental branch to deal with the nasty issue described in #303
Demo is available on https://fivethirtyeight.datasettes.com/fivethirtyeight-ab24e01/most-common-name%2Fsurnames
Here's the CSV version of that page: https://fivethirtyeight.datasettes.com/fivethirtyeight-ab24e01/most-common-name%2Fsurnames.csv","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397534404,https://api.github.com/repos/simonw/datasette/issues/266,397534404,MDEyOklzc3VlQ29tbWVudDM5NzUzNDQwNA==,9599,simonw,2018-06-15T07:13:20Z,2018-06-15T07:13:20Z,OWNER,"Still to add: the streaming version that iterates through all of the pages, as seen in experimental commit https://github.com/simonw/datasette/commit/ced379ea325787b8c3bf0a614daba1fa4856a3bd","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397534498,https://api.github.com/repos/simonw/datasette/issues/266,397534498,MDEyOklzc3VlQ29tbWVudDM5NzUzNDQ5OA==,9599,simonw,2018-06-15T07:13:52Z,2018-06-15T07:13:52Z,OWNER,Also needs documentation.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-389570841,https://api.github.com/repos/simonw/datasette/issues/266,389570841,MDEyOklzc3VlQ29tbWVudDM4OTU3MDg0MQ==,9599,simonw,2018-05-16T15:54:49Z,2018-06-15T07:41:09Z,OWNER,"At the most basic level, this will work based on an extension. Most places you currently put a `.json` extension should also allow a `.csv` extension.
By default this will return the exact results you see on the current page (default max will remain 1000).
## Streaming all records
Where things get interested is *streaming mode*. This will be an option which returns ALL matching records as a streaming CSV file, even if that ends up being millions of records.
I think the best way to build this will be on top of the existing mechanism used to efficiently implement keyset pagination via `_next=` tokens.
## Expanding foreign keys
For tables with foreign key references it would be useful if the CSV format could expand those references to include the labels from `label_column` - maybe via an additional `?_expand=1` option.
When expanding each foreign key column will be shown twice:
rowid,city_id,city_id_label,state","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/233#issuecomment-397637302,https://api.github.com/repos/simonw/datasette/issues/233,397637302,MDEyOklzc3VlQ29tbWVudDM5NzYzNzMwMg==,9599,simonw,2018-06-15T14:24:08Z,2018-06-15T14:55:19Z,OWNER,"I'm going with the terminology ""labels"" here. You'll be able to add ``?_labels=1`` and the JSON will look something like this:
```
{
""rowid"": 233,
""TreeID"": 121240,
""qLegalStatus"": {
""value"" 2,
""label"": ""Private""
}
""qSpecies"": {
""value"": 16,
""label"": ""Sycamore""
}
""qAddress"": ""91 Commonwealth Ave"",
...
}
```
I need this to help build foreign key expansions for CSV files, see #266 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316444720,Option to expose expanded foreign keys in JSON/CSV,
https://github.com/simonw/datasette/issues/233#issuecomment-397648080,https://api.github.com/repos/simonw/datasette/issues/233,397648080,MDEyOklzc3VlQ29tbWVudDM5NzY0ODA4MA==,9599,simonw,2018-06-15T14:56:21Z,2018-06-15T14:56:21Z,OWNER,"I considered including a `""table""` key like this:
```
""qLegalStatus"": {
""value"" 2,
""label"": ""Private"",
""table"": ""qLegalStatus""
}
```
This would help generate the HTML links using just the JSON data. But... I realized that in a list of 50 rows that value would be duplicated 50 times which is a bit nasty.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316444720,Option to expose expanded foreign keys in JSON/CSV,
https://github.com/simonw/datasette/issues/233#issuecomment-397663968,https://api.github.com/repos/simonw/datasette/issues/233,397663968,MDEyOklzc3VlQ29tbWVudDM5NzY2Mzk2OA==,9599,simonw,2018-06-15T15:51:17Z,2018-06-15T15:51:17Z,OWNER,"Nearly done, but I need the HTML view to ignore the `?_labels=1` param (it throws an error at the moment).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316444720,Option to expose expanded foreign keys in JSON/CSV,
https://github.com/simonw/datasette/issues/233#issuecomment-397668427,https://api.github.com/repos/simonw/datasette/issues/233,397668427,MDEyOklzc3VlQ29tbWVudDM5NzY2ODQyNw==,9599,simonw,2018-06-15T16:07:43Z,2018-06-15T16:07:43Z,OWNER,Demo: https://datasette-json-labels-demo.now.sh/fixtures-fda0fea/facetable.json?_labels=1&_shape=array,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316444720,Option to expose expanded foreign keys in JSON/CSV,
https://github.com/simonw/datasette/issues/233#issuecomment-397729319,https://api.github.com/repos/simonw/datasette/issues/233,397729319,MDEyOklzc3VlQ29tbWVudDM5NzcyOTMxOQ==,9599,simonw,2018-06-15T20:10:24Z,2018-06-15T20:10:24Z,OWNER,I'm also going to add the ability to specify individual columns that you want to expand using `?_label=city_id&_label=state_id`,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316444720,Option to expose expanded foreign keys in JSON/CSV,
https://github.com/simonw/datasette/issues/233#issuecomment-397729500,https://api.github.com/repos/simonw/datasette/issues/233,397729500,MDEyOklzc3VlQ29tbWVudDM5NzcyOTUwMA==,9599,simonw,2018-06-15T20:11:14Z,2018-06-15T20:11:14Z,OWNER,The `.json` and `.csv` links displayed on the table page should default to using `?_labels=1` if Datasette detects that there are foreign key expansions available for the page.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316444720,Option to expose expanded foreign keys in JSON/CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397729945,https://api.github.com/repos/simonw/datasette/issues/266,397729945,MDEyOklzc3VlQ29tbWVudDM5NzcyOTk0NQ==,9599,simonw,2018-06-15T20:13:05Z,2018-06-15T20:13:05Z,OWNER,"The ""This data as ..."" area of the page is getting a bit untidy, especially if I'm going to add other download options in the future. I think I'll move the HTML to the page footer (less concerns about taking up lots of space there) and then have a bit of JavaScript that turns it into a show/hide menu of some sort in its current location.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/pull/311#issuecomment-397823913,https://api.github.com/repos/simonw/datasette/issues/311,397823913,MDEyOklzc3VlQ29tbWVudDM5NzgyMzkxMw==,9599,simonw,2018-06-16T16:32:07Z,2018-06-16T16:48:48Z,OWNER,"Still todo:
- [ ] HTML view to obey the ?_labels=1 param (it throws an error at the moment)
- [ ] `?_label=one&_label=2` support for only expanding specific labels
- [ ] Better docs","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",332998752,"?_labels=1 to expand foreign keys (in csv and json), refs #233",
https://github.com/simonw/datasette/issues/233#issuecomment-397824991,https://api.github.com/repos/simonw/datasette/issues/233,397824991,MDEyOklzc3VlQ29tbWVudDM5NzgyNDk5MQ==,9599,simonw,2018-06-16T16:50:31Z,2018-06-16T16:50:42Z,OWNER,"I'm going to support `?_labels=` on HTML views, but I'll allow it to be used to turn them off (they are on by default) using `?_labels=off`.
Related: 7e0caa1e62607c6579101cc0e62bec8899013715 where I added a new `value_as_boolean` helper extracted from how `--config` works in `cli.py`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316444720,Option to expose expanded foreign keys in JSON/CSV,
https://github.com/simonw/datasette/issues/312#issuecomment-397825583,https://api.github.com/repos/simonw/datasette/issues/312,397825583,MDEyOklzc3VlQ29tbWVudDM5NzgyNTU4Mw==,9599,simonw,2018-06-16T17:00:12Z,2018-06-16T17:00:12Z,OWNER,This is already covered by #292 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333000163,"HTML, CSV and JSON views should support ?_col=&_col=",
https://github.com/simonw/datasette/issues/233#issuecomment-397839482,https://api.github.com/repos/simonw/datasette/issues/233,397839482,MDEyOklzc3VlQ29tbWVudDM5NzgzOTQ4Mg==,9599,simonw,2018-06-16T21:21:03Z,2018-06-16T21:21:03Z,OWNER,Should facets always have their labels expanded or should they also obey the `_labels` and `_label` querystring arguments?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316444720,Option to expose expanded foreign keys in JSON/CSV,
https://github.com/simonw/datasette/issues/233#issuecomment-397839583,https://api.github.com/repos/simonw/datasette/issues/233,397839583,MDEyOklzc3VlQ29tbWVudDM5NzgzOTU4Mw==,9599,simonw,2018-06-16T21:23:14Z,2018-06-16T21:23:44Z,OWNER,"I'm a bit torn on naming - choices are:
* `?_labels=on` and `?_label=col1&_label=col2`
* `?_expands=on` (or `?_expand_all=on`) and `?_expand=col1&_expand=col2`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316444720,Option to expose expanded foreign keys in JSON/CSV,
https://github.com/simonw/datasette/issues/233#issuecomment-397840676,https://api.github.com/repos/simonw/datasette/issues/233,397840676,MDEyOklzc3VlQ29tbWVudDM5Nzg0MDY3Ng==,9599,simonw,2018-06-16T21:49:50Z,2018-06-16T21:49:50Z,OWNER,For the moment I'm going with `_labels=`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316444720,Option to expose expanded foreign keys in JSON/CSV,
https://github.com/simonw/datasette/pull/311#issuecomment-397841968,https://api.github.com/repos/simonw/datasette/issues/311,397841968,MDEyOklzc3VlQ29tbWVudDM5Nzg0MTk2OA==,9599,simonw,2018-06-16T22:20:31Z,2018-06-16T22:20:31Z,OWNER,I merged this manually in ed631e690b81e34fcaeaba1f16c9166f1c505990,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",332998752,"?_labels=1 to expand foreign keys (in csv and json), refs #233",
https://github.com/simonw/datasette/issues/233#issuecomment-397842194,https://api.github.com/repos/simonw/datasette/issues/233,397842194,MDEyOklzc3VlQ29tbWVudDM5Nzg0MjE5NA==,9599,simonw,2018-06-16T22:26:21Z,2018-06-16T22:26:21Z,OWNER,"Some demos:
* https://datasette-labels-demo.now.sh/sf-trees-02c8ef1/Street_Tree_List - regular HTML view
* https://datasette-labels-demo.now.sh/sf-trees-02c8ef1/Street_Tree_List?_labels=off - no labels
* https://datasette-labels-demo.now.sh/sf-trees-02c8ef1/Street_Tree_List.json?_labels=on - JSON with all labels
* https://datasette-labels-demo.now.sh/sf-trees-02c8ef1/Street_Tree_List.json?_label=qSpecies&_shape=array - JSON with specific labels in array shape
* https://datasette-labels-demo.now.sh/sf-trees-02c8ef1/Street_Tree_List.csv?_labels=on - CSV with all labels
* https://datasette-labels-demo.now.sh/sf-trees-02c8ef1/Street_Tree_List.csv?_label=qSpecies - CSV with specific labels","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316444720,Option to expose expanded foreign keys in JSON/CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397842246,https://api.github.com/repos/simonw/datasette/issues/266,397842246,MDEyOklzc3VlQ29tbWVudDM5Nzg0MjI0Ng==,9599,simonw,2018-06-16T22:27:59Z,2018-06-16T22:27:59Z,OWNER,"Two demos of the new functionality in #233 as it applies to CSV:
* https://datasette-labels-demo.now.sh/sf-trees-02c8ef1/Street_Tree_List.csv?_labels=on - CSV with all foreign key columns expanded
* https://datasette-labels-demo.now.sh/sf-trees-02c8ef1/Street_Tree_List.csv?_label=qSpecies - CSV with specific columns expanded","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/313#issuecomment-397900434,https://api.github.com/repos/simonw/datasette/issues/313,397900434,MDEyOklzc3VlQ29tbWVudDM5NzkwMDQzNA==,9599,simonw,2018-06-17T19:23:23Z,2018-06-17T19:23:23Z,OWNER,This will require some relatively sophisticated Travis build steps. Useful docs: https://docs.travis-ci.com/user/build-stages/ - useful example: https://docs.travis-ci.com/user/build-stages/deploy-heroku/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333086005,Deploy demo of Datasette on every commit that passes tests,
https://github.com/simonw/datasette/issues/313#issuecomment-397907987,https://api.github.com/repos/simonw/datasette/issues/313,397907987,MDEyOklzc3VlQ29tbWVudDM5NzkwNzk4Nw==,9599,simonw,2018-06-17T21:32:52Z,2018-06-17T21:32:52Z,OWNER,"This very nearly works...
* https://latest.datasette.io/
* https://f0c1722.datasette.io/
But... https://f0c1722.datasette.io/-/versions isn't showing the correct note:
```
{
""datasette"": {
""version"": ""0.22.1""
}
...
```
There should be a `""note""` field there with the full commit hash.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333086005,Deploy demo of Datasette on every commit that passes tests,
https://github.com/simonw/datasette/issues/313#issuecomment-397908093,https://api.github.com/repos/simonw/datasette/issues/313,397908093,MDEyOklzc3VlQ29tbWVudDM5NzkwODA5Mw==,9599,simonw,2018-06-17T21:34:52Z,2018-06-17T21:34:52Z,OWNER,"It looks like all of my test deploys ended up going to the same Zeit deployment ID: https://zeit.co/simonw/datasette-latest/rbmtcedvlj
This is strange... the Dockerfile should be different for each one (due to the differing version-note).
https://github.com/simonw/datasette/commit/db1e6bc182d11f333e6addaa1a6be87625a4e12b#diff-34418c57343344c73271e13b01b7fcd9R255","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333086005,Deploy demo of Datasette on every commit that passes tests,
https://github.com/simonw/datasette/issues/313#issuecomment-397908185,https://api.github.com/repos/simonw/datasette/issues/313,397908185,MDEyOklzc3VlQ29tbWVudDM5NzkwODE4NQ==,9599,simonw,2018-06-17T21:36:50Z,2018-06-17T21:36:50Z,OWNER,"```
The command ""datasette publish now fixtures.db -m fixtures.json --token=$NOW_TOKEN --branch=$TRAVIS_COMMIT --version-note=$TRAVIS_COMMIT"" exited with 0.
```
Partial log of the ``datasette publish now`` output:
```
> Step 5/7 : RUN datasette inspect fixtures.db --inspect-file inspect-data.json
> ---> Running in d373f330e53e
> ---> 09bab386aaa3
> Removing intermediate container d373f330e53e
> Step 6/7 : EXPOSE 8001
> ---> Running in e0fe37b3061c
> ---> 47798440e214
> Removing intermediate container e0fe37b3061c
> Step 7/7 : CMD datasette serve --host 0.0.0.0 fixtures.db --cors --port 8001 --inspect-file inspect-data.json --metadata metadata.json --version-note f0c17229b7a7914d3da02e087dfd0e25d8321448
```
So it looks like `--version-note` is being correctly set there.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333086005,Deploy demo of Datasette on every commit that passes tests,
https://github.com/simonw/datasette/issues/313#issuecomment-397908614,https://api.github.com/repos/simonw/datasette/issues/313,397908614,MDEyOklzc3VlQ29tbWVudDM5NzkwODYxNA==,9599,simonw,2018-06-17T21:44:51Z,2018-06-17T21:45:03Z,OWNER,"Aha!
```1.03s$ now alias --token=$NOW_TOKEN
> Error! Couldn't find a deployment to alias. Please provide one as an argument.
The command ""now alias --token=$NOW_TOKEN"" exited with 1.
```
That explains it. I need to set the same alias in my call to `datasette publish`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333086005,Deploy demo of Datasette on every commit that passes tests,
https://github.com/simonw/datasette/issues/313#issuecomment-397908947,https://api.github.com/repos/simonw/datasette/issues/313,397908947,MDEyOklzc3VlQ29tbWVudDM5NzkwODk0Nw==,9599,simonw,2018-06-17T21:51:34Z,2018-06-17T21:51:34Z,OWNER,"That fixed it! https://958b75c.datasette.io/-/versions
```
{
""python"": {
""version"": ""3.6.5"",
""full"": ""3.6.5 (default, Jun 6 2018, 19:19:24) \n[GCC 6.3.0 20170516]""
},
""datasette"": {
""version"": ""0+unknown"",
""note"": ""958b75c69841ef5913da86e0eb2df634a9b95fda""
},
""sqlite"": {
""version"": ""3.16.2"",
""fts_versions"": [
""FTS5"",
""FTS4"",
""FTS3""
],
""extensions"": {
""json1"": null
}
}
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333086005,Deploy demo of Datasette on every commit that passes tests,
https://github.com/simonw/datasette/issues/266#issuecomment-397912840,https://api.github.com/repos/simonw/datasette/issues/266,397912840,MDEyOklzc3VlQ29tbWVudDM5NzkxMjg0MA==,9599,simonw,2018-06-17T23:13:35Z,2018-06-17T23:16:42Z,OWNER,"This worked! https://github.com/simonw/datasette/commit/5a0a82faf9cf9dd109d76181ed00eea19472087c - it spat out a 76MB CSV when I ran it against the sf-trees demo database.
It was just a quick hack though - it currently ignores `_labels=` and `_dl=` which need to be supported.
I'm going to add a config option for turning full CSV export off just in case any Datasette users are uncomfortable with URLs that churn out that much data in one go.
```
ConfigOption(""allow_csv_stream"", True, """"""
Allow .csv?_stream=1 to download all rows (ignoring max_returned_rows)
"""""".strip()),
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397915258,https://api.github.com/repos/simonw/datasette/issues/266,397915258,MDEyOklzc3VlQ29tbWVudDM5NzkxNTI1OA==,9599,simonw,2018-06-18T00:01:05Z,2018-06-18T00:01:05Z,OWNER,Someone malicious could use a UNION to generate an unpleasantly large CSV response. I'll add another config setting which limits the response size to 100MB but can be turned off by setting it to 0.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397915403,https://api.github.com/repos/simonw/datasette/issues/266,397915403,MDEyOklzc3VlQ29tbWVudDM5NzkxNTQwMw==,9599,simonw,2018-06-18T00:03:17Z,2018-06-18T00:14:37Z,OWNER,"Since CSV streaming export doesn't work for custom SQL queries (since they don't support `_next=` pagination) there's no need to provide a option that disables streams just for custom SQL.
Related: the UI should not show the option to download everything on custom SQL pages.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397916091,https://api.github.com/repos/simonw/datasette/issues/266,397916091,MDEyOklzc3VlQ29tbWVudDM5NzkxNjA5MQ==,9599,simonw,2018-06-18T00:13:43Z,2018-06-18T00:15:50Z,OWNER,I was also worried about the performance of pagination over custom `_sort` orders or views which use offset pagination - but Datasette's SQL time limits should prevent those from getting out of hand. This does mean that a streaming CSV file may be truncated with an error - if this happens we should ensure the error is written out as the last line of the CSV so anyone who tried to import it gets a relevant error message informing them that the export did not complete.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397916321,https://api.github.com/repos/simonw/datasette/issues/266,397916321,MDEyOklzc3VlQ29tbWVudDM5NzkxNjMyMQ==,9599,simonw,2018-06-18T00:17:44Z,2018-06-18T00:18:05Z,OWNER,The export UI could be a GET form controlling various parameters. This would discourage crawlers from hitting the export links and would also allow us to express the full range of export options.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397918264,https://api.github.com/repos/simonw/datasette/issues/266,397918264,MDEyOklzc3VlQ29tbWVudDM5NzkxODI2NA==,9599,simonw,2018-06-18T00:49:35Z,2018-06-18T00:49:35Z,OWNER,"Simpler design: the top of the page will link to basic .json and .csv and ""advanced"" - which will fragment link to an advanced export format the bottom of the page.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397923253,https://api.github.com/repos/simonw/datasette/issues/266,397923253,MDEyOklzc3VlQ29tbWVudDM5NzkyMzI1Mw==,9599,simonw,2018-06-18T01:49:52Z,2018-06-18T03:02:28Z,OWNER,Ideally the downloadable filenames of exported CSVs would differ across different querystring parameters. Maybe S`treet_Trees-56cbd54.csv` where `56cbd54` is a hash of the querystring?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397949002,https://api.github.com/repos/simonw/datasette/issues/266,397949002,MDEyOklzc3VlQ29tbWVudDM5Nzk0OTAwMg==,9599,simonw,2018-06-18T05:53:17Z,2018-06-18T05:53:17Z,OWNER,"Advanced export pane:
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397842667,https://api.github.com/repos/simonw/datasette/issues/266,397842667,MDEyOklzc3VlQ29tbWVudDM5Nzg0MjY2Nw==,9599,simonw,2018-06-16T22:38:15Z,2018-06-18T05:55:11Z,OWNER,"Still todo:
- [x] Streaming version
- [ ] Tidy up the ""This data as ..."" UI
- [x] Default .csv (and .json) links to use `?_labels=on` (only if at least one foreign key detected)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/266#issuecomment-397952129,https://api.github.com/repos/simonw/datasette/issues/266,397952129,MDEyOklzc3VlQ29tbWVudDM5Nzk1MjEyOQ==,9599,simonw,2018-06-18T06:15:36Z,2018-06-18T06:15:51Z,OWNER,Advanced export pane demo: https://latest.datasette.io/fixtures-35b6eb6/facetable?_size=4,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/316#issuecomment-398030903,https://api.github.com/repos/simonw/datasette/issues/316,398030903,MDEyOklzc3VlQ29tbWVudDM5ODAzMDkwMw==,132230,gavinband,2018-06-18T12:00:43Z,2018-06-18T12:00:43Z,NONE,"I should add that I'm using datasette version 0.22, Python 2.7.10 on Mac OS X. Happy to send more info if helpful.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333238932,datasette inspect takes a very long time on large dbs,
https://github.com/simonw/datasette/issues/266#issuecomment-398098582,https://api.github.com/repos/simonw/datasette/issues/266,398098582,MDEyOklzc3VlQ29tbWVudDM5ODA5ODU4Mg==,9599,simonw,2018-06-18T15:40:32Z,2018-06-18T15:40:32Z,OWNER,This is now released in Datasette 0.23! http://datasette.readthedocs.io/en/latest/changelog.html#v0-23,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323681589,Export to CSV,
https://github.com/simonw/datasette/issues/316#issuecomment-398101670,https://api.github.com/repos/simonw/datasette/issues/316,398101670,MDEyOklzc3VlQ29tbWVudDM5ODEwMTY3MA==,9599,simonw,2018-06-18T15:49:35Z,2018-06-18T15:50:38Z,OWNER,"Wow, I've gone as high as 7GB but I've never tried it against 600GB.
`datasette inspect` is indeed expected to take a long time for large databases. That's why it's available as a separate command: by running `datasette inspect` to generate `inspect-data.json` you can execute it just once against a large database and then have `datasette serve` take advantage of that cached metadata (hence avoiding `datasette serve` hanging on startup).
As you spotted, most of the time is spent in those counts. I imagine you don't need those row counts in order for the rest of Datasette to function correctly (they are mainly used for display purposes - on the https://latest.datasette.io/fixtures index page for example).
If your database changes infrequently, for the moment I recommend running `datasette inspect` once to generate the `inspect-data.json` file (let me know how long it takes) and then passing that file to `datasette serve mydb.db --inspect-file=inspect-data.json`
If your database DOES change frequently then this workaround won't help you much. Let me know and I'll see how much work it would take to have those row counts be optional rather than required.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333238932,datasette inspect takes a very long time on large dbs,
https://github.com/simonw/datasette/issues/265#issuecomment-398102537,https://api.github.com/repos/simonw/datasette/issues/265,398102537,MDEyOklzc3VlQ29tbWVudDM5ODEwMjUzNw==,9599,simonw,2018-06-18T15:52:15Z,2018-06-18T15:52:15Z,OWNER,https://latest.datasette.io/ now always hosts the latest version of the code. I've started linking to it from our documentation.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323677499,Add links to example Datasette instances to appropiate places in docs,
https://github.com/simonw/datasette/issues/316#issuecomment-398109204,https://api.github.com/repos/simonw/datasette/issues/316,398109204,MDEyOklzc3VlQ29tbWVudDM5ODEwOTIwNA==,132230,gavinband,2018-06-18T16:12:45Z,2018-06-18T16:12:45Z,NONE,"Hi Simon,
Thanks for the response. Ok I'll try running `datasette inspect` up front.
In principle the db won't change. However, the site's in development and it's likely I'll need to add views and some auxiliary (smaller) tables as I go along. I will need to be careful with this if it involves an inspect step in each iteration, though.
g.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333238932,datasette inspect takes a very long time on large dbs,
https://github.com/simonw/datasette/issues/271#issuecomment-398133924,https://api.github.com/repos/simonw/datasette/issues/271,398133924,MDEyOklzc3VlQ29tbWVudDM5ODEzMzkyNA==,9599,simonw,2018-06-18T17:32:22Z,2018-06-18T17:32:22Z,OWNER,"As seen in #316 inspect is already taking a VERY long time to run against large (600GB) databases.
To get this working I may have to make inspect an optional optimization and run introspection for columns and primary keys in demand.
The one catch here is the `count(*)` queries - Datasette may need to learn not to return full table counts in circumstances where the count has not been pre-calculates and takes more than Xms to generate.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324162476,Mechanism for automatically picking up changes when on-disk .db file changes,
https://github.com/simonw/datasette/issues/188#issuecomment-398778485,https://api.github.com/repos/simonw/datasette/issues/188,398778485,MDEyOklzc3VlQ29tbWVudDM5ODc3ODQ4NQ==,12617395,bsilverm,2018-06-20T14:48:39Z,2018-06-20T14:48:39Z,NONE,This would be a great feature to have!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",309047460,Ability to bundle metadata and templates inside the SQLite file,
https://github.com/simonw/datasette/issues/302#issuecomment-398825294,https://api.github.com/repos/simonw/datasette/issues/302,398825294,MDEyOklzc3VlQ29tbWVudDM5ODgyNTI5NA==,9599,simonw,2018-06-20T17:06:36Z,2018-06-20T17:06:36Z,OWNER,Still a bug in 0.23,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",328171513,test-2.3.sqlite database filename throws a 404,
https://github.com/simonw/datasette/issues/318#issuecomment-398973309,https://api.github.com/repos/simonw/datasette/issues/318,398973309,MDEyOklzc3VlQ29tbWVudDM5ODk3MzMwOQ==,9599,simonw,2018-06-21T04:35:12Z,2018-06-21T04:37:37Z,OWNER,"Demo of fix: the `on_earth` facet on https://latest.datasette.io/fixtures-cafd088/facetable?_facet=planet_int&_facet=on_earth&_facet=city_id
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334148669,Facets with value of 0 displayed incorrectly,
https://github.com/simonw/datasette/issues/321#issuecomment-398976488,https://api.github.com/repos/simonw/datasette/issues/321,398976488,MDEyOklzc3VlQ29tbWVudDM5ODk3NjQ4OA==,9599,simonw,2018-06-21T04:59:33Z,2018-06-21T06:11:02Z,OWNER,"I've added this to the unit tests and the documentation.
Docs: http://datasette.readthedocs.io/en/latest/sql_queries.html#canned-queries
Canned query demo: https://latest.datasette.io/fixtures/neighborhood_search?text=town
New unit test:
https://github.com/simonw/datasette/blob/3683a6b626b2e79f4dc9600d45853ca4ae8de11a/tests/test_api.py#L333-L344
https://github.com/simonw/datasette/blob/3683a6b626b2e79f4dc9600d45853ca4ae8de11a/tests/fixtures.py#L145-L153","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334190959,Wildcard support in query parameters,
https://github.com/simonw/datasette/issues/321#issuecomment-399098080,https://api.github.com/repos/simonw/datasette/issues/321,399098080,MDEyOklzc3VlQ29tbWVudDM5OTA5ODA4MA==,12617395,bsilverm,2018-06-21T13:10:48Z,2018-06-21T13:10:48Z,NONE,"Perfect, thank you!!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334190959,Wildcard support in query parameters,
https://github.com/simonw/datasette/issues/321#issuecomment-399106871,https://api.github.com/repos/simonw/datasette/issues/321,399106871,MDEyOklzc3VlQ29tbWVudDM5OTEwNjg3MQ==,12617395,bsilverm,2018-06-21T13:39:37Z,2018-06-21T13:39:37Z,NONE,"One thing I've noticed with this approach is that the query is executed with no parameters which I do not believe was the case previously. In the case the table contains a lot of data, this adds some time executing the query before the user can enter their input and run it with the parameters they want.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334190959,Wildcard support in query parameters,
https://github.com/simonw/datasette/issues/321#issuecomment-399126228,https://api.github.com/repos/simonw/datasette/issues/321,399126228,MDEyOklzc3VlQ29tbWVudDM5OTEyNjIyOA==,9599,simonw,2018-06-21T14:36:40Z,2018-06-21T14:36:53Z,OWNER,"This seems to fix that:
```
select neighborhood, facet_cities.name, state
from facetable
join facet_cities on facetable.city_id = facet_cities.id
where :text != '' and neighborhood like '%' || :text || '%'
order by neighborhood;
```
Compare this (with empty string): https://latest.datasette.io/fixtures-cafd088?sql=select+neighborhood%2C+facet_cities.name%2C+state%0D%0Afrom+facetable%0D%0A++++join+facet_cities+on+facetable.city_id+%3D+facet_cities.id%0D%0Awhere+%3Atext+%21%3D+%22%22+and+neighborhood+like+%27%25%27+%7C%7C+%3Atext+%7C%7C+%27%25%27%0D%0Aorder+by+neighborhood%3B
To this: https://latest.datasette.io/fixtures-cafd088?sql=select+neighborhood%2C+facet_cities.name%2C+state%0D%0Afrom+facetable%0D%0A++++join+facet_cities+on+facetable.city_id+%3D+facet_cities.id%0D%0Awhere+%3Atext+%21%3D+%22%22+and+neighborhood+like+%27%25%27+%7C%7C+%3Atext+%7C%7C+%27%25%27%0D%0Aorder+by+neighborhood%3B&text=town","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334190959,Wildcard support in query parameters,
https://github.com/simonw/datasette/issues/321#issuecomment-399129220,https://api.github.com/repos/simonw/datasette/issues/321,399129220,MDEyOklzc3VlQ29tbWVudDM5OTEyOTIyMA==,12617395,bsilverm,2018-06-21T14:45:02Z,2018-06-21T14:45:02Z,NONE,Those queries look identical. How can this be prevented if the queries are in a metadata.json file?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334190959,Wildcard support in query parameters,
https://github.com/simonw/datasette/issues/309#issuecomment-399134680,https://api.github.com/repos/simonw/datasette/issues/309,399134680,MDEyOklzc3VlQ29tbWVudDM5OTEzNDY4MA==,9599,simonw,2018-06-21T14:59:57Z,2018-06-21T14:59:57Z,OWNER,I can use Sanic middleware for this: http://sanic.readthedocs.io/en/latest/sanic/middleware.html#responding-early,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",331343824,On 404s with a trailing slash redirect to that page without a trailing slash,
https://github.com/simonw/datasette/issues/319#issuecomment-399139462,https://api.github.com/repos/simonw/datasette/issues/319,399139462,MDEyOklzc3VlQ29tbWVudDM5OTEzOTQ2Mg==,9599,simonw,2018-06-21T15:13:58Z,2018-06-21T15:13:58Z,OWNER,"Demo of fix: https://latest.datasette.io/fixtures-e14e080/searchable_tags
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334149717,Incorrect display of compound primary keys with foreign key relationships,
https://github.com/simonw/datasette/issues/309#issuecomment-399142274,https://api.github.com/repos/simonw/datasette/issues/309,399142274,MDEyOklzc3VlQ29tbWVudDM5OTE0MjI3NA==,9599,simonw,2018-06-21T15:22:02Z,2018-06-21T15:22:02Z,OWNER,Demo: https://latest.datasette.io/fixtures-e14e080/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",331343824,On 404s with a trailing slash redirect to that page without a trailing slash,
https://github.com/simonw/datasette/issues/317#issuecomment-399144688,https://api.github.com/repos/simonw/datasette/issues/317,399144688,MDEyOklzc3VlQ29tbWVudDM5OTE0NDY4OA==,9599,simonw,2018-06-21T15:29:06Z,2018-06-21T15:29:16Z,OWNER,"From https://docs.travis-ci.com/user/deployment/pypi/
> Note that if your PyPI password contains special characters you need to escape them before encrypting your password. Some people have [reported difficulties](https://github.com/travis-ci/dpl/issues/377) connecting to PyPI with passwords containing anything except alphanumeric characters.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333326107,Travis CI fails to upload new releases to PyPI,
https://github.com/simonw/datasette/issues/317#issuecomment-399150285,https://api.github.com/repos/simonw/datasette/issues/317,399150285,MDEyOklzc3VlQ29tbWVudDM5OTE1MDI4NQ==,9599,simonw,2018-06-21T15:45:47Z,2018-06-21T15:45:47Z,OWNER,That fixed it! https://travis-ci.org/simonw/datasette/jobs/395078407 ran successfully and https://pypi.org/project/datasette/ now hosts Datasette 0.23.1 deployed via Travis.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333326107,Travis CI fails to upload new releases to PyPI,
https://github.com/simonw/datasette/issues/319#issuecomment-399154550,https://api.github.com/repos/simonw/datasette/issues/319,399154550,MDEyOklzc3VlQ29tbWVudDM5OTE1NDU1MA==,9599,simonw,2018-06-21T15:58:15Z,2018-06-21T15:58:15Z,OWNER,Fixed here too now: https://registry.datasette.io/registry-c10707b/datasette_tags,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334149717,Incorrect display of compound primary keys with foreign key relationships,
https://github.com/simonw/datasette/issues/314#issuecomment-399156960,https://api.github.com/repos/simonw/datasette/issues/314,399156960,MDEyOklzc3VlQ29tbWVudDM5OTE1Njk2MA==,9599,simonw,2018-06-21T16:04:59Z,2018-06-21T16:04:59Z,OWNER,"Demo of fix: https://latest.datasette.io/fixtures-e14e080/simple_view
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333096176,HTML table does not correctly display entirely blank rows,
https://github.com/simonw/datasette/issues/259#issuecomment-399157944,https://api.github.com/repos/simonw/datasette/issues/259,399157944,MDEyOklzc3VlQ29tbWVudDM5OTE1Nzk0NA==,9599,simonw,2018-06-21T16:07:49Z,2018-06-21T16:07:49Z,OWNER,Thanks to #319 the test suite now includes a m2m table: https://latest.datasette.io/fixtures-e14e080/searchable_tags,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322787470,inspect() should detect many-to-many relationships,
https://github.com/simonw/datasette/issues/321#issuecomment-399171239,https://api.github.com/repos/simonw/datasette/issues/321,399171239,MDEyOklzc3VlQ29tbWVudDM5OTE3MTIzOQ==,9599,simonw,2018-06-21T16:51:08Z,2018-06-21T16:51:29Z,OWNER,"I may have misunderstood your problem here.
I understood that the problem is that when using the `""%"" || :text || ""%""` construct the first hit to that page (with an empty string for `:text`) results in a `where neighborhood like ""%%""` query which is slow because it matches every row in the database.
My fix was to add this to the where clause:
where :text != '' and ...
Which means that when you first load the page the where fails to match any rows and you get no results (and hopefully instant loading times assuming SQLite is smart enough to optimize this away). That's why you don't see any rows returned on this page: https://latest.datasette.io/fixtures-cafd088?sql=select+neighborhood%2C+facet_cities.name%2C+state%0D%0Afrom+facetable%0D%0A++++join+facet_cities+on+facetable.city_id+%3D+facet_cities.id%0D%0Awhere+%3Atext+%21%3D+%22%22+and+neighborhood+like+%27%25%27+%7C%7C+%3Atext+%7C%7C+%27%25%27%0D%0Aorder+by+neighborhood%3B","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334190959,Wildcard support in query parameters,
https://github.com/simonw/datasette/issues/321#issuecomment-398973176,https://api.github.com/repos/simonw/datasette/issues/321,398973176,MDEyOklzc3VlQ29tbWVudDM5ODk3MzE3Ng==,9599,simonw,2018-06-21T04:34:11Z,2018-06-21T16:53:57Z,OWNER,"This is a little bit fiddly, but it's possible to do it using SQLite string concatenation. Here's an example:
```
select * from facetable
where neighborhood like ""%"" || :text || ""%"";
```
Try it here: https://latest.datasette.io/fixtures-35b6eb6?sql=select+*+from+facetable+where+neighborhood+like+%22%25%22+%7C%7C+%3Atext+%7C%7C+%22%25%22%3B&text=town
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334190959,Wildcard support in query parameters,
https://github.com/simonw/datasette/issues/321#issuecomment-399173916,https://api.github.com/repos/simonw/datasette/issues/321,399173916,MDEyOklzc3VlQ29tbWVudDM5OTE3MzkxNg==,12617395,bsilverm,2018-06-21T17:00:10Z,2018-06-21T17:00:10Z,NONE,"Oh I see.. My issue is that the query executes with an empty string prior to the user submitting the parameters. I'll try adding your workaround to some of my queries. Thanks again,","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334190959,Wildcard support in query parameters,
https://github.com/simonw/datasette/issues/326#issuecomment-399721346,https://api.github.com/repos/simonw/datasette/issues/326,399721346,MDEyOklzc3VlQ29tbWVudDM5OTcyMTM0Ng==,9599,simonw,2018-06-24T01:10:26Z,2018-06-24T01:10:26Z,OWNER,"Demo: go to https://vega.github.io/editor/ and paste in the following:
```
{
""data"": {
""url"": ""https://fivethirtyeight.datasettes.com/fivethirtyeight/twitter-ratio%2Fsenators.csv?_size=max&_sort_desc=replies"",
""format"": {
""type"": ""csv""
}
},
""mark"": ""bar"",
""encoding"": {
""x"": {
""field"": ""created_at"",
""type"": ""temporal""
},
""y"": {
""field"": ""replies"",
""type"": ""quantitative""
},
""color"": {
""field"": ""user"",
""type"": ""nominal""
}
}
}
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",335141434,CSV should respect --cors and return cors headers,
https://github.com/simonw/datasette/issues/272#issuecomment-400166540,https://api.github.com/repos/simonw/datasette/issues/272,400166540,MDEyOklzc3VlQ29tbWVudDQwMDE2NjU0MA==,9599,simonw,2018-06-26T03:29:43Z,2018-06-26T03:29:43Z,OWNER,This looks VERY relevant: https://github.com/encode/starlette,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/272#issuecomment-400571521,https://api.github.com/repos/simonw/datasette/issues/272,400571521,MDEyOklzc3VlQ29tbWVudDQwMDU3MTUyMQ==,647359,tomchristie,2018-06-27T07:30:07Z,2018-06-27T07:30:07Z,NONE,"I’m up for helping with this.
Looks like you’d need static files support, which I’m planning on adding a component for. Anything else obviously missing?
For a quick overview it looks very doable - the test client ought to me your test cases stay roughly the same.
Are you using any middleware or other components for the Sanic ecosystem? Do you use cookies or sessions at all?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/328#issuecomment-400903687,https://api.github.com/repos/simonw/datasette/issues/328,400903687,MDEyOklzc3VlQ29tbWVudDQwMDkwMzY4Nw==,9599,simonw,2018-06-28T04:00:18Z,2018-06-28T04:00:18Z,OWNER,Need to ship docker image: #57 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336464733,"Installation instructions, including how to use the docker image",
https://github.com/simonw/datasette/issues/57#issuecomment-400903871,https://api.github.com/repos/simonw/datasette/issues/57,400903871,MDEyOklzc3VlQ29tbWVudDQwMDkwMzg3MQ==,9599,simonw,2018-06-28T04:01:38Z,2018-06-28T04:01:38Z,OWNER,"Shipped to Docker Hub: https://hub.docker.com/r/datasetteproject/datasette/
I did this manually the first time. I'll set Travis up to do this automatically in #329","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",273127694,Ship a Docker image of the whole thing,
https://github.com/simonw/datasette/issues/328#issuecomment-400904514,https://api.github.com/repos/simonw/datasette/issues/328,400904514,MDEyOklzc3VlQ29tbWVudDQwMDkwNDUxNA==,9599,simonw,2018-06-28T04:06:39Z,2018-06-28T04:06:39Z,OWNER,https://datasette.readthedocs.io/en/latest/installation.html#using-docker,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336464733,"Installation instructions, including how to use the docker image",
https://github.com/simonw/datasette/pull/280#issuecomment-401003061,https://api.github.com/repos/simonw/datasette/issues/280,401003061,MDEyOklzc3VlQ29tbWVudDQwMTAwMzA2MQ==,9599,simonw,2018-06-28T11:26:23Z,2018-06-28T11:26:23Z,OWNER,I pushed this to Docker Hub https://hub.docker.com/r/datasetteproject/datasette/ and added notes on how to use it to the documentation: http://datasette.readthedocs.io/en/latest/installation.html#using-docker,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",325373747,Build Dockerfile with recent Sqlite + Spatialite,
https://github.com/simonw/datasette/issues/276#issuecomment-401310732,https://api.github.com/repos/simonw/datasette/issues/276,401310732,MDEyOklzc3VlQ29tbWVudDQwMTMxMDczMg==,82988,psychemedia,2018-06-29T10:05:04Z,2018-06-29T10:07:25Z,CONTRIBUTOR,"@russs Different map projections can presumably be handled on the client side using a leaflet plugin to transform the geometry (eg [kartena/Proj4Leaflet](https://kartena.github.io/Proj4Leaflet/)) although the leaflet side would need to detect or be informed of the original projection?
Another possibility would be to provide an easy way/guidance for users to create an FK'd table containing the WGS84 projection of a non-WGS84 geometry in the original/principle table? This could then as a proxy for serving GeoJSON to the leaflet map?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/276#issuecomment-401312981,https://api.github.com/repos/simonw/datasette/issues/276,401312981,MDEyOklzc3VlQ29tbWVudDQwMTMxMjk4MQ==,45057,russss,2018-06-29T10:14:54Z,2018-06-29T10:14:54Z,CONTRIBUTOR,"> @RusSs Different map projections can presumably be handled on the client side using a leaflet plugin to transform the geometry (eg kartena/Proj4Leaflet) although the leaflet side would need to detect or be informed of the original projection?
Well, as @simonw mentioned, GeoJSON only supports WGS84, and GeoJSON (and/or TopoJSON) is the standard we probably want to aim for. On-the-fly reprojection in spatialite is not an issue anyway, and in general I think you want to be serving stuff to web maps in WGS84 or Web Mercator.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324835838,Handle spatialite geometry columns better,
https://github.com/simonw/datasette/issues/332#issuecomment-401477622,https://api.github.com/repos/simonw/datasette/issues/332,401477622,MDEyOklzc3VlQ29tbWVudDQwMTQ3NzYyMg==,9599,simonw,2018-06-29T21:23:17Z,2018-06-29T21:23:55Z,OWNER,"https://docs.python.org/3/library/json.html#json.dump
> **json.dump**(obj, fp, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, default=None, sort_keys=False, **kw)¶
> If `allow_nan` is false (default: True), then it will be a ValueError to serialize out of range float values (nan, inf, -inf) in strict compliance of the JSON specification. If allow_nan is true, their JavaScript equivalents (NaN, Infinity, -Infinity) will be used.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/332#issuecomment-401478223,https://api.github.com/repos/simonw/datasette/issues/332,401478223,MDEyOklzc3VlQ29tbWVudDQwMTQ3ODIyMw==,9599,simonw,2018-06-29T21:26:12Z,2018-06-29T21:26:19Z,OWNER,"I'm not sure what the correct thing to do here is. I don't want to throw a `ValueError` when trying to render that data as JSON, but I also want to produce JSON that doesn't break when fetched by JavaScript.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/334#issuecomment-403526263,https://api.github.com/repos/simonw/datasette/issues/334,403526263,MDEyOklzc3VlQ29tbWVudDQwMzUyNjI2Mw==,9599,simonw,2018-07-09T15:49:01Z,2018-07-09T15:49:01Z,OWNER,Yup that's definitely a bug.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",339095976,extra_options not passed to heroku publisher,
https://github.com/simonw/datasette/issues/325#issuecomment-403263890,https://api.github.com/repos/simonw/datasette/issues/325,403263890,MDEyOklzc3VlQ29tbWVudDQwMzI2Mzg5MA==,9599,simonw,2018-07-08T05:35:20Z,2018-07-09T17:28:27Z,OWNER,Fixed: https://v0-23-2.datasette.io/fixtures-e14e080/table%2Fwith%2Fslashes.csv / https://v0-23-2.datasette.io/fixtures-e14e080/table%2Fwith%2Fslashes.csv/3,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",335064777,Error on row page if table has slashes in the name and ends in .csv,
https://github.com/simonw/datasette/issues/334#issuecomment-403672561,https://api.github.com/repos/simonw/datasette/issues/334,403672561,MDEyOklzc3VlQ29tbWVudDQwMzY3MjU2MQ==,9599,simonw,2018-07-10T01:45:28Z,2018-07-10T01:45:28Z,OWNER,"Tested with `datasette publish heroku fixtures.db --extra-options=""--config sql_time_limit_ms:4000""`
https://blooming-anchorage-31561.herokuapp.com/-/config","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",339095976,extra_options not passed to heroku publisher,
https://github.com/simonw/datasette/issues/323#issuecomment-403855639,https://api.github.com/repos/simonw/datasette/issues/323,403855639,MDEyOklzc3VlQ29tbWVudDQwMzg1NTYzOQ==,9599,simonw,2018-07-10T15:03:36Z,2018-07-10T15:03:36Z,OWNER,I'm satisified with the improvement we got from the pip wheel cache.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334698969,Speed up Travis CI builds,
https://github.com/simonw/datasette/issues/330#issuecomment-403855963,https://api.github.com/repos/simonw/datasette/issues/330,403855963,MDEyOklzc3VlQ29tbWVudDQwMzg1NTk2Mw==,9599,simonw,2018-07-10T15:04:31Z,2018-07-10T15:04:31Z,OWNER,This relates to #276 - I'm definitely convinced now that displaying a giant `b'...'` blob on the page is not a useful default.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336924199,Limit text display in cells containing large amounts of text,
https://github.com/simonw/datasette/issues/331#issuecomment-403856114,https://api.github.com/repos/simonw/datasette/issues/331,403856114,MDEyOklzc3VlQ29tbWVudDQwMzg1NjExNA==,9599,simonw,2018-07-10T15:04:56Z,2018-07-10T15:04:56Z,OWNER,Great idea.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336936010,Datasette throws error when loading spatialite db without extension loaded,
https://github.com/simonw/datasette/issues/331#issuecomment-403858949,https://api.github.com/repos/simonw/datasette/issues/331,403858949,MDEyOklzc3VlQ29tbWVudDQwMzg1ODk0OQ==,9599,simonw,2018-07-10T15:12:53Z,2018-07-10T15:13:04Z,OWNER,"```
$ datasette airports.sqlite
Serve! files=('airports.sqlite',) on port 8001
Usage: datasette airports.sqlite [OPTIONS] [FILES]...
Error: It looks like you're trying to load a SpatiaLite database without first loading the SpatiaLite module.
Read more: https://datasette.readthedocs.io/en/latest/spatialite.html
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336936010,Datasette throws error when loading spatialite db without extension loaded,
https://github.com/simonw/datasette/issues/316#issuecomment-398133159,https://api.github.com/repos/simonw/datasette/issues/316,398133159,MDEyOklzc3VlQ29tbWVudDM5ODEzMzE1OQ==,9599,simonw,2018-06-18T17:29:59Z,2018-07-10T15:14:53Z,OWNER,"For #271 I've been contemplating having Datasette work against an on-disk database that gets modified without needing to restart the server.
For that to work, I'll have to dramatically change the inspect() mechanism. It may be that inspect becomes an optional optimization in the future.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",333238932,datasette inspect takes a very long time on large dbs,
https://github.com/simonw/datasette/issues/335#issuecomment-403865063,https://api.github.com/repos/simonw/datasette/issues/335,403865063,MDEyOklzc3VlQ29tbWVudDQwMzg2NTA2Mw==,9599,simonw,2018-07-10T15:29:32Z,2018-07-10T15:29:32Z,OWNER,"Huh... from https://docs.brew.sh/Acceptable-Formulae
> We frown on authors submitting their own work unless it is very popular.
Marking this one as ""help wanted"" :)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",339505204,Package datasette for installation using homebrew,
https://github.com/simonw/datasette/issues/335#issuecomment-403863927,https://api.github.com/repos/simonw/datasette/issues/335,403863927,MDEyOklzc3VlQ29tbWVudDQwMzg2MzkyNw==,9599,simonw,2018-07-10T15:26:27Z,2018-07-10T15:29:54Z,OWNER,Here are some useful examples of other Python apps that have been packaged using the recipe described above: https://github.com/Homebrew/homebrew-core/search?utf8=%E2%9C%93&q=virtualenv_install_with_resources&type=,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",339505204,Package datasette for installation using homebrew,
https://github.com/simonw/datasette/issues/335#issuecomment-403866099,https://api.github.com/repos/simonw/datasette/issues/335,403866099,MDEyOklzc3VlQ29tbWVudDQwMzg2NjA5OQ==,9599,simonw,2018-07-10T15:32:14Z,2018-07-10T15:32:14Z,OWNER,"I can host a custom tap without needing to get anything accepted into homebrew-core: https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap
Since my principle goal here is ensuring an easy installation path for people who are familiar with `brew` but don't know how to use pip and Python 3 that could be a good option.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",339505204,Package datasette for installation using homebrew,
https://github.com/simonw/datasette/issues/330#issuecomment-403868584,https://api.github.com/repos/simonw/datasette/issues/330,403868584,MDEyOklzc3VlQ29tbWVudDQwMzg2ODU4NA==,9599,simonw,2018-07-10T15:39:12Z,2018-07-10T16:21:08Z,OWNER,"I think this makes sense for the HTML view (not for JSON or CSV). It could be controlled be a new [config option](http://datasette.readthedocs.io/en/latest/config.html), `truncate_cells_html` - which is on by default but can be turned off.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336924199,Limit text display in cells containing large amounts of text,
https://github.com/simonw/datasette/issues/330#issuecomment-403906747,https://api.github.com/repos/simonw/datasette/issues/330,403906747,MDEyOklzc3VlQ29tbWVudDQwMzkwNjc0Nw==,9599,simonw,2018-07-10T17:39:46Z,2018-07-10T17:39:46Z,OWNER,"```
datasette publish now timezones.db --spatialite \
--extra-options=""--config truncate_cells_html:200"" \
--name=datasette-issue-330-demo \
--branch=master
```
https://datasette-issue-330-demo-sbelwxttfn.now.sh/timezones-3cb9f64/timezones
But https://datasette-issue-330-demo-sbelwxttfn.now.sh/timezones-3cb9f64/timezones/1 displays the full blob.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336924199,Limit text display in cells containing large amounts of text,
https://github.com/simonw/datasette/issues/330#issuecomment-403907193,https://api.github.com/repos/simonw/datasette/issues/330,403907193,MDEyOklzc3VlQ29tbWVudDQwMzkwNzE5Mw==,9599,simonw,2018-07-10T17:41:14Z,2018-07-10T17:41:14Z,OWNER,Documentation: http://datasette.readthedocs.io/en/latest/config.html#truncate-cells-html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336924199,Limit text display in cells containing large amounts of text,
https://github.com/simonw/datasette/issues/191#issuecomment-403908704,https://api.github.com/repos/simonw/datasette/issues/191,403908704,MDEyOklzc3VlQ29tbWVudDQwMzkwODcwNA==,9599,simonw,2018-07-10T17:46:13Z,2018-07-10T17:46:13Z,OWNER,I consider this resolved by #46 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",310533258,Figure out how to bundle a more up-to-date SQLite,
https://github.com/simonw/datasette/issues/139#issuecomment-403909389,https://api.github.com/repos/simonw/datasette/issues/139,403909389,MDEyOklzc3VlQ29tbWVudDQwMzkwOTM4OQ==,9599,simonw,2018-07-10T17:48:18Z,2018-07-10T17:48:18Z,OWNER,This is done! https://github.com/simonw/datasette-vega,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275493851,Build a visualization plugin for Vega,
https://github.com/simonw/datasette/issues/143#issuecomment-403909469,https://api.github.com/repos/simonw/datasette/issues/143,403909469,MDEyOklzc3VlQ29tbWVudDQwMzkwOTQ2OQ==,9599,simonw,2018-07-10T17:48:34Z,2018-07-10T17:48:34Z,OWNER,This is now a dupe of https://github.com/simonw/datasette-vega/issues/4,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275939188,"Mechanism for ""suggested visualizations""",
https://github.com/simonw/datasette/issues/87#issuecomment-403909671,https://api.github.com/repos/simonw/datasette/issues/87,403909671,MDEyOklzc3VlQ29tbWVudDQwMzkwOTY3MQ==,9599,simonw,2018-07-10T17:49:12Z,2018-07-10T17:49:12Z,OWNER,This was fixed by https://github.com/simonw/datasette/commit/6a32684ebba89dfe882e1147b23aa8778479f5d8#diff-354f30a63fb0907d4ad57269548329e3,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",273709194,Configure Travis to release new tags to PyPI,
https://github.com/simonw/datasette/issues/140#issuecomment-403910318,https://api.github.com/repos/simonw/datasette/issues/140,403910318,MDEyOklzc3VlQ29tbWVudDQwMzkxMDMxOA==,9599,simonw,2018-07-10T17:51:11Z,2018-07-10T17:51:11Z,OWNER,This would be a nice example plugin to demonstrate plugin configuration options in #231,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275755475,Heatmap visualization plugin,
https://github.com/simonw/datasette/issues/27#issuecomment-403910774,https://api.github.com/repos/simonw/datasette/issues/27,403910774,MDEyOklzc3VlQ29tbWVudDQwMzkxMDc3NA==,9599,simonw,2018-07-10T17:52:41Z,2018-07-10T17:52:41Z,OWNER,I consider this handled by https://github.com/simonw/datasette-vega,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",267886330,Ability to plot a simple graph,
https://github.com/simonw/datasette/issues/137#issuecomment-345750135,https://api.github.com/repos/simonw/datasette/issues/137,345750135,MDEyOklzc3VlQ29tbWVudDM0NTc1MDEzNQ==,9599,simonw,2017-11-20T16:30:56Z,2018-07-10T17:53:13Z,OWNER,"One possible route: introduce prefixes eg `?a.Trees.age__gt=5&a.Trees._group_count=qSpecies&b.Trees.age__gt=10&b.Trees._group_count=qSpecies`
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275415799,Ability to combine multiple SQL queries on a single graph,
https://github.com/simonw/datasette/issues/140#issuecomment-403939399,https://api.github.com/repos/simonw/datasette/issues/140,403939399,MDEyOklzc3VlQ29tbWVudDQwMzkzOTM5OQ==,9599,simonw,2018-07-10T19:30:17Z,2018-07-10T19:30:41Z,OWNER,Building this using Svelte would also produce a neat example of a plugin that uses Svelte: https://svelte.technology/guide - and if I like it I might part datasette-vega to it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275755475,Heatmap visualization plugin,
https://github.com/simonw/datasette/issues/272#issuecomment-403959704,https://api.github.com/repos/simonw/datasette/issues/272,403959704,MDEyOklzc3VlQ29tbWVudDQwMzk1OTcwNA==,9599,simonw,2018-07-10T20:44:47Z,2018-07-10T20:44:47Z,OWNER,"No cookies or sessions - no POST requests in fact, Datasette just cares about GET (path and querystring) and being able to return custom HTTP headers.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/336#issuecomment-403996143,https://api.github.com/repos/simonw/datasette/issues/336,403996143,MDEyOklzc3VlQ29tbWVudDQwMzk5NjE0Mw==,9599,simonw,2018-07-10T23:21:27Z,2018-07-10T23:21:27Z,OWNER,Easiest way to do this I think would be to make those help blocks separate files in the docs/ directory (publish-help.txt perhaps) and then include them with a sphinx directive: https://reinout.vanrees.org/weblog/2010/12/08/include-external-in-sphinx.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",340039409,Ensure --help examples in docs are always up to date,
https://github.com/simonw/datasette/issues/337#issuecomment-404021589,https://api.github.com/repos/simonw/datasette/issues/337,404021589,MDEyOklzc3VlQ29tbWVudDQwNDAyMTU4OQ==,9599,simonw,2018-07-11T02:07:32Z,2018-07-11T02:07:32Z,OWNER,http://datasette.readthedocs.io/en/latest/publish.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",340065374,Documentation for datasette publish and datasette package,
https://github.com/simonw/datasette/issues/336#issuecomment-404021890,https://api.github.com/repos/simonw/datasette/issues/336,404021890,MDEyOklzc3VlQ29tbWVudDQwNDAyMTg5MA==,9599,simonw,2018-07-11T02:09:25Z,2018-07-11T02:09:25Z,OWNER,"I decided against the unit tests, instead I have a new script called `./update-docs-help.sh` which I can run any time I want to refresh the included documentation: https://github.com/simonw/datasette/commit/aec3ae53237e43b0c268dbf9b58fa265ef38cfe1#diff-cb15a1e5a244bb82ad4afce67f252543","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",340039409,Ensure --help examples in docs are always up to date,
https://github.com/simonw/datasette/issues/335#issuecomment-404208602,https://api.github.com/repos/simonw/datasette/issues/335,404208602,MDEyOklzc3VlQ29tbWVudDQwNDIwODYwMg==,9599,simonw,2018-07-11T15:20:12Z,2018-07-11T15:20:12Z,OWNER,Here's a good example of a homebrew tap: https://github.com/saulpw/homebrew-vd,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",339505204,Package datasette for installation using homebrew,
https://github.com/simonw/datasette/issues/338#issuecomment-404209205,https://api.github.com/repos/simonw/datasette/issues/338,404209205,MDEyOklzc3VlQ29tbWVudDQwNDIwOTIwNQ==,9599,simonw,2018-07-11T15:21:47Z,2018-07-11T15:21:47Z,OWNER,"Oops, opened this in the wrong repo - moved it here: https://github.com/simonw/datasette-vega/issues/13","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",340282796,Only load vegaEmbed if charting tools are enabled,
https://github.com/simonw/datasette/issues/339#issuecomment-404338345,https://api.github.com/repos/simonw/datasette/issues/339,404338345,MDEyOklzc3VlQ29tbWVudDQwNDMzODM0NQ==,9599,simonw,2018-07-11T23:09:24Z,2018-07-11T23:09:24Z,OWNER,"It sounds like you're running into the Sanic default response timeout value of 60 seconds: https://github.com/channelcat/sanic/blob/master/docs/sanic/config.md#builtin-configuration-values
For the moment you can over-ride that using an environment variable like this:
SANIC_RESPONSE_TIMEOUT=6000 datasette fivethirtyeight.db -p 8008 --config sql_time_limit_ms:600000","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",340396247,Expose SANIC_RESPONSE_TIMEOUT config option in a sensible way,
https://github.com/simonw/datasette/issues/272#issuecomment-404514973,https://api.github.com/repos/simonw/datasette/issues/272,404514973,MDEyOklzc3VlQ29tbWVudDQwNDUxNDk3Mw==,647359,tomchristie,2018-07-12T13:38:24Z,2018-07-12T13:38:24Z,NONE,"Okay. I reckon the latest version should have all the kinds of components you'd need:
Recently added ASGI components for Routing and Static Files support, as well as making few tweaks to make sure requests and responses are instantiated efficiently.
Don't have any redirect-to-slash / redirect-to-non-slash stuff out of the box yet, which it looks like you might miss.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/339#issuecomment-404565566,https://api.github.com/repos/simonw/datasette/issues/339,404565566,MDEyOklzc3VlQ29tbWVudDQwNDU2NTU2Ng==,9599,simonw,2018-07-12T16:08:42Z,2018-07-12T16:08:42Z,OWNER,I'm going to turn this into an issue about better supporting the above option.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",340396247,Expose SANIC_RESPONSE_TIMEOUT config option in a sensible way,
https://github.com/simonw/datasette/issues/332#issuecomment-402243153,https://api.github.com/repos/simonw/datasette/issues/332,402243153,MDEyOklzc3VlQ29tbWVudDQwMjI0MzE1Mw==,9599,simonw,2018-07-03T17:58:50Z,2018-07-12T16:10:39Z,OWNER,"I think I'm going to return `null` in the JSON for infinity/nan values by default, but if you send `_nan=1` I will instead return invalid JSON with `Infinity` or `NaN` in it (since you have opted in to getting those and hence should be able to handle them).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/332#issuecomment-404567587,https://api.github.com/repos/simonw/datasette/issues/332,404567587,MDEyOklzc3VlQ29tbWVudDQwNDU2NzU4Nw==,9599,simonw,2018-07-12T16:15:29Z,2018-07-12T16:17:54Z,OWNER,Here's how plotly handled this issue: https://github.com/plotly/plotly.py/pull/203 - see also https://github.com/plotly/plotly.py/blob/213602df6c89b45ce2b811ed2591171c961408e7/plotly/utils.py#L137,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/332#issuecomment-404569003,https://api.github.com/repos/simonw/datasette/issues/332,404569003,MDEyOklzc3VlQ29tbWVudDQwNDU2OTAwMw==,9599,simonw,2018-07-12T16:20:06Z,2018-07-12T16:20:06Z,OWNER,And here's how django-rest-framework did it: https://github.com/encode/django-rest-framework/pull/4918/files,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/332#issuecomment-404574598,https://api.github.com/repos/simonw/datasette/issues/332,404574598,MDEyOklzc3VlQ29tbWVudDQwNDU3NDU5OA==,9599,simonw,2018-07-12T16:39:51Z,2018-07-12T16:39:51Z,OWNER,Since my data is all flat lists of values I don't think I need to customize the JSON encoder itself (no need to deal with nested values). I'll fix the data on its way into the encoder instead. This will also help if I decide to move to uJSON for better performance #48,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/339#issuecomment-404576136,https://api.github.com/repos/simonw/datasette/issues/339,404576136,MDEyOklzc3VlQ29tbWVudDQwNDU3NjEzNg==,12617395,bsilverm,2018-07-12T16:45:08Z,2018-07-12T16:45:08Z,NONE,Thanks for the quick reply. Looks like that is working well.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",340396247,Expose SANIC_RESPONSE_TIMEOUT config option in a sensible way,
https://github.com/simonw/datasette/issues/327#issuecomment-404923318,https://api.github.com/repos/simonw/datasette/issues/327,404923318,MDEyOklzc3VlQ29tbWVudDQwNDkyMzMxOA==,9599,simonw,2018-07-13T18:58:11Z,2018-07-13T18:58:11Z,OWNER,Relevant: https://code.fb.com/data-infrastructure/xars-a-more-efficient-open-source-system-for-self-contained-executables/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",335200136,Explore if SquashFS can be used to shrink size of packaged Docker containers,
https://github.com/simonw/datasette/issues/342#issuecomment-404953877,https://api.github.com/repos/simonw/datasette/issues/342,404953877,MDEyOklzc3VlQ29tbWVudDQwNDk1Mzg3Nw==,9599,simonw,2018-07-13T21:05:12Z,2018-07-13T21:05:12Z,OWNER,That's a good idea. We already do this for tables - e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/most-common-name%2Fsurnames - so having it as an option for canned queries definitely makes sense.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",341123355,Requesting support for query description,
https://github.com/simonw/datasette/issues/342#issuecomment-404954202,https://api.github.com/repos/simonw/datasette/issues/342,404954202,MDEyOklzc3VlQ29tbWVudDQwNDk1NDIwMg==,9599,simonw,2018-07-13T21:06:53Z,2018-07-13T21:07:13Z,OWNER,"https://timezones-api.now.sh/-/metadata currently shows this:
```
{
""databases"": {
""timezones"": {
""license"": ""ODbL"",
""license_url"": ""http://opendatacommons.org/licenses/odbl/"",
""queries"": {
""by_point"": ""select tzid\nfrom\n timezones\nwhere\n within(GeomFromText(\u0027POINT(\u0027 || :longitude || \u0027 \u0027 || :latitude || \u0027)\u0027), timezones.Geometry)\n and rowid in (\n SELECT pkid FROM idx_timezones_Geometry\n where xmin \u003c :longitude\n and xmax \u003e :longitude\n and ymin \u003c :latitude\n and ymax \u003e :latitude\n )""
},
""source"": ""timezone-boundary-builder"",
""source_url"": ""https://github.com/evansiroky/timezone-boundary-builder"",
""tables"": {
""timezones"": {
""license"": ""ODbL"",
""license_url"": ""http://opendatacommons.org/licenses/odbl/"",
""sortable_columns"": [
""tzid""
],
""source"": ""timezone-boundary-builder"",
""source_url"": ""https://github.com/evansiroky/timezone-boundary-builder""
}
}
}
},
""license"": ""ODbL"",
""license_url"": ""http://opendatacommons.org/licenses/odbl/"",
""source"": ""timezone-boundary-builder"",
""source_url"": ""https://github.com/evansiroky/timezone-boundary-builder"",
""title"": ""OpenStreetMap Time Zone Boundaries""
}
```
We could support the value part of the `""queries""` array optionally being a dictionary with the same set of metadata fields supported for a table, plus a new `""sql""` key to hold the SQL for the query.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",341123355,Requesting support for query description,
https://github.com/simonw/datasette/issues/342#issuecomment-404954672,https://api.github.com/repos/simonw/datasette/issues/342,404954672,MDEyOklzc3VlQ29tbWVudDQwNDk1NDY3Mg==,9599,simonw,2018-07-13T21:09:01Z,2018-07-13T21:09:01Z,OWNER,"So it would look like this:
```
{
""databases"": {
""timezones"": {
""license"": ""ODbL"",
""license_url"": ""http://opendatacommons.org/licenses/odbl/"",
""queries"": {
""by_point"": {
""title"": ""Timezones by point"",
""description"": ""Find the timezone for a latitude/longitude point"",
""sql"": ""select tzid\nfrom\n timezones\nwhere\n within(GeomFromText('POINT(' || :longitude || ' ' || :latitude || ')'), timezones.Geometry)\n and rowid in (\n SELECT pkid FROM idx_timezones_Geometry\n where xmin < :longitude\n and xmax > :longitude\n and ymin < :latitude\n and ymax > :latitude\n )""
}
}
}
}
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",341123355,Requesting support for query description,
https://github.com/simonw/datasette/issues/344#issuecomment-405022335,https://api.github.com/repos/simonw/datasette/issues/344,405022335,MDEyOklzc3VlQ29tbWVudDQwNTAyMjMzNQ==,45057,russss,2018-07-14T13:00:48Z,2018-07-14T13:00:48Z,CONTRIBUTOR,"Looks like this was a red herring actually, and heroku had a blip when I was testing it...","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",341229113,datasette publish heroku fails without name provided,
https://github.com/simonw/datasette/pull/345#issuecomment-405025731,https://api.github.com/repos/simonw/datasette/issues/345,405025731,MDEyOklzc3VlQ29tbWVudDQwNTAyNTczMQ==,9599,simonw,2018-07-14T14:04:31Z,2018-07-14T14:04:31Z,OWNER,"Fantastic, we really needed this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",341235633,Allow app names for `datasette publish heroku`,
https://github.com/simonw/datasette/issues/343#issuecomment-405026441,https://api.github.com/repos/simonw/datasette/issues/343,405026441,MDEyOklzc3VlQ29tbWVudDQwNTAyNjQ0MQ==,45057,russss,2018-07-14T14:17:14Z,2018-07-14T14:17:14Z,CONTRIBUTOR,This probably depends on #294.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",341228846,Render boolean fields better by default,
https://github.com/simonw/datasette/issues/294#issuecomment-405026800,https://api.github.com/repos/simonw/datasette/issues/294,405026800,MDEyOklzc3VlQ29tbWVudDQwNTAyNjgwMA==,45057,russss,2018-07-14T14:24:31Z,2018-07-14T14:24:31Z,CONTRIBUTOR,"I had a quick look at this in relation to #343 and I feel like it might be worth modelling the inspected table metadata internally as an object rather than a dict. (We'd still have to serialise it back to JSON.)
There are a few places where we rely on the structure of this metadata dict for various reasons, including in templates (and potentially also in user templates). It would be nice to have a reasonably well defined API for accessing metadata internally so that it's clearer what we're breaking.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327365110,inspect should record column types,
https://github.com/simonw/datasette/issues/342#issuecomment-405138460,https://api.github.com/repos/simonw/datasette/issues/342,405138460,MDEyOklzc3VlQ29tbWVudDQwNTEzODQ2MA==,9599,simonw,2018-07-16T02:42:32Z,2018-07-16T02:42:32Z,OWNER,"Demos:
* https://latest.datasette.io/fixtures/neighborhood_search
* https://timezones-api.now.sh/timezones/by_point
Documentation: http://datasette.readthedocs.io/en/latest/sql_queries.html#canned-queries","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",341123355,Requesting support for query description,
https://github.com/simonw/datasette/issues/332#issuecomment-405968983,https://api.github.com/repos/simonw/datasette/issues/332,405968983,MDEyOklzc3VlQ29tbWVudDQwNTk2ODk4Mw==,9599,simonw,2018-07-18T15:18:57Z,2018-07-18T15:18:57Z,OWNER,Maybe argument should be `?_json_nan=1` since that makes it more explicitly obvious what is going on here.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/308#issuecomment-405971920,https://api.github.com/repos/simonw/datasette/issues/308,405971920,MDEyOklzc3VlQ29tbWVudDQwNTk3MTkyMA==,9599,simonw,2018-07-18T15:27:12Z,2018-07-18T15:27:12Z,OWNER,"It looks like there are a few extra options we should support:
https://devcenter.heroku.com/articles/heroku-cli-commands
```
-t, --team=team team to use
--region=region specify region for the app to run in
--space=space the private space to create the app in
```
Since these differ from the options for Zeit Now I think this means splitting up `datasette publish now` and `datasette publish Heroku` into separate subcommands.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",330826972,"Support extra Heroku apps:create options - region, space, team",
https://github.com/simonw/datasette/issues/333#issuecomment-405975025,https://api.github.com/repos/simonw/datasette/issues/333,405975025,MDEyOklzc3VlQ29tbWVudDQwNTk3NTAyNQ==,9599,simonw,2018-07-18T15:36:11Z,2018-07-18T15:40:04Z,OWNER,"A `force_https_api_urls` config option would work here - if set, Datasette will ignore the incoming protocol and always use https. The `datasette deploy now` command could then add that as an option passed to `datasette serve`.
This is the pattern which is producing incorrect URLs on Zeit Now, because the Sanic `request.url` property is not being correctly set.
https://github.com/simonw/datasette/blob/6e37f091edec35e2706197489f54fff5d890c63c/datasette/views/table.py#L653-L655
Suggested help text:
> Always use https:// for URLs output as part of Datasette API responses","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",338768551,Datasette on Zeit Now returns http URLs for facet and next links,
https://github.com/simonw/datasette/issues/333#issuecomment-405988035,https://api.github.com/repos/simonw/datasette/issues/333,405988035,MDEyOklzc3VlQ29tbWVudDQwNTk4ODAzNQ==,9599,simonw,2018-07-18T16:12:35Z,2018-07-18T16:12:35Z,OWNER,"I'll add a `absolute_url(request, path)` method on the base view class which knows to check the new config option.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",338768551,Datasette on Zeit Now returns http URLs for facet and next links,
https://github.com/simonw/datasette/issues/333#issuecomment-407109113,https://api.github.com/repos/simonw/datasette/issues/333,407109113,MDEyOklzc3VlQ29tbWVudDQwNzEwOTExMw==,9599,simonw,2018-07-23T15:59:02Z,2018-07-23T15:59:02Z,OWNER,I still need to modify `datasette publish now` to set this config option on the instances that it deploys.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",338768551,Datasette on Zeit Now returns http URLs for facet and next links,
https://github.com/simonw/datasette/issues/332#issuecomment-407262311,https://api.github.com/repos/simonw/datasette/issues/332,407262311,MDEyOklzc3VlQ29tbWVudDQwNzI2MjMxMQ==,9599,simonw,2018-07-24T02:43:03Z,2018-07-24T02:43:03Z,OWNER,Actually SQLite doesn't handle NaN at all (it treats it as null) so I'm going to change this ticket to just deal with Infinity and -Infinity.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/332#issuecomment-407262436,https://api.github.com/repos/simonw/datasette/issues/332,407262436,MDEyOklzc3VlQ29tbWVudDQwNzI2MjQzNg==,9599,simonw,2018-07-24T02:43:50Z,2018-07-24T02:43:50Z,OWNER,I'm going with `_json_infinity=1` as the querystring argument.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/332#issuecomment-407262561,https://api.github.com/repos/simonw/datasette/issues/332,407262561,MDEyOklzc3VlQ29tbWVudDQwNzI2MjU2MQ==,9599,simonw,2018-07-24T02:44:39Z,2018-07-24T02:44:39Z,OWNER,According to https://www.mail-archive.com/sqlite-users@mailinglists.sqlite.org/msg110573.html you can insert Infinity/-Infinity in raw SQL (as used by our fixtures) using 1e999 and -1e999.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/332#issuecomment-407267707,https://api.github.com/repos/simonw/datasette/issues/332,407267707,MDEyOklzc3VlQ29tbWVudDQwNzI2NzcwNw==,9599,simonw,2018-07-24T03:20:08Z,2018-07-24T03:20:08Z,OWNER,"Demo:
* https://700d83d.datasette.io/fixtures-dcc1dbf/infinity.json - Infinity converted to Null
* https://700d83d.datasette.io/fixtures-dcc1dbf/infinity.json?_json_infinity=on - invalid JSON containing `Infinity` and `-Infinity`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/332#issuecomment-407267762,https://api.github.com/repos/simonw/datasette/issues/332,407267762,MDEyOklzc3VlQ29tbWVudDQwNzI2Nzc2Mg==,9599,simonw,2018-07-24T03:20:33Z,2018-07-24T03:20:33Z,OWNER,Documentation: http://datasette.readthedocs.io/en/latest/json_api.html#special-json-arguments,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",337141108,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,
https://github.com/simonw/datasette/issues/333#issuecomment-407267966,https://api.github.com/repos/simonw/datasette/issues/333,407267966,MDEyOklzc3VlQ29tbWVudDQwNzI2Nzk2Ng==,9599,simonw,2018-07-24T03:21:42Z,2018-07-24T03:21:42Z,OWNER,Demo: https://700d83d.datasette.io/fixtures-dcc1dbf/facetable.json?_facet=state&_size=5&_labels=on,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",338768551,Datasette on Zeit Now returns http URLs for facet and next links,
https://github.com/simonw/datasette/issues/320#issuecomment-407269243,https://api.github.com/repos/simonw/datasette/issues/320,407269243,MDEyOklzc3VlQ29tbWVudDQwNzI2OTI0Mw==,9599,simonw,2018-07-24T03:30:32Z,2018-07-24T03:30:32Z,OWNER,"* No primary key => no ""object"" option: https://latest.datasette.io/fixtures-dcc1dbf/no_primary_key
* Has a primary key => show ""object"" option: https://latest.datasette.io/fixtures-dcc1dbf/complex_foreign_keys
* Has a next page => has ""stream all rows"" option: https://latest.datasette.io/fixtures-dcc1dbf/no_primary_key
* Has foreign key references = show default-checked ""expand labels"" option: https://latest.datasette.io/fixtures-dcc1dbf/complex_foreign_keys
* Does not have a next page => do not show ""stream all rows"" option: https://latest.datasette.io/fixtures-dcc1dbf/complex_foreign_keys
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",334169932,Need unit tests covering the different states for the advanced export box,
https://github.com/simonw/datasette/issues/298#issuecomment-407274059,https://api.github.com/repos/simonw/datasette/issues/298,407274059,MDEyOklzc3VlQ29tbWVudDQwNzI3NDA1OQ==,9599,simonw,2018-07-24T04:03:05Z,2018-07-24T04:03:05Z,OWNER,Demo: https://latest.datasette.io/fixtures-dcc1dbf?sql=select+%28%27https%3A%2F%2Ftwitter.com%2F%27+%7C%7C+%27simonw%27%29+as+user_url%3B,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327459829,URLify URLs in results from custom SQL statements / views,
https://github.com/simonw/datasette/issues/329#issuecomment-407275996,https://api.github.com/repos/simonw/datasette/issues/329,407275996,MDEyOklzc3VlQ29tbWVudDQwNzI3NTk5Ng==,9599,simonw,2018-07-24T04:18:28Z,2018-07-24T04:18:28Z,OWNER,Hopefully this will do the trick: https://github.com/simonw/datasette/commit/2bdab66772dca51b0c729b4e1063610cb2edd890,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336465018,Travis should push tagged images to Docker Hub for each release,
https://github.com/simonw/datasette/issues/329#issuecomment-407280689,https://api.github.com/repos/simonw/datasette/issues/329,407280689,MDEyOklzc3VlQ29tbWVudDQwNzI4MDY4OQ==,9599,simonw,2018-07-24T04:52:58Z,2018-07-24T04:52:58Z,OWNER,"It almost worked... but I had to fix the `docker login` command: https://github.com/simonw/datasette/commit/3a46d5e3c4278e74c3694f36995ea134bff800bc
Hopefully the next release will be published correctly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336465018,Travis should push tagged images to Docker Hub for each release,
https://github.com/simonw/datasette/issues/336#issuecomment-407450815,https://api.github.com/repos/simonw/datasette/issues/336,407450815,MDEyOklzc3VlQ29tbWVudDQwNzQ1MDgxNQ==,9599,simonw,2018-07-24T15:35:03Z,2018-07-24T15:35:03Z,OWNER,Actually I do like the idea of a unit test that reminds me if I've forgotten to update the included files.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",340039409,Ensure --help examples in docs are always up to date,
https://github.com/simonw/datasette/issues/301#issuecomment-407979065,https://api.github.com/repos/simonw/datasette/issues/301,407979065,MDEyOklzc3VlQ29tbWVudDQwNzk3OTA2NQ==,9599,simonw,2018-07-26T05:17:34Z,2018-07-26T05:17:34Z,OWNER,This code now lives in https://github.com/simonw/datasette/blob/master/datasette/publish/heroku.py,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",328155946,"--spatialite option for ""datasette publish heroku""",
https://github.com/simonw/datasette/issues/217#issuecomment-407980050,https://api.github.com/repos/simonw/datasette/issues/217,407980050,MDEyOklzc3VlQ29tbWVudDQwNzk4MDA1MA==,9599,simonw,2018-07-26T05:24:17Z,2018-07-26T05:24:17Z,OWNER,Documentation: http://datasette.readthedocs.io/en/latest/plugins.html#publish-subcommand-publish,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",314725342,Plugin support for datasette publish,
https://github.com/simonw/datasette/pull/349#issuecomment-407980716,https://api.github.com/repos/simonw/datasette/issues/349,407980716,MDEyOklzc3VlQ29tbWVudDQwNzk4MDcxNg==,9599,simonw,2018-07-26T05:28:54Z,2018-07-26T05:28:54Z,OWNER,"Documentation here: http://datasette.readthedocs.io/en/latest/plugins.html#publish-subcommand-publish
The best way to write a new publish plugin is to check out how the Heroku and Now default plugins are implemented: https://github.com/simonw/datasette/tree/master/datasette/publish","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",344695978,"publish_subcommand hook + default plugins mechanism, used for publish heroku/now",
https://github.com/simonw/datasette/issues/348#issuecomment-407983375,https://api.github.com/repos/simonw/datasette/issues/348,407983375,MDEyOklzc3VlQ29tbWVudDQwNzk4MzM3NQ==,9599,simonw,2018-07-26T05:46:01Z,2018-07-26T05:46:01Z,OWNER,"Oops, forgot to commit those unit tests.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",344656114,"Unit tests for ""datasette publish""",
https://github.com/simonw/datasette/issues/272#issuecomment-408097719,https://api.github.com/repos/simonw/datasette/issues/272,408097719,MDEyOklzc3VlQ29tbWVudDQwODA5NzcxOQ==,9599,simonw,2018-07-26T13:29:38Z,2018-07-26T13:29:38Z,OWNER,It looks like that's a bug in Starlette - filed here: https://github.com/encode/starlette/issues/32,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/272#issuecomment-408093480,https://api.github.com/repos/simonw/datasette/issues/272,408093480,MDEyOklzc3VlQ29tbWVudDQwODA5MzQ4MA==,9599,simonw,2018-07-26T13:15:55Z,2018-07-26T13:46:40Z,OWNER,"I'm now hacking around with an initial version of this in the [starlette branch](https://github.com/simonw/datasette/tree/starlette).
Here's my work in progress, deployed using `datasette publish now fixtures.db -n datasette-starlette-demo --branch=starlette --extra-options=""--asgi""`
https://datasette-starlette-demo.now.sh/
Lots more work to do - the CSS isn't being served correctly for example, it's showing this error when I hit `/-/static/app.css`:
```
INFO: 127.0.0.1 - ""GET /-/static/app.css HTTP/1.1"" 200
ERROR: Exception in ASGI application
Traceback (most recent call last):
File ""/Users/simonw/Dropbox/Development/datasette/venv/lib/python3.6/site-packages/uvicorn/protocols/http/httptools_impl.py"", line 363, in run_asgi
result = await asgi(self.receive, self.send)
File ""/Users/simonw/Dropbox/Development/datasette/venv/lib/python3.6/site-packages/starlette/staticfiles.py"", line 91, in __call__
await response(receive, send)
File ""/Users/simonw/Dropbox/Development/datasette/venv/lib/python3.6/site-packages/starlette/response.py"", line 180, in __call__
{""type"": ""http.response.body"", ""body"": chunk, ""more_body"": False}
File ""/Users/simonw/Dropbox/Development/datasette/venv/lib/python3.6/site-packages/uvicorn/protocols/http/httptools_impl.py"", line 483, in send
raise RuntimeError(""Response content shorter than Content-Length"")
RuntimeError: Response content shorter than Content-Length
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/272#issuecomment-408105251,https://api.github.com/repos/simonw/datasette/issues/272,408105251,MDEyOklzc3VlQ29tbWVudDQwODEwNTI1MQ==,9599,simonw,2018-07-26T13:54:06Z,2018-07-26T13:54:06Z,OWNER,"Tom shipped my fix for that bug already, so https://datasette-starlette-demo.now.sh/ is now serving CSS!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/272#issuecomment-408478935,https://api.github.com/repos/simonw/datasette/issues/272,408478935,MDEyOklzc3VlQ29tbWVudDQwODQ3ODkzNQ==,9599,simonw,2018-07-27T17:00:08Z,2018-07-27T17:00:08Z,OWNER,"Refs
https://github.com/encode/uvicorn/issues/168","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/299#issuecomment-408581551,https://api.github.com/repos/simonw/datasette/issues/299,408581551,MDEyOklzc3VlQ29tbWVudDQwODU4MTU1MQ==,9599,simonw,2018-07-28T04:24:05Z,2018-07-28T04:24:05Z,OWNER,New documentation is now online here: https://datasette.readthedocs.io/en/latest/pages.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",327461381,Documentation covering ALL datasette URLs,
https://github.com/simonw/datasette/issues/259#issuecomment-392214791,https://api.github.com/repos/simonw/datasette/issues/259,392214791,MDEyOklzc3VlQ29tbWVudDM5MjIxNDc5MQ==,9599,simonw,2018-05-25T23:43:15Z,2018-07-29T00:56:03Z,OWNER,"We may need to derive a usable name for each of these relationships that can be used in eg querystring parameters.
The name of the join table is a reasonable choice here. Say the join table is called `event_tags` - the querystring for returning all events that are tagged `badger` could be `/db/events?_m2m_event_tags__tag=badger` perhaps?
But what if `event_tags` has more than one foreign key back to `events`? Might need to specify the column in `events` that is referred back to by `event_tags` somehow in that case.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322787470,inspect() should detect many-to-many relationships,
https://github.com/simonw/datasette/issues/259#issuecomment-409087501,https://api.github.com/repos/simonw/datasette/issues/259,409087501,MDEyOklzc3VlQ29tbWVudDQwOTA4NzUwMQ==,9599,simonw,2018-07-31T04:03:29Z,2018-07-31T04:03:29Z,OWNER,Parent ticket: #354,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",322787470,inspect() should detect many-to-many relationships,
https://github.com/simonw/datasette/issues/355#issuecomment-409087871,https://api.github.com/repos/simonw/datasette/issues/355,409087871,MDEyOklzc3VlQ29tbWVudDQwOTA4Nzg3MQ==,9599,simonw,2018-07-31T04:06:22Z,2018-07-31T04:06:22Z,OWNER,"I started playing with this in the `m2m` branch - work so far: https://github.com/simonw/datasette/compare/295d005ca48747faf046ed30c3c61e7563c61ed2...af4ce463e7518f9d7828b846efd5b528a1905eca
Here's a demo: https://datasette-m2m-work-in-progress.now.sh/russian-ads-e8e09e2/ads?_m2m_ad_targets__target_id=ec3ac&_m2m_ad_targets__target_id=e128e","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",346027040,Table view should support filtering via many-to-many relationships,
https://github.com/simonw/datasette/issues/356#issuecomment-409088967,https://api.github.com/repos/simonw/datasette/issues/356,409088967,MDEyOklzc3VlQ29tbWVudDQwOTA4ODk2Nw==,9599,simonw,2018-07-31T04:14:44Z,2018-07-31T04:14:44Z,OWNER,"Here's the query I'm playing with for facet counts:
https://datasette-m2m-work-in-progress.now.sh/russian-ads-e8e09e2?sql=select+target_id%2C+count%28*%29+as+n+from+ad_targets%0D%0Awhere%0D%0A++target_id+not+in+%28%22ec3ac%22%2C+%22e128e%22%29%0D%0A++and+ad_id+in+%28select+ad_id+from+ad_targets+where+target_id+%3D+%22ec3ac%22%29%0D%0A++and+ad_id+in+%28select+ad_id+from+ad_targets+where+target_id+%3D+%22e128e%22%29%0D%0Agroup+by+target_id+order+by+n+desc%3B
```
select target_id, count(*) as n from ad_targets
where
target_id not in (""ec3ac"", ""e128e"")
and ad_id in (select ad_id from ad_targets where target_id = ""ec3ac"")
and ad_id in (select ad_id from ad_targets where target_id = ""e128e"")
group by target_id order by n desc;
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",346028655,Ability to display facet counts for many-to-many relationships,
https://github.com/simonw/datasette/issues/352#issuecomment-409715112,https://api.github.com/repos/simonw/datasette/issues/352,409715112,MDEyOklzc3VlQ29tbWVudDQwOTcxNTExMg==,9599,simonw,2018-08-01T20:41:04Z,2018-08-01T20:41:04Z,OWNER,The hook is currently only used on the custom SQL results page - it needs to run on table/view pages as well.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",345821500,render_cell(value) plugin hook,
https://github.com/simonw/datasette/issues/352#issuecomment-410485995,https://api.github.com/repos/simonw/datasette/issues/352,410485995,MDEyOklzc3VlQ29tbWVudDQxMDQ4NTk5NQ==,9599,simonw,2018-08-05T00:16:21Z,2018-08-05T00:16:21Z,OWNER,"First plugin using this hook: https://github.com/simonw/datasette-json-html
Hook documentation: http://datasette.readthedocs.io/en/latest/plugins.html#render-cell-value","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",345821500,render_cell(value) plugin hook,
https://github.com/simonw/datasette/issues/352#issuecomment-410580202,https://api.github.com/repos/simonw/datasette/issues/352,410580202,MDEyOklzc3VlQ29tbWVudDQxMDU4MDIwMg==,9599,simonw,2018-08-06T03:39:40Z,2018-08-06T03:39:40Z,OWNER,I used `datasette-json-html` to build this: https://russian-ira-facebook-ads-datasette-whmbonekoj.now.sh/russian-ads-919cbfd/display_ads,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",345821500,render_cell(value) plugin hook,
https://github.com/simonw/datasette/issues/357#issuecomment-410818501,https://api.github.com/repos/simonw/datasette/issues/357,410818501,MDEyOklzc3VlQ29tbWVudDQxMDgxODUwMQ==,9599,simonw,2018-08-06T19:04:54Z,2018-08-06T19:04:54Z,OWNER,Another potential use-case for this hook: loading metadata via a URL,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",348043884,Plugin hook for loading metadata.json,
https://github.com/simonw/datasette/issues/174#issuecomment-412290986,https://api.github.com/repos/simonw/datasette/issues/174,412290986,MDEyOklzc3VlQ29tbWVudDQxMjI5MDk4Ng==,9599,simonw,2018-08-11T17:46:51Z,2018-08-11T17:46:51Z,OWNER,This was fixed in https://github.com/simonw/datasette/commit/89d9fbb91bfc0dd9091b34dbf3cf540ab849cc44,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",281197863,License/Source in footer should inherit from top level,
https://github.com/simonw/datasette/issues/188#issuecomment-412291327,https://api.github.com/repos/simonw/datasette/issues/188,412291327,MDEyOklzc3VlQ29tbWVudDQxMjI5MTMyNw==,9599,simonw,2018-08-11T17:53:17Z,2018-08-11T17:53:17Z,OWNER,"Potential problem: the existing `metadata.json` format looks like this:
```
{
""title"": ""Custom title for your index page"",
""description"": ""Some description text can go here"",
""license"": ""ODbL"",
""license_url"": ""https://opendatacommons.org/licenses/odbl/"",
""databases"": {
""database1"": {
""source"": ""Alternative source"",
""source_url"": ""http://example.com/"",
""tables"": {
""example_table"": {
""description_html"": ""Custom table description"",
""license"": ""CC BY 3.0 US"",
""license_url"": ""https://creativecommons.org/licenses/by/3.0/us/""
}
}
}
}
}
```
This doesn't make sense for metadata that is bundled with a specific database - there's no point in having the `databases` key, instead the content of `database1` in the above example should be at the top level. This also means that if you rename the `*.db` file you won't have to edit its metadata at the same time.
Calling such an embedded file `metadata.json` when the shape is different could be confusing. Maybe call it `database-metadata.json` instead.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",309047460,Ability to bundle metadata and templates inside the SQLite file,
https://github.com/simonw/datasette/issues/231#issuecomment-412291395,https://api.github.com/repos/simonw/datasette/issues/231,412291395,MDEyOklzc3VlQ29tbWVudDQxMjI5MTM5NQ==,9599,simonw,2018-08-11T17:54:41Z,2018-08-11T17:54:41Z,OWNER,"I'm going to separate the issue of enabling and disabling plugins from the existence of the `plugins` key. The format will simply be:
```
{
""plugins"": {
""name-of-plugin"": {
... any structures you like go here, defined by the plugin ...
}
}
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",316323336,metadata.json support for plugin configuration options,
https://github.com/simonw/datasette/issues/238#issuecomment-412291437,https://api.github.com/repos/simonw/datasette/issues/238,412291437,MDEyOklzc3VlQ29tbWVudDQxMjI5MTQzNw==,9599,simonw,2018-08-11T17:55:26Z,2018-08-11T18:02:48Z,OWNER,"On further thought, I'd much rather implement this using some kind of metadata plugin hook - see #357","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",317714268,External metadata.json,
https://github.com/simonw/datasette/issues/185#issuecomment-412299013,https://api.github.com/repos/simonw/datasette/issues/185,412299013,MDEyOklzc3VlQ29tbWVudDQxMjI5OTAxMw==,9599,simonw,2018-08-11T20:14:54Z,2018-08-11T20:14:54Z,OWNER,"I've been worrying about how this one relates to #260 - I'd like to validate metadata (to help protect against people e.g. misspelling `license_url` and then being confused when their license isn't displayed properly), but this issue requests the ability to add arbitrary additional keys to the metadata structure.
I think the solution is to introduce a metadata key called `extra_metadata_keys` which allows you to specifically list the extra keys that you want to enable. Something like this:
```
{
""title"": ""My title"",
""source"": ""Source"",
""source_url"": ""https://www.example.com/"",
""release_date"": ""2018-04-01"",
""extra_metadata_keys"": [""release_date""]
}
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",299760684,Metadata should be a nested arbitrary KV store,
https://github.com/simonw/datasette/issues/359#issuecomment-412356537,https://api.github.com/repos/simonw/datasette/issues/359,412356537,MDEyOklzc3VlQ29tbWVudDQxMjM1NjUzNw==,9599,simonw,2018-08-12T17:01:21Z,2018-08-12T17:01:39Z,OWNER,"Example table: https://latest-code.datasette.io/code/definitions
Here's a query that does facet counting against that column:
https://latest-code.datasette.io/code-a26fa3c?sql=select+count%28*%29+as+n%2C+j.value+from+definitions+join+json_each%28params%29+j+group+by+j.value+order+by+n+desc%3B
```
select count(*) as n, j.value
from definitions join json_each(params) j
group by j.value order by n desc;
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",349827640,Faceted browse against a JSON list of tags,
https://github.com/simonw/datasette/issues/359#issuecomment-412356746,https://api.github.com/repos/simonw/datasette/issues/359,412356746,MDEyOklzc3VlQ29tbWVudDQxMjM1Njc0Ng==,9599,simonw,2018-08-12T17:05:00Z,2018-08-12T17:05:00Z,OWNER,"And here's the query for pulling back every record tagged with a specific tag:
https://latest-code.datasette.io/code-a26fa3c?sql=select+*+from+definitions+where+rowid+in+%28%0D%0A++select+definitions.rowid%0D%0A++from+definitions+join+json_each%28params%29+j%0D%0A++where+j.value+%3D+%3Atag%0D%0A%29&tag=filename
```
select * from definitions where rowid in (
select definitions.rowid
from definitions join json_each(params) j
where j.value = :tag
)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",349827640,Faceted browse against a JSON list of tags,
https://github.com/simonw/datasette/issues/359#issuecomment-412357691,https://api.github.com/repos/simonw/datasette/issues/359,412357691,MDEyOklzc3VlQ29tbWVudDQxMjM1NzY5MQ==,9599,simonw,2018-08-12T17:17:29Z,2018-08-12T17:17:29Z,OWNER,"Note that there doesn't seem to be a way to use indexes (even [indexes on expressions](https://www.sqlite.org/expridx.html)) to speed these up, so this will only ever be effective on smaller data sets, probably in the 10,000-100,000 range. Datasette is often used with smaller data sets so this is still worth pursuing.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",349827640,Faceted browse against a JSON list of tags,
https://github.com/simonw/datasette/issues/185#issuecomment-412663658,https://api.github.com/repos/simonw/datasette/issues/185,412663658,MDEyOklzc3VlQ29tbWVudDQxMjY2MzY1OA==,222245,carlmjohnson,2018-08-13T21:04:11Z,2018-08-13T21:04:11Z,NONE,That seems good to me.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",299760684,Metadata should be a nested arbitrary KV store,
https://github.com/simonw/datasette/issues/360#issuecomment-413386332,https://api.github.com/repos/simonw/datasette/issues/360,413386332,MDEyOklzc3VlQ29tbWVudDQxMzM4NjMzMg==,9599,simonw,2018-08-16T00:51:00Z,2018-08-16T00:51:00Z,OWNER,Relevant: https://github.com/coleifer/pysqlite3/issues/2,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",351017129,Use pysqlite3 if available,
https://github.com/simonw/datasette/issues/360#issuecomment-413387424,https://api.github.com/repos/simonw/datasette/issues/360,413387424,MDEyOklzc3VlQ29tbWVudDQxMzM4NzQyNA==,9599,simonw,2018-08-16T00:57:25Z,2018-08-16T00:57:25Z,OWNER,"I deployed a working demo of this here: https://pysqlite3-datasette.now.sh
I used this command to deploy it:
datasette publish now \
fixtures.db fivethirtyeight.db \
--branch=pysqlite3 \
--install=https://github.com/karlb/pysqlite3/archive/master.zip \
-n pysqlite3-datasette
https://pysqlite3-datasette.now.sh/-/versions confirms version of SQLite is `3.25.0`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",351017129,Use pysqlite3 if available,
https://github.com/simonw/datasette/issues/360#issuecomment-413396812,https://api.github.com/repos/simonw/datasette/issues/360,413396812,MDEyOklzc3VlQ29tbWVudDQxMzM5NjgxMg==,9599,simonw,2018-08-16T01:50:42Z,2018-08-16T01:50:42Z,OWNER,"Now that this has merged into master the command for deploying it can use `--branch=master` instead:
datasette publish now \
fixtures.db fivethirtyeight.db \
--branch=master \
--install=https://github.com/karlb/pysqlite3/archive/master.zip \
-n pysqlite3-datasette
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",351017129,Use pysqlite3 if available,
https://github.com/simonw/datasette/issues/267#issuecomment-414860009,https://api.github.com/repos/simonw/datasette/issues/267,414860009,MDEyOklzc3VlQ29tbWVudDQxNDg2MDAwOQ==,78156,annapowellsmith,2018-08-21T23:57:51Z,2018-08-21T23:57:51Z,NONE,"Looks to me like hashing, redirects and caching were documented as part of https://github.com/simonw/datasette/commit/788a542d3c739da5207db7d1fb91789603cdd336#diff-3021b0e065dce289c34c3b49b3952a07 - so perhaps this can be closed? :tada:","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323716411,"Documentation for URL hashing, redirects and cache policy",
https://github.com/simonw/datasette/issues/350#issuecomment-416659043,https://api.github.com/repos/simonw/datasette/issues/350,416659043,MDEyOklzc3VlQ29tbWVudDQxNjY1OTA0Mw==,9599,simonw,2018-08-28T16:48:19Z,2018-08-28T16:48:19Z,OWNER,Closed in https://github.com/simonw/datasette/commit/0bd41d4cb0a42d7d2baf8b49675418d1482ae39b,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",344701755,Don't list default plugins on /-/plugins,
https://github.com/simonw/datasette/issues/350#issuecomment-416667565,https://api.github.com/repos/simonw/datasette/issues/350,416667565,MDEyOklzc3VlQ29tbWVudDQxNjY2NzU2NQ==,9599,simonw,2018-08-28T17:13:50Z,2018-08-28T17:13:50Z,OWNER,https://b7257a2.datasette.io/-/plugins is now correctly returning an empty list.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",344701755,Don't list default plugins on /-/plugins,
https://github.com/simonw/datasette/issues/362#issuecomment-416727898,https://api.github.com/repos/simonw/datasette/issues/362,416727898,MDEyOklzc3VlQ29tbWVudDQxNjcyNzg5OA==,9599,simonw,2018-08-28T20:24:00Z,2018-08-28T20:24:00Z,OWNER,"Are you talking about these filters here?
I haven't thought much about how those could be made more usable - right now they basically expose all available options, but customizing them for particular use-cases is certainly an interesting potential space.
Could you sketch out a bit more about how your ideal interface here would work?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",352768017,Add option to include/exclude columns in search filters,
https://github.com/simonw/datasette/pull/363#issuecomment-417684877,https://api.github.com/repos/simonw/datasette/issues/363,417684877,MDEyOklzc3VlQ29tbWVudDQxNzY4NDg3Nw==,436032,kevboh,2018-08-31T14:39:45Z,2018-08-31T14:39:45Z,NONE,"It looks like the check passed, not sure why it's showing as running in GH.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",355299310,Search all apps during heroku publish,
https://github.com/simonw/datasette/issues/308#issuecomment-418106781,https://api.github.com/repos/simonw/datasette/issues/308,418106781,MDEyOklzc3VlQ29tbWVudDQxODEwNjc4MQ==,9599,simonw,2018-09-03T12:53:21Z,2018-09-03T12:53:21Z,OWNER,Now that I've split the heroku command out into a separate default plugin this is a much easier thing to add: https://github.com/simonw/datasette/blob/master/datasette/publish/heroku.py,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",330826972,"Support extra Heroku apps:create options - region, space, team",
https://github.com/simonw/datasette/issues/272#issuecomment-418695115,https://api.github.com/repos/simonw/datasette/issues/272,418695115,MDEyOklzc3VlQ29tbWVudDQxODY5NTExNQ==,647359,tomchristie,2018-09-05T11:21:25Z,2018-09-05T11:21:25Z,NONE,"Some notes:
* Starlette just got a bump to 0.3.0 - there's some renamings in there. It's got enough functionality now that you can treat it either as a framework or as a toolkit. Either way the component design is all just *here's an ASGI app* all the way through.
* Uvicorn got a bump to 0.3.3 - Removed some cyclical references that were causing garbage collection to impact performance. Ought to be a decent speed bump.
* Wrt. passing config - Either use a single envvar that points to a config, or use multiple envvars for the config. Uvicorn could get a flag to read a `.env` file, but I don't see ASGI itself having a specific interface there.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/pull/293#issuecomment-420295524,https://api.github.com/repos/simonw/datasette/issues/293,420295524,MDEyOklzc3VlQ29tbWVudDQyMDI5NTUyNA==,11912854,jsancho-gpl,2018-09-11T14:32:45Z,2018-09-11T14:32:45Z,NONE,I close this PR because it's better to use the new one #364 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326987229,Support for external database connectors,
https://github.com/simonw/datasette/issues/329#issuecomment-422821483,https://api.github.com/repos/simonw/datasette/issues/329,422821483,MDEyOklzc3VlQ29tbWVudDQyMjgyMTQ4Mw==,418191,jaywgraves,2018-09-19T14:17:42Z,2018-09-19T14:17:42Z,CONTRIBUTOR,"I'm using the docker image (0.23.2) and notice some differences/bugs between the docs and the published version with canned queries. (submitted a tiny doc fix also)
I was able to build the docker container locally using `master` and I'm using that for now.
Would it be possible to manually push 0.24 to DockerHub until the TravisCI stuff is fixed?
I would like to run this in our Kubernetes cluster but don't want to publish a version in our internal registry if I don't have to.
Thanks!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336465018,Travis should push tagged images to Docker Hub for each release,
https://github.com/simonw/datasette/pull/365#issuecomment-422885014,https://api.github.com/repos/simonw/datasette/issues/365,422885014,MDEyOklzc3VlQ29tbWVudDQyMjg4NTAxNA==,9599,simonw,2018-09-19T17:15:16Z,2018-09-19T17:15:16Z,OWNER,Thanks!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",361764460,fix small doc typo,
https://github.com/simonw/datasette/issues/329#issuecomment-422903031,https://api.github.com/repos/simonw/datasette/issues/329,422903031,MDEyOklzc3VlQ29tbWVudDQyMjkwMzAzMQ==,9599,simonw,2018-09-19T18:07:09Z,2018-09-19T18:07:09Z,OWNER,"The new 0.25 release has been successfully pushed to Docker Hub! https://hub.docker.com/r/datasetteproject/datasette/tags/
One catch: it looks like it didn't update the ""latest"" tag to point at it. Looking into that now.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336465018,Travis should push tagged images to Docker Hub for each release,
https://github.com/simonw/datasette/issues/329#issuecomment-422908130,https://api.github.com/repos/simonw/datasette/issues/329,422908130,MDEyOklzc3VlQ29tbWVudDQyMjkwODEzMA==,9599,simonw,2018-09-19T18:23:02Z,2018-09-19T18:23:02Z,OWNER,"I fixed that by running the following on my laptop:
$ docker pull datasetteproject/datasette:0.25
$ docker tag datasetteproject/datasette:0.25 datasetteproject/datasette:latest
$ docker push datasetteproject/datasette
The `latest` tag now points to the most recent release.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336465018,Travis should push tagged images to Docker Hub for each release,
https://github.com/simonw/datasette/issues/329#issuecomment-422915450,https://api.github.com/repos/simonw/datasette/issues/329,422915450,MDEyOklzc3VlQ29tbWVudDQyMjkxNTQ1MA==,418191,jaywgraves,2018-09-19T18:45:02Z,2018-09-20T10:50:50Z,CONTRIBUTOR,"That works for me. Was able to pull the public image and no errors on my canned query. (~although a small rendering bug. I'll create an issue and if I have time today, a PR to fix~ this turned out to be my error.)
Thanks for the quick response!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336465018,Travis should push tagged images to Docker Hub for each release,
https://github.com/simonw/datasette/issues/292#issuecomment-423543060,https://api.github.com/repos/simonw/datasette/issues/292,423543060,MDEyOklzc3VlQ29tbWVudDQyMzU0MzA2MA==,9599,simonw,2018-09-21T14:06:31Z,2018-09-21T14:09:06Z,OWNER,"I keep on finding new reasons that I want this.
The latest is that I'm playing with the more advanced features of FTS5 - in particular the highlight() function and the ability to sort by rank.
The problem is... in order to do this, I need to properly join against the `_fts` table. Here's an example query:
select
highlight(events_fts, 0, '', ''),
events_fts.rank,
events.*
from events
join events_fts on events.rowid = events_fts.rowid
where events_fts match :search
order by rank
Note that this is a different query from the usual FTS one (which does `where rowid in (select rowid from events_fts...)`) because I need the rank column somewhere I can sort against.
I'd like to be able to use this on the table view page so I can get faceting etc for free, but this is a completely different query from the default. Maybe I need a way to customize the entire query? That feels weird though - why am I not using a view in that case?
Answer: because views can't accept `:search` style parameters. I could use a canned query, but canned queries don't get faceting etc.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",326800219,Mechanism for customizing the SQL used to select specific columns in the table view,
https://github.com/simonw/datasette/issues/328#issuecomment-427261369,https://api.github.com/repos/simonw/datasette/issues/328,427261369,MDEyOklzc3VlQ29tbWVudDQyNzI2MTM2OQ==,13698964,chmaynard,2018-10-05T06:37:06Z,2018-10-05T06:37:06Z,NONE,"```
~ $ docker pull datasetteproject/datasette
~ $ docker run -p 8001:8001 -v `pwd`:/mnt datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db
Usage: datasette -p [OPTIONS] [FILES]...
Error: Invalid value for ""files"": Path ""/mnt/fixtures.db"" does not exist.
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336464733,"Installation instructions, including how to use the docker image",
https://github.com/simonw/datasette/issues/187#issuecomment-427943710,https://api.github.com/repos/simonw/datasette/issues/187,427943710,MDEyOklzc3VlQ29tbWVudDQyNzk0MzcxMA==,1583271,progpow,2018-10-08T18:58:05Z,2018-10-08T18:58:05Z,NONE,"I have same error:
```
Collecting uvloop
Using cached https://files.pythonhosted.org/packages/5c/37/6daa39aac42b2deda6ee77f408bec0419b600e27b89b374b0d440af32b10/uvloop-0.11.2.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File """", line 1, in
File ""C:\Users\sageev\AppData\Local\Temp\pip-install-bq64l8jy\uvloop\setup.py"", line 15, in
raise RuntimeError('uvloop does not support Windows at the moment')
RuntimeError: uvloop does not support Windows at the moment
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",309033998,Windows installation error,
https://github.com/simonw/datasette/issues/366#issuecomment-429737929,https://api.github.com/repos/simonw/datasette/issues/366,429737929,MDEyOklzc3VlQ29tbWVudDQyOTczNzkyOQ==,416374,gfrmin,2018-10-15T07:32:57Z,2018-10-15T07:32:57Z,CONTRIBUTOR,"Very hacky solution is to write now.json file forcing the usage of v1 of Zeit cloud, see https://github.com/slygent/datasette/commit/3ab824793ec6534b6dd87078aa46b11c4fa78ea3
This does work, at least.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",369716228,Default built image size over Zeit Now 100MiB limit,
https://github.com/simonw/datasette/issues/176#issuecomment-431867885,https://api.github.com/repos/simonw/datasette/issues/176,431867885,MDEyOklzc3VlQ29tbWVudDQzMTg2Nzg4NQ==,634572,eads,2018-10-22T15:24:57Z,2018-10-22T15:24:57Z,NONE,"I'd like this as well. It would let me access Datasette-driven projects from GatsbyJS the same way I can access Postgres DBs via Hasura. While I don't see SQLite replacing Postgres for the 50m row datasets I sometimes have to work with, there's a whole class of smaller datasets that are great with Datasette but currently would find another option.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",285168503,Add GraphQL endpoint,
https://github.com/simonw/datasette/issues/366#issuecomment-433680598,https://api.github.com/repos/simonw/datasette/issues/366,433680598,MDEyOklzc3VlQ29tbWVudDQzMzY4MDU5OA==,9599,simonw,2018-10-28T06:38:43Z,2018-10-28T06:38:43Z,OWNER,I've just started running into this as well. Looks like I'll have to anchor to v1 for the moment - I'm hoping the discussion on https://github.com/zeit/now-cli/issues/1523 encourages an increase in this limit policy :/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",369716228,Default built image size over Zeit Now 100MiB limit,
https://github.com/simonw/datasette/issues/371#issuecomment-435767775,https://api.github.com/repos/simonw/datasette/issues/371,435767775,MDEyOklzc3VlQ29tbWVudDQzNTc2Nzc3NQ==,9599,simonw,2018-11-05T06:27:33Z,2018-11-05T06:27:33Z,OWNER,"This would be fantastic - that tutorial looks like many of the details needed for this.
Do you know if Digital Ocean have the ability to provision URLs for a droplet without you needing to buy your own domain name? Heroku have https://example.herokuapp.com/ and Zeit have https://blah.now.sh/ - does Digital Ocean have an equivalent?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",377156339,datasette publish digitalocean plugin,
https://github.com/simonw/datasette/issues/369#issuecomment-435767827,https://api.github.com/repos/simonw/datasette/issues/369,435767827,MDEyOklzc3VlQ29tbWVudDQzNTc2NzgyNw==,9599,simonw,2018-11-05T06:27:55Z,2018-11-05T06:28:48Z,OWNER,"This is a good idea. Basically a version of this bug but on the custom SQL query page:
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",374953006,Interface should show same JSON shape options for custom SQL queries,
https://github.com/simonw/datasette/issues/369#issuecomment-435768450,https://api.github.com/repos/simonw/datasette/issues/369,435768450,MDEyOklzc3VlQ29tbWVudDQzNTc2ODQ1MA==,416374,gfrmin,2018-11-05T06:31:59Z,2018-11-05T06:31:59Z,CONTRIBUTOR,"That would be ideal, but you know better than me whether the CSV streaming trick works for custom SQL queries.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",374953006,Interface should show same JSON shape options for custom SQL queries,
https://github.com/simonw/datasette/issues/329#issuecomment-435772031,https://api.github.com/repos/simonw/datasette/issues/329,435772031,MDEyOklzc3VlQ29tbWVudDQzNTc3MjAzMQ==,9599,simonw,2018-11-05T06:53:28Z,2018-11-05T06:54:10Z,OWNER,"This works now! The `0.25.1` release was the first release which successfully pushed to Docker Hub: https://hub.docker.com/r/datasetteproject/datasette/tags/
Here's the log from the successful Travis release job: https://travis-ci.org/simonw/datasette/jobs/450714602
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336465018,Travis should push tagged images to Docker Hub for each release,
https://github.com/simonw/datasette/issues/371#issuecomment-435862009,https://api.github.com/repos/simonw/datasette/issues/371,435862009,MDEyOklzc3VlQ29tbWVudDQzNTg2MjAwOQ==,82988,psychemedia,2018-11-05T12:48:35Z,2018-11-05T12:48:35Z,CONTRIBUTOR,I think you need to register a domain name you own separately in order to get a non-IP address address? https://www.digitalocean.com/docs/networking/dns/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",377156339,datasette publish digitalocean plugin,
https://github.com/simonw/datasette/issues/370#issuecomment-435974786,https://api.github.com/repos/simonw/datasette/issues/370,435974786,MDEyOklzc3VlQ29tbWVudDQzNTk3NDc4Ng==,9599,simonw,2018-11-05T18:06:56Z,2018-11-05T18:06:56Z,OWNER,"I've been thinking a bit about ways of using Jupyter Notebook more effectively with Datasette (thinks like a `publish_dataframes(df1, df2, df3)` function which publishes some Pandas dataframes and returns you a URL to a new hosted Datasette instance) but you're right, Jupyter Lab is potentially a much more interesting fit.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",377155320,Integration with JupyterLab,
https://github.com/simonw/datasette/issues/374#issuecomment-435976262,https://api.github.com/repos/simonw/datasette/issues/374,435976262,MDEyOklzc3VlQ29tbWVudDQzNTk3NjI2Mg==,9599,simonw,2018-11-05T18:11:10Z,2018-11-05T18:11:10Z,OWNER,"I think there is a useful way forward here though: the image size may be limited to 100MB, but once the instance launches it gets access to a filesystem with a lot more space than that (possibly as much as 15GB given my initial poking around).
So... one potential solution here is to teach Datasette to launch from a smaller image and then download a larger SQLite file from a known URL as part of its initial startup.
Combined with the ability to get Now to always run at least one copy of an instance this could allow Datasette to host much larger SQLite databases on that platform while playing nicely with the Zeit v2 platform.
See also https://github.com/zeit/now-cli/issues/1523","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",377518499,Get Datasette working with Zeit Now v2's 100MB image size limit,
https://github.com/simonw/datasette/issues/370#issuecomment-436037692,https://api.github.com/repos/simonw/datasette/issues/370,436037692,MDEyOklzc3VlQ29tbWVudDQzNjAzNzY5Mg==,82988,psychemedia,2018-11-05T21:15:47Z,2018-11-05T21:18:37Z,CONTRIBUTOR,"In terms of integration with `pandas`, I was pondering two different ways `datasette`/`csvs_to_sqlite` integration may work:
- like [`pandasql`](https://github.com/yhat/pandasql), to provide a SQL query layer either by a direct connection to the sqlite db or via `datasette` API;
- as an improvement of `pandas.to_sql()`, which is a bit ropey (e.g. `pandas.to_sql_from_csvs()`, routing the dataframe to sqlite via `csvs_tosqlite` rather than the dodgy mapping that `pandas` supports).
The `pandas.publish_*` idea could be quite interesting though... Would it be useful/fruitful to think about `publish_` as a complement to [`pandas.to_`](https://pandas.pydata.org/pandas-docs/stable/api.html#id12)?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",377155320,Integration with JupyterLab,
https://github.com/simonw/datasette/issues/370#issuecomment-436042445,https://api.github.com/repos/simonw/datasette/issues/370,436042445,MDEyOklzc3VlQ29tbWVudDQzNjA0MjQ0NQ==,82988,psychemedia,2018-11-05T21:30:42Z,2018-11-05T21:31:48Z,CONTRIBUTOR,"Another route would be something like creating a `datasette` IPython magic for notebooks to take a dataframe and easily render it as a `datasette`. You'd need to run the app in the background rather than block execution in the notebook. Related to that, or to publishing a dataframe in notebook cell for use in other cells in a non-blocking way, there may be cribs in something like https://github.com/micahscopes/nbmultitask .","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",377155320,Integration with JupyterLab,
https://github.com/simonw/datasette/issues/227#issuecomment-439194286,https://api.github.com/repos/simonw/datasette/issues/227,439194286,MDEyOklzc3VlQ29tbWVudDQzOTE5NDI4Ng==,222245,carlmjohnson,2018-11-15T21:20:37Z,2018-11-15T21:20:37Z,NONE,I'm diving back into https://salaries.news.baltimoresun.com and what I really want is the ability to inject the request into my context.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",315960272,prepare_context() plugin hook,
https://github.com/simonw/datasette/issues/120#issuecomment-439421164,https://api.github.com/repos/simonw/datasette/issues/120,439421164,MDEyOklzc3VlQ29tbWVudDQzOTQyMTE2NA==,36796532,ad-si,2018-11-16T15:05:18Z,2018-11-16T15:05:18Z,NONE,This would be an awesome feature ❤️ ,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275087397,Plugin that adds an authentication layer of some sort,
https://github.com/simonw/datasette/issues/374#issuecomment-439762759,https://api.github.com/repos/simonw/datasette/issues/374,439762759,MDEyOklzc3VlQ29tbWVudDQzOTc2Mjc1OQ==,9599,simonw,2018-11-19T03:41:36Z,2018-11-19T03:41:36Z,OWNER,"It turned out Zeit didn't end up shipping the new 100MB-limit Docker-based Zeit 2.0 after all - they ended up going in a completely different direction, towards lambdas instead (which don't really fit the Datasette model): https://zeit.co/blog/now-2
But... as far as I can tell they have introduced the 100MB image size for all free Zeit accounts ever against their 1.0 platform. So we still need to solve this, or free Zeit users won't be able to use `datasette publish now` even while 1.0 is still available.
I made some notes on this here: https://simonwillison.net/2018/Nov/19/smaller-python-docker-images/
I've got it working for the Datasette Publish webapp, but I still need to fix `datasette publish now` to create much smaller patterns.
I know how to do this for regular datasette, but I haven't yet figured out an Alpine Linux pattern for spatialite extras:
https://github.com/simonw/datasette/blob/5e3a432a0caa23837fa58134f69e2f82e4f632a6/datasette/utils.py#L287-L300","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",377518499,Get Datasette working with Zeit Now v2's 100MB image size limit,
https://github.com/simonw/datasette/issues/374#issuecomment-439763196,https://api.github.com/repos/simonw/datasette/issues/374,439763196,MDEyOklzc3VlQ29tbWVudDQzOTc2MzE5Ng==,9599,simonw,2018-11-19T03:45:13Z,2018-11-19T03:45:13Z,OWNER,This looks like it might be a recipe for spatialite Python on Alpine Linux: https://github.com/bentrm/geopython/blob/8e52062d9545f4b7c1f04a3516354a5a9155e31f/Dockerfile,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",377518499,Get Datasette working with Zeit Now v2's 100MB image size limit,
https://github.com/simonw/datasette/issues/374#issuecomment-439763268,https://api.github.com/repos/simonw/datasette/issues/374,439763268,MDEyOklzc3VlQ29tbWVudDQzOTc2MzI2OA==,9599,simonw,2018-11-19T03:45:44Z,2018-11-19T03:45:44Z,OWNER,Another example that might be useful: https://github.com/poc-flask/alpine/blob/8e9f48a2351e106347dab36d08cf21dee865993e/Dockerfile,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",377518499,Get Datasette working with Zeit Now v2's 100MB image size limit,
https://github.com/simonw/datasette/pull/389#issuecomment-440128762,https://api.github.com/repos/simonw/datasette/issues/389,440128762,MDEyOklzc3VlQ29tbWVudDQ0MDEyODc2Mg==,9599,simonw,2018-11-20T03:52:11Z,2018-11-20T03:52:11Z,OWNER,"The problem is Sanic. Here's the error I'm getting:
```
(venv) datasette $ pytest -x
============================================================= test session starts ==============================================================
platform darwin -- Python 3.7.1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /Users/simonw/Dropbox/Development/datasette, inifile:
collected 258 items
tests/test_api.py ...................F
=================================================================== FAILURES ===================================================================
_______________________________________________________ test_table_with_slashes_in_name ________________________________________________________
app_client =
def test_table_with_slashes_in_name(app_client):
response = app_client.get('/fixtures/table%2Fwith%2Fslashes.csv?_shape=objects&_format=json')
> assert response.status == 200
E AssertionError: assert 404 == 200
```
That's because something about how Sanic handles escape characters in URLs changed between 0.7.0 and 0.8.3.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",382471625,Bump dependency versions,
https://github.com/simonw/datasette/pull/390#issuecomment-447677798,https://api.github.com/repos/simonw/datasette/issues/390,447677798,MDEyOklzc3VlQ29tbWVudDQ0NzY3Nzc5OA==,9599,simonw,2018-12-16T21:32:45Z,2018-12-16T21:32:45Z,OWNER,Thanks for spotting this!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",386459810,tiny typo in customization docs,
https://github.com/simonw/datasette/issues/374#issuecomment-448437245,https://api.github.com/repos/simonw/datasette/issues/374,448437245,MDEyOklzc3VlQ29tbWVudDQ0ODQzNzI0NQ==,9599,simonw,2018-12-19T01:35:59Z,2018-12-19T01:35:59Z,OWNER,"Closing this as Zeit went on a different direction with Now v2, so the 100MB limit is no longer a concern.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",377518499,Get Datasette working with Zeit Now v2's 100MB image size limit,
https://github.com/simonw/datasette/issues/393#issuecomment-450943172,https://api.github.com/repos/simonw/datasette/issues/393,450943172,MDEyOklzc3VlQ29tbWVudDQ1MDk0MzE3Mg==,9599,simonw,2019-01-02T18:28:43Z,2019-01-02T18:28:43Z,OWNER,"Definitely a bug, thanks.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",395236066,"CSV export in ""Advanced export"" pane doesn't respect query",
https://github.com/simonw/datasette/issues/393#issuecomment-450943632,https://api.github.com/repos/simonw/datasette/issues/393,450943632,MDEyOklzc3VlQ29tbWVudDQ1MDk0MzYzMg==,9599,simonw,2019-01-02T18:30:20Z,2019-01-02T18:30:20Z,OWNER,"This is the code which is meant to add those options as hidden form fields:
https://github.com/simonw/datasette/blob/fe5b6ea95a973534fe8a44907c0ea2449aae7602/datasette/templates/table.html#L150-L155
It's clearly not working. Need to fix this and add a corresponding unit test.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",395236066,"CSV export in ""Advanced export"" pane doesn't respect query",
https://github.com/simonw/datasette/issues/393#issuecomment-450944166,https://api.github.com/repos/simonw/datasette/issues/393,450944166,MDEyOklzc3VlQ29tbWVudDQ1MDk0NDE2Ng==,9599,simonw,2019-01-02T18:32:12Z,2019-01-02T18:32:12Z,OWNER,"Here's the test that needs updating:
https://github.com/simonw/datasette/blob/8b8ae55e7c8b9e1dceef53f55a330b596ca44d41/tests/test_html.py#L427-L435","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",395236066,"CSV export in ""Advanced export"" pane doesn't respect query",
https://github.com/simonw/datasette/issues/391#issuecomment-450964512,https://api.github.com/repos/simonw/datasette/issues/391,450964512,MDEyOklzc3VlQ29tbWVudDQ1MDk2NDUxMg==,9599,simonw,2019-01-02T19:45:12Z,2019-01-02T19:45:12Z,OWNER,"Thanks, I've fixed this. I had to re-alias it against now:
```
~ $ now alias google-trends-pnwhfwvgqf.now.sh https://google-trends.datasettes.com/
> Assigning alias google-trends.datasettes.com to deployment google-trends-pnwhfwvgqf.now.sh
> Certificate for google-trends.datasettes.com (cert_uXaADIuNooHS3tZ) created [18s]
> Success! google-trends.datasettes.com now points to google-trends-pnwhfwvgqf.now.sh [20s]
```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",392610803,Google Trends example doesn’t work,
https://github.com/simonw/datasette/issues/393#issuecomment-451046123,https://api.github.com/repos/simonw/datasette/issues/393,451046123,MDEyOklzc3VlQ29tbWVudDQ1MTA0NjEyMw==,9599,simonw,2019-01-03T03:05:07Z,2019-01-03T03:05:07Z,OWNER,The fix was released as part of Datasette 0.26 - you can see the fix working here: https://v0-26.datasette.io/fixtures-dd88475/facetable?_facet=planet_int&planet_int=1#export,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",395236066,"CSV export in ""Advanced export"" pane doesn't respect query",
https://github.com/simonw/datasette/issues/393#issuecomment-451047426,https://api.github.com/repos/simonw/datasette/issues/393,451047426,MDEyOklzc3VlQ29tbWVudDQ1MTA0NzQyNg==,9599,simonw,2019-01-03T03:19:04Z,2019-01-03T03:19:04Z,OWNER,https://fivethirtyeight.datasettes.com/-/versions is now running 0.26 - so your initial bug demo is now fixed: https://fivethirtyeight.datasettes.com/fivethirtyeight-c300360/classic-rock%2Fclassic-rock-song-list?Release+Year__exact=1989#export,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",395236066,"CSV export in ""Advanced export"" pane doesn't respect query",
https://github.com/simonw/datasette/issues/393#issuecomment-451415063,https://api.github.com/repos/simonw/datasette/issues/393,451415063,MDEyOklzc3VlQ29tbWVudDQ1MTQxNTA2Mw==,1727065,ltrgoddard,2019-01-04T11:04:08Z,2019-01-04T11:04:08Z,NONE,Awesome - will get myself up and running on 0.26,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",395236066,"CSV export in ""Advanced export"" pane doesn't respect query",
https://github.com/simonw/datasette/issues/394#issuecomment-451704724,https://api.github.com/repos/simonw/datasette/issues/394,451704724,MDEyOklzc3VlQ29tbWVudDQ1MTcwNDcyNA==,9599,simonw,2019-01-06T00:32:23Z,2019-01-06T00:33:44Z,OWNER,"I found a really nice pattern for writing the unit tests for this (though it would look even nicer with a solution to #395)
```python
@pytest.mark.parametrize(""prefix"", [""/prefix/"", ""https://example.com/""])
@pytest.mark.parametrize(""path"", [
""/"",
""/fixtures"",
""/fixtures/compound_three_primary_keys"",
""/fixtures/compound_three_primary_keys/a,a,a"",
""/fixtures/paginated_view"",
])
def test_url_prefix_config(prefix, path):
for client in make_app_client(config={
""url_prefix"": prefix,
}):
response = client.get(path)
soup = Soup(response.body, ""html.parser"")
for a in soup.findAll(""a""):
href = a[""href""]
if href not in {
""https://github.com/simonw/datasette"",
""https://github.com/simonw/datasette/blob/master/LICENSE"",
""https://github.com/simonw/datasette/blob/master/tests/fixtures.py"",
}:
assert href.startswith(prefix), (href, a.parent)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",396212021,base_url configuration setting,
https://github.com/simonw/datasette/issues/397#issuecomment-453251589,https://api.github.com/repos/simonw/datasette/issues/397,453251589,MDEyOklzc3VlQ29tbWVudDQ1MzI1MTU4OQ==,9599,simonw,2019-01-10T20:59:42Z,2019-01-10T20:59:42Z,OWNER,"What version of SQLite are you seeing in Datasette? You can tell by hitting http://localhost:8001/-/versions - e.g. here: https://latest.datasette.io/-/versions
My best guess is that your Python SQLite module is running an older version that doesn't support window functions.
One way you can fix that is with the `pysqlite3` module - try running this in your virtual environment:
pip install git+git://github.com/karlb/pysqlite3
That's using a fork of the official module that embeds a full recent SQLite. See this issue thread for more details: https://github.com/coleifer/pysqlite3/issues/2","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",397129564,Update official datasetteproject/datasette Docker container to SQLite 3.26.0,
https://github.com/simonw/datasette/issues/397#issuecomment-453252024,https://api.github.com/repos/simonw/datasette/issues/397,453252024,MDEyOklzc3VlQ29tbWVudDQ1MzI1MjAyNA==,9599,simonw,2019-01-10T21:00:57Z,2019-01-10T21:00:57Z,OWNER,"Oh I just saw you're using the official Datasette docker package - yeah, that's not bundled with a recent SQLite at the moment. We should update that:
https://github.com/simonw/datasette/blob/5b026115126bedbb66457767e169139146d1c9fd/Dockerfile#L9-L11","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",397129564,Update official datasetteproject/datasette Docker container to SQLite 3.26.0,
https://github.com/simonw/datasette/issues/271#issuecomment-453262703,https://api.github.com/repos/simonw/datasette/issues/271,453262703,MDEyOklzc3VlQ29tbWVudDQ1MzI2MjcwMw==,9599,simonw,2019-01-10T21:35:18Z,2019-01-10T21:35:18Z,OWNER,It turns out this was much easier to support than I expected: https://github.com/simonw/datasette/commit/eac08f0dfc61a99e8887442fc247656d419c76f8,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324162476,Mechanism for automatically picking up changes when on-disk .db file changes,
https://github.com/simonw/datasette/issues/396#issuecomment-453324601,https://api.github.com/repos/simonw/datasette/issues/396,453324601,MDEyOklzc3VlQ29tbWVudDQ1MzMyNDYwMQ==,9599,simonw,2019-01-11T00:55:21Z,2019-01-11T00:55:21Z,OWNER,Demo: https://latest.datasette.io/-/versions,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",397098882,Add pragma compile_options output to /-/versions,
https://github.com/simonw/datasette/issues/397#issuecomment-453330680,https://api.github.com/repos/simonw/datasette/issues/397,453330680,MDEyOklzc3VlQ29tbWVudDQ1MzMzMDY4MA==,9599,simonw,2019-01-11T01:17:11Z,2019-01-11T01:25:33Z,OWNER,"If you pull [the latest image](https://hub.docker.com/r/datasetteproject/datasette) you should get the right SQLite version now:
docker pull datasetteproject/datasette
docker run -p 8001:8001 \
datasetteproject/datasette \
datasette -p 8001 -h 0.0.0.0
http://0.0.0.0:8001/-/versions now gives me:
```
""version"": ""3.26.0""
```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",397129564,Update official datasetteproject/datasette Docker container to SQLite 3.26.0,
https://github.com/simonw/datasette/issues/400#issuecomment-453795040,https://api.github.com/repos/simonw/datasette/issues/400,453795040,MDEyOklzc3VlQ29tbWVudDQ1Mzc5NTA0MA==,9599,simonw,2019-01-13T01:46:30Z,2019-01-13T01:46:30Z,OWNER,I'm really excited about this - it looks like it could be a great plugin.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",398559195,datasette publish cloudrun plugin,
https://github.com/simonw/datasette/issues/399#issuecomment-453874429,https://api.github.com/repos/simonw/datasette/issues/399,453874429,MDEyOklzc3VlQ29tbWVudDQ1Mzg3NDQyOQ==,9599,simonw,2019-01-13T23:09:09Z,2019-01-13T23:09:09Z,OWNER,"It looks like there are two reasons for this:
- The `.git` directory was listed in `.dockerignore` so it wasn't being copied into the build process
- The docker build stage wasn't installing the `git` executable, so it couldn't read the current version
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",398089089,/-/versions for official Docker image returns wrong Datasette version,
https://github.com/simonw/datasette/issues/399#issuecomment-453876023,https://api.github.com/repos/simonw/datasette/issues/399,453876023,MDEyOklzc3VlQ29tbWVudDQ1Mzg3NjAyMw==,9599,simonw,2019-01-13T23:31:59Z,2019-01-13T23:31:59Z,OWNER,"```
docker pull datasetteproject/datasette
docker run -p 8001:8001 datasetteproject/datasette datasette -p 8001 -h 0.0.0.0
```
http://0.0.0.0:8001/-/versions now returns:
```
{
""datasette"": {
""version"": ""0.26.2+0.ga418c8b.dirty""
},
```
I'm not sure why it's showing `.dirty` there.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",398089089,/-/versions for official Docker image returns wrong Datasette version,
https://github.com/simonw/datasette/issues/402#issuecomment-455223551,https://api.github.com/repos/simonw/datasette/issues/402,455223551,MDEyOklzc3VlQ29tbWVudDQ1NTIyMzU1MQ==,9599,simonw,2019-01-17T15:55:06Z,2019-01-17T15:55:06Z,OWNER,"It's new in SQLite 3.26.0 so I will need to figure out how to only apply it in that version or higher.
https://sqlite.org/releaselog/3_26_0.html","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",400340905,Use SQLITE_DBCONFIG_DEFENSIVE plus other recommendations from SQLite security docs,
https://github.com/simonw/datasette/issues/402#issuecomment-455224327,https://api.github.com/repos/simonw/datasette/issues/402,455224327,MDEyOklzc3VlQ29tbWVudDQ1NTIyNDMyNw==,9599,simonw,2019-01-17T15:56:57Z,2019-01-17T15:56:57Z,OWNER,https://sqlite.org/security.html has other recommmendations for apps that accept SQLite files from untrusted sources that we should apply.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",400340905,Use SQLITE_DBCONFIG_DEFENSIVE plus other recommendations from SQLite security docs,
https://github.com/simonw/datasette/issues/401#issuecomment-455230501,https://api.github.com/repos/simonw/datasette/issues/401,455230501,MDEyOklzc3VlQ29tbWVudDQ1NTIzMDUwMQ==,9599,simonw,2019-01-17T16:12:59Z,2019-01-17T16:12:59Z,OWNER,"Datasette-cluster-map doesn't use the new plugin configuration mechanism yet - it really should!
The best example of how to use this mechanism right now is embedded in the Datasette unit tests:
https://github.com/simonw/datasette/blob/b7257a21bf3dfa7353980f343c83a616da44daa7/tests/fixtures.py#L266-L270
https://github.com/simonw/datasette/blob/b7257a21bf3dfa7353980f343c83a616da44daa7/tests/test_plugins.py#L139-L145","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",400229984,How to pass configuration to plugins?,
https://github.com/simonw/datasette/issues/402#issuecomment-455231411,https://api.github.com/repos/simonw/datasette/issues/402,455231411,MDEyOklzc3VlQ29tbWVudDQ1NTIzMTQxMQ==,9599,simonw,2019-01-17T16:15:21Z,2019-01-17T16:15:21Z,OWNER,Unfortunately it looks like there isn't currently a mechanism in the Python sqlite3 library for setting configuration flags like SQLITE_DBCONFIG_DEFENSIVE,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",400340905,Use SQLITE_DBCONFIG_DEFENSIVE plus other recommendations from SQLite security docs,
https://github.com/simonw/datasette/issues/401#issuecomment-455445069,https://api.github.com/repos/simonw/datasette/issues/401,455445069,MDEyOklzc3VlQ29tbWVudDQ1NTQ0NTA2OQ==,9599,simonw,2019-01-18T06:49:07Z,2019-01-18T06:49:07Z,OWNER,I've released a new version of the datasette-cluster-map plugin to illustrate how plugin configuration can work: https://github.com/simonw/datasette-cluster-map/commit/fcc86c450e3df3e6b81c41f31df458923181527a,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",400229984,How to pass configuration to plugins?,
https://github.com/simonw/datasette/issues/403#issuecomment-455445392,https://api.github.com/repos/simonw/datasette/issues/403,455445392,MDEyOklzc3VlQ29tbWVudDQ1NTQ0NTM5Mg==,9599,simonw,2019-01-18T06:51:14Z,2019-01-18T06:51:14Z,OWNER,"I talk about that a bit here: https://simonwillison.net/2018/Oct/4/datasette-ideas/#Bundling_the_data_with_the_code
One of the key ideas behind Datasette is that if your data is read-only you can package it up with the rest of your code - so the normal limitations that apply with hosting services like now.sh no longer prevent you from including a database. The SQLite database is just another static binary file that gets packaged up as part of your deployment.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",400511206,How does persistence work?,
https://github.com/simonw/datasette/issues/401#issuecomment-455520561,https://api.github.com/repos/simonw/datasette/issues/401,455520561,MDEyOklzc3VlQ29tbWVudDQ1NTUyMDU2MQ==,1055831,dazzag24,2019-01-18T11:48:13Z,2019-01-18T11:48:13Z,NONE,"Thanks. I'll take a look at your changes.
I must admit I was struggling to see how to pass info from the python code in __init__.py into the javascript document.addEventListener function.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",400229984,How to pass configuration to plugins?,
https://github.com/simonw/datasette/issues/403#issuecomment-455752238,https://api.github.com/repos/simonw/datasette/issues/403,455752238,MDEyOklzc3VlQ29tbWVudDQ1NTc1MjIzOA==,1794527,ccorcos,2019-01-19T05:47:55Z,2019-01-19T05:47:55Z,NONE,Ah. That makes much more sense. Interesting approach.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",400511206,How does persistence work?,
https://github.com/simonw/datasette/issues/405#issuecomment-457975075,https://api.github.com/repos/simonw/datasette/issues/405,457975075,MDEyOklzc3VlQ29tbWVudDQ1Nzk3NTA3NQ==,9599,simonw,2019-01-28T01:41:51Z,2019-01-28T01:41:51Z,OWNER,Implemented in https://github.com/simonw/datasette/commit/b5dd83981a7dbff571284d4d90a950c740245b05,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",403617881,.json?_nl=on option for exporting newline-delimited JSON,
https://github.com/simonw/datasette/issues/405#issuecomment-457975857,https://api.github.com/repos/simonw/datasette/issues/405,457975857,MDEyOklzc3VlQ29tbWVudDQ1Nzk3NTg1Nw==,9599,simonw,2019-01-28T01:48:37Z,2019-01-28T01:49:00Z,OWNER,"Demo: https://latest.datasette.io/fixtures-dd88475/facetable.json?_shape=array&_nl=on
Also https://b5dd839.datasette.io/fixtures-dd88475/facetable.json?_shape=array&_nl=on","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",403617881,.json?_nl=on option for exporting newline-delimited JSON,
https://github.com/simonw/datasette/pull/404#issuecomment-457976864,https://api.github.com/repos/simonw/datasette/issues/404,457976864,MDEyOklzc3VlQ29tbWVudDQ1Nzk3Njg2NA==,9599,simonw,2019-01-28T01:56:55Z,2019-01-28T01:56:55Z,OWNER,"This failed in Python 3.5:
```
File ""/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/jinja2/environment.py"", line 1020, in render_async
raise NotImplementedError('This feature is not available for this '
NotImplementedError: This feature is not available for this version of Python
```
It looks like this is caused by this feature detection code:
https://github.com/pallets/jinja/blob/a7ba0b637805c53d442e975e3864d3ea38d8743f/jinja2/utils.py#L633-L638","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",403499298,Experiment: run Jinja in async mode,
https://github.com/simonw/sqlite-utils/issues/6#issuecomment-457978729,https://api.github.com/repos/simonw/sqlite-utils/issues/6,457978729,MDEyOklzc3VlQ29tbWVudDQ1Nzk3ODcyOQ==,9599,simonw,2019-01-28T02:12:19Z,2019-01-28T02:12:19Z,OWNER,Will need to solve #7 for this to become truly efficient.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",403624090,"""sqlite-utils insert"" should support newline-delimited JSON",
https://github.com/simonw/sqlite-utils/issues/7#issuecomment-457980966,https://api.github.com/repos/simonw/sqlite-utils/issues/7,457980966,MDEyOklzc3VlQ29tbWVudDQ1Nzk4MDk2Ng==,9599,simonw,2019-01-28T02:29:32Z,2019-01-28T02:29:32Z,OWNER,"Remember to remove this TODO (and turn the `[]` into `()` on this line) as part of this task:
https://github.com/simonw/sqlite-utils/blob/5309c5c7755818323a0f5353bad0de98ecc866be/sqlite_utils/cli.py#L78-L80","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",403625674,.insert_all() should accept a generator and process it efficiently,
https://github.com/simonw/sqlite-utils/issues/7#issuecomment-458011885,https://api.github.com/repos/simonw/sqlite-utils/issues/7,458011885,MDEyOklzc3VlQ29tbWVudDQ1ODAxMTg4NQ==,9599,simonw,2019-01-28T06:25:48Z,2019-01-28T06:25:48Z,OWNER,Re-opening for the second bit involving the cli tool.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",403625674,.insert_all() should accept a generator and process it efficiently,
https://github.com/simonw/sqlite-utils/issues/7#issuecomment-458011906,https://api.github.com/repos/simonw/sqlite-utils/issues/7,458011906,MDEyOklzc3VlQ29tbWVudDQ1ODAxMTkwNg==,9599,simonw,2019-01-28T06:25:55Z,2019-01-28T06:25:55Z,OWNER,"I tested this with a script called `churn_em_out.py`
```
i = 0
while True:
i += 1
print(
'{""id"": I, ""another"": ""row"", ""number"": J}'.replace(""I"", str(i)).replace(
""J"", str(i + 1)
)
)
```
Then I ran this:
```
python churn_em_out.py | \
sqlite-utils insert /tmp/getbig.db stats - \
--nl --batch-size=10000
```
And used `watch 'ls -lah /tmp/getbig.db'` to watch the file growing as it had 10,000 lines of junk committed in batches. The memory used by the process never grew about around 50MB.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",403625674,.insert_all() should accept a generator and process it efficiently,
https://github.com/simonw/datasette/issues/160#issuecomment-459915995,https://api.github.com/repos/simonw/datasette/issues/160,459915995,MDEyOklzc3VlQ29tbWVudDQ1OTkxNTk5NQ==,82988,psychemedia,2019-02-02T00:43:16Z,2019-02-02T00:58:20Z,CONTRIBUTOR,"Do you have any simple working examples of how to use `--static`? Inspection of default served files suggests locations such as `http://example.com/-/static/app.css?0e06ee`.
If `datasette` is being proxied to `http://example.com/foo/datasette`, what form should arguments to `--static` take so that static files are correctly referenced?
Use case is here: https://github.com/psychemedia/jupyterserverproxy-datasette-demo Trying to do a really simple `datasette` demo in MyBinder using jupyter-server-proxy.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",278208011,Ability to bundle and serve additional static files,
https://github.com/simonw/datasette/pull/407#issuecomment-460897973,https://api.github.com/repos/simonw/datasette/issues/407,460897973,MDEyOklzc3VlQ29tbWVudDQ2MDg5Nzk3Mw==,9599,simonw,2019-02-06T04:31:30Z,2019-02-06T04:31:30Z,OWNER,This helped my figure out what to do: https://github.com/heroku/heroku-builds/issues/36,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",407073223,Heroku --include-vcs-ignore,
https://github.com/simonw/datasette/issues/398#issuecomment-460901857,https://api.github.com/repos/simonw/datasette/issues/398,460901857,MDEyOklzc3VlQ29tbWVudDQ2MDkwMTg1Nw==,9599,simonw,2019-02-06T05:01:19Z,2019-02-06T05:01:19Z,OWNER,"I'd really like to use the content-length header here, but Sanic hasn't yet fixed the bug I filed about it: https://github.com/huge-success/sanic/issues/1194","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",398011658,Ensure downloading a 100+MB SQLite database file works,
https://github.com/simonw/datasette/issues/172#issuecomment-460902824,https://api.github.com/repos/simonw/datasette/issues/172,460902824,MDEyOklzc3VlQ29tbWVudDQ2MDkwMjgyNA==,9599,simonw,2019-02-06T05:09:05Z,2019-02-06T05:09:05Z,OWNER,"Demo: https://latest.datasette.io/fixtures-dd88475
![]()
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",280896290,Show size of .db file next to download link,
https://github.com/simonw/datasette/issues/187#issuecomment-463917744,https://api.github.com/repos/simonw/datasette/issues/187,463917744,MDEyOklzc3VlQ29tbWVudDQ2MzkxNzc0NA==,4190962,phoenixjun,2019-02-15T05:58:44Z,2019-02-15T05:58:44Z,NONE,is this supported or not? you can comment if it is not supported so that people like me can stop trying.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",309033998,Windows installation error,
https://github.com/simonw/sqlite-utils/issues/8#issuecomment-464341721,https://api.github.com/repos/simonw/sqlite-utils/issues/8,464341721,MDEyOklzc3VlQ29tbWVudDQ2NDM0MTcyMQ==,82988,psychemedia,2019-02-16T12:08:41Z,2019-02-16T12:08:41Z,NONE,We also get an error if a column name contains a `.`,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",403922644,Problems handling column names containing spaces or - ,
https://github.com/simonw/datasette/issues/187#issuecomment-466325528,https://api.github.com/repos/simonw/datasette/issues/187,466325528,MDEyOklzc3VlQ29tbWVudDQ2NjMyNTUyOA==,2892252,fkuhn,2019-02-22T09:03:50Z,2019-02-22T09:03:50Z,NONE,"I ran into the same issue when trying to install datasette on windows after successfully using it on linux. Unfortunately, there has not been any progress in implementing uvloop for windows - so I recommend not to use it on win. You can read about this issue here:
[https://github.com/MagicStack/uvloop/issues/14](url)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",309033998,Windows installation error,
https://github.com/simonw/sqlite-utils/issues/8#issuecomment-466695500,https://api.github.com/repos/simonw/sqlite-utils/issues/8,466695500,MDEyOklzc3VlQ29tbWVudDQ2NjY5NTUwMA==,9599,simonw,2019-02-23T21:09:03Z,2019-02-23T21:09:03Z,OWNER,"Fixed in https://github.com/simonw/sqlite-utils/commit/228d595f7d10994f34e948888093c2cd290267c4
![]()
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",403922644,Problems handling column names containing spaces or - ,
https://github.com/simonw/sqlite-utils/issues/11#issuecomment-466695672,https://api.github.com/repos/simonw/sqlite-utils/issues/11,466695672,MDEyOklzc3VlQ29tbWVudDQ2NjY5NTY3Mg==,9599,simonw,2019-02-23T21:10:23Z,2019-02-23T21:10:23Z,OWNER,"Rough sketch:
```
+try:
+ import numpy
+except ImportError:
+ numpy = None
+
Column = namedtuple(
""Column"", (""cid"", ""name"", ""type"", ""notnull"", ""default_value"", ""is_pk"")
)
@@ -70,6 +79,22 @@ class Database:
datetime.time: ""TEXT"",
None.__class__: ""TEXT"",
}
+ # If numpy is available, add more types
+ if numpy:
+ col_type_mapping.update({
+ numpy.int8: ""INTEGER"",
+ numpy.int16: ""INTEGER"",
+ numpy.int32: ""INTEGER"",
+ numpy.int64: ""INTEGER"",
+ numpy.uint8: ""INTEGER"",
+ numpy.uint16: ""INTEGER"",
+ numpy.uint32: ""INTEGER"",
+ numpy.uint64: ""INTEGER"",
+ numpy.float16: ""FLOAT"",
+ numpy.float32: ""FLOAT"",
+ numpy.float64: ""FLOAT"",
+ numpy.float128: ""FLOAT"",
+ })
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413740684,Detect numpy types when creating tables,
https://github.com/simonw/sqlite-utils/issues/11#issuecomment-466695695,https://api.github.com/repos/simonw/sqlite-utils/issues/11,466695695,MDEyOklzc3VlQ29tbWVudDQ2NjY5NTY5NQ==,9599,simonw,2019-02-23T21:10:35Z,2019-02-23T21:10:35Z,OWNER,Need to test this both with and without `numpy` installed.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413740684,Detect numpy types when creating tables,
https://github.com/simonw/sqlite-utils/issues/13#issuecomment-466732039,https://api.github.com/repos/simonw/sqlite-utils/issues/13,466732039,MDEyOklzc3VlQ29tbWVudDQ2NjczMjAzOQ==,9599,simonw,2019-02-24T04:07:57Z,2019-02-24T04:07:57Z,OWNER,"Example: http://api.nobelprize.org/v1/laureate.json
This includes affiliations which look like this:
""affiliations"": [
{
""name"": ""Sorbonne University"",
""city"": ""Paris"",
""country"": ""France""
}
]","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413779210,Ability to automatically create IDs from content hash of row,
https://github.com/simonw/sqlite-utils/issues/10#issuecomment-466794069,https://api.github.com/repos/simonw/sqlite-utils/issues/10,466794069,MDEyOklzc3VlQ29tbWVudDQ2Njc5NDA2OQ==,9599,simonw,2019-02-24T16:55:37Z,2019-02-24T16:55:37Z,OWNER,"This was fixed by https://github.com/simonw/sqlite-utils/commit/228d595f7d10994f34e948888093c2cd290267c4 - see also #8
```
>>> db = sqlite_utils.Database("":memory:"")
>>> dfX=pd.DataFrame({'order':range(3),'col2':range(3)})
>>> db[""test""].upsert_all(dfX.to_dict(orient='records'))
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",411066700,Error in upsert if column named 'order',
https://github.com/simonw/sqlite-utils/issues/14#issuecomment-466794369,https://api.github.com/repos/simonw/sqlite-utils/issues/14,466794369,MDEyOklzc3VlQ29tbWVudDQ2Njc5NDM2OQ==,9599,simonw,2019-02-24T16:59:11Z,2019-02-24T16:59:43Z,OWNER,"https://www.sqlite.org/lang_createindex.html
May as well support ``--if-not-exists`` as well.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413842611,Utilities for adding indexes,
https://github.com/simonw/sqlite-utils/issues/14#issuecomment-466800090,https://api.github.com/repos/simonw/sqlite-utils/issues/14,466800090,MDEyOklzc3VlQ29tbWVudDQ2NjgwMDA5MA==,9599,simonw,2019-02-24T18:01:10Z,2019-02-24T18:01:10Z,OWNER,"The `WHERE` clause can be used to create partial indexes: https://www.sqlite.org/partialindex.html
I'm going to ignore it for the moment.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413842611,Utilities for adding indexes,
https://github.com/simonw/sqlite-utils/issues/14#issuecomment-466800210,https://api.github.com/repos/simonw/sqlite-utils/issues/14,466800210,MDEyOklzc3VlQ29tbWVudDQ2NjgwMDIxMA==,9599,simonw,2019-02-24T18:02:23Z,2019-02-24T18:02:23Z,OWNER,Likewise I'm going to ignore indexes on expressions (as opposed to just columns): https://www.sqlite.org/expridx.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413842611,Utilities for adding indexes,
https://github.com/simonw/sqlite-utils/issues/2#issuecomment-466807308,https://api.github.com/repos/simonw/sqlite-utils/issues/2,466807308,MDEyOklzc3VlQ29tbWVudDQ2NjgwNzMwOA==,9599,simonw,2019-02-24T19:18:19Z,2019-02-24T19:18:25Z,OWNER,"Python API:
db[""articles""].add_foreign_key(""author_id"", ""authors"", ""id"")
CLI:
$ sqlite-utils add-foreign-key articles author_id authors id
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",349850687,Mechanism for adding foreign keys to an existing table,
https://github.com/simonw/sqlite-utils/issues/17#issuecomment-466820167,https://api.github.com/repos/simonw/sqlite-utils/issues/17,466820167,MDEyOklzc3VlQ29tbWVudDQ2NjgyMDE2Nw==,9599,simonw,2019-02-24T21:42:33Z,2019-02-24T21:42:33Z,OWNER,"It looks like the type information isn't actually used for anything at all, so this:
https://github.com/simonw/sqlite-utils/blob/f8d3b7cfe5c1950b0749d40eb2640df50b52f651/tests/test_create.py#L97-L103
Could actually be written like this:
```
fresh_db[""m2m""].insert(
{""one_id"": 1, ""two_id"": 1},
foreign_keys=(
(""one_id"", ""one"", ""id""),
(""two_id"", ""two"", ""id""),
),
)
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413868452,Improve and document foreign_keys=... argument to insert/create/etc,
https://github.com/simonw/sqlite-utils/issues/17#issuecomment-466820188,https://api.github.com/repos/simonw/sqlite-utils/issues/17,466820188,MDEyOklzc3VlQ29tbWVudDQ2NjgyMDE4OA==,9599,simonw,2019-02-24T21:42:50Z,2019-02-24T21:42:50Z,OWNER,Sanity checking those foreign keys would be worthwhile.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413868452,Improve and document foreign_keys=... argument to insert/create/etc,
https://github.com/simonw/sqlite-utils/issues/17#issuecomment-466821200,https://api.github.com/repos/simonw/sqlite-utils/issues/17,466821200,MDEyOklzc3VlQ29tbWVudDQ2NjgyMTIwMA==,9599,simonw,2019-02-24T21:55:08Z,2019-02-24T21:55:54Z,OWNER,"This involves a breaking API change. I need to call that out in the README and also fix my two other projects which use the old four-tuple version of `foreign_keys=`:
https://github.com/simonw/db-to-sqlite/blob/c2f8e93bc6bbdfd135de3656ea0f497859ae49ff/db_to_sqlite/cli.py#L30-L42
And
https://github.com/simonw/russian-ira-facebook-ads-datasette/blob/e7106710abdd7bdcae035bedd8bdaba75ae56a12/fetch_and_build_russian_ads.py#L71-L74
I'll also need to set a minimum version for `sqlite-utils` in the `db-to-sqlite` setup.py:
https://github.com/simonw/db-to-sqlite/blob/c2f8e93bc6bbdfd135de3656ea0f497859ae49ff/setup.py#L25","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413868452,Improve and document foreign_keys=... argument to insert/create/etc,
https://github.com/simonw/sqlite-utils/issues/17#issuecomment-466823422,https://api.github.com/repos/simonw/sqlite-utils/issues/17,466823422,MDEyOklzc3VlQ29tbWVudDQ2NjgyMzQyMg==,9599,simonw,2019-02-24T22:20:05Z,2019-02-24T22:20:05Z,OWNER,Re-opening this until I've fixed the other two projects.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413868452,Improve and document foreign_keys=... argument to insert/create/etc,
https://github.com/simonw/sqlite-utils/issues/17#issuecomment-466827533,https://api.github.com/repos/simonw/sqlite-utils/issues/17,466827533,MDEyOklzc3VlQ29tbWVudDQ2NjgyNzUzMw==,9599,simonw,2019-02-24T23:03:29Z,2019-02-24T23:03:29Z,OWNER,Need to put out a new release of `sqlite-utils` so `db-to-sqlite` can depend on it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413868452,Improve and document foreign_keys=... argument to insert/create/etc,
https://github.com/simonw/sqlite-utils/issues/17#issuecomment-466828503,https://api.github.com/repos/simonw/sqlite-utils/issues/17,466828503,MDEyOklzc3VlQ29tbWVudDQ2NjgyODUwMw==,9599,simonw,2019-02-24T23:15:26Z,2019-02-24T23:15:26Z,OWNER,Released: https://sqlite-utils.readthedocs.io/en/latest/changelog.html#v0-14,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413868452,Improve and document foreign_keys=... argument to insert/create/etc,
https://github.com/simonw/sqlite-utils/issues/17#issuecomment-466830869,https://api.github.com/repos/simonw/sqlite-utils/issues/17,466830869,MDEyOklzc3VlQ29tbWVudDQ2NjgzMDg2OQ==,9599,simonw,2019-02-24T23:45:48Z,2019-02-24T23:45:48Z,OWNER,Both projects have been upgraded.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413868452,Improve and document foreign_keys=... argument to insert/create/etc,
https://github.com/simonw/datasette/issues/187#issuecomment-467264937,https://api.github.com/repos/simonw/datasette/issues/187,467264937,MDEyOklzc3VlQ29tbWVudDQ2NzI2NDkzNw==,9599,simonw,2019-02-26T02:14:28Z,2019-02-26T02:14:28Z,OWNER,I'm working on a port of Datasette to Starlette which I think would fix this issue: https://github.com/encode/starlette,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",309033998,Windows installation error,
https://github.com/simonw/datasette/issues/409#issuecomment-472844001,https://api.github.com/repos/simonw/datasette/issues/409,472844001,MDEyOklzc3VlQ29tbWVudDQ3Mjg0NDAwMQ==,43100,Uninen,2019-03-14T13:04:20Z,2019-03-14T13:04:42Z,NONE,It seems this affects the Datasette Publish -site as well: https://github.com/simonw/datasette-publish-support/issues/3,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",408376825,Zeit API v1 does not work for new users - need to migrate to v2,
https://github.com/simonw/datasette/issues/409#issuecomment-472875713,https://api.github.com/repos/simonw/datasette/issues/409,472875713,MDEyOklzc3VlQ29tbWVudDQ3Mjg3NTcxMw==,209967,michaelmcandrew,2019-03-14T14:14:39Z,2019-03-14T14:14:39Z,NONE,also linking this zeit issue in case it is helpful: https://github.com/zeit/now-examples/issues/163#issuecomment-440125769,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",408376825,Zeit API v1 does not work for new users - need to migrate to v2,
https://github.com/simonw/datasette/pull/416#issuecomment-473154643,https://api.github.com/repos/simonw/datasette/issues/416,473154643,MDEyOklzc3VlQ29tbWVudDQ3MzE1NDY0Mw==,9599,simonw,2019-03-15T04:27:47Z,2019-03-15T04:28:00Z,OWNER,"Deployed a demo: https://datasette-optional-hash-demo.now.sh/
datasette publish now \
../demo-databses/russian-ads.db \
../demo-databses/polar-bears.db \
--branch=optional-hash \
-n datasette-optional-hash \
--alias datasette-optional-hash-demo \
--install=datasette-cluster-map \
--install=datasette-json-html
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421348146,URL hashing now optional: turn on with --config hash_urls:1 (#418),
https://github.com/simonw/datasette/pull/416#issuecomment-473156513,https://api.github.com/repos/simonw/datasette/issues/416,473156513,MDEyOklzc3VlQ29tbWVudDQ3MzE1NjUxMw==,9599,simonw,2019-03-15T04:40:29Z,2019-03-15T04:40:29Z,OWNER,"Still TODO: need to figure out what to do about cache TTL. Defaulting to 365 days no longer makes sense without the hash_urls setting.
Maybe drop that setting default to 0?
Here's the setting:
https://github.com/simonw/datasette/blob/9743e1d91b5f0a2b3c1c0bd6ffce8739341f43c4/datasette/app.py#L84-L86
And here's where it takes affect:
https://github.com/simonw/datasette/blob/4462a5ab2817ac0d9ffe20dafbbf27c5c5b81466/datasette/views/base.py#L491-L501","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421348146,URL hashing now optional: turn on with --config hash_urls:1 (#418),
https://github.com/simonw/datasette/issues/414#issuecomment-473156774,https://api.github.com/repos/simonw/datasette/issues/414,473156774,MDEyOklzc3VlQ29tbWVudDQ3MzE1Njc3NA==,9599,simonw,2019-03-15T04:42:06Z,2019-03-15T04:42:06Z,OWNER,"This has been bothering me as well, especially when I try to install `datasette` and `sqlite-utils` at the same time.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",415575624,datasette requires specific version of Click,
https://github.com/simonw/datasette/issues/411#issuecomment-473156905,https://api.github.com/repos/simonw/datasette/issues/411,473156905,MDEyOklzc3VlQ29tbWVudDQ3MzE1NjkwNQ==,9599,simonw,2019-03-15T04:42:58Z,2019-03-15T04:42:58Z,OWNER,"Have you tried this?
MakePoint(:Long || "", "" || :Lat)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",410384988,How to pass named parameter into spatialite MakePoint() function,
https://github.com/simonw/datasette/issues/415#issuecomment-473157770,https://api.github.com/repos/simonw/datasette/issues/415,473157770,MDEyOklzc3VlQ29tbWVudDQ3MzE1Nzc3MA==,9599,simonw,2019-03-15T04:49:03Z,2019-03-15T04:49:03Z,OWNER,"Interesting idea. I can see how this would make sense if you are dealing with really long SQL queries.
My own example of a long query that might benefit from this: https://russian-ads-demo.herokuapp.com/russian-ads-a42c4e8?sql=select%0D%0A++++target_id%2C%0D%0A++++targets.name%2C%0D%0A++++count(*)+as+n%2C%0D%0A++++json_object(%0D%0A++++++++%22href%22%2C+%22%2Frussian-ads%2Ffaceted-targets%3Ftargets%3D%22+||+%0D%0A++++++++++++json_insert(%3Atargets%2C+%27%24[%27+||+json_array_length(%3Atargets)+||+%27]%27%2C+target_id)%0D%0A++++++++%2C%0D%0A++++++++%22label%22%2C+json_insert(%3Atargets%2C+%27%24[%27+||+json_array_length(%3Atargets)+||+%27]%27%2C+target_id)%0D%0A++++)+as+apply_this_facet%2C%0D%0A++++json_object(%0D%0A++++++++%22href%22%2C+%22%2Frussian-ads%2Fdisplay_ads%3F_targets_json%3D%22+||+%0D%0A++++++++++++json_insert(%3Atargets%2C+%27%24[%27+||+json_array_length(%3Atargets)+||+%27]%27%2C+target_id)%0D%0A++++++++%2C%0D%0A++++++++%22label%22%2C+%22See+%22+||+count(*)+||+%22+ads+matching+%22+||+json_insert(%3Atargets%2C+%27%24[%27+||+json_array_length(%3Atargets)+||+%27]%27%2C+target_id)%0D%0A++++)+as+browse_these_ads%0D%0Afrom+ad_targets%0D%0Ajoin+targets+on+ad_targets.target_id+%3D+targets.id%0D%0Awhere%0D%0A++++json_array_length(%3Atargets)+%3D%3D+0+or%0D%0A++++ad_id+in+(%0D%0A++++++++select+ad_id%0D%0A++++++++from+%22ad_targets%22%0D%0A++++++++where+%22ad_targets%22.target_id+in+(select+value+from+json_each(%3Atargets))%0D%0A++++++++group+by+%22ad_targets%22.ad_id%0D%0A++++++++having+count(distinct+%22ad_targets%22.target_id)+%3D+json_array_length(%3Atargets)%0D%0A++++)%0D%0A++++and+target_id+not+in+(select+value+from+json_each(%3Atargets))%0D%0Agroup+by%0D%0A++++target_id+order+by+n+desc%0D%0A&targets=[%22e6200%22]
Having a `show/hide` link would be an easy way to support this in the UI, and those could add/remove a `_hide_sql=1` parameter.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",418329842,Add query parameter to hide SQL textarea,
https://github.com/simonw/datasette/issues/412#issuecomment-473158506,https://api.github.com/repos/simonw/datasette/issues/412,473158506,MDEyOklzc3VlQ29tbWVudDQ3MzE1ODUwNg==,9599,simonw,2019-03-15T04:53:53Z,2019-03-15T04:53:53Z,OWNER,"I've been thinking about how Datasette instances could query each other for a while - it's a really interesting direction.
There are some tricky problems to solve to get this to work. There's a SQLite mechanism called ""virtual table functions"" which can implement things like this, but it's not supported by Python's `sqlite3` module out of the box.
https://github.com/coleifer/sqlite-vtfunc is a library that enables this feature. I experimented with using that to implement a function that scrapes HTML content (with an eye to accessing data from other APIs and Datasette instances) a while ago: https://github.com/coleifer/sqlite-vtfunc/issues/6
The bigger challenge is how to get this kind of thing to behave well within a Python 3 async environment. I have some ideas here but they're going to require some very crafty engineering.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",411257981,Linked Data(sette),
https://github.com/simonw/datasette/pull/416#issuecomment-473159679,https://api.github.com/repos/simonw/datasette/issues/416,473159679,MDEyOklzc3VlQ29tbWVudDQ3MzE1OTY3OQ==,9599,simonw,2019-03-15T05:01:27Z,2019-03-15T05:01:27Z,OWNER,"Also: if the option is False and the user visits a URL with a hash in it, should we redirect them?
I'm inclined to say no: furthermore, I'd be OK continuing to serve a far-future cache header for that case.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421348146,URL hashing now optional: turn on with --config hash_urls:1 (#418),
https://github.com/simonw/datasette/pull/413#issuecomment-473160476,https://api.github.com/repos/simonw/datasette/issues/413,473160476,MDEyOklzc3VlQ29tbWVudDQ3MzE2MDQ3Ng==,9599,simonw,2019-03-15T05:06:37Z,2019-03-15T05:06:37Z,OWNER,Thanks!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",413887019,Update spatialite.rst,
https://github.com/simonw/datasette/pull/416#issuecomment-473160702,https://api.github.com/repos/simonw/datasette/issues/416,473160702,MDEyOklzc3VlQ29tbWVudDQ3MzE2MDcwMg==,9599,simonw,2019-03-15T05:08:13Z,2019-03-15T05:08:13Z,OWNER,This also needs extensive tests to ensure that with the option turned on all of the redirects behave as they should.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421348146,URL hashing now optional: turn on with --config hash_urls:1 (#418),
https://github.com/simonw/datasette/issues/415#issuecomment-473164038,https://api.github.com/repos/simonw/datasette/issues/415,473164038,MDEyOklzc3VlQ29tbWVudDQ3MzE2NDAzOA==,9599,simonw,2019-03-15T05:31:21Z,2019-03-15T05:31:21Z,OWNER,"Demo: https://latest.datasette.io/fixtures-dd88475?sql=select+%2A+from+sortable+order+by+pk1%2C+pk2+limit+101
![]()
v.s. https://latest.datasette.io/fixtures-dd88475?sql=select+%2A+from+sortable+order+by+pk1%2C+pk2+limit+101&_hide_sql=1
![]()
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",418329842,Add query parameter to hide SQL textarea,
https://github.com/simonw/datasette/issues/415#issuecomment-473217334,https://api.github.com/repos/simonw/datasette/issues/415,473217334,MDEyOklzc3VlQ29tbWVudDQ3MzIxNzMzNA==,36796532,ad-si,2019-03-15T09:30:57Z,2019-03-15T09:30:57Z,NONE,"Awesome, thanks! 😁 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",418329842,Add query parameter to hide SQL textarea,
https://github.com/simonw/datasette/issues/417#issuecomment-473308631,https://api.github.com/repos/simonw/datasette/issues/417,473308631,MDEyOklzc3VlQ29tbWVudDQ3MzMwODYzMQ==,9599,simonw,2019-03-15T14:32:13Z,2019-03-15T14:32:13Z,OWNER,"This would allow Datasette to be easily used as a ""data library"" (like a data warehouse but less expectation of big data querying technology such as Presto).
One of the things I learned at the NICAR CAR 2019 conference in Newport Beach is that there is a very real need for some kind of easily accessible data library at most newsrooms.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421546944,Datasette Library,
https://github.com/simonw/datasette/pull/416#issuecomment-473310026,https://api.github.com/repos/simonw/datasette/issues/416,473310026,MDEyOklzc3VlQ29tbWVudDQ3MzMxMDAyNg==,9599,simonw,2019-03-15T14:35:53Z,2019-03-15T14:35:53Z,OWNER,See #418 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421348146,URL hashing now optional: turn on with --config hash_urls:1 (#418),
https://github.com/simonw/datasette/issues/123#issuecomment-473313975,https://api.github.com/repos/simonw/datasette/issues/123,473313975,MDEyOklzc3VlQ29tbWVudDQ3MzMxMzk3NQ==,9599,simonw,2019-03-15T14:45:46Z,2019-03-15T14:45:46Z,OWNER,"I'm reopening this one as part of #417.
Further experience with Python's CSV standard library module has convinced me that pandas is not a required dependency for this. My [sqlite-utils](https://github.com/simonw/sqlite-utils) package can do most of the work here with very few dependencies.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275125561,Datasette serve should accept paths/URLs to CSVs and other file formats,
https://github.com/simonw/datasette/issues/419#issuecomment-473708941,https://api.github.com/repos/simonw/datasette/issues/419,473708941,MDEyOklzc3VlQ29tbWVudDQ3MzcwODk0MQ==,9599,simonw,2019-03-17T19:58:11Z,2019-03-17T19:58:11Z,OWNER,"Some problems to solve:
* Right now Datasette assumes it can always show the count of rows in a table, because this has been pre-calculated. If a database is mutable the pre-calculation trick no longer works, and for giant tables a `select count(*) from X` query can be expensive to run. Maybe we set a time limit on these? If time limit expires show ""many rows""?
* Maintaining a content hash of the table no longer makes sense if it is changing (though interestingly there's a `.sha3sum` built-in SQLite CLI command which takes a hash of the content and stays the same even through vacuum runs). Without that we need a different mechanism for calculating table colours. It also means that we can't do the special dbname-hash URL trick (see #418) at all if the database is opened as mutable.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421551434,"Default to opening files in mutable mode, special option for immutable files",
https://github.com/simonw/datasette/issues/418#issuecomment-473709815,https://api.github.com/repos/simonw/datasette/issues/418,473709815,MDEyOklzc3VlQ29tbWVudDQ3MzcwOTgxNQ==,9599,simonw,2019-03-17T20:08:31Z,2019-03-17T20:08:31Z,OWNER,"In #419 I'm now proposing that Datasette default to opening files in ""mutable"" mode, in which case it would not make sense to support hash URLs for those files at all. So actually this feature will only be available for files that are explicitly opened in immutable mode.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421548881,Hashed URLs should be optional,
https://github.com/simonw/datasette/issues/419#issuecomment-473709883,https://api.github.com/repos/simonw/datasette/issues/419,473709883,MDEyOklzc3VlQ29tbWVudDQ3MzcwOTg4Mw==,9599,simonw,2019-03-17T20:09:47Z,2019-03-17T20:37:45Z,OWNER,"Could I persist the last calculated count for a table and somehow detect if that table has been changed in any way by another process, hence invalidating the cached count (and potentially scheduling a new count)?
https://www.sqlite.org/c3ref/update_hook.html says that `sqlite3_update_hook()` can be used to register a handler invoked on almost all update/insert/delete operations to a specific table... except that it misses out on deletes triggered by `ON CONFLICT REPLACE` and only works for `ROWID` tables.
Also this hook is not exposed in the Python `sqlite3` library - though it may be available using some terrifying `ctypes` hacks: https://stackoverflow.com/a/16920926
So on further research, I think the answer is *no*: I should assume that it won't be possible to cache counts and magically invalidate the cache when the underlying file is changed by another process.
Instead I need to assume that counts will be an expensive operation.
As such, I can introduce a time limit on counts and use that anywhere a count is displayed. If the time limit is exceeded by the `count(*)` query I can show ""many"" instead.
That said... running `count(*)` against a table with 200,000 rows in only takes about 3ms, so even a timeout of 20ms is likely to work fine for tables of around a million rows.
It would be really neat if I could generate a lower bound count in a limited amount of time. If I counted up to 4m rows before the timeout I could show ""more than 4m rows"". No idea if that would be possible though.
Relevant: https://stackoverflow.com/questions/8988915/sqlite-count-slow-on-big-tables - reports of very slow counts on 6GB database file. Consensus seems to be ""yeah, that's just how SQLite is built"" - though there was a suggestion that you can use `select max(ROWID) from table` provided you are certain there have been no deletions.
Also relevant: http://sqlite.1065341.n5.nabble.com/sqlite3-performance-on-select-count-very-slow-for-16-GB-file-td80176.html","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421551434,"Default to opening files in mutable mode, special option for immutable files",
https://github.com/simonw/datasette/issues/419#issuecomment-473712820,https://api.github.com/repos/simonw/datasette/issues/419,473712820,MDEyOklzc3VlQ29tbWVudDQ3MzcxMjgyMA==,9599,simonw,2019-03-17T20:43:23Z,2019-03-17T20:43:51Z,OWNER,"So the differences here are:
* For immutable databases we calculate content hash and table counts; mutable databases we do not
* Immutable databasse open with `file:{}?immutable=1`, mutable databases open with `file:{}?mode=ro`
* Anywhere that shows a table count now needs to call a new method which knows to run `count(*)` with a timeout for mutable databases, read from the precalculated counts for immutable databases
* The url-hash option should no longer be available at all for mutable databases
* New command-line tool syntax: `datasette mutable.db` v.s. `datasette -i immutable.db`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421551434,"Default to opening files in mutable mode, special option for immutable files",
https://github.com/simonw/datasette/issues/419#issuecomment-473713363,https://api.github.com/repos/simonw/datasette/issues/419,473713363,MDEyOklzc3VlQ29tbWVudDQ3MzcxMzM2Mw==,9599,simonw,2019-03-17T20:49:39Z,2019-03-17T20:52:46Z,OWNER,"And a really important difference: the whole model of caching inspect data no longer works for mutable files, because another process might make a change to the database schema (adding a new table for example).
https://fivethirtyeight.datasettes.com/-/inspect
So everywhere that uses `self.ds.inspect()` right now will have to change to calling a routine which knows the difference between mutable and immutable databases and queries for live schema data for mutables while using a cache for immutables.
I'll track this as a separate ticket.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421551434,"Default to opening files in mutable mode, special option for immutable files",
https://github.com/simonw/datasette/issues/420#issuecomment-473713946,https://api.github.com/repos/simonw/datasette/issues/420,473713946,MDEyOklzc3VlQ29tbWVudDQ3MzcxMzk0Ng==,9599,simonw,2019-03-17T20:56:38Z,2019-03-17T20:58:17Z,OWNER,"Some examples:
https://github.com/simonw/datasette/blob/1f54e092306b208125f39d06712b02895eb75168/datasette/views/table.py#L34-L40
https://github.com/simonw/datasette/blob/1f54e092306b208125f39d06712b02895eb75168/datasette/views/table.py#L45-L48
https://github.com/simonw/datasette/blob/1f54e092306b208125f39d06712b02895eb75168/datasette/views/table.py#L62-L65
https://github.com/simonw/datasette/blob/1f54e092306b208125f39d06712b02895eb75168/datasette/views/table.py#L112-L123
https://github.com/simonw/datasette/blob/1f54e092306b208125f39d06712b02895eb75168/datasette/views/index.py#L11-L19
https://github.com/simonw/datasette/blob/afe9aa3ae03c485c5d6652741438d09445a486c1/datasette/views/base.py#L143-L147
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421971339,Fix all the places that currently use .inspect() data,
https://github.com/simonw/datasette/pull/416#issuecomment-473714545,https://api.github.com/repos/simonw/datasette/issues/416,473714545,MDEyOklzc3VlQ29tbWVudDQ3MzcxNDU0NQ==,9599,simonw,2019-03-17T21:03:08Z,2019-03-17T21:04:17Z,OWNER,I'm going to introduce a new config setting: `default_cache_ttl_hashed` - and set the default value for `default_cache_ttl` to 10s (to protect against dog-piling).,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421348146,URL hashing now optional: turn on with --config hash_urls:1 (#418),
https://github.com/simonw/datasette/pull/416#issuecomment-473715254,https://api.github.com/repos/simonw/datasette/issues/416,473715254,MDEyOklzc3VlQ29tbWVudDQ3MzcxNTI1NA==,9599,simonw,2019-03-17T21:11:37Z,2019-03-17T21:11:37Z,OWNER,The code for this has got a bit tricky. I need to make a decision at some point as to if the current request is a hashed_url request (if it includes a DB hash in the URL which is the current correct hash). I then need to be able to use that fact to decide which default TTL value to apply when returning the response.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421348146,URL hashing now optional: turn on with --config hash_urls:1 (#418),
https://github.com/simonw/datasette/pull/416#issuecomment-473717052,https://api.github.com/repos/simonw/datasette/issues/416,473717052,MDEyOklzc3VlQ29tbWVudDQ3MzcxNzA1Mg==,9599,simonw,2019-03-17T21:32:24Z,2019-03-17T21:33:16Z,OWNER,"Since this feature is now controlled by a config setting, I'm inclined to make it also available via a URL parameter.
If you hit this URL:
/fixtures/table.json?_hash=1
We can redirect to:
/fixtures-c2342/table.json
In this way developers can opt-in to a hashed (and hence far-future cached) response on a per-query basis.
This option won't be available against mutable databases though, which are coming in #419
This means that the `hash_urls:1` config basically has the effect of assuming `?_hash=1` on all URLs to mutable databases.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421348146,URL hashing now optional: turn on with --config hash_urls:1 (#418),
https://github.com/simonw/datasette/issues/417#issuecomment-473312514,https://api.github.com/repos/simonw/datasette/issues/417,473312514,MDEyOklzc3VlQ29tbWVudDQ3MzMxMjUxNA==,9599,simonw,2019-03-15T14:42:07Z,2019-03-17T22:12:30Z,OWNER,"A neat ability of Datasette Library would be if it can work against other files that have been dropped into the folder. In particular: if a user drops a CSV file into the folder, how about automatically converting that CSV file to SQLite using [sqlite-utils](https://github.com/simonw/sqlite-utils)?","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421546944,Datasette Library,
|