home / github

Menu
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

7,983 rows

✎ View and edit SQL

This data as json, CSV (advanced)

id ▼ html_url issue_url node_id user created_at updated_at author_association body reactions issue performed_via_github_app
349861461 https://github.com/simonw/datasette/issues/135#issuecomment-349861461 https://api.github.com/repos/simonw/datasette/issues/135 MDEyOklzc3VlQ29tbWVudDM0OTg2MTQ2MQ== simonw 9599 2017-12-07T04:43:12Z 2017-12-07T04:43:12Z OWNER This query looks like it does the right thing: select * from sqlite_master where rootpage = 0 and ( sql like '%VIRTUAL TABLE%USING FTS%content="ElementaryGeometries"%' or ( tbl_name = "ElementaryGeometries" and sql like '%VIRTUAL TABLE%USING FTS%' ) ) Against a table that should not be shown as FTS: https://timezones-now-hrjgkinozh.now.sh/timezones-0d61a90?sql=++++++++select+*+from+sqlite_master%0D%0A++++++++++++where+rootpage+%3D+0%0D%0A++++++++++++and+%28%0D%0A++++++++++++++++sql+like+%27%25VIRTUAL+TABLE%25USING+FTS%25content%3D%22ElementaryGeometries%22%25%27%0D%0A++++++++++++++++or+%28%0D%0A++++++++++++++++++tbl_name+%3D+%22ElementaryGeometries%22%0D%0A++++++++++++++++++and+sql+like+%27%25VIRTUAL+TABLE%25USING+FTS%25%27%0D%0A++++++++++++++++%29%0D%0A++++++++++++%29+ Against a table that SHOULD match: https://sf-trees.now.sh/sf-trees-ebc2ad9?sql=++++++++select+*+from+sqlite_master%0D%0A++++++++++++where+rootpage+%3D+0%0D%0A++++++++++++and+%28%0D%0A++++++++++++++++sql+like+%27%25VIRTUAL+TABLE%25USING+FTS%25content%3D%22Street_Tree_List_fts%22%25%27%0D%0A++++++++++++++++or+%28%0D%0A++++++++++++++++++tbl_name+%3D+%22Street_Tree_List_fts%22%0D%0A++++++++++++++++++and+sql+like+%27%25VIRTUAL+TABLE%25USING+FTS%25%27%0D%0A++++++++++++++++%29%0D%0A++++++++++++%29+ {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} ?_search=x should work if used directly against a FTS virtual table 275179724  
349868849 https://github.com/simonw/datasette/issues/158#issuecomment-349868849 https://api.github.com/repos/simonw/datasette/issues/158 MDEyOklzc3VlQ29tbWVudDM0OTg2ODg0OQ== simonw 9599 2017-12-07T05:41:08Z 2017-12-07T05:41:08Z OWNER I'm happy with this - we have extra_head, content, body_class and title blocks which should provide enough hooks for most reasonable customizations. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ensure default templates are designed to be extended 278190981  
349874052 https://github.com/simonw/datasette/issues/153#issuecomment-349874052 https://api.github.com/repos/simonw/datasette/issues/153 MDEyOklzc3VlQ29tbWVudDM0OTg3NDA1Mg== simonw 9599 2017-12-07T06:17:33Z 2017-12-07T06:17:33Z OWNER In #159 I added a mechanism for easily customizing per-column displays, and I've added documentation showing an example of using this mechanism to set certain columns to display as unescaped HTML: http://datasette.readthedocs.io/en/latest/custom_templates.html#custom-templates This fixes item 3, so I'm closing this ticket! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to customize presentation of specific columns in HTML view 276842536  
349874709 https://github.com/simonw/datasette/issues/164#issuecomment-349874709 https://api.github.com/repos/simonw/datasette/issues/164 MDEyOklzc3VlQ29tbWVudDM0OTg3NDcwOQ== simonw 9599 2017-12-07T06:22:10Z 2017-12-07T06:22:10Z OWNER Example usage: datasette skeleton parlgov.db -m parlgov.json Generates a `parlgov.json` file containing this: { "title": null, "description": null, "description_html": null, "license": null, "license_url": null, "source": null, "source_url": null, "databases": { "parlgov": { "title": null, "description": null, "description_html": null, "license": null, "license_url": null, "source": null, "source_url": null, "queries": {}, "tables": { "info_data_source": { "title": null, "description": null, "description_html": null, "license": null, "license_url": null, "source": null, "source_url": null }, "external_party_castles_mair": { "title": null, "description": null, "description_html": null, "license": null, "license_url": null, "source": null, "source_url": null }, "external_party_chess": { "title": null, "description": null, "description_html": null, "license": null, "license_url": null, "source": null, "source_url": null }, "external_party_huber_inglehart": { "title": null, "description": null, "description_html": null, … {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} datasette skeleton command for kick-starting database and table metadata 280013907  
349874844 https://github.com/simonw/datasette/issues/164#issuecomment-349874844 https://api.github.com/repos/simonw/datasette/issues/164 MDEyOklzc3VlQ29tbWVudDM0OTg3NDg0NA== simonw 9599 2017-12-07T06:22:58Z 2017-12-07T06:22:58Z OWNER This metadata doesn't yet do anything - need to implement #165 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} datasette skeleton command for kick-starting database and table metadata 280013907  
350026183 https://github.com/simonw/datasette/issues/165#issuecomment-350026183 https://api.github.com/repos/simonw/datasette/issues/165 MDEyOklzc3VlQ29tbWVudDM1MDAyNjE4Mw== simonw 9599 2017-12-07T16:47:46Z 2017-12-07T16:47:46Z OWNER Here's an example metadata.json file illustrating custom per-database and per- table metadata: { "title": "Overall datasette title", "description_html": "This is a <em>description with HTML</em>.", "databases": { "db1": { "title": "First database", "description": "This is a string description & has no HTML", "license_url": "http://example.com/", "license": "The example license", "queries": { "canned_query": "select * from table1 limit 3;" }, "tables": { "table1": { "title": "Custom title for table1", "description": "Tables can have descriptions too", "source": "This has a custom source", "source_url": "http://example.com/" } } } } } {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} metadata.json support for per-database and per-table information 280014287  
350026452 https://github.com/simonw/datasette/issues/165#issuecomment-350026452 https://api.github.com/repos/simonw/datasette/issues/165 MDEyOklzc3VlQ29tbWVudDM1MDAyNjQ1Mg== simonw 9599 2017-12-07T16:48:34Z 2017-12-07T16:48:34Z OWNER Needs documentation, see #166 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} metadata.json support for per-database and per-table information 280014287  
350035741 https://github.com/simonw/datasette/issues/166#issuecomment-350035741 https://api.github.com/repos/simonw/datasette/issues/166 MDEyOklzc3VlQ29tbWVudDM1MDAzNTc0MQ== simonw 9599 2017-12-07T17:20:35Z 2017-12-07T17:20:35Z OWNER http://datasette.readthedocs.io/en/latest/metadata.html {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Documentation for metadata.json and datasette skeleton 280023225  
350108113 https://github.com/simonw/datasette/issues/161#issuecomment-350108113 https://api.github.com/repos/simonw/datasette/issues/161 MDEyOklzc3VlQ29tbWVudDM1MDEwODExMw== wsxiaoys 388154 2017-12-07T22:02:24Z 2017-12-07T22:02:24Z NONE It's not throwing the validation error anymore, but i still cannot run following with query: ``` WITH RECURSIVE cnt(x) AS (SELECT 1 UNION ALL SELECT x+1 FROM cnt LIMIT 10) SELECT x FROM cnt; ``` I got `near "WITH": syntax error`. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Support WITH query  278814220  
350125953 https://github.com/simonw/datasette/issues/167#issuecomment-350125953 https://api.github.com/repos/simonw/datasette/issues/167 MDEyOklzc3VlQ29tbWVudDM1MDEyNTk1Mw== simonw 9599 2017-12-07T23:25:28Z 2017-12-07T23:25:28Z OWNER My column/row HTML display logic has got way too convoluted. This is a sign I need to add proper unit tests for it and clean it up. The complexity comes from: * Displaying a rowid for tables that do not have a primary key * Showing an additional Link column for rows with a primary key * Not displaying that Link column on the individual row pages * Trying to get foreign keys working correctly in all cases, e.g. #152 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Nasty bug: last column not being correctly displayed 280315352  
350158037 https://github.com/simonw/datasette/issues/161#issuecomment-350158037 https://api.github.com/repos/simonw/datasette/issues/161 MDEyOklzc3VlQ29tbWVudDM1MDE1ODAzNw== simonw 9599 2017-12-08T02:52:34Z 2017-12-08T02:52:34Z OWNER That might mean your version of SQLite doesn't support that syntax. Unfortunately the version bundled with Python is a bit old - the one built by the Dockerfile in this repo should handle it though. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Support WITH query  278814220  
350182904 https://github.com/simonw/datasette/issues/161#issuecomment-350182904 https://api.github.com/repos/simonw/datasette/issues/161 MDEyOklzc3VlQ29tbWVudDM1MDE4MjkwNA== wsxiaoys 388154 2017-12-08T06:18:12Z 2017-12-08T06:18:12Z NONE You're right..got this resolved after upgrading the sqlite version. Thanks you! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Support WITH query  278814220  
350292364 https://github.com/simonw/datasette/issues/141#issuecomment-350292364 https://api.github.com/repos/simonw/datasette/issues/141 MDEyOklzc3VlQ29tbWVudDM1MDI5MjM2NA== simonw 9599 2017-12-08T15:33:18Z 2017-12-08T15:33:18Z OWNER I can emulate this on OS X using a disk image (Disk Utility -> File -> New Image -> Blank Image...) - once mounted, I get the following: >>> os.link('/tmp/hello', '/Volumes/Untitled/hello') Traceback (most recent call last): File "<stdin>", line 1, in <module> OSError: [Errno 18] Cross-device link: '/tmp/hello' -> '/Volumes/Untitled/hello' I can simulate that in a mock like this: >>> from unittest.mock import patch >>> @patch('os.link') ... def test_link(mock_link): ... mock_link.side_effect = OSError ... mock_link() ... {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} datasette publish can fail if /tmp is on a different device 275814941  
350301248 https://github.com/simonw/datasette/issues/141#issuecomment-350301248 https://api.github.com/repos/simonw/datasette/issues/141 MDEyOklzc3VlQ29tbWVudDM1MDMwMTI0OA== simonw 9599 2017-12-08T16:07:04Z 2017-12-08T16:07:04Z OWNER This fix should work, please have a go with latest master and let me know if you run into any problems. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} datasette publish can fail if /tmp is on a different device 275814941  
350302417 https://github.com/simonw/datasette/issues/154#issuecomment-350302417 https://api.github.com/repos/simonw/datasette/issues/154 MDEyOklzc3VlQ29tbWVudDM1MDMwMjQxNw== simonw 9599 2017-12-08T16:11:24Z 2017-12-08T16:11:24Z OWNER I think I'll do this as a custom Jinja template filter. That way template authors can re-use it for their own static files if they want. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Datasette CSS should include content hash in the URL 276873891  
350323722 https://github.com/simonw/datasette/issues/154#issuecomment-350323722 https://api.github.com/repos/simonw/datasette/issues/154 MDEyOklzc3VlQ29tbWVudDM1MDMyMzcyMg== simonw 9599 2017-12-08T17:35:25Z 2017-12-08T17:35:25Z OWNER If I do this as a querystring parameter I won't need to worry about URL routing. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Datasette CSS should include content hash in the URL 276873891  
350413422 https://github.com/simonw/datasette/pull/168#issuecomment-350413422 https://api.github.com/repos/simonw/datasette/issues/168 MDEyOklzc3VlQ29tbWVudDM1MDQxMzQyMg== simonw 9599 2017-12-09T01:33:40Z 2017-12-09T01:33:40Z OWNER https://github.com/channelcat/sanic/releases/tag/0.7.0 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Upgrade to Sanic 0.7.0 280662866  
350421661 https://github.com/simonw/datasette/issues/167#issuecomment-350421661 https://api.github.com/repos/simonw/datasette/issues/167 MDEyOklzc3VlQ29tbWVudDM1MDQyMTY2MQ== simonw 9599 2017-12-09T03:52:46Z 2017-12-09T03:52:46Z OWNER Input: results from the database, foreign key definitions, primary key definitions, type of page Output: display_columns and display_rows {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Nasty bug: last column not being correctly displayed 280315352  
350424595 https://github.com/simonw/datasette/issues/167#issuecomment-350424595 https://api.github.com/repos/simonw/datasette/issues/167 MDEyOklzc3VlQ29tbWVudDM1MDQyNDU5NQ== simonw 9599 2017-12-09T05:08:27Z 2017-12-09T05:08:27Z OWNER Perhaps the row.html and table.html templates should be passed the same data but should themselves decide if they will display the Link column {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Nasty bug: last column not being correctly displayed 280315352  
350496258 https://github.com/simonw/datasette/issues/160#issuecomment-350496258 https://api.github.com/repos/simonw/datasette/issues/160 MDEyOklzc3VlQ29tbWVudDM1MDQ5NjI1OA== simonw 9599 2017-12-09T18:29:28Z 2017-12-09T18:29:28Z OWNER Example usage: datasette package --static css:extra-css/ --static js:extra-js/ \ sf-trees.db --template-dir templates/ --tag sf-trees --branch master This creates a local Docker image that includes copies of the templates/, extra-css/ and extra-js/ directories. You can then run it like this: docker run -p 8001:8001 sf-trees For publishing to Zeit now: datasette publish now --static css:extra-css/ --static js:extra-js/ \ sf-trees.db --template-dir templates/ --name sf-trees --branch master Example: https://sf-trees-wbihszoazc.now.sh/sf-trees-02c8ef1/Street_Tree_List For publishing to Heroku: datasette publish heroku --static css:extra-css/ --static js:extra-js/ \ sf-trees.db --template-dir templates/ --branch master {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to bundle and serve additional static files 278208011  
350496277 https://github.com/simonw/datasette/issues/157#issuecomment-350496277 https://api.github.com/repos/simonw/datasette/issues/157 MDEyOklzc3VlQ29tbWVudDM1MDQ5NjI3Nw== simonw 9599 2017-12-09T18:29:41Z 2017-12-09T18:29:41Z OWNER Example usage: datasette package --static css:extra-css/ --static js:extra-js/ \ sf-trees.db --template-dir templates/ --tag sf-trees --branch master This creates a local Docker image that includes copies of the templates/, extra-css/ and extra-js/ directories. You can then run it like this: docker run -p 8001:8001 sf-trees For publishing to Zeit now: datasette publish now --static css:extra-css/ --static js:extra-js/ \ sf-trees.db --template-dir templates/ --name sf-trees --branch master Example: https://sf-trees-wbihszoazc.now.sh/sf-trees-02c8ef1/Street_Tree_List For publishing to Heroku: datasette publish heroku --static css:extra-css/ --static js:extra-js/ \ sf-trees.db --template-dir templates/ --branch master {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Teach "datasette publish" about custom template directories 278190321  
350506593 https://github.com/simonw/datasette/issues/170#issuecomment-350506593 https://api.github.com/repos/simonw/datasette/issues/170 MDEyOklzc3VlQ29tbWVudDM1MDUwNjU5Mw== simonw 9599 2017-12-09T21:25:50Z 2017-12-09T21:25:50Z OWNER Turns out this is already supported: https://github.com/simonw/datasette/blob/6bdfcf60760c27e29ff34692d06e62b36aeecc56/datasette/app.py#L307 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Custom template for named canned query 280745470  
350506751 https://github.com/simonw/datasette/issues/170#issuecomment-350506751 https://api.github.com/repos/simonw/datasette/issues/170 MDEyOklzc3VlQ29tbWVudDM1MDUwNjc1MQ== simonw 9599 2017-12-09T21:28:32Z 2017-12-09T21:28:32Z OWNER My mistake, that's using the database name - there isn't a way of customizing for a specific named query yet. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Custom template for named canned query 280745470  
350507155 https://github.com/simonw/datasette/issues/170#issuecomment-350507155 https://api.github.com/repos/simonw/datasette/issues/170 MDEyOklzc3VlQ29tbWVudDM1MDUwNzE1NQ== simonw 9599 2017-12-09T21:35:30Z 2017-12-09T21:35:30Z OWNER Canned query page (/mydatabase/canned-query): query-mydatabase-canned-query.html query-mydatabase.html query.html {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Custom template for named canned query 280745470  
350508049 https://github.com/simonw/datasette/issues/171#issuecomment-350508049 https://api.github.com/repos/simonw/datasette/issues/171 MDEyOklzc3VlQ29tbWVudDM1MDUwODA0OQ== simonw 9599 2017-12-09T21:50:50Z 2017-12-09T21:50:50Z OWNER Quoting the new documentation: You can find out which templates were considered for a specific page by viewing source on that page and looking for an HTML comment at the bottom. The comment will look something like this: <!-- Templates considered: *query-mydb-tz.html, query-mydb.html, query.html --> This example is from the canned query page for a query called "tz" in the database called "mydb". The asterisk shows which template was selected - so in this case, Datasette found a template file called `query-mydb-tz.html` and used that - but if that template had not been found, it would have tried for `query-mydb.html` or the default `query.html`. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} HTML comments specifying custom templates for page 280745746  
350515616 https://github.com/simonw/datasette/issues/167#issuecomment-350515616 https://api.github.com/repos/simonw/datasette/issues/167 MDEyOklzc3VlQ29tbWVudDM1MDUxNTYxNg== simonw 9599 2017-12-10T00:21:58Z 2017-12-10T00:21:58Z OWNER This function signature is pretty gross: https://github.com/simonw/datasette/blob/7a7e4b2ed8c76c6d002a9d707dbc840f6a2abf7f/datasette/app.py#L418 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Nasty bug: last column not being correctly displayed 280315352  
350515985 https://github.com/simonw/datasette/issues/167#issuecomment-350515985 https://api.github.com/repos/simonw/datasette/issues/167 MDEyOklzc3VlQ29tbWVudDM1MDUxNTk4NQ== simonw 9599 2017-12-10T00:28:39Z 2017-12-10T00:28:39Z OWNER A better alternative: ```async def display_columns_and_rows(self, database, table, rows, link_column=False):``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Nasty bug: last column not being correctly displayed 280315352  
350516782 https://github.com/simonw/datasette/issues/167#issuecomment-350516782 https://api.github.com/repos/simonw/datasette/issues/167 MDEyOklzc3VlQ29tbWVudDM1MDUxNjc4Mg== simonw 9599 2017-12-10T00:48:54Z 2017-12-10T00:48:54Z OWNER I can simplify this all by dropping the nicety where if a table is using a rowid the Link column is titled rowid instead. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Nasty bug: last column not being correctly displayed 280315352  
350519711 https://github.com/simonw/datasette/issues/169#issuecomment-350519711 https://api.github.com/repos/simonw/datasette/issues/169 MDEyOklzc3VlQ29tbWVudDM1MDUxOTcxMQ== simonw 9599 2017-12-10T02:04:56Z 2017-12-10T02:04:56Z OWNER Done! https://github.com/simonw/datasette/releases/tag/0.14 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Release v0.14 with templates and static files features 280744309  
350519736 https://github.com/simonw/datasette/issues/153#issuecomment-350519736 https://api.github.com/repos/simonw/datasette/issues/153 MDEyOklzc3VlQ29tbWVudDM1MDUxOTczNg== simonw 9599 2017-12-10T02:06:01Z 2017-12-10T02:06:01Z OWNER @ftrain Datasette 0.14 is now released with all of the above: https://github.com/simonw/datasette/releases/tag/0.14 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to customize presentation of specific columns in HTML view 276842536  
350519821 https://github.com/simonw/datasette/issues/153#issuecomment-350519821 https://api.github.com/repos/simonw/datasette/issues/153 MDEyOklzc3VlQ29tbWVudDM1MDUxOTgyMQ== simonw 9599 2017-12-10T02:08:45Z 2017-12-10T02:08:45Z OWNER Also worth mentioning: as of #160 and #157 the `datasette publish now`, `datasette publish heroku` and `datasette package` commands all know how to bundle up any `--static` or `--template-dir` content and include it in the Docker image / Heroku/Now deployment that gets generated. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to customize presentation of specific columns in HTML view 276842536  
350521619 https://github.com/simonw/datasette/issues/42#issuecomment-350521619 https://api.github.com/repos/simonw/datasette/issues/42 MDEyOklzc3VlQ29tbWVudDM1MDUyMTYxOQ== simonw 9599 2017-12-10T03:02:14Z 2017-12-10T03:02:14Z OWNER I think the `datasette skeleton` command from #164 makes this obsolete. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Homepage UI for editing metadata file 268591332  
350521635 https://github.com/simonw/datasette/issues/52#issuecomment-350521635 https://api.github.com/repos/simonw/datasette/issues/52 MDEyOklzc3VlQ29tbWVudDM1MDUyMTYzNQ== simonw 9599 2017-12-10T03:02:56Z 2017-12-10T03:02:56Z OWNER I don't think this is necessary. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Solution for temporarily uploading DB so it can be built by docker 273026602  
350521711 https://github.com/simonw/datasette/issues/90#issuecomment-350521711 https://api.github.com/repos/simonw/datasette/issues/90 MDEyOklzc3VlQ29tbWVudDM1MDUyMTcxMQ== simonw 9599 2017-12-10T03:05:48Z 2017-12-10T03:05:48Z OWNER I fixed that last issue in c195ee4d46f2577b1943836a8270d84c8341d138 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} datasette publish heroku 273846123  
350521736 https://github.com/simonw/datasette/issues/90#issuecomment-350521736 https://api.github.com/repos/simonw/datasette/issues/90 MDEyOklzc3VlQ29tbWVudDM1MDUyMTczNg== simonw 9599 2017-12-10T03:06:34Z 2017-12-10T03:06:34Z OWNER Heroku is now in the README as of 6bdfcf60760c27e29ff34692d06e62b36aeecc56 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} datasette publish heroku 273846123  
350521780 https://github.com/simonw/datasette/issues/91#issuecomment-350521780 https://api.github.com/repos/simonw/datasette/issues/91 MDEyOklzc3VlQ29tbWVudDM1MDUyMTc4MA== simonw 9599 2017-12-10T03:07:53Z 2017-12-10T03:07:53Z OWNER Won't fix - I think the custom templates and static stuff in https://github.com/simonw/datasette/releases/tag/0.14 renders this obsolete. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Option to serve databases from a different prefix, serve regular content elsewhere 273878873  
350521806 https://github.com/simonw/datasette/issues/138#issuecomment-350521806 https://api.github.com/repos/simonw/datasette/issues/138 MDEyOklzc3VlQ29tbWVudDM1MDUyMTgwNg== simonw 9599 2017-12-10T03:08:26Z 2017-12-10T03:08:36Z OWNER Implemented this in 80bf3afa43e3cb396c7a7c9b168eedbc6fe0fa15 and #165. Didn't use data package though. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Per-database and per-table metadata, probably using data-package 275476839  
350521853 https://github.com/simonw/datasette/issues/123#issuecomment-350521853 https://api.github.com/repos/simonw/datasette/issues/123 MDEyOklzc3VlQ29tbWVudDM1MDUyMTg1Mw== simonw 9599 2017-12-10T03:09:53Z 2017-12-10T03:09:53Z OWNER I'm going to keep this separate in csvs-to-sqlite. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Datasette serve should accept paths/URLs to CSVs and other file formats 275125561  
350527283 https://github.com/simonw/datasette/issues/121#issuecomment-350527283 https://api.github.com/repos/simonw/datasette/issues/121 MDEyOklzc3VlQ29tbWVudDM1MDUyNzI4Mw== simonw 9599 2017-12-10T06:00:47Z 2017-12-10T06:00:47Z OWNER This is also really interesting when combined with the spatialite AsGeoJSON function: http://www.gaia-gis.it/gaia-sins/spatialite-sql-4.2.0.html#p3misc {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} ?_json=foo&_json=bar query string argument  275089535  
353424169 https://github.com/simonw/datasette/issues/175#issuecomment-353424169 https://api.github.com/repos/simonw/datasette/issues/175 MDEyOklzc3VlQ29tbWVudDM1MzQyNDE2OQ== simonw 9599 2017-12-21T18:33:55Z 2017-12-21T18:33:55Z OWNER Done - thanks for curating these: https://github.com/topics/automatic-api {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add project topic "automatic-api" 282971961  
355487646 https://github.com/simonw/datasette/issues/120#issuecomment-355487646 https://api.github.com/repos/simonw/datasette/issues/120 MDEyOklzc3VlQ29tbWVudDM1NTQ4NzY0Ng== nickdirienzo 723567 2018-01-05T07:10:12Z 2018-01-05T07:10:12Z NONE Ah, glad I found this issue. I have private data that I'd like to share to a few different people. Personally, a shared username and password would be sufficient for me, more-or-less Basic Auth. Do you have more complex requirements in mind? I'm not sure if "plugin" means "build a plugin" or "find a plugin" or something else entirely. FWIW, I stumbled upon [sanic-auth](https://github.com/pyx/sanic-auth) which looks like a new project to bring some interfaces around auth to sanic, similar to Flask. Alternatively, it shouldn't be too bad to add in Basic Auth. If we went down that route, that would probably be best built as a separate package for sanic that `datasette` brings in. What are your thoughts around this? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Plugin that adds an authentication layer of some sort 275087397  
356115657 https://github.com/simonw/datasette/issues/176#issuecomment-356115657 https://api.github.com/repos/simonw/datasette/issues/176 MDEyOklzc3VlQ29tbWVudDM1NjExNTY1Nw== wulfmann 4313116 2018-01-08T22:22:32Z 2018-01-08T22:22:32Z NONE This project probably would not be the place for that. This is a layer for sqllite specifically. It solves a similar problem as graphql, so adding that here wouldn't make sense. Here's an example i found from google that uses micro to run a graphql microservice. you'd just then need to connect your db. https://github.com/timneutkens/micro-graphql {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add GraphQL endpoint 285168503  
356161672 https://github.com/simonw/datasette/issues/176#issuecomment-356161672 https://api.github.com/repos/simonw/datasette/issues/176 MDEyOklzc3VlQ29tbWVudDM1NjE2MTY3Mg== yozlet 173848 2018-01-09T02:35:35Z 2018-01-09T02:35:35Z NONE @wulfmann I think I disagree, except I'm not entirely sure what you mean by that first paragraph. The JSON API that Datasette currently exposes is quite different to GraphQL. Furthermore, there's no "just" about connecting micro-graphql to a DB; at least, no more "just" than adding any other API. You still need to configure the schema, which is exactly the kind of thing that Datasette does for JSON API. This is why I think that GraphQL's a good fit here. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add GraphQL endpoint 285168503  
356175667 https://github.com/simonw/datasette/issues/176#issuecomment-356175667 https://api.github.com/repos/simonw/datasette/issues/176 MDEyOklzc3VlQ29tbWVudDM1NjE3NTY2Nw== wulfmann 4313116 2018-01-09T04:19:03Z 2018-01-09T04:19:03Z NONE @yozlet Yes I think that I was confused when I posted my original comment. I see your main point now and am in agreement. {"total_count": 2, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 2, "rocket": 0, "eyes": 0} Add GraphQL endpoint 285168503  
357542404 https://github.com/simonw/datasette/pull/178#issuecomment-357542404 https://api.github.com/repos/simonw/datasette/issues/178 MDEyOklzc3VlQ29tbWVudDM1NzU0MjQwNA== simonw 9599 2018-01-14T21:06:07Z 2018-01-14T21:06:07Z OWNER Thanks for catching this, merged! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} If metadata exists, add it to heroku launch command 287240246  
359697938 https://github.com/simonw/datasette/issues/176#issuecomment-359697938 https://api.github.com/repos/simonw/datasette/issues/176 MDEyOklzc3VlQ29tbWVudDM1OTY5NzkzOA== gijs 7193 2018-01-23T07:17:56Z 2018-01-23T07:17:56Z NONE 👍 I'd like this too! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add GraphQL endpoint 285168503  
360535979 https://github.com/simonw/datasette/issues/179#issuecomment-360535979 https://api.github.com/repos/simonw/datasette/issues/179 MDEyOklzc3VlQ29tbWVudDM2MDUzNTk3OQ== psychemedia 82988 2018-01-25T17:18:24Z 2018-01-25T17:18:24Z CONTRIBUTOR To summarise that thread: - expose full `metadata.json` object to the index page template, eg to allow tables to be referred to by name; - ability to import multiple `metadata.json` files, eg to allow metadata files created for a specific SQLite db to be reused in a datasette referring to several database files; It could also be useful to allow users to import a python file containing custom functions that can that be loaded into scope and made available to custom templates. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} More metadata options for template authors  288438570  
368625350 https://github.com/simonw/datasette/issues/176#issuecomment-368625350 https://api.github.com/repos/simonw/datasette/issues/176 MDEyOklzc3VlQ29tbWVudDM2ODYyNTM1MA== wuhland 7431774 2018-02-26T19:44:11Z 2018-02-26T19:44:11Z NONE great idea! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add GraphQL endpoint 285168503  
370273359 https://github.com/simonw/datasette/issues/185#issuecomment-370273359 https://api.github.com/repos/simonw/datasette/issues/185 MDEyOklzc3VlQ29tbWVudDM3MDI3MzM1OQ== simonw 9599 2018-03-04T23:10:56Z 2018-03-04T23:10:56Z OWNER Are you talking specifically about accessing metadata from HTML templates? That makes a lot of sense, I'll think about how this could work. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Metadata should be a nested arbitrary KV store 299760684  
370461231 https://github.com/simonw/datasette/issues/185#issuecomment-370461231 https://api.github.com/repos/simonw/datasette/issues/185 MDEyOklzc3VlQ29tbWVudDM3MDQ2MTIzMQ== carlmjohnson 222245 2018-03-05T15:43:56Z 2018-03-05T15:44:27Z NONE Yes. I think the simplest implementation is to change lines like ```python metadata = self.ds.metadata.get('databases', {}).get(name, {}) ``` to ```python metadata = { **self.ds.metadata, **self.ds.metadata.get('databases', {}).get(name, {}), } ``` so that specified inner values overwrite outer values, but only if they exist. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Metadata should be a nested arbitrary KV store 299760684  
374810115 https://github.com/simonw/datasette/issues/186#issuecomment-374810115 https://api.github.com/repos/simonw/datasette/issues/186 MDEyOklzc3VlQ29tbWVudDM3NDgxMDExNQ== simonw 9599 2018-03-21T01:21:13Z 2018-03-21T01:21:13Z OWNER Hah, this is exactly the opposite of datasette's default approach to caching, which is to cache everything for as long as possible. I don't think we'll need to add `Cache-Control: no-cache` headers provided we instead set it up so you can turn off Datasette's caching. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} proposal new option to disable user agents cache 306811513  
374811114 https://github.com/simonw/datasette/issues/186#issuecomment-374811114 https://api.github.com/repos/simonw/datasette/issues/186 MDEyOklzc3VlQ29tbWVudDM3NDgxMTExNA== simonw 9599 2018-03-21T01:28:30Z 2018-03-21T01:28:30Z OWNER We actually have this already: https://github.com/simonw/datasette/blob/012fc7c5cd3e9160c9a4c19cc964253e97fb054a/datasette/cli.py#L253-L255 You can disable the cache headers using the `datasette --debug` option. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} proposal new option to disable user agents cache 306811513  
374872202 https://github.com/simonw/datasette/issues/186#issuecomment-374872202 https://api.github.com/repos/simonw/datasette/issues/186 MDEyOklzc3VlQ29tbWVudDM3NDg3MjIwMg== stefanocudini 47107 2018-03-21T09:07:22Z 2018-03-21T09:07:22Z NONE --debug is perfect tnk {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} proposal new option to disable user agents cache 306811513  
376585911 https://github.com/simonw/datasette/issues/185#issuecomment-376585911 https://api.github.com/repos/simonw/datasette/issues/185 MDEyOklzc3VlQ29tbWVudDM3NjU4NTkxMQ== simonw 9599 2018-03-27T16:19:43Z 2018-03-27T16:19:43Z OWNER OK, I have an implementation of this. I realised that not ALL metadata should be inherited: it makes sense for source/source_url/license/license_url to be inherited, but it doesn't make sense for the title and description to be inherited down to the individual databases and tables. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Metadata should be a nested arbitrary KV store 299760684  
376587017 https://github.com/simonw/datasette/issues/185#issuecomment-376587017 https://api.github.com/repos/simonw/datasette/issues/185 MDEyOklzc3VlQ29tbWVudDM3NjU4NzAxNw== simonw 9599 2018-03-27T16:22:59Z 2018-03-27T16:22:59Z OWNER One thing that's missing from this: if you set source/license data at the individual database level they should be inherited by tables within that database. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Metadata should be a nested arbitrary KV store 299760684  
376589591 https://github.com/simonw/datasette/issues/185#issuecomment-376589591 https://api.github.com/repos/simonw/datasette/issues/185 MDEyOklzc3VlQ29tbWVudDM3NjU4OTU5MQ== simonw 9599 2018-03-27T16:30:51Z 2018-03-27T16:30:51Z OWNER Also needed: the ability to unset metadata. If the root metadata specifies a license_url it should be possible to set "license_url": null on a child database or table. The current implementation will ignore null (or empty string) values and default to the top level value. I think the templates themselves should be able to indicate if they want the inherited values or not. That way we could support arbitrary key/values and avoid the application code having special knowledge of license_url etc. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Metadata should be a nested arbitrary KV store 299760684  
376590265 https://github.com/simonw/datasette/issues/185#issuecomment-376590265 https://api.github.com/repos/simonw/datasette/issues/185 MDEyOklzc3VlQ29tbWVudDM3NjU5MDI2NQ== carlmjohnson 222245 2018-03-27T16:32:51Z 2018-03-27T16:32:51Z NONE >I think the templates themselves should be able to indicate if they want the inherited values or not. That way we could support arbitrary key/values and avoid the application code having special knowledge of license_url etc. Yes, you could have `metadata` that works like `metadata` does currently and `inherited_metadata` that works with inheritance. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Metadata should be a nested arbitrary KV store 299760684  
376592044 https://github.com/simonw/datasette/issues/185#issuecomment-376592044 https://api.github.com/repos/simonw/datasette/issues/185 MDEyOklzc3VlQ29tbWVudDM3NjU5MjA0NA== carlmjohnson 222245 2018-03-27T16:38:23Z 2018-03-27T16:38:23Z NONE It would be nice to also allow arbitrary keys (maybe under a parent key called params or something to prevent conflicts). For our datasette project, we just have a bunch of dictionaries defined in the base template for things like site URL and column humanized names: https://github.com/baltimore-sun-data/salaries-datasette/blob/master/templates/base.html It would be cleaner if this were in the metadata.json. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Metadata should be a nested arbitrary KV store 299760684  
376594727 https://github.com/simonw/datasette/issues/188#issuecomment-376594727 https://api.github.com/repos/simonw/datasette/issues/188 MDEyOklzc3VlQ29tbWVudDM3NjU5NDcyNw== simonw 9599 2018-03-27T16:46:49Z 2018-05-28T21:34:34Z OWNER One point of complexity: datasette can be used to bundle multiple .db files into a single "app". I think that's OK. We could require that the `datasette_files` table is present in the first database file passed on the command-line. Or we could even construct a search path and consult multiple versions of the table spread across multiple files. That said... any configuration that corresponds to a specific table should live in the same database file as that table. Ditto for general metadata: if we have license/source information for a specific table or database that information should be able to live in the same .db file as the data. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to bundle metadata and templates inside the SQLite file 309047460  
376604558 https://github.com/simonw/datasette/issues/185#issuecomment-376604558 https://api.github.com/repos/simonw/datasette/issues/185 MDEyOklzc3VlQ29tbWVudDM3NjYwNDU1OA== simonw 9599 2018-03-27T17:16:27Z 2018-03-27T17:16:27Z OWNER I am SO inspired by what you've done with https://salaries.news.baltimoresun.com/ - that's pretty much my ideal use-case for Datasette, and it's by far the most elaborate customization I've seen so far. I'd love to hear other ideas that came up while building that. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Metadata should be a nested arbitrary KV store 299760684  
376614973 https://github.com/simonw/datasette/issues/185#issuecomment-376614973 https://api.github.com/repos/simonw/datasette/issues/185 MDEyOklzc3VlQ29tbWVudDM3NjYxNDk3Mw== carlmjohnson 222245 2018-03-27T17:49:00Z 2018-03-27T17:49:00Z NONE @simonw Other than metadata, the biggest item on wishlist for the salaries project was the ability to reorder by column. Of course, that could be done with a custom SQL query, but we didn't want to have to reimplement all the nav/pagination stuff from scratch. @carolinp, feel free to add your thoughts. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Metadata should be a nested arbitrary KV store 299760684  
376981291 https://github.com/simonw/datasette/issues/189#issuecomment-376981291 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3Njk4MTI5MQ== simonw 9599 2018-03-28T18:06:08Z 2018-03-28T18:06:08Z OWNER Given how unlikely it is that this will pose a real problem I think I like option 1: enable sort-by-column by default for all tables, then allow power users to instead switch to explicit enabling of the functionality in their `metadata.json` if they know their data is too big. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
376983741 https://github.com/simonw/datasette/issues/189#issuecomment-376983741 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3Njk4Mzc0MQ== simonw 9599 2018-03-28T18:12:35Z 2018-03-28T18:12:35Z OWNER I think this can work with a `?_sort=xxx` parameter - and `?_sort=-xxx` to sort in the opposite direction. I'd like to support "sort by X descending, then by Y ascending if there are dupes for X" as well. Two ways that could work: `?_sort=-xxx,yyy` Or... `?_sort=-xxx&_sort=yyy` The second option is probably better in that it makes it easier for columns to have a comma in their name. Is it possible for a SQLite column to start with a `-` character? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
376986668 https://github.com/simonw/datasette/issues/189#issuecomment-376986668 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3Njk4NjY2OA== simonw 9599 2018-03-28T18:21:53Z 2018-03-28T18:21:53Z OWNER Might have to do something special to get sort-by-nulls-last: https://stackoverflow.com/questions/12503120/how-to-do-nulls-last-in-sqlite order by ifnull(column_name, -999999) Would need to figure out a smart way to get the default value - maybe by running a min() or max() against the column first? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377049625 https://github.com/simonw/datasette/issues/189#issuecomment-377049625 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzA0OTYyNQ== simonw 9599 2018-03-28T21:52:05Z 2018-03-28T21:52:05Z OWNER This is a better pattern as you don't have to pick a minimum value: ORDER BY CASE WHEN SOMECOL IS NULL THEN 1 ELSE 0 END, SOMECOL {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377050461 https://github.com/simonw/datasette/issues/189#issuecomment-377050461 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzA1MDQ2MQ== simonw 9599 2018-03-28T21:55:14Z 2018-03-28T22:06:30Z OWNER I think there are actually four kinds of sort order we need to support; * ascending * descending * ascending, nulls last * descending, nulls last It looks like [-blah] is a valid SQLite table name, so mark I descending with a hyphen prefix isn't good. Instead, maybe this: ?_sort_asc=col1&_sort_desc_nulls_last=col2 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377051018 https://github.com/simonw/datasette/issues/189#issuecomment-377051018 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzA1MTAxOA== simonw 9599 2018-03-28T21:57:20Z 2018-03-28T22:00:17Z OWNER I'd like to continue to support _next=token pagination even for custom sort orders. To do that I should include rowid (or general primary key) as the tie breaker on all sorts so I can incorporate that it into the _next= token. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377052634 https://github.com/simonw/datasette/issues/189#issuecomment-377052634 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzA1MjYzNA== simonw 9599 2018-03-28T22:03:16Z 2018-03-28T22:03:16Z OWNER In terms of user interface: the obvious place to put this is as a drop down menu on the column headers. This also means the UI can support combined sort orders. Assuming you are already sorted by county descending and you select the candidate column header, the options could be: * sort all by candidate * sort all by candidate, descending * sort by county descending, then by candidate * sort by county descending, then by candidate descending {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377054358 https://github.com/simonw/datasette/issues/189#issuecomment-377054358 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzA1NDM1OA== simonw 9599 2018-03-28T22:09:25Z 2018-03-28T22:09:25Z OWNER I'm tempted to put these verbose sorting options inline in the page HTML but have them in the table footer so they don't clog up the top half of the page with uninteresting links - then use JavaScript to hoik them out into a dropdown menu attached to each column header. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377055663 https://github.com/simonw/datasette/issues/189#issuecomment-377055663 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzA1NTY2Mw== simonw 9599 2018-03-28T22:14:53Z 2018-03-28T22:14:53Z OWNER There is one other interesting option for auto-enabling/disabling sort: the inspect command could include data about column index presence and whether or not a column has any null values in it. This would allow us to dynamically include a "nulls last" option but only for columns that contain at least one null. It's quite a lot of additional engineering for a very minor feature though, so I think I'll punt on that for the moment. We may find that the _group_count feature can benefit from column value statistics later on though. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377065541 https://github.com/simonw/datasette/issues/190#issuecomment-377065541 https://api.github.com/repos/simonw/datasette/issues/190 MDEyOklzc3VlQ29tbWVudDM3NzA2NTU0MQ== simonw 9599 2018-03-28T22:58:52Z 2018-03-28T22:58:52Z OWNER This is because the SQL we are using here is: select * from compound_primary_key where "pk1" > "d" and "pk2" > "v" order by pk1, pk2 limit 101 This is incorrect. The correct SQL syntax (according to the example on https://www.sqlite.org/rowvalue.html#scrolling_window_queries ) is: select * from compound_primary_key where ("pk1", "pk2") > ("d", "v") order by pk1, pk2 limit 101 BUT... this uses "row values" syntax which was only added to SQLite in version 3.15.0 in October 2016: https://sqlite.org/changes.html#version_3_15_0 The version on https://datasette-issue-190-compound-pks.now.sh/compound-pks-9aafe8f?sql=select+sqlite_version%28%29%3B is 3.8.7.1 from October 2014. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Keyset pagination doesn't work correctly for compound primary keys 309558826  
377066466 https://github.com/simonw/datasette/issues/190#issuecomment-377066466 https://api.github.com/repos/simonw/datasette/issues/190 MDEyOklzc3VlQ29tbWVudDM3NzA2NjQ2Ng== simonw 9599 2018-03-28T23:03:45Z 2018-03-28T23:03:57Z OWNER Without row values syntax, the necessary SQL to retrieve the next page after `d, v` gets a bit gnarly: select * from compound_primary_key where pk1 >= "d" and not (pk1 = "d" and pk2 <= "v") order by pk1, pk2 See https://datasette-issue-190-compound-pks.now.sh/compound-pks-9aafe8f?sql=select+*+from+compound_primary_key+where+pk1+%3E%3D+%22d%22+and+not+%28pk1+%3D+%22d%22+and+pk2+%3C%3D+%22v%22%29+order+by+pk1%2C+pk2 This article was useful for figuring this out: https://use-the-index-luke.com/sql/partial-results/fetch-next-page {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Keyset pagination doesn't work correctly for compound primary keys 309558826  
377067541 https://github.com/simonw/datasette/issues/190#issuecomment-377067541 https://api.github.com/repos/simonw/datasette/issues/190 MDEyOklzc3VlQ29tbWVudDM3NzA2NzU0MQ== simonw 9599 2018-03-28T23:09:18Z 2018-03-28T23:09:51Z OWNER Here's how I generated the table for testing this with 3 compound primary keys: CREATE_SQL = ''' CREATE TABLE compound_three_primary_keys ( pk1 varchar(30), pk2 varchar(30), pk3 varchar(30), content text, PRIMARY KEY (pk1, pk2, pk3) );''' alphabet = 'abcdefghijklmnopqrstuvwxyz' for a in alphabet: for b in alphabet: for c in alphabet: print(''' INSERT INTO compound_three_primary_keys VALUES ('{}', '{}', '{}', '{}'); '''.strip().format(a, b, c, '{}-{}-{}-{}-{}-{}'.format(a,b,c,a,b,c))) {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Keyset pagination doesn't work correctly for compound primary keys 309558826  
377072022 https://github.com/simonw/datasette/issues/190#issuecomment-377072022 https://api.github.com/repos/simonw/datasette/issues/190 MDEyOklzc3VlQ29tbWVudDM3NzA3MjAyMg== simonw 9599 2018-03-28T23:32:24Z 2018-03-28T23:32:24Z OWNER Here's the SQL for a next page with three compound primary keys: https://datasette-issue-190-compound-pks.now.sh/compound-pks-8e99805?sql=select+*+from+compound_three_primary_keys%0D%0Awhere%0D%0A++%28pk1+%3E+%3Apk1%29%0D%0A++++or%0D%0A++%28pk1+%3D+%3Apk1+and+pk2+%3E+%3Apk2%29%0D%0A++++or%0D%0A++%28pk1+%3D+%3Apk1+and+pk2+%3D+%3Apk2+and+pk3+%3E+%3Apk3%29%0D%0Aorder+by+pk1%2C+pk2%2C+pk3%3B%0D%0A%0D%0A%0D%0A&pk1=a&pk2=d&pk3=v ``` select * from compound_three_primary_keys where (pk1 > :pk1) or (pk1 = :pk1 and pk2 > :pk2) or (pk1 = :pk1 and pk2 = :pk2 and pk3 > :pk3) order by pk1, pk2, pk3; ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Keyset pagination doesn't work correctly for compound primary keys 309558826  
377362466 https://github.com/simonw/datasette/issues/189#issuecomment-377362466 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzM2MjQ2Ng== simonw 9599 2018-03-29T20:29:14Z 2018-03-29T20:29:14Z OWNER Alternative idea: by default enable all sorting in the UI. If a table has more than 100,000 rows disable sorting UI except for columns that have an index. Allow this to be overridden in metadata.json {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377454591 https://github.com/simonw/datasette/issues/190#issuecomment-377454591 https://api.github.com/repos/simonw/datasette/issues/190 MDEyOklzc3VlQ29tbWVudDM3NzQ1NDU5MQ== simonw 9599 2018-03-30T06:11:59Z 2018-03-30T06:11:59Z OWNER Re-opening this issue: my fix doesn't play nicely with extra filter arguments. Consider this page: https://datasette-issue-190-compound-pks-not-quite-fixed.now.sh/compound-pks-8e99805/compound_three_primary_keys?content__contains=d The next link is to `?_next=f%2Cz%2Ct&content__contains=z` (that's next of `f,z,t`) but that gives us https://datasette-issue-190-compound-pks-not-quite-fixed.now.sh/compound-pks-8e99805/compound_three_primary_keys?_next=b%2Cx%2Cd&content__contains=d which shows `a,a,d` at the top. Sure enough, the generated SQL looks like this: https://datasette-issue-190-compound-pks-not-quite-fixed.now.sh/compound-pks-8e99805?sql=select+%2A+from+compound_three_primary_keys+where+%22content%22+like+%3Ap0+and+%28%5Bpk1%5D+%3E+%3Ap0%29%0A++or%0A%28%5Bpk1%5D+%3D+%3Ap0+and+%5Bpk2%5D+%3E+%3Ap1%29%0A++or%0A%28%5Bpk1%5D+%3D+%3Ap0+and+%5Bpk2%5D+%3D+%3Ap1+and+%5Bpk3%5D+%3E+%3Ap2%29+order+by+pk1%2C+pk2%2C+pk3+limit+101&p0=%25d%25&p1=b&p2=x&p3=d select * from compound_three_primary_keys where "content" like :p0 and ([pk1] > :p0) or ([pk1] = :p0 and [pk2] > :p1) or ([pk1] = :p0 and [pk2] = :p1 and [pk3] > :p2) order by pk1, pk2, pk3 limit 101 The parameters here are confused. The :p0 should be reserved just for the like clause - the other parameters should be p1, p2 and p3 (not p0, p1 and p2). {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Keyset pagination doesn't work correctly for compound primary keys 309558826  
377457087 https://github.com/simonw/datasette/issues/190#issuecomment-377457087 https://api.github.com/repos/simonw/datasette/issues/190 MDEyOklzc3VlQ29tbWVudDM3NzQ1NzA4Nw== simonw 9599 2018-03-30T06:30:23Z 2018-03-30T06:30:23Z OWNER Interestingly, in deploying a copy of the database to demonstrate this final bug fix I had to use the `--force` argument like so: datasette publish now --branch=master compound-pks.db --force This is because `now` had already deployed a Dockerfile referencing `--branch=master` once already, so it thought nothing had changed and it could re-use that last deployment. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Keyset pagination doesn't work correctly for compound primary keys 309558826  
377457214 https://github.com/simonw/datasette/issues/190#issuecomment-377457214 https://api.github.com/repos/simonw/datasette/issues/190 MDEyOklzc3VlQ29tbWVudDM3NzQ1NzIxNA== simonw 9599 2018-03-30T06:31:15Z 2018-03-30T06:31:15Z OWNER Fixed! https://datasette-issue-190-compound-pks-second-fix.now.sh/compound-pks-8e99805/compound_three_primary_keys?_next=b%2Cx%2Cd&content__contains=d now correctly shows `b,y,d` as the first row on the page. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Keyset pagination doesn't work correctly for compound primary keys 309558826  
377459579 https://github.com/simonw/datasette/issues/189#issuecomment-377459579 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzQ1OTU3OQ== simonw 9599 2018-03-30T06:47:52Z 2018-03-30T06:47:52Z OWNER I'm not entirely sure how to get `_next=` pagination working against sorted collections when a tie-breaker is needed. Consider this data: https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+rowid%2C+*+from+%5Bnfl-wide-receivers%2Fadvanced-historical%5D%0D%0Aorder+by+case+when+career_ranypa+is+null+then+1+else+0+end%2C+career_ranypa%2C+rowid+limit+11 ![2018-03-29 at 11 46 pm](https://user-images.githubusercontent.com/9599/38127549-790c8bd0-33ab-11e8-8d32-66f5d3847c8a.png) If the page size was set to 9 rather than 11, the page divide would be between those two rows with the same value in the `career_ranypa` column. What would the `?_next=` token look like such that the correct row would be returned? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377460127 https://github.com/simonw/datasette/issues/189#issuecomment-377460127 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzQ2MDEyNw== simonw 9599 2018-03-30T06:51:29Z 2018-03-30T06:51:52Z OWNER The problem is that our `_next=` pagination currently works based on a `>` - but for this case a `>=` for the value is needed combined with a `>` on the tie-breaker (which would be the `rowid` column). So I think this is the right SQL: ``` select rowid, * from [nfl-wide-receivers/advanced-historical] where career_ranypa >= -6.331167749 and rowid > 2736 order by case when career_ranypa is null then 1 else 0 end, career_ranypa, rowid limit 11 ``` https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+rowid%2C+*+from+%5Bnfl-wide-receivers%2Fadvanced-historical%5D%0D%0Awhere+career_ranypa+%3E%3D+-6.331167749+and+rowid+%3E+2736%0D%0Aorder+by+case+when+career_ranypa+is+null+then+1+else+0+end%2C+career_ranypa%2C+rowid+limit+11 But how do I encode a `_next` token that means ">= X and > Y"? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377462334 https://github.com/simonw/datasette/issues/189#issuecomment-377462334 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzQ2MjMzNA== simonw 9599 2018-03-30T07:06:21Z 2018-03-30T07:06:21Z OWNER Maybe the answer here is that anything that's encoded in the next token is treated as >= with the exception of columns known to be primary keys, which are treated as > {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377546510 https://github.com/simonw/datasette/issues/189#issuecomment-377546510 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzU0NjUxMA== simonw 9599 2018-03-30T15:13:11Z 2018-03-30T15:13:11Z OWNER Pushed some work-in-progress with failing unit tests here: https://github.com/simonw/datasette/commit/2f8359c6f25768805431c80c74e5ec4213c2b2a6 Here's a demo: https://datasette-column-sort-wip.now.sh/sortable-4bbaa6f/sortable?_sort=sortable - note that the `_sort_desc` and `_sort_nulls_last` options aren't done yet, plus it doesn't correctly paginate (the `_next` tokens do not yet take sorting into account). {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
377547265 https://github.com/simonw/datasette/issues/189#issuecomment-377547265 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3NzU0NzI2NQ== simonw 9599 2018-03-30T15:16:43Z 2018-03-30T15:16:43Z OWNER I think this is the right incantation for a "next" link: https://datasette-column-sort-wip.now.sh/sortable-4bbaa6f?sql=select+*+from+sortable%0D%0Awhere+sortable+%3C%3D+94%0D%0Aand+%28%0D%0A++%28pk1+%3E+%27d%27%29%0D%0A++or%0D%0A++%28pk1+%3D+%27d%27+and+pk2+%3E+%27w%27%29%0D%0A%29%0D%0Aorder+by+sortable+desc%2C+pk1%2C+pk2%0D%0Alimit+7 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
378279612 https://github.com/simonw/datasette/issues/122#issuecomment-378279612 https://api.github.com/repos/simonw/datasette/issues/122 MDEyOklzc3VlQ29tbWVudDM3ODI3OTYxMg== simonw 9599 2018-04-03T14:55:54Z 2018-04-03T14:55:54Z OWNER The new documentation for the `_shape=` parameter is now live at http://datasette.readthedocs.io/en/latest/json_api.html {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Redesign JSON output, ditch jsono, offer variants controlled by parameter instead 275092453  
378281740 https://github.com/simonw/datasette/issues/183#issuecomment-378281740 https://api.github.com/repos/simonw/datasette/issues/183 MDEyOklzc3VlQ29tbWVudDM3ODI4MTc0MA== simonw 9599 2018-04-03T15:01:43Z 2018-04-03T15:01:43Z OWNER I'm having trouble replicating this bug. In particular, I don't understand what you mean by "these are then rendered in the datasette query box using single quotes" - since canned queries aren't displayed in a textarea. Do you have an example database / metadata.json I can use to investigate this further? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Custom Queries - escaping strings 291639118  
378293484 https://github.com/simonw/datasette/pull/181#issuecomment-378293484 https://api.github.com/repos/simonw/datasette/issues/181 MDEyOklzc3VlQ29tbWVudDM3ODI5MzQ4NA== simonw 9599 2018-04-03T15:34:29Z 2018-04-03T15:34:29Z OWNER Here's what this looks like: ![2018-04-03 at 8 32 am](https://user-images.githubusercontent.com/9599/38259345-9e1c75ea-3719-11e8-83c9-2160c6fa079c.png) I need to figure out the right way to handle licensing of bundled software like this - it's MIT licensed which is compatible with Datasette's Apache 2 license, but I feel like bundled licensed software (including codemirror) needs to be recognized in the README or docs somehow. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} add "format sql" button to query page, uses sql-formatter 289425975  
378293599 https://github.com/simonw/datasette/pull/181#issuecomment-378293599 https://api.github.com/repos/simonw/datasette/issues/181 MDEyOklzc3VlQ29tbWVudDM3ODI5MzU5OQ== simonw 9599 2018-04-03T15:34:50Z 2018-04-03T15:36:58Z OWNER Let's only show the "Format SQL" button if the user has JavaScript enabled. We can do that in this code here: https://github.com/bsmithgall/datasette/blob/4a7151a58d6ab7c8404a91beef7083e8a5807cf8/datasette/templates/_codemirror_foot.html#L14-L21 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} add "format sql" button to query page, uses sql-formatter 289425975  
378295376 https://github.com/simonw/datasette/pull/181#issuecomment-378295376 https://api.github.com/repos/simonw/datasette/issues/181 MDEyOklzc3VlQ29tbWVudDM3ODI5NTM3Ng== simonw 9599 2018-04-03T15:39:57Z 2018-04-03T15:39:57Z OWNER On the licensing front: it looks like the way Django handles this is to keep the licensing header in the files intact, e.g. https://github.com/django/django/blob/6deaddcca367d0143c815aaa42342021baa3b41e/django/contrib/admin/static/admin/js/vendor/jquery/jquery.js So for this change, adding a comment at the top of `sql-formatter.min.js` which references the MIT license would do the trick. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} add "format sql" button to query page, uses sql-formatter 289425975  
378297842 https://github.com/simonw/datasette/pull/181#issuecomment-378297842 https://api.github.com/repos/simonw/datasette/issues/181 MDEyOklzc3VlQ29tbWVudDM3ODI5Nzg0Mg== bsmithgall 1957344 2018-04-03T15:47:13Z 2018-04-03T15:47:13Z NONE I can work on that -- would you prefer to inline a `display: hidden` and then have the javascript flip the visibility or include it as css? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} add "format sql" button to query page, uses sql-formatter 289425975  
379142500 https://github.com/simonw/datasette/issues/193#issuecomment-379142500 https://api.github.com/repos/simonw/datasette/issues/193 MDEyOklzc3VlQ29tbWVudDM3OTE0MjUwMA== carlmjohnson 222245 2018-04-06T04:05:58Z 2018-04-06T04:05:58Z NONE You could try pulling out a validate query strings method. If it fails validation build the error object from the message. If it passes, you only need to go down a happy path. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Cleaner mechanism for handling custom errors 310882100  
379555484 https://github.com/simonw/datasette/issues/189#issuecomment-379555484 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3OTU1NTQ4NA== simonw 9599 2018-04-08T14:39:57Z 2018-04-08T14:39:57Z OWNER I'm going to combine the code for explicit sorting with the existing code for _next= pagination - so even tables without an explicit sort order will run through the same code since they are ordered and paginated by primary key. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
379556637 https://github.com/simonw/datasette/issues/48#issuecomment-379556637 https://api.github.com/repos/simonw/datasette/issues/48 MDEyOklzc3VlQ29tbWVudDM3OTU1NjYzNw== simonw 9599 2018-04-08T14:56:52Z 2018-04-08T14:56:52Z OWNER It would be useful to have a microbenchmark in place to help understand how much of a performance benefit this would actually provide. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Switch to ujson 272391665  
379556774 https://github.com/simonw/datasette/issues/189#issuecomment-379556774 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3OTU1Njc3NA== simonw 9599 2018-04-08T14:59:05Z 2018-04-08T14:59:05Z OWNER A common problem with keyset pagination is that it can distort the "total number of rows" logic - every time you navigate to a further page the total rows count can decrease due to the extra arguments in the `where` clause. The `filtered_table_rows` value (see #194) calculated using `count_sql` currently has this problem. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
379556881 https://github.com/simonw/datasette/issues/194#issuecomment-379556881 https://api.github.com/repos/simonw/datasette/issues/194 MDEyOklzc3VlQ29tbWVudDM3OTU1Njg4MQ== simonw 9599 2018-04-08T15:00:48Z 2018-04-08T15:02:35Z OWNER `table_rows_count` is always the *total* number of rows in the table. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Rename table_rows and filtered_table_rows to have _count suffix 312312125  
379556981 https://github.com/simonw/datasette/issues/194#issuecomment-379556981 https://api.github.com/repos/simonw/datasette/issues/194 MDEyOklzc3VlQ29tbWVudDM3OTU1Njk4MQ== simonw 9599 2018-04-08T15:02:23Z 2018-04-08T15:02:23Z OWNER Maybe `table_rows_filtered_count` would be more aesthetically pleasing than `filtered_table_rows_count`. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Rename table_rows and filtered_table_rows to have _count suffix 312312125  
379557743 https://github.com/simonw/datasette/issues/195#issuecomment-379557743 https://api.github.com/repos/simonw/datasette/issues/195 MDEyOklzc3VlQ29tbWVudDM3OTU1Nzc0Mw== simonw 9599 2018-04-08T15:13:18Z 2018-04-08T15:13:18Z OWNER https://github.com/simonw/datasette/blob/446d47fdb005b3776bc06ad8d1f44b01fc2e938b/datasette/app.py#L93-L102 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Run pks_for_table in inspect, executing once at build time rather than constantly 312313496  
379557982 https://github.com/simonw/datasette/issues/189#issuecomment-379557982 https://api.github.com/repos/simonw/datasette/issues/189 MDEyOklzc3VlQ29tbWVudDM3OTU1Nzk4Mg== simonw 9599 2018-04-08T15:16:49Z 2018-04-08T15:16:49Z OWNER A note about views: a view cannot be paginated using keyset pagination because records returned from a view don't have a primary key - so there's no way to reliably distinguish between _next= records when the sorted column has duplicates with the same value. Datasette already takes this into account: views are paginated using offset/limit instead. We can continue to do that even for views that have been sorted using a `_sort` parameter. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to sort (and paginate) by column 309471814  
379559074 https://github.com/simonw/datasette/issues/195#issuecomment-379559074 https://api.github.com/repos/simonw/datasette/issues/195 MDEyOklzc3VlQ29tbWVudDM3OTU1OTA3NA== simonw 9599 2018-04-08T15:31:49Z 2018-04-08T15:31:49Z OWNER While I'm at it, doing the same thing for fts_table detection is worth considering: https://github.com/simonw/datasette/blob/446d47fdb005b3776bc06ad8d1f44b01fc2e938b/datasette/app.py#L598-L603 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Run pks_for_table in inspect, executing once at build time rather than constantly 312313496  
379559214 https://github.com/simonw/datasette/issues/150#issuecomment-379559214 https://api.github.com/repos/simonw/datasette/issues/150 MDEyOklzc3VlQ29tbWVudDM3OTU1OTIxNA== simonw 9599 2018-04-08T15:33:58Z 2018-04-08T15:33:58Z OWNER The single biggest challenge here is expanding foreign key references. This is the blocker that prevents `_group_count` from being useful at the moment. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} _group_count= feature improvements 276704327  
379559319 https://github.com/simonw/datasette/issues/150#issuecomment-379559319 https://api.github.com/repos/simonw/datasette/issues/150 MDEyOklzc3VlQ29tbWVudDM3OTU1OTMxOQ== simonw 9599 2018-04-08T15:35:43Z 2018-04-08T15:35:43Z OWNER From a code point of view, the current mechanism for `_group_count` makes the `TableView` even **more** complicated: https://github.com/simonw/datasette/blob/446d47fdb005b3776bc06ad8d1f44b01fc2e938b/datasette/app.py#L644-L653 Instead, I think if `_group_count` is detected we should generate the SQL and then defer to `self.custom_sql`, like we do for canned queries: https://github.com/simonw/datasette/blob/446d47fdb005b3776bc06ad8d1f44b01fc2e938b/datasette/app.py#L539-L541 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} _group_count= feature improvements 276704327  

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 13.131ms · About: simonw/datasette-graphql