{"html_url": "https://github.com/simonw/datasette/issues/266#issuecomment-389570841", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/266", "id": 389570841, "node_id": "MDEyOklzc3VlQ29tbWVudDM4OTU3MDg0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-05-16T15:54:49Z", "updated_at": "2018-06-15T07:41:09Z", "author_association": "OWNER", "body": "At the most basic level, this will work based on an extension. Most places you currently put a `.json` extension should also allow a `.csv` extension.\r\n\r\nBy default this will return the exact results you see on the current page (default max will remain 1000).\r\n\r\n## Streaming all records\r\n\r\nWhere things get interested is *streaming mode*. This will be an option which returns ALL matching records as a streaming CSV file, even if that ends up being millions of records.\r\n\r\nI think the best way to build this will be on top of the existing mechanism used to efficiently implement keyset pagination via `_next=` tokens.\r\n\r\n## Expanding foreign keys\r\n\r\nFor tables with foreign key references it would be useful if the CSV format could expand those references to include the labels from `label_column` - maybe via an additional `?_expand=1` option.\r\n\r\nWhen expanding each foreign key column will be shown twice:\r\n\r\n rowid,city_id,city_id_label,state", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323681589, "label": "Export to CSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/266#issuecomment-397534196", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/266", "id": 397534196, "node_id": "MDEyOklzc3VlQ29tbWVudDM5NzUzNDE5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-15T07:12:16Z", "updated_at": "2018-06-15T07:12:16Z", "author_association": "OWNER", "body": "The first version of this is now shipped to master. I ended up rewriting most of the experimental branch to deal with the nasty issue described in #303\r\n\r\nDemo is available on https://fivethirtyeight.datasettes.com/fivethirtyeight-ab24e01/most-common-name%2Fsurnames\r\n\r\n\r\n\r\nHere's the CSV version of that page: https://fivethirtyeight.datasettes.com/fivethirtyeight-ab24e01/most-common-name%2Fsurnames.csv", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323681589, "label": "Export to CSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/266#issuecomment-397534404", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/266", "id": 397534404, "node_id": "MDEyOklzc3VlQ29tbWVudDM5NzUzNDQwNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-15T07:13:20Z", "updated_at": "2018-06-15T07:13:20Z", "author_association": "OWNER", "body": "Still to add: the streaming version that iterates through all of the pages, as seen in experimental commit https://github.com/simonw/datasette/commit/ced379ea325787b8c3bf0a614daba1fa4856a3bd", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323681589, "label": "Export to CSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/266#issuecomment-397534498", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/266", "id": 397534498, "node_id": "MDEyOklzc3VlQ29tbWVudDM5NzUzNDQ5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-15T07:13:52Z", "updated_at": "2018-06-15T07:13:52Z", "author_association": "OWNER", "body": "Also needs documentation.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323681589, "label": "Export to CSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/233#issuecomment-397637302", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/233", "id": 397637302, "node_id": "MDEyOklzc3VlQ29tbWVudDM5NzYzNzMwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-15T14:24:08Z", "updated_at": "2018-06-15T14:55:19Z", "author_association": "OWNER", "body": "I'm going with the terminology \"labels\" here. You'll be able to add ``?_labels=1`` and the JSON will look something like this:\r\n```\r\n{\r\n \"rowid\": 233,\r\n \"TreeID\": 121240,\r\n \"qLegalStatus\": {\r\n \"value\" 2,\r\n \"label\": \"Private\"\r\n }\r\n \"qSpecies\": {\r\n \"value\": 16,\r\n \"label\": \"Sycamore\"\r\n }\r\n \"qAddress\": \"91 Commonwealth Ave\",\r\n ...\r\n}\r\n```\r\n\r\nI need this to help build foreign key expansions for CSV files, see #266 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 316444720, "label": "Option to expose expanded foreign keys in JSON/CSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/233#issuecomment-397648080", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/233", "id": 397648080, "node_id": "MDEyOklzc3VlQ29tbWVudDM5NzY0ODA4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-15T14:56:21Z", "updated_at": "2018-06-15T14:56:21Z", "author_association": "OWNER", "body": "I considered including a `\"table\"` key like this:\r\n\r\n```\r\n \"qLegalStatus\": {\r\n \"value\" 2,\r\n \"label\": \"Private\",\r\n \"table\": \"qLegalStatus\"\r\n }\r\n```\r\n\r\nThis would help generate the HTML links using just the JSON data. But... I realized that in a list of 50 rows that value would be duplicated 50 times which is a bit nasty.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 316444720, "label": "Option to expose expanded foreign keys in JSON/CSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/233#issuecomment-397663968", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/233", "id": 397663968, "node_id": "MDEyOklzc3VlQ29tbWVudDM5NzY2Mzk2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-15T15:51:17Z", "updated_at": "2018-06-15T15:51:17Z", "author_association": "OWNER", "body": "Nearly done, but I need the HTML view to ignore the `?_labels=1` param (it throws an error at the moment).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 316444720, "label": "Option to expose expanded foreign keys in JSON/CSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/233#issuecomment-397668427", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/233", "id": 397668427, "node_id": "MDEyOklzc3VlQ29tbWVudDM5NzY2ODQyNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-15T16:07:43Z", "updated_at": "2018-06-15T16:07:43Z", "author_association": "OWNER", "body": "Demo: https://datasette-json-labels-demo.now.sh/fixtures-fda0fea/facetable.json?_labels=1&_shape=array", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 316444720, "label": "Option to expose expanded foreign keys in JSON/CSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/233#issuecomment-397729319", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/233", "id": 397729319, "node_id": "MDEyOklzc3VlQ29tbWVudDM5NzcyOTMxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-15T20:10:24Z", "updated_at": "2018-06-15T20:10:24Z", "author_association": "OWNER", "body": "I'm also going to add the ability to specify individual columns that you want to expand using `?_label=city_id&_label=state_id`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 316444720, "label": "Option to expose expanded foreign keys in JSON/CSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/233#issuecomment-397729500", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/233", "id": 397729500, "node_id": "MDEyOklzc3VlQ29tbWVudDM5NzcyOTUwMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-15T20:11:14Z", "updated_at": "2018-06-15T20:11:14Z", "author_association": "OWNER", "body": "The `.json` and `.csv` links displayed on the table page should default to using `?_labels=1` if Datasette detects that there are foreign key expansions available for the page.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 316444720, "label": "Option to expose expanded foreign keys in JSON/CSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/266#issuecomment-397729945", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/266", "id": 397729945, "node_id": "MDEyOklzc3VlQ29tbWVudDM5NzcyOTk0NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-15T20:13:05Z", "updated_at": "2018-06-15T20:13:05Z", "author_association": "OWNER", "body": "The \"This data as ...\" area of the page is getting a bit untidy, especially if I'm going to add other download options in the future. I think I'll move the HTML to the page footer (less concerns about taking up lots of space there) and then have a bit of JavaScript that turns it into a show/hide menu of some sort in its current location.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323681589, "label": "Export to CSV"}, "performed_via_github_app": null}