{"html_url": "https://github.com/simonw/datasette/issues/189#issuecomment-379555484", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/189", "id": 379555484, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU1NTQ4NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T14:39:57Z", "updated_at": "2018-04-08T14:39:57Z", "author_association": "OWNER", "body": "I'm going to combine the code for explicit sorting with the existing code for _next= pagination - so even tables without an explicit sort order will run through the same code since they are ordered and paginated by primary key.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 309471814, "label": "Ability to sort (and paginate) by column"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/48#issuecomment-379556637", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/48", "id": 379556637, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU1NjYzNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T14:56:52Z", "updated_at": "2018-04-08T14:56:52Z", "author_association": "OWNER", "body": "It would be useful to have a microbenchmark in place to help understand how much of a performance benefit this would actually provide.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 272391665, "label": "Switch to ujson"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/189#issuecomment-379556774", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/189", "id": 379556774, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU1Njc3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T14:59:05Z", "updated_at": "2018-04-08T14:59:05Z", "author_association": "OWNER", "body": "A common problem with keyset pagination is that it can distort the \"total number of rows\" logic - every time you navigate to a further page the total rows count can decrease due to the extra arguments in the `where` clause. The `filtered_table_rows` value (see #194) calculated using `count_sql` currently has this problem.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 309471814, "label": "Ability to sort (and paginate) by column"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/194#issuecomment-379556981", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/194", "id": 379556981, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU1Njk4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T15:02:23Z", "updated_at": "2018-04-08T15:02:23Z", "author_association": "OWNER", "body": "Maybe `table_rows_filtered_count` would be more aesthetically pleasing than `filtered_table_rows_count`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 312312125, "label": "Rename table_rows and filtered_table_rows to have _count suffix"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/194#issuecomment-379556881", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/194", "id": 379556881, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU1Njg4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T15:00:48Z", "updated_at": "2018-04-08T15:02:35Z", "author_association": "OWNER", "body": "`table_rows_count` is always the *total* number of rows in the table. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 312312125, "label": "Rename table_rows and filtered_table_rows to have _count suffix"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/195#issuecomment-379557743", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/195", "id": 379557743, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU1Nzc0Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T15:13:18Z", "updated_at": "2018-04-08T15:13:18Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/blob/446d47fdb005b3776bc06ad8d1f44b01fc2e938b/datasette/app.py#L93-L102", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 312313496, "label": "Run pks_for_table in inspect, executing once at build time rather than constantly"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/189#issuecomment-379557982", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/189", "id": 379557982, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU1Nzk4Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T15:16:49Z", "updated_at": "2018-04-08T15:16:49Z", "author_association": "OWNER", "body": "A note about views: a view cannot be paginated using keyset pagination because records returned from a view don't have a primary key - so there's no way to reliably distinguish between _next= records when the sorted column has duplicates with the same value.\r\n\r\nDatasette already takes this into account: views are paginated using offset/limit instead. We can continue to do that even for views that have been sorted using a `_sort` parameter.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 309471814, "label": "Ability to sort (and paginate) by column"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/195#issuecomment-379559074", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/195", "id": 379559074, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU1OTA3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T15:31:49Z", "updated_at": "2018-04-08T15:31:49Z", "author_association": "OWNER", "body": "While I'm at it, doing the same thing for fts_table detection is worth considering:\r\n\r\nhttps://github.com/simonw/datasette/blob/446d47fdb005b3776bc06ad8d1f44b01fc2e938b/datasette/app.py#L598-L603", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 312313496, "label": "Run pks_for_table in inspect, executing once at build time rather than constantly"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/150#issuecomment-379559214", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/150", "id": 379559214, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU1OTIxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T15:33:58Z", "updated_at": "2018-04-08T15:33:58Z", "author_association": "OWNER", "body": "The single biggest challenge here is expanding foreign key references. This is the blocker that prevents `_group_count` from being useful at the moment.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 276704327, "label": "_group_count= feature improvements"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/150#issuecomment-379559319", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/150", "id": 379559319, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU1OTMxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T15:35:43Z", "updated_at": "2018-04-08T15:35:43Z", "author_association": "OWNER", "body": "From a code point of view, the current mechanism for `_group_count` makes the `TableView` even **more** complicated:\r\n\r\nhttps://github.com/simonw/datasette/blob/446d47fdb005b3776bc06ad8d1f44b01fc2e938b/datasette/app.py#L644-L653\r\n\r\nInstead, I think if `_group_count` is detected we should generate the SQL and then defer to `self.custom_sql`, like we do for canned queries:\r\n\r\nhttps://github.com/simonw/datasette/blob/446d47fdb005b3776bc06ad8d1f44b01fc2e938b/datasette/app.py#L539-L541", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 276704327, "label": "_group_count= feature improvements"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/195#issuecomment-379588602", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/195", "id": 379588602, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU4ODYwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T22:40:16Z", "updated_at": "2018-04-08T22:40:16Z", "author_association": "OWNER", "body": "Could also identify all views for that database, which would save on these queries:\r\n\r\nhttps://github.com/simonw/datasette/blob/b2188f044265c95f7e54860e28107c17d2a6ed2e/datasette/app.py#L543-L545", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 312313496, "label": "Run pks_for_table in inspect, executing once at build time rather than constantly"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/189#issuecomment-379591062", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/189", "id": 379591062, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU5MTA2Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T23:23:12Z", "updated_at": "2018-04-08T23:23:12Z", "author_association": "OWNER", "body": "To break this up into smaller units, the first implementation of this will only support a single `_sort` or `_sort_desc` querystring parameter.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 309471814, "label": "Ability to sort (and paginate) by column"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/189#issuecomment-379592393", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/189", "id": 379592393, "node_id": "MDEyOklzc3VlQ29tbWVudDM3OTU5MjM5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-04-08T23:45:42Z", "updated_at": "2018-04-08T23:46:31Z", "author_association": "OWNER", "body": "Actually next page SQL when sorting looks more like this:\r\n\r\n```\r\nselect rowid, * from [alcohol-consumption/drinks]\r\nwhere \"country\" like :p0\r\nand (\r\n    beer_servings > 111\r\n    or (beer_servings = 111 and rowid > 190)\r\n)\r\norder by beer_servings, rowid limit 101\r\n```\r\n\r\nThe next page after row 190 with sortable value 111 should show either records that are greater than 111 or records that match 111 but have a greater primary key than the last one seen.\r\n\r\nhttps://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+rowid%2C+*+from+%5Balcohol-consumption%2Fdrinks%5D%0D%0Awhere+%22country%22+like+%3Ap0%0D%0Aand+%28%0D%0A++++beer_servings+%3E+111%0D%0A++++or+%28beer_servings+%3D+111+and+rowid+%3E+190%29%0D%0A%29%0D%0Aorder+by+beer_servings%2C+rowid+limit+101&p0=%25a%25", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 309471814, "label": "Ability to sort (and paginate) by column"}, "performed_via_github_app": null}