{"html_url": "https://github.com/simonw/datasette/issues/617#issuecomment-551349022", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/617", "id": 551349022, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MTM0OTAyMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-08T01:56:28Z", "updated_at": "2019-11-08T01:56:28Z", "author_association": "OWNER", "body": "Here's the current monstrosity:\r\nhttps://github.com/simonw/datasette/blob/f9c146b893856a48afa810ebcce1714f30d0d3a2/datasette/views/table.py#L210-L780", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 519613116, "label": "Refactor TableView.data() method"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/617#issuecomment-551349770", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/617", "id": 551349770, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MTM0OTc3MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-08T01:59:20Z", "updated_at": "2019-11-11T00:53:42Z", "author_association": "OWNER", "body": "This is what the code does:\r\n\r\n- Check if the table name is actually a canned query - if so execute that\r\n- Check for other parameters that need to be redirected - e.g. `?_filter_column=x` or `_sort_by_desc=y`\r\n- Turn the table + querystring parameters (`?foo=bar`, `?bar__contains=baz`, `?_where=`, `_sort=` etc) into a SQL query\r\n- Figure out pagination (apply limit, and handle [keyset pagination](https://simonwillison.net/2018/Oct/4/datasette-ideas/#Keyset_pagination))\r\n- Calculate suggested facets against the non-paginated version of the query\r\n- Calculate facet counts against the non-paginated version of the query\r\n- Execute the query\r\n- Expand any foreign key references from the results", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 519613116, "label": "Refactor TableView.data() method"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/617#issuecomment-551349885", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/617", "id": 551349885, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MTM0OTg4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-08T01:59:43Z", "updated_at": "2019-11-08T02:00:52Z", "author_association": "OWNER", "body": "A clean starting point would be to refactor out the \"take a table and a bunch of querystring parameters and turn them into a SQL query\" portion.\r\n\r\nAdded bonus to this: I've long wanted to be able to apply the various trimmings of the TableView (like faceting and foreign key expansion) to other arbitrary SQL queries. Splitting out the code that builds the SELECT query will go a long way to making that a reality.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 519613116, "label": "Refactor TableView.data() method"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/617#issuecomment-552141417", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/617", "id": 552141417, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MjE0MTQxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-09T21:58:40Z", "updated_at": "2019-11-09T21:58:40Z", "author_association": "OWNER", "body": "The function that builds the query could go in a new `datasette.utils.sql` module.\r\n\r\nI can design it to only take simple arguments (the table name, list of columns, list of primary keys and a list of key/value tuples from the query string). This will make it really easy to test (and means it won't need to be async since it won't have to use queries to retrieve those values).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 519613116, "label": "Refactor TableView.data() method"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/617#issuecomment-552253893", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/617", "id": 552253893, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MjI1Mzg5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-11T00:46:42Z", "updated_at": "2021-12-18T01:41:47Z", "author_association": "OWNER", "body": "As noted in https://github.com/simonw/datasette/issues/621#issuecomment-552253208 a common pattern in this method is blocks of code that append new items to the `where_clauses`, `params` and `extra_human_descriptions` arrays. This is a useful refactoring opportunity.\r\n\r\nCode that fits this pattern:\r\n\r\n* The code that builds based on the filters: `where_clauses, params = filters.build_where_clauses(table)` and `human_description_en = filters.human_description_en(extra=extra_human_descriptions)`\r\n* Code that handles `?_where=`: `where_clauses.extend(request.args[\"_where\"])` - though note that this also appends to a `extra_wheres_for_ui` array which nothing else uses\r\n* The `_through=` code, see #621 for details\r\n* The code that deals with `?_search=` FTS\r\n\r\nThe keyset pagination code modifies `where_clauses` and `params` too, but I don't think it's quite going to work with the same abstraction that would cover the above examples.\r\n\r\n[UPDATE December 2021 - this comment became the basis for a new `filters_from_request` plugin hook, see also #473]", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 519613116, "label": "Refactor TableView.data() method"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/617#issuecomment-552254238", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/617", "id": 552254238, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MjI1NDIzOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-11T00:49:57Z", "updated_at": "2019-11-11T00:51:35Z", "author_association": "OWNER", "body": "I experimented with a `table_filter()` plugin hook a while ago which looks very much like the abstraction I'm now talking about here: https://github.com/simonw/datasette/commit/5116c4ec8aed5091e1f75415424b80f613518dc6 - more details here: https://simonwillison.net/2018/Aug/6/russian-facebook-ads/#Weird_implementation_details_106", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 519613116, "label": "Refactor TableView.data() method"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/617#issuecomment-552254753", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/617", "id": 552254753, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MjI1NDc1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-11T00:54:51Z", "updated_at": "2019-11-11T00:54:51Z", "author_association": "OWNER", "body": "That `table_filter()` plugin hook should probably be renamed though, since it could now apply to the regular custom SQL view as well as the table view.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 519613116, "label": "Refactor TableView.data() method"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/617#issuecomment-555111571", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/617", "id": 555111571, "node_id": "MDEyOklzc3VlQ29tbWVudDU1NTExMTU3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-18T17:04:56Z", "updated_at": "2019-11-18T17:04:56Z", "author_association": "OWNER", "body": "I made some comments on an annotated version of this method a few days ago: https://gist.github.com/simonw/66aca058195d77bae5f614ef73352eb5", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 519613116, "label": "Refactor TableView.data() method"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/617#issuecomment-991755013", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/617", "id": 991755013, "node_id": "IC_kwDOBm6k_c47HPsF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T19:17:11Z", "updated_at": "2021-12-11T19:17:11Z", "author_association": "OWNER", "body": "This work is now happening in #1518 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 519613116, "label": "Refactor TableView.data() method"}, "performed_via_github_app": null}