{"html_url": "https://github.com/simonw/datasette/issues/1345#issuecomment-851113886", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1345", "id": 851113886, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTExMzg4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T01:52:27Z", "updated_at": "2021-05-31T01:52:27Z", "author_association": "OWNER", "body": "Related issue: visit https://latest.datasette.io/fixtures/facetable?_facet=state and click \"Hide this column\" on the \"state\" cog menu and you get https://latest.datasette.io/fixtures/facetable?_facet=state&_nocol=state which shows an error:\r\n\r\n> ### Invalid SQL\r\n> no such column: state\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 904071938, "label": "?_nocol= does not interact well with default facets"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1345#issuecomment-851114806", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1345", "id": 851114806, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTExNDgwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T01:55:56Z", "updated_at": "2021-05-31T01:55:56Z", "author_association": "OWNER", "body": "Two options here:\r\n\r\n- Don't provide users with options that will lead to this situation - so no \"Hide this column\" option on pages that are already faceted by that column\r\n- Ignore facet selections for columns which are no longer visible\r\n\r\nI think I like the first option more.\r\n\r\nI could partially implement that in the `table.js` JavaScript by looking at the `?_facet=` parameters... but that won't cover the case where the facet is happening because of default facets configured in `metadata.yml`.\r\n\r\nInstead the JavaScript should look for evidence in the DOM that specific facets are enabled. This could also help me cover other types of faceting, such as `?_facet_array=` or even custom facets provided by plugins.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 904071938, "label": "?_nocol= does not interact well with default facets"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1345#issuecomment-851115805", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1345", "id": 851115805, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTExNTgwNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T01:59:39Z", "updated_at": "2021-05-31T01:59:39Z", "author_association": "OWNER", "body": "Maybe there's a concept here of the columns that are required by a selected facet? Those can then be included as `data-` attributes on the page, which will then impact which \"Hide this column\" options are available.\r\n\r\nI can also use them to provide a better error message than \"no such column: state\" - I can verify that `?_nocol` and `?_col` have not been used to disable the required columns.\r\n\r\nThere is one other option here: I could still include the columns that are known to be needed for faceting in the faceting SQL queries, but leave them out of the query that is used to return the results! That's actually a pretty tempting (albeit more complex) option.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 904071938, "label": "?_nocol= does not interact well with default facets"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1345#issuecomment-851116105", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1345", "id": 851116105, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTExNjEwNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T02:00:44Z", "updated_at": "2021-05-31T02:00:44Z", "author_association": "OWNER", "body": "Maybe there's a short-term and longer-term solution for this - where the long-term solution is to use different columns in the faceting selects, while the short-term solution is to disable \"Hide this column\" for certain things.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 904071938, "label": "?_nocol= does not interact well with default facets"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1345#issuecomment-851117233", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1345", "id": 851117233, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTExNzIzMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T02:04:35Z", "updated_at": "2021-05-31T02:04:35Z", "author_association": "OWNER", "body": "That long-term solution may not be too difficult. The facets are calculated against `sql_no_limit` which is constructed here:\r\n\r\nhttps://github.com/simonw/datasette/blob/7b106e106000713bbee31b34d694b3dadbd4818c/datasette/views/table.py#L659-L665\r\n\r\nAnd used here:\r\n\r\nhttps://github.com/simonw/datasette/blob/7b106e106000713bbee31b34d694b3dadbd4818c/datasette/views/table.py#L706-L718\r\n\r\nCrucially, `sql_no_limit` is ONLY used for faceting - nothing else uses it anywhere. So constructing it before constructing `sql` and taking `?_col=` and `?_nocol=` into account may not be a complex change.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 904071938, "label": "?_nocol= does not interact well with default facets"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1345#issuecomment-851120306", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1345", "id": 851120306, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTEyMDMwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T02:14:36Z", "updated_at": "2021-05-31T02:14:36Z", "author_association": "OWNER", "body": "Yes! This was easier than I thought. I'm going with that solution - where facets are calculated against all columns, ignoring `?_col=` and `?_nocol=` entirely.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 904071938, "label": "?_nocol= does not interact well with default facets"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1350#issuecomment-851122934", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1350", "id": 851122934, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTEyMjkzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T02:23:47Z", "updated_at": "2021-05-31T02:23:47Z", "author_association": "OWNER", "body": "I think `?_facets_off=1` is a good design for this.\r\n\r\nI considered `?_facet=` with an empty string but that's a bit weird, and it's not clear what that would do if used like `?_facet=state&_facet=`.\r\n\r\nSo it definitely needs to be a separate named option.\r\n\r\nActually, since I have `?_nocol=` I'm going to call this `?_nofacets=1` instead as that's a little more consistent.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906977719, "label": "?_nofacets=1 query string argument for disabling facets and suggested facets"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1345#issuecomment-851127314", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1345", "id": 851127314, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTEyNzMxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T02:40:26Z", "updated_at": "2021-05-31T02:40:44Z", "author_association": "OWNER", "body": "Demo: https://latest.datasette.io/fixtures/facetable?_facet=state&_nocol=state - the state column is not selected but facet by state still works: \r\n\r\n<img width=\"1134\" alt=\"fixtures__facetable__15_rows_and_Why_Russians_do_not_smile\" src=\"https://user-images.githubusercontent.com/9599/120132273-110a2780-c198-11eb-91a5-cafb77a78c97.png\">\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 904071938, "label": "?_nocol= does not interact well with default facets"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1349#issuecomment-851128524", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1349", "id": 851128524, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTEyODUyNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T02:44:44Z", "updated_at": "2021-05-31T02:44:44Z", "author_association": "OWNER", "body": "Now that I have `?_nofacets=1` I can use that to fix this. The challenge is that in this block of code I need to modify the incoming request's query string arguments, which isn't something I've done before: https://github.com/simonw/datasette/blob/0a7621f96f8ad14da17e7172e8a7bce24ef78966/datasette/views/base.py#L263-L270\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906385991, "label": "CSV ?_stream=on redundantly calculates facets for every page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1349#issuecomment-851129019", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1349", "id": 851129019, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTEyOTAxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T02:46:38Z", "updated_at": "2021-05-31T02:46:38Z", "author_association": "OWNER", "body": "I think the right way to do this is to construct a new `Request` with a modified ASGI scope: https://github.com/simonw/datasette/blob/c5ae1197a208e1b034c88882e3ac865813a40980/datasette/utils/asgi.py#L88-L99", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906385991, "label": "CSV ?_stream=on redundantly calculates facets for every page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1349#issuecomment-851129464", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1349", "id": 851129464, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTEyOTQ2NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T02:48:06Z", "updated_at": "2021-05-31T02:48:06Z", "author_association": "OWNER", "body": "Actually there is precedent for swapping out `request.scope` for a new scope, as seen here in the routing code: https://github.com/simonw/datasette/blob/c5ae1197a208e1b034c88882e3ac865813a40980/datasette/app.py#L1117-L1122", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906385991, "label": "CSV ?_stream=on redundantly calculates facets for every page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1349#issuecomment-851131999", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1349", "id": 851131999, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTEzMTk5OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T02:57:25Z", "updated_at": "2021-05-31T02:57:25Z", "author_association": "OWNER", "body": "I'm having a really hard time figuring out how to unit test this - ideally I'd monitor which SQL queries are executed using the tracing mechanism, but that's not set up to work with anything other than HTML or JSON outputs: https://github.com/simonw/datasette/blob/c5ae1197a208e1b034c88882e3ac865813a40980/datasette/tracer.py#L125-L134", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906385991, "label": "CSV ?_stream=on redundantly calculates facets for every page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1349#issuecomment-851133125", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1349", "id": 851133125, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTEzMzEyNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T03:01:48Z", "updated_at": "2021-05-31T03:01:48Z", "author_association": "OWNER", "body": "I think it's worth getting `?_trace=1` to work with streaming CSV - this would have helped me spot this issue a long time ago.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906385991, "label": "CSV ?_stream=on redundantly calculates facets for every page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1351#issuecomment-851133471", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1351", "id": 851133471, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MTEzMzQ3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-31T03:02:59Z", "updated_at": "2021-05-31T03:02:59Z", "author_association": "OWNER", "body": "Since traces only work with `text/html` and JSON at the moment, the easiest way to do this will be to wrap generated CSV in a HTML page in a textarea if the user specified `?_trace=1`:\r\n\r\nhttps://github.com/simonw/datasette/blob/c5ae1197a208e1b034c88882e3ac865813a40980/datasette/tracer.py#L125-L134", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906993731, "label": "Get `?_trace=1` working with CSV and streaming CSVs"}, "performed_via_github_app": null}