html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/simonw/datasette/issues/1293#issuecomment-899915829,https://api.github.com/repos/simonw/datasette/issues/1293,899915829,IC_kwDOBm6k_c41o6A1,9599,simonw,2021-08-17T01:02:35Z,2021-08-17T01:02:35Z,OWNER,"New approach: this time I'm building a simplified executor for the bytecode operations themselves.
```python
def execute_operations(operations, max_iterations = 100, trace=None):
    trace = trace or (lambda *args: None)
    registers: Dict[int, Any] = {}
    cursors: Dict[int, Tuple[str, Dict]] = {}
    instruction_pointer = 0
    iterations = 0
    result_row = None
    while True:
        iterations += 1
        if iterations > max_iterations:
            break
        operation = operations[instruction_pointer]
        trace(instruction_pointer, dict(operation))
        opcode = operation[""opcode""]
        if opcode == ""Init"":
            if operation[""p2""] != 0:
                instruction_pointer = operation[""p2""]
                continue
            else:
                instruction_pointer += 1
                continue
        elif opcode == ""Goto"":
            instruction_pointer = operation[""p2""]
            continue
        elif opcode == ""Halt"":
            break
        elif opcode == ""OpenRead"":
            cursors[operation[""p1""]] = (""database_table"", {
                ""rootpage"": operation[""p2""],
                ""connection"": operation[""p3""],
            })
        elif opcode == ""OpenEphemeral"":
            cursors[operation[""p1""]] = (""ephemeral"", {
                ""num_columns"": operation[""p2""],
                ""index_keys"": [],
            })
        elif opcode == ""MakeRecord"":
            registers[operation[""p3""]] = (""MakeRecord"", {
                ""registers"": list(range(operation[""p1""] + operation[""p2""]))
            })
        elif opcode == ""IdxInsert"":
            record = registers[operation[""p2""]]
            cursors[operation[""p1""]][1][""index_keys""].append(record)
        elif opcode == ""Rowid"":
            registers[operation[""p2""]] = (""rowid"", {
                ""table"": operation[""p1""]
            })
        elif opcode == ""Sequence"":
            registers[operation[""p2""]] = (""sequence"", {
                ""next_from_cursor"": operation[""p1""]
            })
        elif opcode == ""Column"":
            registers[operation[""p3""]] = (""column"", {
                ""cursor"": operation[""p1""],
                ""column_offset"": operation[""p2""]
            })
        elif opcode == ""ResultRow"":
            p1 = operation[""p1""]
            p2 = operation[""p2""]
            trace(""ResultRow: "", list(range(p1, p1 + p2)), registers)
            result_row = [registers.get(i) for i in range(p1, p1 + p2)]
        elif opcode == ""Integer"":
            registers[operation[""p2""]] = (""Integer"", operation[""p1""])
        elif opcode == ""String8"":
            registers[operation[""p2""]] = (""String"", operation[""p4""])
        instruction_pointer += 1
    return {""registers"": registers, ""cursors"": cursors, ""result_row"": result_row}
```
Results are promising!
```
execute_operations(db.execute(""explain select 'hello', 55, rowid, * from searchable"").fetchall())

{'registers': {1: ('String', 'hello'),
  2: ('Integer', 55),
  3: ('rowid', {'table': 0}),
  4: ('rowid', {'table': 0}),
  5: ('column', {'cursor': 0, 'column_offset': 1}),
  6: ('column', {'cursor': 0, 'column_offset': 2}),
  7: ('column', {'cursor': 0, 'column_offset': 3})},
 'cursors': {0: ('database_table', {'rootpage': 32, 'connection': 0})},
 'result_row': [('String', 'hello'),
  ('Integer', 55),
  ('rowid', {'table': 0}),
  ('rowid', {'table': 0}),
  ('column', {'cursor': 0, 'column_offset': 1}),
  ('column', {'cursor': 0, 'column_offset': 2}),
  ('column', {'cursor': 0, 'column_offset': 3})]}
```
Here's what happens with a union across three tables:
```
execute_operations(db.execute(f""""""
explain select data as content from binary_data
union
select pk as content from complex_foreign_keys
union
select name as content from facet_cities
""""""}).fetchall())

{'registers': {1: ('column', {'cursor': 4, 'column_offset': 0}),
  2: ('MakeRecord', {'registers': [0, 1, 2, 3]}),
  3: ('column', {'cursor': 0, 'column_offset': 1}),
  4: ('column', {'cursor': 3, 'column_offset': 0})},
 'cursors': {3: ('ephemeral',
   {'num_columns': 1,
    'index_keys': [('MakeRecord', {'registers': [0, 1]}),
     ('MakeRecord', {'registers': [0, 1]}),
     ('MakeRecord', {'registers': [0, 1, 2, 3]})]}),
  2: ('database_table', {'rootpage': 44, 'connection': 0}),
  4: ('database_table', {'rootpage': 24, 'connection': 0}),
  0: ('database_table', {'rootpage': 42, 'connection': 0})},
 'result_row': [('column', {'cursor': 3, 'column_offset': 0})]}
```
Note how the result_row refers to cursor 3, which is an ephemeral table which had three different sets of `MakeRecord` index keys assigned to it - indicating that the output column is NOT from the same underlying table source.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,Show column metadata plus links for foreign keys on arbitrary query results,
https://github.com/simonw/datasette/issues/1438#issuecomment-900500824,https://api.github.com/repos/simonw/datasette/issues/1438,900500824,IC_kwDOBm6k_c41rI1Y,9599,simonw,2021-08-17T17:38:16Z,2021-08-17T17:38:16Z,OWNER,"Relevant template code: https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/templates/query.html#L71

`renderers` comes from here: https://github.com/simonw/datasette/blob/2883098770fc66e50183b2b231edbde20848d4d6/datasette/views/base.py#L593-L608","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",972918533,Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions,
https://github.com/simonw/datasette/issues/1438#issuecomment-900502364,https://api.github.com/repos/simonw/datasette/issues/1438,900502364,IC_kwDOBm6k_c41rJNc,9599,simonw,2021-08-17T17:40:41Z,2021-08-17T17:40:41Z,OWNER,Bug is likely in `path_with_format` itself: https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/utils/__init__.py#L710-L729,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",972918533,Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions,
https://github.com/simonw/datasette/issues/1438#issuecomment-900513267,https://api.github.com/repos/simonw/datasette/issues/1438,900513267,IC_kwDOBm6k_c41rL3z,9599,simonw,2021-08-17T17:57:05Z,2021-08-17T17:57:05Z,OWNER,"I'm having trouble replicating this bug outside of Vercel. Against Cloud Run: view-source:https://latest.datasette.io/fixtures?sql=select+*+from+searchable+where+text1+like+%22%25cat%25%22

The HTML here is:

```html
<p class=""export-links"">This data as
  <a href=""/fixtures.json?sql=select+*+from+searchable+where+text1+like+%22%25cat%25%22"">json</a>, 
 ...
  <a href=""/fixtures.csv?sql=select+*+from+searchable+where+text1+like+%22%25cat%25%22&amp;_size=max"">CSV</a>
</p>
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",972918533,Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions,
https://github.com/simonw/datasette/issues/1438#issuecomment-900516826,https://api.github.com/repos/simonw/datasette/issues/1438,900516826,IC_kwDOBm6k_c41rMva,9599,simonw,2021-08-17T18:02:27Z,2021-08-17T18:02:27Z,OWNER,"The key difference I can spot between Vercel and Cloud Run is that `+` in a query string gets converted to `%20` by Vercel before it gets to my app, but does not for Cloud Run:
```
# Vercel
~ % curl -s 'https://til.simonwillison.net/-/asgi-scope?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22%0D%0A' | rg 'query_string' -C 2
 'method': 'GET',
 'path': '/-/asgi-scope',
 'query_string': b'sql=select%20*%20from%20tunes%20where%20name%20like%20%22%25'
                 b'wise%20maid%25%22%0D%0A',
 'raw_path': b'/-/asgi-scope',

# Cloud Run
~ % curl -s 'https://latest-with-plugins.datasette.io/-/asgi-scope?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22%0D%0A' | rg 'query_string' -C 2
 'method': 'GET',
 'path': '/-/asgi-scope',
 'query_string': b'sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%2'
                 b'2%0D%0A',
 'raw_path': b'/-/asgi-scope',
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",972918533,Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions,
https://github.com/simonw/datasette/issues/1438#issuecomment-900518343,https://api.github.com/repos/simonw/datasette/issues/1438,900518343,IC_kwDOBm6k_c41rNHH,9599,simonw,2021-08-17T18:04:42Z,2021-08-17T18:04:42Z,OWNER,Here's how `request.query_string` works: https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/utils/asgi.py#L86-L88,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",972918533,Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions,
https://github.com/simonw/datasette/issues/1438#issuecomment-900681413,https://api.github.com/repos/simonw/datasette/issues/1438,900681413,IC_kwDOBm6k_c41r07F,9599,simonw,2021-08-17T22:47:44Z,2021-08-17T22:47:44Z,OWNER,I deployed another copy of `fixtures.db` on Vercel at https://til.simonwillison.net/fixtures so I can compare it with `fixtures.db` on Cloud Run at https://latest.datasette.io/fixtures,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",972918533,Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions,
https://github.com/simonw/datasette/issues/1438#issuecomment-900690998,https://api.github.com/repos/simonw/datasette/issues/1438,900690998,IC_kwDOBm6k_c41r3Q2,9599,simonw,2021-08-17T23:11:16Z,2021-08-17T23:12:25Z,OWNER,"I have completely failed to replicate this initial bug - but it's still there on the `thesession.vercel.app` deployment (even though my own deployments to Vercel do not exhibit it). Here's a one-liner to replicate it against that deployment:

`curl -s 'https://thesession.vercel.app/thesession?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22' | rg '.csv'`

Whit outputs this:

`<p class=""export-links"">This data as <a href=""/thesession.json?sql=select * from tunes where name like &#34;%wise maid%&#34;"">json</a>, <a href=""/thesession.csv?sql=select * from tunes where name like &#34;%wise maid%&#34;&amp;_size=max"">CSV</a></p>`

It looks like, rather than being URL-encoded, the original query string is somehow making it through to Jinja and then being auto-escaped there.

The weird thing is that the equivalent query executed against my `til.simonwillison.net` Vercel instance does this:

`curl -s 'https://til.simonwillison.net/fixtures?sql=select+*+from+searchable+where+text1+like+%22%25a%25%22' | rg '.csv'`

`<p class=""export-links"">This data as <a href=""/fixtures.json?sql=select%20*%20from%20searchable%20where%20text1%20like%20%22%25a%25%22"">json</a>, <a href=""/fixtures.csv?sql=select%20*%20from%20searchable%20where%20text1%20like%20%22%25a%25%22&amp;_size=max"">CSV</a></p>`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",972918533,Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions,
https://github.com/simonw/datasette/issues/1439#issuecomment-900699670,https://api.github.com/repos/simonw/datasette/issues/1439,900699670,IC_kwDOBm6k_c41r5YW,9599,simonw,2021-08-17T23:34:23Z,2021-08-17T23:34:23Z,OWNER,"The challenge comes down to telling the difference between the following:

- `/db/table` - an HTML table page
- `/db/table.csv` - the CSV version of `/db/table`
- `/db/table.csv` - no this one is actually a database table called `table.csv`
- `/db/table.csv.csv` - the CSV version of `/db/table.csv`
- `/db/table.csv.csv.csv` and so on...","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",973139047,Rethink how .ext formats (v.s. ?_format=) works before 1.0,
https://github.com/simonw/datasette/issues/1439#issuecomment-900705226,https://api.github.com/repos/simonw/datasette/issues/1439,900705226,IC_kwDOBm6k_c41r6vK,9599,simonw,2021-08-17T23:50:32Z,2021-08-17T23:50:47Z,OWNER,"An alternative solution would be to use some form of escaping for the characters that form the name of the table.

The obvious way to do this would be URL-encoding - but it doesn't hold for `.` characters. The hex for that is `%2E` but watch what happens with that in a URL:

```
# Against Cloud Run:
curl -s 'https://datasette.io/-/asgi-scope/foo/bar%2Fbaz%2E' | rg path
 'path': '/-/asgi-scope/foo/bar/baz.',
 'raw_path': b'/-/asgi-scope/foo/bar%2Fbaz.',
 'root_path': '',
# Against Vercel:
curl -s 'https://til.simonwillison.net/-/asgi-scope/foo/bar%2Fbaz%2E' | rg path
 'path': '/-/asgi-scope/foo/bar%2Fbaz%2E',
 'raw_path': b'/-/asgi-scope/foo/bar%2Fbaz%2E',
 'root_path': '',
```
Surprisingly in this case Vercel DOES keep it intact, but Cloud Run does not.

It's still no good though: I need a solution that works on Vercel, Cloud Run and every other potential hosting provider too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",973139047,Rethink how .ext formats (v.s. ?_format=) works before 1.0,