html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/datasette/issues/2065#issuecomment-1524616740,https://api.github.com/repos/simonw/datasette/issues/2065,1524616740,IC_kwDOBm6k_c5a384k,9599,simonw,2023-04-27T03:38:21Z,2023-04-27T03:38:21Z,OWNER,"Tried this: rye install https://github.com/simonw/datasette/archive/refs/heads/main.zip But got this error: Error: Expected one of `@`, `(`, `<`, `=`, `>`, `~`, `!`, `;`, found `:` I instead downloaded that file and ran: rye install main.zip This worked! And now: ``` ~/.rye/tools/main-zip/bin/datasette --version ``` ``` datasette, version 1.0a2 ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652,Datasette cannot be installed with Rye, https://github.com/simonw/datasette/issues/2066#issuecomment-1524637376,https://api.github.com/repos/simonw/datasette/issues/2066,1524637376,IC_kwDOBm6k_c5a4B7A,9599,simonw,2023-04-27T03:50:19Z,2023-04-27T03:50:19Z,OWNER,"Having trouble replicating this on my laptop. I tried a fresh virtual environment with fresh packages (in case this is a `httpx` change) but this passed: pytest tests/test_csv.py","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,Failing test: httpx.InvalidURL: URL too long, https://github.com/simonw/datasette/issues/2066#issuecomment-1524638233,https://api.github.com/repos/simonw/datasette/issues/2066,1524638233,IC_kwDOBm6k_c5a4CIZ,9599,simonw,2023-04-27T03:50:51Z,2023-04-27T03:50:51Z,OWNER,Failure was on 3.7. I'll try that.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,Failing test: httpx.InvalidURL: URL too long, https://github.com/simonw/datasette/issues/2066#issuecomment-1524648995,https://api.github.com/repos/simonw/datasette/issues/2066,1524648995,IC_kwDOBm6k_c5a4Ewj,9599,simonw,2023-04-27T03:56:42Z,2023-04-27T03:57:11Z,OWNER,"I don't have 3.7 locally. Trying to install it with `pyenv`. brew install pyenv Then: pyenv install --list | grep 3.7 Installing: pyenv install 3.7.16 Output: Installed Python-3.7.16 to /Users/simon/.pyenv/versions/3.7.16 So the executable is `/Users/simon/.pyenv/versions/3.7.16/bin/python`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,Failing test: httpx.InvalidURL: URL too long, https://github.com/simonw/datasette/issues/2066#issuecomment-1524655203,https://api.github.com/repos/simonw/datasette/issues/2066,1524655203,IC_kwDOBm6k_c5a4GRj,9599,simonw,2023-04-27T03:59:56Z,2023-04-27T03:59:56Z,OWNER,"In a fresh Datasette checkout I ran: pipenv shell --python /Users/simon/.pyenv/versions/3.7.16/bin/python That gave me a virtual environment with 3.7.16 Python. Then I ran: pip install -e '.[test]' Weirdly that gave me a `which pytest` of `/opt/homebrew/bin/pytest` which ran the tests on 3.11. I figured out the location of the virtual environment with `which python` and then ran this: ``` % /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/bin/pytest tests/test_csv.py ============================================================================================== test session starts =============================================================================================== platform darwin -- Python 3.7.16, pytest-7.3.1, pluggy-1.0.0 SQLite: 3.39.5 rootdir: /private/tmp/datasette configfile: pytest.ini plugins: asyncio-0.21.0, timeout-2.1.0, xdist-3.2.1, anyio-3.6.2 asyncio: mode=strict collected 15 items tests/test_csv.py ..........F.... [100%] ==================================================================================================== FAILURES ==================================================================================================== ________________________________________________________________________________________________ test_max_csv_mb _________________________________________________________________________________________________ app_client_csv_max_mb_one = def test_max_csv_mb(app_client_csv_max_mb_one): response = app_client_csv_max_mb_one.get( ( ""/fixtures.csv?sql=select+'{}'+"" ""from+compound_three_primary_keys&_stream=1&_size=max"" > ).format(""abcdefg"" * 10000) ) /private/tmp/datasette/tests/test_csv.py:161: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/asgiref/sync.py:240: in __call__ return call_result.result() /Users/simon/.pyenv/versions/3.7.16/lib/python3.7/concurrent/futures/_base.py:428: in result return self.__get_result() /Users/simon/.pyenv/versions/3.7.16/lib/python3.7/concurrent/futures/_base.py:384: in __get_result raise self._exception /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/asgiref/sync.py:306: in main_wrap result = await self.awaitable(*args, **kwargs) /private/tmp/datasette/datasette/utils/testing.py:76: in get headers=headers, /private/tmp/datasette/datasette/utils/testing.py:167: in _request content=post_body, /private/tmp/datasette/datasette/app.py:1787: in request method, self._fix(path, avoid_path_rewrites), **kwargs /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:1528: in request extensions=extensions, /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:346: in build_request url = self._merge_url(url) /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:376: in _merge_url merge_url = URL(url) /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_urls.py:113: in __init__ self._uri_reference = urlparse(url, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ url = ""http://localhost/fixtures.csv?sql=select+'abcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcde...gabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefg'+from+compound_three_primary_keys&_stream=1&_size=max"" kwargs = {} def urlparse(url: str = """", **kwargs: typing.Optional[str]) -> ParseResult: # Initial basic checks on allowable URLs. # --------------------------------------- # Hard limit the maximum allowable URL length. if len(url) > MAX_URL_LENGTH: > raise InvalidURL(""URL too long"") E httpx.InvalidURL: URL too long /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_urlparse.py:155: InvalidURL ============================================================================================ short test summary info ============================================================================================= FAILED tests/test_csv.py::test_max_csv_mb - httpx.InvalidURL: URL too long ========================================================================================== 1 failed, 14 passed in 1.83s ========================================================================================== (datasette) simon@Simons-MacBook-Pro datasette % ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,Failing test: httpx.InvalidURL: URL too long, https://github.com/simonw/datasette/issues/2066#issuecomment-1524659084,https://api.github.com/repos/simonw/datasette/issues/2066,1524659084,IC_kwDOBm6k_c5a4HOM,9599,simonw,2023-04-27T04:02:07Z,2023-04-27T04:02:07Z,OWNER,"This is the failing test: https://github.com/simonw/datasette/blob/249fcf8e3e2a90e763f41b080c1b9ec8017f5005/tests/test_csv.py#L156-L167","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,Failing test: httpx.InvalidURL: URL too long, https://github.com/simonw/datasette/issues/2066#issuecomment-1524660603,https://api.github.com/repos/simonw/datasette/issues/2066,1524660603,IC_kwDOBm6k_c5a4Hl7,9599,simonw,2023-04-27T04:02:55Z,2023-04-27T04:02:55Z,OWNER,"In the debugger: ``` (Pdb) MAX_URL_LENGTH 65536 ``` Weird this only seems to be a problem with `httpx` on Python 3.7 though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,Failing test: httpx.InvalidURL: URL too long, https://github.com/simonw/datasette/issues/2066#issuecomment-1524666049,https://api.github.com/repos/simonw/datasette/issues/2066,1524666049,IC_kwDOBm6k_c5a4I7B,9599,simonw,2023-04-27T04:06:18Z,2023-04-27T04:06:18Z,OWNER,"Most recent `httpx` release is 0.24 a couple of weeks ago. Here's what changed in that version: https://github.com/encode/httpx/compare/0.23.3...0.24.0 It looks like that URL limit is new: https://github.com/encode/httpx/commit/57daabf673705954afa94686c0002801c93d31f3#diff-78d8d93b5dd4c77d99c3e2b46b7286ba71a8fd60e92d8bd4eee5fb200b4f87bfR149-R155 ```python def urlparse(url: str = """", **kwargs: typing.Optional[str]) -> ParseResult: # Initial basic checks on allowable URLs. # --------------------------------------- # Hard limit the maximum allowable URL length. if len(url) > MAX_URL_LENGTH: raise InvalidURL(""URL too long"") ``` https://github.com/encode/httpx/blob/32e25497a36e6222cc64a758c98275b450dac28d/httpx/_urlparse.py#L153-L155","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,Failing test: httpx.InvalidURL: URL too long, https://github.com/simonw/datasette/issues/2066#issuecomment-1524669124,https://api.github.com/repos/simonw/datasette/issues/2066,1524669124,IC_kwDOBm6k_c5a4JrE,9599,simonw,2023-04-27T04:10:44Z,2023-04-27T04:10:52Z,OWNER,"I need an alternative way of generating a long string with a shorter URL. ```sql select group_concat('abcabcabc', '') from json_each(json_array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)) ``` https://latest.datasette.io/_memory?sql=select+group_concat%28%27abcabcabc%27%2C+%27%27%29+from+json_each%28json_array%281%2C+2%2C+3%2C+4%2C+5%2C+6%2C+7%2C+8%2C+9%2C+10%29%29","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,Failing test: httpx.InvalidURL: URL too long, https://github.com/simonw/datasette/issues/2066#issuecomment-1524675817,https://api.github.com/repos/simonw/datasette/issues/2066,1524675817,IC_kwDOBm6k_c5a4LTp,9599,simonw,2023-04-27T04:21:03Z,2023-04-27T04:21:03Z,OWNER,I went with this to generate the long string: https://github.com/simonw/datasette/blob/0b0c5cd7a94fe3f151a3e10261b5c84ee64f2f18/tests/test_csv.py#L157-L176,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,Failing test: httpx.InvalidURL: URL too long, https://github.com/simonw/datasette/issues/2066#issuecomment-1524680160,https://api.github.com/repos/simonw/datasette/issues/2066,1524680160,IC_kwDOBm6k_c5a4MXg,9599,simonw,2023-04-27T04:27:50Z,2023-04-27T04:27:50Z,OWNER,That fixed it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,Failing test: httpx.InvalidURL: URL too long, https://github.com/simonw/datasette/issues/2065#issuecomment-1524699863,https://api.github.com/repos/simonw/datasette/issues/2065,1524699863,IC_kwDOBm6k_c5a4RLX,9599,simonw,2023-04-27T04:56:22Z,2023-04-27T04:56:22Z,OWNER,Turned this into a TIL: https://til.simonwillison.net/python/rye,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652,Datasette cannot be installed with Rye, https://github.com/simonw/datasette/issues/2065#issuecomment-1524707628,https://api.github.com/repos/simonw/datasette/issues/2065,1524707628,IC_kwDOBm6k_c5a4TEs,9599,simonw,2023-04-27T05:06:44Z,2023-04-27T05:06:44Z,OWNER,"I need `pip` as a dependency too: ``` % ~/.rye/tools/main-zip/bin/datasette install datasette-graphql Traceback (most recent call last): File ""/Users/simon/.rye/tools/main-zip/bin/datasette"", line 8, in sys.exit(cli()) ^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1130, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1055, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 760, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/datasette/cli.py"", line 365, in install run_module(""pip"", run_name=""__main__"") File """", line 222, in run_module File """", line 142, in _get_module_details ImportError: No module named pip ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652,Datasette cannot be installed with Rye, https://github.com/simonw/datasette/issues/2065#issuecomment-1524709988,https://api.github.com/repos/simonw/datasette/issues/2065,1524709988,IC_kwDOBm6k_c5a4Tpk,9599,simonw,2023-04-27T05:09:36Z,2023-04-27T05:09:36Z,OWNER,"That fixed it - after installing `main.zip` again I ran this and it worked: ~/.rye/tools/main-zip/bin/datasette install datasette-graphql ``` % ~/.rye/tools/main-zip/bin/datasette plugins [ { ""name"": ""datasette-graphql"", ""static"": true, ""templates"": true, ""version"": ""2.2"", ""hooks"": [ ""database_actions"", ""extra_template_vars"", ""menu_links"", ""register_routes"", ""startup"", ""table_actions"" ] } ] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652,Datasette cannot be installed with Rye,