id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason
735852274,MDU6SXNzdWU3MzU4NTIyNzQ=,1082,DigitalOcean buildpack memory errors for large sqlite db?,39538958,open,0,,,3,2020-11-04T06:35:32Z,2020-11-04T19:35:44Z,,NONE,,"1. Have a sqlite db stored in Dropbox
2. Previously tried the Digital Ocean build pack minimal approach (e.g. Procfile, requirements.txt, bin/post_compile)
3. bin/post_compile with wget from Dropbox
4. download of large sqlite db is successful
5. log reveals that when building Docker container, Digital Ocean runs out of memory for 5gb+ sqlite db but works fine for 2gb+ sqlite db",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1082/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1690765434,I_kwDOBm6k_c5kxwh6,2067,Litestream-restored db: errors on 3.11 and 3.10.8; but works on py3.10.7 and 3.10.6,39538958,open,0,,,1,2023-05-01T12:42:28Z,2023-05-03T00:16:03Z,,NONE,,"Hi! Wondering if this issue is limited to my local system or if it affects others as well.
It seems like 3.11 errors out on a ""litestream-restored"" database. On further investigation, it also appears to conk out on 3.10.8 but works on 3.10.7 and 3.10.6.
To demo issue I created a test database, replicated it to an aws s3 bucket, then restored the same under various .pyenv-versioned shells where I test whether I can read the database via the sqlite3 cli.
```sh
# create new shell with 3.11.3
litestream restore -o data/db.sqlite s3://mytestbucketxx/db
sqlite3 data/db.sqlite
# SQLite version 3.41.2 2023-03-22 11:56:21
# Enter "".help"" for usage hints.
# sqlite> .tables
# _litestream_lock _litestream_seq movie
# sqlite>
```
However this get me an `OperationalError` when reading via datasette:
Error on 3.11.3 and 3.10.8
```sh
datasette data/db.sqlite
```
```console
/tester/.venv/lib/python3.11/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
warnings.warn(""pkg_resources is deprecated as an API"", DeprecationWarning)
Traceback (most recent call last):
File ""/tester/.venv/bin/datasette"", line 8, in
sys.exit(cli())
^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1130, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 760, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 143, in wrapped
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 615, in serve
asyncio.get_event_loop().run_until_complete(check_databases(ds))
File ""/Users/mv/.pyenv/versions/3.11.3/lib/python3.11/asyncio/base_events.py"", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 660, in check_databases
await database.execute_fn(check_connection)
File ""/tester/.venv/lib/python3.11/site-packages/datasette/database.py"", line 213, in execute_fn
return await asyncio.get_event_loop().run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/mv/.pyenv/versions/3.11.3/lib/python3.11/concurrent/futures/thread.py"", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/datasette/database.py"", line 211, in in_thread
return fn(conn)
^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/datasette/utils/__init__.py"", line 951, in check_connection
for r in conn.execute(
^^^^^^^^^^^^^
sqlite3.OperationalError: unable to open database file
```
Works on 3.10.7, 3.10.6
```sh
# create new shell with 3.10.7 / 3.10.6
litestream restore -o data/db.sqlite s3://mytestbucketxx/db
datasette data/db.sqlite
# ...
# INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)
```
In both scenarios, the only dependencies were the pinned python version and the latest Datasette version 0.64.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2067/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,