html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502556111,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1502556111,IC_kwDOCGYnMM5Zjy_P,9599,simonw,2023-04-11T01:28:41Z,2023-04-11T01:28:41Z,OWNER,"Investigating this one now. The `sqlite-utils` test suite passes without errors on my Python 3.11.2 installation... but it fails with this error on a Python 3.9.6 installation. In the broken version's virtual environment directory I ran this: ``` cat pyvenv.cfg home = /Applications/Xcode.app/Contents/Developer/usr/bin implementation = CPython version_info = 3.9.6.final.0 virtualenv = 20.17.1 include-system-site-packages = false base-prefix = /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9 base-exec-prefix = /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9 base-executable = /Applications/Xcode.app/Contents/Developer/usr/bin/python3 ``` So it looks like the Xcode `python3` has ""defensive"" mode turned on for SQLite. As far as I can tell there's no way to turn it OFF again in Python. My virtual environment that DOES work has this: ``` home = /opt/homebrew/opt/python@3.11/bin implementation = CPython version_info = 3.11.2.final.0 virtualenv = 20.17.1 include-system-site-packages = false base-prefix = /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11 base-exec-prefix = /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11 base-executable = /opt/homebrew/opt/python@3.11/bin/python3.11 ``` So the Python 3.11 I installed through Homebrew doesn't exhibit this bug.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified, https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502557629,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1502557629,IC_kwDOCGYnMM5ZjzW9,9599,simonw,2023-04-11T01:30:12Z,2023-04-11T01:30:12Z,OWNER,"I'll ask on the SQLite forum if it's possible to toggle that mode on and off using regular SQL. My hunch is that it isn't. In which case `sqlite-utils` should at least know how to catch this error and display a much more readable error message, maybe with a link to further documentation. A utility function that can detect this mode would be really useful too. I'd probably have to do a test that tries to modify `sqlite_master` on a new in-memory database to catch if it's possible or not.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified, https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502559442,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1502559442,IC_kwDOCGYnMM5ZjzzS,9599,simonw,2023-04-11T01:32:30Z,2023-04-11T01:33:27Z,OWNER,"This seems to work: ```python import sqlite3 db = sqlite3.connect("":memory:"") db.executescript("""""" PRAGMA writable_schema = 1; UPDATE sqlite_master SET sql = 'CREATE TABLE [foos] (id integer primary key)'; PRAGMA writable_schema = 0; """""") ``` It succeeds on my Python 3.11 and raises the following exception on my broken Python 3.9: ``` sqlite3.OperationalError: table sqlite_master may not be modified ``` Removing the `PRAGMA writable_schema = 1;` causes the same exception to be raised on both Pythons.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified, https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1504245029,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1504245029,IC_kwDOCGYnMM5ZqPUl,9599,simonw,2023-04-11T23:13:41Z,2023-04-11T23:14:39Z,OWNER,"I also tested this against the current `ubuntu:latest` Docker image (on an M2 Mac), in Python 3.10 and 3.11: ``` docker run -it ubuntu:latest /bin/bash ``` Then in the container: ``` apt-get update apt-get install python3 python3 # pasted in the above recipe apt install software-properties-common add-apt-repository ppa:deadsnakes/ppa apt install python3.11 python3.11 # pasted it in again ``` In both cases the Python code did not raise an exception, which suggests that on Ubuntu those two Python versions do not have the defensive mode set.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified, https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502543165,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13,1502543165,IC_kwDODD6af85Zjv09,9599,simonw,2023-04-11T01:10:36Z,2023-04-11T01:11:47Z,MEMBER,"I just had that error myself on macOS while running the tests: ``` ERROR tests/test_save_checkin.py::test_tables - sqlite3.OperationalError: table sqlite_master may not be modified ERROR tests/test_save_checkin.py::test_venue - sqlite3.OperationalError: table sqlite_master may not be modified ERROR tests/test_save_checkin.py::test_event - sqlite3.OperationalError: table sqlite_master may not be modified ERROR tests/test_save_checkin.py::test_sticker - sqlite3.OperationalError: table sqlite_master may not be modified ERROR tests/test_save_checkin.py::test_likes - sqlite3.OperationalError: table sqlite_master may not be modified ERROR tests/test_save_checkin.py::test_with_ - sqlite3.OperationalError: table sqlite_master may not be modified ERROR tests/test_save_checkin.py::test_users - sqlite3.OperationalError: table sqlite_master may not be modified ERROR tests/test_save_checkin.py::test_photos - sqlite3.OperationalError: table sqlite_master may not be modified ERROR tests/test_save_checkin.py::test_posts - sqlite3.OperationalError: table sqlite_master may not be modified ERROR tests/test_save_checkin.py::test_view - sqlite3.OperationalError: table sqlite_master may not be modified ``` `pytest --pdb` shows it happening in the bit that adds foreign keys: ``` > /Users/simon/.local/share/virtualenvs/swarm-to-sqlite-daPW7yIJ/lib/python3.9/site-packages/sqlite_utils/db.py(1096)add_foreign_keys() -> cursor.execute( (Pdb) list 1096 >> cursor.execute( 1097 ""UPDATE sqlite_master SET sql = ? WHERE name = ?"", 1098 (new_sql, table_name), 1099 ) 1100 cursor.execute(""PRAGMA schema_version = %d"" % (schema_version + 1)) 1101 -> cursor.execute(""PRAGMA writable_schema = 0"") 1102 # Have to VACUUM outside the transaction to ensure .foreign_keys property 1103 # can see the newly created foreign key. 1104 self.vacuum() ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1373210675,fails before generating views. ERR: table sqlite_master may not be modified, https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502546045,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13,1502546045,IC_kwDODD6af85Zjwh9,9599,simonw,2023-04-11T01:14:50Z,2023-04-11T01:14:50Z,MEMBER,"Related: - https://github.com/simonw/sqlite-utils/issues/235","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1373210675,fails before generating views. ERR: table sqlite_master may not be modified, https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502629219,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13,1502629219,IC_kwDODD6af85ZkE1j,9599,simonw,2023-04-11T03:15:26Z,2023-04-11T03:15:26Z,MEMBER,"OK, I figured this out. Unfortunately it's an error that occurs on Python versions that have defensive mode turned on, and it doesn't look like there's a way to turn that mode off. See notes above.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1373210675,fails before generating views. ERR: table sqlite_master may not be modified, https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502629404,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13,1502629404,IC_kwDODD6af85ZkE4c,9599,simonw,2023-04-11T03:15:47Z,2023-04-11T03:46:17Z,MEMBER,"I think `swarm-to-sqlite` needs to avoid this error, maybe by setting up foreign keys in another way - or even by skipping foreign keys entirely on databases that don't support this kind of operation.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1373210675,fails before generating views. ERR: table sqlite_master may not be modified, https://github.com/simonw/datasette/issues/2057#issuecomment-1503832422,https://api.github.com/repos/simonw/datasette/issues/2057,1503832422,IC_kwDOBm6k_c5Zoqlm,9599,simonw,2023-04-11T17:42:57Z,2023-04-11T17:46:42Z,OWNER,"I ran this prompt against ChatGPT with the Browsing alpha: > ```python > if pkg_resources.resource_isdir(plugin.__name__, ""static""): > static_path = pkg_resources.resource_filename( > plugin.__name__, ""static"" > ) > if pkg_resources.resource_isdir(plugin.__name__, ""templates""): > templates_path = pkg_resources.resource_filename( > plugin.__name__, ""templates"" > ) > ``` > This code gives a deprecation warning in Python 3.11 - fix it It looked up the fix for me: And suggested: ```python import importlib.resources # Replace pkg_resources.resource_isdir with importlib.resources.files().is_file() if importlib.resources.files(plugin.__name__).joinpath(""static"").is_file(): static_path = importlib.resources.as_file( importlib.resources.files(plugin.__name__).joinpath(""static"") ) if importlib.resources.files(plugin.__name__).joinpath(""templates"").is_file(): templates_path = importlib.resources.as_file( importlib.resources.files(plugin.__name__).joinpath(""templates"") ) ``` This looks wrong to me - I would expect something like `is_directory()` not `is_file()` for telling if `static/` is a directory.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1503833906,https://api.github.com/repos/simonw/datasette/issues/2057,1503833906,IC_kwDOBm6k_c5Zoq8y,9599,simonw,2023-04-11T17:44:16Z,2023-04-11T17:45:45Z,OWNER,"Another prompt: > How to fix this: > > `pkg_resources.get_distribution(package).version` Response: ```python import importlib.metadata # Get the version number of the specified package package_version = importlib.metadata.version(package) ``` That seems to work: ```pycon >>> import importlib.metadata >>> importlib.metadata.version(""datasette"") '0.64.2' >>> importlib.metadata.version(""pluggy"") '1.0.0' >>> importlib.metadata.version(""not-a-package"") ... importlib.metadata.PackageNotFoundError: No package metadata was found for not-a-package ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1503838640,https://api.github.com/repos/simonw/datasette/issues/2057,1503838640,IC_kwDOBm6k_c5ZosGw,9599,simonw,2023-04-11T17:48:23Z,2023-04-11T17:48:23Z,OWNER,"> This looks wrong to me - I would expect something like `is_directory()` not `is_file()` for telling if `static/` is a directory. I was right about that: ```pycon >>> importlib.resources.files('datasette_graphql') PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql') >>> importlib.resources.files('datasette_graphql').joinpath(""static"") PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql/static') >>> p = importlib.resources.files('datasette_graphql').joinpath(""static"") >>> p PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql/static') >>> p.is_ p.is_absolute() p.is_char_device() p.is_fifo() p.is_mount() p.is_reserved() p.is_symlink() p.is_block_device() p.is_dir() p.is_file() p.is_relative_to( p.is_socket() >>> p.is_dir() True >>> p.is_file() False ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2058#issuecomment-1504291892,https://api.github.com/repos/simonw/datasette/issues/2058,1504291892,IC_kwDOBm6k_c5Zqaw0,9599,simonw,2023-04-11T23:58:45Z,2023-04-11T23:58:45Z,OWNER,"I thought it might relate to the ""defensive mode"" issue described here: - https://github.com/simonw/sqlite-utils/issues/235 But I have since determined that the Datasette official Docker image does NOT run anything in defensive mode, so I don't think it's related to that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1663399821,"500 ""attempt to write a readonly database"" error caused by ""PRAGMA schema_version""", https://github.com/simonw/datasette/issues/2058#issuecomment-1504292145,https://api.github.com/repos/simonw/datasette/issues/2058,1504292145,IC_kwDOBm6k_c5Zqa0x,9599,simonw,2023-04-11T23:58:59Z,2023-04-11T23:58:59Z,OWNER,Asked on the SQLite Forum if anyone has seen this before: https://sqlite.org/forum/forumpost/793a2ed75b,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1663399821,"500 ""attempt to write a readonly database"" error caused by ""PRAGMA schema_version""",