{"html_url": "https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502556111", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/235", "id": 1502556111, "node_id": "IC_kwDOCGYnMM5Zjy_P", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T01:28:41Z", "updated_at": "2023-04-11T01:28:41Z", "author_association": "OWNER", "body": "Investigating this one now.\r\n\r\nThe `sqlite-utils` test suite passes without errors on my Python 3.11.2 installation... but it fails with this error on a Python 3.9.6 installation.\r\n\r\nIn the broken version's virtual environment directory I ran this:\r\n```\r\ncat pyvenv.cfg \r\nhome = /Applications/Xcode.app/Contents/Developer/usr/bin\r\nimplementation = CPython\r\nversion_info = 3.9.6.final.0\r\nvirtualenv = 20.17.1\r\ninclude-system-site-packages = false\r\nbase-prefix = /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9\r\nbase-exec-prefix = /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9\r\nbase-executable = /Applications/Xcode.app/Contents/Developer/usr/bin/python3\r\n```\r\nSo it looks like the Xcode `python3` has \"defensive\" mode turned on for SQLite.\r\n\r\nAs far as I can tell there's no way to turn it OFF again in Python.\r\n\r\nMy virtual environment that DOES work has this:\r\n\r\n```\r\nhome = /opt/homebrew/opt/python@3.11/bin\r\nimplementation = CPython\r\nversion_info = 3.11.2.final.0\r\nvirtualenv = 20.17.1\r\ninclude-system-site-packages = false\r\nbase-prefix = /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11\r\nbase-exec-prefix = /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11\r\nbase-executable = /opt/homebrew/opt/python@3.11/bin/python3.11\r\n```\r\nSo the Python 3.11 I installed through Homebrew doesn't exhibit this bug.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 810618495, "label": "Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502557629", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/235", "id": 1502557629, "node_id": "IC_kwDOCGYnMM5ZjzW9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T01:30:12Z", "updated_at": "2023-04-11T01:30:12Z", "author_association": "OWNER", "body": "I'll ask on the SQLite forum if it's possible to toggle that mode on and off using regular SQL. My hunch is that it isn't.\r\n\r\nIn which case `sqlite-utils` should at least know how to catch this error and display a much more readable error message, maybe with a link to further documentation.\r\n\r\nA utility function that can detect this mode would be really useful too. I'd probably have to do a test that tries to modify `sqlite_master` on a new in-memory database to catch if it's possible or not.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 810618495, "label": "Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502559442", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/235", "id": 1502559442, "node_id": "IC_kwDOCGYnMM5ZjzzS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T01:32:30Z", "updated_at": "2023-04-11T01:33:27Z", "author_association": "OWNER", "body": "This seems to work:\r\n```python\r\nimport sqlite3\r\ndb = sqlite3.connect(\":memory:\")\r\ndb.executescript(\"\"\"\r\nPRAGMA writable_schema = 1;\r\nUPDATE sqlite_master SET sql = 'CREATE TABLE [foos] (id integer primary key)';\r\nPRAGMA writable_schema = 0;\r\n\"\"\")\r\n```\r\nIt succeeds on my Python 3.11 and raises the following exception on my broken Python 3.9:\r\n```\r\nsqlite3.OperationalError: table sqlite_master may not be modified\r\n```\r\nRemoving the `PRAGMA writable_schema = 1;` causes the same exception to be raised on both Pythons.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 810618495, "label": "Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1504245029", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/235", "id": 1504245029, "node_id": "IC_kwDOCGYnMM5ZqPUl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T23:13:41Z", "updated_at": "2023-04-11T23:14:39Z", "author_association": "OWNER", "body": "I also tested this against the current `ubuntu:latest` Docker image (on an M2 Mac), in Python 3.10 and 3.11:\r\n```\r\ndocker run -it ubuntu:latest /bin/bash\r\n```\r\nThen in the container:\r\n```\r\napt-get update\r\napt-get install python3\r\npython3\r\n# pasted in the above recipe\r\napt install software-properties-common\r\nadd-apt-repository ppa:deadsnakes/ppa\r\napt install python3.11\r\npython3.11 \r\n# pasted it in again\r\n```\r\nIn both cases the Python code did not raise an exception, which suggests that on Ubuntu those two Python versions do not have the defensive mode set.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 810618495, "label": "Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502543165", "issue_url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13", "id": 1502543165, "node_id": "IC_kwDODD6af85Zjv09", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T01:10:36Z", "updated_at": "2023-04-11T01:11:47Z", "author_association": "MEMBER", "body": "I just had that error myself on macOS while running the tests:\r\n```\r\nERROR tests/test_save_checkin.py::test_tables - sqlite3.OperationalError: table sqlite_master may not be modified\r\nERROR tests/test_save_checkin.py::test_venue - sqlite3.OperationalError: table sqlite_master may not be modified\r\nERROR tests/test_save_checkin.py::test_event - sqlite3.OperationalError: table sqlite_master may not be modified\r\nERROR tests/test_save_checkin.py::test_sticker - sqlite3.OperationalError: table sqlite_master may not be modified\r\nERROR tests/test_save_checkin.py::test_likes - sqlite3.OperationalError: table sqlite_master may not be modified\r\nERROR tests/test_save_checkin.py::test_with_ - sqlite3.OperationalError: table sqlite_master may not be modified\r\nERROR tests/test_save_checkin.py::test_users - sqlite3.OperationalError: table sqlite_master may not be modified\r\nERROR tests/test_save_checkin.py::test_photos - sqlite3.OperationalError: table sqlite_master may not be modified\r\nERROR tests/test_save_checkin.py::test_posts - sqlite3.OperationalError: table sqlite_master may not be modified\r\nERROR tests/test_save_checkin.py::test_view - sqlite3.OperationalError: table sqlite_master may not be modified\r\n```\r\n`pytest --pdb` shows it happening in the bit that adds foreign keys:\r\n```\r\n> /Users/simon/.local/share/virtualenvs/swarm-to-sqlite-daPW7yIJ/lib/python3.9/site-packages/sqlite_utils/db.py(1096)add_foreign_keys()\r\n-> cursor.execute(\r\n(Pdb) list\r\n1096 >>\t                cursor.execute(\r\n1097 \t                    \"UPDATE sqlite_master SET sql = ? WHERE name = ?\",\r\n1098 \t                    (new_sql, table_name),\r\n1099 \t                )\r\n1100 \t            cursor.execute(\"PRAGMA schema_version = %d\" % (schema_version + 1))\r\n1101 ->\t            cursor.execute(\"PRAGMA writable_schema = 0\")\r\n1102 \t        # Have to VACUUM outside the transaction to ensure .foreign_keys property\r\n1103 \t        # can see the newly created foreign key.\r\n1104 \t        self.vacuum()\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1373210675, "label": "fails before generating views. ERR:  table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502546045", "issue_url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13", "id": 1502546045, "node_id": "IC_kwDODD6af85Zjwh9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T01:14:50Z", "updated_at": "2023-04-11T01:14:50Z", "author_association": "MEMBER", "body": "Related:\r\n- https://github.com/simonw/sqlite-utils/issues/235", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1373210675, "label": "fails before generating views. ERR:  table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502629219", "issue_url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13", "id": 1502629219, "node_id": "IC_kwDODD6af85ZkE1j", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T03:15:26Z", "updated_at": "2023-04-11T03:15:26Z", "author_association": "MEMBER", "body": "OK, I figured this out. Unfortunately it's an error that occurs on Python versions that have defensive mode turned on, and it doesn't look like there's a way to turn that mode off. See notes above.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1373210675, "label": "fails before generating views. ERR:  table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502629404", "issue_url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13", "id": 1502629404, "node_id": "IC_kwDODD6af85ZkE4c", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T03:15:47Z", "updated_at": "2023-04-11T03:46:17Z", "author_association": "MEMBER", "body": "I think `swarm-to-sqlite` needs to avoid this error, maybe by setting up foreign keys in another way - or even by skipping foreign keys entirely on databases that don't support this kind of operation.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1373210675, "label": "fails before generating views. ERR:  table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1503832422", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1503832422, "node_id": "IC_kwDOBm6k_c5Zoqlm", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T17:42:57Z", "updated_at": "2023-04-11T17:46:42Z", "author_association": "OWNER", "body": "I ran this prompt against ChatGPT with the Browsing alpha:\r\n\r\n> ```python\r\n>                 if pkg_resources.resource_isdir(plugin.__name__, \"static\"):\r\n>                     static_path = pkg_resources.resource_filename(\r\n>                         plugin.__name__, \"static\"\r\n>                     )\r\n>                 if pkg_resources.resource_isdir(plugin.__name__, \"templates\"):\r\n>                     templates_path = pkg_resources.resource_filename(\r\n>                         plugin.__name__, \"templates\"\r\n>                     )\r\n> ```\r\n> This code gives a deprecation warning in Python 3.11 - fix it\r\n\r\nIt looked up the fix for me:\r\n\r\n<img width=\"632\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/231245259-af4d962c-e631-420b-8312-fd9657a23bbd.png\">\r\n\r\nAnd suggested:\r\n\r\n```python\r\nimport importlib.resources\r\n\r\n# Replace pkg_resources.resource_isdir with importlib.resources.files().is_file()\r\nif importlib.resources.files(plugin.__name__).joinpath(\"static\").is_file():\r\n    static_path = importlib.resources.as_file(\r\n        importlib.resources.files(plugin.__name__).joinpath(\"static\")\r\n    )\r\nif importlib.resources.files(plugin.__name__).joinpath(\"templates\").is_file():\r\n    templates_path = importlib.resources.as_file(\r\n        importlib.resources.files(plugin.__name__).joinpath(\"templates\")\r\n    )\r\n```\r\nThis looks wrong to me - I would expect something like `is_directory()` not `is_file()` for telling if `static/` is a directory.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1503833906", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1503833906, "node_id": "IC_kwDOBm6k_c5Zoq8y", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T17:44:16Z", "updated_at": "2023-04-11T17:45:45Z", "author_association": "OWNER", "body": "Another prompt:\r\n\r\n> How to fix this:\r\n>\r\n> `pkg_resources.get_distribution(package).version`\r\n\r\nResponse:\r\n\r\n<img width=\"641\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/231245559-dd943527-818a-45aa-9833-b9db8c7ca87e.png\">\r\n\r\n```python\r\nimport importlib.metadata\r\n\r\n# Get the version number of the specified package\r\npackage_version = importlib.metadata.version(package)\r\n```\r\n\r\nThat seems to work:\r\n\r\n```pycon\r\n>>> import importlib.metadata\r\n>>> importlib.metadata.version(\"datasette\")\r\n'0.64.2'\r\n>>> importlib.metadata.version(\"pluggy\")\r\n'1.0.0'\r\n>>> importlib.metadata.version(\"not-a-package\")\r\n...\r\nimportlib.metadata.PackageNotFoundError: No package metadata was found for not-a-package\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1503838640", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1503838640, "node_id": "IC_kwDOBm6k_c5ZosGw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T17:48:23Z", "updated_at": "2023-04-11T17:48:23Z", "author_association": "OWNER", "body": "> This looks wrong to me - I would expect something like `is_directory()` not `is_file()` for telling if `static/` is a directory.\r\n\r\nI was right about that:\r\n\r\n```pycon\r\n>>> importlib.resources.files('datasette_graphql')\r\nPosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql')\r\n>>> importlib.resources.files('datasette_graphql').joinpath(\"static\")\r\nPosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql/static')\r\n>>> p = importlib.resources.files('datasette_graphql').joinpath(\"static\")\r\n>>> p\r\nPosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql/static')\r\n>>> p.is_\r\np.is_absolute()     p.is_char_device()  p.is_fifo()         p.is_mount()        p.is_reserved()     p.is_symlink()     \r\np.is_block_device() p.is_dir()          p.is_file()         p.is_relative_to(   p.is_socket()      \r\n>>> p.is_dir()\r\nTrue\r\n>>> p.is_file()\r\nFalse\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2058#issuecomment-1504291892", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2058", "id": 1504291892, "node_id": "IC_kwDOBm6k_c5Zqaw0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T23:58:45Z", "updated_at": "2023-04-11T23:58:45Z", "author_association": "OWNER", "body": "I thought it might relate to the \"defensive mode\" issue described here:\r\n\r\n- https://github.com/simonw/sqlite-utils/issues/235\r\n\r\nBut I have since determined that the Datasette official Docker image does NOT run anything in defensive mode, so I don't think it's related to that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1663399821, "label": "500 \"attempt to write a readonly database\" error caused by \"PRAGMA schema_version\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2058#issuecomment-1504292145", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2058", "id": 1504292145, "node_id": "IC_kwDOBm6k_c5Zqa0x", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-11T23:58:59Z", "updated_at": "2023-04-11T23:58:59Z", "author_association": "OWNER", "body": "Asked on the SQLite Forum if anyone has seen this before: https://sqlite.org/forum/forumpost/793a2ed75b", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1663399821, "label": "500 \"attempt to write a readonly database\" error caused by \"PRAGMA schema_version\""}, "performed_via_github_app": null}