{"html_url": "https://github.com/simonw/datasette/issues/1744#issuecomment-1129184908", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1744", "id": 1129184908, "node_id": "IC_kwDOBm6k_c5DTf6M", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-05-17T18:25:57Z", "updated_at": "2022-05-17T18:25:57Z", "author_association": "OWNER", "body": "I knocked out a quick prototype of this and it worked!\r\n\r\n    datasette ~/Library/Application\\ Support/Google/Chrome/Default/History --nolock\r\n\r\nHere's the prototype diff:\r\n\r\n```diff\r\ndiff --git a/datasette/app.py b/datasette/app.py\r\nindex b7b8437..f43700d 100644\r\n--- a/datasette/app.py\r\n+++ b/datasette/app.py\r\n@@ -213,6 +213,7 @@ class Datasette:\r\n         config_dir=None,\r\n         pdb=False,\r\n         crossdb=False,\r\n+        nolock=False,\r\n     ):\r\n         assert config_dir is None or isinstance(\r\n             config_dir, Path\r\n@@ -238,6 +239,7 @@ class Datasette:\r\n         self.databases = collections.OrderedDict()\r\n         self._refresh_schemas_lock = asyncio.Lock()\r\n         self.crossdb = crossdb\r\n+        self.nolock = nolock\r\n         if memory or crossdb or not self.files:\r\n             self.add_database(Database(self, is_memory=True), name=\"_memory\")\r\n         # memory_name is a random string so that each Datasette instance gets its own\r\ndiff --git a/datasette/cli.py b/datasette/cli.py\r\nindex 3c6e1b2..7e44665 100644\r\n--- a/datasette/cli.py\r\n+++ b/datasette/cli.py\r\n@@ -452,6 +452,11 @@ def uninstall(packages, yes):\r\n     is_flag=True,\r\n     help=\"Enable cross-database joins using the /_memory database\",\r\n )\r\n+@click.option(\r\n+    \"--nolock\",\r\n+    is_flag=True,\r\n+    help=\"Ignore locking and open locked files in read-only mode\",\r\n+)\r\n @click.option(\r\n     \"--ssl-keyfile\",\r\n     help=\"SSL key file\",\r\n@@ -486,6 +491,7 @@ def serve(\r\n     open_browser,\r\n     create,\r\n     crossdb,\r\n+    nolock,\r\n     ssl_keyfile,\r\n     ssl_certfile,\r\n     return_instance=False,\r\n@@ -545,6 +551,7 @@ def serve(\r\n         version_note=version_note,\r\n         pdb=pdb,\r\n         crossdb=crossdb,\r\n+        nolock=nolock,\r\n     )\r\n \r\n     # if files is a single directory, use that as config_dir=\r\ndiff --git a/datasette/database.py b/datasette/database.py\r\nindex 44d3266..fa55804 100644\r\n--- a/datasette/database.py\r\n+++ b/datasette/database.py\r\n@@ -89,6 +89,8 @@ class Database:\r\n         # mode=ro or immutable=1?\r\n         if self.is_mutable:\r\n             qs = \"?mode=ro\"\r\n+            if self.ds.nolock:\r\n+                qs += \"&nolock=1\"\r\n         else:\r\n             qs = \"?immutable=1\"\r\n         assert not (write and not self.is_mutable)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239008850, "label": "`--nolock` feature for opening locked databases"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1744#issuecomment-1129185356", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1744", "id": 1129185356, "node_id": "IC_kwDOBm6k_c5DTgBM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-05-17T18:26:26Z", "updated_at": "2022-05-17T18:26:26Z", "author_association": "OWNER", "body": "Not sure how to test this - I'd need to open my own lock against a database file somehow.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239008850, "label": "`--nolock` feature for opening locked databases"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1744#issuecomment-1129187486", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1744", "id": 1129187486, "node_id": "IC_kwDOBm6k_c5DTgie", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-05-17T18:28:49Z", "updated_at": "2022-05-17T18:28:49Z", "author_association": "OWNER", "body": "I think I do that with `fcntl.flock()`: https://docs.python.org/3/library/fcntl.html#fcntl.flock", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239008850, "label": "`--nolock` feature for opening locked databases"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1744#issuecomment-1129241873", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1744", "id": 1129241873, "node_id": "IC_kwDOBm6k_c5DTt0R", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-05-17T19:33:16Z", "updated_at": "2022-05-17T19:33:16Z", "author_association": "OWNER", "body": "I'm going to skip adding a test for this - the test logic would have to be pretty convoluted to exercise it properly, and it's a pretty minor and low-risk feature in the scheme of things.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239008850, "label": "`--nolock` feature for opening locked databases"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1744#issuecomment-1129241283", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1744", "id": 1129241283, "node_id": "IC_kwDOBm6k_c5DTtrD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-05-17T19:32:35Z", "updated_at": "2022-05-17T19:32:35Z", "author_association": "OWNER", "body": "I tried writing a test like this:\r\n\r\n```python\r\n@pytest.mark.parametrize(\"locked\", (True, False))\r\ndef test_locked_sqlite_db(tmp_path_factory, locked):\r\n    dir = tmp_path_factory.mktemp(\"test_locked_sqlite_db\")\r\n    test_db = str(dir / \"test.db\")\r\n    sqlite3.connect(test_db).execute(\"create table t (id integer primary key)\")\r\n    if locked:\r\n        fp = open(test_db, \"w\")\r\n        fcntl.lockf(fp.fileno(), fcntl.LOCK_EX)\r\n    runner = CliRunner()\r\n    result = runner.invoke(\r\n        cli,\r\n        [\r\n            \"serve\",\r\n            \"--memory\",\r\n            \"--get\",\r\n            \"/test\",\r\n        ],\r\n        catch_exceptions=False,\r\n    )\r\n```\r\nBut it didn't work, because the test runs in the same process - so taking an exclusive lock on that file didn't cause an error when the test later tried to access it via Datasette!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239008850, "label": "`--nolock` feature for opening locked databases"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1744#issuecomment-1129243427", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1744", "id": 1129243427, "node_id": "IC_kwDOBm6k_c5DTuMj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-05-17T19:35:02Z", "updated_at": "2022-05-17T19:35:02Z", "author_association": "OWNER", "body": "One thing to note is that the `datasette-copy-to-memory` plugin broke with a locked file, because it does this: https://github.com/simonw/datasette-copy-to-memory/blob/d541c18a78ae6f707a8f9b1e7fc4c020a9f68f2e/datasette_copy_to_memory/__init__.py#L27\r\n```python\r\ntmp.execute(\"ATTACH DATABASE ? AS _copy_from\", [db.path])\r\n```\r\nThat would need to use a URI filename too for it to work with locked files.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239008850, "label": "`--nolock` feature for opening locked databases"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1744#issuecomment-1129251699", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1744", "id": 1129251699, "node_id": "IC_kwDOBm6k_c5DTwNz", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-05-17T19:44:47Z", "updated_at": "2022-05-17T19:46:38Z", "author_association": "OWNER", "body": "Updated docs: https://docs.datasette.io/en/latest/getting_started.html#using-datasette-on-your-own-computer and https://docs.datasette.io/en/latest/cli-reference.html#datasette-serve-help", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239008850, "label": "`--nolock` feature for opening locked databases"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1745#issuecomment-1129252603", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1745", "id": 1129252603, "node_id": "IC_kwDOBm6k_c5DTwb7", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-05-17T19:45:51Z", "updated_at": "2022-05-17T19:45:51Z", "author_association": "OWNER", "body": "Now documented here: https://docs.datasette.io/en/latest/contributing.html#running-cog", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239080102, "label": "Documentation on running cog"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/425#issuecomment-1129332959", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/425", "id": 1129332959, "node_id": "IC_kwDOCGYnMM5DUEDf", "user": {"value": 102771161, "label": "McEazy2700"}, "created_at": "2022-05-17T21:27:02Z", "updated_at": "2022-05-17T21:27:02Z", "author_association": "NONE", "body": "Hi, I'm trying to deploy my site using elasticbeanstalk and I keep getting this same error :\r\ndeterministic=True requires SQLite 3.8.3 or higher\r\n\r\nI saw your previous solution that involves editing sqlite-utils/sqlite_utils/db.py file, but I'm curious as to how that will work in production.", "reactions": "{\"total_count\": 5, \"+1\": 5, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1203842656, "label": "`sqlite3.NotSupportedError`: deterministic=True requires SQLite 3.8.3 or higher"}, "performed_via_github_app": null}