{"html_url": "https://github.com/simonw/datasette/issues/283#issuecomment-855369819", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/283", "id": 855369819, "node_id": "MDEyOklzc3VlQ29tbWVudDg1NTM2OTgxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-06T09:40:18Z", "updated_at": "2021-06-06T09:40:18Z", "author_association": "OWNER", "body": "> One note on using this pragma I got an error on starting datasette `no such table: pragma_database_list`.\r\n> \r\n> I diagnosed this to an older version of sqlite3 (3.14.2) and upgrading to a newer version (3.34.2) fixed the issue.\r\n\r\nThat issue is fixed in #1276.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 325958506, "label": "Support cross-database joins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/394#issuecomment-642522285", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/394", "id": 642522285, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjUyMjI4NQ==", "user": {"value": 58298410, "label": "LVerneyPEReN"}, "created_at": "2020-06-11T09:15:19Z", "updated_at": "2020-06-11T09:15:19Z", "author_association": "NONE", "body": "Hi @wragge,\r\n\r\nThis looks great, thanks for the share! I refactored it into a self-contained function, binding on a random available TCP port (multi-user context). I am using subprocess API directly since the `%run` magic was leaving defunct process behind :/\r\n\r\n![image](https://user-images.githubusercontent.com/58298410/84367566-b5d0d500-abd4-11ea-96e2-f5c05a28e506.png)\r\n\r\n```python\r\nimport socket\r\n\r\nfrom signal import SIGINT\r\nfrom subprocess import Popen, PIPE\r\n\r\nfrom IPython.display import display, HTML\r\nfrom notebook.notebookapp import list_running_servers\r\n\r\n\r\ndef get_free_tcp_port():\r\n \"\"\"\r\n Get a free TCP port.\r\n \"\"\"\r\n tcp = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\r\n tcp.bind(('', 0))\r\n _, port = tcp.getsockname()\r\n tcp.close()\r\n return port\r\n\r\n\r\ndef datasette(database):\r\n \"\"\"\r\n Run datasette on an SQLite database.\r\n \"\"\"\r\n # Get current running servers\r\n servers = list_running_servers()\r\n\r\n # Get the current base url\r\n base_url = next(servers)['base_url']\r\n\r\n # Get a free port\r\n port = get_free_tcp_port()\r\n\r\n # Create a base url for Datasette suing the proxy path\r\n proxy_url = f'{base_url}proxy/absolute/{port}/'\r\n\r\n # Display a link to Datasette\r\n display(HTML(f'
View Datasette (Click on the stop button to close the Datasette server)
'))\r\n\r\n # Launch Datasette\r\n with Popen(\r\n [\r\n 'python', '-m', 'datasette', '--',\r\n database,\r\n '--port', str(port),\r\n '--config', f'base_url:{proxy_url}'\r\n ],\r\n stdout=PIPE,\r\n stderr=PIPE,\r\n bufsize=1,\r\n universal_newlines=True\r\n ) as p:\r\n print(p.stdout.readline(), end='')\r\n while True:\r\n try:\r\n line = p.stderr.readline()\r\n if not line:\r\n break\r\n print(line, end='')\r\n exit_code = p.poll()\r\n except KeyboardInterrupt:\r\n p.send_signal(SIGINT)\r\n```\r\n\r\nIdeally, I'd like some extra magic to notify users when they are leaving the closing the notebook tab and make them terminate the running datasette processes. I'll be looking for it.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 396212021, "label": "base_url configuration setting"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/26#issuecomment-501541902", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/26", "id": 501541902, "node_id": "MDEyOklzc3VlQ29tbWVudDUwMTU0MTkwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-13T04:15:22Z", "updated_at": "2019-06-13T16:55:42Z", "author_association": "OWNER", "body": "So maybe something like this:\r\n```\r\ncurl https://api.github.com/repos/simonw/datasette/pulls?state=all | \\\r\n sqlite-utils insert git.db pulls - \\\r\n --flatten=base \\\r\n --flatten=head \\\r\n --extract=user:users:id \\\r\n --extract=head_repo.license:licenses:key \\\r\n --extract=head_repo.owner:users \\\r\n --extract=head_repo\r\n --extract=base_repo.license:licenses:key \\\r\n --extract=base_repo.owner:users \\\r\n --extract=base_repo\r\n```\r\nIs the order of those nested `--extract` lines significant I wonder? It would be nice if the order didn't matter and the code figured out the right execution plan on its own.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 455486286, "label": "Mechanism for turning nested JSON into foreign keys / many-to-many"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/509#issuecomment-749749948", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/509", "id": 749749948, "node_id": "MDEyOklzc3VlQ29tbWVudDc0OTc0OTk0OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-12-22T20:03:10Z", "updated_at": "2020-12-22T20:03:10Z", "author_association": "OWNER", "body": "If you open multiple files with the same filename, e.g. like this:\r\n\r\n datasette fixtures.db templates/fixtures.db plugins/fixtures.db\r\n\r\nYou'll now get this:\r\n\r\n\r\n", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 456568880, "label": "Support opening multiple databases with the same stem"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/526#issuecomment-1259693536", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/526", "id": 1259693536, "node_id": "IC_kwDOBm6k_c5LFWXg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-27T15:42:55Z", "updated_at": "2022-09-27T15:42:55Z", "author_association": "OWNER", "body": "It's interesting to note WHY the time limit works against this so well.\r\n\r\nThe time limit as-implemented looks like this:\r\n\r\nhttps://github.com/simonw/datasette/blob/5f9f567acbc58c9fcd88af440e68034510fb5d2b/datasette/utils/__init__.py#L181-L201\r\n\r\nThe key here is `conn.set_progress_handler(handler, n)` - which specifies that the handler function should be called every `n` SQLite operations.\r\n\r\nThe handler function then checks to see if too much time has transpired and conditionally cancels the query.\r\n\r\nThis also doubles up as a \"maximum number of operations\" guard, which is what's happening when you attempt to fetch an infinite number of rows from an infinite table.\r\n\r\nThat limit code could even be extended to say \"exit the query after either 5s or 50,000,000 operations\".\r\n\r\nI don't think that's necessary though.\r\n\r\nTo be honest I'm having trouble with the idea of dropping `max_returned_rows` mainly because what Datasette does (allow arbitrary untrusted SQL queries) is dangerous, so I've designed in multiple redundant defence-in-depth mechanisms right from the start.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459882902, "label": "Stream all results for arbitrary SQL and canned queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/594#issuecomment-547373739", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/594", "id": 547373739, "node_id": "MDEyOklzc3VlQ29tbWVudDU0NzM3MzczOQ==", "user": {"value": 2680980, "label": "willingc"}, "created_at": "2019-10-29T11:21:52Z", "updated_at": "2019-10-29T11:21:52Z", "author_association": "NONE", "body": "Just an FYI for folks wishing to run datasette with Python 3.8, I was able to successfully use datasette with the following in a virtual environment:\r\n\r\n```\r\npip install uvloop==0.14.0rc1\r\npip install uvicorn==0.9.1\r\n```\r\n", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 506297048, "label": "upgrade to uvicorn-0.9 to be Python-3.8 friendly"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/594#issuecomment-552276247", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/594", "id": 552276247, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MjI3NjI0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-11T03:13:00Z", "updated_at": "2019-11-11T03:13:00Z", "author_association": "OWNER", "body": "#622", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 506297048, "label": "upgrade to uvicorn-0.9 to be Python-3.8 friendly"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-797158641", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 797158641, "node_id": "MDEyOklzc3VlQ29tbWVudDc5NzE1ODY0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-12T00:59:49Z", "updated_at": "2021-03-12T00:59:49Z", "author_association": "OWNER", "body": "> Challenge: what's the equivalent for PostgreSQL of opening a database in read only mode? Will I have to talk users through creating read only credentials?\r\n\r\nIt looks like the answer to this is yes - I'll need users to setup read-only credentials. Here's a TIL about that: https://til.simonwillison.net/postgresql/read-only-postgresql-user", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/716#issuecomment-609455243", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/716", "id": 609455243, "node_id": "MDEyOklzc3VlQ29tbWVudDYwOTQ1NTI0Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-05T17:47:33Z", "updated_at": "2020-04-05T17:47:33Z", "author_association": "OWNER", "body": "You start `git bisect` by giving it a known bad commit and a known good one:\r\n```\r\ngit bisect start master 286ed28 \r\n```\r\nThen you tell it to start running your script:\r\n```\r\ngit bisect run python ../datasette-issue-716/check_view_name.py\r\n```\r\nHere's what I got:\r\n```\r\n(datasette) ~/Dropbox/Development/datasette $ git bisect start master 286ed28\r\nBisecting: 30 revisions left to test after this (roughly 5 steps)\r\n[dc80e779a2e708b2685fc641df99e6aae9ad6f97] Handle scope path if it is a string\r\n(datasette) ~/Dropbox/Development/datasette $ git bisect run python ../datasette-issue-716/check_view_name.py\r\nrunning python ../datasette-issue-716/check_view_name.py\r\nTraceback (most recent call last):\r\n...\r\nBisecting: 15 revisions left to test after this (roughly 4 steps)\r\n[7c6a9c35299f251f9abfb03fd8e85143e4361709] Better tests for prepare_connection() plugin hook, refs #678\r\nrunning python ../datasette-issue-716/check_view_name.py\r\nTraceback (most recent call last):\r\n...\r\nBisecting: 7 revisions left to test after this (roughly 3 steps)\r\n[0091dfe3e5a3db94af8881038d3f1b8312bb857d] More reliable tie-break ordering for facet results\r\nrunning python ../datasette-issue-716/check_view_name.py\r\nTraceback (most recent call last):\r\n...\r\nBisecting: 3 revisions left to test after this (roughly 2 steps)\r\n[ce12244037b60ba0202c814871218c1dab38d729] Release notes for 0.35\r\nrunning python ../datasette-issue-716/check_view_name.py\r\nTraceback (most recent call last):\r\n...\r\nBisecting: 0 revisions left to test after this (roughly 1 step)\r\n[4d7dae9eb75e5430c3ee3c369bb5cd9ba0a148bc] Added a bunch more plugins to the Ecosystem page\r\nrunning python ../datasette-issue-716/check_view_name.py\r\nTraceback (most recent call last):\r\n...\r\n70b915fb4bc214f9d064179f87671f8a378aa127 is the first bad commit\r\ncommit 70b915fb4bc214f9d064179f87671f8a378aa127\r\nAuthor: Simon Willison