{"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-585870836", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 585870836, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NTg3MDgzNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-13T17:19:23Z", "updated_at": "2020-02-13T17:19:37Z", "author_association": "OWNER", "body": "I'm excited about https://github.com/MagicStack/asyncpg for this - it's a true async PostgreSQL library (my SQLite queries run in a threadpool right now) with extremely impressive performance benchmarks, from the team behind `uvloop`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-585872538", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 585872538, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NTg3MjUzOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-13T17:22:54Z", "updated_at": "2020-02-13T17:22:54Z", "author_association": "OWNER", "body": "A couple of things I'd like to support:\r\n- The same datasette instance can have both PostgreSQL and SQLite databases attached to it, and both types will be listed on the homepage.\r\n- The full test suite runs against both SQLite and PostgreSQL, with as few changes as possible (maybe a few `skipIf()` decorators for features not available on both). Probably easiest do do this with some kind of flag - e.g. `pytest --database=sqlite` v.s. `pytest --database=postgresql`\r\n\r\nI can implement that with this in `conftest.py`:\r\n```python\r\ndef pytest_addoption(parser):\r\n parser.addoption(\"--database\", action=\"store\", default=\"sqlite\")\r\n```\r\nSee https://stackoverflow.com/questions/40880259/how-to-pass-arguments-in-pytest-by-command-line", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-585873401", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 585873401, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NTg3MzQwMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-13T17:24:38Z", "updated_at": "2020-02-13T17:24:38Z", "author_association": "OWNER", "body": "The biggest difference between the two will be around introspection. I searched the codebase for potential introspection queries, defined as `select ... from sqlite_master` and `pragma ...` statements.\r\n\r\n- 7 results for sqlite_master in database.py\r\n- 3 results in utils/__init__.py\r\n- (Ignoring the 4 in inspect.py)\r\n- 1 result for \u201cpragma table_info\u201d in app.py\r\n- 2 in utils/__init__.py\r\n- 2 results for \u201cpragma foreign_key_list\u201d in utils/__init__.py\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-585874699", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 585874699, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NTg3NDY5OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-13T17:26:58Z", "updated_at": "2020-02-13T17:27:13Z", "author_association": "OWNER", "body": "First page to get working: the database view page, which shows a list of tables.\r\n\r\nIn the code this is entirely implemented with calls to the `Database` class, which is a good starting point - I can drop in a new version of that class that talks to PostgreSQL instead:\r\n\r\nhttps://github.com/simonw/datasette/blob/0091dfe3e5a3db94af8881038d3f1b8312bb857d/datasette/views/database.py#L25-L66", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-585883109", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 585883109, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NTg4MzEwOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-13T17:44:08Z", "updated_at": "2020-02-13T17:44:08Z", "author_association": "OWNER", "body": "Introspecting columns in PostgreSQL:\r\n```\r\nIn [3]: sql = \"\"\"SELECT * \r\n ...: FROM information_schema.columns \r\n ...: WHERE table_schema = 'public' \r\n ...: AND table_name = 'blog_blogmark' \r\n ...: ;\"\"\" \r\n```\r\nEach column looks like this:\r\n```pythton\r\n{'table_catalog': 'simonwillisonblog',\r\n 'table_schema': 'public',\r\n 'table_name': 'blog_blogmark',\r\n 'column_name': 'id',\r\n 'ordinal_position': 1,\r\n 'column_default': \"nextval('blog_blogmark_id_seq'::regclass)\",\r\n 'is_nullable': 'NO',\r\n 'data_type': 'integer',\r\n 'character_maximum_length': None,\r\n 'character_octet_length': None,\r\n 'numeric_precision': 32,\r\n 'numeric_precision_radix': 2,\r\n 'numeric_scale': 0,\r\n 'datetime_precision': None,\r\n 'interval_type': None,\r\n 'interval_precision': None,\r\n 'character_set_catalog': None,\r\n 'character_set_schema': None,\r\n 'character_set_name': None,\r\n 'collation_catalog': None,\r\n 'collation_schema': None,\r\n 'collation_name': None,\r\n 'domain_catalog': None,\r\n 'domain_schema': None,\r\n 'domain_name': None,\r\n 'udt_catalog': 'simonwillisonblog',\r\n 'udt_schema': 'pg_catalog',\r\n 'udt_name': 'int4',\r\n 'scope_catalog': None,\r\n 'scope_schema': None,\r\n 'scope_name': None,\r\n 'maximum_cardinality': None,\r\n 'dtd_identifier': '1',\r\n 'is_self_referencing': 'NO',\r\n 'is_identity': 'NO',\r\n 'identity_generation': None,\r\n 'identity_start': None,\r\n 'identity_increment': None,\r\n 'identity_maximum': None,\r\n 'identity_minimum': None,\r\n 'identity_cycle': 'NO',\r\n 'is_generated': 'NEVER',\r\n 'generation_expression': None,\r\n 'is_updatable': 'YES'}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-585885242", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 585885242, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NTg4NTI0Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-13T17:48:27Z", "updated_at": "2020-02-13T17:48:27Z", "author_association": "OWNER", "body": "Finding out the primary keys for a table: https://wiki.postgresql.org/wiki/Retrieve_primary_key_columns", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-585885812", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 585885812, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NTg4NTgxMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-13T17:49:33Z", "updated_at": "2020-02-13T17:49:33Z", "author_association": "OWNER", "body": "```\r\nIn [12]: await conn.fetch(\"\"\"SELECT a.attname, format_type(a.atttypid, a.atttypmod) AS data_type \r\n ...: FROM pg_index i \r\n ...: JOIN pg_attribute a ON a.attrelid = i.indrelid \r\n ...: AND a.attnum = ANY(i.indkey) \r\n ...: WHERE i.indrelid = 'blog_blogmark'::regclass \r\n ...: AND i.indisprimary;\"\"\") \r\nOut[12]: []\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-585903240", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 585903240, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NTkwMzI0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-13T18:28:42Z", "updated_at": "2020-02-13T18:29:14Z", "author_association": "OWNER", "body": "Challenge: what's the equivalent for PostgreSQL of opening a database in read only mode? Will I have to talk users through creating read only credentials? Can I do this at runtime somehow? Can I detect if the connection has write permission and disable the arbitrary query feature?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-585962849", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 585962849, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NTk2Mjg0OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-13T20:44:18Z", "updated_at": "2020-02-13T20:46:53Z", "author_association": "OWNER", "body": "Got the database page working! It lists tables, their columns and their row count.\r\n\r\nGot the table page partially working! It can list rows.\r\n\r\nIt can't apply filters yet (because PostgreSQL `$1` parameters differ from SQLite `:blah` parameters) and faceting doesn't work because PostgreSQL requires that subqueries have an alias:\r\n```\r\nsubquery in FROM must have an alias HINT: For example, FROM (SELECT ...) [AS] foo.\r\n```\r\nStill a pretty promising start though!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null}