{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065386352", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065386352, "node_id": "IC_kwDOCGYnMM4_gIFw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T18:41:37Z", "updated_at": "2022-03-11T18:41:37Z", "author_association": "OWNER", "body": "I like `add-generated-column` - feels very clear to me, and is a nice place for adding logic that checks if the DB version supports it or not and shows a useful error.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065389386", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065389386, "node_id": "IC_kwDOCGYnMM4_gI1K", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T18:42:53Z", "updated_at": "2022-03-11T21:40:51Z", "author_association": "OWNER", "body": "The Python API could be:\r\n```python\r\ndb[table_name].add_generated_column(\"field\", str, \"json_extract(data, '$.field')\", stored=True)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065402557", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065402557, "node_id": "IC_kwDOCGYnMM4_gMC9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T19:01:08Z", "updated_at": "2022-03-11T21:42:25Z", "author_association": "OWNER", "body": "Just spotted this in https://www.sqlite.org/gencol.html\r\n\r\n> The only functional difference is that one cannot add new STORED columns using the [ALTER TABLE ADD COLUMN](https://www.sqlite.org/lang_altertable.html#altertabaddcol) command. Only VIRTUAL columns can be added using ALTER TABLE.\r\n\r\nSo to add stored columns to an existing table we would need to use the `.transform()` trick. Which implies that this should actually be a capability of the various `.create()` methods, since transform works by creating a new table with those and then copying across the old data.\r\n\r\nHere's where `.transform()` calls `.create_table_sql()` under the hood:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/9388edf57aa15719095e3cf0952c1653cd070c9b/sqlite_utils/db.py#L1627-L1637", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065440445", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065440445, "node_id": "IC_kwDOCGYnMM4_gVS9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T19:52:15Z", "updated_at": "2022-03-11T19:52:15Z", "author_association": "OWNER", "body": "Two new parameters to `.create_table()` and friends:\r\n\r\n - `generated={...}` - generated column definitions\r\n - `generated_stored={...}` generated stored column definitions\r\n \r\nThese columns will be added at the end of the table, but you can use the `column_order=` parameter to apply a different order.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065458729", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065458729, "node_id": "IC_kwDOCGYnMM4_gZwp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T19:58:50Z", "updated_at": "2022-03-11T20:00:25Z", "author_association": "OWNER", "body": "I'm coming round to your suggestion to have this as extra arguments to `sqlite-utils add-column` now, especially since you also need to pass a column type.\r\n\r\nI'd like to come up with syntax for `sqlite-utils create-table` as well.\r\n\r\nhttps://sqlite-utils.datasette.io/en/stable/cli-reference.html#create-table\r\n\r\nMaybe extra `--generated-stored colname expression` (and `--generated`) options would work there.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065477258", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065477258, "node_id": "IC_kwDOCGYnMM4_geSK", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2022-03-11T20:14:59Z", "updated_at": "2022-03-11T20:14:59Z", "author_association": "CONTRIBUTOR", "body": "Good call on adding this to `create-table`, especially for stored columns. Having the stored/virtual split might make this tricky to implement, but I haven't gone any farther than thinking about what the CLI looks like. I'm going to try making the SQL side work first and figure that'll tell me more about what it needs.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065596417", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065596417, "node_id": "IC_kwDOCGYnMM4_g7YB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T22:30:15Z", "updated_at": "2022-03-11T22:30:15Z", "author_association": "OWNER", "body": "I tried it out in Jupyter and it works as advertised:\r\n\r\n<img width=\"1076\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/157980825-b362a162-b774-4fb5-8ec0-a7f539e62bd7.png\">\r\n\r\nIntrospection is a bit weird: there doesn't seem to be a way to introspect generated columns outside of parsing the stored SQL schema for the columns at the moment! And the `.columns` method doesn't return them at all:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/433813612ff9b4b501739fd7543bef0040dd51fe/sqlite_utils/db.py#L1207-L1213\r\n\r\nHere's why:\r\n\r\n```\r\n>>> db.execute(\"PRAGMA table_info('t')\").fetchall()\r\n[(0, 'body', 'TEXT', 0, None, 0)]\r\n>>> db.execute(\"PRAGMA table_xinfo('t')\").fetchall()\r\n[(0, 'body', 'TEXT', 0, None, 0, 0), (1, 'd', 'INT', 0, None, 0, 2)]\r\n```\r\nSo `table_xinfo()` is needed to get back columns including generated columns: https://www.sqlite.org/pragma.html#pragma_table_xinfo\r\n\r\n> **PRAGMA** *schema.***table_xinfo(***table-name***);**\r\n>\r\n> This pragma returns one row for each column in the named table, including [hidden columns](https://www.sqlite.org/vtab.html#hiddencol) in virtual tables. The output is the same as for [PRAGMA table_info](https://www.sqlite.org/pragma.html#pragma_table_info) except that hidden columns are shown rather than being omitted.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065597709", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065597709, "node_id": "IC_kwDOCGYnMM4_g7sN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T22:32:43Z", "updated_at": "2022-03-11T22:32:43Z", "author_association": "OWNER", "body": "Trying to figure out what that extra field in `table_info` compared to `table_xinfo` is:\r\n```\r\n>>> list(db.query(\"PRAGMA table_xinfo('t')\"))\r\n[{'cid': 0,\r\n  'name': 'body',\r\n  'type': 'TEXT',\r\n  'notnull': 0,\r\n  'dflt_value': None,\r\n  'pk': 0,\r\n  'hidden': 0},\r\n {'cid': 1,\r\n  'name': 'd',\r\n  'type': 'INT',\r\n  'notnull': 0,\r\n  'dflt_value': None,\r\n  'pk': 0,\r\n  'hidden': 2}]\r\n``\r\nPresumably `hidden` 0 v.s 2 v.s. other values has meaning.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}