html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/sqlite-utils/issues/413#issuecomment-1065245831,https://api.github.com/repos/simonw/sqlite-utils/issues/413,1065245831,IC_kwDOCGYnMM4_flyH,9599,simonw,2022-03-11T15:59:14Z,2022-03-11T15:59:14Z,OWNER,"Hint from https://twitter.com/AdamChainz/status/1502311047612575745 > Try: > > `autodoc_typehints = 'description'` > > For a list-of-arguments format > > https://sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_typehints","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166587040,Display autodoc type information more legibly, https://github.com/simonw/sqlite-utils/issues/413#issuecomment-1065247619,https://api.github.com/repos/simonw/sqlite-utils/issues/413,1065247619,IC_kwDOCGYnMM4_fmOD,9599,simonw,2022-03-11T16:01:20Z,2022-03-11T16:01:20Z,OWNER,"Definitely an improvement! It does highlight that I'm not currently using the `:param XXX: description` syntax though, which should move my descriptions of each parameter into that generated list.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166587040,Display autodoc type information more legibly, https://github.com/simonw/sqlite-utils/issues/413#issuecomment-1065249883,https://api.github.com/repos/simonw/sqlite-utils/issues/413,1065249883,IC_kwDOCGYnMM4_fmxb,9599,simonw,2022-03-11T16:03:35Z,2022-03-11T16:03:35Z,OWNER,"Applying this change fixes that: ```diff diff --git a/sqlite_utils/db.py b/sqlite_utils/db.py index 3bc528f..2a79711 100644 --- a/sqlite_utils/db.py +++ b/sqlite_utils/db.py @@ -2293,18 +2293,18 @@ class Table(Queryable): """""" Apply conversion function ``fn`` to every value in the specified columns. - - ``columns`` - a single column or list of string column names to convert. - - ``fn`` - a callable that takes a single argument, ``value``, and returns it converted. - - ``output`` - optional string column name to write the results to (defaults to the input column). - - ``output_type`` - if the output column needs to be created, this is the type that will be used + :param columns: a single column or list of string column names to convert. + :param fn: a callable that takes a single argument, ``value``, and returns it converted. + :param output: optional string column name to write the results to (defaults to the input column). + :param output_type: if the output column needs to be created, this is the type that will be used for the new column. - - ``drop`` - boolean, should the original column be dropped once the conversion is complete? - - ``multi`` - boolean, if ``True`` the return value of ``fn(value)`` will be expected to be a + :param drop: boolean, should the original column be dropped once the conversion is complete? + :param multi: boolean, if ``True`` the return value of ``fn(value)`` will be expected to be a dictionary, and new columns will be created for each key of that dictionary. - - ``where`` - a SQL fragment to use as a ``WHERE`` clause to limit the rows to which the conversion + :param where: a SQL fragment to use as a ``WHERE`` clause to limit the rows to which the conversion is applied, for example ``age > ?`` or ``age > :age``. - - ``where_args`` - a list of arguments (if using ``?``) or a dictionary (if using ``:age``). - - ``show_progress`` - boolean, should a progress bar be displayed? + :param where_args: a list of arguments (if using ``?``) or a dictionary (if using ``:age``). + :param show_progress: boolean, should a progress bar be displayed? See :ref:`python_api_convert`. """""" ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166587040,Display autodoc type information more legibly, https://github.com/simonw/datasette/issues/1634#issuecomment-1065334891,https://api.github.com/repos/simonw/datasette/issues/1634,1065334891,IC_kwDOBm6k_c4_f7hr,208018,dholth,2022-03-11T17:38:08Z,2022-03-11T17:38:08Z,NONE,I noticed the image was large when using fly. Is it possible to use a -slim base?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1131295060,Update Dockerfile generated by `datasette publish`, https://github.com/simonw/sqlite-utils/issues/413#issuecomment-1065345515,https://api.github.com/repos/simonw/sqlite-utils/issues/413,1065345515,IC_kwDOCGYnMM4_f-Hr,9599,simonw,2022-03-11T17:52:22Z,2022-03-11T17:52:22Z,OWNER,"Well this is a huge improvement! https://sqlite-utils.datasette.io/en/latest/reference.html#sqlite_utils.db.Table.insert I'm not crazy about the `extracts=` thing though - I wonder if there's a neat way to customize that to be less verbose?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166587040,Display autodoc type information more legibly, https://github.com/simonw/sqlite-utils/issues/413#issuecomment-1065357081,https://api.github.com/repos/simonw/sqlite-utils/issues/413,1065357081,IC_kwDOCGYnMM4_gA8Z,9599,simonw,2022-03-11T18:07:10Z,2022-03-11T18:07:10Z,OWNER,I'm really happy with this improvement.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166587040,Display autodoc type information more legibly, https://github.com/simonw/sqlite-utils/issues/414#issuecomment-1065377926,https://api.github.com/repos/simonw/sqlite-utils/issues/414,1065377926,IC_kwDOCGYnMM4_gGCG,9599,simonw,2022-03-11T18:34:05Z,2022-03-11T18:34:05Z,OWNER,"Two options: - Delete and recreate the release on GitHub, triggering it to be fixed on Read The Docs (as the `stable` version) - but cancel the push to PyPI, since that platform doesn't allow package versions to be over-written and in this case since the changelog file isn't included in the PyPI package there should be no change at all - Push a 3.25.2 release. I'm going to try and do that first option.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166731361,I forgot to include the changelog in the 3.25.1 release, https://github.com/simonw/sqlite-utils/issues/414#issuecomment-1065378902,https://api.github.com/repos/simonw/sqlite-utils/issues/414,1065378902,IC_kwDOCGYnMM4_gGRW,9599,simonw,2022-03-11T18:35:26Z,2022-03-11T18:35:26Z,OWNER,I deleted both the release and the tag from GitHub using the web interface.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166731361,I forgot to include the changelog in the 3.25.1 release, https://github.com/simonw/sqlite-utils/issues/414#issuecomment-1065379528,https://api.github.com/repos/simonw/sqlite-utils/issues/414,1065379528,IC_kwDOCGYnMM4_gGbI,9599,simonw,2022-03-11T18:36:17Z,2022-03-11T18:36:17Z,OWNER,"I created a new tag and release: https://github.com/simonw/sqlite-utils/releases/tag/3.25.1 And I cancelled the publish workflow: https://github.com/simonw/sqlite-utils/actions/runs/1970415399","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166731361,I forgot to include the changelog in the 3.25.1 release, https://github.com/simonw/sqlite-utils/issues/414#issuecomment-1065380286,https://api.github.com/repos/simonw/sqlite-utils/issues/414,1065380286,IC_kwDOCGYnMM4_gGm-,9599,simonw,2022-03-11T18:37:23Z,2022-03-11T18:37:23Z,OWNER,"On ReadTheDocs that triggered a new `stable` build but it didn't seem to trigger a new build of `3.25.1`: https://readthedocs.org/projects/sqlite-utils/builds/ ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166731361,I forgot to include the changelog in the 3.25.1 release, https://github.com/simonw/sqlite-utils/issues/414#issuecomment-1065381047,https://api.github.com/repos/simonw/sqlite-utils/issues/414,1065381047,IC_kwDOCGYnMM4_gGy3,9599,simonw,2022-03-11T18:38:27Z,2022-03-11T18:38:27Z,OWNER,"OK that fixed it here: https://sqlite-utils.datasette.io/en/stable/changelog.html I'm going to trigger a rebuild of `3.25.1` too: ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166731361,I forgot to include the changelog in the 3.25.1 release, https://github.com/simonw/sqlite-utils/issues/414#issuecomment-1065382145,https://api.github.com/repos/simonw/sqlite-utils/issues/414,1065382145,IC_kwDOCGYnMM4_gHEB,9599,simonw,2022-03-11T18:39:05Z,2022-03-11T18:39:05Z,OWNER,"https://sqlite-utils.datasette.io/en/3.25.1/changelog.html is currently wrong: ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166731361,I forgot to include the changelog in the 3.25.1 release, https://github.com/simonw/sqlite-utils/issues/414#issuecomment-1065384183,https://api.github.com/repos/simonw/sqlite-utils/issues/414,1065384183,IC_kwDOCGYnMM4_gHj3,9599,simonw,2022-03-11T18:40:39Z,2022-03-11T18:40:39Z,OWNER,"That fixed it: ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1166731361,I forgot to include the changelog in the 3.25.1 release, https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065386352,https://api.github.com/repos/simonw/sqlite-utils/issues/411,1065386352,IC_kwDOCGYnMM4_gIFw,9599,simonw,2022-03-11T18:41:37Z,2022-03-11T18:41:37Z,OWNER,"I like `add-generated-column` - feels very clear to me, and is a nice place for adding logic that checks if the DB version supports it or not and shows a useful error.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1160034488,Support for generated columns, https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065389386,https://api.github.com/repos/simonw/sqlite-utils/issues/411,1065389386,IC_kwDOCGYnMM4_gI1K,9599,simonw,2022-03-11T18:42:53Z,2022-03-11T21:40:51Z,OWNER,"The Python API could be: ```python db[table_name].add_generated_column(""field"", str, ""json_extract(data, '$.field')"", stored=True) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1160034488,Support for generated columns, https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065402557,https://api.github.com/repos/simonw/sqlite-utils/issues/411,1065402557,IC_kwDOCGYnMM4_gMC9,9599,simonw,2022-03-11T19:01:08Z,2022-03-11T21:42:25Z,OWNER,"Just spotted this in https://www.sqlite.org/gencol.html > The only functional difference is that one cannot add new STORED columns using the [ALTER TABLE ADD COLUMN](https://www.sqlite.org/lang_altertable.html#altertabaddcol) command. Only VIRTUAL columns can be added using ALTER TABLE. So to add stored columns to an existing table we would need to use the `.transform()` trick. Which implies that this should actually be a capability of the various `.create()` methods, since transform works by creating a new table with those and then copying across the old data. Here's where `.transform()` calls `.create_table_sql()` under the hood: https://github.com/simonw/sqlite-utils/blob/9388edf57aa15719095e3cf0952c1653cd070c9b/sqlite_utils/db.py#L1627-L1637","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1160034488,Support for generated columns, https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065440445,https://api.github.com/repos/simonw/sqlite-utils/issues/411,1065440445,IC_kwDOCGYnMM4_gVS9,9599,simonw,2022-03-11T19:52:15Z,2022-03-11T19:52:15Z,OWNER,"Two new parameters to `.create_table()` and friends: - `generated={...}` - generated column definitions - `generated_stored={...}` generated stored column definitions These columns will be added at the end of the table, but you can use the `column_order=` parameter to apply a different order.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1160034488,Support for generated columns, https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065458729,https://api.github.com/repos/simonw/sqlite-utils/issues/411,1065458729,IC_kwDOCGYnMM4_gZwp,9599,simonw,2022-03-11T19:58:50Z,2022-03-11T20:00:25Z,OWNER,"I'm coming round to your suggestion to have this as extra arguments to `sqlite-utils add-column` now, especially since you also need to pass a column type. I'd like to come up with syntax for `sqlite-utils create-table` as well. https://sqlite-utils.datasette.io/en/stable/cli-reference.html#create-table Maybe extra `--generated-stored colname expression` (and `--generated`) options would work there.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1160034488,Support for generated columns, https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065477258,https://api.github.com/repos/simonw/sqlite-utils/issues/411,1065477258,IC_kwDOCGYnMM4_geSK,25778,eyeseast,2022-03-11T20:14:59Z,2022-03-11T20:14:59Z,CONTRIBUTOR,"Good call on adding this to `create-table`, especially for stored columns. Having the stored/virtual split might make this tricky to implement, but I haven't gone any farther than thinking about what the CLI looks like. I'm going to try making the SQL side work first and figure that'll tell me more about what it needs.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1160034488,Support for generated columns, https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065596417,https://api.github.com/repos/simonw/sqlite-utils/issues/411,1065596417,IC_kwDOCGYnMM4_g7YB,9599,simonw,2022-03-11T22:30:15Z,2022-03-11T22:30:15Z,OWNER,"I tried it out in Jupyter and it works as advertised: Introspection is a bit weird: there doesn't seem to be a way to introspect generated columns outside of parsing the stored SQL schema for the columns at the moment! And the `.columns` method doesn't return them at all: https://github.com/simonw/sqlite-utils/blob/433813612ff9b4b501739fd7543bef0040dd51fe/sqlite_utils/db.py#L1207-L1213 Here's why: ``` >>> db.execute(""PRAGMA table_info('t')"").fetchall() [(0, 'body', 'TEXT', 0, None, 0)] >>> db.execute(""PRAGMA table_xinfo('t')"").fetchall() [(0, 'body', 'TEXT', 0, None, 0, 0), (1, 'd', 'INT', 0, None, 0, 2)] ``` So `table_xinfo()` is needed to get back columns including generated columns: https://www.sqlite.org/pragma.html#pragma_table_xinfo > **PRAGMA** *schema.***table_xinfo(***table-name***);** > > This pragma returns one row for each column in the named table, including [hidden columns](https://www.sqlite.org/vtab.html#hiddencol) in virtual tables. The output is the same as for [PRAGMA table_info](https://www.sqlite.org/pragma.html#pragma_table_info) except that hidden columns are shown rather than being omitted.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1160034488,Support for generated columns, https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065597709,https://api.github.com/repos/simonw/sqlite-utils/issues/411,1065597709,IC_kwDOCGYnMM4_g7sN,9599,simonw,2022-03-11T22:32:43Z,2022-03-11T22:32:43Z,OWNER,"Trying to figure out what that extra field in `table_info` compared to `table_xinfo` is: ``` >>> list(db.query(""PRAGMA table_xinfo('t')"")) [{'cid': 0, 'name': 'body', 'type': 'TEXT', 'notnull': 0, 'dflt_value': None, 'pk': 0, 'hidden': 0}, {'cid': 1, 'name': 'd', 'type': 'INT', 'notnull': 0, 'dflt_value': None, 'pk': 0, 'hidden': 2}] `` Presumably `hidden` 0 v.s 2 v.s. other values has meaning.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1160034488,Support for generated columns,