html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-807642041,https://api.github.com/repos/simonw/sqlite-utils/issues/251,807642041,MDEyOklzc3VlQ29tbWVudDgwNzY0MjA0MQ==,9599,2021-03-25T22:39:22Z,2021-03-25T22:39:22Z,OWNER,"Here's the full current implementation of that tool: https://github.com/simonw/sqlite-transform/blob/0.5/sqlite_transform/cli.py

My current plan is to make this functionality available as the following:

    sqlite-utils convert jsonsplit mydb.db mytable mycolumn
    sqlite-utils convert parsedatetime mydb.db mytable mycolumn
    sqlite-utils convert parsedate mydb.db mytable mycolumn
    sqlite-utils convert lambda mydb.db mytable mycolumn --code='str(value).upper()'

","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-807647791,https://api.github.com/repos/simonw/sqlite-utils/issues/251,807647791,MDEyOklzc3VlQ29tbWVudDgwNzY0Nzc5MQ==,9599,2021-03-25T22:42:48Z,2021-03-25T22:44:31Z,OWNER,"Idea: enhance `lambda` to allow it to return a dictionary of values, which will then be used to populate new columns. Use a `--multicolumn` option to indicate this:

    sqlite-utils convert lambda mydb.db mytable mycolumn \
      --code '{""first_name"": value.split()[0], ""last_name"": value.split()[1]}' \
      --multicolumn --drop

The `--drop` means ""drop the `mycolumn` column after making this change"".

Maybe `--multi` is a better name than `--multicolumn` here, since either way it's going to need additional explanation somewhere.

Would this overlap with #239 at all?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-886122696,https://api.github.com/repos/simonw/sqlite-utils/issues/251,886122696,IC_kwDOCGYnMM400SjI,9599,2021-07-24T23:21:32Z,2021-07-24T23:21:32Z,OWNER,"> ```
> sqlite-utils convert jsonsplit mydb.db mytable mycolumn
> sqlite-utils convert parsedatetime mydb.db mytable mycolumn
> sqlite-utils convert parsedate mydb.db mytable mycolumn
> sqlite-utils convert lambda mydb.db mytable mycolumn --code='str(value).upper()'
> ```

This is a bit verbose - and having added `--multi` and `--output` the `lambda` command keeps getting more and more flexible compared to the others.

New idea: ditch the sub-sub-commands and move the `jsonsplit` and `parsedate` recipes to be options of `convert` - maybe like this:

    sqlite-utils convert my.db mytable col1 --jsonsplit

or:

    sqlite-utils convert my.db mytable col1 --recipe jsonsplit

or `-r jsonsplit` for short.

","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890443079,https://api.github.com/repos/simonw/sqlite-utils/issues/251,890443079,IC_kwDOCGYnMM41ExVH,9599,2021-08-01T03:46:43Z,2021-08-01T03:46:43Z,OWNER,"Note that there's already a concept of `conversions` which might be confused with `convert`? https://sqlite-utils.datasette.io/en/stable/python-api.html#converting-column-values-using-sql-functions

```python
db[""example""].insert({
    ""name"": ""The Bigfoot Discovery Museum""
}, conversions={""name"": ""upper(?)""})
```
I think that's OK though - that's a Python library feature, `sqlite-utils convert` is a CLI thing.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446166,https://api.github.com/repos/simonw/sqlite-utils/issues/251,890446166,IC_kwDOCGYnMM41EyFW,9599,2021-08-01T04:14:26Z,2021-08-01T04:14:26Z,OWNER,"Problem with the `-r/--recipe` idea: the `parsedate` and `parsedatetime` and `jsonsplit` recipes in the current `sqlite-transform` tool all take additional options.

For `sqlite-transform parsedate` and `parsedatetime`:

```python
@click.option(
    ""--dayfirst"",
    is_flag=True,
    help=""Assume day comes first in ambiguous dates, e.g. 03/04/05"",
)
@click.option(
    ""--yearfirst"",
    is_flag=True,
    help=""Assume year comes first in ambiguous dates, e.g. 03/04/05"",
)
```
For `jsonsplit`:
```python
@click.option(""--delimiter"", default="","", help=""Delimiter to split on"")
@click.option(
    ""--type"",
    type=click.Choice((""int"", ""float"")),
    help=""Type to use for values - int or float (defaults to string)"",
)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446506,https://api.github.com/repos/simonw/sqlite-utils/issues/251,890446506,IC_kwDOCGYnMM41EyKq,9599,2021-08-01T04:16:36Z,2021-08-01T04:16:36Z,OWNER,"Back to the design board then. One way to handle this would be the long-form:

```
sqlite-utils convert jsonsplit mydb.db mytable mycolumn
sqlite-utils convert parsedatetime mydb.db mytable mycolumn
sqlite-utils convert parsedate mydb.db mytable mycolumn
sqlite-utils convert lambda mydb.db mytable mycolumn --code='str(value).upper()'
```
I like the idea that `lambda` is the default action, but in this form it's required that the second argument (the word after `convert`) be the name of the recipe that is being applied to avoid any potential confusion with the database filename.

An ugly solution would be to make all four of those options available on `sqlite-utils convert` - and return an error if you try and use one of those without specifying the accompanying recipe. That's a bit gross though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446808,https://api.github.com/repos/simonw/sqlite-utils/issues/251,890446808,IC_kwDOCGYnMM41EyPY,9599,2021-08-01T04:18:18Z,2021-08-01T04:28:18Z,OWNER,"Or.... how about making the `parsedate()` and `parsedatetime()` and `jsonsplit()` functions available within the namespace that is configured for the `--code` block?

Then you could do something like this:

```
sqlite-utils convert mydb.db mytable col --code 'parsedatetime(value, dayfirst=True)'
sqlite-utils convert mydb.db mytable col --code 'jsonsplit(value, delimiter="":"")'
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446943,https://api.github.com/repos/simonw/sqlite-utils/issues/251,890446943,IC_kwDOCGYnMM41EyRf,9599,2021-08-01T04:19:09Z,2021-08-01T04:19:09Z,OWNER,"That's a pretty neat fix, though it's a bit more challenging on the documentation front - maybe the help text for `sqlite-utils convert --help` gets a fair bit longer?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890447102,https://api.github.com/repos/simonw/sqlite-utils/issues/251,890447102,IC_kwDOCGYnMM41EyT-,9599,2021-08-01T04:20:18Z,2021-08-01T04:29:26Z,OWNER,"I could stick them in a `recipe` namespace so you do this:
```
sqlite-utils convert mydb.db mytable col --code 'recipe.parsedatetime(value, dayfirst=True)'
sqlite-utils convert mydb.db mytable col --code 'recipe.jsonsplit(value, delimiter="":"")'
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890448119,https://api.github.com/repos/simonw/sqlite-utils/issues/251,890448119,IC_kwDOCGYnMM41Eyj3,9599,2021-08-01T04:28:05Z,2021-08-01T04:30:28Z,OWNER,"In which case I think `--code` should be a positional argument instead:
```
sqlite-utils convert mydb.db mytable col 'recipe.parsedatetime(value, dayfirst=True)'
sqlite-utils convert mydb.db mytable col 'recipe.jsonsplit(value, delimiter="":"")'
sqlite-utils convert mydb.db mytable col 'recipe.jsonsplit(value, delimiter="":"")'
sqlite-utils convert mydb.db mytable col '{""lower"": value.lower(), ""upper"": value.upper()}' --multi
```
One problem with this: we already accept one or more columns. I think that's OK though since the code is now a required argument, so it means we have to treat everything between the table and the final code argument as a column.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890448190,https://api.github.com/repos/simonw/sqlite-utils/issues/251,890448190,IC_kwDOCGYnMM41Eyk-,9599,2021-08-01T04:28:49Z,2021-08-01T04:28:49Z,OWNER,"Would make sense to accept code from standard input too:

    echo 'value.upper()' | sqlite-utils convert my.db mytable col -","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890448623,https://api.github.com/repos/simonw/sqlite-utils/issues/251,890448623,IC_kwDOCGYnMM41Eyrv,9599,2021-08-01T04:33:30Z,2021-08-01T04:33:30Z,OWNER,"I've started an implementation in the `convert` branch - no documentation yet, and I've not implemented the recipes.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890552827,https://api.github.com/repos/simonw/sqlite-utils/issues/251,890552827,IC_kwDOCGYnMM41FMH7,9599,2021-08-01T16:52:00Z,2021-08-01T16:52:00Z,OWNER,I'll finish the work on this in a PR.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890553783,https://api.github.com/repos/simonw/sqlite-utils/issues/251,890553783,IC_kwDOCGYnMM41FMW3,9599,2021-08-01T16:59:09Z,2021-08-01T16:59:09Z,OWNER,I'm going with `recipes.jsonsplit()` rather than `recipe.jsonsplit()` because the Python module containing the recipes will be called `recipes`. I'll set up a `r.jsonsplit()` shortcut too as a convenience.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-891380382,https://api.github.com/repos/simonw/sqlite-utils/issues/251,891380382,IC_kwDOCGYnMM41IWKe,9599,2021-08-02T22:39:46Z,2021-08-02T22:39:46Z,OWNER,Documentation: https://sqlite-utils.datasette.io/en/stable/cli.html#converting-data-in-columns,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",841377702,