html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/simonw/sqlite-utils/issues/356#issuecomment-997492872,https://api.github.com/repos/simonw/sqlite-utils/issues/356,997492872,IC_kwDOCGYnMM47dIiI,9599,simonw,2021-12-20T00:23:31Z,2021-12-20T00:23:31Z,OWNER,"I think this should work on JSON, or CSV, or individual lines, or the entire content at once.

So I'll require `--lines --convert ...` to import individual lines, or `--all --convert` to run the conversion against the entire input at once.

What would `--lines` or `--all` do without `--convert`? Maybe insert records as `{""line"": ""line of text""}` or `{""all"": ""whole input}`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1077431957,`sqlite-utils insert --convert` option,
https://github.com/simonw/sqlite-utils/issues/356#issuecomment-997496931,https://api.github.com/repos/simonw/sqlite-utils/issues/356,997496931,IC_kwDOCGYnMM47dJhj,9599,simonw,2021-12-20T00:39:14Z,2021-12-20T00:39:52Z,OWNER,"```
% sqlite-utils insert --help
Usage: sqlite-utils insert [OPTIONS] PATH TABLE JSON_FILE

  Insert records from JSON file into a table, creating the table if it does
  not already exist.

  Input should be a JSON array of objects, unless --nl or --csv is used.

Options:
  --pk TEXT                 Columns to use as the primary key, e.g. id
  --nl                      Expect newline-delimited JSON
  --flatten                 Flatten nested JSON objects
  -c, --csv                 Expect CSV
  --tsv                     Expect TSV
  --convert TEXT            Python code to convert each item
  --import TEXT             Python modules to import
  --delimiter TEXT          Delimiter to use for CSV files
  --quotechar TEXT          Quote character to use for CSV/TSV
  --sniff                   Detect delimiter and quote character
  --no-headers              CSV file has no header row
  --batch-size INTEGER      Commit every X records
  --alter                   Alter existing table to add any missing columns
  --not-null TEXT           Columns that should be created as NOT NULL
  --default <TEXT TEXT>...  Default value that should be set for a column
  --encoding TEXT           Character encoding for input, defaults to utf-8
  -d, --detect-types        Detect types for columns in CSV/TSV data
  --load-extension TEXT     SQLite extensions to load
  --silent                  Do not show progress bar
  --ignore                  Ignore records if pk already exists
  --replace                 Replace records if pk already exists
  --truncate                Truncate table before inserting records, if table
                            already exists
  -h, --help                Show this message and exit.
```
I can add a bunch of extra help at the top there to explain all of this stuff. That ""Input should be a JSON array of objects"" bit could be expanded to several paragraphs.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1077431957,`sqlite-utils insert --convert` option,
https://github.com/simonw/sqlite-utils/issues/356#issuecomment-997497262,https://api.github.com/repos/simonw/sqlite-utils/issues/356,997497262,IC_kwDOCGYnMM47dJmu,9599,simonw,2021-12-20T00:40:15Z,2021-12-20T00:40:15Z,OWNER,`--flatten` could do with a better description too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1077431957,`sqlite-utils insert --convert` option,
https://github.com/simonw/sqlite-utils/issues/163#issuecomment-997502242,https://api.github.com/repos/simonw/sqlite-utils/issues/163,997502242,IC_kwDOCGYnMM47dK0i,9599,simonw,2021-12-20T00:56:45Z,2021-12-20T00:56:52Z,OWNER,"> Maybe `sqlite-utils` should absorb all of the functionality from `sqlite-transform` - having two separate tools doesn't necessarily make sense.

I implemented that in:
- #251","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",706001517,Idea: conversions= could take Python functions,
https://github.com/simonw/sqlite-utils/issues/356#issuecomment-997507074,https://api.github.com/repos/simonw/sqlite-utils/issues/356,997507074,IC_kwDOCGYnMM47dMAC,9599,simonw,2021-12-20T01:10:06Z,2021-12-20T01:16:11Z,OWNER,"Work-in-progress improved help:
```
Usage: sqlite-utils insert [OPTIONS] PATH TABLE FILE

  Insert records from FILE into a table, creating the table if it does not
  already exist.

  By default the input is expected to be a JSON array of objects. Or:

  - Use --nl for newline-delimited JSON objects
  - Use --csv or --tsv for comma-separated or tab-separated input
  - Use --lines to write each incoming line to a column called ""line""
  - Use --all to write the entire input to a column called ""all""

  You can also use --convert to pass a fragment of Python code that will be
  used to convert each input.

  Your Python code will be passed a ""row"" variable representing the imported
  row, and can return a modified row.

  If you are using --lines your code will be passed a ""line"" variable, and for
  --all an ""all"" variable.

Options:
  --pk TEXT                 Columns to use as the primary key, e.g. id
  --flatten                 Flatten nested JSON objects, so {""a"": {""b"": 1}}
                            becomes {""a_b"": 1}
  --nl                      Expect newline-delimited JSON
  -c, --csv                 Expect CSV input
  --tsv                     Expect TSV input
  --lines                   Treat each line as a single value called 'line'
  --all                     Treat input as a single value called 'all'
  --convert TEXT            Python code to convert each item
  --import TEXT             Python modules to import
  --delimiter TEXT          Delimiter to use for CSV files
  --quotechar TEXT          Quote character to use for CSV/TSV
  --sniff                   Detect delimiter and quote character
  --no-headers              CSV file has no header row
  --batch-size INTEGER      Commit every X records
  --alter                   Alter existing table to add any missing columns
  --not-null TEXT           Columns that should be created as NOT NULL
  --default <TEXT TEXT>...  Default value that should be set for a column
  --encoding TEXT           Character encoding for input, defaults to utf-8
  -d, --detect-types        Detect types for columns in CSV/TSV data
  --load-extension TEXT     SQLite extensions to load
  --silent                  Do not show progress bar
  --ignore                  Ignore records if pk already exists
  --replace                 Replace records if pk already exists
  --truncate                Truncate table before inserting records, if table
                            already exists
  -h, --help                Show this message and exit.
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1077431957,`sqlite-utils insert --convert` option,
https://github.com/simonw/sqlite-utils/issues/356#issuecomment-997508728,https://api.github.com/repos/simonw/sqlite-utils/issues/356,997508728,IC_kwDOCGYnMM47dMZ4,9599,simonw,2021-12-20T01:14:43Z,2021-12-20T01:14:43Z,OWNER,(This makes me want `--extract` from #352 even more.),"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1077431957,`sqlite-utils insert --convert` option,
https://github.com/simonw/datasette/issues/1547#issuecomment-997511968,https://api.github.com/repos/simonw/datasette/issues/1547,997511968,IC_kwDOBm6k_c47dNMg,127565,wragge,2021-12-20T01:21:59Z,2021-12-20T01:21:59Z,CONTRIBUTOR,"I've installed the alpha version but get an error when starting up Datasette:

```
Traceback (most recent call last):
  File ""/Users/tim/.pyenv/versions/stock-exchange/bin/datasette"", line 5, in <module>
    from datasette.cli import cli
  File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/cli.py"", line 15, in <module>
    from .app import Datasette, DEFAULT_SETTINGS, SETTINGS, SQLITE_LIMIT_ATTACHED, pm
  File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/app.py"", line 31, in <module>
    from .views.database import DatabaseDownload, DatabaseView
  File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/views/database.py"", line 25, in <module>
    from datasette.plugins import pm
  File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/plugins.py"", line 29, in <module>
    mod = importlib.import_module(plugin)
  File ""/Users/tim/.pyenv/versions/3.8.5/lib/python3.8/importlib/__init__.py"", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/filters.py"", line 9, in <module>
    @hookimpl(specname=""filters_from_request"")
TypeError: __call__() got an unexpected keyword argument 'specname'
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1076388044,Writable canned queries fail to load custom templates,
https://github.com/simonw/datasette/issues/1575#issuecomment-997513177,https://api.github.com/repos/simonw/datasette/issues/1575,997513177,IC_kwDOBm6k_c47dNfZ,9599,simonw,2021-12-20T01:24:25Z,2021-12-20T01:24:25Z,OWNER,Looks like `specname` is new in Pluggy 1.0: https://github.com/pytest-dev/pluggy/blob/main/CHANGELOG.rst#pluggy-100-2021-08-25,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1084257842,__call__() got an unexpected keyword argument 'specname',
https://github.com/simonw/datasette/issues/1547#issuecomment-997513369,https://api.github.com/repos/simonw/datasette/issues/1547,997513369,IC_kwDOBm6k_c47dNiZ,9599,simonw,2021-12-20T01:24:43Z,2021-12-20T01:24:43Z,OWNER,"@wragge thanks, that's a bug! Working on that in #1575.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1076388044,Writable canned queries fail to load custom templates,
https://github.com/simonw/datasette/issues/1547#issuecomment-997514220,https://api.github.com/repos/simonw/datasette/issues/1547,997514220,IC_kwDOBm6k_c47dNvs,9599,simonw,2021-12-20T01:26:25Z,2021-12-20T01:26:25Z,OWNER,"OK, this should hopefully fix that for you:

    pip install https://github.com/simonw/datasette/archive/f36e010b3b69ada104b79d83c7685caf9359049e.zip","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1076388044,Writable canned queries fail to load custom templates,
https://github.com/simonw/datasette/issues/1547#issuecomment-997519202,https://api.github.com/repos/simonw/datasette/issues/1547,997519202,IC_kwDOBm6k_c47dO9i,127565,wragge,2021-12-20T01:36:58Z,2021-12-20T01:36:58Z,CONTRIBUTOR,"Yep, that works -- thanks!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1076388044,Writable canned queries fail to load custom templates,
https://github.com/simonw/datasette/pull/1554#issuecomment-998354538,https://api.github.com/repos/simonw/datasette/issues/1554,998354538,IC_kwDOBm6k_c47ga5q,9599,simonw,2021-12-20T23:52:04Z,2021-12-20T23:52:04Z,OWNER,Abandoning this since it didn't work how I wanted.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1079129258,TableView refactor,