{"html_url": "https://github.com/simonw/sqlite-utils/issues/356#issuecomment-997492872", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/356", "id": 997492872, "node_id": "IC_kwDOCGYnMM47dIiI", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-20T00:23:31Z", "updated_at": "2021-12-20T00:23:31Z", "author_association": "OWNER", "body": "I think this should work on JSON, or CSV, or individual lines, or the entire content at once.\r\n\r\nSo I'll require `--lines --convert ...` to import individual lines, or `--all --convert` to run the conversion against the entire input at once.\r\n\r\nWhat would `--lines` or `--all` do without `--convert`? Maybe insert records as `{\"line\": \"line of text\"}` or `{\"all\": \"whole input}`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077431957, "label": "`sqlite-utils insert --convert` option"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/356#issuecomment-997496931", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/356", "id": 997496931, "node_id": "IC_kwDOCGYnMM47dJhj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-20T00:39:14Z", "updated_at": "2021-12-20T00:39:52Z", "author_association": "OWNER", "body": "```\r\n% sqlite-utils insert --help\r\nUsage: sqlite-utils insert [OPTIONS] PATH TABLE JSON_FILE\r\n\r\n  Insert records from JSON file into a table, creating the table if it does\r\n  not already exist.\r\n\r\n  Input should be a JSON array of objects, unless --nl or --csv is used.\r\n\r\nOptions:\r\n  --pk TEXT                 Columns to use as the primary key, e.g. id\r\n  --nl                      Expect newline-delimited JSON\r\n  --flatten                 Flatten nested JSON objects\r\n  -c, --csv                 Expect CSV\r\n  --tsv                     Expect TSV\r\n  --convert TEXT            Python code to convert each item\r\n  --import TEXT             Python modules to import\r\n  --delimiter TEXT          Delimiter to use for CSV files\r\n  --quotechar TEXT          Quote character to use for CSV/TSV\r\n  --sniff                   Detect delimiter and quote character\r\n  --no-headers              CSV file has no header row\r\n  --batch-size INTEGER      Commit every X records\r\n  --alter                   Alter existing table to add any missing columns\r\n  --not-null TEXT           Columns that should be created as NOT NULL\r\n  --default <TEXT TEXT>...  Default value that should be set for a column\r\n  --encoding TEXT           Character encoding for input, defaults to utf-8\r\n  -d, --detect-types        Detect types for columns in CSV/TSV data\r\n  --load-extension TEXT     SQLite extensions to load\r\n  --silent                  Do not show progress bar\r\n  --ignore                  Ignore records if pk already exists\r\n  --replace                 Replace records if pk already exists\r\n  --truncate                Truncate table before inserting records, if table\r\n                            already exists\r\n  -h, --help                Show this message and exit.\r\n```\r\nI can add a bunch of extra help at the top there to explain all of this stuff. That \"Input should be a JSON array of objects\" bit could be expanded to several paragraphs.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077431957, "label": "`sqlite-utils insert --convert` option"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/356#issuecomment-997497262", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/356", "id": 997497262, "node_id": "IC_kwDOCGYnMM47dJmu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-20T00:40:15Z", "updated_at": "2021-12-20T00:40:15Z", "author_association": "OWNER", "body": "`--flatten` could do with a better description too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077431957, "label": "`sqlite-utils insert --convert` option"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/163#issuecomment-997502242", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/163", "id": 997502242, "node_id": "IC_kwDOCGYnMM47dK0i", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-20T00:56:45Z", "updated_at": "2021-12-20T00:56:52Z", "author_association": "OWNER", "body": "> Maybe `sqlite-utils` should absorb all of the functionality from `sqlite-transform` - having two separate tools doesn't necessarily make sense.\r\n\r\nI implemented that in:\r\n- #251", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 706001517, "label": "Idea: conversions= could take Python functions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/356#issuecomment-997507074", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/356", "id": 997507074, "node_id": "IC_kwDOCGYnMM47dMAC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-20T01:10:06Z", "updated_at": "2021-12-20T01:16:11Z", "author_association": "OWNER", "body": "Work-in-progress improved help:\r\n```\r\nUsage: sqlite-utils insert [OPTIONS] PATH TABLE FILE\r\n\r\n  Insert records from FILE into a table, creating the table if it does not\r\n  already exist.\r\n\r\n  By default the input is expected to be a JSON array of objects. Or:\r\n\r\n  - Use --nl for newline-delimited JSON objects\r\n  - Use --csv or --tsv for comma-separated or tab-separated input\r\n  - Use --lines to write each incoming line to a column called \"line\"\r\n  - Use --all to write the entire input to a column called \"all\"\r\n\r\n  You can also use --convert to pass a fragment of Python code that will be\r\n  used to convert each input.\r\n\r\n  Your Python code will be passed a \"row\" variable representing the imported\r\n  row, and can return a modified row.\r\n\r\n  If you are using --lines your code will be passed a \"line\" variable, and for\r\n  --all an \"all\" variable.\r\n\r\nOptions:\r\n  --pk TEXT                 Columns to use as the primary key, e.g. id\r\n  --flatten                 Flatten nested JSON objects, so {\"a\": {\"b\": 1}}\r\n                            becomes {\"a_b\": 1}\r\n  --nl                      Expect newline-delimited JSON\r\n  -c, --csv                 Expect CSV input\r\n  --tsv                     Expect TSV input\r\n  --lines                   Treat each line as a single value called 'line'\r\n  --all                     Treat input as a single value called 'all'\r\n  --convert TEXT            Python code to convert each item\r\n  --import TEXT             Python modules to import\r\n  --delimiter TEXT          Delimiter to use for CSV files\r\n  --quotechar TEXT          Quote character to use for CSV/TSV\r\n  --sniff                   Detect delimiter and quote character\r\n  --no-headers              CSV file has no header row\r\n  --batch-size INTEGER      Commit every X records\r\n  --alter                   Alter existing table to add any missing columns\r\n  --not-null TEXT           Columns that should be created as NOT NULL\r\n  --default <TEXT TEXT>...  Default value that should be set for a column\r\n  --encoding TEXT           Character encoding for input, defaults to utf-8\r\n  -d, --detect-types        Detect types for columns in CSV/TSV data\r\n  --load-extension TEXT     SQLite extensions to load\r\n  --silent                  Do not show progress bar\r\n  --ignore                  Ignore records if pk already exists\r\n  --replace                 Replace records if pk already exists\r\n  --truncate                Truncate table before inserting records, if table\r\n                            already exists\r\n  -h, --help                Show this message and exit.\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077431957, "label": "`sqlite-utils insert --convert` option"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/356#issuecomment-997508728", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/356", "id": 997508728, "node_id": "IC_kwDOCGYnMM47dMZ4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-20T01:14:43Z", "updated_at": "2021-12-20T01:14:43Z", "author_association": "OWNER", "body": "(This makes me want `--extract` from #352 even more.)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077431957, "label": "`sqlite-utils insert --convert` option"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1547#issuecomment-997511968", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1547", "id": 997511968, "node_id": "IC_kwDOBm6k_c47dNMg", "user": {"value": 127565, "label": "wragge"}, "created_at": "2021-12-20T01:21:59Z", "updated_at": "2021-12-20T01:21:59Z", "author_association": "CONTRIBUTOR", "body": "I've installed the alpha version but get an error when starting up Datasette:\r\n\r\n```\r\nTraceback (most recent call last):\r\n  File \"/Users/tim/.pyenv/versions/stock-exchange/bin/datasette\", line 5, in <module>\r\n    from datasette.cli import cli\r\n  File \"/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/cli.py\", line 15, in <module>\r\n    from .app import Datasette, DEFAULT_SETTINGS, SETTINGS, SQLITE_LIMIT_ATTACHED, pm\r\n  File \"/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/app.py\", line 31, in <module>\r\n    from .views.database import DatabaseDownload, DatabaseView\r\n  File \"/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/views/database.py\", line 25, in <module>\r\n    from datasette.plugins import pm\r\n  File \"/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/plugins.py\", line 29, in <module>\r\n    mod = importlib.import_module(plugin)\r\n  File \"/Users/tim/.pyenv/versions/3.8.5/lib/python3.8/importlib/__init__.py\", line 127, in import_module\r\n    return _bootstrap._gcd_import(name[level:], package, level)\r\n  File \"/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/filters.py\", line 9, in <module>\r\n    @hookimpl(specname=\"filters_from_request\")\r\nTypeError: __call__() got an unexpected keyword argument 'specname'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1076388044, "label": "Writable canned queries fail to load custom templates"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1575#issuecomment-997513177", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1575", "id": 997513177, "node_id": "IC_kwDOBm6k_c47dNfZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-20T01:24:25Z", "updated_at": "2021-12-20T01:24:25Z", "author_association": "OWNER", "body": "Looks like `specname` is new in Pluggy 1.0: https://github.com/pytest-dev/pluggy/blob/main/CHANGELOG.rst#pluggy-100-2021-08-25", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1084257842, "label": "__call__() got an unexpected keyword argument 'specname'"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1547#issuecomment-997513369", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1547", "id": 997513369, "node_id": "IC_kwDOBm6k_c47dNiZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-20T01:24:43Z", "updated_at": "2021-12-20T01:24:43Z", "author_association": "OWNER", "body": "@wragge thanks, that's a bug! Working on that in #1575.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1076388044, "label": "Writable canned queries fail to load custom templates"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1547#issuecomment-997514220", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1547", "id": 997514220, "node_id": "IC_kwDOBm6k_c47dNvs", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-20T01:26:25Z", "updated_at": "2021-12-20T01:26:25Z", "author_association": "OWNER", "body": "OK, this should hopefully fix that for you:\r\n\r\n    pip install https://github.com/simonw/datasette/archive/f36e010b3b69ada104b79d83c7685caf9359049e.zip", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1076388044, "label": "Writable canned queries fail to load custom templates"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1547#issuecomment-997519202", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1547", "id": 997519202, "node_id": "IC_kwDOBm6k_c47dO9i", "user": {"value": 127565, "label": "wragge"}, "created_at": "2021-12-20T01:36:58Z", "updated_at": "2021-12-20T01:36:58Z", "author_association": "CONTRIBUTOR", "body": "Yep, that works -- thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1076388044, "label": "Writable canned queries fail to load custom templates"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1554#issuecomment-998354538", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1554", "id": 998354538, "node_id": "IC_kwDOBm6k_c47ga5q", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-20T23:52:04Z", "updated_at": "2021-12-20T23:52:04Z", "author_association": "OWNER", "body": "Abandoning this since it didn't work how I wanted.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1079129258, "label": "TableView refactor"}, "performed_via_github_app": null}