{"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890553783", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 890553783, "node_id": "IC_kwDOCGYnMM41FMW3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T16:59:09Z", "updated_at": "2021-08-01T16:59:09Z", "author_association": "OWNER", "body": "I'm going with `recipes.jsonsplit()` rather than `recipe.jsonsplit()` because the Python module containing the recipes will be called `recipes`. I'll set up a `r.jsonsplit()` shortcut too as a convenience.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/303#issuecomment-890553014", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/303", "id": 890553014, "node_id": "IC_kwDOCGYnMM41FMK2", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2021-08-01T16:53:35Z", "updated_at": "2021-08-02T04:45:19Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\n> Merging [#303](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (4c3bf97) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/c7e8d72be9fe8fe0811f685a18eebc637662d41b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (c7e8d72) will **increase** coverage by `0.24%`.\n> The diff coverage is `100.00%`.\n\n[![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/303/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n```diff\n@@ Coverage Diff @@\n## main #303 +/- ##\n==========================================\n+ Coverage 96.04% 96.28% +0.24% \n==========================================\n Files 4 5 +1 \n Lines 1998 2129 +131 \n==========================================\n+ Hits 1919 2050 +131 \n Misses 79 79 \n```\n\n\n| [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [sqlite\\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/303/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `94.91% <100.00%> (+0.36%)` | :arrow_up: |\n| [sqlite\\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/303/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.97% <100.00%> (+0.10%)` | :arrow_up: |\n| [sqlite\\_utils/recipes.py](https://codecov.io/gh/simonw/sqlite-utils/pull/303/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL3JlY2lwZXMucHk=) | `100.00% <100.00%> (\u00f8)` | |\n| [sqlite\\_utils/utils.py](https://codecov.io/gh/simonw/sqlite-utils/pull/303/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL3V0aWxzLnB5) | `92.39% <100.00%> (+0.48%)` | :arrow_up: |\n\n------\n\n[Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n> `\u0394 = absolute (impact)`, `\u00f8 = not affected`, `? = missing data`\n> Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [c7e8d72...4c3bf97](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 957536983, "label": "sqlite-utils convert command and db[table].convert(...) method"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890552827", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 890552827, "node_id": "IC_kwDOCGYnMM41FMH7", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T16:52:00Z", "updated_at": "2021-08-01T16:52:00Z", "author_association": "OWNER", "body": "I'll finish the work on this in a PR.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/302#issuecomment-890548009", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/302", "id": 890548009, "node_id": "IC_kwDOCGYnMM41FK8p", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T16:18:13Z", "updated_at": "2021-08-01T16:18:13Z", "author_association": "OWNER", "body": "Basic API design:\r\n\r\n db[table].convert(\"headline\", lambda v: v.upper())\r\n\r\nYou can pass a list of columns instead of a single game column string to apply it to multiple columns.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 957529248, "label": "Python library version of `sqlite-utils convert`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890448623", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 890448623, "node_id": "IC_kwDOCGYnMM41Eyrv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T04:33:30Z", "updated_at": "2021-08-01T04:33:30Z", "author_association": "OWNER", "body": "I've started an implementation in the `convert` branch - no documentation yet, and I've not implemented the recipes.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890448190", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 890448190, "node_id": "IC_kwDOCGYnMM41Eyk-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T04:28:49Z", "updated_at": "2021-08-01T04:28:49Z", "author_association": "OWNER", "body": "Would make sense to accept code from standard input too:\r\n\r\n echo 'value.upper()' | sqlite-utils convert my.db mytable col -", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890448119", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 890448119, "node_id": "IC_kwDOCGYnMM41Eyj3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T04:28:05Z", "updated_at": "2021-08-01T04:30:28Z", "author_association": "OWNER", "body": "In which case I think `--code` should be a positional argument instead:\r\n```\r\nsqlite-utils convert mydb.db mytable col 'recipe.parsedatetime(value, dayfirst=True)'\r\nsqlite-utils convert mydb.db mytable col 'recipe.jsonsplit(value, delimiter=\":\")'\r\nsqlite-utils convert mydb.db mytable col 'recipe.jsonsplit(value, delimiter=\":\")'\r\nsqlite-utils convert mydb.db mytable col '{\"lower\": value.lower(), \"upper\": value.upper()}' --multi\r\n```\r\nOne problem with this: we already accept one or more columns. I think that's OK though since the code is now a required argument, so it means we have to treat everything between the table and the final code argument as a column.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890447102", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 890447102, "node_id": "IC_kwDOCGYnMM41EyT-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T04:20:18Z", "updated_at": "2021-08-01T04:29:26Z", "author_association": "OWNER", "body": "I could stick them in a `recipe` namespace so you do this:\r\n```\r\nsqlite-utils convert mydb.db mytable col --code 'recipe.parsedatetime(value, dayfirst=True)'\r\nsqlite-utils convert mydb.db mytable col --code 'recipe.jsonsplit(value, delimiter=\":\")'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446943", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 890446943, "node_id": "IC_kwDOCGYnMM41EyRf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T04:19:09Z", "updated_at": "2021-08-01T04:19:09Z", "author_association": "OWNER", "body": "That's a pretty neat fix, though it's a bit more challenging on the documentation front - maybe the help text for `sqlite-utils convert --help` gets a fair bit longer?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446808", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 890446808, "node_id": "IC_kwDOCGYnMM41EyPY", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T04:18:18Z", "updated_at": "2021-08-01T04:28:18Z", "author_association": "OWNER", "body": "Or.... how about making the `parsedate()` and `parsedatetime()` and `jsonsplit()` functions available within the namespace that is configured for the `--code` block?\r\n\r\nThen you could do something like this:\r\n\r\n```\r\nsqlite-utils convert mydb.db mytable col --code 'parsedatetime(value, dayfirst=True)'\r\nsqlite-utils convert mydb.db mytable col --code 'jsonsplit(value, delimiter=\":\")'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446506", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 890446506, "node_id": "IC_kwDOCGYnMM41EyKq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T04:16:36Z", "updated_at": "2021-08-01T04:16:36Z", "author_association": "OWNER", "body": "Back to the design board then. One way to handle this would be the long-form:\r\n\r\n```\r\nsqlite-utils convert jsonsplit mydb.db mytable mycolumn\r\nsqlite-utils convert parsedatetime mydb.db mytable mycolumn\r\nsqlite-utils convert parsedate mydb.db mytable mycolumn\r\nsqlite-utils convert lambda mydb.db mytable mycolumn --code='str(value).upper()'\r\n```\r\nI like the idea that `lambda` is the default action, but in this form it's required that the second argument (the word after `convert`) be the name of the recipe that is being applied to avoid any potential confusion with the database filename.\r\n\r\nAn ugly solution would be to make all four of those options available on `sqlite-utils convert` - and return an error if you try and use one of those without specifying the accompanying recipe. That's a bit gross though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446166", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 890446166, "node_id": "IC_kwDOCGYnMM41EyFW", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T04:14:26Z", "updated_at": "2021-08-01T04:14:26Z", "author_association": "OWNER", "body": "Problem with the `-r/--recipe` idea: the `parsedate` and `parsedatetime` and `jsonsplit` recipes in the current `sqlite-transform` tool all take additional options.\r\n\r\nFor `sqlite-transform parsedate` and `parsedatetime`:\r\n\r\n```python\r\n@click.option(\r\n \"--dayfirst\",\r\n is_flag=True,\r\n help=\"Assume day comes first in ambiguous dates, e.g. 03/04/05\",\r\n)\r\n@click.option(\r\n \"--yearfirst\",\r\n is_flag=True,\r\n help=\"Assume year comes first in ambiguous dates, e.g. 03/04/05\",\r\n)\r\n```\r\nFor `jsonsplit`:\r\n```python\r\n@click.option(\"--delimiter\", default=\",\", help=\"Delimiter to split on\")\r\n@click.option(\r\n \"--type\",\r\n type=click.Choice((\"int\", \"float\")),\r\n help=\"Type to use for values - int or float (defaults to string)\",\r\n)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890443079", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 890443079, "node_id": "IC_kwDOCGYnMM41ExVH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T03:46:43Z", "updated_at": "2021-08-01T03:46:43Z", "author_association": "OWNER", "body": "Note that there's already a concept of `conversions` which might be confused with `convert`? https://sqlite-utils.datasette.io/en/stable/python-api.html#converting-column-values-using-sql-functions\r\n\r\n```python\r\ndb[\"example\"].insert({\r\n \"name\": \"The Bigfoot Discovery Museum\"\r\n}, conversions={\"name\": \"upper(?)\"})\r\n```\r\nI think that's OK though - that's a Python library feature, `sqlite-utils convert` is a CLI thing.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1411#issuecomment-890441844", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1411", "id": 890441844, "node_id": "IC_kwDOBm6k_c41ExB0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-01T03:27:30Z", "updated_at": "2021-08-01T03:27:30Z", "author_association": "OWNER", "body": "Confirmed: https://latest.datasette.io/fixtures/neighborhood_search?text=cork&_hide_sql=1 no longer exhibits the bug.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 957345476, "label": "Canned query ?sql= is pointlessly echoed in query string starting from hidden mode"}, "performed_via_github_app": null}