{"html_url": "https://github.com/simonw/datasette/pull/1685#issuecomment-1079018557", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1685", "id": 1079018557, "node_id": "IC_kwDOBm6k_c5AUIQ9", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2022-03-25T13:16:48Z", "updated_at": "2022-03-25T13:16:48Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1685?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\n> Merging [#1685](https://codecov.io/gh/simonw/datasette/pull/1685?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (933ce47) into [main](https://codecov.io/gh/simonw/datasette/commit/c496f2b663ff0cef908ffaaa68b8cb63111fb5f2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (c496f2b) will **not change** coverage.\n> The diff coverage is `n/a`.\n\n```diff\n@@           Coverage Diff           @@\n##             main    #1685   +/-   ##\n=======================================\n  Coverage   91.74%   91.74%           \n=======================================\n  Files          34       34           \n  Lines        4565     4565           \n=======================================\n  Hits         4188     4188           \n  Misses        377      377           \n```\n\n\n\n------\n\n[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1685?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n> `\u0394 = absolute <relative> (impact)`, `\u00f8 = not affected`, `? = missing data`\n> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1685?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [c496f2b...933ce47](https://codecov.io/gh/simonw/datasette/pull/1685?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1180778860, "label": "Update jinja2 requirement from <3.1.0,>=2.10.3 to >=2.10.3,<3.2.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1079243535", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/420", "id": 1079243535, "node_id": "IC_kwDOCGYnMM5AU_MP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-25T17:25:12Z", "updated_at": "2022-03-25T17:25:12Z", "author_association": "OWNER", "body": "That documentation is split across a few places. This is the only bit that talks about `def convert()` pattern right now:\r\n\r\n- https://sqlite-utils.datasette.io/en/stable/cli.html#converting-data-in-columns\r\n\r\nBut that's for `sqlite-utils convert` - the documentation for `sqlite-utils insert --convert` at https://sqlite-utils.datasette.io/en/stable/cli.html#applying-conversions-while-inserting-data doesn't mention it.\r\n\r\nSince both `sqlite-utils convert` and `sqlite-utils insert --convert` apply the same rules to the code, they should link to a shared explanation in the documentation.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1178546862, "label": "Document how to use a `--convert` function that runs initialization code first"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1079376283", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/420", "id": 1079376283, "node_id": "IC_kwDOCGYnMM5AVfmb", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-25T19:39:30Z", "updated_at": "2022-03-25T19:43:35Z", "author_association": "OWNER", "body": "Actually this doesn't work as I thought. This demo shows that the initialization code is run once per item, not a single time at the start of the run:\r\n```\r\n% sqlite-utils insert dogs.db dogs dogs.json --convert '\r\nimport random\r\nprint(\"seeding\")\r\nrandom.seed(10)\r\nprint(random.random())\r\n\r\ndef convert(row):\r\n    print(row)\r\n    row[\"random_score\"] = random.random()\r\n'\r\nseeding\r\n0.5714025946899135\r\nseeding\r\n0.5714025946899135\r\nseeding\r\n0.5714025946899135\r\nseeding\r\n0.5714025946899135\r\n```\r\nAlso that `print(row)` line is not being printed anywhere that gets to the console for some reason.\r\n\r\n... my mistake, that happened because I changed this line in order to try to get local imports to work:\r\n```python\r\n    try:\r\n        exec(code, globals, locals)\r\n        return globals[\"convert\"]\r\n    except (AttributeError, SyntaxError, NameError, KeyError, TypeError):\r\n```\r\nIt should be `locals[\"convert\"]`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1178546862, "label": "Document how to use a `--convert` function that runs initialization code first"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1079384771", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/420", "id": 1079384771, "node_id": "IC_kwDOCGYnMM5AVhrD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-25T19:51:34Z", "updated_at": "2022-03-25T19:53:01Z", "author_association": "OWNER", "body": "This works:\r\n```\r\n% sqlite-utils insert dogs.db dogs dogs.json --convert '\r\nimport random\r\nprint(\"seeding\")\r\nrandom.seed(10)\r\nprint(random.random())\r\n\r\ndef convert(row):\r\n    global random\r\n    print(row)\r\n    row[\"random_score\"] = random.random()\r\n'\r\nseeding\r\n0.5714025946899135\r\n{'id': 1, 'name': 'Cleo'}\r\n{'id': 2, 'name': 'Pancakes'}\r\n{'id': 3, 'name': 'New dog'}\r\n(sqlite-utils) sqlite-utils % sqlite-utils rows dogs.db dogs\r\n[{\"id\": 1, \"name\": \"Cleo\", \"random_score\": 0.4288890546751146},\r\n {\"id\": 2, \"name\": \"Pancakes\", \"random_score\": 0.5780913011344704},\r\n {\"id\": 3, \"name\": \"New dog\", \"random_score\": 0.20609823213950174}]\r\n```\r\nHaving to use `global random` inside the function is frustrating but apparently necessary. https://stackoverflow.com/a/56552138/6083", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1178546862, "label": "Document how to use a `--convert` function that runs initialization code first"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1079404281", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/420", "id": 1079404281, "node_id": "IC_kwDOCGYnMM5AVmb5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-25T20:19:50Z", "updated_at": "2022-03-25T20:19:50Z", "author_association": "OWNER", "body": "Now documented here: https://sqlite-utils.datasette.io/en/latest/cli.html#using-a-convert-function-to-execute-initialization", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1178546862, "label": "Document how to use a `--convert` function that runs initialization code first"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/422#issuecomment-1079406708", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/422", "id": 1079406708, "node_id": "IC_kwDOCGYnMM5AVnB0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-25T20:23:21Z", "updated_at": "2022-03-25T20:23:21Z", "author_association": "OWNER", "body": "Fixing this would require a bump to 4.0 because it would break existing code.\r\n\r\nThe alternative would be to introduce a new `ignore_nulls=True` parameter which users can change to `ignore_nulls=False`. Or come up with better wording for that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1181236173, "label": "Reconsider not running convert functions against null values"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/421#issuecomment-1079407962", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/421", "id": 1079407962, "node_id": "IC_kwDOCGYnMM5AVnVa", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-25T20:25:10Z", "updated_at": "2022-03-25T20:25:18Z", "author_association": "OWNER", "body": "Can you share either your whole `global.db` table or a shrunk down example that illustrates the bug?\r\n\r\nMy hunch is that you may have a table or column with a name that triggers the error.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1180427792, "label": "\"Error: near \"(\": syntax error\" when using sqlite-utils indexes CLI"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/417#issuecomment-1079441621", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/417", "id": 1079441621, "node_id": "IC_kwDOCGYnMM5AVvjV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-25T21:18:37Z", "updated_at": "2022-03-25T21:18:37Z", "author_association": "OWNER", "body": "Updated documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#inserting-newline-delimited-json", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1175744654, "label": "insert fails on JSONL with whitespace"}, "performed_via_github_app": null}