html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248527646,https://api.github.com/repos/simonw/sqlite-utils/issues/486,1248527646,IC_kwDOCGYnMM5KawUe,22429695,2022-09-15T19:34:59Z,2022-09-15T20:23:12Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/486?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **96.47**% // Head: **96.52**% // Increases project coverage by **`+0.04%`** :tada: > Coverage data is based on head [(`0acbc68`)](https://codecov.io/gh/simonw/sqlite-utils/pull/486?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`d9b9e07`)](https://codecov.io/gh/simonw/sqlite-utils/commit/d9b9e075f07a20f1137cd2e34ed5d3f1a3db4ad8?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch coverage: 100.00% of modified lines in pull request are covered. > :exclamation: Current head 0acbc68 differs from pull request most recent head d5db749. Consider uploading reports for the commit d5db749 to get more accurate results
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #486 +/- ## ========================================== + Coverage 96.47% 96.52% +0.04% ========================================== Files 6 6 Lines 2642 2646 +4 ========================================== + Hits 2549 2554 +5 + Misses 93 92 -1 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/486?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/486/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.86% <100.00%> (ø)` | | | [sqlite\_utils/utils.py](https://codecov.io/gh/simonw/sqlite-utils/pull/486/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL3V0aWxzLnB5) | `94.98% <100.00%> (+0.47%)` | :arrow_up: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/486?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1366512990, https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248565396,https://api.github.com/repos/simonw/sqlite-utils/issues/486,1248565396,IC_kwDOCGYnMM5Ka5iU,9599,2022-09-15T20:12:50Z,2022-09-15T20:12:50Z,OWNER,"Annoying `mypy` test failure: ``` /Users/runner/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/numpy/__init__.pyi:636: error: Positional-only parameters are only supported in Python 3.8 and greater ``` Looks like this: - https://github.com/python/mypy/issues/13627","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1366512990, https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248567323,https://api.github.com/repos/simonw/sqlite-utils/issues/486,1248567323,IC_kwDOCGYnMM5Ka6Ab,9599,2022-09-15T20:14:45Z,2022-09-15T20:14:45Z,OWNER,"There's a fix for `mypy` that has landed but isn't out in a release yet: - https://github.com/python/mypy/issues/13385 For the moment looks like pinning to Python 3.10.6 could help. Need to figure out how to do that in GitHub Actions though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1366512990, https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248568775,https://api.github.com/repos/simonw/sqlite-utils/issues/486,1248568775,IC_kwDOCGYnMM5Ka6XH,9599,2022-09-15T20:16:14Z,2022-09-15T20:16:14Z,OWNER,"https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#using-the-python-version-input says can set the full version: ``` - uses: actions/setup-python@v4 with: python-version: ""3.10.6"" ``` I'll try that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1366512990, https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248582147,https://api.github.com/repos/simonw/sqlite-utils/issues/486,1248582147,IC_kwDOCGYnMM5Ka9oD,9599,2022-09-15T20:29:17Z,2022-09-15T20:29:17Z,OWNER,This looks good to me. I need to run some manual tests before merging (it's a good sign that the automated tests pass though).,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1366512990, https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248591268,https://api.github.com/repos/simonw/sqlite-utils/issues/486,1248591268,IC_kwDOCGYnMM5Ka_2k,9599,2022-09-15T20:36:02Z,2022-09-15T20:40:03Z,OWNER,"I had a big CSV file lying around, I converted it to other formats like this: sqlite-utils insert /tmp/t.db t /tmp/en.openfoodfacts.org.products.csv --csv sqlite-utils rows /tmp/t.db t --nl > /tmp/big.nl sqlite-utils rows /tmp/t.db t > /tmp/big.json Then tested the progress bar like this: sqlite-utils insert /tmp/t2.db t /tmp/big.nl --nl Output: ``` sqlite-utils insert /tmp/t2.db t /tmp/big.nl --nl [------------------------------------] 0% [#######-----------------------------] 20% 00:00:20 ``` With `--silent` it is silent. And for regular JSON: ``` sqlite-utils insert /tmp/t3.db t /tmp/big.json [####################################] 100% ``` This is actually not doing the right thing. The problem is that `sqlite-utils` doesn't include a streaming JSON parser, so it instead reads that entire JSON file into memory first (exhausting the progress bar to 100% instantly) and then does the rest of the work in-memory while the bar sticks at 100%.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1366512990, https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248593835,https://api.github.com/repos/simonw/sqlite-utils/issues/486,1248593835,IC_kwDOCGYnMM5KbAer,9599,2022-09-15T20:37:14Z,2022-09-15T20:37:14Z,OWNER,"I'm going to land this anyway. The lack of a streaming JSON parser is a separate issue, I don't think it should block landing this improvement.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1366512990,