issue_comments
9,947 rows sorted by html_url descending
This data as json, CSV (advanced)
user 367 ✖
- simonw 8,389
- codecov[bot] 182
- fgregg 79
- eyeseast 65
- russss 39
- psychemedia 35
- abdusco 26
- bgrins 24
- cldellow 24
- mroswell 22
- dependabot[bot] 20
- aborruso 19
- chapmanjacobd 18
- chrismp 18
- brandonrobertz 15
- hydrosquall 15
- RhetTbull 15
- jacobian 14
- carlmjohnson 14
- asg017 14
- tballison 13
- wragge 12
- tsibley 11
- rixx 11
- stonebig 11
- frafra 10
- terrycojones 10
- dracos 10
- rayvoelker 10
- maxhawkins 9
- clausjuhl 9
- bobwhitelock 9
- 20after4 8
- UtahDave 8
- tomchristie 8
- bsilverm 8
- 4l1fe 8
- rgieseke 7
- zaneselvans 7
- mhalle 7
- zeluspudding 7
- cobiadigital 7
- jefftriplett 6
- simonwiles 6
- mcarpenter 6
- khimaros 6
- CharlesNepote 6
- ocdtrekkie 6
- davidbgk 5
- khusmann 5
- jaywgraves 5
- rdmurphy 5
- MarkusH 5
- lovasoa 5
- Mjboothaus 5
- dazzag24 5
- ar-jan 5
- xavdid 5
- davidhaley 5
- SteadBytes 5
- dependabot-preview[bot] 5
- jayvdb 4
- fs111 4
- bollwyvl 4
- amjith 4
- ctb 4
- Btibert3 4
- dholth 4
- r4vi 4
- jsfenfen 4
- glasnt 4
- jungle-boogie 4
- ColinMaudry 4
- kbaikov 4
- JBPressac 4
- nitinpaultifr 4
- Kabouik 4
- henry501 4
- pjamargh 4
- benpickles 3
- frankieroberto 3
- obra 3
- janimo 3
- atomotic 3
- ghing 3
- briandorsey 3
- pkoppstein 3
- yozlet 3
- yschimke 3
- philroche 3
- macropin 3
- camallen 3
- wsxiaoys 3
- johnfelipe 3
- mdrovdahl 3
- xrotwang 3
- robroc 3
- dmick 3
- betatim 3
- dufferzafar 3
- Florents-Tselai 3
- ashishdotme 3
- henrikek 3
- swyxio 3
- Segerberg 3
- blairdrummond 3
- jsancho-gpl 3
- kevindkeogh 3
- gk7279 3
- daniel-butler 3
- learning4life 3
- mattmalcher 3
- FabianHertwig 3
- polyrand 3
- justmars 3
- garethr 2
- danp 2
- nelsonjchen 2
- dsisnero 2
- hubgit 2
- ftrain 2
- chrishas35 2
- tannewt 2
- ingenieroariel 2
- coleifer 2
- gavinband 2
- aviflax 2
- tholo 2
- mungewell 2
- frankier 2
- lchski 2
- tmaier 2
- gfrmin 2
- amitkoth 2
- mcint 2
- frosencrantz 2
- eads 2
- virtadpt 2
- leafgarland 2
- glyph 2
- rafguns 2
- strada 2
- adipasquale 2
- eelkevdbos 2
- ligurio 2
- n8henrie 2
- soobrosa 2
- nathancahill 2
- mustafa0x 2
- davidleejy 2
- bsmithgall 2
- noslouch 2
- willingc 2
- nattaylor 2
- durkie 2
- raynae 2
- cclauss 2
- wulfmann 2
- philshem 2
- bram2000 2
- zzeleznick 2
- chris48s 2
- plpxsk 2
- jeqo 2
- nickvazz 2
- aaronyih1 2
- luxint 2
- jussiarpalahti 2
- sachaj 2
- lagolucas 2
- stevecrawshaw 2
- chekos 2
- ad-si 2
- smithdc1 2
- gsajko 2
- jcmkk3 2
- null92 2
- rachelll4 2
- tunguyenatwork 2
- LVerneyPEReN 2
- MichaelTiemannOSC 2
- tmcl-it 2
- anotherjesse 1
- jarib 1
- jokull 1
- fernand0 1
- llimllib 1
- gijs 1
- blaine 1
- ashanan 1
- gravis 1
- nkirsch 1
- tomdyson 1
- mrchrisadams 1
- dkam 1
- harperreed 1
- nileshtrivedi 1
- furilo 1
- kindly 1
- adamwolf 1
- prabhur 1
- palfrey 1
- dmd 1
- pquentin 1
- rubenv 1
- Uninen 1
- rtanglao 1
- carsonyl 1
- nryberg 1
- step21 1
- stefanocudini 1
- rcoup 1
- scoates 1
- hpk42 1
- annapowellsmith 1
- aslakr 1
- thorn0 1
- yurivish 1
- pax 1
- lucapette 1
- jmelloy 1
- Krazybug 1
- dvhthomas 1
- dckc 1
- phubbard 1
- sethvincent 1
- andrewdotn 1
- meatcar 1
- aitoehigie 1
- julienma 1
- michaelmcandrew 1
- drewda 1
- stiles 1
- saulpw 1
- adamalton 1
- terinjokes 1
- thadk 1
- robintw 1
- astrojuanlu 1
- ipmb 1
- steren 1
- aidansteele 1
- mikepqr 1
- 0x1997 1
- jonafato 1
- gwk 1
- knutwannheden 1
- davidszotten 1
- chrislkeller 1
- kevboh 1
- eaubin 1
- yunzheng 1
- lfdebrux 1
- karlcow 1
- heyarne 1
- ryanfox 1
- sopel 1
- cephillips 1
- ryascott 1
- sirnacnud 1
- simonrjones 1
- justinpinkney 1
- merwok 1
- mattkiefer 1
- snth 1
- adarshp 1
- joshmgrant 1
- bcongdon 1
- nickdirienzo 1
- adamjonas 1
- hannseman 1
- kaihendry 1
- urbas 1
- metamoof 1
- brimstone 1
- adamchainz 1
- PabloLerma 1
- heussd 1
- RayBB 1
- BryantD 1
- limar 1
- drkane 1
- Gagravarr 1
- esagara 1
- agguser 1
- dyllan-to-you 1
- justinallen 1
- jordaneremieff 1
- wdccdw 1
- wpears 1
- progpow 1
- DavidPratten 1
- ltrgoddard 1
- costrouc 1
- jratike80 1
- ment4list 1
- ccorcos 1
- choldgraf 1
- qqilihq 1
- fidiego 1
- OverkillGuy 1
- QAInsights 1
- secretGeek 1
- fkuhn 1
- jameslittle230 1
- Profpatsch 1
- dskrad 1
- kwladyka 1
- Carib0u 1
- fatihky 1
- phoenixjun 1
- JesperTreetop 1
- wenhoujx 1
- bapowell 1
- yairlenga 1
- louispotok 1
- ChristopherWilks 1
- Maltazar 1
- eumiro 1
- wuhland 1
- foscoj 1
- dvot197007 1
- kokes 1
- RamiAwar 1
- csusanu 1
- rprimet 1
- metab0t 1
- spdkils 1
- sturzl 1
- jrdmb 1
- robmarkcole 1
- jfeiwell 1
- coisnepe 1
- chmaynard 1
- erlend-aasland 1
- amlestin 1
- alecstein 1
- bendnorman 1
- noklam 1
- jakewilkins 1
- Thomascountz 1
- eigenfoo 1
- GmGniap 1
- rdtq 1
- AnkitKundariya 1
- LucasElArruda 1
- duarteocarmo 1
- mattiaborsoi 1
- sarcasticadmin 1
- yqlbu 1
- abeyerpath 1
- b0b5h4rp13 1
- Rik-de-Kort 1
- patricktrainer 1
- xmichele 1
- miuku 1
- philipp-heinrich 1
- thewchan 1
- publicmatt 1
- izzues 1
- thisismyfuckingusername 1
- kirajano 1
- J450n-4-W 1
- mlaparie 1
- Dhyanesh97 1
- knowledgecamp12 1
- McEazy2700 1
id | html_url ▲ | issue_url | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
1029682294 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029682294 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49X7R2 | simonw 9599 | 2022-02-04T05:53:26Z | 2022-02-04T05:53:26Z | OWNER | This looks fantastic, thanks for all of the work you put into this! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029370537 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029370537 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WvKp | eyeseast 25778 | 2022-02-03T20:25:58Z | 2022-02-03T20:25:58Z | CONTRIBUTOR | OK, I moved all the GIS helpers into `db.py` as methods on `Database` and `Table`, and I put `find_spatialite` back in `utils.py`. I deleted `gis.py`, since there's nothing left it. Docs and tests are updated and passing. I think this is better. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029338360 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029338360 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WnT4 | eyeseast 25778 | 2022-02-03T19:43:56Z | 2022-02-03T19:43:56Z | CONTRIBUTOR | Works for me. I was just looking at how the FTS extensions work and they're just methods, too. So this can be consistent with that. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029335225 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029335225 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49Wmi5 | simonw 9599 | 2022-02-03T19:39:40Z | 2022-02-03T19:39:40Z | OWNER | > I thought about adding these as methods on `Database` and `Table`, and I'm back and forth on it for the same reasons you are. It's certainly cleaner, and it's clearer what you're operating on. I could go either way. > > I do sort of like having all the Spatialite stuff in its own module, just because it's built around an extension you might not have or want, but I don't know if that's a good reason to have a different API. > > You could have `init_spatialite` add methods to `Database` and `Table`, so they're only there if you have Spatialite set up. Is that too clever? It feels too clever. Yeah that's too clever. You know what? I'm pretty confident we are both massively over-thinking this. We should put the methods on `Database` and `Table`! API simplicity and consistency matters more than vague concerns about purity. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029326568 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029326568 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49Wkbo | eyeseast 25778 | 2022-02-03T19:28:26Z | 2022-02-03T19:28:26Z | CONTRIBUTOR | > `from sqlite_utils.utils import find_spatialite` is part of the documented API already: > > https://sqlite-utils.datasette.io/en/3.22.1/python-api.html#finding-spatialite > > To avoid needing to bump the major version number to 4 to indicate a backwards incompatible change, we should keep a `from .gis import find_spatialite` line at the top of `utils.py` such that any existing code with that documented import continues to work. This is fixed now. I had to take out the type annotations for `Database` and `Table` to avoid a circular import, but that's fine and may be moot if these become class methods. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029306428 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029306428 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49Wfg8 | eyeseast 25778 | 2022-02-03T19:03:43Z | 2022-02-03T19:03:43Z | CONTRIBUTOR | I thought about adding these as methods on `Database` and `Table`, and I'm back and forth on it for the same reasons you are. It's certainly cleaner, and it's clearer what you're operating on. I could go either way. I do sort of like having all the Spatialite stuff in its own module, just because it's built around an extension you might not have or want, but I don't know if that's a good reason to have a different API. You could have `init_spatialite` add methods to `Database` and `Table`, so they're only there if you have Spatialite set up. Is that too clever? It feels too clever. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029297971 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029297971 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49Wdcz | simonw 9599 | 2022-02-03T18:52:50Z | 2022-02-03T18:52:50Z | OWNER | I'm not sure I like `name="geometry"` as the default argument to `add_geometry_column` - mainly because of this example here: ```python add_geometry_column(db["locations"], "POINT") create_spatial_index(db["locations"], "geometry") ``` I had to go and look at the code to figure out if `"POINT"` was the name of the column - and I don't like how inconsistent it looks next to the following `create_spatial_index()` call where you DO need to pass the column name. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029296782 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029296782 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WdKO | simonw 9599 | 2022-02-03T18:51:21Z | 2022-02-03T18:51:21Z | OWNER | What do you think about adding these as methods on the `Database` class instead? Then you could do: ```python # This is with an optional argument, which if omitted runs find_spatialite() for you: db.init_spatialite() # Instead of: init_spatialite(db, find_spatialite()) ``` Likewise, the `add_geometry_column` and `create_spatial_index` methods could live on `Table`: ```python # Instead of this: add_geometry_column(db["locations"], "POINT", "geometry") create_spatial_index(db["locations"], "geometry") # Could have this: db["locations"].add_geometry_column("POINT") db["locations"].create_spatial_index("geometry") ``` On the one hand, this is much more consistent with the existing `sqlite-utils` Python API. But on the other hand... this is mixing SpatiaLite functionality directly into the core classes. Is that a good idea, seeing as SpatiaLite is both an optional extension (which can be tricky to install) AND something that has a very different release cadence and quality-of-documentation from SQLite itself? There's a third option: the SpatiaLite could exist on subclasses of `Database` and `Table` - so the above examples would look something like this: ```python from sqlite_utils.gis import SpatiaLiteDatabase db = SpatiaLiteDatabase("geo.db") db.init_spatialite() db["locations"].add_geometry_column("POINT") db["locations"].create_spatial_index("geometry") ``` On the one hand, this would keep the SpatiaLite-specific stuff out of the core Database/Table classes. But it feels a bit untidy to me, especially since it raises the spectre of someone who was already subclassing Database for some reason now needing to instead subclass `SpatiaLiteDatabase` (not too keen on that capitalization) - or even (horror) trying to dabble with multiple inheritance, which can only lead to pain. So I don't have a strong opinion formed on this question yet! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029285985 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029285985 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49Wahh | simonw 9599 | 2022-02-03T18:37:48Z | 2022-02-03T18:37:48Z | OWNER | `from sqlite_utils.utils import find_spatialite` is part of the documented API already: https://sqlite-utils.datasette.io/en/3.22.1/python-api.html#finding-spatialite To avoid needing to bump the major version number to 4 to indicate a backwards incompatible change, we should keep a `from .gis import find_spatialite` line at the top of `utils.py` such that any existing code with that documented import continues to work. | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029273853 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029273853 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WXj9 | simonw 9599 | 2022-02-03T18:23:30Z | 2022-02-03T18:31:21Z | OWNER | OK, this change makes a bunch of sense to me - and also raises some interesting questions about future additions to `sqlite-utils` with regards to SpatiaLite. Would `sqlite-utils add-geometry-column ...` be a good CLI enhancement. for example? I see you've already talked about that in #79 - moving this conversation there! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029180984 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029180984 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WA44 | eyeseast 25778 | 2022-02-03T16:42:04Z | 2022-02-03T16:42:04Z | CONTRIBUTOR | Fixed my spelling. That's a useful thing. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029177700 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029177700 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WAFk | codecov[bot] 22429695 | 2022-02-03T16:38:45Z | 2022-02-04T05:52:39Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/385?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#385](https://codecov.io/gh/simonw/sqlite-utils/pull/385?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (af86b17) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/74586d3cb26fa3cc3412721985ecdc1864c2a31d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (74586d3) will **decrease** coverage by `0.61%`. > The diff coverage is `28.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/385/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/385?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #385 +/- ## ========================================== - Coverage 96.52% 95.91% -0.62% ========================================== Files 6 6 Lines 2389 2421 +32 ========================================== + Hits 2306 2322 +16 - Misses 83 99 +16 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/385?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/385/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.69% <ø> (+0.15%)` … | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029177015 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029177015 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49V_63 | simonw 9599 | 2022-02-03T16:38:02Z | 2022-02-03T16:38:02Z | OWNER | Sorry had missed this - tests should run now. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1029175907 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029175907 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49V_pj | eyeseast 25778 | 2022-02-03T16:36:54Z | 2022-02-03T16:36:54Z | CONTRIBUTOR | @simonw Not sure if you've seen this, but any chance you can run the tests? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
514339130 | https://github.com/simonw/sqlite-utils/pull/38#issuecomment-514339130 | https://api.github.com/repos/simonw/sqlite-utils/issues/38 | MDEyOklzc3VlQ29tbWVudDUxNDMzOTEzMA== | simonw 9599 | 2019-07-23T18:57:33Z | 2019-07-23T18:57:33Z | OWNER | This should switch to using `.get()`, introduced in https://github.com/simonw/sqlite-utils/commit/c65b67ca46f70e2da46a5b945f4ed358173262e9 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | table.update() method 467862459 | |
511219187 | https://github.com/simonw/sqlite-utils/pull/38#issuecomment-511219187 | https://api.github.com/repos/simonw/sqlite-utils/issues/38 | MDEyOklzc3VlQ29tbWVudDUxMTIxOTE4Nw== | simonw 9599 | 2019-07-14T17:05:26Z | 2019-07-14T17:05:26Z | OWNER | I think I need to solve #36 (compound primary key support) first. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | table.update() method 467862459 | |
1009534817 | https://github.com/simonw/sqlite-utils/pull/377#issuecomment-1009534817 | https://api.github.com/repos/simonw/sqlite-utils/issues/377 | IC_kwDOCGYnMM48LEdh | simonw 9599 | 2022-01-11T02:09:38Z | 2022-01-11T02:09:38Z | OWNER | I tested this like so: ``` % wget 'https://raw.githubusercontent.com/wri/global-power-plant-database/master/output_database/global_power_plant_database.csv' % sqlite-utils create-database test.db % sqlite-utils create-table test.db power_plants url text owner text % sqlite-utils schema test.db CREATE TABLE [power_plants] ( [url] TEXT, [owner] TEXT ); % sqlite-utils bulk test.db 'insert into power_plants (url, owner) values (:url, :owner)' global_power_plant_database.csv --csv [------------------------------------] 0% [###################################-] 99% % sqlite-utils tables --counts test.db -t table count ------------ ------- power_plants 33643 ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `sqlite-utils bulk` command 1097477582 | |
1009532125 | https://github.com/simonw/sqlite-utils/pull/377#issuecomment-1009532125 | https://api.github.com/repos/simonw/sqlite-utils/issues/377 | IC_kwDOCGYnMM48LDzd | simonw 9599 | 2022-01-11T02:03:35Z | 2022-01-11T02:03:35Z | OWNER | Documentation: https://github.com/simonw/sqlite-utils/blob/f4ea0d32c0543373eefaa9b9f3911eb07549eecb/docs/cli.rst#executing-sql-in-bulk | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `sqlite-utils bulk` command 1097477582 | |
1009531863 | https://github.com/simonw/sqlite-utils/pull/377#issuecomment-1009531863 | https://api.github.com/repos/simonw/sqlite-utils/issues/377 | IC_kwDOCGYnMM48LDvX | codecov[bot] 22429695 | 2022-01-11T02:03:00Z | 2022-01-11T02:03:00Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/377?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#377](https://codecov.io/gh/simonw/sqlite-utils/pull/377?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (f4ea0d3) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/b6dad08a8389736b7e960cfe9bc719cfc21a98f5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (b6dad08) will **decrease** coverage by `0.01%`. > The diff coverage is `96.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/377/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/377?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #377 +/- ## ========================================== - Coverage 96.52% 96.50% -0.02% ========================================== Files 6 6 Lines 2330 2378 +48 ========================================== + Hits 2249 2295 +46 - Misses 81 83 +2 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/377?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/377/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.49% <94.28%> (-0.0… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `sqlite-utils bulk` command 1097477582 | |
1009272446 | https://github.com/simonw/sqlite-utils/pull/367#issuecomment-1009272446 | https://api.github.com/repos/simonw/sqlite-utils/issues/367 | IC_kwDOCGYnMM48KEZ- | simonw 9599 | 2022-01-10T19:31:08Z | 2022-01-10T19:31:08Z | OWNER | I'm going to implement this in a separate commit from this PR. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Initial prototype of .analyze() methods 1097041471 | |
1008158799 | https://github.com/simonw/sqlite-utils/pull/367#issuecomment-1008158799 | https://api.github.com/repos/simonw/sqlite-utils/issues/367 | IC_kwDOCGYnMM48F0hP | codecov[bot] 22429695 | 2022-01-08T21:36:55Z | 2022-01-09T02:34:44Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/367?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#367](https://codecov.io/gh/simonw/sqlite-utils/pull/367?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (9848eaa) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/a8f9cc6f64f299830834428509940d448b82b4ed?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (a8f9cc6) will **decrease** coverage by `0.20%`. > The diff coverage is `50.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/367/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/367?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #367 +/- ## ========================================== - Coverage 96.44% 96.24% -0.21% ========================================== Files 5 6 +1 Lines 2307 2317 +10 ========================================== + Hits 2225 2230 +5 - Misses 82 87 +5 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/367?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/367/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.15% <28.57%> (-0.42%)` … | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Initial prototype of .analyze() methods 1097041471 | |
1006315145 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006315145 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-yaJ | simonw 9599 | 2022-01-06T06:20:51Z | 2022-01-06T06:20:51Z | OWNER | This is all documented. I'm going to rebase-merge it to keep the individual commits. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006311742 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006311742 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-xk- | simonw 9599 | 2022-01-06T06:12:19Z | 2022-01-06T06:12:19Z | OWNER | Got that working: ``` % echo 'This is cool' | sqlite-utils insert words.db words - --text --convert '({"word": w} for w in text.split())' % sqlite-utils dump words.db BEGIN TRANSACTION; CREATE TABLE [words] ( [word] TEXT ); INSERT INTO "words" VALUES('This'); INSERT INTO "words" VALUES('is'); INSERT INTO "words" VALUES('cool'); COMMIT; ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006309834 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006309834 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-xHK | simonw 9599 | 2022-01-06T06:08:01Z | 2022-01-06T06:08:01Z | OWNER | For `--text` the conversion function should be allowed to return an iterable instead of a dictionary, in which case it will be treated as the full list of records to be inserted. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006301546 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006301546 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-vFq | simonw 9599 | 2022-01-06T05:44:47Z | 2022-01-06T05:44:47Z | OWNER | Just need documentation for `--convert` now against the various different types of input. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006300280 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006300280 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-ux4 | simonw 9599 | 2022-01-06T05:40:45Z | 2022-01-06T05:40:45Z | OWNER | I'm going to rename `--all` to `--text`: > - Use `--text` to write the entire input to a column called "text" To avoid that clash with Python's `all()` function. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006299778 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006299778 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-uqC | simonw 9599 | 2022-01-06T05:39:10Z | 2022-01-06T05:39:10Z | OWNER | `all` is a bad variable name because it clashes with the Python `all()` built-in function. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006295276 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006295276 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-tjs | simonw 9599 | 2022-01-06T05:26:11Z | 2022-01-06T05:26:11Z | OWNER | Here's the traceback if your `--convert` function doesn't return a dict right now: ``` % sqlite-utils insert /tmp/all.db blah /tmp/log.log --convert 'all.upper()' --all Traceback (most recent call last): File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/bin/sqlite-utils", line 33, in <module> sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')()) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1137, in __call__ return self.main(*args, **kwargs) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1062, in main rv = self.invoke(ctx) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1668, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 763, in invoke return __callback(*args, **kwargs) File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 949, in insert insert_upsert_implementation( File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 834, in insert_upsert_implementation db[table].insert_all( File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py", line 2602, in insert_all first_record = next(records) File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py", line 3044, in fix_square_braces for record in records: File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 831, in <genexpr> docs = (decode_base64_values(doc) for doc in docs) File "/Users/simon/Dropbox/Development/s… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006294777 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006294777 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-tb5 | simonw 9599 | 2022-01-06T05:24:54Z | 2022-01-06T05:24:54Z | OWNER | > I added a custom error message for if the user's `--convert` code doesn't return a dict. That turned out to be a bad idea because it meant exhausting the iterator early for the check - before we got to the `.insert_all()` code that breaks the iterator up into chunks. I tried fixing that with `itertools.tee()` to run the generator twice but that's grossly memory-inefficient for large imports. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006288444 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006288444 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-r48 | simonw 9599 | 2022-01-06T05:07:10Z | 2022-01-06T05:07:10Z | OWNER | And here's a demo of `--convert` used with `--all` - I added a custom error message for if the user's `--convert` code doesn't return a dict. ``` % sqlite-utils insert /tmp/all.db blah /tmp/log.log --convert 'all.upper()' --all Error: Records returned by your --convert function must be dicts % sqlite-utils insert /tmp/all.db blah /tmp/log.log --convert '{"all": all.upper()}' --all % sqlite-utils dump /tmp/all.db BEGIN TRANSACTION; CREATE TABLE [blah] ( [all] TEXT ); INSERT INTO "blah" VALUES('INFO: 127.0.0.1:60581 - "GET / HTTP/1.1" 200 OK INFO: 127.0.0.1:60581 - "GET /FOO/-/STATIC/APP.CSS?CEAD5A HTTP/1.1" 200 OK INFO: 127.0.0.1:60581 - "GET /FAVICON.ICO HTTP/1.1" 200 OK INFO: 127.0.0.1:60581 - "GET /FOO/TIDDLYWIKI HTTP/1.1" 200 OK INFO: 127.0.0.1:60581 - "GET /FOO/-/STATIC/APP.CSS?CEAD5A HTTP/1.1" 200 OK INFO: 127.0.0.1:60584 - "GET /FOO/-/STATIC/SQL-FORMATTER-2.3.3.MIN.JS HTTP/1.1" 200 OK INFO: 127.0.0.1:60586 - "GET /FOO/-/STATIC/CODEMIRROR-5.57.0.MIN.JS HTTP/1.1" 200 OK INFO: 127.0.0.1:60585 - "GET /FOO/-/STATIC/CODEMIRROR-5.57.0.MIN.CSS HTTP/1.1" 200 OK INFO: 127.0.0.1:60588 - "GET /FOO/-/STATIC/CODEMIRROR-5.57.0-SQL.MIN.JS HTTP/1.1" 200 OK INFO: 127.0.0.1:60587 - "GET /FOO/-/STATIC/CM-RESIZE-1.0.1.MIN.JS HTTP/1.1" 200 OK INFO: 127.0.0.1:60586 - "GET /FOO/TIDDLYWIKI/TIDDLERS HTTP/1.1" 200 OK INFO: 127.0.0.1:60586 - "GET /FOO/-/STATIC/APP.CSS?CEAD5A HTTP/1.1" 200 OK INFO: 127.0.0.1:60584 - "GET /FOO/-/STATIC/TABLE.JS HTTP/1.1" 200 OK '); COMMIT; ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006284673 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006284673 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-q-B | simonw 9599 | 2022-01-06T04:55:52Z | 2022-01-06T04:55:52Z | OWNER | Test code that just worked for me: ``` sqlite-utils insert /tmp/blah.db blah /tmp/log.log --convert ' bits = line.split() return dict([("b_{}".format(i), bit) for i, bit in enumerate(bits)])' --lines ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006232013 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006232013 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-eHN | simonw 9599 | 2022-01-06T02:21:35Z | 2022-01-06T02:21:35Z | OWNER | I'm having second thoughts about this bit: > Your Python code will be passed a "row" variable representing the imported row, and can return a modified row. > > If you are using `--lines` your code will be passed a "line" variable, and for `--all` an "all" variable. The code in question is this: https://github.com/simonw/sqlite-utils/blob/500a35ad4d91c8a6232134ce9406efec11bedff8/sqlite_utils/utils.py#L296-L303 Do I really want to add the complexity of supporting different variable names there? I think always using `value` might be better. Except... `value` made sense for the existing `sqlite-utils convert` command where you are running a conversion function against the value for the column in the current row - is it confusing if applied to lines or documents or `all`? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006230411 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006230411 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-duL | simonw 9599 | 2022-01-06T02:17:35Z | 2022-01-06T02:17:35Z | OWNER | Documentation: https://github.com/simonw/sqlite-utils/blob/33223856ff7fe746b7b77750fbe5b218531d0545/docs/cli.rst#inserting-unstructured-data-with---lines-and---all - I went with a single section titled "Inserting unstructured data with --lines and --all" | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006220129 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006220129 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-bNh | simonw 9599 | 2022-01-06T01:52:26Z | 2022-01-06T01:52:26Z | OWNER | I'm going to refactor all of the tests for `sqlite-utils insert` into a new `test_cli_insert.py` module. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006219956 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006219956 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-bK0 | codecov[bot] 22429695 | 2022-01-06T01:51:54Z | 2022-01-06T06:22:25Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/361?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#361](https://codecov.io/gh/simonw/sqlite-utils/pull/361?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (b7f0b88) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/f3fd8613113d21d44238a6ec54b375f5aa72c4e0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (f3fd861) will **decrease** coverage by `0.05%`. > The diff coverage is `92.85%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/361/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/361?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #361 +/- ## ========================================== - Coverage 96.49% 96.44% -0.06% ========================================== Files 5 5 Lines 2283 2306 +23 ========================================== + Hits 2203 2224 +21 - Misses 80 82 +2 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/361?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/361/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.49% <92.00%> (-0.1… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
1006219848 | https://github.com/simonw/sqlite-utils/pull/361#issuecomment-1006219848 | https://api.github.com/repos/simonw/sqlite-utils/issues/361 | IC_kwDOCGYnMM47-bJI | simonw 9599 | 2022-01-06T01:51:36Z | 2022-01-06T01:51:36Z | OWNER | So far I've just implemented the new help: ``` % sqlite-utils insert --help Usage: sqlite-utils insert [OPTIONS] PATH TABLE FILE Insert records from FILE into a table, creating the table if it does not already exist. By default the input is expected to be a JSON array of objects. Or: - Use --nl for newline-delimited JSON objects - Use --csv or --tsv for comma-separated or tab-separated input - Use --lines to write each incoming line to a column called "line" - Use --all to write the entire input to a column called "all" You can also use --convert to pass a fragment of Python code that will be used to convert each input. Your Python code will be passed a "row" variable representing the imported row, and can return a modified row. If you are using --lines your code will be passed a "line" variable, and for --all an "all" variable. Options: --pk TEXT Columns to use as the primary key, e.g. id --flatten Flatten nested JSON objects, so {"a": {"b": 1}} becomes {"a_b": 1} --nl Expect newline-delimited JSON -c, --csv Expect CSV input --tsv Expect TSV input --lines Treat each line as a single value called 'line' --all Treat input as a single value called 'all' --convert TEXT Python code to convert each item --import TEXT Python modules to import --delimiter TEXT Delimiter to use for CSV files --quotechar TEXT Quote character to use for CSV/TSV --sniff Detect delimiter and quote character --no-headers CSV file has no header row --batch-size INTEGER Commit every X records --alter Alter existing table to add any missing columns --not-null TEXT Columns that should be created as NOT NULL --default <TEXT TEXT>... Default value that should be set for a column --e… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | --lines and --text and --convert and --import 1094890366 | |
991397907 | https://github.com/simonw/sqlite-utils/pull/347#issuecomment-991397907 | https://api.github.com/repos/simonw/sqlite-utils/issues/347 | IC_kwDOCGYnMM47F4gT | simonw 9599 | 2021-12-11T01:01:40Z | 2021-12-11T01:01:40Z | OWNER | The change I made to that test in #354 might help with this. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against pysqlite3 running SQLite 3.37 1066603133 | |
982137888 | https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982137888 | https://api.github.com/repos/simonw/sqlite-utils/issues/347 | IC_kwDOCGYnMM46ijwg | simonw 9599 | 2021-11-29T23:50:54Z | 2021-11-29T23:50:54Z | OWNER | If I'm going to `skipIf()` those tests I need a way to check if `pysqlite3` is being used. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against pysqlite3 running SQLite 3.37 1066603133 | |
982137293 | https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982137293 | https://api.github.com/repos/simonw/sqlite-utils/issues/347 | IC_kwDOCGYnMM46ijnN | simonw 9599 | 2021-11-29T23:49:29Z | 2021-11-29T23:49:29Z | OWNER | A short term fix would be to skip those tests against `pysqlite3` - but longer term it would be good to address the underlying issue, particularly for the WAL ones (the FTS ones aren't too worrying since if you deliberately try and break the FTS table it's not hugely problematic if you corrupt your database). | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against pysqlite3 running SQLite 3.37 1066603133 | |
982136747 | https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982136747 | https://api.github.com/repos/simonw/sqlite-utils/issues/347 | IC_kwDOCGYnMM46ijer | simonw 9599 | 2021-11-29T23:48:05Z | 2021-11-29T23:48:05Z | OWNER | Some interesting test failures in the version that runs with `pysqlite3`: ``` =========================== short test summary info ============================ FAILED tests/test_cli.py::test_enable_wal - assert 0 == 1 FAILED tests/test_cli.py::test_disable_wal - pysqlite3.dbapi2.OperationalErro... FAILED tests/test_fts.py::test_rebuild_fts[searchable] - pysqlite3.dbapi2.Dat... FAILED tests/test_fts.py::test_rebuild_fts[searchable_fts] - pysqlite3.dbapi2... FAILED tests/test_wal.py::test_enable_disable_wal - pysqlite3.dbapi2.Operatio... ================== 5 failed, 750 passed, 3 skipped in 15.20s =================== ``` https://github.com/simonw/sqlite-utils/runs/4360759085 The WAL errors look like this: ``` E pysqlite3.dbapi2.OperationalError: cannot change into wal mode from within a transaction ``` Triggered by a call to `db.enable_wal()` The FTS errors are caused by tests that try to deliberately corrupt the FTS index by running `fresh_db["searchable_fts_data"].delete_where()` - and then rebuilding it using `rebuild_fts()`: ``` @pytest.mark.parametrize("table_to_fix", ["searchable", "searchable_fts"]) def test_rebuild_fts(fresh_db, table_to_fix): table = fresh_db["searchable"] table.insert(search_records[0]) table.enable_fts(["text", "country"]) # Run a search rows = list(table.search("tanuki")) assert len(rows) == 1 assert { "rowid": 1, "text": "tanuki are running tricksters", "country": "Japan", "not_searchable": "foo", }.items() <= rows[0].items() # Delete from searchable_fts_data fresh_db["searchable_fts_data"].delete_where() # This should have broken the index with pytest.raises(sqlite3.DatabaseError): list(table.search("tanuki")) # Running rebuild_fts() should fix it > fresh_db[table_to_fix].rebuild_fts() tests/test_fts.py:277: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against pysqlite3 running SQLite 3.37 1066603133 | |
982133970 | https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982133970 | https://api.github.com/repos/simonw/sqlite-utils/issues/347 | IC_kwDOCGYnMM46iizS | simonw 9599 | 2021-11-29T23:41:17Z | 2021-11-29T23:41:17Z | OWNER | Took a bit of experimenting to get both `mypy` AND `flake8` to ignore the same line. The incantation that worked was this one: https://github.com/simonw/sqlite-utils/blob/f990e134aa8219b687ff6c261330f36824b5df36/sqlite_utils/utils.py#L8 Order here matters - this did NOT work for both tools: ```python from sqlite3.dump import _iterdump as iterdump # noqa: F401 # type: ignore ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against pysqlite3 running SQLite 3.37 1066603133 | |
982129727 | https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982129727 | https://api.github.com/repos/simonw/sqlite-utils/issues/347 | IC_kwDOCGYnMM46ihw_ | simonw 9599 | 2021-11-29T23:31:58Z | 2021-11-29T23:31:58Z | OWNER | It failed on other Python versions with `mypy`: ``` sqlite_utils/utils.py:8: error: Cannot find implementation or library stub for module named "sqlite3.dump" sqlite_utils/utils.py:8: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against pysqlite3 running SQLite 3.37 1066603133 | |
982129218 | https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982129218 | https://api.github.com/repos/simonw/sqlite-utils/issues/347 | IC_kwDOCGYnMM46ihpC | simonw 9599 | 2021-11-29T23:31:02Z | 2021-11-29T23:31:02Z | OWNER | Here's the test run that's installing `pysqlite3` and that version of SQLite: https://github.com/simonw/sqlite-utils/runs/4360663292 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against pysqlite3 running SQLite 3.37 1066603133 | |
982126665 | https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982126665 | https://api.github.com/repos/simonw/sqlite-utils/issues/347 | IC_kwDOCGYnMM46ihBJ | simonw 9599 | 2021-11-29T23:26:01Z | 2021-11-29T23:33:48Z | OWNER | https://github.com/simonw/sqlite-utils/blob/93b059dd230eae9eaae472b7fbabd4a66feeb79d/.github/workflows/test.yml#L11-L20 This configuration means that the numpy=0, Python=3.10, os=Ubuntu build will additionally use `pysqlite3` with the SQLite 3.37.0. It's failing right now: https://github.com/simonw/sqlite-utils/runs/4360593156 - because `pysqlite3` doesn't provide `.iterdump()`. I can use the workaround from this comment: https://github.com/coleifer/pysqlite3/issues/24#issuecomment-982081267 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against pysqlite3 running SQLite 3.37 1066603133 | |
982123183 | https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982123183 | https://api.github.com/repos/simonw/sqlite-utils/issues/347 | IC_kwDOCGYnMM46igKv | codecov[bot] 22429695 | 2021-11-29T23:20:35Z | 2021-12-11T01:02:19Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/347?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#347](https://codecov.io/gh/simonw/sqlite-utils/pull/347?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (71b6c38) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/213a0ff177f23a35f3b235386366ff132eb879f1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (213a0ff) will **increase** coverage by `0.00%`. > The diff coverage is `100.00%`. > :exclamation: Current head 71b6c38 differs from pull request most recent head 1a7ef2f. Consider uploading reports for the commit 1a7ef2f to get more accurate results [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/347/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/347?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #347 +/- ## ======================================= Coverage 96.51% 96.52% ======================================= Files 5 5 Lines 2270 2271 +1 ======================================= + Hits 2191 2192 +1 Misses 79 79 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/347?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/347/diff?src=pr&el=tree&utm_medium=referral&utm_sou… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against pysqlite3 running SQLite 3.37 1066603133 | |
962259527 | https://github.com/simonw/sqlite-utils/pull/337#issuecomment-962259527 | https://api.github.com/repos/simonw/sqlite-utils/issues/337 | IC_kwDOCGYnMM45WupH | urbas 771193 | 2021-11-05T22:33:02Z | 2021-11-05T22:33:02Z | NONE | Smokes, it looks like there was a bug in click 8.0.2 (fixed in 8.0.3: https://github.com/pallets/click/issues/2089). Meaning this PR is not needed. Closing. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Default values for `--attach` and `--param` options 1046271107 | |
979442854 | https://github.com/simonw/sqlite-utils/pull/333#issuecomment-979442854 | https://api.github.com/repos/simonw/sqlite-utils/issues/333 | IC_kwDOCGYnMM46YRym | simonw 9599 | 2021-11-25T19:47:26Z | 2021-11-25T19:47:26Z | OWNER | I just remembered that there's one other place that this could fit: as a Datasette "insert" plugin. This is vaporware at the moment, but the idea is that Datasette itself could grow a mechanism for importing data, that's driven by plugins. Out of the box Datasette would be able to import CSV and CSV files, similar to `sqlite-utils insert ... --csv` - but plugins would then be able to add support for additional format such as GeoJSON or - in this case - Parquet. The neat thing about having it as a Datasette plugin is that one plugin would enable three different ways of importing data: 1. Via a new `datasette insert ...` CLI option (similar to `sqlite-utils`) 2. Via a web form upload interface, where authenticated Datasette users would be able to upload files 3. Via an API interface, where files could be programatically submitted to a running Datasette server I started fleshing out this idea quite a while ago but didn't make much concrete progress, maybe I should revisit it: - https://github.com/simonw/datasette/issues/1160 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add functionality to read Parquet files. 1039037439 | |
979345527 | https://github.com/simonw/sqlite-utils/pull/333#issuecomment-979345527 | https://api.github.com/repos/simonw/sqlite-utils/issues/333 | IC_kwDOCGYnMM46X6B3 | Florents-Tselai 2118708 | 2021-11-25T16:31:47Z | 2021-11-25T16:31:47Z | NONE | Thanks for your reply @simonw . Tbh, my first attempt was actually the `parquet-to-sqlite` package but I already had Makefiles that relied on `SQLite-utils` and it was less intrusive to my workflow. Maybe I'll revisit that decision. FYI: there's a `[sqlite-parquet-vtable](https://github.com/cldellow/sqlite-parquet-vtable)` I don't think plugins make much sense either. Probably defeats the purpose of simplicity: simple database along with a pip-able package. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add functionality to read Parquet files. 1039037439 | |
974754412 | https://github.com/simonw/sqlite-utils/pull/333#issuecomment-974754412 | https://api.github.com/repos/simonw/sqlite-utils/issues/333 | IC_kwDOCGYnMM46GZJs | simonw 9599 | 2021-11-21T04:35:32Z | 2021-11-21T04:35:32Z | OWNER | Some other recent projects (like trying to get this library to work in JupyterLite) have made me much more cautious about adding new dependencies, especially dependencies like `pyarrow` which require custom C/Rust extensions. There are a few ways this could work though: - Have this as an optional dependency feature - so it only works if the user installs `pyarrow` as well - Implement this as a separate tool, `parquet-to-sqlite` - which could itself depend on `sqlite-utils` - Add a concept of "plugins" to `sqlite-utils`, similar to how those work in Datasette: https://docs.datasette.io/en/stable/plugins.html My favourite option is `parquet-to-sqlite` because that can be built without any additional changes to `sqlite-utils` at all! I find the concept of plugins for `sqlite-utils` interesting. I've so far not had quite enough potential use-cases to convince me this is worthwhile (especially since it should be very easy to build out separate tools entirely), but I'm ready to be convinced that a plugin mechanism would be worthwhile. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add functionality to read Parquet files. 1039037439 | |
942752844 | https://github.com/simonw/sqlite-utils/pull/330#issuecomment-942752844 | https://api.github.com/repos/simonw/sqlite-utils/issues/330 | IC_kwDOCGYnMM44MURM | codecov[bot] 22429695 | 2021-10-13T22:00:31Z | 2021-10-13T22:11:30Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/330?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#330](https://codecov.io/gh/simonw/sqlite-utils/pull/330?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (fc3de90) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/718a8f61bcaed39c04d5d223104056213f8c8516?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (718a8f6) will **not change** coverage. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/330/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/330?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #330 +/- ## ======================================= Coverage 96.59% 96.59% ======================================= Files 5 5 Lines 2230 2230 ======================================= Hits 2154 2154 Misses 76 76 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/330?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](htt… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against Python 3.10 1025726600 | |
916119657 | https://github.com/simonw/sqlite-utils/pull/326#issuecomment-916119657 | https://api.github.com/repos/simonw/sqlite-utils/issues/326 | IC_kwDOCGYnMM42muBp | meatcar 191622 | 2021-09-09T13:54:10Z | 2021-09-09T13:54:10Z | CONTRIBUTOR | dupe of #293? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against 3.10-dev 991237645 | |
915321467 | https://github.com/simonw/sqlite-utils/pull/326#issuecomment-915321467 | https://api.github.com/repos/simonw/sqlite-utils/issues/326 | IC_kwDOCGYnMM42jrJ7 | codecov[bot] 22429695 | 2021-09-08T15:05:53Z | 2021-09-08T15:05:53Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/326?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#326](https://codecov.io/gh/simonw/sqlite-utils/pull/326?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (078a087) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/77c240df56068341561e95e4a412cbfa24dc5bc7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (77c240d) will **not change** coverage. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/326/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/326?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #326 +/- ## ======================================= Coverage 96.58% 96.58% ======================================= Files 5 5 Lines 2223 2223 ======================================= Hits 2147 2147 Misses 76 76 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/326?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](htt… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against 3.10-dev 991237645 | |
968384988 | https://github.com/simonw/sqlite-utils/pull/324#issuecomment-968384988 | https://api.github.com/repos/simonw/sqlite-utils/issues/324 | IC_kwDOCGYnMM45uGHc | simonw 9599 | 2021-11-14T23:25:16Z | 2021-11-14T23:25:16Z | OWNER | Yes this was absolutely the intention! Thanks, I wonder how often I've made that mistake in other projects? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Use python-dateutil package instead of dateutils 988013247 | |
968401459 | https://github.com/simonw/sqlite-utils/pull/322#issuecomment-968401459 | https://api.github.com/repos/simonw/sqlite-utils/issues/322 | IC_kwDOCGYnMM45uKIz | simonw 9599 | 2021-11-15T00:26:42Z | 2021-11-15T00:26:42Z | OWNER | This relates to the fact that dictionaries, lists and tuples get special treatment and are converted to JSON strings, using this code: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L2937-L2947 So the `COLUMN_TYPE_MAPPING` should include those too - right now it looks like this: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L165-L188 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add dict type to be mapped as TEXT in sqllite 979612115 | |
905040902 | https://github.com/simonw/sqlite-utils/pull/321#issuecomment-905040902 | https://api.github.com/repos/simonw/sqlite-utils/issues/321 | IC_kwDOCGYnMM418dQG | simonw 9599 | 2021-08-24T23:25:03Z | 2021-08-24T23:25:03Z | OWNER | I'm going to skip this test on windows. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Ability to insert file contents as text, in addition to blob 978537855 | |
905040307 | https://github.com/simonw/sqlite-utils/pull/321#issuecomment-905040307 | https://api.github.com/repos/simonw/sqlite-utils/issues/321 | IC_kwDOCGYnMM418dGz | simonw 9599 | 2021-08-24T23:23:36Z | 2021-08-24T23:23:36Z | OWNER | https://discuss.python.org/t/pep-597-use-utf-8-for-default-text-file-encoding/1819 says: > Currently, `TextIOWrapper` uses `locale.getpreferredencoding(False)` (hereinafter called “locale encoding”) when encoding is not specified. > ... > Package authors using macOS or Linux may forget that the default encoding is not always UTF-8. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Ability to insert file contents as text, in addition to blob 978537855 | |
905039576 | https://github.com/simonw/sqlite-utils/pull/321#issuecomment-905039576 | https://api.github.com/repos/simonw/sqlite-utils/issues/321 | IC_kwDOCGYnMM418c7Y | simonw 9599 | 2021-08-24T23:21:29Z | 2021-08-24T23:21:29Z | OWNER | Hah, the error here is actually: ``` > assert result.exit_code == 1, result.output E AssertionError: E E assert 0 == 1 E + where 0 = <Result okay>.exit_code ``` So I was expecting an error, but instead the command worked. I suspect this is because on Windows the default character set may not be UTF-8? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Ability to insert file contents as text, in addition to blob 978537855 | |
905037323 | https://github.com/simonw/sqlite-utils/pull/321#issuecomment-905037323 | https://api.github.com/repos/simonw/sqlite-utils/issues/321 | IC_kwDOCGYnMM418cYL | simonw 9599 | 2021-08-24T23:15:29Z | 2021-08-24T23:15:29Z | OWNER | Huh, tests are failing but only on Windows! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Ability to insert file contents as text, in addition to blob 978537855 | |
905022931 | https://github.com/simonw/sqlite-utils/pull/321#issuecomment-905022931 | https://api.github.com/repos/simonw/sqlite-utils/issues/321 | IC_kwDOCGYnMM418Y3T | codecov[bot] 22429695 | 2021-08-24T22:38:38Z | 2021-08-24T23:27:26Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#321](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (db2dd6d) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/9258f4bd8450c951900de998a7bf81ca9b45a014?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (9258f4b) will **increase** coverage by `0.16%`. > The diff coverage is `100.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/321/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #321 +/- ## ========================================== + Coverage 96.41% 96.58% +0.16% ========================================== Files 5 5 Lines 2206 2223 +17 ========================================== + Hits 2127 2147 +20 + Misses 79 76 -3 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/321/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.69% <100.00%> (+0… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Ability to insert file contents as text, in addition to blob 978537855 | |
506784912 | https://github.com/simonw/sqlite-utils/pull/32#issuecomment-506784912 | https://api.github.com/repos/simonw/sqlite-utils/issues/32 | MDEyOklzc3VlQ29tbWVudDUwNjc4NDkxMg== | simonw 9599 | 2019-06-28T15:55:13Z | 2019-06-28T15:55:13Z | OWNER | Documentation, to be added as a subsection of https://sqlite-utils.readthedocs.io/en/latest/python-api.html#adding-foreign-key-constraints ### Adding multiple foreign key constraints at once The final step in adding a new foreign key to a SQLite database is to run VACUUM, to ensure the new foreign key is available in future introspection queries. VACUUM against a large (multi-GB) database can take several minutes or longer. If you are adding multiple foreign keys using table.add_foreign_key(...) these can quickly add up. Instead, you can use db.add_foreign_keys(...) to add multiple foreign keys within a single transaction. This method takes a list of four-tuples, each one specifying a table, column, other_table and other_column. Here's an example adding two foreign keys at once: db.add_foreign_keys([ ("dogs", "breed_id", "breeds", "id"), ("dogs", "home_town_id", "towns", "id") ]) | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | db.add_foreign_keys() method 462094937 | |
896284722 | https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896284722 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | IC_kwDOCGYnMM41bDgy | simonw 9599 | 2021-08-10T20:08:03Z | 2021-08-10T20:08:21Z | OWNER | Spotted a rogue backtick: ![A0147E27-7506-49B0-BEFB-20D99BBFEBAD](https://user-images.githubusercontent.com/9599/128927930-b3333dee-a385-409b-a945-f108e6ea40df.jpeg) | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add reference page to documentation using Sphinx autodoc 965143346 | |
896200682 | https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896200682 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | IC_kwDOCGYnMM41au_q | simonw 9599 | 2021-08-10T18:03:40Z | 2021-08-10T18:03:40Z | OWNER | Adding type signatures to `create_table()` and `.create_table_sql()` is a bit too involved, I'll do that in a separate issue. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add reference page to documentation using Sphinx autodoc 965143346 | |
896186025 | https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896186025 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | IC_kwDOCGYnMM41arap | simonw 9599 | 2021-08-10T17:42:51Z | 2021-08-10T17:42:51Z | OWNER | That worked! https://sqlite-utils.datasette.io/en/autodoc/reference.html | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add reference page to documentation using Sphinx autodoc 965143346 | |
896182934 | https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896182934 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | IC_kwDOCGYnMM41aqqW | simonw 9599 | 2021-08-10T17:38:44Z | 2021-08-10T17:38:44Z | OWNER | From https://docs.readthedocs.io/en/stable/config-file/v2.html#packages it looks like I can tell Read The Docs to run `pip install -e .` using a `.readthedocs.yaml` configuration: ```yaml version: 2 sphinx: configuration: docs/conf.py python: version: "3.9" install: - method: pip path: . extra_requirements: - docs ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add reference page to documentation using Sphinx autodoc 965143346 | |
896180956 | https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896180956 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | IC_kwDOCGYnMM41aqLc | simonw 9599 | 2021-08-10T17:35:51Z | 2021-08-10T17:35:51Z | OWNER | Reading the rest of https://sphinx-rtd-tutorial.readthedocs.io/en/latest/sphinx-config.html#autodoc-configuration it suggests using a `requirements.txt` file to install dependencies - but I use `setup.py` for that so I need to figure out a different pattern here. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add reference page to documentation using Sphinx autodoc 965143346 | |
896175438 | https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896175438 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | IC_kwDOCGYnMM41ao1O | simonw 9599 | 2021-08-10T17:28:19Z | 2021-08-10T17:28:19Z | OWNER | https://sphinx-rtd-tutorial.readthedocs.io/en/latest/sphinx-config.html#autodoc-configuration says do something like this at the top of `conf.py`: ```python import os import sys sys.path.insert(0, os.path.abspath('../../simpleble/')) ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add reference page to documentation using Sphinx autodoc 965143346 | |
896174456 | https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896174456 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | IC_kwDOCGYnMM41aol4 | simonw 9599 | 2021-08-10T17:27:01Z | 2021-08-10T17:27:01Z | OWNER | Docs are now building at https://sqlite-utils.datasette.io/en/autodoc/reference.html But there's a problem! The page is semi-blank: <img width="663" alt="Reference_—_sqlite-utils_3_15-6-gc11ff89_documentation" src="https://user-images.githubusercontent.com/9599/128906373-7463e0c9-3b08-4b15-a73a-84b8ba6ab342.png"> I need to teach Read The Docs how to ensure `sqlite_utils` is available for introspection. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add reference page to documentation using Sphinx autodoc 965143346 | |
896162082 | https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896162082 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | IC_kwDOCGYnMM41alki | codecov[bot] 22429695 | 2021-08-10T17:10:39Z | 2021-08-10T23:07:35Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#312](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (43bc064) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/ee469e3122d6f5973ec2584c1580d930daca2e7c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (ee469e3) will **decrease** coverage by `0.02%`. > The diff coverage is `96.84%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/312/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #312 +/- ## ========================================== - Coverage 96.30% 96.28% -0.03% ========================================== Files 5 5 Lines 2168 2179 +11 ========================================== + Hits 2088 2098 +10 - Misses 80 81 +1 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/312/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.91% <96.84%> (-0.08%)` … | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add reference page to documentation using Sphinx autodoc 965143346 | |
896156971 | https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896156971 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | IC_kwDOCGYnMM41akUr | simonw 9599 | 2021-08-10T17:04:22Z | 2021-08-10T17:05:59Z | OWNER | I'm going to get Read The Docs to build the docs for this branch too - on https://readthedocs.org/projects/sqlite-utils/versions/ I am clicking this button: <img width="881" alt="Versions___Read_the_Docs" src="https://user-images.githubusercontent.com/9599/128903015-0f2649e7-34a8-4dfd-984c-b1b5bd8a98be.png"> I then set it to "active" (so pushes to the branch will build it) and "hidden" (so it wouldn't show up in search or in the navigation menu). https://docs.readthedocs.io/en/stable/versions.html#version-states <img width="846" alt="autodoc_-_sqlite-utils___Read_the_Docs" src="https://user-images.githubusercontent.com/9599/128903217-efe827c6-4f54-42dd-943e-ddf9ab3a05d8.png"> | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add reference page to documentation using Sphinx autodoc 965143346 | |
896154028 | https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896154028 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | IC_kwDOCGYnMM41ajms | simonw 9599 | 2021-08-10T17:01:06Z | 2021-08-10T17:01:06Z | OWNER | On Python 3.6: ``` sqlite_utils/db.py:366: in Database def tables(self) -> List[Table]: E NameError: name 'Table' is not defined ``` Python 3.7 can fix this with `from __future__ import annotations` but since we still support 3.6 I'll have to use a string instead. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add reference page to documentation using Sphinx autodoc 965143346 | |
890553014 | https://github.com/simonw/sqlite-utils/pull/303#issuecomment-890553014 | https://api.github.com/repos/simonw/sqlite-utils/issues/303 | IC_kwDOCGYnMM41FMK2 | codecov[bot] 22429695 | 2021-08-01T16:53:35Z | 2021-08-02T04:45:19Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#303](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (4c3bf97) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/c7e8d72be9fe8fe0811f685a18eebc637662d41b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (c7e8d72) will **increase** coverage by `0.24%`. > The diff coverage is `100.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/303/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #303 +/- ## ========================================== + Coverage 96.04% 96.28% +0.24% ========================================== Files 4 5 +1 Lines 1998 2129 +131 ========================================== + Hits 1919 2050 +131 Misses 79 79 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/303/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `94.91% <100.00%> (+0… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils convert command and db[table].convert(...) method 957536983 | |
868728092 | https://github.com/simonw/sqlite-utils/pull/293#issuecomment-868728092 | https://api.github.com/repos/simonw/sqlite-utils/issues/293 | MDEyOklzc3VlQ29tbWVudDg2ODcyODA5Mg== | simonw 9599 | 2021-06-25T17:39:35Z | 2021-06-25T17:39:35Z | OWNER | Here's more about this problem: https://github.com/numpy/numpy/issues/15947 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against Python 3.10-dev 929748885 | |
868134040 | https://github.com/simonw/sqlite-utils/pull/293#issuecomment-868134040 | https://api.github.com/repos/simonw/sqlite-utils/issues/293 | MDEyOklzc3VlQ29tbWVudDg2ODEzNDA0MA== | simonw 9599 | 2021-06-25T01:49:44Z | 2021-06-25T01:50:33Z | OWNER | Test failed on 3.10 with `numpy` on macOS: ``` sqlite_utils/__init__.py:1: in <module> 11 from .db import Database 12 sqlite_utils/db.py:48: in <module> 13 import numpy as np # type: ignore 14 ../../../hostedtoolcache/Python/3.10.0-beta.3/x64/lib/python3.10/site-packages/numpy/__init__.py:391: in <module> 15 raise RuntimeError(msg) 16 E RuntimeError: Polyfit sanity test emitted a warning, most likely due to using a buggy Accelerate backend. If you compiled yourself, more information is available at https://numpy.org/doc/stable/user/building.html#accelerated-blas-lapack-libraries Otherwise report this to the vendor that provided NumPy. 17 E RankWarning: Polyfit may be poorly conditioned 18 Error: Process completed with exit code 4. ``` | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1} | Test against Python 3.10-dev 929748885 | |
868125750 | https://github.com/simonw/sqlite-utils/pull/293#issuecomment-868125750 | https://api.github.com/repos/simonw/sqlite-utils/issues/293 | MDEyOklzc3VlQ29tbWVudDg2ODEyNTc1MA== | codecov[bot] 22429695 | 2021-06-25T01:42:43Z | 2021-06-25T01:42:43Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/293?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#293](https://codecov.io/gh/simonw/sqlite-utils/pull/293?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (ae0f46a) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/747be6057d09a4e5d9d726e29d5cf99b10c59dea?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (747be60) will **not change** coverage. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/293/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/293?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #293 +/- ## ======================================= Coverage 96.03% 96.03% ======================================= Files 4 4 Lines 1994 1994 ======================================= Hits 1915 1915 Misses 79 79 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/293?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](htt… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Test against Python 3.10-dev 929748885 | |
864092515 | https://github.com/simonw/sqlite-utils/pull/277#issuecomment-864092515 | https://api.github.com/repos/simonw/sqlite-utils/issues/277 | MDEyOklzc3VlQ29tbWVudDg2NDA5MjUxNQ== | simonw 9599 | 2021-06-18T14:47:57Z | 2021-06-18T14:47:57Z | OWNER | This is a neat improvement. | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0} | add -h support closes #276 923612361 | |
863205049 | https://github.com/simonw/sqlite-utils/pull/277#issuecomment-863205049 | https://api.github.com/repos/simonw/sqlite-utils/issues/277 | MDEyOklzc3VlQ29tbWVudDg2MzIwNTA0OQ== | codecov[bot] 22429695 | 2021-06-17T12:40:49Z | 2021-06-17T12:40:49Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/277?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#277](https://codecov.io/gh/simonw/sqlite-utils/pull/277?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (abbd324) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/a19ce1a4d0048d389411cfe11a5dbe4c503720e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (a19ce1a) will **increase** coverage by `0.00%`. > The diff coverage is `100.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/277/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/277?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #277 +/- ## ======================================= Coverage 96.06% 96.06% ======================================= Files 4 4 Lines 1828 1829 +1 ======================================= + Hits 1756 1757 +1 Misses 72 72 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/277?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/277/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `94.03% <100.00%> (+<0.01%)` | :arrow_up: | ------… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | add -h support closes #276 923612361 | |
864099764 | https://github.com/simonw/sqlite-utils/pull/273#issuecomment-864099764 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | MDEyOklzc3VlQ29tbWVudDg2NDA5OTc2NA== | simonw 9599 | 2021-06-18T14:59:27Z | 2021-06-18T14:59:27Z | OWNER | I'm going to merge this as-is and work on the JSON/TSV support in a separate issue. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils memory command for directly querying CSV/JSON data 922099793 | |
862817185 | https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862817185 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | MDEyOklzc3VlQ29tbWVudDg2MjgxNzE4NQ== | codecov[bot] 22429695 | 2021-06-17T00:15:34Z | 2021-06-17T00:15:34Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/273?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > :exclamation: No coverage uploaded for pull request base (`main@78aebb6`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#section-missing-base-commit). > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/273/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/273?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #273 +/- ## ======================================= Coverage ? 96.10% ======================================= Files ? 4 Lines ? 1873 Branches ? 0 ======================================= Hits ? 1800 Misses ? 73 Partials ? 0 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/273?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/273?src=pr&el=footer&utm_medium=referr… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils memory command for directly querying CSV/JSON data 922099793 | |
862605436 | https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862605436 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | MDEyOklzc3VlQ29tbWVudDg2MjYwNTQzNg== | simonw 9599 | 2021-06-16T18:19:05Z | 2021-06-16T18:19:05Z | OWNER | `--attach` documentation: https://github.com/simonw/sqlite-utils/blob/192dc2c5b73bd836ab8e2e5fed4b36c6ea02f250/docs/cli.rst#joining-in-memory-data-against-existing-databases-using-attach | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils memory command for directly querying CSV/JSON data 922099793 | |
862046009 | https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862046009 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | MDEyOklzc3VlQ29tbWVudDg2MjA0NjAwOQ== | simonw 9599 | 2021-06-16T05:15:38Z | 2021-06-16T05:15:38Z | OWNER | I'm going to add a `--encoding` option - it will affect ALL CSV input files, so if you have CSV files with different encodings you'll need to sort that mess out yourself (likely by importing each CSV file separately into a database using `sqlite-utils insert` with different `--encoding` values). | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils memory command for directly querying CSV/JSON data 922099793 | |
862045639 | https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862045639 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | MDEyOklzc3VlQ29tbWVudDg2MjA0NTYzOQ== | simonw 9599 | 2021-06-16T05:14:38Z | 2021-06-16T05:14:38Z | OWNER | Can't share much code though since a bunch of that `insert` stuff is specific to that command - showing progress bars, returning errors on illegal option combinations etc. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils memory command for directly querying CSV/JSON data 922099793 | |
862045438 | https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862045438 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | MDEyOklzc3VlQ29tbWVudDg2MjA0NTQzOA== | simonw 9599 | 2021-06-16T05:14:00Z | 2021-06-16T05:14:00Z | OWNER | I should probably refactor the CSV/JSON/loading stuff into a function in `utils.py` in order to share some of the implementation with the existing `sqlite-utils insert` code: https://github.com/simonw/sqlite-utils/blob/287cdcae8908916687f2ecccc87c38549d004ac6/sqlite_utils/cli.py#L691-L734 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils memory command for directly querying CSV/JSON data 922099793 | |
862043974 | https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862043974 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | MDEyOklzc3VlQ29tbWVudDg2MjA0Mzk3NA== | simonw 9599 | 2021-06-16T05:10:12Z | 2021-06-16T05:10:12Z | OWNER | I can stop promoting `:memory:` here and promote `memory` instead: https://github.com/simonw/sqlite-utils/blob/c7234cae8336b8525034e8f917d82dd0699abd42/docs/cli.rst#L83-L86 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils memory command for directly querying CSV/JSON data 922099793 | |
862042110 | https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862042110 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | MDEyOklzc3VlQ29tbWVudDg2MjA0MjExMA== | simonw 9599 | 2021-06-16T05:05:51Z | 2021-06-16T05:06:11Z | OWNER | Initial documentation is here: https://github.com/simonw/sqlite-utils/blob/c7234cae8336b8525034e8f917d82dd0699abd42/docs/cli.rst#running-queries-directly-against-csv-data It only talks about CSV at the moment - needs to be updated to mention JSON too once that is implemented. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils memory command for directly querying CSV/JSON data 922099793 | |
843702392 | https://github.com/simonw/sqlite-utils/pull/258#issuecomment-843702392 | https://api.github.com/repos/simonw/sqlite-utils/issues/258 | MDEyOklzc3VlQ29tbWVudDg0MzcwMjM5Mg== | simonw 9599 | 2021-05-19T02:47:37Z | 2021-05-19T02:47:37Z | OWNER | I'm going to merge this and add a test - thanks! | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fixing insert from JSON containing strings with non-ascii characters … 868191959 | |
843705533 | https://github.com/simonw/sqlite-utils/pull/254#issuecomment-843705533 | https://api.github.com/repos/simonw/sqlite-utils/issues/254 | MDEyOklzc3VlQ29tbWVudDg0MzcwNTUzMw== | simonw 9599 | 2021-05-19T02:57:22Z | 2021-05-19T02:57:22Z | OWNER | Thanks! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix incorrect create-table cli description 857280617 | |
901344634 | https://github.com/simonw/sqlite-utils/pull/247#issuecomment-901344634 | https://api.github.com/repos/simonw/sqlite-utils/issues/247 | IC_kwDOCGYnMM41uW16 | codecov[bot] 22429695 | 2021-08-18T18:42:54Z | 2021-08-18T18:42:54Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#247](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (af989af) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/1fe73c898b44695052f1a9ca832818d50cecf662?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (1fe73c8) will **decrease** coverage by `0.03%`. > The diff coverage is `85.71%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/247/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #247 +/- ## ========================================== - Coverage 96.28% 96.24% -0.04% ========================================== Files 5 5 Lines 2179 2186 +7 ========================================== + Hits 2098 2104 +6 - Misses 81 82 +1 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/247/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.84% <85.71%> (-0.08%)` … | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | FTS quote functionality from datasette 832687563 | |
901338988 | https://github.com/simonw/sqlite-utils/pull/247#issuecomment-901338988 | https://api.github.com/repos/simonw/sqlite-utils/issues/247 | IC_kwDOCGYnMM41uVds | simonw 9599 | 2021-08-18T18:33:39Z | 2021-08-18T18:33:39Z | OWNER | This was also requested in #296. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | FTS quote functionality from datasette 832687563 | |
843705721 | https://github.com/simonw/sqlite-utils/pull/245#issuecomment-843705721 | https://api.github.com/repos/simonw/sqlite-utils/issues/245 | MDEyOklzc3VlQ29tbWVudDg0MzcwNTcyMQ== | simonw 9599 | 2021-05-19T02:58:01Z | 2021-05-19T02:58:01Z | OWNER | Thanks very much. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Correct some typos 830803173 | |
843705806 | https://github.com/simonw/sqlite-utils/pull/244#issuecomment-843705806 | https://api.github.com/repos/simonw/sqlite-utils/issues/244 | MDEyOklzc3VlQ29tbWVudDg0MzcwNTgwNg== | simonw 9599 | 2021-05-19T02:58:18Z | 2021-05-19T02:58:18Z | OWNER | Thanks! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Typo in upsert example 820468864 | |
778841547 | https://github.com/simonw/sqlite-utils/pull/225#issuecomment-778841547 | https://api.github.com/repos/simonw/sqlite-utils/issues/225 | MDEyOklzc3VlQ29tbWVudDc3ODg0MTU0Nw== | simonw 9599 | 2021-02-14T21:04:13Z | 2021-02-14T21:04:13Z | OWNER | I added a test and fixed this in #234 - thanks for the fix. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | fix for problem in Table.insert_all on search for columns per chunk of rows 797159961 | |
778834504 | https://github.com/simonw/sqlite-utils/pull/225#issuecomment-778834504 | https://api.github.com/repos/simonw/sqlite-utils/issues/225 | MDEyOklzc3VlQ29tbWVudDc3ODgzNDUwNA== | simonw 9599 | 2021-02-14T20:09:30Z | 2021-02-14T20:09:30Z | OWNER | Thanks for this. I'm going to try and get the test suite to run in Windows on GitHub Actions. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | fix for problem in Table.insert_all on search for columns per chunk of rows 797159961 | |
778828495 | https://github.com/simonw/sqlite-utils/pull/224#issuecomment-778828495 | https://api.github.com/repos/simonw/sqlite-utils/issues/224 | MDEyOklzc3VlQ29tbWVudDc3ODgyODQ5NQ== | simonw 9599 | 2021-02-14T19:31:06Z | 2021-02-14T19:31:06Z | OWNER | I'm going to add a `offset=` parameter to support this case. Thanks for the suggestion! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add fts offset docs. 792297010 | |
765678057 | https://github.com/simonw/sqlite-utils/pull/224#issuecomment-765678057 | https://api.github.com/repos/simonw/sqlite-utils/issues/224 | MDEyOklzc3VlQ29tbWVudDc2NTY3ODA1Nw== | polyrand 37962604 | 2021-01-22T20:53:06Z | 2021-01-23T20:13:27Z | NONE | I'm using the FTS methods in sqlite-utils for this website: [drwn.io](https://drwn.io/). I wanted to get pagination to have some kind of infinite scrolling in the landing page, and I ended up using that. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add fts offset docs. 792297010 | |
743956666 | https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743956666 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | MDEyOklzc3VlQ29tbWVudDc0Mzk1NjY2Ng== | simonw 9599 | 2020-12-13T05:44:49Z | 2020-12-13T05:44:49Z | OWNER | Example output: ``` % sqlite-utils analyze-tables github.db tags tags.repo: (1/3) Total rows: 261 Null rows: 0 Blank rows: 0 Distinct values: 14 Most common: 88: 107914493 75: 140912432 27: 206156866 21: 207052882 17: 197431109 8: 197882382 5: 256834907 5: 205429375 4: 248903544 3: 206202864 Least common: 1: 209590345 2: 206649770 2: 303218369 3: 206202864 3: 213286752 4: 248903544 5: 205429375 5: 256834907 8: 197882382 17: 197431109 tags.name: (2/3) Total rows: 261 Null rows: 0 Blank rows: 0 Distinct values: 175 Most common: 10: 0.2 9: 0.1 7: 0.3 6: 0.4 5: 0.7 5: 0.5 5: 0.1a 4: 0.9 4: 0.8 4: 0.6 Least common: 1: 0.1.1 1: 0.11.1 1: 0.1a2 1: 0.20.1 1: 0.21.1 1: 0.21.2 1: 0.21.3 1: 0.22 1: 0.22.1 1: 0.23 tags.sha: (3/3) Total rows: 261 Null rows: 0 Blank rows: 0 Distinct values: 261 ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils analyze-tables command and table.analyze_column() method 763320133 | |
743708524 | https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708524 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | MDEyOklzc3VlQ29tbWVudDc0MzcwODUyNA== | simonw 9599 | 2020-12-12T05:48:20Z | 2020-12-12T05:48:32Z | OWNER | ``` % sqlite-utils analyze-tables ../datasette/fixtures.db facetable --column pk 1/1: ColumnDetails(table='facetable', column='pk', total_rows=15, num_null=0, num_blank=0, num_distinct=15, most_common=None, least_common=None) ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils analyze-tables command and table.analyze_column() method 763320133 | |
743708325 | https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708325 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | MDEyOklzc3VlQ29tbWVudDc0MzcwODMyNQ== | simonw 9599 | 2020-12-12T05:46:27Z | 2020-12-12T05:46:27Z | OWNER | It would be neat if you could optionally specify a subset of columns to analyze, using `-c` or `--column`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils analyze-tables command and table.analyze_column() method 763320133 | |
743708169 | https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708169 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | MDEyOklzc3VlQ29tbWVudDc0MzcwODE2OQ== | simonw 9599 | 2020-12-12T05:44:46Z | 2020-12-12T05:44:46Z | OWNER | If there are less than ten values is it worth outputting them twice, once in `most_common` and then in reverse in `least_common`? Feels redundant - I think I should leave `least_common` empty in that case. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils analyze-tables command and table.analyze_column() method 763320133 | |
743708080 | https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708080 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | MDEyOklzc3VlQ29tbWVudDc0MzcwODA4MA== | simonw 9599 | 2020-12-12T05:43:45Z | 2020-12-12T05:43:45Z | OWNER | CLI output looks like this at the moment, which is bad: ``` % sqlite-utils analyze-tables ../datasette/fixtures.db facetable 1/10: ColumnDetails(table='facetable', column='pk', total_rows=15, num_null=0, num_blank=0, num_distinct=15, most_common=None, least_common=None) 2/10: ColumnDetails(table='facetable', column='created', total_rows=15, num_null=0, num_blank=0, num_distinct=4, most_common=[('2019-01-17 08:00:00', 4), ('2019-01-15 08:00:00', 4), ('2019-01-14 08:00:00', 4), ('2019-01-16 08:00:00', 3)], least_common=[('2019-01-16 08:00:00', 3), ('2019-01-14 08:00:00', 4), ('2019-01-15 08:00:00', 4), ('2019-01-17 08:00:00', 4)]) 3/10: ColumnDetails(table='facetable', column='planet_int', total_rows=15, num_null=0, num_blank=0, num_distinct=2, most_common=[(1, 14), (2, 1)], least_common=[(2, 1), (1, 14)]) 4/10: ColumnDetails(table='facetable', column='on_earth', total_rows=15, num_null=0, num_blank=0, num_distinct=2, most_common=[(1, 14), (0, 1)], least_common=[(0, 1), (1, 14)]) 5/10: ColumnDetails(table='facetable', column='state', total_rows=15, num_null=0, num_blank=0, num_distinct=3, most_common=[('CA', 10), ('MI', 4), ('MC', 1)], least_common=[('MC', 1), ('MI', 4), ('CA', 10)]) 6/10: ColumnDetails(table='facetable', column='city_id', total_rows=15, num_null=0, num_blank=0, num_distinct=4, most_common=[(1, 6), (3, 4), (2, 4), (4, 1)], least_common=[(4, 1), (2, 4), (3, 4), (1, 6)]) 7/10: ColumnDetails(table='facetable', column='neighborhood', total_rows=15, num_null=0, num_blank=0, num_distinct=14, most_common=[('Downtown', 2), ('Tenderloin', 1), ('SOMA', 1), ('Mission', 1), ('Mexicantown', 1), ('Los Feliz', 1), ('Koreatown', 1), ('Hollywood', 1), ('Hayes Valley', 1), ('Greektown', 1)], least_common=[('Arcadia Planitia', 1), ('Bernal Heights', 1), ('Corktown', 1), ('Dogpatch', 1), ('Greektown', 1), ('Hayes Valley', 1), ('Hollywood', 1), ('Koreatown', 1), ('Los Feliz', 1), ('Mexicantown', 1)]) 8/10: ColumnDetails(table='facetable', column='tags', total_rows=15, num_null=0, num_blank=0, num_distinct=3,… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils analyze-tables command and table.analyze_column() method 763320133 | |
743707969 | https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743707969 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | MDEyOklzc3VlQ29tbWVudDc0MzcwNzk2OQ== | simonw 9599 | 2020-12-12T05:42:26Z | 2020-12-12T05:43:06Z | OWNER | Should truncate values in the least/most common JSON array to a sensible length, otherwise you end up with stuff like this: ```json [ [ "b'\\x00\\x05barry\\x03\\x01\\x02\\x00\\x00\\x03cat\\x03\\x01\\x03\\x00\\x00\\x03dog\\x08\\x01\\x01\\x01\\x03\\x00\\x01\\x03\\x00\\x00\\x07panther\\x05\\x01\\x01\\x02\\x02\\x00\\x01\\x03uma\\x05\\x02\\x01\\x02\\x02\\x00\\x00\\x04sara\\x05\\x02\\x01\\x01\\x02\\x00\\x00\\x05terry\\x08\\x01\\x01\\x01\\x02\\x00\\x01\\x02\\x00\\x00\\x06weasel\\x05\\x02\\x01\\x01\\x03\\x00'", 1 ] ] ``` This example also shows that binary values (like those in `_fts` tables) look a bit weird, but I think I'm OK with that since binary data can't be represented neatly in JSON anyway. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils analyze-tables command and table.analyze_column() method 763320133 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
created_at (date) >1000 ✖