issue_comments
928 rows where author_association = "NONE" sorted by issue_url descending
This data as json, CSV (advanced)
Suggested facets: created_at (date)
id | html_url | issue_url ▲ | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
928790381 | https://github.com/simonw/sqlite-utils/issues/98#issuecomment-928790381 | https://api.github.com/repos/simonw/sqlite-utils/issues/98 | IC_kwDOCGYnMM43XDdt | patricktrainer 36834097 | 2021-09-28T04:38:44Z | 2021-09-28T04:38:44Z | NONE | Hi @simonw - wondering if you might be able to shed some light here. I've seemed to reproduce this issue. Here's the stacktrace: ``` ... db["potholes"].insert(pothole, pk='id', alter=True, replace=True) ... Traceback (most recent call last): File "<stdin>", line 3, in <module> File "/Users/patricktrainer/.pyenv/versions/3.9.0/lib/python3.9/site-packages/sqlite_utils/db.py", line 2481, in insert return self.insert_all( File "/Users/patricktrainer/.pyenv/versions/3.9.0/lib/python3.9/site-packages/sqlite_utils/db.py", line 2596, in insert_all self.insert_chunk( File "/Users/patricktrainer/.pyenv/versions/3.9.0/lib/python3.9/site-packages/sqlite_utils/db.py", line 2424, in insert_chunk row = list(self.rows_where("rowid = ?", [self.last_rowid]))[0] IndexError: list index out of range ``` Interesting enough, I found that omitting the `pk` param does not throw the error. Let me know how I can help out! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records 597671518 | |
614073859 | https://github.com/simonw/sqlite-utils/issues/97#issuecomment-614073859 | https://api.github.com/repos/simonw/sqlite-utils/issues/97 | MDEyOklzc3VlQ29tbWVudDYxNDA3Mzg1OQ== | betatim 1448859 | 2020-04-15T14:29:30Z | 2020-04-15T14:29:30Z | NONE | Woah! Thanks a lot. Next time I will add a more obvious/explicit "if you like this idea let me know I'd love to work on it to get my feet wet here" :D | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Adding a "recreate" flag to the `Database` constructor 593751293 | |
593122605 | https://github.com/simonw/sqlite-utils/issues/89#issuecomment-593122605 | https://api.github.com/repos/simonw/sqlite-utils/issues/89 | MDEyOklzc3VlQ29tbWVudDU5MzEyMjYwNQ== | chrishas35 35075 | 2020-03-01T17:33:11Z | 2020-03-01T17:33:11Z | NONE | If you're happy with the proposed implementation, I have code & tests written that I'll get ready for a PR. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Ability to customize columns used by extracts= feature 573578548 | |
586683572 | https://github.com/simonw/sqlite-utils/issues/86#issuecomment-586683572 | https://api.github.com/repos/simonw/sqlite-utils/issues/86 | MDEyOklzc3VlQ29tbWVudDU4NjY4MzU3Mg== | foscoj 8149512 | 2020-02-16T09:03:54Z | 2020-02-16T09:03:54Z | NONE | Probably the best option to just throw the error. Is there any active dev chan where we could post the issue to python sqlite3? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Problem with square bracket in CSV column name 564579430 | |
464341721 | https://github.com/simonw/sqlite-utils/issues/8#issuecomment-464341721 | https://api.github.com/repos/simonw/sqlite-utils/issues/8 | MDEyOklzc3VlQ29tbWVudDQ2NDM0MTcyMQ== | psychemedia 82988 | 2019-02-16T12:08:41Z | 2019-02-16T12:08:41Z | NONE | We also get an error if a column name contains a `.` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Problems handling column names containing spaces or - 403922644 | |
482994231 | https://github.com/simonw/sqlite-utils/issues/8#issuecomment-482994231 | https://api.github.com/repos/simonw/sqlite-utils/issues/8 | MDEyOklzc3VlQ29tbWVudDQ4Mjk5NDIzMQ== | psychemedia 82988 | 2019-04-14T15:04:07Z | 2019-04-14T15:29:33Z | NONE | PLEASE IGNORE THE BELOW... I did a package update and rebuilt the kernel I was working in... may just have been an old version of sqlite_utils, seems to be working now. (Too many containers / too many environments!) Has an issue been reintroduced here with FTS? eg I'm getting an error thrown by spaces in column names here: ``` /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order) def enable_fts(self, columns, fts_version="FTS5"): --> 329 "Enables FTS on the specified columns" 330 sql = """ 331 CREATE VIRTUAL TABLE "{table}_fts" USING {fts_version} ( ``` when trying an `insert_all`. Also, if a col has a `.` in it, I seem to get: ``` /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order) 327 jsonify_if_needed(record.get(key, None)) for key in all_columns 328 ) --> 329 result = self.db.conn.execute(sql, values) 330 self.db.conn.commit() 331 self.last_id = result.lastrowid OperationalError: near ".": syntax error ``` (Can't post a worked minimal example right now; racing trying to build something against a live timing screen that will stop until next weekend in an hour or two...) PS Hmmm I did a test and they seem to work; I must be messing up s/where else... ``` import sqlite3 from sqlite_utils import Database dbname='testingDB_sqlite_utils.db' #!rm $dbname conn = sqlite3.connect(dbname, timeout=10) #Setup database tables c = conn.cursor() setup=''' CREATE TABLE IF NOT EXISTS "test1" ( "NO" INTEGER, "NAME" TEXT ); CREATE TABLE IF NOT EXISTS "test2" ( "NO" INTEGER, `TIME OF DAY` TEXT ); CREATE TABLE IF NOT EXISTS "test3" ( "NO" INTEGER, `AVG. SPEED (MPH)` FLOAT ); ''' c.executescript(setup) DB = Database(conn) … | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Problems handling column names containing spaces or - 403922644 | |
614440032 | https://github.com/simonw/sqlite-utils/issues/76#issuecomment-614440032 | https://api.github.com/repos/simonw/sqlite-utils/issues/76 | MDEyOklzc3VlQ29tbWVudDYxNDQ0MDAzMg== | metab0t 10501166 | 2020-04-16T06:23:29Z | 2020-04-16T06:23:29Z | NONE | Thanks for your hard work! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | order_by mechanism 549287310 | |
571138093 | https://github.com/simonw/sqlite-utils/issues/73#issuecomment-571138093 | https://api.github.com/repos/simonw/sqlite-utils/issues/73 | MDEyOklzc3VlQ29tbWVudDU3MTEzODA5Mw== | psychemedia 82988 | 2020-01-06T13:28:31Z | 2020-01-06T13:28:31Z | NONE | I think I actually had several issues in play... The missing key was one, but I think there is also an issue as per below. For example, in the following: ```python def init_testdb(dbname='test.db'): if os.path.exists(dbname): os.remove(dbname) conn = sqlite3.connect(dbname) db = Database(conn) return conn, db conn, db = init_testdb() c = conn.cursor() c.executescript('CREATE TABLE "test1" ("Col1" TEXT, "Col2" TEXT, PRIMARY KEY ("Col1"));') c.executescript('CREATE TABLE "test2" ("Col1" TEXT, "Col2" TEXT, PRIMARY KEY ("Col1"));') print('Test 1...') for i in range(3): db['test1'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) db['test2'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) print('Test 2...') for i in range(3): db['test1'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) db['test2'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}, {'Col1':'c','Col2':'x'}], pk=('Col1')) print('Done...') --------------------------------------------------------------------------- Test 1... Test 2... IndexError: list index out of range --------------------------------------------------------------------------- IndexError Traceback (most recent call last) <ipython-input-763-444132ca189f> in <module> 22 print('Test 2...') 23 for i in range(3): ---> 24 db['test1'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) 25 db['test2'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}, 26 {'Col1':'c','Col2':'x'}], pk=('Col1')) /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in upsert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, extracts) 1157 alter=alter, 1158 extracts=extracts, -> 1… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | upsert_all() throws issue when upserting to empty table 545407916 | |
573047321 | https://github.com/simonw/sqlite-utils/issues/73#issuecomment-573047321 | https://api.github.com/repos/simonw/sqlite-utils/issues/73 | MDEyOklzc3VlQ29tbWVudDU3MzA0NzMyMQ== | psychemedia 82988 | 2020-01-10T14:02:56Z | 2020-01-10T14:09:23Z | NONE | Hmmm... just tried with installs from pip and the repo (v2.0.0 and v2.0.1) and I get the error each time (start of second run through the second loop). Could it be sqlite3? I'm on 3.30.1. UPDATE: just tried it on jupyter.org/try and I get the error there, too. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | upsert_all() throws issue when upserting to empty table 545407916 | |
580745213 | https://github.com/simonw/sqlite-utils/issues/73#issuecomment-580745213 | https://api.github.com/repos/simonw/sqlite-utils/issues/73 | MDEyOklzc3VlQ29tbWVudDU4MDc0NTIxMw== | psychemedia 82988 | 2020-01-31T14:02:38Z | 2020-01-31T14:21:09Z | NONE | So the conundrum continues.. The simple test case above now runs, but if I upsert a large number of new records (successfully) and then try to upsert a fewer number of new records to a different table, I get the same error. If I run the same upserts again (which in the first case means there are no new records to add, because they were already added), the second upsert works correctly. It feels as if the number of items added via an upsert >> the number of items I try to add in an upsert immediately after, I get the error. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | upsert_all() throws issue when upserting to empty table 545407916 | |
575799104 | https://github.com/simonw/sqlite-utils/issues/70#issuecomment-575799104 | https://api.github.com/repos/simonw/sqlite-utils/issues/70 | MDEyOklzc3VlQ29tbWVudDU3NTc5OTEwNA== | LucasElArruda 26292069 | 2020-01-17T21:20:17Z | 2020-01-17T21:20:17Z | NONE | Omg sorry I took so long to reply! On SQL we can say how the foreign key behaves when it is deleted or updated on the parent table (see https://www.sqlitetutorial.net/sqlite-foreign-key/ for more details). I did not see clearly how to create tables with this feature on sqlite-utils library. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Implement ON DELETE and ON UPDATE actions for foreign keys 539204432 | |
710768396 | https://github.com/simonw/sqlite-utils/issues/69#issuecomment-710768396 | https://api.github.com/repos/simonw/sqlite-utils/issues/69 | MDEyOklzc3VlQ29tbWVudDcxMDc2ODM5Ng== | aborruso 30607 | 2020-10-17T07:46:59Z | 2020-10-17T07:46:59Z | NONE | Great @simonw thank you very much | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Feature request: enable extensions loading 534507142 | |
1638910473 | https://github.com/simonw/sqlite-utils/issues/567#issuecomment-1638910473 | https://api.github.com/repos/simonw/sqlite-utils/issues/567 | IC_kwDOCGYnMM5hr8oJ | asg017 15178711 | 2023-07-17T21:27:41Z | 2023-07-17T21:27:41Z | NONE | Another use-case: I want to make a `sqlite-utils` plugin that'll help me insert data into Datasette. ```bash sqlite-utils insert-datasette \ --token $DATASETTE_API_KEY \ https://latest.datasette.io/fixtures/my-table \ 'select ...' ``` This could also be a datasette plugin (ex `datasette upload-data ...`, but you can also think of `sqlite-utils` plugins that upload to S3, a postgres table, other DBMS's, etc.) | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Plugin system 1801394744 | |
1606237836 | https://github.com/simonw/sqlite-utils/pull/560#issuecomment-1606237836 | https://api.github.com/repos/simonw/sqlite-utils/issues/560 | IC_kwDOCGYnMM5fvT6M | codecov[bot] 22429695 | 2023-06-25T19:49:45Z | 2023-06-26T08:20:59Z | NONE | ## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/560?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage: **`80.55`**% and project coverage change: **`-0.15`** :warning: > Comparison is base [(`2747257`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/2747257a3334d55e890b40ec58fada57ae8cfbfd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.36% compared to head [(`5e7d27e`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/560?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.22%. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #560 +/- ## ========================================== - Coverage 96.36% 96.22% -0.15% ========================================== Files 6 6 Lines 2726 2752 +26 ========================================== + Hits 2627 2648 +21 - Misses 99 104 +5 ``` | [Impacted Files](https://app.codecov.io/gh/simonw/sqlite-utils/pull/560?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/utils.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/560?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL3V0aWxzLnB5) | `94.56% <63.63%> (-0.62%)` | :arrow_down: | | [sqlite\_utils/db.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/560?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.33% <86.36%> (-0.20%)` | :arrow_down: |… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Use sqlean if available in environment 1773458985 | |
1557607516 | https://github.com/simonw/sqlite-utils/issues/554#issuecomment-1557607516 | https://api.github.com/repos/simonw/sqlite-utils/issues/554 | IC_kwDOCGYnMM5c1zRc | xavdid 1231935 | 2023-05-22T17:18:33Z | 2023-05-22T17:18:33Z | NONE | Oh and for context - this goes away if I use `.upsert` instead of `insert(..., ignore=True)`, but I don't want to update the value if it's written, just do an insert if it's new. The code is basically: ```py def save_items(table, items): db["users"].insert(build_user(items[0]), pk="id",ignore=True) db[table].insert_all(items) if comments := fetch_comments(): save_items('comments', comments) if posts := fetch_posts(): save_items('posts', posts) ``` So either `comments` or `post` could create the relevant user if those items exist. In cases where they _both_ exist, I get this error. I need the `pk` because either call could create the table. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `IndexError` when doing `.insert(..., pk='id')` after `insert_all` 1720096994 | |
1556287870 | https://github.com/simonw/sqlite-utils/pull/553#issuecomment-1556287870 | https://api.github.com/repos/simonw/sqlite-utils/issues/553 | IC_kwDOCGYnMM5cwxF- | codecov[bot] 22429695 | 2023-05-21T20:45:58Z | 2023-05-21T20:57:08Z | NONE | ## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/553?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch and project coverage have no change. > Comparison is base [(`e240133`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/e240133b11588d31dc22c632f7a7ca636c72978d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.36% compared to head [(`0b81794`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/553?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.36%. > :exclamation: Current head 0b81794 differs from pull request most recent head 21036a5. Consider uploading reports for the commit 21036a5 to get more accurate results <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #553 +/- ## ======================================= Coverage 96.36% 96.36% ======================================= Files 6 6 Lines 2726 2726 ======================================= Hits 2627 2627 Misses 99 99 ``` </details> [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/sqlite-utils/pull/553?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Reformatted CLI examples in docs 1718635018 | |
1556241555 | https://github.com/simonw/sqlite-utils/pull/549#issuecomment-1556241555 | https://api.github.com/repos/simonw/sqlite-utils/issues/549 | IC_kwDOCGYnMM5cwlyT | codecov[bot] 22429695 | 2023-05-21T17:57:24Z | 2023-05-21T18:28:44Z | NONE | ## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/549?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage: **`83.33`**% and project coverage change: **`+0.06`** :tada: > Comparison is base [(`b3b100d`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/b3b100d7f5b2a76ccd4bfe8b0301a29e321d0375?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.30% compared to head [(`948692a`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/549?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.36%. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #549 +/- ## ========================================== + Coverage 96.30% 96.36% +0.06% ========================================== Files 6 6 Lines 2707 2726 +19 ========================================== + Hits 2607 2627 +20 + Misses 100 99 -1 ``` | [Impacted Files](https://app.codecov.io/gh/simonw/sqlite-utils/pull/549?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/549?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.22% <83.33%> (-0.03%)` | :arrow_down: | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/simonw/sqlite-utils/pull/549/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) </details> [:umbrella: View fu… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | TUI powered by Trogon 1718586377 | |
1556213031 | https://github.com/simonw/sqlite-utils/pull/546#issuecomment-1556213031 | https://api.github.com/repos/simonw/sqlite-utils/issues/546 | IC_kwDOCGYnMM5cwe0n | codecov[bot] 22429695 | 2023-05-21T15:56:05Z | 2023-05-21T16:18:03Z | NONE | ## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/546?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage: **`93.75`**% and no project coverage change. > Comparison is base [(`b3b100d`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/b3b100d7f5b2a76ccd4bfe8b0301a29e321d0375?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.30% compared to head [(`9f23e68`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/546?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.31%. > :exclamation: Current head 9f23e68 differs from pull request most recent head 2eca17d. Consider uploading reports for the commit 2eca17d to get more accurate results <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #546 +/- ## ======================================= Coverage 96.30% 96.31% ======================================= Files 6 6 Lines 2707 2712 +5 ======================================= + Hits 2607 2612 +5 Misses 100 100 ``` | [Impacted Files](https://app.codecov.io/gh/simonw/sqlite-utils/pull/546?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/546?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.37% <90.90%> (+<0.01%)` | :arrow_up: | | [sqlite\_utils/cli.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/546?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Analyze tables options: --common-limit, --no-most, --no-least 1718550688 | |
1537744000 | https://github.com/simonw/sqlite-utils/issues/540#issuecomment-1537744000 | https://api.github.com/repos/simonw/sqlite-utils/issues/540 | IC_kwDOCGYnMM5bqByA | pquentin 42327 | 2023-05-08T04:56:12Z | 2023-05-08T04:56:12Z | NONE | Hey @simonw, urllib3 maintainer here :wave: Sorry for breaking your CI. I understand you may prefer to pin the Python version, but note that specifying just `python: "3"` will get you the latest. We use that in urllib3: https://github.com/urllib3/urllib3/blob/main/.readthedocs.yml I can open PRs to sqlite-utils / datasette if you're interested | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sphinx.builders.linkcheck build error 1699184583 | |
524300388 | https://github.com/simonw/sqlite-utils/issues/54#issuecomment-524300388 | https://api.github.com/repos/simonw/sqlite-utils/issues/54 | MDEyOklzc3VlQ29tbWVudDUyNDMwMDM4OA== | ftrain 20264 | 2019-08-23T12:41:09Z | 2019-08-23T12:41:09Z | NONE | Extremely cool and easy to understand. Thank you! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Ability to list views, and to access db["view_name"].rows / rows_where / etc 480961330 | |
1538975545 | https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538975545 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | IC_kwDOCGYnMM5buuc5 | xavdid 1231935 | 2023-05-08T20:06:35Z | 2023-05-08T20:06:35Z | NONE | perfect, thank you! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `table.upsert_all` fails to write rows when `not_null` is present 1695428235 | |
1506200813 | https://github.com/simonw/sqlite-utils/pull/537#issuecomment-1506200813 | https://api.github.com/repos/simonw/sqlite-utils/issues/537 | IC_kwDOCGYnMM5Zxszt | codecov[bot] 22429695 | 2023-04-13T01:45:22Z | 2023-04-13T01:45:22Z | NONE | ## [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/537?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage: **`100.00`**% and no project coverage change. > Comparison is base [(`c0251cc`)](https://codecov.io/gh/simonw/sqlite-utils/commit/c0251cc9271260de73b4227859a51fab9b4cb745?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.25% compared to head [(`a75abeb`)](https://codecov.io/gh/simonw/sqlite-utils/pull/537?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.25%. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #537 +/- ## ======================================= Coverage 96.25% 96.25% ======================================= Files 6 6 Lines 2671 2673 +2 ======================================= + Hits 2571 2573 +2 Misses 100 100 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/537?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/537?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.33% <100.00%> (+<0.01%)` | :arrow_up: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&u… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Support self-referencing FKs in `Table.create` 1665200812 | |
1500893216 | https://github.com/simonw/sqlite-utils/pull/536#issuecomment-1500893216 | https://api.github.com/repos/simonw/sqlite-utils/issues/536 | IC_kwDOCGYnMM5ZddAg | codecov[bot] 22429695 | 2023-04-08T13:35:42Z | 2023-04-08T13:35:42Z | NONE | ## [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/536?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch and project coverage have no change. > Comparison is base [(`c0251cc`)](https://codecov.io/gh/simonw/sqlite-utils/commit/c0251cc9271260de73b4227859a51fab9b4cb745?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.25% compared to head [(`cea05dc`)](https://codecov.io/gh/simonw/sqlite-utils/pull/536?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.25%. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #536 +/- ## ======================================= Coverage 96.25% 96.25% ======================================= Files 6 6 Lines 2671 2671 ======================================= Hits 2571 2571 Misses 100 100 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/536?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/utils.py](https://codecov.io/gh/simonw/sqlite-utils/pull/536?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL3V0aWxzLnB5) | `95.13% <ø> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Si… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add paths for homebrew on Apple silicon 1659525418 | |
1592617271 | https://github.com/simonw/sqlite-utils/issues/535#issuecomment-1592617271 | https://api.github.com/repos/simonw/sqlite-utils/issues/535 | IC_kwDOCGYnMM5e7Wk3 | erlend-aasland 13780613 | 2023-06-15T08:39:49Z | 2023-06-15T08:39:49Z | NONE | > piping to `jq` is good enough usually ... or `python -m json.tool`[^1], if you don't have `jq` installed. [^1]: no fancy colouring, like `jq`; only pretty-printing | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | rows: --transpose or psql extended view-like functionality 1655860104 | |
1465038901 | https://github.com/simonw/sqlite-utils/pull/531#issuecomment-1465038901 | https://api.github.com/repos/simonw/sqlite-utils/issues/531 | IC_kwDOCGYnMM5XUrg1 | codecov[bot] 22429695 | 2023-03-11T22:29:19Z | 2023-04-07T18:19:49Z | NONE | ## [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/531?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage: **`100.00`**% and project coverage change: **`+0.03`** :tada: > Comparison is base [(`c0251cc`)](https://codecov.io/gh/simonw/sqlite-utils/commit/c0251cc9271260de73b4227859a51fab9b4cb745?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.25% compared to head [(`afdf618`)](https://codecov.io/gh/simonw/sqlite-utils/pull/531?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.29%. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #531 +/- ## ========================================== + Coverage 96.25% 96.29% +0.03% ========================================== Files 6 6 Lines 2671 2671 ========================================== + Hits 2571 2572 +1 + Misses 100 99 -1 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/531?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/utils.py](https://codecov.io/gh/simonw/sqlite-utils/pull/531?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL3V0aWxzLnB5) | `95.13% <ø> (ø)` | | | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/531?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.26% <100.00%> (+0.09%)` | :arrow_up: | Help us with your feedback. Take ten second… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add paths for homebrew on Apple silicon 1620164673 | |
1419390560 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419390560 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5Umi5g | 4l1fe 21095447 | 2023-02-06T16:43:47Z | 2023-02-06T16:43:47Z | NONE | > SQLite doesn't have a native `DATETIME` type. It stores dates internally as strings and then has [functions](https://www.sqlite.org/lang_datefunc.html) to work with date-like strings. Yes it's weird. That's correct. But my issue is about the application level libraries that, i suppose, have better data understanding if see a specific type such as `DATETIME`. I'm writing data with **dataset** i've mentioned. The lib changes its behavior depending on a type. I saw different behavior with types `DATETIME, FLOAT, TEXT`. Dataset, for their part, is built upon Sqlalchemy, you know what it is. To be honest, i didn't dive into the details of why the behavior changes, but when i altered manually by other util a type of column to `DATETIME` things got back to normal. On the matter, can i achieve it with Sqlite Utils at the moment? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1419734229 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419734229 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5Un2zV | cldellow 193185 | 2023-02-06T20:53:28Z | 2023-02-06T21:16:29Z | NONE | I think it's not currently possible: sqlite-utils requires that it be one of `integer`, `text`, `float`, `blob` ([see code](https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/cli.py#L2266)) IMO, this is a bit of friction and it would be nice if it was more permissive. SQLite permits developers to use any data type when creating a table. For example, this is a perfectly cromulent sqlite session that creates a table with columns of type `baz` and `bar`: ``` sqlite> create table foo(column1 baz, column2 bar); sqlite> .schema foo CREATE TABLE foo(column1 baz, column2 bar); sqlite> select * from pragma_table_info('foo'); cid name type notnull dflt_value pk ---------- ---------- ---------- ---------- ---------- ---------- 0 column1 baz 0 0 1 column2 bar 0 0 ``` The idea is that the application developer will know what meaning to ascribe to those types. For example, I'm working on a plugin to Datasette. Dates are tricky to handle. If you have some existing rows, you can look at the values in them to know how a user is serializing the dates -- as an ISO 8601 string? An RFC 3339 string? With millisecond precision? With timezone offset? But if you don't yet have any rows, you have to guess. If the column is of type `TEXT`, you don't even know that it's meant to hold a date! In this case, my plugin will look to see if the column is of type `DATE` or `DATETIME`, and assume a certain representation when writing. Perhaps there is an argument that sqlite-utils is trying to conform to SQLite's strict mode, and that is why it limits the choices. In strict mode, SQLite requires that the data type be one of `INT`, `INTEGER`, `REAL`, `TEXT`, `BLOB`, `ANY`. But that can't be the case -- sqlite-utils supports `FLOAT`, which is not one of the valid types in strict mode, and it rejects `INT`, `REAL` and `ANY`, which _are_ valid. | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1419740776 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419740776 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5Un4Zo | cldellow 193185 | 2023-02-06T20:59:01Z | 2023-02-06T20:59:01Z | NONE | That said, it looks like the check is only enforced at the CLI level. If you use the API directly, I think it'll work. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1420496447 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1420496447 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5Uqw4_ | 4l1fe 21095447 | 2023-02-07T09:57:38Z | 2023-02-07T09:57:38Z | NONE | > That said, it looks like the check is only enforced at the CLI level. If you use the API directly, I think it'll work. It works, but a column becomes `TEXT` ```python In [1]: import sqlite_utils In [2]: db = sqlite_utils.Database('events.sqlite') In [3]: table = db['cards.chunk.get'] In [4]: table.columns_dict Out[4]: {'id': int, 'timestamp': float, 'data_chunk_number': int, 'user_id': str, 'meta_duplication_source_id': int, 'context_sort_attribute': str, 'context_sort_order': str} In [5]: from datetime import datetime In [7]: table.transform(types={'timestamp': datetime}) In [8]: table.columns_dict Out[8]: {'id': int, 'timestamp': str, 'data_chunk_number': int, 'user_id': str, 'meta_duplication_source_id': int, 'context_sort_attribute': str, 'context_sort_order': str} ``` ```bash ❯ sqlite-utils schema events.sqlite cards.chunk.get CREATE TABLE "cards.chunk.get" ( [id] INTEGER PRIMARY KEY NOT NULL, [timestamp] TEXT, ... ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1420809773 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1420809773 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5Ur9Yt | cldellow 193185 | 2023-02-07T13:53:01Z | 2023-02-07T13:53:01Z | NONE | Ah, it looks like that is controlled by this dict: https://github.com/simonw/sqlite-utils/blob/main/sqlite_utils/db.py#L178 I suspect you could overwrite the datetime entry to achieve what you want | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1420966995 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1420966995 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5UsjxT | 4l1fe 21095447 | 2023-02-07T15:29:28Z | 2023-02-07T15:29:28Z | NONE | I could, of course. Doest it worth bringing such the improvement to the library? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1420992261 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1420992261 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5Usp8F | cldellow 193185 | 2023-02-07T15:45:58Z | 2023-02-07T15:45:58Z | NONE | I'd support that, but I'm not the author of this library. One challenge is that would be a breaking change. Do you see a way to enable it without affecting existing users or bumping the major version number? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1421022917 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421022917 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5UsxbF | 4l1fe 21095447 | 2023-02-07T16:06:03Z | 2023-02-07T16:08:58Z | NONE | > Do you see a way to enable it without affecting existing users or bumping the major version number? I don't see a clean solution, only extending code with a side variable that tells us we want to apply advanced types instead of basic. it could be a similiar command like `tranform-v2 --type column DATETIME` or a cli option `transform --adv-type column DATETIME` along with a dict that contains the advanced types. Then with knowledge that we run an advanced command we take that dictionary somehow, we can wrap the current and new dictionaries by a superdict and work with it everywhere according to the knowledge. This way shouldn't affect users who are using the previous lib versions and it have to be merged in the next major one. But this way looks a bad design, too messy. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1421033725 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421033725 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5Us0D9 | cldellow 193185 | 2023-02-07T16:12:13Z | 2023-02-07T16:12:13Z | NONE | I think the bigger issue is that `sqlite-utils` mixes mechanism (it implements the [12-step way to alter SQLite tables](https://www.sqlite.org/lang_altertable.html#otheralter)) and policy (it has an opinionated stance on what column types should be used). That might be a design choice to make it accessible to users by providing a reasonable set of defaults, but it doesn't quite fit my use case. It might make sense to extract a separate library that provides just the mechanisms, and then `sqlite-utils` would sit on top of that library with its opinionated set of policies. That would be a very big change, though. I might take a stab at extracting the library, but just for the table schema migration piece, not all the other features that `sqlite-utils` supports. I wouldn't expect `sqlite-utils` to depend on it. Part of my motivation is that I want to provide some other abilities, too, like support for CHECK constraints. I see that the issue in this repo (https://github.com/simonw/sqlite-utils/issues/358) proposes a bunch of short-hand constraints, which I wouldn't want to accidentally expose to people -- I want a layer that is a 1:1 mapping to SQLite. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1421052195 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421052195 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5Us4kj | 4l1fe 21095447 | 2023-02-07T16:23:17Z | 2023-02-07T16:23:57Z | NONE | Isn't your suggestion too fundamental for the utility? The bigger flexibility, the bigger complexity. Your idea make sense defenitely, but how often do you make schema changes? And how many people could benefit from it, what do you think? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1421055590 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421055590 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5Us5Zm | 4l1fe 21095447 | 2023-02-07T16:25:31Z | 2023-02-07T16:25:31Z | NONE | > Ah, it looks like that is controlled by this dict: https://github.com/simonw/sqlite-utils/blob/main/sqlite_utils/db.py#L178 > > I suspect you could overwrite the datetime entry to achieve what you want And thank you for pointing me to it. At least, i can make a monkey patch for my need... | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1421081939 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421081939 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5Us_1T | cldellow 193185 | 2023-02-07T16:42:25Z | 2023-02-07T16:43:42Z | NONE | Ha, yes, I might end up making something very niche. That's OK. I'm building a UI for [Datasette](https://datasette.io/) that lets users make schema changes, so it's important to me that the tool work in a non-surprising way -- if you ask for a column of type X, you should get type X. If the column or table previously had CHECK constraints, they shouldn't be silently removed. And so on. I had hoped that I could just lean on sqlite-utils, but I think it's a little too surprising. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1421177666 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421177666 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5UtXNC | 4l1fe 21095447 | 2023-02-07T17:39:00Z | 2023-02-07T17:39:00Z | NONE | > lets users make schema changes, so it's important to me that the tool work in a non-surprising way -- if you ask for a column of type X, you should get type X. If the column or table previously had CHECK constraints, they shouldn't be silently removed I've got your concern. Let's see if we will be replied on it and i'll close the issue some later. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1422681850 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1422681850 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5UzGb6 | 4l1fe 21095447 | 2023-02-08T14:25:50Z | 2023-02-08T14:29:09Z | NONE | I live the patch here for others: _original code_ ```shell $ which sqlite-utils | xargs cat ``` ```python #!/usr/bin/python3 # -*- coding: utf-8 -*- import re import sys from sqlite_utils.cli import cli if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) sys.exit(cli()) ``` _patched/sqlite-utils.py_ ```python #!/usr/bin/python3 # -*- coding: utf-8 -*- import re import sys from sqlite_utils.cli import cli # New imports from unittest.mock import patch from sqlite_utils.cli import VALID_COLUMN_TYPES if __name__ == '__main__': # Choices of the option `--type` cli.commands['transform'].params[2].type.types[1].choices.append('DATETIME') # The dicts has to be extended with a new type with patch.dict('sqlite_utils.db.COLUMN_TYPE_MAPPING', {'DATETIME': 'DATETIME'}),\ patch('sqlite_utils.cli.VALID_COLUMN_TYPES', VALID_COLUMN_TYPES + ("DATETIME", )): # Command is unchanged sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) sys.exit(cli()) ``` And now it's working ```bash $ sqlite-utils schema events.sqlite cards.chunk.get CREATE TABLE "cards.chunk.get" ( [id] INTEGER PRIMARY KEY NOT NULL, [timestamp] TEXT, ) $ python patched/sqlite-utils.py transform events.sqlite cards.chunk.get --type timestamp DATETIME $ sqlite-utils schema events.sqlite cards.chunk.get CREATE TABLE "cards.chunk.get" ( [id] INTEGER PRIMARY KEY NOT NULL, [timestamp] DATETIME, ) ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Transformation type `--type DATETIME` 1572766460 | |
1339837520 | https://github.com/simonw/sqlite-utils/issues/516#issuecomment-1339837520 | https://api.github.com/repos/simonw/sqlite-utils/issues/516 | IC_kwDOCGYnMM5P3ExQ | briandorsey 122043 | 2022-12-06T19:02:30Z | 2022-12-06T19:02:30Z | NONE | `--verbose` or `--verbosity=ABC` were the flags I looked for. Expected to see them at a global level near `--version`. But only sharing because that's where I looked first, I don't have a strong opinion on the exact wording/location. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Feature request: output number of ignored/replaced rows for insert command 1479914599 | |
1339839767 | https://github.com/simonw/sqlite-utils/issues/516#issuecomment-1339839767 | https://api.github.com/repos/simonw/sqlite-utils/issues/516 | IC_kwDOCGYnMM5P3FUX | briandorsey 122043 | 2022-12-06T19:04:17Z | 2022-12-06T19:04:17Z | NONE | Current behavior is different when importing via stdin vs. a file. Imports from a file give a progress bar. For this new request, I'd love to see total imported and total ignored/replaced in both cases. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Feature request: output number of ignored/replaced rows for insert command 1479914599 | |
1339844639 | https://github.com/simonw/sqlite-utils/issues/516#issuecomment-1339844639 | https://api.github.com/repos/simonw/sqlite-utils/issues/516 | IC_kwDOCGYnMM5P3Ggf | briandorsey 122043 | 2022-12-06T19:08:13Z | 2022-12-06T19:08:13Z | NONE | Reference: tqdm (https://tqdm.github.io/) shows a progress bar when total is known, and falls back to counting units of work done for streams. File input vs. stdin seems like a similar situation. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Feature request: output number of ignored/replaced rows for insert command 1479914599 | |
1318431389 | https://github.com/simonw/sqlite-utils/issues/510#issuecomment-1318431389 | https://api.github.com/repos/simonw/sqlite-utils/issues/510 | IC_kwDOCGYnMM5Olaqd | ar-jan 1176293 | 2022-11-17T10:36:28Z | 2022-11-17T10:36:28Z | NONE | The virtual table's _config `version: 4` seems to indicate FTS5. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Cannot enable FTS5 despite it being available 1434911255 | |
1320394127 | https://github.com/simonw/sqlite-utils/issues/510#issuecomment-1320394127 | https://api.github.com/repos/simonw/sqlite-utils/issues/510 | IC_kwDOCGYnMM5Os52P | ar-jan 1176293 | 2022-11-18T18:37:51Z | 2022-11-18T18:37:51Z | NONE | I guess it is not incorrect when it says the version is `4`, though it is confusing. Maybe it doesn't even refer to FTS4/FTS5 versions, but something else? In any case, it's not related to sqlite-utils, but SQLite itself. | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Cannot enable FTS5 despite it being available 1434911255 | |
1297754631 | https://github.com/simonw/sqlite-utils/pull/508#issuecomment-1297754631 | https://api.github.com/repos/simonw/sqlite-utils/issues/508 | IC_kwDOCGYnMM5NWioH | codecov[bot] 22429695 | 2022-10-31T22:14:48Z | 2022-10-31T22:53:59Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/508?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **96.25**% // Head: **96.09**% // Decreases project coverage by **`-0.15%`** :warning: > Coverage data is based on head [(`2d6a149`)](https://codecov.io/gh/simonw/sqlite-utils/pull/508?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`529110e`)](https://codecov.io/gh/simonw/sqlite-utils/commit/529110e7d8c4a6b1bbf5fb61f2e29d72aa95a611?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch coverage: 63.63% of modified lines in pull request are covered. > :exclamation: Current head 2d6a149 differs from pull request most recent head 43a8c4c. Consider uploading reports for the commit 43a8c4c to get more accurate results <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #508 +/- ## ========================================== - Coverage 96.25% 96.09% -0.16% ========================================== Files 4 4 Lines 2401 2407 +6 ========================================== + Hits 2311 2313 +2 - Misses 90 94 +4 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/508?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/508/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `96.79% <63.63%> (-0.30%)` | :arrow_down: | Help us with your feedback. Take ten seconds to… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Allow surrogates in parameters 1430563092 | |
1275248761 | https://github.com/simonw/sqlite-utils/pull/499#issuecomment-1275248761 | https://api.github.com/repos/simonw/sqlite-utils/issues/499 | IC_kwDOCGYnMM5MAsB5 | codecov[bot] 22429695 | 2022-10-11T20:38:13Z | 2022-10-11T20:38:13Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/499?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **96.37**% // Head: **96.25**% // Decreases project coverage by **`-0.11%`** :warning: > Coverage data is based on head [(`a25e838`)](https://codecov.io/gh/simonw/sqlite-utils/pull/499?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`d792dad`)](https://codecov.io/gh/simonw/sqlite-utils/commit/d792dad1cf5f16525da81b1e162fb71d469995f3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch coverage: 40.00% of modified lines in pull request are covered. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #499 +/- ## ========================================== - Coverage 96.37% 96.25% -0.12% ========================================== Files 4 4 Lines 2401 2406 +5 ========================================== + Hits 2314 2316 +2 - Misses 87 90 +3 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/499?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/499/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.09% <40.00%> (-0.23%)` | :arrow_down: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | feat: recreate fts triggers after table transform 1405196044 | |
1274097701 | https://github.com/simonw/sqlite-utils/pull/498#issuecomment-1274097701 | https://api.github.com/repos/simonw/sqlite-utils/issues/498 | IC_kwDOCGYnMM5L8TAl | codecov[bot] 22429695 | 2022-10-11T05:13:37Z | 2022-10-11T05:13:37Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/498?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **96.37**% // Head: **96.37**% // Increases project coverage by **`+0.00%`** :tada: > Coverage data is based on head [(`75577e5`)](https://codecov.io/gh/simonw/sqlite-utils/pull/498?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`d792dad`)](https://codecov.io/gh/simonw/sqlite-utils/commit/d792dad1cf5f16525da81b1e162fb71d469995f3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch coverage: 100.00% of modified lines in pull request are covered. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #498 +/- ## ======================================= Coverage 96.37% 96.37% ======================================= Files 4 4 Lines 2401 2402 +1 ======================================= + Hits 2314 2315 +1 Misses 87 87 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/498?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/498/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.32% <100.00%> (+<0.01%)` | :arrow_up: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | fix: enable-fts permanently save triggers 1404013495 | |
1294408928 | https://github.com/simonw/sqlite-utils/issues/496#issuecomment-1294408928 | https://api.github.com/repos/simonw/sqlite-utils/issues/496 | IC_kwDOCGYnMM5NJxzg | justmars 39538958 | 2022-10-28T03:36:56Z | 2022-10-28T03:37:50Z | NONE | With respect to the typing of Table class itself, my interim solution: ```python from sqlite_utils.db import Table def tbl(self, table_name: str) -> Table: tbl = self.db[table_name] if isinstance(tbl, Table): return tbl raise Exception(f"Missing {table_name=}") ``` With respect to @chapmanjacobd concern on the `DEFAULT` being an empty class, have also been using `# type: ignore`, e.g. ```python @classmethod def insert_list(cls, areas: list[str]): return meta.tbl(meta.Areas).insert_all( ({"area": a} for a in areas), ignore=True # type: ignore ) ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | devrel/python api: Pylance type hinting 1393202060 | |
1532481862 | https://github.com/simonw/sqlite-utils/issues/496#issuecomment-1532481862 | https://api.github.com/repos/simonw/sqlite-utils/issues/496 | IC_kwDOCGYnMM5bV9FG | xavdid 1231935 | 2023-05-03T05:53:26Z | 2023-05-03T05:53:26Z | NONE | Would love to put our heads together for improvements here. I _think_ anything that is `argname=DEFAULT` needs to be typed as `argname: str | Default = DEFAULT` (replacing `str` with the appropriate type(s)). We may be able to get clever and tie the types to that key in the `_defaults` dict (definitely possible in Typescript, but I'm less familiar with advanced python types). Right now, all args are typed as `Default`, which means all callers get type errors. As for table/view, given that they don't have the same methods, it would be nice to be able to get one or the other specifically. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | devrel/python api: Pylance type hinting 1393202060 | |
1257063174 | https://github.com/simonw/sqlite-utils/issues/490#issuecomment-1257063174 | https://api.github.com/repos/simonw/sqlite-utils/issues/490 | IC_kwDOCGYnMM5K7UMG | jeqo 6180701 | 2022-09-24T20:50:15Z | 2022-09-24T20:50:15Z | NONE | 🤯 this is beautiful. Thanks @simonw ! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Ability to insert multi-line files 1382457780 | |
1257072258 | https://github.com/simonw/sqlite-utils/issues/490#issuecomment-1257072258 | https://api.github.com/repos/simonw/sqlite-utils/issues/490 | IC_kwDOCGYnMM5K7WaC | jeqo 6180701 | 2022-09-24T22:01:05Z | 2022-09-24T22:01:05Z | NONE | For completeness, the regex requires a bit more dark magic to capture the following lines, here is a _working_ expression: https://regex101.com/r/rsuEcs/1 ``` sqlite-utils insert /tmp/log.db log multiline.log --text --convert " import re r = re.compile(r'^(?P<datetime>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})(?:\:\s)(?P<log>(.*\s\s.*|.*)+)', re.MULTILINE) def convert(text): return [m.groupdict() for m in r.finditer(text)] " ``` ``` BEGIN TRANSACTION; CREATE TABLE [log] ( [datetime] TEXT, [log] TEXT ); INSERT INTO "log" VALUES('2022-03-01T12:04:52','Here is a log message that spans multiple lines'); INSERT INTO "log" VALUES('2022-03-01T12:04:52','This is a single line'); INSERT INTO "log" VALUES('2022-03-01T12:04:52','Here is another message that spans multiple lines'); COMMIT; ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Ability to insert multi-line files 1382457780 | |
1364141224 | https://github.com/simonw/sqlite-utils/issues/488#issuecomment-1364141224 | https://api.github.com/repos/simonw/sqlite-utils/issues/488 | IC_kwDOCGYnMM5RTySo | ar-jan 1176293 | 2022-12-23T17:38:55Z | 2022-12-23T17:38:55Z | NONE | > text columns containing empty strings should not be rewritten to null. I would actually appreciate an option to do just that for text columns as well. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `sqlite-utils transform` should set empty strings to null when converting text columns to integer/float 1373224657 | |
1242513223 | https://github.com/simonw/sqlite-utils/issues/487#issuecomment-1242513223 | https://api.github.com/repos/simonw/sqlite-utils/issues/487 | IC_kwDOCGYnMM5KDz9H | ryanfox 540968 | 2022-09-09T22:01:10Z | 2022-09-09T22:01:10Z | NONE | Perfect. Apologies, I searched but didn’t see that one for some reason. Thank you. On Fri, Sep 9, 2022 at 3:04 PM Simon Willison ***@***.***> wrote: > This isn't supported yet - there's an older issue for that here: > > - #117 <https://github.com/simonw/sqlite-utils/issues/117> > > — > Reply to this email directly, view it on GitHub > <https://github.com/simonw/sqlite-utils/issues/487#issuecomment-1242409766>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAEECKBODHZCKWTN5OO6RI3V5OJ5RANCNFSM6AAAAAAQIV3YIU> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> > | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Specify foreign key against compound key in other table 1367835380 | |
1248527646 | https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248527646 | https://api.github.com/repos/simonw/sqlite-utils/issues/486 | IC_kwDOCGYnMM5KawUe | codecov[bot] 22429695 | 2022-09-15T19:34:59Z | 2022-09-15T20:23:12Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/486?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **96.47**% // Head: **96.52**% // Increases project coverage by **`+0.04%`** :tada: > Coverage data is based on head [(`0acbc68`)](https://codecov.io/gh/simonw/sqlite-utils/pull/486?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`d9b9e07`)](https://codecov.io/gh/simonw/sqlite-utils/commit/d9b9e075f07a20f1137cd2e34ed5d3f1a3db4ad8?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch coverage: 100.00% of modified lines in pull request are covered. > :exclamation: Current head 0acbc68 differs from pull request most recent head d5db749. Consider uploading reports for the commit d5db749 to get more accurate results <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #486 +/- ## ========================================== + Coverage 96.47% 96.52% +0.04% ========================================== Files 6 6 Lines 2642 2646 +4 ========================================== + Hits 2549 2554 +5 + Misses 93 92 -1 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/486?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/486/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.86% <100.00%> (ø)` | | | [sqlite\_utils/utils.py](https://codecov.io/gh/simonw/sqlite… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | progressbar for inserts/upserts of all fileformats, closes #485 1366512990 | |
1232091957 | https://github.com/simonw/sqlite-utils/pull/480#issuecomment-1232091957 | https://api.github.com/repos/simonw/sqlite-utils/issues/480 | IC_kwDOCGYnMM5JcDs1 | codecov[bot] 22429695 | 2022-08-30T19:45:23Z | 2022-08-31T03:34:37Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/480?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **96.47**% // Head: **96.47**% // Increases project coverage by **`+0.00%`** :tada: > Coverage data is based on head [(`3dbfca6`)](https://codecov.io/gh/simonw/sqlite-utils/pull/480?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`087753c`)](https://codecov.io/gh/simonw/sqlite-utils/commit/087753cd42c406f1e060c1822dcd9b5fda3d60f4?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch coverage: 100.00% of modified lines in pull request are covered. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #480 +/- ## ======================================= Coverage 96.47% 96.47% ======================================= Files 6 6 Lines 2640 2642 +2 ======================================= + Hits 2547 2549 +2 Misses 93 93 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/480?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/480/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.32% <100.00%> (+<0.01%)` | :arrow_up: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | search_sql add include_rank option 1355433619 | |
1238815924 | https://github.com/simonw/sqlite-utils/issues/477#issuecomment-1238815924 | https://api.github.com/repos/simonw/sqlite-utils/issues/477 | IC_kwDOCGYnMM5J1tS0 | thewchan 49702524 | 2022-09-07T01:46:24Z | 2022-09-07T01:46:24Z | NONE | Will do! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Conda Forge 1353441389 | |
1229449018 | https://github.com/simonw/sqlite-utils/issues/474#issuecomment-1229449018 | https://api.github.com/repos/simonw/sqlite-utils/issues/474 | IC_kwDOCGYnMM5JR-c6 | hubgit 14294 | 2022-08-28T12:40:13Z | 2022-08-28T12:40:13Z | NONE | Creating the table before inserting is a useful workaround, thanks. It does require figuring out the `create table` syntax and listing all the fields manually, though, which loses some of the magic of sqlite-utils. I was expecting to find an option like `--headers=foo,bar` (or `--header-row='foo\tbar'`, if that would be easier) - not necessarily that exact syntax, but something that would essentially be treated the same as having a header row in the file. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add an option for specifying column names when inserting CSV data 1353074021 | |
1229130453 | https://github.com/simonw/sqlite-utils/pull/473#issuecomment-1229130453 | https://api.github.com/repos/simonw/sqlite-utils/issues/473 | IC_kwDOCGYnMM5JQwrV | codecov[bot] 22429695 | 2022-08-27T05:55:12Z | 2022-08-27T05:55:18Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/473?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#473](https://codecov.io/gh/simonw/sqlite-utils/pull/473?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (72db599) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/a46a5e3a9e03dcdd8c84a92e4a5dbfa02ba461fa?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (a46a5e3) will **decrease** coverage by `0.07%`. > The diff coverage is `50.00%`. ```diff @@ Coverage Diff @@ ## main #473 +/- ## ========================================== - Coverage 96.67% 96.60% -0.08% ========================================== Files 6 6 Lines 2587 2590 +3 ========================================== + Hits 2501 2502 +1 - Misses 86 88 +2 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/473?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/473/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.94% <50.00%> (-0.18%)` | :arrow_down: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Support entrypoints for `--load-extension` 1352953535 | |
1229430228 | https://github.com/simonw/sqlite-utils/issues/471#issuecomment-1229430228 | https://api.github.com/repos/simonw/sqlite-utils/issues/471 | IC_kwDOCGYnMM5JR53U | stonebig 4312421 | 2022-08-28T10:43:35Z | 2022-08-28T10:43:35Z | NONE | Is it still nfortunately slow and tricky when playing with floats ? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils query --functions mechanism for registering extra functions 1352932716 | |
1224412060 | https://github.com/simonw/sqlite-utils/pull/468#issuecomment-1224412060 | https://api.github.com/repos/simonw/sqlite-utils/issues/468 | IC_kwDOCGYnMM5I-wuc | codecov[bot] 22429695 | 2022-08-23T17:30:24Z | 2022-08-27T23:11:44Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/468?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#468](https://codecov.io/gh/simonw/sqlite-utils/pull/468?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (fc38480) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/c5f8a2eb1a81a18b52825cc649112f71fe419b12?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (c5f8a2e) will **decrease** coverage by `0.12%`. > The diff coverage is `88.88%`. > :exclamation: Current head fc38480 differs from pull request most recent head 2f6a64f. Consider uploading reports for the commit 2f6a64f to get more accurate results ```diff @@ Coverage Diff @@ ## main #468 +/- ## ========================================== - Coverage 96.60% 96.47% -0.13% ========================================== Files 6 6 Lines 2590 2640 +50 ========================================== + Hits 2502 2547 +45 - Misses 88 93 +5 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/468?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/468/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.86% <66.66%> (-0.09%)` | :arrow_down: | | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/468/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) |… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | db[table].create(..., transform=True) and create-table --transform 1348294436 | |
1218565147 | https://github.com/simonw/sqlite-utils/pull/465#issuecomment-1218565147 | https://api.github.com/repos/simonw/sqlite-utils/issues/465 | IC_kwDOCGYnMM5IodQb | codecov[bot] 22429695 | 2022-08-17T22:44:40Z | 2022-08-17T23:21:22Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/465?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#465](https://codecov.io/gh/simonw/sqlite-utils/pull/465?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (bb90d33) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/45e24deffea042b5db7ab84cd1eb63b3ed9bb9da?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (45e24de) will **not change** coverage. > The diff coverage is `n/a`. ```diff @@ Coverage Diff @@ ## main #465 +/- ## ======================================= Coverage 96.64% 96.64% ======================================= Files 6 6 Lines 2566 2566 ======================================= Hits 2480 2480 Misses 86 86 ``` Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | beanbag-docutils>=2.0 1342357149 | |
1203207940 | https://github.com/simonw/sqlite-utils/pull/462#issuecomment-1203207940 | https://api.github.com/repos/simonw/sqlite-utils/issues/462 | IC_kwDOCGYnMM5Ht38E | codecov[bot] 22429695 | 2022-08-02T20:58:38Z | 2022-08-02T21:05:58Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/462?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#462](https://codecov.io/gh/simonw/sqlite-utils/pull/462?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (0a14a03) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/1856002e3c0fcc9f09f72ab7d97ad8c75f6de7df?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (1856002) will **not change** coverage. > The diff coverage is `n/a`. ```diff @@ Coverage Diff @@ ## main #462 +/- ## ======================================= Coverage 96.64% 96.64% ======================================= Files 6 6 Lines 2566 2566 ======================================= Hits 2480 2480 Misses 86 86 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/462?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/462?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [1856002...0a14a03](https://codecov.io/gh/simonw/sqlite-utils/pull/462?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Discord badge 1326391841 | |
1202947160 | https://github.com/simonw/sqlite-utils/pull/460#issuecomment-1202947160 | https://api.github.com/repos/simonw/sqlite-utils/issues/460 | IC_kwDOCGYnMM5Hs4RY | codecov[bot] 22429695 | 2022-08-02T16:20:42Z | 2022-08-02T20:53:19Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/460?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#460](https://codecov.io/gh/simonw/sqlite-utils/pull/460?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (f21f4cc) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/1856002e3c0fcc9f09f72ab7d97ad8c75f6de7df?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (1856002) will **not change** coverage. > The diff coverage is `n/a`. ```diff @@ Coverage Diff @@ ## main #460 +/- ## ======================================= Coverage 96.64% 96.64% ======================================= Files 6 6 Lines 2566 2566 ======================================= Hits 2480 2480 Misses 86 86 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/460?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/460?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [1856002...f21f4cc](https://codecov.io/gh/simonw/sqlite-utils/pull/460?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Cross-link CLI to Python docs 1326087800 | |
592999503 | https://github.com/simonw/sqlite-utils/issues/46#issuecomment-592999503 | https://api.github.com/repos/simonw/sqlite-utils/issues/46 | MDEyOklzc3VlQ29tbWVudDU5Mjk5OTUwMw== | chrishas35 35075 | 2020-02-29T22:08:20Z | 2020-02-29T22:08:20Z | NONE | @simonw any thoughts on allow extracts to specify the lookup column name? If I'm understanding the documentation right, `.lookup()` allows you to define the "value" column (the documentation uses name), but when you use `extracts` keyword as part of `.insert()`, `.upsert()` etc. the lookup must be done against a column named "value". I have an existing lookup table that I've populated with columns "id" and "name" as opposed to "id" and "value", and seems I can't use `extracts=`, unless I'm missing something... Initial thought on how to do this would be to allow the dictionary value to be a tuple of table name column pair... so: ``` table = db.table("trees", extracts={"species_id": ("Species", "name"}) ``` I haven't dug too much into the existing code yet, but does this make sense? Worth doing? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | extracts= option for insert/update/etc 471780443 | |
1197090890 | https://github.com/simonw/sqlite-utils/pull/457#issuecomment-1197090890 | https://api.github.com/repos/simonw/sqlite-utils/issues/457 | IC_kwDOCGYnMM5HWihK | codecov[bot] 22429695 | 2022-07-27T17:40:14Z | 2022-07-27T17:40:40Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/457?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#457](https://codecov.io/gh/simonw/sqlite-utils/pull/457?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (80b3907) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/9e6cceac1c0e086429e2d308b700e59cc53a1991?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (9e6ccea) will **not change** coverage. > The diff coverage is `n/a`. ```diff @@ Coverage Diff @@ ## main #457 +/- ## ======================================= Coverage 96.64% 96.64% ======================================= Files 6 6 Lines 2566 2566 ======================================= Hits 2480 2480 Misses 86 86 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/457?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/457?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [9e6ccea...80b3907](https://codecov.io/gh/simonw/sqlite-utils/pull/457?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Link to installation instructions 1319881016 | |
1190449764 | https://github.com/simonw/sqlite-utils/issues/456#issuecomment-1190449764 | https://api.github.com/repos/simonw/sqlite-utils/issues/456 | IC_kwDOCGYnMM5G9NJk | jcmkk3 45919695 | 2022-07-20T15:45:54Z | 2022-07-20T15:45:54Z | NONE | > hadley wickham's melt and reshape could be good inspo: http://had.co.nz/reshape/introduction.pdf Note that Hadley has since implemented `pivot_longer` and `pivot_wider` instead of the previous verbs/functions that he used. Those can be found in the tidyr package and are probably the best reference which includes all of the learnings from years of user feedback. https://tidyr.tidyverse.org/articles/pivot.html | {"total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | feature request: pivot command 1310243385 | |
1189130490 | https://github.com/simonw/sqlite-utils/pull/455#issuecomment-1189130490 | https://api.github.com/repos/simonw/sqlite-utils/issues/455 | IC_kwDOCGYnMM5G4LD6 | codecov[bot] 22429695 | 2022-07-19T14:32:10Z | 2022-07-19T14:32:10Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/455?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#455](https://codecov.io/gh/simonw/sqlite-utils/pull/455?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (1b35a92) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/855bce8c3823718def13e0b8928c58bf857e41b2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (855bce8) will **not change** coverage. > The diff coverage is `n/a`. ```diff @@ Coverage Diff @@ ## main #455 +/- ## ======================================= Coverage 96.64% 96.64% ======================================= Files 6 6 Lines 2566 2566 ======================================= Hits 2480 2480 Misses 86 86 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/455?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/455/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.54% <ø> (ø)` | | ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/455?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not a… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | in extract code, check equality with IS instead of = for nulls 1309542173 | |
1185949850 | https://github.com/simonw/sqlite-utils/pull/452#issuecomment-1185949850 | https://api.github.com/repos/simonw/sqlite-utils/issues/452 | IC_kwDOCGYnMM5GsCia | codecov[bot] 22429695 | 2022-07-15T21:15:45Z | 2022-07-15T21:18:34Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/452?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#452](https://codecov.io/gh/simonw/sqlite-utils/pull/452?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (eef350f) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/42440d6345c242ee39778045e29143fb550bd2c2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (42440d6) will **increase** coverage by `0.00%`. > The diff coverage is `100.00%`. ```diff @@ Coverage Diff @@ ## main #452 +/- ## ======================================= Coverage 96.60% 96.61% ======================================= Files 6 6 Lines 2534 2540 +6 ======================================= + Hits 2448 2454 +6 Misses 86 86 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/452?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/452/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.53% <100.00%> (+0.01%)` | :arrow_up: | ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/452?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolu… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add duplicate table feature 1299760627 | |
1162498734 | https://github.com/simonw/sqlite-utils/issues/448#issuecomment-1162498734 | https://api.github.com/repos/simonw/sqlite-utils/issues/448 | IC_kwDOCGYnMM5FSlKu | mungewell 236907 | 2022-06-22T00:43:45Z | 2022-06-22T00:43:45Z | NONE | Attempted to test on a machine with a new version of Python, but install failed with an error message for the 'click' package. ``` C:\WINDOWS\system32>"c:\Program Files\Python310\python.exe" Python 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> quit() C:\WINDOWS\system32>cd C:\Users\swood\Downloads\sqlite-utils-main-20220621\sqlite-utils-main C:\Users\swood\Downloads\sqlite-utils-main-20220621\sqlite-utils-main>"c:\Program Files\Python310\python.exe" setup.py install running install running bdist_egg running egg_info ... Installed c:\program files\python310\lib\site-packages\click_default_group_wheel-1.2.2-py3.10.egg Searching for click Downloading https://files.pythonhosted.org/packages/3d/da/f3bbf30f7e71d881585d598f67f4424b2cc4c68f39849542e81183218017/click-default-group-wheel-1.2.2.tar.gz#sha256=e90da42d92c03e88a12ed0c0b69c8a29afb5d36e3dc8d29c423ba4219e6d7747 Best match: click default-group-wheel-1.2.2 Processing click-default-group-wheel-1.2.2.tar.gz Writing C:\Users\swood\AppData\Local\Temp\easy_install-aiaj0_eh\click-default-group-wheel-1.2.2\setup.cfg Running click-default-group-wheel-1.2.2\setup.py -q bdist_egg --dist-dir C:\Users\swood\AppData\Local\Temp\easy_install-aiaj0_eh\click-default-group-wheel-1.2.2\egg-dist-tmp-z61a4h8n zip_safe flag not set; analyzing archive contents... removing 'c:\program files\python310\lib\site-packages\click_default_group_wheel-1.2.2-py3.10.egg' (and everything under it) Copying click_default_group_wheel-1.2.2-py3.10.egg to c:\program files\python310\lib\site-packages click-default-group-wheel 1.2.2 is already the active version in easy-install.pth Installed c:\program files\python310\lib\site-packages\click_default_group_wheel-1.2.2-py3.10.egg error: The 'click' distribution was not found and is required by click-default-group-wheel, sqlite-utils ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Reading rows from a file => AttributeError: '_io.StringIO' object has no attribute 'readinto' 1279144769 | |
1162500525 | https://github.com/simonw/sqlite-utils/issues/448#issuecomment-1162500525 | https://api.github.com/repos/simonw/sqlite-utils/issues/448 | IC_kwDOCGYnMM5FSlmt | mungewell 236907 | 2022-06-22T00:46:43Z | 2022-06-22T00:46:43Z | NONE | [log.txt](https://github.com/simonw/sqlite-utils/files/8953589/log.txt) | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Reading rows from a file => AttributeError: '_io.StringIO' object has no attribute 'readinto' 1279144769 | |
1155515426 | https://github.com/simonw/sqlite-utils/issues/441#issuecomment-1155515426 | https://api.github.com/repos/simonw/sqlite-utils/issues/441 | IC_kwDOCGYnMM5E38Qi | betatim 1448859 | 2022-06-14T17:53:43Z | 2022-06-14T17:53:43Z | NONE | That would be handy (additional where filters) but I think the trick with the `with` statement is already an order of magnitude better than what I had thought of, so my problem is solved by it (plus I got to learn about `with` today!) | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Combining `rows_where()` and `search()` to limit which rows are searched 1257724585 | |
1139426398 | https://github.com/simonw/sqlite-utils/issues/439#issuecomment-1139426398 | https://api.github.com/repos/simonw/sqlite-utils/issues/439 | IC_kwDOCGYnMM5D6kRe | frafra 4068 | 2022-05-27T09:04:05Z | 2022-05-27T10:44:54Z | NONE | This code works: ```python import csv import sqlite_utils db = sqlite_utils.Database("test.db") reader = csv.DictReader(open("csv", encoding="utf-16-le").read().split("\r\n"), delimiter=";") db["test"].insert_all(reader, pk="Id") ``` I used `iconv` to change the encoding; sqlite-utils can import the resulting file, even if it stops at 98 %: ``` sqlite-utils insert --csv test test.db clean [------------------------------------] 0% [###################################-] 98% 00:00:00 ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Misleading progress bar against utf-16-le CSV input 1250495688 | |
1139379923 | https://github.com/simonw/sqlite-utils/issues/438#issuecomment-1139379923 | https://api.github.com/repos/simonw/sqlite-utils/issues/438 | IC_kwDOCGYnMM5D6Y7T | frafra 4068 | 2022-05-27T08:05:01Z | 2022-05-27T08:05:01Z | NONE | I tried to debug it using `pdb`, but it looks `sqlite-utils` catches the exception, so it is not quick to figure out where the failure is happening. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | illegal UTF-16 surrogate 1250161887 | |
1139392769 | https://github.com/simonw/sqlite-utils/issues/438#issuecomment-1139392769 | https://api.github.com/repos/simonw/sqlite-utils/issues/438 | IC_kwDOCGYnMM5D6cEB | frafra 4068 | 2022-05-27T08:21:53Z | 2022-05-27T08:21:53Z | NONE | Argument were specified in the wrong order. `PATH TABLE FILE` can be misleading :) | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | illegal UTF-16 surrogate 1250161887 | |
1139484453 | https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1139484453 | https://api.github.com/repos/simonw/sqlite-utils/issues/433 | IC_kwDOCGYnMM5D6ycl | frafra 4068 | 2022-05-27T10:20:08Z | 2022-05-27T10:20:08Z | NONE | I can confirm. This only happens with sqlite-utils. I am using gnome-terminal with bash. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | CLI eats my cursor 1239034903 | |
1416486796 | https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1416486796 | https://api.github.com/repos/simonw/sqlite-utils/issues/433 | IC_kwDOCGYnMM5Ubd-M | alecstein 16236421 | 2023-02-03T22:32:10Z | 2023-02-03T22:32:10Z | NONE | Came here to say that I also have this issue. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | CLI eats my cursor 1239034903 | |
1578840450 | https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1578840450 | https://api.github.com/repos/simonw/sqlite-utils/issues/433 | IC_kwDOCGYnMM5eGzGC | jonafato 392720 | 2023-06-06T14:09:04Z | 2023-06-06T14:09:04Z | NONE | I also ran into this recently. See below for a patch for one possible solution (tested via "it works on my machine", but I don't expect that this behavior would vary a whole lot across terminal emulators and shells). Another possible solution might be to subclass click's `ProgressBar` to keep the logic within the original context manager. Happy to send a PR or for this patch to serve as the basis for a fix that someone else authors. ```patch diff --git a/sqlite_utils/utils.py b/sqlite_utils/utils.py index 06c1a4c..530a3a3 100644 --- a/sqlite_utils/utils.py +++ b/sqlite_utils/utils.py @@ -147,14 +147,23 @@ def decode_base64_values(doc): class UpdateWrapper: - def __init__(self, wrapped, update): + def __init__(self, wrapped, update, render_finish): self._wrapped = wrapped self._update = update + self._render_finish = render_finish def __iter__(self): - for line in self._wrapped: - self._update(len(line)) - yield line + return self + + def __next__(self): + try: + line = next(self._wrapped) + except StopIteration as e: + self._render_finish() + raise + + self._update(len(line)) + return line def read(self, size=-1): data = self._wrapped.read(size) @@ -178,7 +187,7 @@ def file_progress(file, silent=False, **kwargs): else: file_length = os.path.getsize(file.name) with click.progressbar(length=file_length, **kwargs) as bar: - yield UpdateWrapper(file, bar.update) + yield UpdateWrapper(file, bar.update, bar.render_finish) class Format(enum.Enum): ``` | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 1, "eyes": 0} | CLI eats my cursor 1239034903 | |
1640826795 | https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1640826795 | https://api.github.com/repos/simonw/sqlite-utils/issues/433 | IC_kwDOCGYnMM5hzQer | J450n-4-W 76528036 | 2023-07-18T19:08:50Z | 2023-07-18T19:08:50Z | NONE | Came here to report this, but instead I'll confirm the issue across two terminal emulators (Gnome Terminal and Alacritty) on Pop_OS! 22.04 (currently based on Ubuntu/Gnome). Also messes up the formatting of the terminal. Can also confirm that reset fixes it until the next sqlite-utils command. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | CLI eats my cursor 1239034903 | |
1126295407 | https://github.com/simonw/sqlite-utils/issues/431#issuecomment-1126295407 | https://api.github.com/repos/simonw/sqlite-utils/issues/431 | IC_kwDOCGYnMM5DIedv | rafguns 738408 | 2022-05-13T17:47:32Z | 2022-05-13T17:47:32Z | NONE | I'd be happy to write a PR for this, if you think it's worth having. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Allow making m2m relation of a table to itself 1227571375 | |
1164460052 | https://github.com/simonw/sqlite-utils/issues/431#issuecomment-1164460052 | https://api.github.com/repos/simonw/sqlite-utils/issues/431 | IC_kwDOCGYnMM5FaEAU | rafguns 738408 | 2022-06-23T14:12:51Z | 2022-06-23T14:12:51Z | NONE | Yeah, I think I prefer your suggestion: it seems cleaner than my initial `left_name=`/`right_name=` idea. Perhaps one downside is that it's less obvious what the role of each field is: in this example, is `people_id_1` a reference to parent or child? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Allow making m2m relation of a table to itself 1227571375 | |
1116336340 | https://github.com/simonw/sqlite-utils/issues/430#issuecomment-1116336340 | https://api.github.com/repos/simonw/sqlite-utils/issues/430 | IC_kwDOCGYnMM5CifDU | rayvoelker 9308268 | 2022-05-03T17:03:31Z | 2022-05-03T17:03:31Z | NONE | So, the good news is that it appears that setting one of those PRAGMA statements fixed the issue of `table.extract()` method call on this large database completing (that I described above.) The bad news is that I'm not sure which one! I wonder if it's something system / environment specific about SQLite, or maybe something else going on. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Document how to use `PRAGMA temp_store` to avoid errors when running VACUUM against huge databases 1224112817 | |
1129332959 | https://github.com/simonw/sqlite-utils/issues/425#issuecomment-1129332959 | https://api.github.com/repos/simonw/sqlite-utils/issues/425 | IC_kwDOCGYnMM5DUEDf | McEazy2700 102771161 | 2022-05-17T21:27:02Z | 2022-05-17T21:27:02Z | NONE | Hi, I'm trying to deploy my site using elasticbeanstalk and I keep getting this same error : deterministic=True requires SQLite 3.8.3 or higher I saw your previous solution that involves editing sqlite-utils/sqlite_utils/db.py file, but I'm curious as to how that will work in production. | {"total_count": 5, "+1": 5, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `sqlite3.NotSupportedError`: deterministic=True requires SQLite 3.8.3 or higher 1203842656 | |
1509951952 | https://github.com/simonw/sqlite-utils/issues/425#issuecomment-1509951952 | https://api.github.com/repos/simonw/sqlite-utils/issues/425 | IC_kwDOCGYnMM5aAAnQ | Dhyanesh97 89400147 | 2023-04-15T20:14:58Z | 2023-04-15T20:14:58Z | NONE | is this change released ? Because when we run docker containers issue still persists for production deployments. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `sqlite3.NotSupportedError`: deterministic=True requires SQLite 3.8.3 or higher 1203842656 | |
1081079506 | https://github.com/simonw/sqlite-utils/issues/421#issuecomment-1081079506 | https://api.github.com/repos/simonw/sqlite-utils/issues/421 | IC_kwDOCGYnMM5Ab_bS | learning4life 24938923 | 2022-03-28T19:58:55Z | 2022-03-28T20:05:57Z | NONE | Sure, it is from the documentation example: [Extracting columns into a separate table](https://sqlite-utils.datasette.io/en/stable/cli.html#extracting-columns-into-a-separate-table) ``` wget "https://github.com/wri/global-power-plant-database/blob/232a6666/output_database/global_power_plant_database.csv?raw=true" sqlite-utils insert global.db power_plants \ 'global_power_plant_database.csv?raw=true' --csv # Extract those columns: sqlite-utils extract global.db power_plants country country_long \ --table countries \ --fk-column country_id \ --rename country_long name ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | "Error: near "(": syntax error" when using sqlite-utils indexes CLI 1180427792 | |
1082476727 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1082476727 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5AhUi3 | strada 770231 | 2022-03-29T23:52:38Z | 2022-03-29T23:52:38Z | NONE | @simonw Thanks for looking into it and documenting the solution! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Document how to use a `--convert` function that runs initialization code first 1178546862 | |
1076662556 | https://github.com/simonw/sqlite-utils/pull/419#issuecomment-1076662556 | https://api.github.com/repos/simonw/sqlite-utils/issues/419 | IC_kwDOCGYnMM5ALJEc | codecov[bot] 22429695 | 2022-03-23T18:12:47Z | 2022-03-23T18:12:47Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/419?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#419](https://codecov.io/gh/simonw/sqlite-utils/pull/419?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (228f736) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/93fa79d30b1531bea281d0eb6b925c4e61bc1aa6?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (93fa79d) will **not change** coverage. > The diff coverage is `n/a`. ```diff @@ Coverage Diff @@ ## main #419 +/- ## ======================================= Coverage 96.55% 96.55% ======================================= Files 6 6 Lines 2498 2498 ======================================= Hits 2412 2412 Misses 86 86 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/419?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/419?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [93fa79d...228f736](https://codecov.io/gh/simonw/sqlite-utils/pull/419?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Ignore common generated files 1178484369 | |
1074256603 | https://github.com/simonw/sqlite-utils/issues/417#issuecomment-1074256603 | https://api.github.com/repos/simonw/sqlite-utils/issues/417 | IC_kwDOCGYnMM5AB9rb | blaine 9954 | 2022-03-21T18:19:41Z | 2022-03-21T18:19:41Z | NONE | That makes sense; just a little hint that points folks towards doing the right thing might be helpful! fwiw, the reason I was using jq in the first place was just a quick way to extract one attribute from an actual JSON array. When I initially imported it, I got a table with a bunch of embedded JSON values, rather than a native table, because each array entry had two attributes, one with the data I _actually_ wanted. Not sure how common a use-case this is, though (and easily fixed, aside from the jq weirdness!) | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | insert fails on JSONL with whitespace 1175744654 | |
1116684581 | https://github.com/simonw/sqlite-utils/issues/416#issuecomment-1116684581 | https://api.github.com/repos/simonw/sqlite-utils/issues/416 | IC_kwDOCGYnMM5Cj0El | mattkiefer 638427 | 2022-05-03T21:36:49Z | 2022-05-03T21:36:49Z | NONE | Thanks for addressing this @simonw! However, I just reinstalled sqlite-utils 3.26.1 and get an `ParserError: Unknown string format: None`: ``` sqlite-utils --version sqlite-utils, version 3.26.1 ``` ``` sqlite-utils convert idfpr.db license "Original Issue Date" "r.parsedate(value)" Traceback (most recent call last): File "/home/matt/.local/lib/python3.9/site-packages/sqlite_utils/db.py", line 2514, in convert_value return fn(v) File "<string>", line 2, in fn File "/home/matt/.local/lib/python3.9/site-packages/sqlite_utils/recipes.py", line 19, in parsedate parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst) File "/usr/lib/python3/dist-packages/dateutil/parser/_parser.py", line 1374, in parse return DEFAULTPARSER.parse(timestr, **kwargs) File "/usr/lib/python3/dist-packages/dateutil/parser/_parser.py", line 649, in parse raise ParserError("Unknown string format: %s", timestr) dateutil.parser._parser.ParserError: Unknown string format: None Traceback (most recent call last): File "/home/matt/.local/bin/sqlite-utils", line 8, in <module> sys.exit(cli()) File "/usr/lib/python3/dist-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3/dist-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/usr/lib/python3/dist-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/lib/python3/dist-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/lib/python3/dist-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/home/matt/.local/lib/python3.9/site-packages/sqlite_utils/cli.py", line 2707, in convert db[table].convert( File "/home/matt/.local/lib/python3.9/site-packages/sqlite_utils/db.py", line 2530, in convert self.db.execute(sql, where_args or []) File "/home/matt/.local/lib/python3.9/site-packages/s… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Options for how `r.parsedate()` should handle invalid dates 1173023272 | |
1059652834 | https://github.com/simonw/sqlite-utils/issues/412#issuecomment-1059652834 | https://api.github.com/repos/simonw/sqlite-utils/issues/412 | IC_kwDOCGYnMM4_KQTi | zaneselvans 596279 | 2022-03-05T02:14:40Z | 2022-03-05T02:14:40Z | NONE | We do a lot of `df.to_sql()` to write into sqlite, mostly in [this moddule](https://github.com/catalyst-cooperative/pudl/blob/main/src/pudl/load.py#L25) | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Optional Pandas integration 1160182768 | |
1059097969 | https://github.com/simonw/sqlite-utils/issues/408#issuecomment-1059097969 | https://api.github.com/repos/simonw/sqlite-utils/issues/408 | IC_kwDOCGYnMM4_II1x | learning4life 24938923 | 2022-03-04T11:55:21Z | 2022-03-04T11:55:21Z | NONE | Thanks @simonw I will test it after my vacation 👍 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `deterministic=True` fails on versions of SQLite prior to 3.8.3 1145882578 | |
1066139147 | https://github.com/simonw/sqlite-utils/issues/408#issuecomment-1066139147 | https://api.github.com/repos/simonw/sqlite-utils/issues/408 | IC_kwDOCGYnMM4_i_4L | learning4life 24938923 | 2022-03-13T16:45:00Z | 2022-03-13T16:54:09Z | NONE | @simonw Now I get this: ``` (app-root) sqlite-utils indexes global.db --table Error: near "(": syntax error (app-root) sqlite-utils --version sqlite-utils, version 3.25.1 (app-root) sqlite3 --version 3.36.0 2021-06-18 18:36:39 (app-root) python --version Python 3.8.11 ``` Dockerfile ``` FROM centos/python-38-centos7 USER root RUN yum update -y RUN yum upgrade -y # epel RUN yum -y install epel-release && yum clean all # SQLite RUN yum -y install zlib-devel geos geos-devel proj proj-devel freexl freexl-devel libxml2-devel WORKDIR /build/ COPY sqlite-autoconf-3360000.tar.gz ./ RUN tar -zxf sqlite-autoconf-3360000.tar.gz WORKDIR /build/sqlite-autoconf-3360000 RUN ./configure RUN make RUN make install # RUN /opt/app-root/bin/python3.8 -m pip install --upgrade pip RUN pip install sqlite-utils ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `deterministic=True` fails on versions of SQLite prior to 3.8.3 1145882578 | |
1040519196 | https://github.com/simonw/sqlite-utils/pull/407#issuecomment-1040519196 | https://api.github.com/repos/simonw/sqlite-utils/issues/407 | IC_kwDOCGYnMM4-BRAc | codecov[bot] 22429695 | 2022-02-15T16:52:21Z | 2022-02-15T18:12:03Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/407?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#407](https://codecov.io/gh/simonw/sqlite-utils/pull/407?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (a974da5) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/e7f040106b5f5a892ebd984f19b21c605e87c142?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (e7f0401) will **increase** coverage by `0.71%`. > The diff coverage is `85.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/407/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/407?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #407 +/- ## ========================================== + Coverage 95.91% 96.62% +0.71% ========================================== Files 6 6 Lines 2421 2460 +39 ========================================== + Hits 2322 2377 +55 + Misses 99 83 -16 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/407?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/407/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.76% <85.00%> (+0.0… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add SpatiaLite helpers to CLI 1138948786 | |
1041313679 | https://github.com/simonw/sqlite-utils/issues/406#issuecomment-1041313679 | https://api.github.com/repos/simonw/sqlite-utils/issues/406 | IC_kwDOCGYnMM4-ES-P | psychemedia 82988 | 2022-02-16T09:59:51Z | 2022-02-16T10:00:10Z | NONE | The `CustomColumnType()` approach looks good. This pushes you into the mindspace that you are defining and working with a custom column type. When creating the table, you could then error, or at least warn, if someone wasn't setting a column on a `type` or a custom column type, which I guess is where `mypy` comes in? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Creating tables with custom datatypes 1128466114 | |
1041363433 | https://github.com/simonw/sqlite-utils/issues/406#issuecomment-1041363433 | https://api.github.com/repos/simonw/sqlite-utils/issues/406 | IC_kwDOCGYnMM4-EfHp | psychemedia 82988 | 2022-02-16T10:57:03Z | 2022-02-16T10:57:19Z | NONE | Wondering if this actually relates to https://github.com/simonw/sqlite-utils/issues/402 ? I also wonder if this would be a sensible approach for eg registering `pint` based quantity conversions into and out of the db, perhaps storing the quantity as a serialised `magnitude measurement` single column string? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Creating tables with custom datatypes 1128466114 | |
1248440137 | https://github.com/simonw/sqlite-utils/issues/406#issuecomment-1248440137 | https://api.github.com/repos/simonw/sqlite-utils/issues/406 | IC_kwDOCGYnMM5Kaa9J | psychemedia 82988 | 2022-09-15T18:13:50Z | 2022-09-15T18:13:50Z | NONE | I was wondering if you have any more thoughts on this? I have a tangible use case now: adding a "vector" column to a database to support semantic search using doc2vec embeddings ([example](https://psychemedia.github.io/storynotes/Lang_Doc2Vec.html); note that the `vtfunc` package may no longer be reliable...). | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Creating tables with custom datatypes 1128466114 | |
1041325398 | https://github.com/simonw/sqlite-utils/issues/402#issuecomment-1041325398 | https://api.github.com/repos/simonw/sqlite-utils/issues/402 | IC_kwDOCGYnMM4-EV1W | psychemedia 82988 | 2022-02-16T10:12:48Z | 2022-02-16T10:18:55Z | NONE | > My hunch is that the case where you want to consider input from more than one column will actually be pretty rare - the only case I can think of where I would want to do that is for latitude/longitude columns Other possible pairs: unconventional date/datetime and timezone pairs eg `2022-02-16::17.00, London`; or more generally, numerical value and unit of measurement pairs (eg if you want to cast into and out of different measurement units using packages like `pint`) or currencies etc. Actually, in that case, I guess you may be presenting things that are unit typed already, and so a conversion would need to parse things into an appropriate, possibly two column `value, unit` format. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Advanced class-based `conversions=` mechanism 1125297737 | |
1030807433 | https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1030807433 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | IC_kwDOCGYnMM49cN-J | chris48s 6025893 | 2022-02-06T10:54:09Z | 2022-02-06T10:54:09Z | NONE | > Interesting that some accept an SRID and others do not - presumably GeomFromGeoJSON() always uses SRID=4326? The ewtk/ewkb ones don't accept an SRID is because ewkt encodes the SRID in the string, so you would do this with a wkt string: `GeomFromText('POINT(529090 179645)', 27700)` but for ewkt it would be `GeomFromEWKT('SRID=27700;POINT(529090 179645)')` The specs for KML and GeoJSON specify a Coordinate Reference System for the format - https://datatracker.ietf.org/doc/html/rfc7946#section-4 - https://docs.opengeospatial.org/is/12-007r2/12-007r2.html#1274 GML can specify the SRID in the XML at feature level e.g: ``` <gml:Point srsName="EPSG:27700"> <gml:coordinates>529090, 179645</gml:coordinates> </gml:Point> ``` There's a few more obscure formats in there, but broadly I think it is safe to assume an SRID param exists on the function for cases where the SRID is not implied by or specified in the input format. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Make it easier to insert geometries, with documentation and maybe code 1124731464 | |
1548913065 | https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1548913065 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | IC_kwDOCGYnMM5cUomp | chrislkeller 433780 | 2023-05-16T03:11:03Z | 2023-05-16T03:11:52Z | NONE | Using this thread and some [other resources](https://sqlite-utils.datasette.io/en/stable/cli.html#spatialite-helpers) I managed to cobble together a couple of sqlite-utils lines to add a geometry column for a table that already has a lat/lng column. ``` # add a geometry column sqlite-utils add-geometry-column [db name] [table name] geometry --type POINT --srid 4326 # add a point for each row to geometry column sqlite-utils --load-extension=spatialite [db name] 'update [table name] SET Geometry=MakePoint(longitude, latitude, 4326);' ``` | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Make it easier to insert geometries, with documentation and maybe code 1124731464 | |
1029177700 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029177700 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WAFk | codecov[bot] 22429695 | 2022-02-03T16:38:45Z | 2022-02-04T05:52:39Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/385?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#385](https://codecov.io/gh/simonw/sqlite-utils/pull/385?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (af86b17) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/74586d3cb26fa3cc3412721985ecdc1864c2a31d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (74586d3) will **decrease** coverage by `0.61%`. > The diff coverage is `28.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/385/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/385?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #385 +/- ## ========================================== - Coverage 96.52% 95.91% -0.62% ========================================== Files 6 6 Lines 2389 2421 +32 ========================================== + Hits 2306 2322 +16 - Misses 83 99 +16 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/385?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/385/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.69% <ø> (+0.15%)` … | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add new spatialite helper methods 1102899312 | |
1009531863 | https://github.com/simonw/sqlite-utils/pull/377#issuecomment-1009531863 | https://api.github.com/repos/simonw/sqlite-utils/issues/377 | IC_kwDOCGYnMM48LDvX | codecov[bot] 22429695 | 2022-01-11T02:03:00Z | 2022-01-11T02:03:00Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/377?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#377](https://codecov.io/gh/simonw/sqlite-utils/pull/377?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (f4ea0d3) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/b6dad08a8389736b7e960cfe9bc719cfc21a98f5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (b6dad08) will **decrease** coverage by `0.01%`. > The diff coverage is `96.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/377/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/377?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #377 +/- ## ========================================== - Coverage 96.52% 96.50% -0.02% ========================================== Files 6 6 Lines 2330 2378 +48 ========================================== + Hits 2249 2295 +46 - Misses 81 83 +2 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/377?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/377/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.49% <94.28%> (-0.0… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `sqlite-utils bulk` command 1097477582 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
updated_at (date) >30 ✖