home / github

Menu
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

721 rows where author_association = "NONE" sorted by issue_url

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

author_association 1 ✖

  • NONE · 721 ✖
id html_url issue_url ▼ node_id user created_at updated_at author_association body reactions issue performed_via_github_app
748436115 https://github.com/dogsheep/dogsheep-photos/issues/15#issuecomment-748436115 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/15 MDEyOklzc3VlQ29tbWVudDc0ODQzNjExNQ== nickvazz 8573886 2020-12-19T07:43:38Z 2020-12-19T07:47:36Z NONE Hey Simon! I really enjoy datasette so far, just started trying it out today following your iPhone photos [example](https://simonwillison.net/2020/May/21/dogsheep-photos/). I am not sure if you had run into this or not, but it seems like they might have changed one of the column names from `ZGENERICASSET` to `ZASSET`. Should I open a PR? Would change: - [here](https://github.com/dogsheep/dogsheep-photos/blob/master/dogsheep_photos/cli.py#L209-L213) - [here](https://github.com/dogsheep/dogsheep-photos/blob/master/dogsheep_photos/cli.py#L238) - [here](https://github.com/dogsheep/dogsheep-photos/blob/master/dogsheep_photos/cli.py#L240) {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Expose scores from ZCOMPUTEDASSETATTRIBUTES 612151767  
633234781 https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633234781 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20 MDEyOklzc3VlQ29tbWVudDYzMzIzNDc4MQ== dmd 41439 2020-05-24T13:56:13Z 2020-05-24T13:56:13Z NONE As that seems to be closed, can you give a hint on how to make this work? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Ability to serve thumbnailed Apple Photo from its place on disk 613006393  
748436195 https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-748436195 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21 MDEyOklzc3VlQ29tbWVudDc0ODQzNjE5NQ== nickvazz 8573886 2020-12-19T07:44:32Z 2020-12-19T07:44:49Z NONE I have also run into this a bit, would it be possible to post your `requirements.txt` so I can try and reproduce your [blog post](https://simonwillison.net/2020/May/21/dogsheep-photos/)? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} bpylist.archiver.CircularReference: archive has a cycle with uid(13) 615474990  
751125270 https://github.com/dogsheep/dogsheep-photos/issues/28#issuecomment-751125270 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/28 MDEyOklzc3VlQ29tbWVudDc1MTEyNTI3MA== jmelloy 129786 2020-12-24T22:26:22Z 2020-12-24T22:26:22Z NONE This comes around if you’ve run the photo export without running an s3 upload. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Invalid SQL no such table: main.uploads 624490929  
934207940 https://github.com/dogsheep/dogsheep-photos/issues/3#issuecomment-934207940 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/3 IC_kwDOD079W843ruHE jratike80 1751612 2021-10-05T08:57:41Z 2021-10-05T08:57:41Z NONE Maybe the exif-loader from the SpatiaLite project could be useful as a reference even it is written in C and it also saves images as blobs https://www.gaia-gis.it/fossil/spatialite-tools/file?name=exif_loader.c&ci=tip. The tool is also integrated into the spatialite-gui application. I found some user documentation from the web archive http://web.archive.org/web/20180629041238/https://www.gaia-gis.it/spatialite-2.3.1/spatialite-exif-2.3.1.html. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Import EXIF data into SQLite - lens used, ISO, aperture etc 602533481  
811362316 https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-811362316 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31 MDEyOklzc3VlQ29tbWVudDgxMTM2MjMxNg== PabloLerma 871250 2021-03-31T19:14:39Z 2021-03-31T19:14:39Z NONE 👋 could I help somehow for this to be merged? As Big Sur is going to be more used as the time goes I think it would be nice to merge and publish a new version. Nice work! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Update for Big Sur 771511344  
1035717429 https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-1035717429 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31 IC_kwDOD079W849u8s1 harperreed 18504 2022-02-11T01:55:38Z 2022-02-11T01:55:38Z NONE I would love this merged! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Update for Big Sur 771511344  
791053721 https://github.com/dogsheep/dogsheep-photos/issues/32#issuecomment-791053721 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/32 MDEyOklzc3VlQ29tbWVudDc5MTA1MzcyMQ== dsisnero 6213 2021-03-05T00:31:27Z 2021-03-05T00:31:27Z NONE I am getting the same thing for US West (N. California) us-west-1 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} KeyError: 'Contents' on running upload 803333769  
882091516 https://github.com/dogsheep/dogsheep-photos/issues/32#issuecomment-882091516 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/32 IC_kwDOD079W840k6X8 aaronyih1 10793464 2021-07-18T17:29:39Z 2021-07-18T17:33:02Z NONE Same here for US West (N. California) us-west-1. Running on Catalina. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} KeyError: 'Contents' on running upload 803333769  
884688833 https://github.com/dogsheep/dogsheep-photos/issues/32#issuecomment-884688833 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/32 IC_kwDOD079W840u0fB aaronyih1 10793464 2021-07-22T06:40:25Z 2021-07-22T06:40:25Z NONE The solution here is to upload an image to the bucket first. It is caused because it does not properly handle the case when there are no images in the bucket. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} KeyError: 'Contents' on running upload 803333769  
777951854 https://github.com/dogsheep/dogsheep-photos/issues/33#issuecomment-777951854 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/33 MDEyOklzc3VlQ29tbWVudDc3Nzk1MTg1NA== leafgarland 675335 2021-02-12T03:54:39Z 2021-02-12T03:54:39Z NONE I think that is a typo in the docs, you can use > dogsheep-photos apple-photos photos.db {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} photo-to-sqlite: command not found 803338729  
778002092 https://github.com/dogsheep/dogsheep-photos/issues/33#issuecomment-778002092 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/33 MDEyOklzc3VlQ29tbWVudDc3ODAwMjA5Mg== robmarkcole 11855322 2021-02-12T06:19:32Z 2021-02-12T06:19:32Z NONE hi @leafgarland that results in a new error: ``` (venv) (base) Robins-MacBook:datasette robin$ dogsheep-photos apple-photos photos.db Traceback (most recent call last): File "/Users/robin/datasette/venv/bin/dogsheep-photos", line 8, in <module> sys.exit(cli()) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/dogsheep_photos/cli.py", line 206, in apple_photos db.conn.execute( sqlite3.OperationalError: no such table: attached.ZGENERICASSET ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} photo-to-sqlite: command not found 803338729  
778014990 https://github.com/dogsheep/dogsheep-photos/issues/33#issuecomment-778014990 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/33 MDEyOklzc3VlQ29tbWVudDc3ODAxNDk5MA== leafgarland 675335 2021-02-12T06:54:14Z 2021-02-12T06:54:14Z NONE Ahh, that might be because macOS Big Sur has changed the structure of the photos db. Might need to wait for a later release, there is a PR which adds support for Big Sur. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} photo-to-sqlite: command not found 803338729  
813249000 https://github.com/dogsheep/dogsheep-photos/issues/35#issuecomment-813249000 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/35 MDEyOklzc3VlQ29tbWVudDgxMzI0OTAwMA== ligurio 1151557 2021-04-05T07:37:57Z 2021-04-05T07:37:57Z NONE There are trained ML models used in Photoprism: - https://dl.photoprism.org/tensorflow/nasnet.zip - https://dl.photoprism.org/tensorflow/nsfw.zip {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Support to annotate photos on other than macOS OSes 842695374  
1006708046 https://github.com/dogsheep/dogsheep-photos/pull/36#issuecomment-1006708046 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/36 IC_kwDOD079W848ASVO scoates 71983 2022-01-06T16:04:46Z 2022-01-06T16:04:46Z NONE This one got me, today, too. 👍 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Correct naming of tool in readme 988493790  
906015471 https://github.com/dogsheep/dogsheep-photos/issues/7#issuecomment-906015471 https://api.github.com/repos/dogsheep/dogsheep-photos/issues/7 IC_kwDOD079W842ALLv dkam 18232 2021-08-26T02:01:01Z 2021-08-26T02:01:01Z NONE Perceptual hashes might be what you're after : http://phash.org {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Integrate image content hashing 602585497  
1021264135 https://github.com/dogsheep/dogsheep.github.io/pull/6#issuecomment-1021264135 https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/6 IC_kwDODMzF1s4830EH ligurio 1151557 2022-01-25T14:52:40Z 2022-01-25T14:52:40Z NONE @simonw, could you review? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add testres-db tool 842765105  
777690332 https://github.com/dogsheep/evernote-to-sqlite/issues/11#issuecomment-777690332 https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/11 MDEyOklzc3VlQ29tbWVudDc3NzY5MDMzMg== dskrad 3613583 2021-02-11T18:16:01Z 2021-02-11T18:16:01Z NONE I solved this issue by modifying line 31 of utils.py content = ET.tostring(ET.fromstring(content_xml.strip())).decode("utf-8") {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} XML parse error 792851444  
911772943 https://github.com/dogsheep/evernote-to-sqlite/issues/14#issuecomment-911772943 https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/14 IC_kwDOEhK-wc42WI0P step21 46968 2021-09-02T14:53:11Z 2021-09-02T14:53:11Z NONE Additionally, assuming the line numbers match up with the provided enenx file, the mentioned line plus one before and after is as follows: ``` <![CDATA[>]]> </span></div> <div style="padding: 0px; font-family: Arial, sans-serif; font-size: 12px; line-height: 16px; white-space: pre-wrap;"><br style=" padding: 0px;"/> ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} xml.etree.ElementTree.Parse Error - mismatched tag 986829194  
765495861 https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765495861 https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1 MDEyOklzc3VlQ29tbWVudDc2NTQ5NTg2MQ== cobiadigital 25372415 2021-01-22T15:44:00Z 2021-01-22T15:44:00Z NONE Risk of autoimmune disorders: https://www.snpedia.com/index.php/Genotype ``` select rsid, genotype, case genotype when 'AA' then '2x risk of rheumatoid arthritis and other autoimmune diseases' when 'GG' then 'Normal risk for autoimmune disorders' end as interpretation from genome where rsid = 'rs2476601' ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Figure out some interesting example SQL queries 496415321  
765498984 https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765498984 https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1 MDEyOklzc3VlQ29tbWVudDc2NTQ5ODk4NA== cobiadigital 25372415 2021-01-22T15:48:25Z 2021-01-22T15:49:33Z NONE The "Warrior Gene" https://www.snpedia.com/index.php/Rs4680 ``` select rsid, genotype, case genotype when 'AA' then '(worrier) advantage in memory and attention tasks' when 'AG' then 'Intermediate dopamine levels, other effects' when 'GG' then '(warrior) multiple associations, see details' end as interpretation from genome where rsid = 'rs4680' ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Figure out some interesting example SQL queries 496415321  
765502845 https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765502845 https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1 MDEyOklzc3VlQ29tbWVudDc2NTUwMjg0NQ== cobiadigital 25372415 2021-01-22T15:53:19Z 2021-01-22T15:53:19Z NONE rs7903146 Influences risk of Type-2 diabetes https://www.snpedia.com/index.php/Rs7903146 ``` select rsid, genotype, case genotype when 'CC' then 'Normal (lower) risk of Type 2 Diabetes and Gestational Diabetes.' when 'CT' then '1.4x increased risk for diabetes (and perhaps colon cancer).' when 'TT' then '2x increased risk for Type-2 diabetes' end as interpretation from genome where rsid = 'rs7903146' ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Figure out some interesting example SQL queries 496415321  
765506901 https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765506901 https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1 MDEyOklzc3VlQ29tbWVudDc2NTUwNjkwMQ== cobiadigital 25372415 2021-01-22T15:58:41Z 2021-01-22T15:58:58Z NONE Both rs10757274 and rs2383206 can both indicate higher risks of heart disease https://www.snpedia.com/index.php/Rs2383206 ``` select rsid, genotype, case genotype when 'AA' then 'Normal' when 'AG' then '~1.2x increased risk for heart disease' when 'GG' then '~1.3x increased risk for heart disease' end as interpretation from genome where rsid = 'rs10757274' ``` ``` select rsid, genotype, case genotype when 'AA' then 'Normal' when 'AG' then '1.4x increased risk for heart disease' when 'GG' then '1.7x increased risk for heart disease' end as interpretation from genome where rsid = 'rs2383206' ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Figure out some interesting example SQL queries 496415321  
765523517 https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765523517 https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1 MDEyOklzc3VlQ29tbWVudDc2NTUyMzUxNw== cobiadigital 25372415 2021-01-22T16:20:25Z 2021-01-22T16:20:25Z NONE rs53576: the oxytocin receptor (OXTR) gene ``` select rsid, genotype, case genotype when 'AA' then 'Lack of empathy?' when 'AG' then 'Lack of empathy?' when 'GG' then 'Optimistic and empathetic; handle stress well' end as interpretation from genome where rsid = 'rs53576' ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Figure out some interesting example SQL queries 496415321  
765525338 https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765525338 https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1 MDEyOklzc3VlQ29tbWVudDc2NTUyNTMzOA== cobiadigital 25372415 2021-01-22T16:22:44Z 2021-01-22T16:22:44Z NONE rs1333049 associated with coronary artery disease https://www.snpedia.com/index.php/Rs1333049 ``` select rsid, genotype, case genotype when 'CC' then '1.9x increased risk for coronary artery disease' when 'CG' then '1.5x increased risk for CAD' when 'GG' then 'normal' end as interpretation from genome where rsid = 'rs1333049' ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Figure out some interesting example SQL queries 496415321  
831004775 https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-831004775 https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1 MDEyOklzc3VlQ29tbWVudDgzMTAwNDc3NQ== cobiadigital 25372415 2021-05-03T03:46:23Z 2021-05-03T03:46:23Z NONE RS1800955 is related to novelty seeking and ADHD https://www.snpedia.com/index.php/Rs1800955 `select rsid, genotype, case genotype when 'CC' then 'increased susceptibility to novelty seeking' when 'CT' then 'increased susceptibility to novelty seeking' when 'TT' then 'normal' end as interpretation from genome where rsid = 'rs1800955'` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Figure out some interesting example SQL queries 496415321  
605439685 https://github.com/dogsheep/github-to-sqlite/issues/15#issuecomment-605439685 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/15 MDEyOklzc3VlQ29tbWVudDYwNTQzOTY4NQ== garethr 2029 2020-03-28T12:17:01Z 2020-03-28T12:17:01Z NONE That looks great, thanks! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Assets table with downloads 544571092  
571412923 https://github.com/dogsheep/github-to-sqlite/issues/16#issuecomment-571412923 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/16 MDEyOklzc3VlQ29tbWVudDU3MTQxMjkyMw== jayvdb 15092 2020-01-07T03:06:46Z 2020-01-07T03:06:46Z NONE I re-tried after doing `auth`, and I get the same result. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Exception running first command: IndexError: list index out of range 546051181  
602136481 https://github.com/dogsheep/github-to-sqlite/issues/16#issuecomment-602136481 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/16 MDEyOklzc3VlQ29tbWVudDYwMjEzNjQ4MQ== jayvdb 15092 2020-03-22T02:08:57Z 2020-03-22T02:08:57Z NONE I'd love to be using your library as a better cached gh layer for a new library I have built, replacing large parts of the very ugly https://github.com/jayvdb/pypidb/blob/master/pypidb/_github.py , and then probably being able to rebuild the setuppy chunk as a feature here at a later stage. I would also need tokenless and netrc support, but I would be happy to add those bits. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Exception running first command: IndexError: list index out of range 546051181  
622279374 https://github.com/dogsheep/github-to-sqlite/issues/33#issuecomment-622279374 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/33 MDEyOklzc3VlQ29tbWVudDYyMjI3OTM3NA== garethr 2029 2020-05-01T07:12:47Z 2020-05-01T07:12:47Z NONE I also go it working with: ```yaml run: echo ${{ secrets.github_token }} | github-to-sqlite auth ``` {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Fall back to authentication via ENV 609950090  
623038148 https://github.com/dogsheep/github-to-sqlite/issues/38#issuecomment-623038148 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/38 MDEyOklzc3VlQ29tbWVudDYyMzAzODE0OA== zzeleznick 5779832 2020-05-03T01:18:57Z 2020-05-03T01:18:57Z NONE Thanks, @simonw! I feel a little foolish in hindsight, but I'm on the same page now and am glad to have discovered first-hand a motivation for this `repos_starred` use case. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} [Feature Request] Support Repo Name in Search 🥺 611284481  
623044643 https://github.com/dogsheep/github-to-sqlite/issues/38#issuecomment-623044643 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/38 MDEyOklzc3VlQ29tbWVudDYyMzA0NDY0Mw== zzeleznick 5779832 2020-05-03T02:34:32Z 2020-05-03T02:34:32Z NONE 1. More than glad to share feedback from the sidelines as a [starrer](https://github-to-sqlite.dogsheep.net/github?sql=select%0D%0A++starred_at%2C%0D%0A++starred_by%2C%0D%0A++full_name+as+repo_name%0D%0Afrom%0D%0A++repos_starred%0D%0Awhere%0D%0A++starred_by+%3D+%22zzeleznick%22%0D%0Aorder+by%0D%0A++starred_at+desc). ``` -- Motivation: -- Datasette is a data hammer and I'm looking for nails -- e.g. Find which repos a user has starred => trigger a TBD downstream action select starred_at, starred_by, full_name as repo_name from repos_starred where starred_by = "zzeleznick" order by starred_at desc ``` | starred_at | starred_by | repo_name | | --- | --- | --- | | 2020-02-11T01:08:59Z | zzeleznick | dogsheep/twitter-to-sqlite | | 2020-01-11T21:57:34Z | zzeleznick | simonw/datasette | 2. In my day job, I use [airflow](https://github.com/apache/airflow), and that's the mental model I'm bringing to [datasette](https://github.com/simonw/datasette). 3. I see your project like [twitter-to-sqlite](https://github.com/dogsheep/twitter-to-sqlite) akin to [Operators](https://airflow.apache.org/docs/stable/_api/index.html#pythonapi-operators) in Airflow world. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} [Feature Request] Support Repo Name in Search 🥺 611284481  
860895838 https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-860895838 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64 MDEyOklzc3VlQ29tbWVudDg2MDg5NTgzOA== khimaros 231498 2021-06-14T18:23:21Z 2021-06-14T21:37:35Z NONE i have a basic working version at https://github.com/khimaros/github-to-sqlite this can be tested with `github-to-sqlite events.db khimaros/events` caveat: the GitHub API doesn't seem to provide a complete history of events. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} feature: support "events" 920636216  
861035862 https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861035862 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64 MDEyOklzc3VlQ29tbWVudDg2MTAzNTg2Mg== khimaros 231498 2021-06-14T22:29:20Z 2021-06-14T22:29:20Z NONE it looks like the v4 GraphQL API is the only way to get data beyond 90 days from GitHub. this is significant change, but may be worth considering in the future. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} feature: support "events" 920636216  
861087651 https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861087651 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64 MDEyOklzc3VlQ29tbWVudDg2MTA4NzY1MQ== khimaros 231498 2021-06-15T00:48:37Z 2021-06-15T00:48:37Z NONE @simonw -- i've created an omega-query that fetched most of what was interesting to me for a single user. found by poking around in the "Explorer" tab in https://docs.github.com/en/graphql/overview/explorer note: pagination is still required via `first` and `last` but it seems to allow unlimited history. ``` query MyQuery { __typename user(login: "<user>") { id pinnedItems(first: 100) { edges { node } } pullRequests(first: 100) { nodes { body title state createdAt } } createdAt issues(first: 100) { pageInfo { endCursor startCursor } nodes { title url createdAt body } } issueComments(first: 100) { edges { node { id updatedAt url body } } } repositories(first: 100) { nodes { createdAt description parent { name } pinnedIssues(first: 100) { edges { node { id } } } pinnedDiscussions(first: 100) { edges { node { id } } } } } starredRepositories(first: 100) { edges { node { id } } } } } ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} feature: support "events" 920636216  
885964242 https://github.com/dogsheep/github-to-sqlite/pull/65#issuecomment-885964242 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/65 IC_kwDODFdgUs40zr3S khimaros 231498 2021-07-23T23:45:35Z 2021-07-23T23:45:35Z NONE @simonw is this PR of interest to you? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} basic support for events 923270900  
929651819 https://github.com/dogsheep/github-to-sqlite/pull/66#issuecomment-929651819 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/66 IC_kwDODFdgUs43aVxr sarcasticadmin 30531572 2021-09-28T21:50:31Z 2021-09-28T21:50:31Z NONE @simonw any feedback/thoughts? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add --merged-by flag to pull-requests sub command 975161924  
1073152522 https://github.com/dogsheep/google-takeout-to-sqlite/issues/10#issuecomment-1073152522 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/10 IC_kwDODFE5qs4_9wIK csusanu 9290214 2022-03-20T02:38:07Z 2022-03-20T02:38:07Z NONE [This line](https://github.com/dogsheep/google-takeout-to-sqlite/blob/e54e544427f1cc3ea8189f0e95f54046301a8645/google_takeout_to_sqlite/utils.py) needs to say `"MyActivity.json"` instead of `"My Activity.json"`. Google must have changed the file name. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} sqlite3.OperationalError: no such table: main.my_activity 1123393829  
747130908 https://github.com/dogsheep/google-takeout-to-sqlite/issues/2#issuecomment-747130908 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/2 MDEyOklzc3VlQ29tbWVudDc0NzEzMDkwOA== khimaros 231498 2020-12-17T00:47:04Z 2020-12-17T00:47:43Z NONE it looks like almost all of the memory consumption is coming from `json.load()`. another direction here may be to use the new "Semantic Location History" data which is already broken down by year and month. it also provides much more interesting data, such as estimated address, form of travel, etc. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} killed by oomkiller on large location-history 769376447  
780817596 https://github.com/dogsheep/google-takeout-to-sqlite/issues/4#issuecomment-780817596 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4 MDEyOklzc3VlQ29tbWVudDc4MDgxNzU5Ng== UtahDave 306240 2021-02-17T20:01:35Z 2021-02-17T20:01:35Z NONE I've got this almost working. Just needs some polish {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Feature Request: Gmail 778380836  
781451701 https://github.com/dogsheep/google-takeout-to-sqlite/issues/4#issuecomment-781451701 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4 MDEyOklzc3VlQ29tbWVudDc4MTQ1MTcwMQ== Btibert3 203343 2021-02-18T16:06:21Z 2021-02-18T16:06:21Z NONE Awesome! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Feature Request: Gmail 778380836  
783688547 https://github.com/dogsheep/google-takeout-to-sqlite/issues/4#issuecomment-783688547 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4 MDEyOklzc3VlQ29tbWVudDc4MzY4ODU0Nw== UtahDave 306240 2021-02-22T21:31:28Z 2021-02-22T21:31:28Z NONE @Btibert3 I've opened a PR with my initial attempt at this. Would you be willing to give this a try? https://github.com/dogsheep/google-takeout-to-sqlite/pull/5 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Feature Request: Gmail 778380836  
790198930 https://github.com/dogsheep/google-takeout-to-sqlite/issues/4#issuecomment-790198930 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4 MDEyOklzc3VlQ29tbWVudDc5MDE5ODkzMA== Btibert3 203343 2021-03-04T00:58:40Z 2021-03-04T00:58:40Z NONE I am just seeing this sorry, yes! I will kick the tires later on tonight. My apologies for the delay. {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Feature Request: Gmail 778380836  
790934616 https://github.com/dogsheep/google-takeout-to-sqlite/issues/4#issuecomment-790934616 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4 MDEyOklzc3VlQ29tbWVudDc5MDkzNDYxNg== Btibert3 203343 2021-03-04T20:54:44Z 2021-03-04T20:54:44Z NONE Sorry for the delay, I got sidetracked after class last night. I am getting the following error: ``` /content# google-takeout-to-sqlite mbox takeout.db Takeout/Mail/gmail.mbox Usage: google-takeout-to-sqlite [OPTIONS] COMMAND [ARGS]...Try 'google-takeout-to-sqlite --help' for help. Error: No such command 'mbox'. ``` On the box, I installed with pip after cloning: https://github.com/UtahDave/google-takeout-to-sqlite.git {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Feature Request: Gmail 778380836  
783794520 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-783794520 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 MDEyOklzc3VlQ29tbWVudDc4Mzc5NDUyMA== UtahDave 306240 2021-02-23T01:13:54Z 2021-02-23T01:13:54Z NONE Also, @simonw I created a test based off the existing tests. I think it's working correctly {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
784638394 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-784638394 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 MDEyOklzc3VlQ29tbWVudDc4NDYzODM5NA== UtahDave 306240 2021-02-24T00:36:18Z 2021-02-24T00:36:18Z NONE I noticed that @simonw is using black for formatting. I ran black on my additions in this PR. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
790389335 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790389335 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 MDEyOklzc3VlQ29tbWVudDc5MDM4OTMzNQ== UtahDave 306240 2021-03-04T07:32:04Z 2021-03-04T07:32:04Z NONE > The command takes quite a while to start running, presumably because this line causes it to have to scan the WHOLE file in order to generate a count: > > https://github.com/dogsheep/google-takeout-to-sqlite/blob/a3de045eba0fae4b309da21aa3119102b0efc576/google_takeout_to_sqlite/utils.py#L66-L67 > > I'm fine with waiting though. It's not like this is a command people run every day - and without that count we can't show a progress bar, which seems pretty important for a process that takes this long. The wait is from python loading the mbox file. This happens regardless if you're getting the length of the mbox. The mbox module is on the slow side. It is possible to do one's own parsing of the mbox, but I kind of wanted to avoid doing that. {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
790391711 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790391711 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 MDEyOklzc3VlQ29tbWVudDc5MDM5MTcxMQ== UtahDave 306240 2021-03-04T07:36:24Z 2021-03-04T07:36:24Z NONE > Looks like you're doing this: > > ```python > elif message.get_content_type() == "text/plain": > body = message.get_payload(decode=True) > ``` > > So presumably that decodes to a unicode string? > > I imagine the reason the column is a `BLOB` for me is that `sqlite-utils` determines the column type based on the first batch of items - https://github.com/simonw/sqlite-utils/blob/09c3386f55f766b135b6a1c00295646c4ae29bec/sqlite_utils/db.py#L1927-L1928 - and I got unlucky and had something in my first batch that wasn't a unicode string. Ah, that's good to know. I think explicitly creating the tables will be a great improvement. I'll add that. Also, I noticed after I opened this PR that the `message.get_payload()` is being deprecated in favor of `message.get_content()` or something like that. I'll see if that handles the decoding better, too. Thanks for the feedback. I should have time tomorrow to put together some improvements. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
791089881 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-791089881 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 MDEyOklzc3VlQ29tbWVudDc5MTA4OTg4MQ== maxhawkins 28565 2021-03-05T02:03:19Z 2021-03-05T02:03:19Z NONE I just tried to run this on a small VPS instance with 2GB of memory and it crashed out of memory while processing a 12GB mbox from Takeout. Is it possible to stream the emails to sqlite instead of loading it all into memory and upserting at once? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
791530093 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-791530093 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 MDEyOklzc3VlQ29tbWVudDc5MTUzMDA5Mw== UtahDave 306240 2021-03-05T16:28:07Z 2021-03-05T16:28:07Z NONE > I just tried to run this on a small VPS instance with 2GB of memory and it crashed out of memory while processing a 12GB mbox from Takeout. > > Is it possible to stream the emails to sqlite instead of loading it all into memory and upserting at once? @maxhawkins a limitation of the python mbox module is it loads the entire mbox into memory. I did find another approach to this problem that didn't use the builtin python mbox module and created a generator so that it didn't have to load the whole mbox into memory. I was hoping to use standard library modules, but this might be a good reason to investigate that approach a bit more. My worry is making sure a custom processor handles all the ins and outs of the mbox format correctly. Hm. As I'm writing this, I thought of something. I think I can parse each message one at a time, and then use an mbox function to load each message using the python mbox module. That way the mbox module can still deal with the specifics of the mbox format, but I can use a generator. I'll give that a try. Thanks for the feedback @maxhawkins and @simonw. I'll give that a try. @simonw can we hold off on merging this until I can test this new approach? {"total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
849708617 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-849708617 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 MDEyOklzc3VlQ29tbWVudDg0OTcwODYxNw== maxhawkins 28565 2021-05-27T15:01:42Z 2021-05-27T15:01:42Z NONE Any updates? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
884672647 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-884672647 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 IC_kwDODFE5qs40uwiH maxhawkins 28565 2021-07-22T05:56:31Z 2021-07-22T14:03:08Z NONE How does this commit look? https://github.com/maxhawkins/google-takeout-to-sqlite/commit/72802a83fee282eb5d02d388567731ba4301050d It seems that Takeout's mbox format is pretty simple, so we can get away with just splitting the file on lines begining with `From `. My commit just splits the file every time a line starts with `From ` and uses `email.message_from_bytes` to parse each chunk. I was able to load a 12GB takeout mbox without the program using more than a couple hundred MB of memory during the import process. It does make us lose the progress bar, but maybe I can add that back in a later commit. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
885022230 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-885022230 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 IC_kwDODFE5qs40wF4W maxhawkins 28565 2021-07-22T15:51:46Z 2021-07-22T15:51:46Z NONE One thing I noticed is this importer doesn't save attachments along with the body of the emails. It would be nice if those got stored as blobs in a separate attachments table so attachments can be included while fetching search results. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
885094284 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-885094284 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 IC_kwDODFE5qs40wXeM maxhawkins 28565 2021-07-22T17:41:32Z 2021-07-22T17:41:32Z NONE I added a follow-up commit that deals with emails that don't have a `Date` header: https://github.com/maxhawkins/google-takeout-to-sqlite/commit/4bc70103582c10802c85a523ef1e99a8a2154aa9 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
885098025 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-885098025 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 IC_kwDODFE5qs40wYYp UtahDave 306240 2021-07-22T17:47:50Z 2021-07-22T17:47:50Z NONE Hi @maxhawkins , I'm sorry, I haven't had any time to work on this. I'll have some time tomorrow to test your commits. I think they look great. I'm great with your commits superseding my initial attempt here. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
888075098 https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-888075098 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 IC_kwDODFE5qs407vNa maxhawkins 28565 2021-07-28T07:18:56Z 2021-07-28T07:18:56Z NONE > I'm not sure why but my most recent import, when displayed in Datasette, looks like this: > > <img alt="mbox__mbox_emails__753_446_rows" width="574" src="https://user-images.githubusercontent.com/9599/109985836-0ab00080-7cba-11eb-97d5-0631a0835b61.png"> I did some investigation into this issue and made a fix [here](https://github.com/dogsheep/google-takeout-to-sqlite/pull/8/commits/8ee555c2889a38ff42b95664ee074b4a01a82f06). The problem was that some messages (like gchat logs) don't have a `Message-Id` and we need to use `X-GM-THRID` as the pkey instead. @simonw While looking into this I found something unexpected about how sqlite_utils handles upserts if the pkey column is `None`. When the pkey is NULL I'd expect the function to either use rowid or throw an exception. Instead, it seems upsert_all creates a row where all columns are NULL instead of using the values provided as parameters. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} WIP: Add Gmail takeout mbox import 813880401  
894581223 https://github.com/dogsheep/google-takeout-to-sqlite/pull/8#issuecomment-894581223 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8 IC_kwDODFE5qs41Ujnn maxhawkins 28565 2021-08-07T00:57:48Z 2021-08-07T00:57:48Z NONE Just added two more fixes: * Added parsing for rfc 2047 encoded unicode headers * Body is now stored as TEXT rather than a BLOB regardless of what order the messages are parsed in. I was able to run this on my Takeout export and everything seems to work fine. @simonw let me know if this looks good to merge. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add Gmail takeout mbox import (v2) 954546309  
896378525 https://github.com/dogsheep/google-takeout-to-sqlite/pull/8#issuecomment-896378525 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8 IC_kwDODFE5qs41baad maxhawkins 28565 2021-08-10T23:28:45Z 2021-08-10T23:28:45Z NONE I added parsing of text/html emails using BeautifulSoup. Around half of the emails in my archive don't include a text/plain payload so adding html parsing makes a good chunk of them searchable. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add Gmail takeout mbox import (v2) 954546309  
1002735370 https://github.com/dogsheep/google-takeout-to-sqlite/pull/8#issuecomment-1002735370 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8 IC_kwDODFE5qs47xIcK Btibert3 203343 2021-12-29T18:58:23Z 2021-12-29T18:58:23Z NONE @maxhawkins how hard would it be to add an entry to the table that includes the HTML version of the email, if it exists? I just attempted your the PR branch on a very small mbox file, and it worked great. My use case is a research project and I need to access more than just the body plain text. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add Gmail takeout mbox import (v2) 954546309  
1003437288 https://github.com/dogsheep/google-takeout-to-sqlite/pull/8#issuecomment-1003437288 https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8 IC_kwDODFE5qs47zzzo maxhawkins 28565 2021-12-31T19:06:20Z 2021-12-31T19:06:20Z NONE > @maxhawkins how hard would it be to add an entry to the table that includes the HTML version of the email, if it exists? I just attempted your the PR branch on a very small mbox file, and it worked great. My use case is a research project and I need to access more than just the body plain text. Shouldn't be hard. The easiest way is probably to remove the `if body.content_type == "text/html"` clause from [utils.py:254](https://github.com/dogsheep/google-takeout-to-sqlite/pull/8/commits/8e6d487b697ce2e8ad885acf613a157bfba84c59#diff-25ad9dd1ced1b8bfc37fda8444819c803232c08891e4af3d4064aa205d8174eaR254) and just return content directly without parsing. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add Gmail takeout mbox import (v2) 954546309  
711083698 https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711083698 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11 MDEyOklzc3VlQ29tbWVudDcxMTA4MzY5OA== jarib 572 2020-10-17T21:39:15Z 2020-10-17T21:39:15Z NONE Nice! Works perfectly. Thanks for the quick response and great tooling in general. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} export.xml file name varies with different language settings 723838331  
877805513 https://github.com/dogsheep/healthkit-to-sqlite/issues/12#issuecomment-877805513 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/12 MDEyOklzc3VlQ29tbWVudDg3NzgwNTUxMw== Mjboothaus 956433 2021-07-11T14:03:01Z 2021-07-11T14:03:01Z NONE Hi Simon -- just experimenting with your excellent software! Up to this point in time I have been using the (paid) [HealthFit App](https://apps.apple.com/au/app/healthfit/id1202650514) to export my workouts from my Apple Watch, one walk at the time into either .GPX or .FIT format and then using another library to suck it into Python and eventually here to my "Emmaus Walking" app: https://share.streamlit.io/mjboothaus/emmaus_walking/emmaus_walking/app.py I just used `healthkit-to-sqlite` to convert my export.zip file and it all "just worked". I did notice the issue with various numeric fields being stored in the SQLite db as TEXT for now and just thought I'd flag it - but you're already self-reported this issue. Keep up the great work! I was curious if you have any thoughts about periodically exporting "export.zip" and how to just update the SQLite file instead of re-creating it each time. Hopefully Apple will give some thought to managing this data in a more sensible fashion as it grows over time. Ideally one could pull it from iCloud (where it is allegedly being backed up). {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Some workout columns should be float, not text 727848625  
877874117 https://github.com/dogsheep/healthkit-to-sqlite/issues/12#issuecomment-877874117 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/12 MDEyOklzc3VlQ29tbWVudDg3Nzg3NDExNw== Mjboothaus 956433 2021-07-11T23:03:37Z 2021-07-11T23:03:37Z NONE P.s. wondering if you have explored using the spatialite functionality with the location data in workouts? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Some workout columns should be float, not text 727848625  
904642396 https://github.com/dogsheep/healthkit-to-sqlite/pull/13#issuecomment-904642396 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/13 IC_kwDOC8tyDs41679c FabianHertwig 32016596 2021-08-24T13:27:40Z 2021-08-24T13:28:26Z NONE This would fix #21 and make #22 obsolete. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} SQLite does not have case sensitive columns 743071410  
798436026 https://github.com/dogsheep/healthkit-to-sqlite/issues/14#issuecomment-798436026 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/14 MDEyOklzc3VlQ29tbWVudDc5ODQzNjAyNg== n8henrie 1234956 2021-03-13T14:23:16Z 2021-03-13T14:23:16Z NONE This PR allows my import to succeed. It looks like some events don't have an `id`, but do have `HKExternalUUID` (which gets turned into `metadata_HKExternalUUID`), so I use this as a fallback. If a record has neither of these, I changed it to just print the record (for debugging) and `return`. For some odd reason this ran fine at first, and now (after removing the generated db and trying again) I'm getting a different error (duplicate column name). Looks like it may have run when I had two successive runs without remembering to delete the db in between. Will try to refactor. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} UNIQUE constraint failed: workouts.id 771608692  
798468572 https://github.com/dogsheep/healthkit-to-sqlite/issues/14#issuecomment-798468572 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/14 MDEyOklzc3VlQ29tbWVudDc5ODQ2ODU3Mg== n8henrie 1234956 2021-03-13T14:47:31Z 2021-03-13T14:47:31Z NONE Ok, new PR works. I'm not `git` enough so I just force-pushed over the old one. I still end up with a lot of activities that are missing an `id` and therefore skipped (since this is used as the primary key). For example: ``` {'workoutActivityType': 'HKWorkoutActivityTypeRunning', 'duration': '35.31666666666667', 'durationUnit': 'min', 'totalDistance': '4.010870267636999', 'totalDistanceUnit': 'mi', 'totalEnergyBurned': '660.3516235351562', 'totalEnergyBurnedUnit': 'Cal', 'sourceName': 'Strava', 'sourceVersion': '22810', 'creationDate': '2020-07-16 13:38:26 -0700', 'startDate': '2020-07-16 06:38:26 -0700', 'endDate': '2020-07-16 07:13:45 -0700'} ``` I also end up with some unhappy characters (in the skipped events), such as: `'sourceName': 'Nathan’s Apple\xa0Watch',`. But it's successfully making it through the file, and the resulting db opens in datasette, so I'd call that progress. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} UNIQUE constraint failed: workouts.id 771608692  
1073123231 https://github.com/dogsheep/healthkit-to-sqlite/issues/14#issuecomment-1073123231 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/14 IC_kwDOC8tyDs4_9o-f lchski 343884 2022-03-19T22:39:29Z 2022-03-19T22:39:29Z NONE I have this issue, too, with a fresh export. None of my `Workout` entries in `export.xml` have an `id` key, though [the sample `export.xml` in the tests folder doesn’t either](https://github.com/dogsheep/healthkit-to-sqlite/blob/main/tests/zip_contents/apple_health_export/export.xml#L14-L21), so I don’t think this is the culprit. Indeed, it seems @simonw is using the [`hash_id` function from `sqlite_utils`](https://sqlite-utils.datasette.io/en/stable/python-api.html#setting-an-id-based-on-the-hash-of-the-row-contents), which creates a column (`id`, in this case) based on a hash of the row’s contents. When I run the script, a `workouts` table is created, with one entry: my first workout. No `workout_points` table is created, as [I’d expect from `utils.py`](https://github.com/dogsheep/healthkit-to-sqlite/blob/main/healthkit_to_sqlite/utils.py#L89-L90). I then get essentially the same error as noted in this thread: ```Importing from HealthKit [###################################-] 98% 00:00:01 Traceback (most recent call last): File "/Users/lchski/.pyenv/versions/3.10.3/bin/healthkit-to-sqlite", line 8, in <module> sys.exit(cli()) File "/Users/lchski/.pyenv/versions/3.10.3/lib/python3.10/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/Users/lchski/.pyenv/versions/3.10.3/lib/python3.10/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/Users/lchski/.pyenv/versions/3.10.3/lib/python3.10/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/lchski/.pyenv/versions/3.10.3/lib/python3.10/site-packages/click/core.py", line 754, in invoke return __callback(*args, **kwargs) File "/Users/lchski/.pyenv/versions/3.10.3/lib/python3.10/site-packages/healthkit_to_sqlite/cli.py", line 57, in cli convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf) File "/Users/lchski/.pyenv/versions/3.10.3/lib/python3.10/site-packages/health… {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} UNIQUE constraint failed: workouts.id 771608692  
1073139067 https://github.com/dogsheep/healthkit-to-sqlite/issues/14#issuecomment-1073139067 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/14 IC_kwDOC8tyDs4_9s17 lchski 343884 2022-03-20T00:54:18Z 2022-03-20T00:54:18Z NONE Update: this appears to be because of running the command twice without clearing the DB in between. Tries to insert a Workout that already exists, causing a collision on the (auto-generated) `id` column. Had a different error with a clean DB, likely due to the workout points format; will make a new issue for that. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} UNIQUE constraint failed: workouts.id 771608692  
903950096 https://github.com/dogsheep/healthkit-to-sqlite/issues/21#issuecomment-903950096 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/21 IC_kwDOC8tyDs414S8Q FabianHertwig 32016596 2021-08-23T17:00:59Z 2021-08-23T17:00:59Z NONE I think the issue is that I have records like these: ```xml <Record type="HKQuantityTypeIdentifierDietaryCholesterol" sourceName="MyFitnessPal" sourceVersion="35120" unit="mg" creationDate="2021-07-04 20:55:27 +0200" startDate="2021-07-04 20:55:00 +0200" endDate="2021-07-04 20:55:00 +0200" value="124"> <MetadataEntry key="meal" value="Dinner"/> <MetadataEntry key="Meal" value="Dinner"/> </Record> ``` And if sqlite is case insensitive, then `metadata_meal` and `metadata_Meal` result in the same column. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Duplicate Column 977128935  
904641261 https://github.com/dogsheep/healthkit-to-sqlite/pull/22#issuecomment-904641261 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/22 IC_kwDOC8tyDs4167rt FabianHertwig 32016596 2021-08-24T13:26:20Z 2021-08-24T13:26:20Z NONE Did not see that #13 fixes the same issue in a similar way. You can decide which one to merge ;) {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Make sure that case-insensitive column names are unique 978086284  
514745798 https://github.com/dogsheep/healthkit-to-sqlite/issues/9#issuecomment-514745798 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/9 MDEyOklzc3VlQ29tbWVudDUxNDc0NTc5OA== tholo 166463 2019-07-24T18:25:36Z 2019-07-24T18:25:36Z NONE This is on macOS 10.14.6, with Python 3.7.4, packages in the virtual environment: ``` Package Version ------------------- ------- aiofiles 0.4.0 Click 7.0 click-default-group 1.2.1 datasette 0.29.2 h11 0.8.1 healthkit-to-sqlite 0.3.1 httptools 0.0.13 hupper 1.8.1 importlib-metadata 0.18 Jinja2 2.10.1 MarkupSafe 1.1.1 Pint 0.8.1 pip 19.2.1 pluggy 0.12.0 setuptools 41.0.1 sqlite-utils 1.7 tabulate 0.8.3 uvicorn 0.8.4 uvloop 0.12.2 websockets 7.0 zipp 0.5.2 ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Too many SQL variables 472429048  
515370687 https://github.com/dogsheep/healthkit-to-sqlite/issues/9#issuecomment-515370687 https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/9 MDEyOklzc3VlQ29tbWVudDUxNTM3MDY4Nw== tholo 166463 2019-07-26T09:01:19Z 2019-07-26T09:01:19Z NONE Yes, that did fix the issue I was seeing — it will now import my complete HealthKit data. Thorsten > On Jul 25, 2019, at 23:07, Simon Willison <notifications@github.com> wrote: > > @tholo <https://github.com/tholo> this should be fixed in just-released version 0.3.2 - could you run a pip install -U healthkit-to-sqlite and let me know if it works for you now? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub <https://github.com/dogsheep/healthkit-to-sqlite/issues/9?email_source=notifications&email_token=AABIUPYTWYOYSLEAFS7TLMLQBKIBBA5CNFSM4IGSXNNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD23TDNQ#issuecomment-515322294>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABIUP7DCBQ37SESQL7D4WTQBKIBBANCNFSM4IGSXNNA>. > {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Too many SQL variables 472429048  
774726123 https://github.com/dogsheep/pocket-to-sqlite/issues/9#issuecomment-774726123 https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/9 MDEyOklzc3VlQ29tbWVudDc3NDcyNjEyMw== jfeiwell 12669260 2021-02-07T18:21:08Z 2021-02-07T18:21:08Z NONE @simonw any ideas here? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} SSL Error 801780625  
774730656 https://github.com/dogsheep/pocket-to-sqlite/issues/9#issuecomment-774730656 https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/9 MDEyOklzc3VlQ29tbWVudDc3NDczMDY1Ng== merwok 635179 2021-02-07T18:45:04Z 2021-02-07T18:45:04Z NONE That URL uses TLS 1.3, but maybe only if the client supports it. It could be your Python version or your SSL library that’s not recent enough. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} SSL Error 801780625  
941274088 https://github.com/dogsheep/swarm-to-sqlite/issues/12#issuecomment-941274088 https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/12 IC_kwDODD6af844GrPo fs111 33631 2021-10-12T18:31:57Z 2021-10-12T18:31:57Z NONE I am running into the same problem. Is there any workaround? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 403 when getting token 951817328  
645515103 https://github.com/dogsheep/twitter-to-sqlite/issues/47#issuecomment-645515103 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/47 MDEyOklzc3VlQ29tbWVudDY0NTUxNTEwMw== hpk42 73579 2020-06-17T17:30:01Z 2020-06-17T17:30:01Z NONE It's the one with python3.7:: >>> sqlite3.sqlite_version '3.11.0' On Wed, Jun 17, 2020 at 10:24 -0700, Simon Willison wrote: > That means your version of SQLite is old enough that it doesn't support the FTS5 extension. > > Could you share what operating system you're running, and what the output is that you get from running this? > > python -c 'import sqlite3; print(sqlite3.connect(":memory:").execute("select sqlite_version()").fetchone()[0])' > > I can teach this tool to fall back on FTS4 if FTS5 isn't available. > > -- > You are receiving this because you authored the thread. > Reply to this email directly or view it on GitHub: > https://github.com/dogsheep/twitter-to-sqlite/issues/47#issuecomment-645512127 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Fall back to FTS4 if FTS5 is not available 639542974  
691501132 https://github.com/dogsheep/twitter-to-sqlite/issues/50#issuecomment-691501132 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/50 MDEyOklzc3VlQ29tbWVudDY5MTUwMTEzMg== bcongdon 706257 2020-09-12T14:48:10Z 2020-09-12T14:48:10Z NONE This seems to be an issue even with larger values of `--stop_after`: ``` $ twitter-to-sqlite favorites twitter.db --stop_after=2000 Importing favorites [####################################] 198 $ ``` {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} favorites --stop_after=N stops after min(N, 200) 698791218  
729484478 https://github.com/dogsheep/twitter-to-sqlite/issues/52#issuecomment-729484478 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/52 MDEyOklzc3VlQ29tbWVudDcyOTQ4NDQ3OA== fatihky 4169772 2020-11-18T07:12:45Z 2020-11-18T07:12:45Z NONE I'm so sorry that you already have `--since_id` option and that's enough for the case I've mentioned. Thank you for this excellent tool! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Discussion: Adding support for fetching only fresh tweets 745393298  
748436453 https://github.com/dogsheep/twitter-to-sqlite/issues/53#issuecomment-748436453 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/53 MDEyOklzc3VlQ29tbWVudDc0ODQzNjQ1Mw== anotherjesse 27 2020-12-19T07:47:01Z 2020-12-19T07:47:01Z NONE I think this should probably be closed as won't fix. Attempting to make a patch for this I realized that the since_id would limit to tweets posted since that since_id, not when it was favorited. So favoriting something in the older would be missed if you used `--since` with a cron script Better to just use `--stop_after` set to a couple hundred {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} --since support for favorites 771324837  
767888743 https://github.com/dogsheep/twitter-to-sqlite/issues/54#issuecomment-767888743 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54 MDEyOklzc3VlQ29tbWVudDc2Nzg4ODc0Mw== henry501 19328961 2021-01-26T23:07:41Z 2021-01-26T23:07:41Z NONE My import got much further with the applied fixes than 0.21.3, but not 100%. I do appear to have all of the tweets imported at least. Not sure when I'll have a chance to look further to try to fix or see what didn't make it into the import. Here's my output: ``` direct-messages-group: not yet implemented branch-links: not yet implemented periscope-expired-broadcasts: not yet implemented direct-messages: not yet implemented mute: not yet implemented periscope-comments-made-by-user: not yet implemented periscope-ban-information: not yet implemented periscope-profile-description: not yet implemented screen-name-change: not yet implemented manifest: not yet implemented fleet: not yet implemented user-link-clicks: not yet implemented periscope-broadcast-metadata: not yet implemented contact: not yet implemented fleet-mute: not yet implemented device-token: not yet implemented protected-history: not yet implemented direct-message-mute: not yet implemented Traceback (most recent call last): File "/Users/henry/.local/share/virtualenvs/python-sqlite-testing-mF3G2xKl/bin/twitter-to-sqlite", line 33, in <module> sys.exit(load_entry_point('twitter-to-sqlite==0.21.3', 'console_scripts', 'twitter-to-sqlite')()) File "/Users/henry/.local/share/virtualenvs/python-sqlite-testing-mF3G2xKl/lib/python3.9/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/Users/henry/.local/share/virtualenvs/python-sqlite-testing-mF3G2xKl/lib/python3.9/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/Users/henry/.local/share/virtualenvs/python-sqlite-testing-mF3G2xKl/lib/python3.9/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/henry/.local/share/virtualenvs/python-sqlite-testing-mF3G2xKl/lib/python3.9/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/henry/.local/share/virtualenvs/… {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Archive import appears to be broken on recent exports 779088071  
927312650 https://github.com/dogsheep/twitter-to-sqlite/issues/54#issuecomment-927312650 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54 IC_kwDODEm0Qs43RasK danp 2182 2021-09-26T14:09:51Z 2021-09-26T14:09:51Z NONE Similar trouble with ageinfo using 0.22. Here's what my ageinfo.js file looks like: ``` window.YTD.ageinfo.part0 = [ { "ageMeta" : { } } ] ``` Commenting out the registration for ageinfo in archive.py gets my archive to import. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Archive import appears to be broken on recent exports 779088071  
769973212 https://github.com/dogsheep/twitter-to-sqlite/issues/56#issuecomment-769973212 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/56 MDEyOklzc3VlQ29tbWVudDc2OTk3MzIxMg== gsajko 42315895 2021-01-29T18:29:02Z 2021-01-29T18:31:55Z NONE I think it was with `twitter-to-sqlite home-timeline home.db -a auth.json --since` and Im using only this command to grab tweets from cron tab `2,7,12,17,22,27,32,37,42,47,52,57 * * * * run-one /home/gsajko/miniconda3/bin/twitter-to-sqlite home-timeline /home/gsajko/work/custom_twitter_feed/home.db -a /home/gsajko/work/custom_twitter_feed/auth/auth.json --since` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Not all quoted statuses get fetched? 796736607  
772408273 https://github.com/dogsheep/twitter-to-sqlite/issues/56#issuecomment-772408273 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/56 MDEyOklzc3VlQ29tbWVudDc3MjQwODI3Mw== gsajko 42315895 2021-02-03T10:36:36Z 2021-02-03T10:36:36Z NONE I figured it out. Those tweets are in database, because somebody quote tweeted them, or retweeted them. And if you grab quoted tweet or reweeted tweet from other tweet json, It doesn't grab all of the details. So if someone quote tweeted a quote tweet, the second quote tweet won't have `quoted_status`. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Not all quoted statuses get fetched? 796736607  
860063190 https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-860063190 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57 MDEyOklzc3VlQ29tbWVudDg2MDA2MzE5MA== stiles 232237 2021-06-12T14:46:44Z 2021-06-12T14:46:44Z NONE I'm having the same issue (same versions of python and twitter-to-sqlite). It's the `user-timeline` command. Other commands are working. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Error: Use either --since or --since_id, not both 907645813  
1001222213 https://github.com/dogsheep/twitter-to-sqlite/issues/62#issuecomment-1001222213 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/62 IC_kwDODEm0Qs47rXBF sw-yx 6764957 2021-12-26T17:59:25Z 2021-12-26T17:59:25Z NONE just confirmed that this error does not occur when i use my public main account. gets more interesting! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} KeyError: 'created_at' for private accounts? 1088816961  
1049775451 https://github.com/dogsheep/twitter-to-sqlite/issues/62#issuecomment-1049775451 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/62 IC_kwDODEm0Qs4-kk1b miuku 43036882 2022-02-24T11:43:31Z 2022-02-24T11:43:31Z NONE i seem to have fixed this issue by applying for [elevated API access](https://developer.twitter.com/en/portal/products/elevated) {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} KeyError: 'created_at' for private accounts? 1088816961  
1050123919 https://github.com/dogsheep/twitter-to-sqlite/issues/62#issuecomment-1050123919 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/62 IC_kwDODEm0Qs4-l56P sw-yx 6764957 2022-02-24T18:10:18Z 2022-02-24T18:10:18Z NONE gonna close this for now since i'm not actively working on it. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} KeyError: 'created_at' for private accounts? 1088816961  
344864254 https://github.com/simonw/datasette/issues/100#issuecomment-344864254 https://api.github.com/repos/simonw/datasette/issues/100 MDEyOklzc3VlQ29tbWVudDM0NDg2NDI1NA== coisnepe 13304454 2017-11-16T09:25:10Z 2017-11-16T09:25:10Z NONE @simonw I see. I upgraded sanic-jinja2 and jinja2: it now works flawlessly. Thank you! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} TemplateAssertionError: no filter named 'tojson' 274160723  
705890365 https://github.com/simonw/datasette/pull/1000#issuecomment-705890365 https://api.github.com/repos/simonw/datasette/issues/1000 MDEyOklzc3VlQ29tbWVudDcwNTg5MDM2NQ== codecov[bot] 22429695 2020-10-09T00:03:29Z 2020-10-09T16:07:03Z NONE # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1000?src=pr&el=h1) Report > Merging [#1000](https://codecov.io/gh/simonw/datasette/pull/1000?src=pr&el=desc) into [main](https://codecov.io/gh/simonw/datasette/commit/7249ac5ca04b5ddc6517750326ee7e522cc49145?el=desc) will **increase** coverage by `0.15%`. > The diff coverage is `100.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1000/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1000?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1000 +/- ## ========================================== + Coverage 84.37% 84.52% +0.15% ========================================== Files 28 28 Lines 3871 3878 +7 ========================================== + Hits 3266 3278 +12 + Misses 605 600 -5 ``` | [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1000?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1000/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `96.34% <100.00%> (+0.02%)` | :arrow_up: | | [datasette/cli.py](https://codecov.io/gh/simonw/datasette/pull/1000/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL2NsaS5weQ==) | `74.35% <100.00%> (ø)` | | | [datasette/utils/testing.py](https://codecov.io/gh/simonw/datasette/pull/1000/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3V0aWxzL3Rlc3RpbmcucHk=) | `95.16% <100.00%> (-4.84%)` | :arrow_down: | | [datasette/views/base.py](https://codecov.io/gh/simonw/datasette/pull/1000/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3ZpZXdzL2Jhc2UucHk=) | `93.94% <100.00%> (+0.11%)` | :arrow_up: | | [datasette/utils/asgi.py](https://codecov.io/gh/simonw/datasette/pull/1000/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3V0aWxzL2FzZ2kucHk=) | `91.92% <0.00%> (ø)` | | | [datasette/views/special.py](https://codecov.io/gh/simonw/datasette/pull/1000/diff?… {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} datasette.client internal requests mechanism 717746043  
706302863 https://github.com/simonw/datasette/issues/1003#issuecomment-706302863 https://api.github.com/repos/simonw/datasette/issues/1003 MDEyOklzc3VlQ29tbWVudDcwNjMwMjg2Mw== mhalle 649467 2020-10-09T17:17:06Z 2020-10-09T17:17:06Z NONE I agree on the descriptive and python-consistent naming. There is already a tojson, but frankly i find the "to" and "from" confusing in a text templating language where what's a string and what's data isn't 100% transparent. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} from_json jinja2 filter 718238967  
706383750 https://github.com/simonw/datasette/pull/1008#issuecomment-706383750 https://api.github.com/repos/simonw/datasette/issues/1008 MDEyOklzc3VlQ29tbWVudDcwNjM4Mzc1MA== codecov[bot] 22429695 2020-10-09T20:17:29Z 2020-10-09T20:17:29Z NONE # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1008?src=pr&el=h1) Report > Merging [#1008](https://codecov.io/gh/simonw/datasette/pull/1008?src=pr&el=desc) into [main](https://codecov.io/gh/simonw/datasette/commit/1bdbc8aa7f4fd7a768d456146e44da86cb1b36d1?el=desc) will **increase** coverage by `0.00%`. > The diff coverage is `100.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1008/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1008?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1008 +/- ## ======================================= Coverage 84.55% 84.56% ======================================= Files 28 28 Lines 3878 3880 +2 ======================================= + Hits 3279 3281 +2 Misses 599 599 ``` | [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1008?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1008/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `96.35% <100.00%> (+0.01%)` | :arrow_up: | ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1008?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1008?src=pr&el=footer). Last update [1bdbc8a...4085898](https://codecov.io/gh/simonw/datasette/pull/1008?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Add json_loads and json_dumps jinja2 filters 718395987  
344597274 https://github.com/simonw/datasette/issues/101#issuecomment-344597274 https://api.github.com/repos/simonw/datasette/issues/101 MDEyOklzc3VlQ29tbWVudDM0NDU5NzI3NA== eaubin 450244 2017-11-15T13:48:55Z 2017-11-15T13:48:55Z NONE This is a duplicate of https://github.com/simonw/datasette/issues/100 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} TemplateAssertionError: no filter named 'tojson' 274161964  
707123799 https://github.com/simonw/datasette/pull/1017#issuecomment-707123799 https://api.github.com/repos/simonw/datasette/issues/1017 MDEyOklzc3VlQ29tbWVudDcwNzEyMzc5OQ== codecov[bot] 22429695 2020-10-12T13:34:51Z 2020-10-12T13:34:51Z NONE # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1017?src=pr&el=h1) Report > Merging [#1017](https://codecov.io/gh/simonw/datasette/pull/1017?src=pr&el=desc) into [main](https://codecov.io/gh/simonw/datasette/commit/acf07a67722aa74828744726187690b59d342494?el=desc) will **not change** coverage. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1017/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1017?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1017 +/- ## ======================================= Coverage 84.56% 84.56% ======================================= Files 28 28 Lines 3882 3882 ======================================= Hits 3283 3283 Misses 599 599 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1017?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1017?src=pr&el=footer). Last update [acf07a6...f30d9da](https://codecov.io/gh/simonw/datasette/pull/1017?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Update janus requirement from <0.6,>=0.4 to >=0.4,<0.7 719381863  
707125737 https://github.com/simonw/datasette/pull/1018#issuecomment-707125737 https://api.github.com/repos/simonw/datasette/issues/1018 MDEyOklzc3VlQ29tbWVudDcwNzEyNTczNw== codecov[bot] 22429695 2020-10-12T13:38:35Z 2020-10-12T13:38:35Z NONE # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1018?src=pr&el=h1) Report > Merging [#1018](https://codecov.io/gh/simonw/datasette/pull/1018?src=pr&el=desc) into [main](https://codecov.io/gh/simonw/datasette/commit/acf07a67722aa74828744726187690b59d342494?el=desc) will **not change** coverage. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1018/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1018?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1018 +/- ## ======================================= Coverage 84.56% 84.56% ======================================= Files 28 28 Lines 3882 3882 ======================================= Hits 3283 3283 Misses 599 599 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1018?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1018?src=pr&el=footer). Last update [acf07a6...4b021be](https://codecov.io/gh/simonw/datasette/pull/1018?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Update asgiref requirement from ~=3.2.10 to >=3.2.10,<3.4.0 719382156  
708693231 https://github.com/simonw/datasette/pull/1022#issuecomment-708693231 https://api.github.com/repos/simonw/datasette/issues/1022 MDEyOklzc3VlQ29tbWVudDcwODY5MzIzMQ== codecov[bot] 22429695 2020-10-14T22:26:41Z 2020-10-14T22:26:41Z NONE # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1022?src=pr&el=h1) Report > Merging [#1022](https://codecov.io/gh/simonw/datasette/pull/1022?src=pr&el=desc) into [main](https://codecov.io/gh/simonw/datasette/commit/7f2edb5dd2074dce0090659021991695a984844b?el=desc) will **not change** coverage. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1022/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1022?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1022 +/- ## ======================================= Coverage 84.60% 84.60% ======================================= Files 28 28 Lines 3892 3892 ======================================= Hits 3293 3293 Misses 599 599 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1022?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1022?src=pr&el=footer). Last update [7f2edb5...7cef70a](https://codecov.io/gh/simonw/datasette/pull/1022?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Fix table name in spatialite example command 721830990  
711407607 https://github.com/simonw/datasette/pull/1030#issuecomment-711407607 https://api.github.com/repos/simonw/datasette/issues/1030 MDEyOklzc3VlQ29tbWVudDcxMTQwNzYwNw== codecov[bot] 22429695 2020-10-18T19:31:31Z 2020-10-19T08:01:51Z NONE # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1030?src=pr&el=h1) Report > Merging [#1030](https://codecov.io/gh/simonw/datasette/pull/1030?src=pr&el=desc) into [main](https://codecov.io/gh/simonw/datasette/commit/568bd7bbf590861687db8c318f3d8cfcd1dfb47a?el=desc) will **decrease** coverage by `0.10%`. > The diff coverage is `68.75%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1030/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1030?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1030 +/- ## ========================================== - Coverage 84.63% 84.53% -0.11% ========================================== Files 28 28 Lines 3892 3905 +13 ========================================== + Hits 3294 3301 +7 - Misses 598 604 +6 ``` | [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1030?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [datasette/utils/\_\_init\_\_.py](https://codecov.io/gh/simonw/datasette/pull/1030/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3V0aWxzL19faW5pdF9fLnB5) | `93.35% <68.75%> (-0.79%)` | :arrow_down: | | [datasette/views/index.py](https://codecov.io/gh/simonw/datasette/pull/1030/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3ZpZXdzL2luZGV4LnB5) | `96.49% <0.00%> (-1.76%)` | :arrow_down: | ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1030?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1030?src=pr&el=footer). Last update [568bd7b...e082533](https://codecov.io/gh/simonw/datasette/pull/1030?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Make `package` command deal with a configuration directory argument 723982480  
711792622 https://github.com/simonw/datasette/pull/1031#issuecomment-711792622 https://api.github.com/repos/simonw/datasette/issues/1031 MDEyOklzc3VlQ29tbWVudDcxMTc5MjYyMg== codecov[bot] 22429695 2020-10-19T07:57:17Z 2020-10-19T07:57:17Z NONE # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1031?src=pr&el=h1) Report > Merging [#1031](https://codecov.io/gh/simonw/datasette/pull/1031?src=pr&el=desc) into [main](https://codecov.io/gh/simonw/datasette/commit/568bd7bbf590861687db8c318f3d8cfcd1dfb47a?el=desc) will **decrease** coverage by `0.02%`. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1031/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1031?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1031 +/- ## ========================================== - Coverage 84.63% 84.60% -0.03% ========================================== Files 28 28 Lines 3892 3892 ========================================== - Hits 3294 3293 -1 - Misses 598 599 +1 ``` | [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1031?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [datasette/cli.py](https://codecov.io/gh/simonw/datasette/pull/1031/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL2NsaS5weQ==) | `74.22% <ø> (ø)` | | | [datasette/views/index.py](https://codecov.io/gh/simonw/datasette/pull/1031/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3ZpZXdzL2luZGV4LnB5) | `96.49% <0.00%> (-1.76%)` | :arrow_down: | ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1031?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1031?src=pr&el=footer). Last update [568bd7b...7e7eaa4](https://codecov.io/gh/simonw/datasette/pull/1031?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Fallback to databases in inspect-data.json when no -i options are passed 724369025  
714289680 https://github.com/simonw/datasette/pull/1031#issuecomment-714289680 https://api.github.com/repos/simonw/datasette/issues/1031 MDEyOklzc3VlQ29tbWVudDcxNDI4OTY4MA== frankier 299380 2020-10-22T07:23:52Z 2020-10-22T07:23:52Z NONE The bug is that currently when there are databases passed in, but no -i flag, e.g. in configuration directory mode, inclusion in inspect-data.json does not automatically cause databases to be considered immutable, as described in the documentation. The reason is that the -i flag is specified multiple=True, which means when it is not passed in we will get an empty list [], rather than None. So the current code decides that no databases are immutable rather than falling back to inspect-data.json -- as is presumably intended. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Fallback to databases in inspect-data.json when no -i options are passed 724369025  
744003454 https://github.com/simonw/datasette/pull/1031#issuecomment-744003454 https://api.github.com/repos/simonw/datasette/issues/1031 MDEyOklzc3VlQ29tbWVudDc0NDAwMzQ1NA== frankier 299380 2020-12-13T12:52:56Z 2020-12-13T12:52:56Z NONE Please let me know if there's anything I can do to help get this merged. This is causing problems for me because it means when I build my Docker image my databases aren't considered immutable, which I would like them to be so that a download link is produced. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Fallback to databases in inspect-data.json when no -i options are passed 724369025  
712397537 https://github.com/simonw/datasette/issues/1032#issuecomment-712397537 https://api.github.com/repos/simonw/datasette/issues/1032 MDEyOklzc3VlQ29tbWVudDcxMjM5NzUzNw== saulpw 236498 2020-10-19T19:37:55Z 2020-10-19T19:37:55Z NONE python-dateutil is awesome, but it can only guess at one date at a time. So if you have a column of dates that are (presumably) in the same format, it can't use the full set of dates to deduce the format. Also, once it has parsed a date, you can't get the format it used, whether to parse or render other dates. These limitations prevent it from being a silver bullet for date parsing, though they're not enough for me to stop using it! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bring date parsing into Datasette core 724878151  

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 229.614ms · About: simonw/datasette-graphql