html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app https://github.com/dogsheep/github-to-sqlite/issues/38#issuecomment-623038148,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/38,623038148,MDEyOklzc3VlQ29tbWVudDYyMzAzODE0OA==,5779832,2020-05-03T01:18:57Z,2020-05-03T01:18:57Z,NONE,"Thanks, @simonw! I feel a little foolish in hindsight, but I'm on the same page now and am glad to have discovered first-hand a motivation for this `repos_starred` use case.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",611284481, https://github.com/dogsheep/github-to-sqlite/issues/38#issuecomment-623038378,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/38,623038378,MDEyOklzc3VlQ29tbWVudDYyMzAzODM3OA==,9599,2020-05-03T01:21:13Z,2020-05-03T01:21:13Z,MEMBER,No this is really useful feedback! I'm so close to this project that I miss what's not obvious to people dropping in for the first time.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",611284481, https://github.com/dogsheep/github-to-sqlite/issues/38#issuecomment-623044643,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/38,623044643,MDEyOklzc3VlQ29tbWVudDYyMzA0NDY0Mw==,5779832,2020-05-03T02:34:32Z,2020-05-03T02:34:32Z,NONE,"1. More than glad to share feedback from the sidelines as a [starrer](https://github-to-sqlite.dogsheep.net/github?sql=select%0D%0A++starred_at%2C%0D%0A++starred_by%2C%0D%0A++full_name+as+repo_name%0D%0Afrom%0D%0A++repos_starred%0D%0Awhere%0D%0A++starred_by+%3D+%22zzeleznick%22%0D%0Aorder+by%0D%0A++starred_at+desc). ``` -- Motivation: -- Datasette is a data hammer and I'm looking for nails -- e.g. Find which repos a user has starred => trigger a TBD downstream action select starred_at, starred_by, full_name as repo_name from repos_starred where starred_by = ""zzeleznick"" order by starred_at desc ``` | starred_at | starred_by | repo_name | | --- | --- | --- | | 2020-02-11T01:08:59Z | zzeleznick | dogsheep/twitter-to-sqlite | | 2020-01-11T21:57:34Z | zzeleznick | simonw/datasette | 2. In my day job, I use [airflow](https://github.com/apache/airflow), and that's the mental model I'm bringing to [datasette](https://github.com/simonw/datasette). 3. I see your project like [twitter-to-sqlite](https://github.com/dogsheep/twitter-to-sqlite) akin to [Operators](https://airflow.apache.org/docs/stable/_api/index.html#pythonapi-operators) in Airflow world.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",611284481, https://github.com/dogsheep/github-to-sqlite/issues/39#issuecomment-675509550,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/39,675509550,MDEyOklzc3VlQ29tbWVudDY3NTUwOTU1MA==,9599,2020-08-18T14:23:56Z,2020-08-18T14:23:56Z,MEMBER,I think this is fixed: https://github-to-sqlite.dogsheep.net/github/issues?_facet=repo,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613777056, https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-531517138,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,531517138,MDEyOklzc3VlQ29tbWVudDUzMTUxNzEzOA==,9599,2019-09-14T21:59:59Z,2019-09-14T21:59:59Z,MEMBER,"Paginate through https://api.github.com/repos/simonw/datasette/stargazers Send `Accept: application/vnd.github.v3.star+json` to get the `starred_at` dates.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730, https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-550388354,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,550388354,MDEyOklzc3VlQ29tbWVudDU1MDM4ODM1NA==,9599,2019-11-06T16:26:55Z,2019-11-06T16:26:55Z,MEMBER,"Here's a query I figured out using a window function that shows cumulative stargazers over time: ```sql select yyyymmdd, sum(n) over ( order by yyyymmdd rows unbounded preceding ) as cumulative_count from ( select substr(starred_at, 0, 11) as yyyymmdd, count(*) as n from stars group by yyyymmdd ) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730, https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-622990947,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,622990947,MDEyOklzc3VlQ29tbWVudDYyMjk5MDk0Nw==,9599,2020-05-02T17:54:16Z,2020-05-02T17:54:16Z,MEMBER,"I could add that window function query as a view, but only if the detected version of SQLite supports window functions.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730, https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-623000814,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,623000814,MDEyOklzc3VlQ29tbWVudDYyMzAwMDgxNA==,9599,2020-05-02T19:15:23Z,2020-05-02T19:15:23Z,MEMBER,"I'm not going to do the `--sql` bit just yet. I have patterns for working around this for other commands which are working fine: https://github.com/dogsheep/github-to-sqlite/blob/d00a53061556dc403c166b443d141c4e1adbd64a/.github/workflows/deploy-demo.yml#L53-L70","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730, https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-623004836,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,623004836,MDEyOklzc3VlQ29tbWVudDYyMzAwNDgzNg==,9599,2020-05-02T19:49:51Z,2020-05-02T19:49:51Z,MEMBER,"Alternative pattern: ``` sqlite-utils releases.db 'select full_name from repos' --csv --no-headers \ | tr -d '\r' \ | xargs github-to-sqlite stargazers stars.db ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730, https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-623006154,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,623006154,MDEyOklzc3VlQ29tbWVudDYyMzAwNjE1NA==,9599,2020-05-02T20:01:39Z,2020-05-02T20:01:54Z,MEMBER,Needs tests and documentation. I shipped it early to check that the live demo works.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730, https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-623007441,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,623007441,MDEyOklzc3VlQ29tbWVudDYyMzAwNzQ0MQ==,9599,2020-05-02T20:13:37Z,2020-05-02T20:13:37Z,MEMBER,Datasette cumulative stars over time: https://github-to-sqlite.dogsheep.net/github?sql=select%0D%0A++yyyymmdd%2C%0D%0A++sum%28n%29+over+%28%0D%0A++++order+by%0D%0A++++++yyyymmdd+rows+unbounded+preceding%0D%0A++%29+as+cumulative_count%0D%0Afrom%0D%0A++%28%0D%0A++++select%0D%0A++++++substr%28starred_at%2C+0%2C+11%29+as+yyyymmdd%2C%0D%0A++++++count%28*%29+as+n%0D%0A++++from%0D%0A++++++stars%0D%0A++++where+repo+%3D+107914493%0D%0A++++group+by%0D%0A++++++yyyymmdd%0D%0A++%29,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730, https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-623010272,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,623010272,MDEyOklzc3VlQ29tbWVudDYyMzAxMDI3Mg==,9599,2020-05-02T20:39:14Z,2020-05-02T20:39:14Z,MEMBER,"Graph of cumulative stars for Datasette over time: https://github-to-sqlite.dogsheep.net/github?sql=select%0D%0A++yyyymmdd%2C%0D%0A++sum%28n%29+over+%28%0D%0A++++order+by%0D%0A++++++yyyymmdd+rows+unbounded+preceding%0D%0A++%29+as+cumulative_count%0D%0Afrom%0D%0A++%28%0D%0A++++select%0D%0A++++++substr%28starred_at%2C+0%2C+11%29+as+yyyymmdd%2C%0D%0A++++++count%28*%29+as+n%0D%0A++++from%0D%0A++++++stars%0D%0A++++where+repo+%3D+107914493%0D%0A++++group+by%0D%0A++++++yyyymmdd%0D%0A++%29#g.mark=line&g.x_column=yyyymmdd&g.x_type=temporal&g.y_column=cumulative_count&g.y_type=quantitative Stars per day (as a label bar chart, so very wide): https://github-to-sqlite.dogsheep.net/github?sql=%0D%0A++++select%0D%0A++++++substr%28starred_at%2C+0%2C+11%29+as+yyyymmdd%2C%0D%0A++++++count%28*%29+as+n%0D%0A++++from%0D%0A++++++stars%0D%0A++++where+repo+%3D+107914493%0D%0A++++group+by%0D%0A++++++yyyymmdd%0D%0A++#g.mark=bar&g.x_column=yyyymmdd&g.x_type=ordinal&g.y_column=n&g.y_type=quantitative ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730, https://github.com/dogsheep/github-to-sqlite/issues/40#issuecomment-643393506,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/40,643393506,MDEyOklzc3VlQ29tbWVudDY0MzM5MzUwNg==,9599,2020-06-12T17:21:14Z,2020-06-12T17:21:14Z,MEMBER,"I only install SQLite for this: https://github.com/dogsheep/github-to-sqlite/blob/c0d54e0260468be38152293df5abd775c068495d/.github/workflows/deploy-demo.yml#L77-L78 I'm going to remove the need to install sqlite3 by making this possible with sqlite-utils: https://github.com/simonw/sqlite-utils/issues/115","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637899539, https://github.com/dogsheep/github-to-sqlite/issues/40#issuecomment-643414646,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/40,643414646,MDEyOklzc3VlQ29tbWVudDY0MzQxNDY0Ng==,9599,2020-06-12T18:06:48Z,2020-06-12T18:06:48Z,MEMBER,That fixed it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637899539, https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653947916,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653947916,MDEyOklzc3VlQ29tbWVudDY1Mzk0NzkxNg==,9599,2020-07-05T22:40:47Z,2020-07-05T22:40:47Z,MEMBER,Might be that it's not got enough RAM. I'll try deploying to a larger instance.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727, https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653960989,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653960989,MDEyOklzc3VlQ29tbWVudDY1Mzk2MDk4OQ==,9599,2020-07-06T00:32:34Z,2020-07-06T00:32:34Z,MEMBER,Same error.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727, https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962418,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653962418,MDEyOklzc3VlQ29tbWVudDY1Mzk2MjQxOA==,9599,2020-07-06T00:41:38Z,2020-07-06T00:41:38Z,MEMBER,"https://console.cloud.google.com/run/detail/us-central1/github-to-sqlite/logs?project=datasette-222320 has some clues. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727, https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962530,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653962530,MDEyOklzc3VlQ29tbWVudDY1Mzk2MjUzMA==,9599,2020-07-06T00:42:13Z,2020-07-06T00:42:13Z,MEMBER,So it looks like it's the ASGI lifespan change I made in https://github.com/simonw/datasette/commit/16f592247a2a0e140ada487e9972645406dcae69 - It must be incompatible with one of the plugins.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727, https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962669,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653962669,MDEyOklzc3VlQ29tbWVudDY1Mzk2MjY2OQ==,9599,2020-07-06T00:42:57Z,2020-07-06T00:42:57Z,MEMBER,"https://github-to-sqlite.dogsheep.net/-/plugins ```json [ { ""name"": ""datasette-json-html"", ""static"": false, ""templates"": false, ""version"": ""0.6"", ""hooks"": [ ""prepare_connection"", ""render_cell"" ] }, { ""name"": ""datasette-render-markdown"", ""static"": false, ""templates"": false, ""version"": ""1.1.2"", ""hooks"": [ ""extra_template_vars"", ""render_cell"" ] }, { ""name"": ""datasette-pretty-json"", ""static"": false, ""templates"": false, ""version"": ""0.2"", ""hooks"": [ ""render_cell"" ] }, { ""name"": ""datasette-search-all"", ""static"": false, ""templates"": true, ""version"": ""0.2.1"", ""hooks"": [ ""asgi_wrapper"", ""extra_template_vars"" ] }, { ""name"": ""datasette-vega"", ""static"": true, ""templates"": false, ""version"": ""0.6.2"", ""hooks"": [ ""extra_css_urls"", ""extra_js_urls"" ] } ] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727, https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962708,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653962708,MDEyOklzc3VlQ29tbWVudDY1Mzk2MjcwOA==,9599,2020-07-06T00:43:10Z,2020-07-06T00:43:10Z,MEMBER,I bet it's datasette-search-all.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727, https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653966670,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653966670,MDEyOklzc3VlQ29tbWVudDY1Mzk2NjY3MA==,9599,2020-07-06T01:07:02Z,2020-07-06T01:07:02Z,MEMBER,OK that fix worked.https://github.com/dogsheep/github-to-sqlite/runs/839764768?check_suite_focus=true,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727, https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660536265,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43,660536265,MDEyOklzc3VlQ29tbWVudDY2MDUzNjI2NQ==,9599,2020-07-18T20:15:12Z,2020-07-18T20:15:12Z,MEMBER,"I want to create a SQL query which shows me all of my repositories that have commits that are NOT in the most recent release. The releases table doesn't have enough information for this because it doesn't tell you the commit hash associated with each release, just the tag: https://github-to-sqlite.dogsheep.net/github/releases","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660355904, https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660547502,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43,660547502,MDEyOklzc3VlQ29tbWVudDY2MDU0NzUwMg==,9599,2020-07-18T21:50:37Z,2020-07-18T21:50:37Z,MEMBER,"``` $ github-to-sqlite tags tags.db simonw/datasette dogsheep/github-to-sqlite $ sqlite-utils tables tags.db --counts [{""table"": ""users"", ""count"": 2}, {""table"": ""licenses"", ""count"": 1}, {""table"": ""repos"", ""count"": 2}, {""table"": ""tags"", ""count"": 76}, {""table"": ""licenses_fts"", ""count"": 1}, {""table"": ""licenses_fts_data"", ""count"": 3}, {""table"": ""licenses_fts_idx"", ""count"": 1}, {""table"": ""licenses_fts_docsize"", ""count"": 1}, {""table"": ""licenses_fts_config"", ""count"": 1}, {""table"": ""repos_fts"", ""count"": 2}, {""table"": ""repos_fts_data"", ""count"": 3}, {""table"": ""repos_fts_idx"", ""count"": 1}, {""table"": ""repos_fts_docsize"", ""count"": 2}, {""table"": ""repos_fts_config"", ""count"": 1}, {""table"": ""users_fts"", ""count"": 2}, {""table"": ""users_fts_data"", ""count"": 3}, {""table"": ""users_fts_idx"", ""count"": 1}, {""table"": ""users_fts_docsize"", ""count"": 2}, {""table"": ""users_fts_config"", ""count"": 1}] $ sqlite-utils rows tags.db tags [{""repo_id"": 107914493, ""name"": ""0.45"", ""sha"": ""f1f581b7ffcd5d8f3ae6c1c654d813a6641410eb""}, {""repo_id"": 107914493, ""name"": ""0.45a5"", ""sha"": ""676bb64c877d73f8ff496cef4632f5a8a5a9283c""}, {""repo_id"": 107914493, ""name"": ""0.45a4"", ""sha"": ""265483173bc8341dc02c8b782b9b59d2ce8bbedb""}, {""repo_id"": 107914493, ""name"": ""0.45a3"", ""sha"": ""1f55a4a2b68fa65e56a28baeb7f44122fdeca7e7""}, {""repo_id"": 107914493, ""name"": ""0.45a2"", ""sha"": ""1a5b7d318fa923edfcefd3df8f64dae2e9c49d3f""}, {""repo_id"": 107914493, ""name"": ""0.45a1"", ""sha"": ""b59b92b1b0517cf18fa748ff9d0a0bf86298dd43""}, {""repo_id"": 107914493, ""name"": ""0.45a0"", ""sha"": ""dda932d818b34ccab11730a76554f0a3748d8348""}, {""repo_id"": 107914493, ""name"": ""0.44"", ""sha"": ""b906030235efbdff536405d66078f4868ce0d3bd""}, {""repo_id"": 107914493, ""name"": ""0.43"", ""sha"": ""d56f402822df102f9cf1a9a056449d01a15e3aae""}, {""repo_id"": 107914493, ""name"": ""0.42"", ""sha"": ""af6c6c5d6f929f951c0e63bfd1c82e37a071b50f""}, {""repo_id"": 107914493, ""name"": ""0.41"", ""sha"": ""182e5c8745c94576718315f7596ccc81e5e2417b""}, {""repo_id"": 107914493, ""name"": ""0.40"", ""sha"": ""8da108193b08abf140716f8ac499f32309dfe9cf""}, {""repo_id"": 107914493, ""name"": ""0.39"", ""sha"": ""dedd775512daee49925882654f252df61a9e3b6d""}, {""repo_id"": 107914493, ""name"": ""0.38"", ""sha"": ""7e357abbc38dcc9d19a2f1df3252668a48e941e4""}, {""repo_id"": 107914493, ""name"": ""0.37.1"", ""sha"": ""be20e6991eac2baa9b43e9b26ae209bae805ede5""}, {""repo_id"": 107914493, ""name"": ""0.37"", ""sha"": ""c9e6841482b299fceadc5ad548c2dbf58a8f1227""}, {""repo_id"": 107914493, ""name"": ""0.36"", ""sha"": ""b031fe97636b80b05fec409ee1dffb7d044fd4e9""}, {""repo_id"": 107914493, ""name"": ""0.35"", ""sha"": ""30b6f71b306a43605c99bef79302ed5cb22d1924""}, {""repo_id"": 107914493, ""name"": ""0.34"", ""sha"": ""e7f60d2a9b59752e20de8412f7b0a3e9a5359a31""}, {""repo_id"": 107914493, ""name"": ""0.33"", ""sha"": ""59e7014c8a0f4102d7dc79f517540c55c49e1554""}, {""repo_id"": 107914493, ""name"": ""0.32"", ""sha"": ""a95bedb9c423fa6d772c93ef47bc40f13a5bea50""}, {""repo_id"": 107914493, ""name"": ""0.31.2"", ""sha"": ""b51f258d00bb3c3b401f15d46a1fbd50394dbe1c""}, {""repo_id"": 107914493, ""name"": ""0.31.1"", ""sha"": ""a22c7761b61baa61b8e3da7d30887468d61d6b83""}, {""repo_id"": 107914493, ""name"": ""0.31"", ""sha"": ""7f89928062b1a1fdb2625a946f7cd5161e597401""}, {""repo_id"": 107914493, ""name"": ""0.30.2"", ""sha"": ""2bf7ce5f517d772a16d7855a35a8a75d4456aad7""}, {""repo_id"": 107914493, ""name"": ""0.30.1"", ""sha"": ""3ca290e0db03bb4747e24203c445873f74512107""}, {""repo_id"": 107914493, ""name"": ""0.30"", ""sha"": ""8050f9e1ece9afd0236ad38c6458c12a4ad917e6""}, {""repo_id"": 107914493, ""name"": ""0.29.3"", ""sha"": ""0fc8afde0eb5ef677f4ac31601540d6168c8208d""}, {""repo_id"": 107914493, ""name"": ""0.29.2"", ""sha"": ""6abe6faff6b035e9334dd05f8c741ae9b7a47440""}, {""repo_id"": 107914493, ""name"": ""0.29.1"", ""sha"": ""2a94f3719fb2c4335fcda374fa92f87272b02d34""}, {""repo_id"": 107914493, ""name"": ""0.29"", ""sha"": ""fb7ee8e0ad59a15083234a48e935525f6e7257dd""}, {""repo_id"": 107914493, ""name"": ""0.28"", ""sha"": ""e518f76c5f5dd0138032bfb26387f5bb91086a3f""}, {""repo_id"": 107914493, ""name"": ""0.27.1"", ""sha"": ""3f3f29ac9afe7c41ffc48a3bd2af473a53eecc8a""}, {""repo_id"": 107914493, ""name"": ""0.27"", ""sha"": ""436b8bc1d17c2ab415800ab209204f94e7f7929e""}, {""repo_id"": 107914493, ""name"": ""0.26.2"", ""sha"": ""a418c8b44f82d456be523c8690cf7236bb648c22""}, {""repo_id"": 107914493, ""name"": ""0.26.1"", ""sha"": ""4722acc73ce761556b18f5dcbe36b7fef2ee2c69""}, {""repo_id"": 107914493, ""name"": ""0.26"", ""sha"": ""424e146697309a54c05d5d1ba1f840849ddbafdc""}, {""repo_id"": 107914493, ""name"": ""0.25.2"", ""sha"": ""b5128fc53fce6a1bf3b16bad9f318451bc1d1263""}, {""repo_id"": 107914493, ""name"": ""0.25.1"", ""sha"": ""3dc0b3fa8c9b9bd81540ffe20c8b7e7a72465274""}, {""repo_id"": 107914493, ""name"": ""0.25"", ""sha"": ""57a71377c992753327a16b417daf79df7f506dd1""}, {""repo_id"": 107914493, ""name"": ""0.24"", ""sha"": ""28872a1fa789f314b0342f4e6182f1c78d6e2bca""}, {""repo_id"": 107914493, ""name"": ""0.23.2"", ""sha"": ""6df6f712b36f0fe75694174906e31242427a8d1d""}, {""repo_id"": 107914493, ""name"": ""0.23.1"", ""sha"": ""dea86b9fba78e032ad09673e884e764387daf209""}, {""repo_id"": 107914493, ""name"": ""0.23"", ""sha"": ""e04f5b0d348ef7275a0a5ab9eb53527105132885""}, {""repo_id"": 107914493, ""name"": ""0.22.1"", ""sha"": ""5d6252788230d168ba09f379d1d2af867e3302ab""}, {""repo_id"": 107914493, ""name"": ""0.22"", ""sha"": ""558d9d7bfef3dd633eb16389281b67d42c9bdeef""}, {""repo_id"": 107914493, ""name"": ""0.21"", ""sha"": ""403211de632cd15f0820cc9399305fc43c187b47""}, {""repo_id"": 107914493, ""name"": ""0.20"", ""sha"": ""3a5d7951ce8f35118ffdd7f8d86e09b909e1218c""}, {""repo_id"": 107914493, ""name"": ""0.19"", ""sha"": ""ba9bfa583179c25aaef94b1f44da7eba74620b9a""}, {""repo_id"": 107914493, ""name"": ""0.18"", ""sha"": ""43ae15c0d14b3e968e8d5bfef72ac0c39783c3a2""}, {""repo_id"": 107914493, ""name"": ""0.17"", ""sha"": ""fb988ace7c7e2bee5ac142a0eab22431d0675a77""}, {""repo_id"": 107914493, ""name"": ""0.16"", ""sha"": ""b6539ff04502536bd1fa96e3b1430bdafc456826""}, {""repo_id"": 107914493, ""name"": ""0.15"", ""sha"": ""7706fe0c67aba5cfe905c7906cae9e0c43cd75b2""}, {""repo_id"": 107914493, ""name"": ""0.14"", ""sha"": ""2edc652df6d786e4f2c3f073e3567002d248be09""}, {""repo_id"": 107914493, ""name"": ""0.13"", ""sha"": ""c160f15c3937f8fbe581276f811e8c58f9137bb1""}, {""repo_id"": 107914493, ""name"": ""0.12"", ""sha"": ""51bdd67691bd69082ae7690af8b905f06050ee80""}, {""repo_id"": 107914493, ""name"": ""0.11"", ""sha"": ""b0f3d4e375655f0764f3137dbcede324f9bbc0cb""}, {""repo_id"": 107914493, ""name"": ""0.10"", ""sha"": ""5928c11ee798a232aa4096706cd47e639d1c9fc2""}, {""repo_id"": 107914493, ""name"": ""0.9"", ""sha"": ""d75f423b6fcfc074b7c6f8f7679da8876f181edd""}, {""repo_id"": 107914493, ""name"": ""0.8"", ""sha"": ""fe279ab7b4ae99dab295d5cf4d39ad06d782997e""}, {""repo_id"": 107914493, ""name"": ""0.7"", ""sha"": ""6b3b05b6db0d2a7b7cec8b8dbb4ddc5e12a376b2""}, {""repo_id"": 207052882, ""name"": ""2.3"", ""sha"": ""7090e43d804724ef3b31ae5ca9efd6ac05f76cbc""}, {""repo_id"": 207052882, ""name"": ""2.2"", ""sha"": ""4fe69783b55465e7692a807d3a02a710f69c9c42""}, {""repo_id"": 207052882, ""name"": ""2.1"", ""sha"": ""9d7aed336c8e62bf372caa800cb4aae3985cbae9""}, {""repo_id"": 207052882, ""name"": ""2.0"", ""sha"": ""44611df1524a03ce305405e5902c9615e3c73a72""}, {""repo_id"": 207052882, ""name"": ""1.1"", ""sha"": ""5cd34bd07d704487d48ac741ee5da5317afe88d2""}, {""repo_id"": 207052882, ""name"": ""1.0.1"", ""sha"": ""3b7ab5685de89fcb6fc92d320c0e24b17be05570""}, {""repo_id"": 207052882, ""name"": ""1.0"", ""sha"": ""1ea30c8fb1d080bd5e38c577e3ad20bb527a2fe6""}, {""repo_id"": 207052882, ""name"": ""0.7"", ""sha"": ""e35eec4343aa560c58c1634cc228d0d46c442304""}, {""repo_id"": 207052882, ""name"": ""0.6"", ""sha"": ""9eb737090fafd0e5a7e314be48402374d99e9828""}, {""repo_id"": 207052882, ""name"": ""0.5"", ""sha"": ""ae9035f8fe5aff1c54bff4c6b4c2e808a44f0f2a""}, {""repo_id"": 207052882, ""name"": ""0.4"", ""sha"": ""8c6251c31a05c58c2bfbef114247642d1b3dbb44""}, {""repo_id"": 207052882, ""name"": ""0.3"", ""sha"": ""f697f247468516aa4ee13b1862b59e0dba18d00f""}, {""repo_id"": 207052882, ""name"": ""0.2"", ""sha"": ""0fe96bc50fb3d7b1c7e4577db0ddf207eaeebbb9""}, {""repo_id"": 207052882, ""name"": ""0.1.1"", ""sha"": ""321e0284c64dc48b2143311009886293c05edb07""}, {""repo_id"": 207052882, ""name"": ""0.1"", ""sha"": ""7387c88a3f84704548e81d43b91615c02b61a957""}] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660355904, https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660548780,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43,660548780,MDEyOklzc3VlQ29tbWVudDY2MDU0ODc4MA==,9599,2020-07-18T22:02:37Z,2020-07-18T23:05:56Z,MEMBER,https://github-to-sqlite.dogsheep.net/github/tags?_facet=repo,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660355904, https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660551397,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43,660551397,MDEyOklzc3VlQ29tbWVudDY2MDU1MTM5Nw==,9599,2020-07-18T22:27:32Z,2020-07-18T23:05:45Z,MEMBER,"```sql with most_recent_releases as ( with ranked as ( select repo, tag_name, published_at, row_number() OVER ( partition BY repo ORDER BY published_at DESC ) rank FROM releases ) select * from ranked where rank = 1 ) select repos.full_name as repo, most_recent_releases.tag_name as release, commits.committer_date as release_commit_date, ( select count(*) from commits c2 where c2.repo = repos.id and c2.committer_date > commits.committer_date ) as commits_since_release, 'https://github.com/' || repos.full_name || '/compare/' || most_recent_releases.tag_name || '...' || repos.default_branch as view_commits from most_recent_releases join repos on most_recent_releases.repo = repos.id join tags on tags.repo = repos.id and tags.name = most_recent_releases.tag_name join commits on tags.sha = commits.sha order by commits_since_release desc ``` repo | release | release_commit_date | commits_since_release | view_commits -- | -- | -- | -- | -- simonw/datasette | 0.45 | 2020-07-01T21:43:07Z | 9 | https://github.com/simonw/datasette/compare/0.45...master dogsheep/twitter-to-sqlite | 0.21.1 | 2020-04-30T18:20:43Z | 2 | https://github.com/dogsheep/twitter-to-sqlite/compare/0.21.1...master dogsheep/github-to-sqlite | 2.3 | 2020-07-09T23:26:34Z | 2 | https://github.com/dogsheep/github-to-sqlite/compare/2.3...master dogsheep/dogsheep-photos | 0.4.1 | 2020-05-25T20:11:20Z | 2 | https://github.com/dogsheep/dogsheep-photos/compare/0.4.1...master dogsheep/swarm-to-sqlite | 0.3.1 | 2020-03-28T02:29:41Z | 1 | https://github.com/dogsheep/swarm-to-sqlite/compare/0.3.1...master dogsheep/hacker-news-to-sqlite | 0.3.1 | 2020-03-21T22:39:34Z | 1 | https://github.com/dogsheep/hacker-news-to-sqlite/compare/0.3.1...master simonw/sqlite-utils | 2.11 | 2020-07-08T17:36:07Z | 0 | https://github.com/simonw/sqlite-utils/compare/2.11...master dogsheep/healthkit-to-sqlite | 0.5 | 2020-03-28T01:50:51Z | 0 | https://github.com/dogsheep/healthkit-to-sqlite/compare/0.5...master dogsheep/inaturalist-to-sqlite | 0.2 | 2020-03-24T00:35:44Z | 0 | https://github.com/dogsheep/inaturalist-to-sqlite/compare/0.2...master dogsheep/genome-to-sqlite | 0.1 | 2019-09-19T15:38:10Z | 0 | https://github.com/dogsheep/genome-to-sqlite/compare/0.1...master dogsheep/pocket-to-sqlite | 0.2 | 2020-03-27T22:23:16Z | 0 | https://github.com/dogsheep/pocket-to-sqlite/compare/0.2...master https://github-to-sqlite.dogsheep.net/github?sql=with+most_recent_releases+as+%28%0D%0A++with+ranked+as+%28%0D%0A++++select%0D%0A++++++repo%2C%0D%0A++++++tag_name%2C%0D%0A++++++published_at%2C%0D%0A++++++row_number%28%29+OVER+%28%0D%0A++++++++partition+BY+repo%0D%0A++++++++ORDER+BY%0D%0A++++++++++published_at+DESC%0D%0A++++++%29+rank%0D%0A++++FROM%0D%0A++++++releases%0D%0A++%29%0D%0A++select%0D%0A++++*%0D%0A++from%0D%0A++++ranked%0D%0A++where%0D%0A++++rank+%3D+1%0D%0A%29%0D%0Aselect%0D%0A++repos.full_name+as+repo%2C%0D%0A++most_recent_releases.tag_name+as+release%2C%0D%0A++commits.committer_date+as+release_commit_date%2C%0D%0A++%28%0D%0A++++select%0D%0A++++++count%28*%29%0D%0A++++from%0D%0A++++++commits+c2%0D%0A++++where%0D%0A++++++c2.repo+%3D+repos.id%0D%0A++++++and+c2.committer_date+%3E+commits.committer_date%0D%0A++%29+as+commits_since_release%2C%0D%0A++%27https%3A%2F%2Fgithub.com%2F%27+%7C%7C+repos.full_name+%7C%7C+%27%2Fcompare%2F%27+%7C%7C+most_recent_releases.tag_name+%7C%7C+%27...%27+%7C%7C+repos.default_branch+as+view_commits%0D%0Afrom%0D%0A++most_recent_releases%0D%0A++join+repos+on+most_recent_releases.repo+%3D+repos.id%0D%0A++join+tags+on+tags.repo+%3D+repos.id%0D%0A++and+tags.name+%3D+most_recent_releases.tag_name%0D%0A++join+commits+on+tags.sha+%3D+commits.sha%0D%0Aorder+by%0D%0A++commits_since_release+desc","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660355904, https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660553646,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45,660553646,MDEyOklzc3VlQ29tbWVudDY2MDU1MzY0Ng==,9599,2020-07-18T22:51:41Z,2020-07-18T22:51:41Z,MEMBER,"I could fix this by putting `REFRESH_DB` in a commit message: https://github.com/dogsheep/github-to-sqlite/blob/4ae4aa6f172344b19ff3513707195ee6d2654bd4/.github/workflows/deploy-demo.yml#L41-L46 But... doing so would lose the data I've collected in https://github-to-sqlite.dogsheep.net/github/dependents?_sort_desc=first_seen_utc concerning the first time each dependent repo was spotted.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660429601, https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660553711,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45,660553711,MDEyOklzc3VlQ29tbWVudDY2MDU1MzcxMQ==,9599,2020-07-18T22:52:16Z,2020-07-18T22:52:16Z,MEMBER,"I think the best fix is to download the `github.db` database, manually fix it and then manually deploy it to Cloud Run from my laptop.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660429601, https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660554162,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45,660554162,MDEyOklzc3VlQ29tbWVudDY2MDU1NDE2Mg==,9599,2020-07-18T22:56:58Z,2020-07-18T22:56:58Z,MEMBER,"Manually fixing the database: ``` $ wget 'https://github-to-sqlite.dogsheep.net/github.db' --2020-07-18 15:52:33-- https://github-to-sqlite.dogsheep.net/github.db Resolving github-to-sqlite.dogsheep.net (github-to-sqlite.dogsheep.net)... 172.217.5.115 Connecting to github-to-sqlite.dogsheep.net (github-to-sqlite.dogsheep.net)|172.217.5.115|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 14626816 (14M) [application/octet-stream] Saving to: ‘github.db’ github.db 100%[============================================================================>] 13.95M 1.22MB/s in 18s 2020-07-18 15:52:53 (773 KB/s) - ‘github.db’ saved [14626816/14626816] $ sqlite3 github.db SQLite version 3.28.0 2019-04-15 14:49:49 Enter "".help"" for usage hints. sqlite> drop table tags; sqlite> ^D $ github-to-sqlite tags github.db simonw/datasette simonw/sqlite-utils dogsheep/healthkit-to-sqlite dogsheep/swarm-to-sqlite dogsheep/twitter-to-sqlite dogsheep/inaturalist-to-sqlite dogsheep/google-takeout-to-sqlite dogsheep/github-to-sqlite dogsheep/genome-to-sqlite dogsheep/pocket-to-sqlite dogsheep/hacker-news-to-sqlite dogsheep/dogsheep-photos $ sqlite-utils tables github.db --counts [{""table"": ""users"", ""count"": 4048}, {""table"": ""repos"", ""count"": 210}, ... {""table"": ""stars"", ""count"": 4140}, {""table"": ""tags"", ""count"": 188}] $ sqlite-utils rows github.db tags [{""repo"": 107914493, ""name"": ""0.45"", ""sha"": ""f1f581b7ffcd5d8f3ae6c1c654d813a6641410eb""}, {""repo"": 107914493, ""name"": ""0.45a5"", ""sha"": ""676bb64c877d73f8ff496cef4632f5a8a5a9283c""}, ... ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660429601, https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660554299,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45,660554299,MDEyOklzc3VlQ29tbWVudDY2MDU1NDI5OQ==,9599,2020-07-18T22:58:24Z,2020-07-18T23:02:52Z,MEMBER,"Deploying the fixed version like this: ``` $ gcloud config set run/region us-central1 $ gcloud config set project datasette-222320 $ datasette publish cloudrun /tmp/github.db \ -m demo-metadata.json \ --service github-to-sqlite \ --install=py-gfm \ --install='datasette-search-all>=0.3' \ --install='datasette-render-markdown>=1.1.2' \ --install=datasette-pretty-json \ --install=datasette-json-html \ --install=datasette-vega ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660429601, https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660554811,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45,660554811,MDEyOklzc3VlQ29tbWVudDY2MDU1NDgxMQ==,9599,2020-07-18T23:03:13Z,2020-07-18T23:03:13Z,MEMBER,https://github-to-sqlite.dogsheep.net/github/tags now shows a `repo` column instead of a `repo_id` column.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660429601, https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-675259273,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46,675259273,MDEyOklzc3VlQ29tbWVudDY3NTI1OTI3Mw==,9599,2020-08-18T05:28:32Z,2020-08-18T05:28:32Z,MEMBER,"Oh that's interesting - i didn't realize ""reviews"" were a separate concept. I'd definitely accept a pull request adding those!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",664485022, https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-735482187,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46,735482187,MDEyOklzc3VlQ29tbWVudDczNTQ4MjE4Nw==,9599,2020-11-30T00:20:11Z,2020-11-30T00:20:11Z,MEMBER,"Pull request are now added, thanks to @adamjonas.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",664485022, https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-735482546,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46,735482546,MDEyOklzc3VlQ29tbWVudDczNTQ4MjU0Ng==,9599,2020-11-30T00:22:02Z,2020-11-30T00:22:02Z,MEMBER,"As for reviews... here's the output of `github-to-sqlite get https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48/reviews --accept 'application/vnd.github.v3+json'` ```json [ { ""id"": 503368921, ""node_id"": ""MDE3OlB1bGxSZXF1ZXN0UmV2aWV3NTAzMzY4OTIx"", ""user"": { ""login"": ""simonw"", ""id"": 9599, ""node_id"": ""MDQ6VXNlcjk1OTk="", ""avatar_url"": ""https://avatars0.githubusercontent.com/u/9599?u=5968723deb1a55b82620e106f5ca58e9b11a0942&v=4"", ""gravatar_id"": """", ""url"": ""https://api.github.com/users/simonw"", ""html_url"": ""https://github.com/simonw"", ""followers_url"": ""https://api.github.com/users/simonw/followers"", ""following_url"": ""https://api.github.com/users/simonw/following{/other_user}"", ""gists_url"": ""https://api.github.com/users/simonw/gists{/gist_id}"", ""starred_url"": ""https://api.github.com/users/simonw/starred{/owner}{/repo}"", ""subscriptions_url"": ""https://api.github.com/users/simonw/subscriptions"", ""organizations_url"": ""https://api.github.com/users/simonw/orgs"", ""repos_url"": ""https://api.github.com/users/simonw/repos"", ""events_url"": ""https://api.github.com/users/simonw/events{/privacy}"", ""received_events_url"": ""https://api.github.com/users/simonw/received_events"", ""type"": ""User"", ""site_admin"": false }, ""body"": """", ""state"": ""CHANGES_REQUESTED"", ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-503368921"", ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"", ""author_association"": ""MEMBER"", ""_links"": { ""html"": { ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-503368921"" }, ""pull_request"": { ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"" } }, ""submitted_at"": ""2020-10-06T21:28:40Z"", ""commit_id"": ""4f33b850bd37829262dd29e1c520afffebedc19c"" }, { ""id"": 513118561, ""node_id"": ""MDE3OlB1bGxSZXF1ZXN0UmV2aWV3NTEzMTE4NTYx"", ""user"": { ""login"": ""adamjonas"", ""id"": 755825, ""node_id"": ""MDQ6VXNlcjc1NTgyNQ=="", ""avatar_url"": ""https://avatars1.githubusercontent.com/u/755825?v=4"", ""gravatar_id"": """", ""url"": ""https://api.github.com/users/adamjonas"", ""html_url"": ""https://github.com/adamjonas"", ""followers_url"": ""https://api.github.com/users/adamjonas/followers"", ""following_url"": ""https://api.github.com/users/adamjonas/following{/other_user}"", ""gists_url"": ""https://api.github.com/users/adamjonas/gists{/gist_id}"", ""starred_url"": ""https://api.github.com/users/adamjonas/starred{/owner}{/repo}"", ""subscriptions_url"": ""https://api.github.com/users/adamjonas/subscriptions"", ""organizations_url"": ""https://api.github.com/users/adamjonas/orgs"", ""repos_url"": ""https://api.github.com/users/adamjonas/repos"", ""events_url"": ""https://api.github.com/users/adamjonas/events{/privacy}"", ""received_events_url"": ""https://api.github.com/users/adamjonas/received_events"", ""type"": ""User"", ""site_admin"": false }, ""body"": """", ""state"": ""COMMENTED"", ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-513118561"", ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"", ""author_association"": ""CONTRIBUTOR"", ""_links"": { ""html"": { ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-513118561"" }, ""pull_request"": { ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"" } }, ""submitted_at"": ""2020-10-20T20:45:05Z"", ""commit_id"": ""4f33b850bd37829262dd29e1c520afffebedc19c"" }, { ""id"": 513127529, ""node_id"": ""MDE3OlB1bGxSZXF1ZXN0UmV2aWV3NTEzMTI3NTI5"", ""user"": { ""login"": ""adamjonas"", ""id"": 755825, ""node_id"": ""MDQ6VXNlcjc1NTgyNQ=="", ""avatar_url"": ""https://avatars1.githubusercontent.com/u/755825?v=4"", ""gravatar_id"": """", ""url"": ""https://api.github.com/users/adamjonas"", ""html_url"": ""https://github.com/adamjonas"", ""followers_url"": ""https://api.github.com/users/adamjonas/followers"", ""following_url"": ""https://api.github.com/users/adamjonas/following{/other_user}"", ""gists_url"": ""https://api.github.com/users/adamjonas/gists{/gist_id}"", ""starred_url"": ""https://api.github.com/users/adamjonas/starred{/owner}{/repo}"", ""subscriptions_url"": ""https://api.github.com/users/adamjonas/subscriptions"", ""organizations_url"": ""https://api.github.com/users/adamjonas/orgs"", ""repos_url"": ""https://api.github.com/users/adamjonas/repos"", ""events_url"": ""https://api.github.com/users/adamjonas/events{/privacy}"", ""received_events_url"": ""https://api.github.com/users/adamjonas/received_events"", ""type"": ""User"", ""site_admin"": false }, ""body"": """", ""state"": ""COMMENTED"", ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-513127529"", ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"", ""author_association"": ""CONTRIBUTOR"", ""_links"": { ""html"": { ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-513127529"" }, ""pull_request"": { ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"" } }, ""submitted_at"": ""2020-10-20T20:57:33Z"", ""commit_id"": ""3a0d5c498f9faae4e40aab204cd01b965a4f61f3"" } ] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",664485022, https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-735483604,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46,735483604,MDEyOklzc3VlQ29tbWVudDczNTQ4MzYwNA==,9599,2020-11-30T00:26:50Z,2020-11-30T00:26:50Z,MEMBER,"It seems like there's a lot missing from that - those aren't particularly interesting given the data that is returned. From the docs at https://docs.github.com/en/free-pro-team@latest/rest/reference/pulls#reviews it looks like each review consists of multiple comments, and the comments are where the useful material is - https://docs.github.com/en/free-pro-team@latest/rest/reference/pulls#list-comments-for-a-pull-request-review `github-to-sqlite get https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48/reviews/503368921/comments --accept 'application/vnd.github.v3+json'` ```json [ { ""id"": 500603838, ""node_id"": ""MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDUwMDYwMzgzOA=="", ""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500603838"", ""pull_request_review_id"": 503368921, ""diff_hunk"": ""@@ -0,0 +1,370 @@\n+[\n+ {\n+ \""url\"": \""https://api.github.com/repos/simonw/datasette/pulls/571\"",\n+ \""id\"": 313384926,\n+ \""node_id\"": \""MDExOlB1bGxSZXF1ZXN0MzEzMzg0OTI2\"",\n+ \""html_url\"": \""https://github.com/simonw/datasette/pull/571\"",\n+ \""diff_url\"": \""https://github.com/simonw/datasette/pull/571.diff\"",\n+ \""patch_url\"": \""https://github.com/simonw/datasette/pull/571.patch\"",\n+ \""issue_url\"": \""https://api.github.com/repos/simonw/datasette/issues/571\"",\n+ \""number\"": 571,\n+ \""state\"": \""closed\"",\n+ \""locked\"": false,\n+ \""title\"": \""detect_fts now works with alternative table escaping\"",\n+ \""user\"": {\n+ \""login\"": \""simonw\"",\n+ \""id\"": 9599,\n+ \""node_id\"": \""MDQ6VXNlcjk1OTk=\"",\n+ \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"",\n+ \""gravatar_id\"": \""\"",\n+ \""url\"": \""https://api.github.com/users/simonw\"",\n+ \""html_url\"": \""https://github.com/simonw\"",\n+ \""followers_url\"": \""https://api.github.com/users/simonw/followers\"",\n+ \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"",\n+ \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"",\n+ \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"",\n+ \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"",\n+ \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"",\n+ \""repos_url\"": \""https://api.github.com/users/simonw/repos\"",\n+ \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"",\n+ \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"",\n+ \""type\"": \""User\"",\n+ \""site_admin\"": false\n+ },\n+ \""body\"": \""Fixes #570\"",\n+ \""created_at\"": \""2019-09-03T00:23:39Z\"",\n+ \""updated_at\"": \""2019-09-03T00:32:28Z\"",\n+ \""closed_at\"": \""2019-09-03T00:32:28Z\"",\n+ \""merged_at\"": \""2019-09-03T00:32:28Z\"",\n+ \""merge_commit_sha\"": \""2dc5c8dc259a0606162673d394ba8cc1c6f54428\"",\n+ \""assignee\"": null,\n+ \""assignees\"": [\n+\n+ ],\n+ \""requested_reviewers\"": [\n+\n+ ],\n+ \""requested_teams\"": [\n+\n+ ],\n+ \""labels\"": [\n+\n+ ],\n+ \""milestone\"": null,\n+ \""draft\"": false,\n+ \""commits_url\"": \""https://api.github.com/repos/simonw/datasette/pulls/571/commits\"",\n+ \""review_comments_url\"": \""https://api.github.com/repos/simonw/datasette/pulls/571/comments\"",\n+ \""review_comment_url\"": \""https://api.github.com/repos/simonw/datasette/pulls/comments{/number}\"",\n+ \""comments_url\"": \""https://api.github.com/repos/simonw/datasette/issues/571/comments\"",\n+ \""statuses_url\"": \""https://api.github.com/repos/simonw/datasette/statuses/a85239f69261c10f1a9f90514c8b5d113cb94585\"",\n+ \""head\"": {\n+ \""label\"": \""simonw:detect-fts\"",\n+ \""ref\"": \""detect-fts\"",\n+ \""sha\"": \""a85239f69261c10f1a9f90514c8b5d113cb94585\"",\n+ \""user\"": {\n+ \""login\"": \""simonw\"",\n+ \""id\"": 9599,\n+ \""node_id\"": \""MDQ6VXNlcjk1OTk=\"",\n+ \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"",\n+ \""gravatar_id\"": \""\"",\n+ \""url\"": \""https://api.github.com/users/simonw\"",\n+ \""html_url\"": \""https://github.com/simonw\"",\n+ \""followers_url\"": \""https://api.github.com/users/simonw/followers\"",\n+ \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"",\n+ \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"",\n+ \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"",\n+ \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"",\n+ \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"",\n+ \""repos_url\"": \""https://api.github.com/users/simonw/repos\"",\n+ \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"",\n+ \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"",\n+ \""type\"": \""User\"",\n+ \""site_admin\"": false\n+ },\n+ \""repo\"": {\n+ \""id\"": 107914493,\n+ \""node_id\"": \""MDEwOlJlcG9zaXRvcnkxMDc5MTQ0OTM=\"",\n+ \""name\"": \""datasette\"",\n+ \""full_name\"": \""simonw/datasette\"",\n+ \""private\"": false,\n+ \""owner\"": {\n+ \""login\"": \""simonw\"",\n+ \""id\"": 9599,\n+ \""node_id\"": \""MDQ6VXNlcjk1OTk=\"",\n+ \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"",\n+ \""gravatar_id\"": \""\"",\n+ \""url\"": \""https://api.github.com/users/simonw\"",\n+ \""html_url\"": \""https://github.com/simonw\"",\n+ \""followers_url\"": \""https://api.github.com/users/simonw/followers\"",\n+ \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"",\n+ \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"",\n+ \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"",\n+ \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"",\n+ \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"",\n+ \""repos_url\"": \""https://api.github.com/users/simonw/repos\"",\n+ \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"",\n+ \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"",\n+ \""type\"": \""User\"",\n+ \""site_admin\"": false\n+ },\n+ \""html_url\"": \""https://github.com/simonw/datasette\"",\n+ \""description\"": \""An open source multi-tool for exploring and publishing data\"",\n+ \""fork\"": false,\n+ \""url\"": \""https://api.github.com/repos/simonw/datasette\"",\n+ \""forks_url\"": \""https://api.github.com/repos/simonw/datasette/forks\"",\n+ \""keys_url\"": \""https://api.github.com/repos/simonw/datasette/keys{/key_id}\"",\n+ \""collaborators_url\"": \""https://api.github.com/repos/simonw/datasette/collaborators{/collaborator}\"",\n+ \""teams_url\"": \""https://api.github.com/repos/simonw/datasette/teams\"",\n+ \""hooks_url\"": \""https://api.github.com/repos/simonw/datasette/hooks\"",\n+ \""issue_events_url\"": \""https://api.github.com/repos/simonw/datasette/issues/events{/number}\"",\n+ \""events_url\"": \""https://api.github.com/repos/simonw/datasette/events\"",\n+ \""assignees_url\"": \""https://api.github.com/repos/simonw/datasette/assignees{/user}\"",\n+ \""branches_url\"": \""https://api.github.com/repos/simonw/datasette/branches{/branch}\"",\n+ \""tags_url\"": \""https://api.github.com/repos/simonw/datasette/tags\"",\n+ \""blobs_url\"": \""https://api.github.com/repos/simonw/datasette/git/blobs{/sha}\"",\n+ \""git_tags_url\"": \""https://api.github.com/repos/simonw/datasette/git/tags{/sha}\"",\n+ \""git_refs_url\"": \""https://api.github.com/repos/simonw/datasette/git/refs{/sha}\"",\n+ \""trees_url\"": \""https://api.github.com/repos/simonw/datasette/git/trees{/sha}\"",\n+ \""statuses_url\"": \""https://api.github.com/repos/simonw/datasette/statuses/{sha}\"",\n+ \""languages_url\"": \""https://api.github.com/repos/simonw/datasette/languages\"",\n+ \""stargazers_url\"": \""https://api.github.com/repos/simonw/datasette/stargazers\"",\n+ \""contributors_url\"": \""https://api.github.com/repos/simonw/datasette/contributors\"",\n+ \""subscribers_url\"": \""https://api.github.com/repos/simonw/datasette/subscribers\"",\n+ \""subscription_url\"": \""https://api.github.com/repos/simonw/datasette/subscription\"",\n+ \""commits_url\"": \""https://api.github.com/repos/simonw/datasette/commits{/sha}\"",\n+ \""git_commits_url\"": \""https://api.github.com/repos/simonw/datasette/git/commits{/sha}\"",\n+ \""comments_url\"": \""https://api.github.com/repos/simonw/datasette/comments{/number}\"",\n+ \""issue_comment_url\"": \""https://api.github.com/repos/simonw/datasette/issues/comments{/number}\"",\n+ \""contents_url\"": \""https://api.github.com/repos/simonw/datasette/contents/{+path}\"",\n+ \""compare_url\"": \""https://api.github.com/repos/simonw/datasette/compare/{base}...{head}\"",\n+ \""merges_url\"": \""https://api.github.com/repos/simonw/datasette/merges\"",\n+ \""archive_url\"": \""https://api.github.com/repos/simonw/datasette/{archive_format}{/ref}\"",\n+ \""downloads_url\"": \""https://api.github.com/repos/simonw/datasette/downloads\"",\n+ \""issues_url\"": \""https://api.github.com/repos/simonw/datasette/issues{/number}\"",\n+ \""pulls_url\"": \""https://api.github.com/repos/simonw/datasette/pulls{/number}\"",\n+ \""milestones_url\"": \""https://api.github.com/repos/simonw/datasette/milestones{/number}\"",\n+ \""notifications_url\"": \""https://api.github.com/repos/simonw/datasette/notifications{?since,all,participating}\"",\n+ \""labels_url\"": \""https://api.github.com/repos/simonw/datasette/labels{/name}\"",\n+ \""releases_url\"": \""https://api.github.com/repos/simonw/datasette/releases{/id}\"",\n+ \""deployments_url\"": \""https://api.github.com/repos/simonw/datasette/deployments\"",\n+ \""created_at\"": \""2017-10-23T00:39:03Z\"",\n+ \""updated_at\"": \""2020-07-27T20:42:15Z\"",\n+ \""pushed_at\"": \""2020-07-26T01:21:05Z\"",\n+ \""git_url\"": \""git://github.com/simonw/datasette.git\"",\n+ \""ssh_url\"": \""git@github.com:simonw/datasette.git\"",\n+ \""clone_url\"": \""https://github.com/simonw/datasette.git\"",\n+ \""svn_url\"": \""https://github.com/simonw/datasette\"",\n+ \""homepage\"": \""http://datasette.readthedocs.io/\"",\n+ \""size\"": 3487,\n+ \""stargazers_count\"": 3642,\n+ \""watchers_count\"": 3642,\n+ \""language\"": \""Python\"",\n+ \""has_issues\"": true,\n+ \""has_projects\"": false,\n+ \""has_downloads\"": true,\n+ \""has_wiki\"": true,\n+ \""has_pages\"": false,\n+ \""forks_count\"": 206,\n+ \""mirror_url\"": null,\n+ \""archived\"": false,\n+ \""disabled\"": false,\n+ \""open_issues_count\"": 190,\n+ \""license\"": {\n+ \""key\"": \""apache-2.0\"",\n+ \""name\"": \""Apache License 2.0\"",\n+ \""spdx_id\"": \""Apache-2.0\"",\n+ \""url\"": \""https://api.github.com/licenses/apache-2.0\"",\n+ \""node_id\"": \""MDc6TGljZW5zZTI=\""\n+ },\n+ \""forks\"": 206,\n+ \""open_issues\"": 190,\n+ \""watchers\"": 3642,\n+ \""default_branch\"": \""master\""\n+ }\n+ },\n+ \""base\"": {\n+ \""label\"": \""simonw:master\"",\n+ \""ref\"": \""master\"",\n+ \""sha\"": \""f04deebec4f3842f7bd610cd5859de529f77d50e\"",\n+ \""user\"": {\n+ \""login\"": \""simonw\"",\n+ \""id\"": 9599,\n+ \""node_id\"": \""MDQ6VXNlcjk1OTk=\"",\n+ \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"",\n+ \""gravatar_id\"": \""\"",\n+ \""url\"": \""https://api.github.com/users/simonw\"",\n+ \""html_url\"": \""https://github.com/simonw\"",\n+ \""followers_url\"": \""https://api.github.com/users/simonw/followers\"",\n+ \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"",\n+ \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"",\n+ \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"",\n+ \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"",\n+ \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"",\n+ \""repos_url\"": \""https://api.github.com/users/simonw/repos\"",\n+ \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"",\n+ \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"",\n+ \""type\"": \""User\"",\n+ \""site_admin\"": false\n+ },\n+ \""repo\"": {\n+ \""id\"": 107914493,\n+ \""node_id\"": \""MDEwOlJlcG9zaXRvcnkxMDc5MTQ0OTM=\"",\n+ \""name\"": \""datasette\"",\n+ \""full_name\"": \""simonw/datasette\"",\n+ \""private\"": false,\n+ \""owner\"": {\n+ \""login\"": \""simonw\"",\n+ \""id\"": 9599,\n+ \""node_id\"": \""MDQ6VXNlcjk1OTk=\"",\n+ \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"",\n+ \""gravatar_id\"": \""\"",\n+ \""url\"": \""https://api.github.com/users/simonw\"",\n+ \""html_url\"": \""https://github.com/simonw\"",\n+ \""followers_url\"": \""https://api.github.com/users/simonw/followers\"",\n+ \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"",\n+ \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"",\n+ \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"",\n+ \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"",\n+ \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"",\n+ \""repos_url\"": \""https://api.github.com/users/simonw/repos\"",\n+ \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"",\n+ \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"",\n+ \""type\"": \""User\"",\n+ \""site_admin\"": false\n+ },\n+ \""html_url\"": \""https://github.com/simonw/datasette\"",\n+ \""description\"": \""An open source multi-tool for exploring and publishing data\"",\n+ \""fork\"": false,\n+ \""url\"": \""https://api.github.com/repos/simonw/datasette\"",\n+ \""forks_url\"": \""https://api.github.com/repos/simonw/datasette/forks\"",\n+ \""keys_url\"": \""https://api.github.com/repos/simonw/datasette/keys{/key_id}\"",\n+ \""collaborators_url\"": \""https://api.github.com/repos/simonw/datasette/collaborators{/collaborator}\"",\n+ \""teams_url\"": \""https://api.github.com/repos/simonw/datasette/teams\"",\n+ \""hooks_url\"": \""https://api.github.com/repos/simonw/datasette/hooks\"",\n+ \""issue_events_url\"": \""https://api.github.com/repos/simonw/datasette/issues/events{/number}\"",\n+ \""events_url\"": \""https://api.github.com/repos/simonw/datasette/events\"",\n+ \""assignees_url\"": \""https://api.github.com/repos/simonw/datasette/assignees{/user}\"",\n+ \""branches_url\"": \""https://api.github.com/repos/simonw/datasette/branches{/branch}\"",\n+ \""tags_url\"": \""https://api.github.com/repos/simonw/datasette/tags\"",\n+ \""blobs_url\"": \""https://api.github.com/repos/simonw/datasette/git/blobs{/sha}\"",\n+ \""git_tags_url\"": \""https://api.github.com/repos/simonw/datasette/git/tags{/sha}\"",\n+ \""git_refs_url\"": \""https://api.github.com/repos/simonw/datasette/git/refs{/sha}\"",\n+ \""trees_url\"": \""https://api.github.com/repos/simonw/datasette/git/trees{/sha}\"",\n+ \""statuses_url\"": \""https://api.github.com/repos/simonw/datasette/statuses/{sha}\"",\n+ \""languages_url\"": \""https://api.github.com/repos/simonw/datasette/languages\"",\n+ \""stargazers_url\"": \""https://api.github.com/repos/simonw/datasette/stargazers\"",\n+ \""contributors_url\"": \""https://api.github.com/repos/simonw/datasette/contributors\"",\n+ \""subscribers_url\"": \""https://api.github.com/repos/simonw/datasette/subscribers\"",\n+ \""subscription_url\"": \""https://api.github.com/repos/simonw/datasette/subscription\"",\n+ \""commits_url\"": \""https://api.github.com/repos/simonw/datasette/commits{/sha}\"",\n+ \""git_commits_url\"": \""https://api.github.com/repos/simonw/datasette/git/commits{/sha}\"",\n+ \""comments_url\"": \""https://api.github.com/repos/simonw/datasette/comments{/number}\"",\n+ \""issue_comment_url\"": \""https://api.github.com/repos/simonw/datasette/issues/comments{/number}\"",\n+ \""contents_url\"": \""https://api.github.com/repos/simonw/datasette/contents/{+path}\"",\n+ \""compare_url\"": \""https://api.github.com/repos/simonw/datasette/compare/{base}...{head}\"",\n+ \""merges_url\"": \""https://api.github.com/repos/simonw/datasette/merges\"",\n+ \""archive_url\"": \""https://api.github.com/repos/simonw/datasette/{archive_format}{/ref}\"",\n+ \""downloads_url\"": \""https://api.github.com/repos/simonw/datasette/downloads\"",\n+ \""issues_url\"": \""https://api.github.com/repos/simonw/datasette/issues{/number}\"",\n+ \""pulls_url\"": \""https://api.github.com/repos/simonw/datasette/pulls{/number}\"",\n+ \""milestones_url\"": \""https://api.github.com/repos/simonw/datasette/milestones{/number}\"",\n+ \""notifications_url\"": \""https://api.github.com/repos/simonw/datasette/notifications{?since,all,participating}\"",\n+ \""labels_url\"": \""https://api.github.com/repos/simonw/datasette/labels{/name}\"",\n+ \""releases_url\"": \""https://api.github.com/repos/simonw/datasette/releases{/id}\"",\n+ \""deployments_url\"": \""https://api.github.com/repos/simonw/datasette/deployments\"",\n+ \""created_at\"": \""2017-10-23T00:39:03Z\"",\n+ \""updated_at\"": \""2020-07-27T20:42:15Z\"",\n+ \""pushed_at\"": \""2020-07-26T01:21:05Z\"",\n+ \""git_url\"": \""git://github.com/simonw/datasette.git\"",\n+ \""ssh_url\"": \""git@github.com:simonw/datasette.git\"",\n+ \""clone_url\"": \""https://github.com/simonw/datasette.git\"",\n+ \""svn_url\"": \""https://github.com/simonw/datasette\"",\n+ \""homepage\"": \""http://datasette.readthedocs.io/\"",\n+ \""size\"": 3487,\n+ \""stargazers_count\"": 3642,\n+ \""watchers_count\"": 3642,\n+ \""language\"": \""Python\"",\n+ \""has_issues\"": true,\n+ \""has_projects\"": false,\n+ \""has_downloads\"": true,\n+ \""has_wiki\"": true,\n+ \""has_pages\"": false,\n+ \""forks_count\"": 206,\n+ \""mirror_url\"": null,\n+ \""archived\"": false,\n+ \""disabled\"": false,\n+ \""open_issues_count\"": 190,\n+ \""license\"": {\n+ \""key\"": \""apache-2.0\"",\n+ \""name\"": \""Apache License 2.0\"",\n+ \""spdx_id\"": \""Apache-2.0\"",\n+ \""url\"": \""https://api.github.com/licenses/apache-2.0\"",\n+ \""node_id\"": \""MDc6TGljZW5zZTI=\""\n+ },\n+ \""forks\"": 206,\n+ \""open_issues\"": 190,\n+ \""watchers\"": 3642,\n+ \""default_branch\"": \""master\""\n+ }\n+ },\n+ \""_links\"": {\n+ \""self\"": {\n+ \""href\"": \""https://api.github.com/repos/simonw/datasette/pulls/571\""\n+ },\n+ \""html\"": {\n+ \""href\"": \""https://github.com/simonw/datasette/pull/571\""\n+ },\n+ \""issue\"": {\n+ \""href\"": \""https://api.github.com/repos/simonw/datasette/issues/571\""\n+ },\n+ \""comments\"": {\n+ \""href\"": \""https://api.github.com/repos/simonw/datasette/issues/571/comments\""\n+ },\n+ \""review_comments\"": {\n+ \""href\"": \""https://api.github.com/repos/simonw/datasette/pulls/571/comments\""\n+ },\n+ \""review_comment\"": {\n+ \""href\"": \""https://api.github.com/repos/simonw/datasette/pulls/comments{/number}\""\n+ },\n+ \""commits\"": {\n+ \""href\"": \""https://api.github.com/repos/simonw/datasette/pulls/571/commits\""\n+ },\n+ \""statuses\"": {\n+ \""href\"": \""https://api.github.com/repos/simonw/datasette/statuses/a85239f69261c10f1a9f90514c8b5d113cb94585\""\n+ }\n+ },\n+ \""author_association\"": \""OWNER\"",\n+ \""active_lock_reason\"": null,\n+ \""merged\"": true,\n+ \""mergeable\"": null,\n+ \""rebaseable\"": null,\n+ \""mergeable_state\"": \""unknown\"",\n+ \""merged_by\"": {"", ""path"": ""tests/pull_requests.json"", ""position"": 342, ""original_position"": 342, ""commit_id"": ""3a0d5c498f9faae4e40aab204cd01b965a4f61f3"", ""user"": { ""login"": ""simonw"", ""id"": 9599, ""node_id"": ""MDQ6VXNlcjk1OTk="", ""avatar_url"": ""https://avatars0.githubusercontent.com/u/9599?u=5968723deb1a55b82620e106f5ca58e9b11a0942&v=4"", ""gravatar_id"": """", ""url"": ""https://api.github.com/users/simonw"", ""html_url"": ""https://github.com/simonw"", ""followers_url"": ""https://api.github.com/users/simonw/followers"", ""following_url"": ""https://api.github.com/users/simonw/following{/other_user}"", ""gists_url"": ""https://api.github.com/users/simonw/gists{/gist_id}"", ""starred_url"": ""https://api.github.com/users/simonw/starred{/owner}{/repo}"", ""subscriptions_url"": ""https://api.github.com/users/simonw/subscriptions"", ""organizations_url"": ""https://api.github.com/users/simonw/orgs"", ""repos_url"": ""https://api.github.com/users/simonw/repos"", ""events_url"": ""https://api.github.com/users/simonw/events{/privacy}"", ""received_events_url"": ""https://api.github.com/users/simonw/received_events"", ""type"": ""User"", ""site_admin"": false }, ""body"": ""Running this should create a `merged_by` column on the `pull_requests` table which is a foreign key to the `users` table."", ""created_at"": ""2020-10-06T21:22:47Z"", ""updated_at"": ""2020-10-20T20:56:33Z"", ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500603838"", ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"", ""author_association"": ""MEMBER"", ""_links"": { ""self"": { ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500603838"" }, ""html"": { ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500603838"" }, ""pull_request"": { ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"" } }, ""original_commit_id"": ""4f33b850bd37829262dd29e1c520afffebedc19c"" }, { ""id"": 500606198, ""node_id"": ""MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDUwMDYwNjE5OA=="", ""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500606198"", ""pull_request_review_id"": 503368921, ""diff_hunk"": ""@@ -0,0 +1,124 @@\n+from github_to_sqlite import utils\n+import pytest\n+import pathlib\n+import sqlite_utils\n+from sqlite_utils.db import ForeignKey\n+import json\n+\n+\n+@pytest.fixture\n+def pull_requests():\n+ return json.load(open(pathlib.Path(__file__).parent / \""pull_requests.json\""))\n+\n+\n+@pytest.fixture\n+def db(pull_requests):\n+ db = sqlite_utils.Database(memory=True)\n+ db[\""repos\""].insert(\n+ {\""id\"": 1},\n+ pk=\""id\"",\n+ columns={\""organization\"": int, \""topics\"": str, \""name\"": str, \""description\"": str},\n+ )\n+ utils.save_pull_requests(db, pull_requests, {\""id\"": 1})\n+ return db\n+\n+\n+def test_tables(db):\n+ assert {\""pull_requests\"", \""users\"", \""repos\"", \""milestones\""} == set(\n+ db.table_names()\n+ )\n+ assert {\n+ ForeignKey(\n+ table=\""pull_requests\"", column=\""repo\"", other_table=\""repos\"", other_column=\""id\""\n+ ),\n+ ForeignKey(\n+ table=\""pull_requests\"",\n+ column=\""milestone\"",\n+ other_table=\""milestones\"",\n+ other_column=\""id\"",\n+ ),\n+ ForeignKey(\n+ table=\""pull_requests\"", column=\""assignee\"", other_table=\""users\"", other_column=\""id\""\n+ ),\n+ ForeignKey(\n+ table=\""pull_requests\"", column=\""user\"", other_table=\""users\"", other_column=\""id\""\n+ ),\n+ } == set(db[\""pull_requests\""].foreign_keys)\n+\n+\n+def test_pull_requests(db):\n+ pull_request_rows = list(db[\""pull_requests\""].rows)\n+ assert [\n+ {\n+ 'id': 313384926,\n+ 'node_id': 'MDExOlB1bGxSZXF1ZXN0MzEzMzg0OTI2',\n+ 'number': 571,\n+ 'state': 'closed',\n+ 'locked': 0,\n+ 'title': 'detect_fts now works with alternative table escaping',\n+ 'user': 9599,\n+ 'body': 'Fixes #570',\n+ 'created_at': '2019-09-03T00:23:39Z',\n+ 'updated_at': '2019-09-03T00:32:28Z',\n+ 'closed_at': '2019-09-03T00:32:28Z',\n+ 'merged_at': '2019-09-03T00:32:28Z',\n+ 'merge_commit_sha': '2dc5c8dc259a0606162673d394ba8cc1c6f54428',\n+ 'assignee': None,\n+ 'milestone': None,\n+ 'draft': 0,\n+ 'head': 'a85239f69261c10f1a9f90514c8b5d113cb94585',\n+ 'base': 'f04deebec4f3842f7bd610cd5859de529f77d50e',\n+ 'author_association': 'OWNER',\n+ 'merged': 1,\n+ 'mergeable': None,\n+ 'rebaseable': None,\n+ 'mergeable_state': 'unknown',\n+ 'merged_by': '{\""login\"": \""simonw\"", \""id\"": 9599, \""node_id\"": \""MDQ6VXNlcjk1OTk=\"", \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"", \""gravatar_id\"": \""\"", \""url\"": \""https://api.github.com/users/simonw\"", \""html_url\"": \""https://github.com/simonw\"", \""followers_url\"": \""https://api.github.com/users/simonw/followers\"", \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"", \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"", \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"", \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"", \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"", \""repos_url\"": \""https://api.github.com/users/simonw/repos\"", \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"", \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"", \""type\"": \""User\"", \""site_admin\"": false}',"", ""path"": ""tests/test_pull_requests.py"", ""position"": null, ""original_position"": 76, ""commit_id"": ""3a0d5c498f9faae4e40aab204cd01b965a4f61f3"", ""user"": { ""login"": ""simonw"", ""id"": 9599, ""node_id"": ""MDQ6VXNlcjk1OTk="", ""avatar_url"": ""https://avatars0.githubusercontent.com/u/9599?u=5968723deb1a55b82620e106f5ca58e9b11a0942&v=4"", ""gravatar_id"": """", ""url"": ""https://api.github.com/users/simonw"", ""html_url"": ""https://github.com/simonw"", ""followers_url"": ""https://api.github.com/users/simonw/followers"", ""following_url"": ""https://api.github.com/users/simonw/following{/other_user}"", ""gists_url"": ""https://api.github.com/users/simonw/gists{/gist_id}"", ""starred_url"": ""https://api.github.com/users/simonw/starred{/owner}{/repo}"", ""subscriptions_url"": ""https://api.github.com/users/simonw/subscriptions"", ""organizations_url"": ""https://api.github.com/users/simonw/orgs"", ""repos_url"": ""https://api.github.com/users/simonw/repos"", ""events_url"": ""https://api.github.com/users/simonw/events{/privacy}"", ""received_events_url"": ""https://api.github.com/users/simonw/received_events"", ""type"": ""User"", ""site_admin"": false }, ""body"": ""See above - this should be 9599, an integer reference to the row in the users table."", ""created_at"": ""2020-10-06T21:27:43Z"", ""updated_at"": ""2020-10-20T20:56:33Z"", ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500606198"", ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"", ""author_association"": ""MEMBER"", ""_links"": { ""self"": { ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500606198"" }, ""html"": { ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500606198"" }, ""pull_request"": { ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"" } }, ""original_commit_id"": ""4f33b850bd37829262dd29e1c520afffebedc19c"" }, { ""id"": 500606665, ""node_id"": ""MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDUwMDYwNjY2NQ=="", ""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500606665"", ""pull_request_review_id"": 503368921, ""diff_hunk"": ""@@ -0,0 +1,124 @@\n+from github_to_sqlite import utils\n+import pytest\n+import pathlib\n+import sqlite_utils\n+from sqlite_utils.db import ForeignKey\n+import json\n+\n+\n+@pytest.fixture\n+def pull_requests():\n+ return json.load(open(pathlib.Path(__file__).parent / \""pull_requests.json\""))\n+\n+\n+@pytest.fixture\n+def db(pull_requests):\n+ db = sqlite_utils.Database(memory=True)\n+ db[\""repos\""].insert(\n+ {\""id\"": 1},\n+ pk=\""id\"",\n+ columns={\""organization\"": int, \""topics\"": str, \""name\"": str, \""description\"": str},\n+ )\n+ utils.save_pull_requests(db, pull_requests, {\""id\"": 1})\n+ return db\n+\n+\n+def test_tables(db):\n+ assert {\""pull_requests\"", \""users\"", \""repos\"", \""milestones\""} == set(\n+ db.table_names()\n+ )\n+ assert {\n+ ForeignKey(\n+ table=\""pull_requests\"", column=\""repo\"", other_table=\""repos\"", other_column=\""id\""\n+ ),\n+ ForeignKey(\n+ table=\""pull_requests\"",\n+ column=\""milestone\"",\n+ other_table=\""milestones\"",\n+ other_column=\""id\"",\n+ ),\n+ ForeignKey(\n+ table=\""pull_requests\"", column=\""assignee\"", other_table=\""users\"", other_column=\""id\""\n+ ),\n+ ForeignKey(\n+ table=\""pull_requests\"", column=\""user\"", other_table=\""users\"", other_column=\""id\""\n+ ),\n+ } == set(db[\""pull_requests\""].foreign_keys)\n+\n+\n+def test_pull_requests(db):\n+ pull_request_rows = list(db[\""pull_requests\""].rows)\n+ assert [\n+ {\n+ 'id': 313384926,"", ""path"": ""tests/test_pull_requests.py"", ""position"": null, ""original_position"": 53, ""commit_id"": ""3a0d5c498f9faae4e40aab204cd01b965a4f61f3"", ""user"": { ""login"": ""simonw"", ""id"": 9599, ""node_id"": ""MDQ6VXNlcjk1OTk="", ""avatar_url"": ""https://avatars0.githubusercontent.com/u/9599?u=5968723deb1a55b82620e106f5ca58e9b11a0942&v=4"", ""gravatar_id"": """", ""url"": ""https://api.github.com/users/simonw"", ""html_url"": ""https://github.com/simonw"", ""followers_url"": ""https://api.github.com/users/simonw/followers"", ""following_url"": ""https://api.github.com/users/simonw/following{/other_user}"", ""gists_url"": ""https://api.github.com/users/simonw/gists{/gist_id}"", ""starred_url"": ""https://api.github.com/users/simonw/starred{/owner}{/repo}"", ""subscriptions_url"": ""https://api.github.com/users/simonw/subscriptions"", ""organizations_url"": ""https://api.github.com/users/simonw/orgs"", ""repos_url"": ""https://api.github.com/users/simonw/repos"", ""events_url"": ""https://api.github.com/users/simonw/events{/privacy}"", ""received_events_url"": ""https://api.github.com/users/simonw/received_events"", ""type"": ""User"", ""site_admin"": false }, ""body"": ""Minor detail: I use Black for this repo, which requires double quotes - running \""black .\"" in the root directory (with the latest version of Black) should handle this for you."", ""created_at"": ""2020-10-06T21:28:31Z"", ""updated_at"": ""2020-10-20T20:56:33Z"", ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500606665"", ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"", ""author_association"": ""MEMBER"", ""_links"": { ""self"": { ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500606665"" }, ""html"": { ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500606665"" }, ""pull_request"": { ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"" } }, ""original_commit_id"": ""4f33b850bd37829262dd29e1c520afffebedc19c"" } ] ``` That's a lot more interesting.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",664485022, https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-735483820,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46,735483820,MDEyOklzc3VlQ29tbWVudDczNTQ4MzgyMA==,9599,2020-11-30T00:27:47Z,2020-11-30T00:27:47Z,MEMBER,"So it looks like anything that pulls reviews needs to pull each review, then for each one pull the comments. I'm going to consider this blocked on smarter rate limit handling in #51.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",664485022, https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-1359468823,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46,1359468823,IC_kwDODFdgUs5RB9kX,1839645,2022-12-20T14:39:39Z,2022-12-20T14:40:15Z,NONE,"Just a quick +1 to this one from me - I would like to do a better job of tracking who is reviewing one another's pull requests in repositories, since this is a specific kind of maintenance work that I think often goes unrewarded. I can't seem to figure this out just by looking at the `pull_request` or `issue_comments` tables, so I think it would be helpful to support PR reviews natively (even if just for summary statistics). Alternatively if there is a way in the API to tell if an issue comment is part of a review, then perhaps you could quickly calculate the number of unique reviews that an author performed. But that was beyond my SQL-foo :-) ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",664485022, https://github.com/dogsheep/github-to-sqlite/issues/47#issuecomment-675523053,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/47,675523053,MDEyOklzc3VlQ29tbWVudDY3NTUyMzA1Mw==,9599,2020-08-18T14:45:53Z,2020-08-18T14:45:53Z,MEMBER,"``` % github-to-sqlite emojis emojis.db --fetch [########----------------------------] 397/1682 23% 00:03:43 ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681086659, https://github.com/dogsheep/github-to-sqlite/pull/48#issuecomment-704503719,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/48,704503719,MDEyOklzc3VlQ29tbWVudDcwNDUwMzcxOQ==,755825,2020-10-06T19:26:59Z,2020-10-06T19:26:59Z,CONTRIBUTOR,ref #46 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681228542, https://github.com/dogsheep/github-to-sqlite/pull/48#issuecomment-704553385,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/48,704553385,MDEyOklzc3VlQ29tbWVudDcwNDU1MzM4NQ==,9599,2020-10-06T21:07:44Z,2020-10-06T21:07:44Z,MEMBER,"Sorry for not looking at this sooner, trying it out now - pull request looks great!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681228542, https://github.com/dogsheep/github-to-sqlite/issues/5#issuecomment-549094229,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/5,549094229,MDEyOklzc3VlQ29tbWVudDU0OTA5NDIyOQ==,9599,2019-11-03T00:44:03Z,2019-11-03T00:44:03Z,MEMBER,Might not need an incomplete boolean - may be possible to handle this with `alter=True` and then by filtering for users with null values in certain columns.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493671014, https://github.com/dogsheep/github-to-sqlite/issues/5#issuecomment-602806530,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/5,602806530,MDEyOklzc3VlQ29tbWVudDYwMjgwNjUzMA==,9599,2020-03-23T19:23:30Z,2020-03-23T19:23:30Z,MEMBER,Yeah `alter=True` means that this isn't necessary.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493671014, https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693773191,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693773191,MDEyOklzc3VlQ29tbWVudDY5Mzc3MzE5MQ==,9599,2020-09-17T02:39:26Z,2020-09-17T02:39:26Z,MEMBER,I'm going to start with `github-to-sqlite get` and `github-to-sqlite post` - I may add `put` and suchlike later on.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756, https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693775622,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693775622,MDEyOklzc3VlQ29tbWVudDY5Mzc3NTYyMg==,9599,2020-09-17T02:48:34Z,2020-09-17T02:48:34Z,MEMBER,I'd like a `--paginate` option that does the same thing as https://github.com/simonw/paginate-json,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756, https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693788032,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693788032,MDEyOklzc3VlQ29tbWVudDY5Mzc4ODAzMg==,9599,2020-09-17T03:35:22Z,2020-09-17T03:35:22Z,MEMBER,Documentation: https://github.com/dogsheep/github-to-sqlite/blob/b02bf135485c0a7a3768868967f45a6b5e515289/README.md#making-authenticated-api-calls,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756, https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693788387,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693788387,MDEyOklzc3VlQ29tbWVudDY5Mzc4ODM4Nw==,9599,2020-09-17T03:36:47Z,2020-09-17T03:36:58Z,MEMBER,"Fun demo of the `--nl` option: github-to-sqlite get /users/simonw/repos --paginate --nl | sqlite-utils insert simonw.db repos - --nl ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756, https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693789129,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693789129,MDEyOklzc3VlQ29tbWVudDY5Mzc4OTEyOQ==,9599,2020-09-17T03:40:01Z,2020-09-17T03:40:01Z,MEMBER,"Bug with endpoints that return dictionaries rather than arrays: ``` github-to-sqlite get /users/simonw [ ""login"", ""id"", ""node_id"", ""avatar_url"", ""gravatar_id"", ""url"", ""html_url"", ""followers_url"", ""following_url"", ""gists_url"", ""starred_url"", ""subscriptions_url"", ""organizations_url"", ""repos_url"", ""events_url"", ""received_events_url"", ""type"", ""site_admin"", ""name"", ""company"", ""blog"", ""location"", ""email"", ""hireable"", ""bio"", ""twitter_username"", ""public_repos"", ""public_gists"", ""followers"", ""following"", ""created_at"", ""updated_at"" ] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756, https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693794700,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693794700,MDEyOklzc3VlQ29tbWVudDY5Mzc5NDcwMA==,9599,2020-09-17T04:02:39Z,2020-09-17T04:02:39Z,MEMBER,It would be useful if you could pass an `--accept` option to this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756, https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-711569063,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,711569063,MDEyOklzc3VlQ29tbWVudDcxMTU2OTA2Mw==,9599,2020-10-19T05:01:29Z,2020-10-19T05:01:29Z,MEMBER,"Demo of `--accept`: github-to-sqlite get /repos/simonw/datasette/readme --accept 'application/vnd.github.VERSION.html' ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756, https://github.com/dogsheep/github-to-sqlite/issues/51#issuecomment-735484186,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/51,735484186,MDEyOklzc3VlQ29tbWVudDczNTQ4NDE4Ng==,9599,2020-11-30T00:29:31Z,2020-11-30T00:29:31Z,MEMBER,"This just caused a failure in deploying the demo: https://github.com/dogsheep/github-to-sqlite/runs/1471304407?check_suite_focus=true ``` File ""/opt/hostedtoolcache/Python/3.8.6/x64/bin/github-to-sqlite"", line 33, in sys.exit(load_entry_point('github-to-sqlite', 'console_scripts', 'github-to-sqlite')()) File ""/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/click/core.py"", line 829, in __call__ return self.main(*args, **kwargs) File ""/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/click/core.py"", line 782, in main rv = self.invoke(ctx) File ""/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/click/core.py"", line 610, in invoke return callback(*args, **kwargs) File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py"", line 142, in issue_comments for comment in utils.fetch_issue_comments(repo, token, issue): File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 380, in fetch_issue_comments for comments in paginate(url, headers): File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 472, in paginate raise GitHubError.from_response(response) github_to_sqlite.utils.GitHubError: ('API rate limit exceeded for user ID 9599.', 403) Error: Process completed with exit code 1. ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703246031, https://github.com/dogsheep/github-to-sqlite/issues/51#issuecomment-770150526,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/51,770150526,MDEyOklzc3VlQ29tbWVudDc3MDE1MDUyNg==,22578954,2021-01-30T03:44:19Z,2021-01-30T03:47:24Z,CONTRIBUTOR,I don't have much experience with github's rate limiting. In my day job we use the [tenacity library](https://github.com/jd/tenacity) to handle http errors we get.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703246031, https://github.com/dogsheep/github-to-sqlite/issues/51#issuecomment-1208757153,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/51,1208757153,IC_kwDODFdgUs5IDCuh,9020979,2022-08-09T00:29:44Z,2022-08-09T00:29:44Z,NONE,"I've been looking into how to to get this data out of Github (especially now there are ""secondary rate limits"" without an advertised allowance separate from the regular rate limits. I've had decent success with the Airbyte github extractor (aside from one data quality issue https://github.com/airbytehq/airbyte/pull/15420 ). Airbyte splits data extraction between the GraphQL and REST endpoints depending on the resource type, but they're very comprehensive. https://github.com/airbytehq/airbyte/blob/306a75ef5370728e0912cf52a1a898a530db0c90/airbyte-integrations/connectors/source-github/source_github/streams.py#L22-L122 Before this, I tried a few solutions in my own custom wrapper mentioned in this thread + its children https://github.com/PyGithub/PyGithub/issues/1989 , but they weren't working as expected.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703246031, https://github.com/dogsheep/github-to-sqlite/issues/51#issuecomment-1279224780,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/51,1279224780,IC_kwDODFdgUs5MP2vM,7908073,2022-10-14T16:34:07Z,2022-10-14T16:34:07Z,NONE,"also, it says that authenticated requests have a much higher ""rate limit"". Unauthenticated requests only get 60 req/hour ?? seems more like a quota than a ""rate limit"" (although I guess that is semantic equivalence) You would want to use `x-ratelimit-reset` ``` time.sleep(r['x-ratelimit-reset'] + 1 - time.time()) ``` But a more complete solution would bring authenticated requests to the other subcommands. I'm surprised only `github-to-sqlite get` is using the `--auth=` CLI flag","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703246031, https://github.com/dogsheep/github-to-sqlite/issues/53#issuecomment-735485677,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/53,735485677,MDEyOklzc3VlQ29tbWVudDczNTQ4NTY3Nw==,9599,2020-11-30T00:36:09Z,2020-11-30T00:36:09Z,MEMBER,Given rate limits (see #51) this command might be better implemented by running a `git clone` into a temporary directory - doing so would retrieve all of the files in one go.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",753000405, https://github.com/dogsheep/github-to-sqlite/issues/54#issuecomment-735464438,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54,735464438,MDEyOklzc3VlQ29tbWVudDczNTQ2NDQzOA==,9599,2020-11-29T21:57:08Z,2020-11-29T21:57:08Z,MEMBER,Inspired by this tweet from Michael Heap https://twitter.com/mheap/status/1333108608817631238,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",753026003, https://github.com/dogsheep/github-to-sqlite/issues/54#issuecomment-735464493,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54,735464493,MDEyOklzc3VlQ29tbWVudDczNTQ2NDQ5Mw==,9599,2020-11-29T21:57:32Z,2020-11-29T21:57:32Z,MEMBER,`$ github-to-sqlite workflows github.db simonw/datasette dogsheep/github-to-sqlite`,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",753026003, https://github.com/dogsheep/github-to-sqlite/issues/54#issuecomment-735465708,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54,735465708,MDEyOklzc3VlQ29tbWVudDczNTQ2NTcwOA==,9599,2020-11-29T22:08:46Z,2020-11-29T22:08:46Z,MEMBER,"Demo: - https://github-to-sqlite.dogsheep.net/github/steps?_facet=repo - https://github-to-sqlite.dogsheep.net/github/workflows - https://github-to-sqlite.dogsheep.net/github/jobs","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",753026003, https://github.com/dogsheep/github-to-sqlite/issues/58#issuecomment-746734412,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/58,746734412,MDEyOklzc3VlQ29tbWVudDc0NjczNDQxMg==,9599,2020-12-16T17:58:56Z,2020-12-16T17:58:56Z,MEMBER,"I'm going to rewrite those `` links to `` - but only if a corresponding `id=""user-content-filtering-tables""` element exists.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",769150394, https://github.com/dogsheep/github-to-sqlite/issues/58#issuecomment-746735889,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/58,746735889,MDEyOklzc3VlQ29tbWVudDc0NjczNTg4OQ==,9599,2020-12-16T17:59:50Z,2020-12-16T17:59:50Z,MEMBER,"I don't want to add a full HTML parser (like BeautifulSoup) as a dependency for this feature. Since the HTML comes from a single, trusted source (GitHub) I could probably handle this using [regular expressions](https://stackoverflow.com/a/1732454).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",769150394, https://github.com/dogsheep/github-to-sqlite/pull/59#issuecomment-751375487,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/59,751375487,MDEyOklzc3VlQ29tbWVudDc1MTM3NTQ4Nw==,631242,2020-12-26T17:08:44Z,2020-12-26T17:08:44Z,CONTRIBUTOR,"Hi @simonw, do I need to do anything else for this PR to be considered to be included? I've tried using this project and it is quite nice to be able to explore a repository, but noticed that a couple commands don't allow you to use authorization from the environment variable.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",771872303, https://github.com/dogsheep/github-to-sqlite/pull/59#issuecomment-844250232,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/59,844250232,MDEyOklzc3VlQ29tbWVudDg0NDI1MDIzMg==,9599,2021-05-19T16:08:10Z,2021-05-19T16:08:10Z,MEMBER,Thanks for catching this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",771872303, https://github.com/dogsheep/github-to-sqlite/pull/59#issuecomment-846413174,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/59,846413174,MDEyOklzc3VlQ29tbWVudDg0NjQxMzE3NA==,631242,2021-05-22T14:06:19Z,2021-05-22T14:06:19Z,CONTRIBUTOR,Thanks Simon!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",771872303, https://github.com/dogsheep/github-to-sqlite/issues/6#issuecomment-541387822,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/6,541387822,MDEyOklzc3VlQ29tbWVudDU0MTM4NzgyMg==,9599,2019-10-13T05:27:39Z,2019-10-13T05:27:39Z,MEMBER,This should be fixed by https://github.com/dogsheep/github-to-sqlite/commit/552543a74970f8a3a3f87f887be23a0c6eb1cb5b,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",504238461, https://github.com/dogsheep/github-to-sqlite/issues/6#issuecomment-541387941,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/6,541387941,MDEyOklzc3VlQ29tbWVudDU0MTM4Nzk0MQ==,9599,2019-10-13T05:30:19Z,2019-10-13T05:30:19Z,MEMBER,Fix released in 0.5: https://github.com/dogsheep/github-to-sqlite/releases/tag/0.5,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",504238461, https://github.com/dogsheep/github-to-sqlite/issues/60#issuecomment-770069864,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/60,770069864,MDEyOklzc3VlQ29tbWVudDc3MDA2OTg2NA==,22578954,2021-01-29T21:52:05Z,2021-02-12T18:29:43Z,CONTRIBUTOR,"For the purposes below I am assuming the organization I would get all the repositories and their related commits from is called `gh-organization`. The github's owner id of gh-orgnization is `123456789`. ```bash github-to-sqlite repos github.db gh-organization ``` I'm on a windows computer running git bash to be able to use the `|` command. This works for me ```bash sqlite3 github.db ""SELECT full_name FROM repos WHERE owner = '123456789';"" | tr '\n\r' ' ' | xargs | { read repos; github-to-sqlite commits github.db $repos; } ``` On a pure linux system I think this would work because the new line character is normally `\n` ```bash sqlite3 github.db ""SELECT full_name FROM repos WHERE owner = '123456789';"" | tr '\n' ' ' | xargs | { read repos; github-to-sqlite commits github.db $repos; }` ``` As expected I ran into rate limit issues #51 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",797097140, https://github.com/dogsheep/github-to-sqlite/issues/60#issuecomment-770071568,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/60,770071568,MDEyOklzc3VlQ29tbWVudDc3MDA3MTU2OA==,9599,2021-01-29T21:56:15Z,2021-01-29T21:56:15Z,MEMBER,"I really like the way you're using pipes here - really smart. It's similar to how I build the demo database in this GitHub Actions workflow: https://github.com/dogsheep/github-to-sqlite/blob/62dfd3bc4014b108200001ef4bc746feb6f33b45/.github/workflows/deploy-demo.yml#L52-L82 `twitter-to-sqlite` actually has a mechanism for doing this kind of thing, documented at https://github.com/dogsheep/twitter-to-sqlite#providing-input-from-a-sql-query-with---sql-and---attach It lets you do things like: ``` $ twitter-to-sqlite users-lookup my.db --sql=""select follower_id from following"" --ids ``` Maybe I should add something similar to `github-to-sqlite`? Feels like it could be really useful.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",797097140, https://github.com/dogsheep/github-to-sqlite/issues/60#issuecomment-770112248,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/60,770112248,MDEyOklzc3VlQ29tbWVudDc3MDExMjI0OA==,22578954,2021-01-30T00:01:03Z,2021-01-30T01:14:42Z,CONTRIBUTOR,"Yes that would be cool! I wouldn't mind helping. Is this the meat of it? https://github.com/dogsheep/twitter-to-sqlite/blob/21fc1cad6dd6348c67acff90a785b458d3a81275/twitter_to_sqlite/utils.py#L512 It looks like the cli option is added with this decorator : https://github.com/dogsheep/twitter-to-sqlite/blob/21fc1cad6dd6348c67acff90a785b458d3a81275/twitter_to_sqlite/cli.py#L14 I looked a bit at utils.py in the GitHub repository. I was surprised at the amount of manual mapping of the API response you had to do to get this to work.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",797097140, https://github.com/dogsheep/github-to-sqlite/pull/61#issuecomment-844249385,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/61,844249385,MDEyOklzc3VlQ29tbWVudDg0NDI0OTM4NQ==,9599,2021-05-19T16:07:06Z,2021-05-19T16:07:06Z,MEMBER,Thanks!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",797108702, https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-860895838,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,860895838,MDEyOklzc3VlQ29tbWVudDg2MDg5NTgzOA==,231498,2021-06-14T18:23:21Z,2021-06-14T21:37:35Z,NONE,"i have a basic working version at https://github.com/khimaros/github-to-sqlite this can be tested with `github-to-sqlite events.db khimaros/events` caveat: the GitHub API doesn't seem to provide a complete history of events.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216, https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861035862,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,861035862,MDEyOklzc3VlQ29tbWVudDg2MTAzNTg2Mg==,231498,2021-06-14T22:29:20Z,2021-06-14T22:29:20Z,NONE,"it looks like the v4 GraphQL API is the only way to get data beyond 90 days from GitHub. this is significant change, but may be worth considering in the future.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216, https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861041597,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,861041597,MDEyOklzc3VlQ29tbWVudDg2MTA0MTU5Nw==,9599,2021-06-14T22:44:54Z,2021-06-14T22:44:54Z,MEMBER,Have you found a way to access events in GraphQL? I can only see way to access a timeline of events for a single issue or a single pull request. See also https://github.community/t/get-event-equivalent-for-v4/13600/2,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216, https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861042050,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,861042050,MDEyOklzc3VlQ29tbWVudDg2MTA0MjA1MA==,9599,2021-06-14T22:45:42Z,2021-06-14T22:45:42Z,MEMBER,I'm definitely interested in supporting events in this tool - see #14.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216, https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861087651,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,861087651,MDEyOklzc3VlQ29tbWVudDg2MTA4NzY1MQ==,231498,2021-06-15T00:48:37Z,2021-06-15T00:48:37Z,NONE,"@simonw -- i've created an omega-query that fetched most of what was interesting to me for a single user. found by poking around in the ""Explorer"" tab in https://docs.github.com/en/graphql/overview/explorer note: pagination is still required via `first` and `last` but it seems to allow unlimited history. ``` query MyQuery { __typename user(login: """") { id pinnedItems(first: 100) { edges { node } } pullRequests(first: 100) { nodes { body title state createdAt } } createdAt issues(first: 100) { pageInfo { endCursor startCursor } nodes { title url createdAt body } } issueComments(first: 100) { edges { node { id updatedAt url body } } } repositories(first: 100) { nodes { createdAt description parent { name } pinnedIssues(first: 100) { edges { node { id } } } pinnedDiscussions(first: 100) { edges { node { id } } } } } starredRepositories(first: 100) { edges { node { id } } } } } ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216, https://github.com/dogsheep/github-to-sqlite/pull/65#issuecomment-885964242,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/65,885964242,IC_kwDODFdgUs40zr3S,231498,2021-07-23T23:45:35Z,2021-07-23T23:45:35Z,NONE,@simonw is this PR of interest to you?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",923270900, https://github.com/dogsheep/github-to-sqlite/pull/65#issuecomment-1266141699,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/65,1266141699,IC_kwDODFdgUs5Ld8oD,231498,2022-10-03T22:35:03Z,2022-10-03T22:35:03Z,NONE,"@simonw rebased against latest, please let me know if i should drop this PR.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",923270900, https://github.com/dogsheep/github-to-sqlite/pull/66#issuecomment-929651819,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/66,929651819,IC_kwDODFdgUs43aVxr,30531572,2021-09-28T21:50:31Z,2021-09-28T21:50:31Z,NONE,@simonw any feedback/thoughts? ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",975161924, https://github.com/dogsheep/github-to-sqlite/issues/69#issuecomment-985928838,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/69,985928838,IC_kwDODFdgUs46xBSG,9599,2021-12-04T00:34:52Z,2021-12-04T00:34:52Z,MEMBER,"First attempt at this: ```sql select 'issues' as ""table"", id, node_id, title, user, created_at, body, repo from issues union all select 'issue_comments' as ""table"", issue_comments.id, issue_comments.node_id, '' as title, issue_comments.user, issue_comments.created_at, issue_comments.body, issues.repo from issue_comments join issues on issues.id = issue_comments.issue order by created_at desc ``` https://github-to-sqlite.dogsheep.net/github?sql=select+%27issues%27+as+%22table%22%2C+id%2C+node_id%2C+title%2C+user%2C+created_at%2C+body%2C+repo%0D%0Afrom+issues%0D%0Aunion+all%0D%0Aselect+%27issue_comments%27+as+%22table%22%2C+issue_comments.id%2C+issue_comments.node_id%2C+%27%27+as+title%2C+issue_comments.user%2C+issue_comments.created_at%2C+issue_comments.body%2C+issues.repo%0D%0Afrom+issue_comments+join+issues+on+issues.id+%3D+issue_comments.issue%0D%0Aorder+by+created_at+desc","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1071071397, https://github.com/dogsheep/github-to-sqlite/issues/7#issuecomment-541721437,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/7,541721437,MDEyOklzc3VlQ29tbWVudDU0MTcyMTQzNw==,9599,2019-10-14T14:44:12Z,2019-10-14T14:44:12Z,MEMBER,Docs: https://github.com/dogsheep/github-to-sqlite/blob/0.5/README.md#retrieving-issue-comments-for-a-repository,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506276893, https://github.com/dogsheep/github-to-sqlite/issues/72#issuecomment-1105474232,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/72,1105474232,IC_kwDODFdgUs5B5DK4,9599,2022-04-21T17:02:15Z,2022-04-21T17:02:15Z,MEMBER,"That's interesting - yeah it looks like the number of pages can be derived from the `Link` header, which is enough information to show a progress bar, probably using Click just to avoid adding another dependency. https://docs.github.com/en/rest/guides/traversing-with-pagination","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1211283427, https://github.com/dogsheep/github-to-sqlite/pull/73#issuecomment-1188223299,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/73,1188223299,IC_kwDODFdgUs5G0tlD,9599,2022-07-18T19:40:06Z,2022-07-18T19:40:06Z,MEMBER,Thanks!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1261884917, https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188223933,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74,1188223933,IC_kwDODFdgUs5G0tu9,9599,2022-07-18T19:40:50Z,2022-07-18T19:42:41Z,MEMBER,"Here's how the demo is deployed: https://github.com/dogsheep/github-to-sqlite/blob/dbac2e5dd8a562b45d8255a265859cf8020ca22a/.github/workflows/deploy-demo.yml#L103-L119 I'm suspicious of `py-gfm`, which is used like this: https://github.com/dogsheep/github-to-sqlite/blob/dbac2e5dd8a562b45d8255a265859cf8020ca22a/demo-metadata.json#L49-L51","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1308461063, https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188225625,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74,1188225625,IC_kwDODFdgUs5G0uJZ,9599,2022-07-18T19:41:52Z,2022-07-18T19:41:52Z,MEMBER,"https://github.com/Zopieux/py-gfm says that library is no longer maintained, and suggests https://github.com/Zopieux/pycmarkgfm as an alternative.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1308461063, https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188228964,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74,1188228964,IC_kwDODFdgUs5G0u9k,9599,2022-07-18T19:45:30Z,2022-07-18T19:47:35Z,MEMBER,"`pycmarkgfm` doesn't implement the Markdown plugin extension I was using. I'm going to drop the GFM rendering from the demo, and just treat it as regular markdown.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1308461063, https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188233729,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74,1188233729,IC_kwDODFdgUs5G0wIB,9599,2022-07-18T19:51:02Z,2022-07-18T19:51:02Z,MEMBER,Takes 30m to deploy the demo!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1308461063, https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188317682,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74,1188317682,IC_kwDODFdgUs5G1Eny,9599,2022-07-18T21:14:22Z,2022-07-18T21:14:22Z,MEMBER,That fixed it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1308461063, https://github.com/dogsheep/github-to-sqlite/pull/76#issuecomment-1238190601,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/76,1238190601,IC_kwDODFdgUs5JzUoJ,2757699,2022-09-06T13:58:20Z,2022-09-06T13:59:08Z,NONE,"Tested PR just now in private org, fetched >2k repos infos flawlessly! poetry run github-to-sqlite repos --organization github.db MYORG","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1363280254, https://github.com/dogsheep/github-to-sqlite/issues/79#issuecomment-1498167714,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/79,1498167714,IC_kwDODFdgUs5ZTDmi,9599,2023-04-05T21:12:55Z,2023-04-05T21:12:55Z,MEMBER,"It's a rate limiting problem: https://github.com/dogsheep/github-to-sqlite/actions/runs/4622674009/jobs/8175633155 ``` File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py"", line 407, in commits utils.save_commits(db, commits, repo_full[""id""]) File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 635, in save_commits for commit in commits: File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 417, in fetch_commits for commits in paginate(url, headers): File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 478, in paginate raise GitHubError.from_response(response) github_to_sqlite.utils.GitHubError: ('API rate limit exceeded for user ID 9599.', 403) Error: Process completed with exit code 1. ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1570375808, https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-549094195,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8,549094195,MDEyOklzc3VlQ29tbWVudDU0OTA5NDE5NQ==,9599,2019-11-03T00:43:16Z,2019-11-03T00:43:28Z,MEMBER,"Also need to take #5 into account - if this command creates incomplete user records, how do we repair them? And make sure that if we run this command first any future commands that populate users don't break (probably just a case of using `alter=True` in a few places).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516763727, https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-549230583,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8,549230583,MDEyOklzc3VlQ29tbWVudDU0OTIzMDU4Mw==,9599,2019-11-04T05:49:26Z,2019-11-04T05:49:26Z,MEMBER,Adding the view from #10 would be useful here too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516763727, https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-549233778,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8,549233778,MDEyOklzc3VlQ29tbWVudDU0OTIzMzc3OA==,9599,2019-11-04T06:14:40Z,2019-11-04T06:14:40Z,MEMBER,"Spotted a tricky problem: running `github-to-sqlite starred stargazers.db` results in an incomplete `simonw` record. It creates a proper record for me thanks to this bit: https://github.com/dogsheep/github-to-sqlite/blob/ea07274667a08c67907e8bfbbccb6f0fb95ce817/github_to_sqlite/cli.py#L120-L126 But then... when it gets to the `datasette` repository which I have starred it over-writes my full user record with one that's missing most of the details, thanks to this bit: https://github.com/dogsheep/github-to-sqlite/blob/ea07274667a08c67907e8bfbbccb6f0fb95ce817/github_to_sqlite/utils.py#L117-L124 I need to find a way of NOT over-writing a good record with a thinner one. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516763727, https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-594154644,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8,594154644,MDEyOklzc3VlQ29tbWVudDU5NDE1NDY0NA==,9599,2020-03-03T20:33:57Z,2020-03-03T20:33:57Z,MEMBER,`sqlite-utils` supports proper upserts now so this problem should be easy to fix.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516763727, https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-623006004,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8,623006004,MDEyOklzc3VlQ29tbWVudDYyMzAwNjAwNA==,9599,2020-05-02T20:00:26Z,2020-05-02T20:00:26Z,MEMBER,I'm abandoning this in favour of a new implementation.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516763727, https://github.com/dogsheep/google-takeout-to-sqlite/issues/10#issuecomment-1073152522,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/10,1073152522,IC_kwDODFE5qs4_9wIK,9290214,2022-03-20T02:38:07Z,2022-03-20T02:38:07Z,NONE,"[This line](https://github.com/dogsheep/google-takeout-to-sqlite/blob/e54e544427f1cc3ea8189f0e95f54046301a8645/google_takeout_to_sqlite/utils.py) needs to say `""MyActivity.json""` instead of `""My Activity.json""`. Google must have changed the file name.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1123393829, https://github.com/dogsheep/google-takeout-to-sqlite/issues/2#issuecomment-747126777,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/2,747126777,MDEyOklzc3VlQ29tbWVudDc0NzEyNjc3Nw==,9599,2020-12-17T00:36:52Z,2020-12-17T00:36:52Z,MEMBER,The memory profiler tricks I used in https://github.com/dogsheep/healthkit-to-sqlite/issues/7 could help figure out what's going on here.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",769376447, https://github.com/dogsheep/google-takeout-to-sqlite/issues/2#issuecomment-747130908,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/2,747130908,MDEyOklzc3VlQ29tbWVudDc0NzEzMDkwOA==,231498,2020-12-17T00:47:04Z,2020-12-17T00:47:43Z,NONE,"it looks like almost all of the memory consumption is coming from `json.load()`. another direction here may be to use the new ""Semantic Location History"" data which is already broken down by year and month. it also provides much more interesting data, such as estimated address, form of travel, etc.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",769376447, https://github.com/dogsheep/google-takeout-to-sqlite/issues/4#issuecomment-780817596,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4,780817596,MDEyOklzc3VlQ29tbWVudDc4MDgxNzU5Ng==,306240,2021-02-17T20:01:35Z,2021-02-17T20:01:35Z,NONE,I've got this almost working. Just needs some polish,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",778380836, https://github.com/dogsheep/google-takeout-to-sqlite/issues/4#issuecomment-781451701,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4,781451701,MDEyOklzc3VlQ29tbWVudDc4MTQ1MTcwMQ==,203343,2021-02-18T16:06:21Z,2021-02-18T16:06:21Z,NONE,Awesome!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",778380836, https://github.com/dogsheep/google-takeout-to-sqlite/issues/4#issuecomment-783688547,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4,783688547,MDEyOklzc3VlQ29tbWVudDc4MzY4ODU0Nw==,306240,2021-02-22T21:31:28Z,2021-02-22T21:31:28Z,NONE,"@Btibert3 I've opened a PR with my initial attempt at this. Would you be willing to give this a try? https://github.com/dogsheep/google-takeout-to-sqlite/pull/5","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",778380836, https://github.com/dogsheep/google-takeout-to-sqlite/issues/4#issuecomment-790198930,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4,790198930,MDEyOklzc3VlQ29tbWVudDc5MDE5ODkzMA==,203343,2021-03-04T00:58:40Z,2021-03-04T00:58:40Z,NONE,"I am just seeing this sorry, yes! I will kick the tires later on tonight. My apologies for the delay.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",778380836, https://github.com/dogsheep/google-takeout-to-sqlite/issues/4#issuecomment-790934616,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4,790934616,MDEyOklzc3VlQ29tbWVudDc5MDkzNDYxNg==,203343,2021-03-04T20:54:44Z,2021-03-04T20:54:44Z,NONE,"Sorry for the delay, I got sidetracked after class last night. I am getting the following error: ``` /content# google-takeout-to-sqlite mbox takeout.db Takeout/Mail/gmail.mbox Usage: google-takeout-to-sqlite [OPTIONS] COMMAND [ARGS]...Try 'google-takeout-to-sqlite --help' for help. Error: No such command 'mbox'. ``` On the box, I installed with pip after cloning: https://github.com/UtahDave/google-takeout-to-sqlite.git","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",778380836, https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-783794520,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,783794520,MDEyOklzc3VlQ29tbWVudDc4Mzc5NDUyMA==,306240,2021-02-23T01:13:54Z,2021-02-23T01:13:54Z,NONE,"Also, @simonw I created a test based off the existing tests. I think it's working correctly","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401, https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-784638394,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,784638394,MDEyOklzc3VlQ29tbWVudDc4NDYzODM5NA==,306240,2021-02-24T00:36:18Z,2021-02-24T00:36:18Z,NONE,I noticed that @simonw is using black for formatting. I ran black on my additions in this PR.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,