id,node_id,number,state,locked,title,user,user_label,body,created_at,updated_at,closed_at,merged_at,merge_commit_sha,assignee,assignee_label,milestone,milestone_label,draft,head,base,author_association,repo,repo_label,url,merged_by,merged_by_label,auto_merge
335980246,MDExOlB1bGxSZXF1ZXN0MzM1OTgwMjQ2,8,closed,0,"stargazers command, refs #4",9599,simonw,Needs tests. Refs #4.,2019-11-03T00:37:36Z,2020-05-02T20:00:27Z,2020-05-02T20:00:26Z,,db25bdf8cee4c3e2d730cf269eb9a903b51cdb41,,,,,0,ea07274667a08c67907e8bfbbccb6f0fb95ce817,ae9035f8fe5aff1c54bff4c6b4c2e808a44f0f2a,MEMBER,207052882,github-to-sqlite,https://github.com/dogsheep/github-to-sqlite/pull/8,,,
469651732,MDExOlB1bGxSZXF1ZXN0NDY5NjUxNzMy,48,closed,0,Add pull requests,755825,adamjonas,"ref #46 

Issues don't have merge information on them, which means that PRs need to be pulled separately.

Did my best to mimic the API of issues.",2020-08-18T17:58:44Z,2020-11-29T23:51:09Z,2020-11-29T23:51:09Z,2020-11-29T23:51:09Z,b37f55549461cfe0731b57623f315860b3db49d0,,,,,0,3a0d5c498f9faae4e40aab204cd01b965a4f61f3,16d271253f4ea71b261d2d228b926c7bc1a7e660,CONTRIBUTOR,207052882,github-to-sqlite,https://github.com/dogsheep/github-to-sqlite/pull/48,,,
543246535,MDExOlB1bGxSZXF1ZXN0NTQzMjQ2NTM1,59,closed,0,Remove unneeded exists=True for -a/--auth flag.,631242,frosencrantz,The file does not need to exist when using an environment variable.,2020-12-21T06:03:55Z,2021-05-22T14:06:19Z,2021-05-19T16:08:12Z,2021-05-19T16:08:12Z,70dffca351375e6f542969c72ebc43c6d393d99c,,,,,0,79745bed50b7344c5cbb17a08215dc20d58b9416,d19d7db034bf7c3adcae37b9ab6f365d569605b3,CONTRIBUTOR,207052882,github-to-sqlite,https://github.com/dogsheep/github-to-sqlite/pull/59,,,
564172140,MDExOlB1bGxSZXF1ZXN0NTY0MTcyMTQw,61,closed,0,fixing typo in get cli help text,22578954,daniel-butler,,2021-01-29T18:57:04Z,2021-05-19T16:07:09Z,2021-05-19T16:07:09Z,2021-05-19T16:07:09Z,ba8cf3e9bb5f4f8740bd4b9eed28f1464d7f6b9a,,,,,0,7ac6efc3a873facafa72192b58e28c6e8a79f744,62dfd3bc4014b108200001ef4bc746feb6f33b45,CONTRIBUTOR,207052882,github-to-sqlite,https://github.com/dogsheep/github-to-sqlite/pull/61,,,
672053811,MDExOlB1bGxSZXF1ZXN0NjcyMDUzODEx,65,open,0,basic support for events,231498,khimaros,"a quick first pass at implementing the feature requested in https://github.com/dogsheep/github-to-sqlite/issues/64

testing instructions:

```
$ github-to-sqlite events events.db user/khimaros
```

if the specified user is the authenticated user, it will also include private events.

caveat: pagination appears to be broken (i don't see `next` in the response JSON from GitHub)",2021-06-17T00:51:30Z,2022-10-03T22:35:03Z,,,0a252a06a15e307c8a67b2e0aac0907e2566bf19,,,,,0,82da9f91deda81d92ec64c9eda960aa64340c169,0e45b72312a0756e5a562effbba08cb8de1e480b,FIRST_TIME_CONTRIBUTOR,207052882,github-to-sqlite,https://github.com/dogsheep/github-to-sqlite/pull/65,,,
716357982,MDExOlB1bGxSZXF1ZXN0NzE2MzU3OTgy,66,open,0,Add --merged-by flag to pull-requests sub command,30531572,sarcasticadmin,"## Description

Proposing a solution to the API limitation for `merged_by` in pull_requests. Specifically the following called out in the readme:

```
Note that the merged_by column on the pull_requests table will only be populated for pull requests that are loaded using the --pull-request option - the GitHub API does not return this field for pull requests that are loaded in bulk.
```

This approach might cause larger repos to hit rate limits called out in https://github.com/dogsheep/github-to-sqlite/issues/51 but seems to work well in the repos I tested and included below.

## Old Behavior
- Had to list out the pull-requests individually via multiple `--pull-request` flags

## New Behavior

- `--merged-by` flag for getting 'merge_by' information out of pull-requests without having to specify individual PR numbers.

# Testing

Picking some repo that has more than one merger (datasette only has 1 😉 )

```
$ github-to-sqlite pull-requests ./github.db opnsense/tools --merged-by
$ echo ""select id, url, merged_by from pull_requests;"" | sqlite3 ./github.db 
83533612|https://github.com/opnsense/tools/pull/39|1915288
102632885|https://github.com/opnsense/tools/pull/43|1915288
149114810|https://github.com/opnsense/tools/pull/57|1915288
160394495|https://github.com/opnsense/tools/pull/64|1915288
163308408|https://github.com/opnsense/tools/pull/67|1915288
169723264|https://github.com/opnsense/tools/pull/69|1915288
171381422|https://github.com/opnsense/tools/pull/72|1915288
179938195|https://github.com/opnsense/tools/pull/77|1915288
196233824|https://github.com/opnsense/tools/pull/82|1915288
215289964|https://github.com/opnsense/tools/pull/93|
219696100|https://github.com/opnsense/tools/pull/97|1915288
223664843|https://github.com/opnsense/tools/pull/99|
228446172|https://github.com/opnsense/tools/pull/103|1915288
238930434|https://github.com/opnsense/tools/pull/110|1915288
255507110|https://github.com/opnsense/tools/pull/119|1915288
255980675|https://github.com/opnsense/tools/pull/120|1915288
261906770|https://github.com/opnsense/tools/pull/125|
263800503|https://github.com/opnsense/tools/pull/127|1915288
264038685|https://github.com/opnsense/tools/pull/128|1915288
264696704|https://github.com/opnsense/tools/pull/129|1915288
266660547|https://github.com/opnsense/tools/pull/130|1915288
273120409|https://github.com/opnsense/tools/pull/133|1915288
274370803|https://github.com/opnsense/tools/pull/135|
276600629|https://github.com/opnsense/tools/pull/139|
277303655|https://github.com/opnsense/tools/pull/141|1915288
293033714|https://github.com/opnsense/tools/pull/145|
294827649|https://github.com/opnsense/tools/pull/146|
295140008|https://github.com/opnsense/tools/pull/147|1915288
305690829|https://github.com/opnsense/tools/pull/150|9783985
307077931|https://github.com/opnsense/tools/pull/152|1915288
321782100|https://github.com/opnsense/tools/pull/155|
337265672|https://github.com/opnsense/tools/pull/160|
337267484|https://github.com/opnsense/tools/pull/161|1915288
368251763|https://github.com/opnsense/tools/pull/169|
428262505|https://github.com/opnsense/tools/pull/181|
437557011|https://github.com/opnsense/tools/pull/182|1915288
447079893|https://github.com/opnsense/tools/pull/185|
461822092|https://github.com/opnsense/tools/pull/191|
463290142|https://github.com/opnsense/tools/pull/193|1915288
470112962|https://github.com/opnsense/tools/pull/194|1915288
472644649|https://github.com/opnsense/tools/pull/195|1915288
488696898|https://github.com/opnsense/tools/pull/198|
513289902|https://github.com/opnsense/tools/pull/201|
522530265|https://github.com/opnsense/tools/pull/203|
564443347|https://github.com/opnsense/tools/pull/213|
597579516|https://github.com/opnsense/tools/pull/220|1915288
602860357|https://github.com/opnsense/tools/pull/221|1915288
608744738|https://github.com/opnsense/tools/pull/222|1915288
623279673|https://github.com/opnsense/tools/pull/228|1915288
664656182|https://github.com/opnsense/tools/pull/233|
664781786|https://github.com/opnsense/tools/pull/234|1915288
670683636|https://github.com/opnsense/tools/pull/235|1915288
683150764|https://github.com/opnsense/tools/pull/237|
685016233|https://github.com/opnsense/tools/pull/238|
687099825|https://github.com/opnsense/tools/pull/239|1915288
715705652|https://github.com/opnsense/tools/pull/244|1915288
715721248|https://github.com/opnsense/tools/pull/245|1915288
```
`userid` are now present for those PRs that were merged.

Without the flag the `merged_by` behavior remains missing as expected when get PRs bulk:

```
$ github-to-sqlite pull-requests ./github.db opnsense/tools
$ echo ""select id, url, merged_by from pull_requests;"" | sqlite3 ./github.db 
83533612|https://github.com/opnsense/tools/pull/39|
102632885|https://github.com/opnsense/tools/pull/43|
149114810|https://github.com/opnsense/tools/pull/57|
160394495|https://github.com/opnsense/tools/pull/64|
163308408|https://github.com/opnsense/tools/pull/67|
169723264|https://github.com/opnsense/tools/pull/69|
171381422|https://github.com/opnsense/tools/pull/72|
179938195|https://github.com/opnsense/tools/pull/77|
196233824|https://github.com/opnsense/tools/pull/82|
215289964|https://github.com/opnsense/tools/pull/93|
219696100|https://github.com/opnsense/tools/pull/97|
223664843|https://github.com/opnsense/tools/pull/99|
228446172|https://github.com/opnsense/tools/pull/103|
238930434|https://github.com/opnsense/tools/pull/110|
255507110|https://github.com/opnsense/tools/pull/119|
255980675|https://github.com/opnsense/tools/pull/120|
261906770|https://github.com/opnsense/tools/pull/125|
263800503|https://github.com/opnsense/tools/pull/127|
264038685|https://github.com/opnsense/tools/pull/128|
264696704|https://github.com/opnsense/tools/pull/129|
266660547|https://github.com/opnsense/tools/pull/130|
273120409|https://github.com/opnsense/tools/pull/133|
274370803|https://github.com/opnsense/tools/pull/135|
276600629|https://github.com/opnsense/tools/pull/139|
277303655|https://github.com/opnsense/tools/pull/141|
293033714|https://github.com/opnsense/tools/pull/145|
294827649|https://github.com/opnsense/tools/pull/146|
295140008|https://github.com/opnsense/tools/pull/147|
305690829|https://github.com/opnsense/tools/pull/150|
307077931|https://github.com/opnsense/tools/pull/152|
321782100|https://github.com/opnsense/tools/pull/155|
337265672|https://github.com/opnsense/tools/pull/160|
337267484|https://github.com/opnsense/tools/pull/161|
368251763|https://github.com/opnsense/tools/pull/169|
428262505|https://github.com/opnsense/tools/pull/181|
437557011|https://github.com/opnsense/tools/pull/182|
447079893|https://github.com/opnsense/tools/pull/185|
461822092|https://github.com/opnsense/tools/pull/191|
463290142|https://github.com/opnsense/tools/pull/193|
470112962|https://github.com/opnsense/tools/pull/194|
472644649|https://github.com/opnsense/tools/pull/195|
488696898|https://github.com/opnsense/tools/pull/198|
513289902|https://github.com/opnsense/tools/pull/201|
522530265|https://github.com/opnsense/tools/pull/203|
564443347|https://github.com/opnsense/tools/pull/213|
597579516|https://github.com/opnsense/tools/pull/220|
602860357|https://github.com/opnsense/tools/pull/221|
608744738|https://github.com/opnsense/tools/pull/222|
623279673|https://github.com/opnsense/tools/pull/228|
664656182|https://github.com/opnsense/tools/pull/233|
664781786|https://github.com/opnsense/tools/pull/234|
670683636|https://github.com/opnsense/tools/pull/235|
683150764|https://github.com/opnsense/tools/pull/237|
685016233|https://github.com/opnsense/tools/pull/238|
687099825|https://github.com/opnsense/tools/pull/239|
715705652|https://github.com/opnsense/tools/pull/244|
715721248|https://github.com/opnsense/tools/pull/245|
```

Individual PRs passed via `--pull-request` flag behaves as expected (unchanged):

```
$ github-to-sqlite pull-requests ./github.db opnsense/tools --pull-request 39 --pull-request 237
$ echo ""select id, url, merged_by from pull_requests;"" | sqlite3 ./github.db
83533612|https://github.com/opnsense/tools/pull/39|1915288
683150764|https://github.com/opnsense/tools/pull/237|
```
> Picking 1 PR that has a merged_by (39) and one that does not (237)",2021-08-20T00:57:55Z,2021-09-28T21:50:31Z,,,6b4276d9469e4579c81588ac9e3d128026d919a0,,,,,0,a92a31d5d446022baeaf7f3c9ea107094637e64d,ed3752022e45b890af63996efec804725e95d0d4,FIRST_TIME_CONTRIBUTOR,207052882,github-to-sqlite,https://github.com/dogsheep/github-to-sqlite/pull/66,,,
721686721,MDExOlB1bGxSZXF1ZXN0NzIxNjg2NzIx,67,open,0,Replacing step ID key with step_id,16374374,jshcmpbll,"Workflows that have an `id` in any step result in the following error when running `workflows`:

e.g.`github-to-sqlite workflows github.db nixos/nixpkgs`

```Traceback (most recent call last):
  File ""/usr/local/bin/github-to-sqlite"", line 8, in <module>
    sys.exit(cli())
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1137, in __call__
    return self.main(*args, **kwargs)
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1062, in main
    rv = self.invoke(ctx)
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1668, in invoke```Traceback (most recent call last):
  File ""/usr/local/bin/github-to-sqlite"", line 8, in <module>
    sys.exit(cli())
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1137, in __call__
    return self.main(*args, **kwargs)
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1062, in main
    rv = self.invoke(ctx)
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 763, in invoke
    return __callback(*args, **kwargs)
  File ""/usr/local/lib/python3.8/dist-packages/github_to_sqlite/cli.py"", line 601, in workflows
    utils.save_workflow(db, repo_id, filename, content)
  File ""/usr/local/lib/python3.8/dist-packages/github_to_sqlite/utils.py"", line 865, in save_workflow
    db[""steps""].insert_all(
  File ""/usr/local/lib/python3.8/dist-packages/sqlite_utils/db.py"", line 2596, in insert_all
    self.insert_chunk(
  File ""/usr/local/lib/python3.8/dist-packages/sqlite_utils/db.py"", line 2378, in insert_chunk
    result = self.db.execute(query, params)
  File ""/usr/local/lib/python3.8/dist-packages/sqlite_utils/db.py"", line 419, in execute
    return self.conn.execute(sql, parameters)
sqlite3.IntegrityError: datatype mismatch
```

 - [Information about the ID key in a step for GHA](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsid)
 - [An example workflow from a public repo](https://github.com/NixOS/nixpkgs/blob/b4cc66827745e525ce7bb54659845ac89788a597/.github/workflows/direct-push.yml#L16)

# Changes
I'm proposing that the key for `id` in step is replaced with `step_id` so that it no longer interferes with the table `id` for tracking the record.

Special thanks to @sarcasticadmin @egiffen and @ruebenramirez for helping a bit on this 😄 ",2021-08-28T01:26:41Z,2021-08-28T01:27:00Z,,,9f73c9bf29dec9a1482d9af56b9fac271869585c,,,,,0,9b5acceb25cf48b00e9c6c8293358b036440deb2,ed3752022e45b890af63996efec804725e95d0d4,FIRST_TIME_CONTRIBUTOR,207052882,github-to-sqlite,https://github.com/dogsheep/github-to-sqlite/pull/67,,,
747742034,PR_kwDODFdgUs4skaNS,68,open,0,Add support for retrieving teams / members,68329,philwills,Adds a method for retrieving all the teams within an organisation and all the members in those teams. The latter is stored as a join table `team_members` beteween `teams` and `users`.,2021-10-01T15:55:02Z,2021-10-01T15:59:53Z,,,f46e276c356c893370d5893296f4b69f08baf02c,,,,,0,cc838e87b1eb19b299f277a07802923104f35ce2,ed3752022e45b890af63996efec804725e95d0d4,FIRST_TIME_CONTRIBUTOR,207052882,github-to-sqlite,https://github.com/dogsheep/github-to-sqlite/pull/68,,,
862538586,PR_kwDODFdgUs4zaUta,70,open,0,scrape-dependents: enable paging through package menu option if present,36061055,stanbiryukov,Some repos organize network dependents by a Package toggle. This PR adds the ability to page through those options and scrape underlying dependents.,2022-02-24T15:07:25Z,2022-02-24T15:07:25Z,,,36cca3584a07d88d1e505111d1b23294d66ba73e,,,,,0,cc8f276a474525e55ed0bcacb0cd8cc560f89614,751bc900366ca52e662ea383b858cbf4365093d9,FIRST_TIME_CONTRIBUTOR,207052882,github-to-sqlite,https://github.com/dogsheep/github-to-sqlite/pull/70,,,
959140599,PR_kwDODFdgUs45K1L3,73,closed,0,Fixing 'NoneType' object has no attribute 'items',1224205,empjustine,"Under some conditions, GitHub caches removed starred repositories and ends up leaving dangling `None` user references.

    Traceback (most recent call last):
      File ""/home/dogsheep/dogsheep/github-to-sqlite/bin/github-to-sqlite"", line 8, in <module>
        sys.exit(cli())
      File ""/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py"", line 1130, in __call__
        return self.main(*args, **kwargs)
      File ""/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py"", line 1055, in main
        rv = self.invoke(ctx)
      File ""/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py"", line 1657, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File ""/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py"", line 1404, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File ""/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py"", line 760, in invoke
        return __callback(*args, **kwargs)
      File ""/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/cli.py"", line 181, in starred
        utils.save_stars(db, user, stars)
      File ""/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/utils.py"", line 494, in save_stars
        repo_id = save_repo(db, repo)
      File ""/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/utils.py"", line 308, in save_repo
        to_save[""owner""] = save_user(db, to_save[""owner""])
      File ""/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/utils.py"", line 229, in save_user
        for key, value in user.items()
    AttributeError: 'NoneType' object has no attribute 'items'",2022-06-06T13:58:11Z,2022-07-18T19:40:12Z,2022-07-18T19:40:12Z,2022-07-18T19:40:12Z,dbac2e5dd8a562b45d8255a265859cf8020ca22a,,,,,0,d7c06886f3bb95085a3af3b2a21547e41556cc6e,a6e237f75a4b86963d91dcb5c9582e3a1b3349d6,CONTRIBUTOR,207052882,github-to-sqlite,https://github.com/dogsheep/github-to-sqlite/pull/73,,,
1047561919,PR_kwDODFdgUs4-cIa_,76,open,0,Add organization support to repos command,2757699,OverkillGuy,"New --organization flag to signify all given ""usernames"" are private
orgs. Adapts API URL to the organization path instead.

Not the best implementation, but a first draft to talk around

Fixes #75 (badly, no tests, overly vague, untested)",2022-09-06T13:21:42Z,2022-09-06T13:59:08Z,,,1514acfa87f57261547bc3d7fc4f161e34285d76,,,,,0,bb959b46e8a7647755c14dee180fdd5209451954,ace13ec3d98090d99bd71871c286a4a612c96a50,FIRST_TIME_CONTRIBUTOR,207052882,github-to-sqlite,https://github.com/dogsheep/github-to-sqlite/pull/76,,,