html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606850008,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606850008,MDEyOklzc3VlQ29tbWVudDYwNjg1MDAwOA==,9599,simonw,2020-03-31T20:13:59Z,2020-04-01T00:23:00Z,MEMBER,"Table design for `since_ids` table:

type | key | since_id
--- | --- | ---
1 | 124324 | 2347239847293
2 | 99ff9cefff5cbfd804f7cd43e2b27ced8addbe8d | 2125947927344

Primary compound key on `(category, key)`

`type` is also a foreign key to a `since_id_types` table with `id` and `name` columns (probably created using https://sqlite-utils.readthedocs.io/en/stable/python-api.html#working-with-lookup-tables )","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606850453,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606850453,MDEyOklzc3VlQ29tbWVudDYwNjg1MDQ1Mw==,9599,simonw,2020-03-31T20:14:58Z,2020-04-01T03:03:50Z,MEMBER,Actually I'll hard-code the population of `since_id_types` to get known ID constants.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606998669,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606998669,MDEyOklzc3VlQ29tbWVudDYwNjk5ODY2OQ==,9599,simonw,2020-04-01T02:57:36Z,2020-04-01T02:57:36Z,MEMBER,"The tricky thing here is thinking about the interaction between the recorded since_id and a desire to run the initial import.

The first time you run `twitter-to-sqlite user-timeline db.db username` we want to fetch as many tweets from that user as possible - probably around 3,200 before the API limitations cut us off.

We need to record the maximum ID from those as the `since_id` - which we will see on the very first page we paginate through. That way next time we run the command with `--since` we will only fetch new tweets.

But what happens if our initial import is cancelled after only a few tweets? We risk never pulling in the rest of the tweets.

Not sure if I need to solve this at all or if I should instead trust users to run the command a second time without `--since` if they think they didn't retrieve anything the first time.

I had considered letting `--stop_after=` over-ride `--since` but that doesn't actually make sense - if you send a since_id to the Twitter API you'll never get back more tweets than exist after that ID, so the `--stop_after` would not make a meaningful difference.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-607003655,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,607003655,MDEyOklzc3VlQ29tbWVudDYwNzAwMzY1NQ==,9599,simonw,2020-04-01T03:18:00Z,2020-04-01T03:18:00Z,MEMBER,I've got this working for the `user-timeline` command.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-607010634,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,607010634,MDEyOklzc3VlQ29tbWVudDYwNzAxMDYzNA==,9599,simonw,2020-04-01T03:45:16Z,2020-04-01T03:45:16Z,MEMBER,"OK, fix is applied to everything now.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-607010791,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10,607010791,MDEyOklzc3VlQ29tbWVudDYwNzAxMDc5MQ==,9599,simonw,2020-04-01T03:45:48Z,2020-04-01T03:45:48Z,MEMBER,I'm happy with the recent work I did on this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",492297930,Rethink progress bars for various commands,
https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607011421,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40,607011421,MDEyOklzc3VlQ29tbWVudDYwNzAxMTQyMQ==,9599,simonw,2020-04-01T03:47:37Z,2020-04-01T03:55:08Z,MEMBER,"Actually a single table with a `type` integer ID referencing a `count_history_types` table would better match the way I implemented the `since_ids` table:

https://github.com/dogsheep/twitter-to-sqlite/blob/4b6c8d8c1cc6fefdb566ec8506157133f47c569a/twitter_to_sqlite/utils.py#L331-L341

In which case the compound primary key would be `type`, `user`, `datetime`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590669793,Feature: record history of follower counts,
https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607011972,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40,607011972,MDEyOklzc3VlQ29tbWVudDYwNzAxMTk3Mg==,9599,simonw,2020-04-01T03:49:02Z,2020-04-01T03:50:01Z,MEMBER,"I want the datetime value to look like `2020-04-01T03:34:58+00:00` (the format returned by the Twitter API which I am storing in other tables at the moment).
```
>>> datetime.utcnow().isoformat().split('.')[0] + '+00:00'
'2020-04-01T03:49:52+00:00'
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590669793,Feature: record history of follower counts,
https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607019151,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40,607019151,MDEyOklzc3VlQ29tbWVudDYwNzAxOTE1MQ==,9599,simonw,2020-04-01T04:11:10Z,2020-04-01T04:11:10Z,MEMBER,"In testing this collects a LOT of data. I'm going to skip tracking favourites_count and statuses_count and just track followers, friends and listed instead.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590669793,Feature: record history of follower counts,