commits
1 row where message = "Moved all SQLite queries to threads SQLite operations are blocking, but we're running everything in Sanic, an asyncio web framework, so blocking operations are bad - a long-running DB operation could hold up the entire server. Instead, I've moved all SQLite operations into threads. These are managed by a concurrent.futures ThreadPoolExecutor. This means I can run up to X queries in parallel, and I can continue to queue up additional incoming HTTP traffic while the threadpool is busy. Each thread is responsible for managing its own SQLite connections - one per database. These are cached in a threadlocal. Since we are working with immutable, read-only SQLite databases it should be safe to share SQLite objects across threads. On this assumption I'm using the check_same_thread=False option. Opening a database connection looks like this: conn = sqlite3.connect( 'file:filename.db?immutable=1', uri=True, check_same_thread=False, ) The following articles were helpful in figuring this out: * https://pymotw.com/3/asyncio/executors.html * https://marlinux.wordpress.com/2017/05/19/python-3-6-asyncio-sqlalchemy/ Closes #45. Refs #38." and raw_committer = "2946d096d0cdefdc017559e6b57e87658736e843"
This data as json, CSV (advanced)
Suggested facets: author_date (date), committer_date (date)
sha ▼ | author_date | committer_date | raw_author | raw_committer | repo | author | committer |
---|---|---|---|---|---|---|---|
31b21f5c5e15fc3acab7fabb170c1da71dc3c98c | 2017-11-05T02:21:44Z | 2017-11-05T02:21:44Z | Simon Willison 2946d096d0cdefdc017559e6b57e87658736e843 | Simon Willison 2946d096d0cdefdc017559e6b57e87658736e843 | datasette 107914493 | simonw 9599 | simonw 9599 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [commits] ( [sha] TEXT PRIMARY KEY, [message] TEXT, [author_date] TEXT, [committer_date] TEXT, [raw_author] TEXT REFERENCES [raw_authors]([id]), [raw_committer] TEXT REFERENCES [raw_authors]([id]), [repo] INTEGER REFERENCES [repos]([id]), [author] INTEGER REFERENCES [users]([id]), [committer] INTEGER REFERENCES [users]([id]) ); CREATE INDEX [idx_commits_committer] ON [commits] ([committer]); CREATE INDEX [idx_commits_author] ON [commits] ([author]); CREATE INDEX [idx_commits_repo] ON [commits] ([repo]); CREATE INDEX [idx_commits_raw_committer] ON [commits] ([raw_committer]); CREATE INDEX [idx_commits_raw_author] ON [commits] ([raw_author]);