sha,message,author_date,committer_date,raw_author,raw_committer,repo,author,committer 31b21f5c5e15fc3acab7fabb170c1da71dc3c98c,"Moved all SQLite queries to threads SQLite operations are blocking, but we're running everything in Sanic, an asyncio web framework, so blocking operations are bad - a long-running DB operation could hold up the entire server. Instead, I've moved all SQLite operations into threads. These are managed by a concurrent.futures ThreadPoolExecutor. This means I can run up to X queries in parallel, and I can continue to queue up additional incoming HTTP traffic while the threadpool is busy. Each thread is responsible for managing its own SQLite connections - one per database. These are cached in a threadlocal. Since we are working with immutable, read-only SQLite databases it should be safe to share SQLite objects across threads. On this assumption I'm using the check_same_thread=False option. Opening a database connection looks like this: conn = sqlite3.connect( 'file:filename.db?immutable=1', uri=True, check_same_thread=False, ) The following articles were helpful in figuring this out: * https://pymotw.com/3/asyncio/executors.html * https://marlinux.wordpress.com/2017/05/19/python-3-6-asyncio-sqlalchemy/ Closes #45. Refs #38.",2017-11-05T02:21:44Z,2017-11-05T02:21:44Z,2946d096d0cdefdc017559e6b57e87658736e843,2946d096d0cdefdc017559e6b57e87658736e843,107914493,9599,9599