html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/datasette/issues/607#issuecomment-546752311,https://api.github.com/repos/simonw/datasette/issues/607,546752311,MDEyOklzc3VlQ29tbWVudDU0Njc1MjMxMQ==,8431341,zeluspudding,2019-10-28T00:37:10Z,2019-10-28T00:37:10Z,NONE,"UPDATE: According to tips suggested in [Squeezing Performance from SQLite: Indexes? Indexes!](https://medium.com/@JasonWyatt/squeezing-performance-from-sqlite-indexes-indexes-c4e175f3c346) I have added an index to my large table and benchmarked query speeds in the case where I want to return `all rows`, `rows exactly equal to 'Musk Elon'` and, `rows like 'musk'`. Indexing reduced query time for each of those measures and **dramatically** reduced the time to return `rows exactly equal to 'Musk Elon'` as shown below: > table: edgar_idx > rows: 16,428,090 rows > **indexed: False** > Return all rows where company name exactly equal to Musk Elon > query: select rowid, * from edgar_idx where ""company"" = :p0 order by rowid limit 101 > query time: Query took 21821.031ms > > Return all rows where company name contains Musk > query: select rowid, * from edgar_idx where ""company"" like :p0 order by rowid limit 101 > query time: Query took 20505.029ms > > Return everything > query: select rowid, * from edgar_idx order by rowid limit 101 > query time: Query took 7985.011ms > > **indexed: True** > Return all rows where company name exactly equal to Musk Elon > query: select rowid, * from edgar_idx where ""company"" = :p0 order by rowid limit 101 > query time: Query took 30.0ms > > Return all rows where company name contains Musk > query: select rowid, * from edgar_idx where ""company"" like :p0 order by rowid limit 101 > query time: Query took 13340.019ms > > Return everything > query: select rowid, * from edgar_idx order by rowid limit 101 > query time: Query took 2190.003ms So indexing reduced query time for an exact match to ""Musk Elon"" from almost `22 seconds` to `30.0ms`. **That's amazing and truly promising!** However, an autocomplete feature relies on fuzzy / incomplete matching, which is more similar to the `contains 'musk'` query... Unfortunately, that takes 13 seconds even after indexing. So the hunt for a fast fuzzy / autocomplete search capability persists.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",512996469,Ways to improve fuzzy search speed on larger data sets?,