html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/simonw/datasette/issues/1510#issuecomment-1073366630,https://api.github.com/repos/simonw/datasette/issues/1510,1073366630,IC_kwDOBm6k_c4_-kZm,9599,simonw,2022-03-20T22:59:33Z,2022-03-20T22:59:33Z,OWNER,"I really like the idea of making this effectively the same thing as the fully documented, stable JSON API that comes as part of 1.0. If you want to know what will be available to your templates, consult the API documentation.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1054244712,Datasette 1.0 documented template context (maybe via API docs),
https://github.com/simonw/datasette/issues/1674#issuecomment-1073366436,https://api.github.com/repos/simonw/datasette/issues/1674,1073366436,IC_kwDOBm6k_c4_-kWk,9599,simonw,2022-03-20T22:58:40Z,2022-03-20T22:58:40Z,OWNER,"This will probably happen as part of turning this into an officially documented API that serves the template context for the homepage:
- #1510","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1174717287,Tweak design of /.json,
https://github.com/simonw/datasette/issues/1355#issuecomment-1073362979,https://api.github.com/repos/simonw/datasette/issues/1355,1073362979,IC_kwDOBm6k_c4_-jgj,9599,simonw,2022-03-20T22:38:53Z,2022-03-20T22:38:53Z,OWNER,"Built a research prototype:
```diff
diff --git a/datasette/app.py b/datasette/app.py
index 5c8101a..5cd3e63 100644
--- a/datasette/app.py
+++ b/datasette/app.py
@@ -1,6 +1,7 @@
import asyncio
import asgi_csrf
import collections
+import contextlib
import datetime
import functools
import glob
@@ -1490,3 +1491,11 @@ class DatasetteClient:
return await client.request(
method, self._fix(path, avoid_path_rewrites), **kwargs
)
+
+ @contextlib.asynccontextmanager
+ async def stream(self, method, path, **kwargs):
+ async with httpx.AsyncClient(app=self.app) as client:
+ print(""async with as client"")
+ async with client.stream(method, self._fix(path), **kwargs) as response:
+ print(""async with client.stream about to yield response"")
+ yield response
diff --git a/datasette/cli.py b/datasette/cli.py
index 3c6e1b2..3025ead 100644
--- a/datasette/cli.py
+++ b/datasette/cli.py
@@ -585,11 +585,19 @@ def serve(
asyncio.get_event_loop().run_until_complete(check_databases(ds))
if get:
- client = TestClient(ds)
- response = client.get(get)
- click.echo(response.text)
- exit_code = 0 if response.status == 200 else 1
- sys.exit(exit_code)
+
+ async def _run_get():
+ print(""_run_get"")
+ async with ds.client.stream(""GET"", get) as response:
+ print(""Got response:"", response)
+ async for chunk in response.aiter_bytes(chunk_size=1024):
+ print("" chunk"")
+ sys.stdout.buffer.write(chunk)
+ sys.stdout.buffer.flush()
+ exit_code = 0 if response.status_code == 200 else 1
+ sys.exit(exit_code)
+
+ asyncio.get_event_loop().run_until_complete(_run_get())
return
# Start the server
```
But for some reason it didn't appear to stream out the response - it would print this out:
```
% datasette covid.db --get '/covid/ny_times_us_counties.csv?_size=10&_stream=on'
_run_get
async with as client
```
And then hang. I would expect it to start printing out chunks of CSV data here, but instead it looks like it waited for everything to be generated before returning anything to the console.
No idea why. I dropped this for the moment.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910088936,datasette --get should efficiently handle streaming CSV,
https://github.com/simonw/datasette/issues/1673#issuecomment-1073361986,https://api.github.com/repos/simonw/datasette/issues/1673,1073361986,IC_kwDOBm6k_c4_-jRC,9599,simonw,2022-03-20T22:31:41Z,2022-03-20T22:34:06Z,OWNER,"Maybe it's because `supports_table_xinfo()` creates a brand new in-memory SQLite connection every time you call it?
https://github.com/simonw/datasette/blob/798f075ef9b98819fdb564f9f79c78975a0f71e8/datasette/utils/sqlite.py#L22-L35
Actually no, I'm caching that already:
https://github.com/simonw/datasette/blob/798f075ef9b98819fdb564f9f79c78975a0f71e8/datasette/utils/sqlite.py#L12-L19","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1174708375,Streaming CSV spends a lot of time in `table_column_details`,
https://github.com/simonw/datasette/issues/1672#issuecomment-1073355818,https://api.github.com/repos/simonw/datasette/issues/1672,1073355818,IC_kwDOBm6k_c4_-hwq,9599,simonw,2022-03-20T21:52:38Z,2022-03-20T21:52:38Z,OWNER,"That means taking on these issues:
- https://github.com/simonw/datasette/issues/1101
- https://github.com/simonw/datasette/issues/1096
- https://github.com/simonw/datasette/issues/1062","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1174697144,Refactor CSV handling code out of DataView,
https://github.com/simonw/datasette/issues/1660#issuecomment-1073355032,https://api.github.com/repos/simonw/datasette/issues/1660,1073355032,IC_kwDOBm6k_c4_-hkY,9599,simonw,2022-03-20T21:46:43Z,2022-03-20T21:46:43Z,OWNER,I think the way to get rid of most of the remaining complexity in `DataView` is to refactor how CSV stuff works - pulling it in line with other export factors and extracting the streaming mechanism. Opening a fresh issue for that.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1170144879,Refactor and simplify Datasette routing and views,
https://github.com/simonw/sqlite-utils/issues/140#issuecomment-1073330388,https://api.github.com/repos/simonw/sqlite-utils/issues/140,1073330388,IC_kwDOCGYnMM4_-bjU,9599,simonw,2022-03-20T19:44:39Z,2022-03-20T19:45:45Z,OWNER,"Alternative idea for specifying types: accept a Python expression, then use Python type literal syntax. For example:
```
sqlite-utils insert-files gifs.db images *.gif \
-c path -c md5 -c last_modified:mtime \
-a file_type '""gif""'
```
Where `-a` indicates an additional column.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688351054,Idea: insert-files mechanism for adding extra columns with fixed values,
https://github.com/dogsheep/google-takeout-to-sqlite/issues/10#issuecomment-1073152522,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/10,1073152522,IC_kwDODFE5qs4_9wIK,9290214,csusanu,2022-03-20T02:38:07Z,2022-03-20T02:38:07Z,NONE,"[This line](https://github.com/dogsheep/google-takeout-to-sqlite/blob/e54e544427f1cc3ea8189f0e95f54046301a8645/google_takeout_to_sqlite/utils.py) needs to say `""MyActivity.json""` instead of `""My Activity.json""`. Google must have changed the file name.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1123393829,sqlite3.OperationalError: no such table: main.my_activity,
https://github.com/simonw/datasette/issues/1669#issuecomment-1073143413,https://api.github.com/repos/simonw/datasette/issues/1669,1073143413,IC_kwDOBm6k_c4_9t51,9599,simonw,2022-03-20T01:24:36Z,2022-03-20T01:24:36Z,OWNER,https://github.com/simonw/datasette/releases/tag/0.61a0,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1174404647,Release 0.61 alpha,
https://github.com/dogsheep/healthkit-to-sqlite/issues/14#issuecomment-1073139067,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/14,1073139067,IC_kwDOC8tyDs4_9s17,343884,lchski,2022-03-20T00:54:18Z,2022-03-20T00:54:18Z,NONE,"Update: this appears to be because of running the command twice without clearing the DB in between. Tries to insert a Workout that already exists, causing a collision on the (auto-generated) `id` column. Had a different error with a clean DB, likely due to the workout points format; will make a new issue for that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",771608692,UNIQUE constraint failed: workouts.id,
https://github.com/simonw/datasette/issues/1669#issuecomment-1073137170,https://api.github.com/repos/simonw/datasette/issues/1669,1073137170,IC_kwDOBm6k_c4_9sYS,9599,simonw,2022-03-20T00:35:52Z,2022-03-20T00:35:52Z,OWNER,https://github.com/simonw/datasette/compare/0.60.2...main,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1174404647,Release 0.61 alpha,
https://github.com/simonw/datasette/issues/1668#issuecomment-1073136896,https://api.github.com/repos/simonw/datasette/issues/1668,1073136896,IC_kwDOBm6k_c4_9sUA,9599,simonw,2022-03-20T00:33:23Z,2022-03-20T00:33:23Z,OWNER,I'm going to release this as a 0.61 alpha so I can more easily depend on it from `datasette-hashed-urls`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1174306154,"Introduce concept of a database `route`, separate from its name",
https://github.com/simonw/datasette/issues/1668#issuecomment-1073136686,https://api.github.com/repos/simonw/datasette/issues/1668,1073136686,IC_kwDOBm6k_c4_9sQu,9599,simonw,2022-03-20T00:31:13Z,2022-03-20T00:31:13Z,OWNER,"That demo is now live:
- https://latest.datasette.io/alternative-route
- https://latest.datasette.io/alternative-route/attraction_characteristic","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1174306154,"Introduce concept of a database `route`, separate from its name",
https://github.com/simonw/datasette/issues/1668#issuecomment-1073135433,https://api.github.com/repos/simonw/datasette/issues/1668,1073135433,IC_kwDOBm6k_c4_9r9J,9599,simonw,2022-03-20T00:20:36Z,2022-03-20T00:20:36Z,OWNER,"Building this plugin instantly revealed that all of the links - on the homepage and the database page and so on - are incorrect:
```python
from datasette import hookimpl
@hookimpl
def startup(datasette):
db = datasette.get_database(""fixtures2"")
db.route = ""alternative-route""
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1174306154,"Introduce concept of a database `route`, separate from its name",
https://github.com/simonw/datasette/issues/1668#issuecomment-1073134816,https://api.github.com/repos/simonw/datasette/issues/1668,1073134816,IC_kwDOBm6k_c4_9rzg,9599,simonw,2022-03-20T00:16:22Z,2022-03-20T00:16:22Z,OWNER,I'm going to add a `fixtures2.db` database which has that as the name but `alternative-route` as the route. I'll set that up using a custom plugin in the `plugins/` folder that gets deployed by https://github.com/simonw/datasette/blob/main/.github/workflows/deploy-latest.yml,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1174306154,"Introduce concept of a database `route`, separate from its name",
https://github.com/simonw/datasette/issues/1668#issuecomment-1073134206,https://api.github.com/repos/simonw/datasette/issues/1668,1073134206,IC_kwDOBm6k_c4_9rp-,9599,simonw,2022-03-20T00:12:03Z,2022-03-20T00:12:03Z,OWNER,I'd like to have a live demo of this up on `latest.datasette.io` too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1174306154,"Introduce concept of a database `route`, separate from its name",