html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/simonw/datasette/pull/518#issuecomment-504782618,https://api.github.com/repos/simonw/datasette/issues/518,504782618,MDEyOklzc3VlQ29tbWVudDUwNDc4MjYxOA==,9599,simonw,2019-06-23T20:05:44Z,2019-06-23T20:05:59Z,OWNER,"**Replacement for @app.listener(""before_server_start"")** - this is what the [ASGI lifespan protocol](https://asgi.readthedocs.io/en/latest/specs/lifespan.html) is for.
I know Uvicorn supports this because it keeps saying `ASGI 'lifespan' protocol appears unsupported` on the console.
I think the solution here will be to introduce another ASGI wrapper class similar to `AsgiTracer`. I'll model this on the example in the ASGI lifespan spec.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459587155,Port Datasette from Sanic to ASGI + Uvicorn,
https://github.com/simonw/datasette/issues/514#issuecomment-504789231,https://api.github.com/repos/simonw/datasette/issues/514,504789231,MDEyOklzc3VlQ29tbWVudDUwNDc4OTIzMQ==,7936571,chrismp,2019-06-23T21:35:33Z,2019-06-23T21:35:33Z,NONE,"@russss
Thanks, just one more thing.
I edited `datasette.service`:
```
[Unit]
Description=Datasette
After=network.target
[Service]
Type=simple
User=chris
WorkingDirectory=/home/chris/digital-library
ExecStart=/home/chris/Env/datasette/bin/datasette serve -h 0.0.0.0 databases/*.db --cors --metadata metadata.json
Restart=on-failure
[Install]
WantedBy=multi-user.target
```
Then ran:
```
$ sudo systemctl daemon-reload
$ sudo systemctl enable datasette
$ sudo systemctl start datasette
```
But the logs from `journalctl` show this datasette error:
```
Jun 23 23:31:41 ns331247 datasette[1771]: Error: Invalid value for ""[FILES]..."": Path ""databases/*.db"" does not exist.
Jun 23 23:31:44 ns331247 datasette[1778]: Usage: datasette serve [OPTIONS] [FILES]...
Jun 23 23:31:44 ns331247 datasette[1778]: Try ""datasette serve --help"" for help.
```
But the `databases` directory does exist in the directory specified by `WorkingDirectory`. Is this a datasette problem or did I write something incorrectly in the `.service` file?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459397625,Documentation with recommendations on running Datasette in production without using Docker,
https://github.com/simonw/datasette/issues/272#issuecomment-504759683,https://api.github.com/repos/simonw/datasette/issues/272,504759683,MDEyOklzc3VlQ29tbWVudDUwNDc1OTY4Mw==,9599,simonw,2019-06-23T14:57:50Z,2019-06-23T14:57:50Z,OWNER,"All of the tests are now passing!
I still need a solution for this:
https://github.com/simonw/datasette/blob/5bd510b01adae3f719e4426b9bfbc346a946ba5c/datasette/app.py#L706-L714
I think the answer is ASGI lifespan, which is supported by Uvicorn. https://asgi.readthedocs.io/en/latest/specs/lifespan.html#startup","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/272#issuecomment-504761039,https://api.github.com/repos/simonw/datasette/issues/272,504761039,MDEyOklzc3VlQ29tbWVudDUwNDc2MTAzOQ==,9599,simonw,2019-06-23T15:15:41Z,2019-06-23T15:18:36Z,OWNER,"And now the tests are all passing!
Still to do:
* Use `raw_path` so table names containing `/` can work correctly
* Get ?_trace=1 working again
* Replacement for `@app.listener(""before_server_start"")`
* Replace Sanic request object with my own request class, so I can remove Sanic dependency","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/pull/518#issuecomment-504765145,https://api.github.com/repos/simonw/datasette/issues/518,504765145,MDEyOklzc3VlQ29tbWVudDUwNDc2NTE0NQ==,9599,simonw,2019-06-23T16:04:37Z,2019-06-23T16:04:37Z,OWNER,"Another bug: JSON is being served without a content-type header:
```
~ $ curl -i 'http://127.0.0.1:8001/fivethirtyeight/ahca-polls%2Fahca_polls.json'
HTTP/1.1 200 OK
date: Sun, 23 Jun 2019 16:04:01 GMT
server: uvicorn
referrer-policy: no-referrer
transfer-encoding: chunked
{""database"": ""fivethirtyeight"", ""table"": ""ahca-polls/ahca_polls"", ...
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459587155,Port Datasette from Sanic to ASGI + Uvicorn,
https://github.com/simonw/datasette/issues/272#issuecomment-504754433,https://api.github.com/repos/simonw/datasette/issues/272,504754433,MDEyOklzc3VlQ29tbWVudDUwNDc1NDQzMw==,9599,simonw,2019-06-23T13:51:53Z,2019-06-23T13:51:53Z,OWNER,"CSV tests all pass as of https://github.com/simonw/datasette/commit/ff9efa668ebc33f17ef9b30139960e29906a18fb
This code could be a lot neater though. At the very least I'm going to refactor `datasette/utils.py` into a `datasette/utils` package and put all of my new ASGI utilities in `datasette/utils/asgi.py`
The way I implemented streaming on top of a writer object (inspired by Sanic) is a bit of a weird hack. I think I'd rather use an abstraction where my view functions can yield chunks of body data.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/pull/518#issuecomment-504795648,https://api.github.com/repos/simonw/datasette/issues/518,504795648,MDEyOklzc3VlQ29tbWVudDUwNDc5NTY0OA==,9599,simonw,2019-06-23T23:07:06Z,2019-06-23T23:07:06Z,OWNER,"For the request object.... what are the fields of it I actually use?
* `request.url`
* `request.query_string`
* `request.path`
* `request.method`
* `request.args`
* `request.raw_args`
ALL of those are things that can be derived from the `scope` - so I think my new `Request` class (in `utils/asgi.py`) is just going to be a wrapper around a `scope`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459587155,Port Datasette from Sanic to ASGI + Uvicorn,
https://github.com/simonw/datasette/issues/272#issuecomment-504759842,https://api.github.com/repos/simonw/datasette/issues/272,504759842,MDEyOklzc3VlQ29tbWVudDUwNDc1OTg0Mg==,9599,simonw,2019-06-23T15:00:06Z,2019-06-23T15:00:06Z,OWNER,I also need to actually take advantage of `raw_path` such that pages like https://fivethirtyeight.datasettes.com/fivethirtyeight/twitter-ratio%2Fsenators can be correctly served.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/issues/514#issuecomment-504793379,https://api.github.com/repos/simonw/datasette/issues/514,504793379,MDEyOklzc3VlQ29tbWVudDUwNDc5MzM3OQ==,9599,simonw,2019-06-23T22:31:29Z,2019-06-23T22:31:48Z,OWNER,"I suggest trying a full path in `ExecStart` like this:
`ExecStart=/home/chris/Env/datasette/bin/datasette serve -h 0.0.0.0 /home/chris/digital-library/databases/*.db --cors --metadata /home/chris/digital-library/metadata.json`
That should eliminate the chance of some kind of path confusion.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459397625,Documentation with recommendations on running Datasette in production without using Docker,
https://github.com/simonw/datasette/issues/272#issuecomment-504761165,https://api.github.com/repos/simonw/datasette/issues/272,504761165,MDEyOklzc3VlQ29tbWVudDUwNDc2MTE2NQ==,9599,simonw,2019-06-23T15:17:07Z,2019-06-23T15:17:07Z,OWNER,I'm going to move the remaining work into a pull request.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/pull/518#issuecomment-504798977,https://api.github.com/repos/simonw/datasette/issues/518,504798977,MDEyOklzc3VlQ29tbWVudDUwNDc5ODk3Nw==,9599,simonw,2019-06-23T23:52:38Z,2019-06-23T23:52:38Z,OWNER,"Last thing is to replace `sanic.response`:
* `response.text("""")`
* `response.html()`
* `response.redirect(path)`
* `response.HTTPResponse`
Implementations here: https://github.com/huge-success/sanic/blob/0.7.0/sanic/response.py#L175-L285","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459587155,Port Datasette from Sanic to ASGI + Uvicorn,
https://github.com/simonw/datasette/issues/272#issuecomment-504710331,https://api.github.com/repos/simonw/datasette/issues/272,504710331,MDEyOklzc3VlQ29tbWVudDUwNDcxMDMzMQ==,9599,simonw,2019-06-23T01:08:45Z,2019-06-23T01:08:45Z,OWNER,"Lots still to do:
* Static files are not being served
* Streaming CSV files don't work
* Tests all fail
* Some URLs (e.g. the 'next' link on tables) are incorrect
I'm going to work on getting the unit test framework to be ASGI-compatible next.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/pull/518#issuecomment-504762887,https://api.github.com/repos/simonw/datasette/issues/518,504762887,MDEyOklzc3VlQ29tbWVudDUwNDc2Mjg4Nw==,9599,simonw,2019-06-23T15:38:58Z,2019-06-23T15:38:58Z,OWNER,"Mystery solved: that's because I'm constructing my own scope object and testing via `ApplicationCommunicator` rather than exercising Uvicorn directly.
https://github.com/simonw/datasette/blob/d60fbfcae2658e71cab6d7b3b9f53f8d895064ef/tests/fixtures.py#L42-L57
I don't want to introduce the complexity of launching a real Uvicorn as part of the tests, so I guess I'll have to carefully update my `ApplicationCommunicator` test harness to more correctly emulate real life.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459587155,Port Datasette from Sanic to ASGI + Uvicorn,
https://github.com/simonw/datasette/issues/272#issuecomment-504754552,https://api.github.com/repos/simonw/datasette/issues/272,504754552,MDEyOklzc3VlQ29tbWVudDUwNDc1NDU1Mg==,9599,simonw,2019-06-23T13:53:39Z,2019-06-23T13:53:39Z,OWNER,"Next test to fix (because by new test harness doesn't actually obey the `allow_redirects=` parameter):
```
_____________ test_database_page_redirects_with_url_hash _____________
app_client_with_hash =
def test_database_page_redirects_with_url_hash(app_client_with_hash):
response = app_client_with_hash.get(""/fixtures"", allow_redirects=False)
assert response.status == 302
response = app_client_with_hash.get(""/fixtures"")
> assert ""fixtures"" in response.text
E AssertionError: assert 'fixtures' in ''
E + where '' = .text
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324188953,Port Datasette to ASGI,
https://github.com/simonw/datasette/pull/518#issuecomment-504765738,https://api.github.com/repos/simonw/datasette/issues/518,504765738,MDEyOklzc3VlQ29tbWVudDUwNDc2NTczOA==,9599,simonw,2019-06-23T16:11:49Z,2019-06-23T16:20:44Z,OWNER,"OK, for **Get ?_trace=1 working again**. The old code lives in two places:
https://github.com/simonw/datasette/blob/35429f90894321eda7f2db31b9ea7976f31f73ac/datasette/app.py#L546-L560
And then:
https://github.com/simonw/datasette/blob/35429f90894321eda7f2db31b9ea7976f31f73ac/datasette/app.py#L653-L672
So it's stashing something on the request to tell the rest of the code it should be tracing, then using that collected data from the request to add information to the final body.
One possible shape for the replacement is a new ASGI middleware that wraps everything else. We don't have a mutable request object here though, so we will need to untangle this entirely from the request object.
Also tricky is that in ASGI land we handle streams - we don't usually wait around for the entire response body to be compiled for us. This means the code that modifies the response (adding to the JSON or appending inside the `