html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/simonw/datasette/issues/639#issuecomment-558432868,https://api.github.com/repos/simonw/datasette/issues/639,558432868,MDEyOklzc3VlQ29tbWVudDU1ODQzMjg2OA==,9599,simonw,2019-11-26T02:40:06Z,2019-11-26T02:40:06Z,OWNER,"Unfortunately I don't think it's possible to do this with Heroku. Heroku treats all deployments as total replacements - that's part of how they achieve zero-downtime deployments, since they run the new deployment at the same time as the old deployment and then switch traffic over at the load balancer.

I did have one idea that's relevant here: #238 - which would provide a mechanism for `metadata.json` to be hosted on a separate URL (e.g. a gist) and have Datasette periodically fetch a new copy. I closed that in favour of #357 - a plugin hook for loading metadata. That's still something I'm interested in exploring.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",527670799,updating metadata.json without recreating the app,
https://github.com/simonw/datasette/issues/357#issuecomment-558432963,https://api.github.com/repos/simonw/datasette/issues/357,558432963,MDEyOklzc3VlQ29tbWVudDU1ODQzMjk2Mw==,9599,simonw,2019-11-26T02:40:31Z,2019-11-26T02:40:31Z,OWNER,A plugin hook for this would enable #639. Renaming this issue.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",348043884,Plugin hook for loading metadata.json,
https://github.com/simonw/datasette/issues/639#issuecomment-558437707,https://api.github.com/repos/simonw/datasette/issues/639,558437707,MDEyOklzc3VlQ29tbWVudDU1ODQzNzcwNw==,172847,pkoppstein,2019-11-26T03:02:53Z,2019-11-26T03:03:29Z,NONE,"@simonw - Thanks for the reply!

My reading of the heroku documents is that if one sets things up using git, then one can use ""git push"" (from a {local, GitHub, GitLab} git repository to Heroku) to ""update"" a Heroku deployment, but I'm not sure exactly how this works. However, assuming there is some way to use ""git push"" to update the Heroku deployment, the question becomes how can one do this in conjunction with datasette.

Again based on my reading the heroku documents, it would seem that the following should work (but it doesn't quite):

1) Use datasette to create a deployment (named MYAPP)
2) Put it in maintenance mode
3) heroku git:clone -a MYAPP
  -- This results in an empty repository (as expected)
4) In another directory, heroku slugs:download -a MYAPP
5) Copy the downloaded slug into the repository
6) Make some change to metadata.json
6) Commit and push it back
7) Take the deployment out of maintenance mode
8) Refresh the deployment

Using the heroku console, I've verified that the edits appear on heroku, but somehow they are not reflected in the running app.

I'm hopeful that with some small tweak or perhaps the addition of a bit of voodoo, this strategy will work.  

I think it will be important to get this working for another reason: getting Heroku, Cloudcube, and datasette to work together, to overcome the slug size limitation so that large SQLite databases can be deployed to Heroku using Datasette.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",527670799,updating metadata.json without recreating the app,
https://github.com/simonw/datasette/issues/639#issuecomment-558439989,https://api.github.com/repos/simonw/datasette/issues/639,558439989,MDEyOklzc3VlQ29tbWVudDU1ODQzOTk4OQ==,9599,simonw,2019-11-26T03:14:27Z,2019-11-26T03:14:27Z,OWNER,@jacobian does this sound like something that could work?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",527670799,updating metadata.json without recreating the app,
https://github.com/simonw/datasette/issues/641#issuecomment-558443464,https://api.github.com/repos/simonw/datasette/issues/641,558443464,MDEyOklzc3VlQ29tbWVudDU1ODQ0MzQ2NA==,9599,simonw,2019-11-26T03:30:02Z,2019-11-26T03:30:02Z,OWNER,https://datasette.readthedocs.io/en/latest/custom_templates.html#serving-static-files,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",528442126,Better documentation for --static option,
https://github.com/simonw/datasette/issues/357#issuecomment-558446045,https://api.github.com/repos/simonw/datasette/issues/357,558446045,MDEyOklzc3VlQ29tbWVudDU1ODQ0NjA0NQ==,9599,simonw,2019-11-26T03:43:17Z,2019-11-26T03:43:17Z,OWNER,I think only one plugin gets to work at a time. The plugin can return a dictionary which is used for live lookups of metadata every time it's accessed - which means the plugin can itself mutate that dictionary.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",348043884,Plugin hook for loading metadata.json,
https://github.com/simonw/datasette/issues/357#issuecomment-558459823,https://api.github.com/repos/simonw/datasette/issues/357,558459823,MDEyOklzc3VlQ29tbWVudDU1ODQ1OTgyMw==,9599,simonw,2019-11-26T04:55:44Z,2019-11-26T04:56:24Z,OWNER,"This needs to play nicely with `asyncio` - which means that the plugin hook needs to be able to interact with the event loop somehow.

That said... I don't particularly want to change everywhere that accesses metadata into a `await` call. So this is tricky.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",348043884,Plugin hook for loading metadata.json,
https://github.com/simonw/datasette/issues/357#issuecomment-558461851,https://api.github.com/repos/simonw/datasette/issues/357,558461851,MDEyOklzc3VlQ29tbWVudDU1ODQ2MTg1MQ==,9599,simonw,2019-11-26T05:05:21Z,2019-11-26T05:05:21Z,OWNER,"Here's an example plugin I set up using the experimental hook in d11fd2cbaa6b31933b1319f81b5d1520726cb0b6

```python
import json
from datasette import hookimpl
import threading
import requests
import time


def change_over_time(m, metadata_value):
    while True:
        print(metadata_value)
        fetched = requests.get(metadata_value).json()
        counter = m[""counter""]
        m.clear()
        m[""counter""] = counter + 1
        m.update(fetched)
        m[""counter""] += 1
        m[""title""] = ""{} {}"".format(m.get(""title"", """"), m[""counter""])
        time.sleep(10)


@hookimpl(trylast=True)
def load_metadata(metadata_value):
    m = {
        ""counter"": 0,
    }
    x = threading.Thread(target=change_over_time, args=(m, metadata_value), daemon=True)
    x.start()
    x.setName(""datasette-metadata-counter"")
    return m
```
It runs a separate thread that fetches the provided URL every 10 seconds:

```
datasette -m metadata.json --memory -p 8069 -m https://gist.githubusercontent.com/simonw/e8e4fcd7c0a9c951f7dd976921992157/raw/b702d18a6a078a0fb94ef1cee62e11a3396e0336/demo-metadata.json
```
I learned a bunch of things from this prototype.

First, this is the wrong place to run the code:

https://github.com/simonw/datasette/blob/d11fd2cbaa6b31933b1319f81b5d1520726cb0b6/datasette/cli.py#L337-L343

I wanted the plugin hook to be able to receive a `datasette` instance, so implementations could potentially run their own database queries. Calling the hook in the CLI function here happens BEFORE the `Datasette()` instance is created, so that doesn't work.

I wanted to build a demo of a plugin that would load metadata periodically from an external URL (see #238) - but this threaded implementation is pretty naive. It results in a hit every 10 seconds even if no-one is using Datasette!

A smarter implementation would be to fetch and cache the results - then only re-fetch them if more than 10 seconds have passed since the last time the metadata was accessed.

But... doing this neatly requires asyncio - and the plugin isn't running inside an event loop (since `uvicorn.run(ds.app()...)` has not run yet so the event loop hasn't even started).

I could try and refactor everything so that all calls to read from `metadata` happen via `await`, but this feels like a pretty invasive change. It would be necessary if metadata might be read via a SQL query though.

Or maybe I could set it up so the plugin can start itself running in the event loop and call back to the `datasette` object to update metadata whenever it feels like it?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",348043884,Plugin hook for loading metadata.json,
https://github.com/simonw/datasette/issues/639#issuecomment-558687342,https://api.github.com/repos/simonw/datasette/issues/639,558687342,MDEyOklzc3VlQ29tbWVudDU1ODY4NzM0Mg==,21148,jacobian,2019-11-26T15:40:00Z,2019-11-26T15:40:00Z,CONTRIBUTOR,"A bit of background: the reason `heroku git:clone` brings down an empty directory is because `datasette publish heroku` uses the [builds API](https://devcenter.heroku.com/articles/build-and-release-using-the-api), rather than a `git push`, to release the app. I originally did this because it seemed like a lower bar than having a working `git`, but the downside is, as you found out, that tweaking the created app is hard. 

So there's one option -- change `datasette publish heroku` to use `git push` instead of `heroku builds:create`.

@pkoppstein - what you suggested seems like it ought to work (you don't need maintenance mode, though). I'm not sure why it doesn't.

You could also look into using the [slugs API](https://devcenter.heroku.com/articles/platform-api-deploying-slugs) to download the slug, change `metadata.json`, re-pack and re-upload the slug.

Ultimately though I think I think @simonw's idea of reading `metadata.json` from an external source might be better (#357). Reading from an alternate URL would be fine, or you could also just stuff the whole `metadata.json` into a Heroku config var, and write a plugin to read it from there. 

Hope this helps a bit!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",527670799,updating metadata.json without recreating the app,
https://github.com/simonw/datasette/issues/639#issuecomment-558852316,https://api.github.com/repos/simonw/datasette/issues/639,558852316,MDEyOklzc3VlQ29tbWVudDU1ODg1MjMxNg==,172847,pkoppstein,2019-11-26T22:54:23Z,2019-11-26T22:54:23Z,NONE,"@jacobian - Thanks for your help.  Having to upload an entire slug each time a small change is needed in `metadata.json` seems no better than the current situation so I probably won't go down that rabbit hole just yet.  In any case, the really important goal is moving the SQLite file out of Heroku in a way that the Heroku app can still read it efficiently.  Is this possible?  Is Cloudcube the right place to start?  Is there any alternative?  ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",527670799,updating metadata.json without recreating the app,