html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/datasette/issues/1522#issuecomment-974557766,https://api.github.com/repos/simonw/datasette/issues/1522,974557766,IC_kwDOBm6k_c46FpJG,9599,simonw,2021-11-20T00:35:25Z,2021-11-20T00:35:25Z,OWNER,Wrote a TIL about `--build-arg` and Cloud Run: https://til.simonwillison.net/cloudrun/using-build-args-with-cloud-run,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974558076,https://api.github.com/repos/simonw/datasette/issues/1522,974558076,IC_kwDOBm6k_c46FpN8,9599,simonw,2021-11-20T00:36:56Z,2021-11-20T00:36:56Z,OWNER,That 503 error is _really_ frustrating: I have a deploy running at https://apache-proxy-demo.datasette.io/prefix/ and after a fresh deploy it serves 503 errors for quite a while - then eventually starts working.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1519#issuecomment-974558267,https://api.github.com/repos/simonw/datasette/issues/1519,974558267,IC_kwDOBm6k_c46FpQ7,9599,simonw,2021-11-20T00:37:57Z,2021-11-20T00:37:57Z,OWNER,Thanks to #1522 I have a live demo that exhibits this bug now: https://apache-proxy-demo.datasette.io/prefix/fixtures/attraction_characteristic,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058790545,base_url is omitted in JSON and CSV views, https://github.com/simonw/datasette/issues/1519#issuecomment-974559176,https://api.github.com/repos/simonw/datasette/issues/1519,974559176,IC_kwDOBm6k_c46FpfI,9599,simonw,2021-11-20T00:42:08Z,2021-11-20T00:42:08Z,OWNER,"> In the meantime I can catch these errors by changing the test to run each path twice, once with and once without the prefix. This should accurately simulate how Apache is working here. This worked, I managed to get the tests to fail! Here's the change I made: ```diff diff --git a/tests/test_html.py b/tests/test_html.py index f24165b..dbdfe59 100644 --- a/tests/test_html.py +++ b/tests/test_html.py @@ -1614,12 +1614,19 @@ def test_metadata_sort_desc(app_client): ""/fixtures/compound_three_primary_keys/a,a,a"", ""/fixtures/paginated_view"", ""/fixtures/facetable"", + ""/fixtures?sql=select+1"", ], ) -def test_base_url_config(app_client_base_url_prefix, path): +@pytest.mark.parametrize(""use_prefix"", (True, False)) +def test_base_url_config(app_client_base_url_prefix, path, use_prefix): client = app_client_base_url_prefix - response = client.get(""/prefix/"" + path.lstrip(""/"")) + path_to_get = path + if use_prefix: + path_to_get = ""/prefix/"" + path.lstrip(""/"") + response = client.get(path_to_get) soup = Soup(response.body, ""html.parser"") + if path == ""/fixtures?sql=select+1"": + assert False for el in soup.findAll([""a"", ""link"", ""script""]): if ""href"" in el.attrs: href = el[""href""] @@ -1642,11 +1649,12 @@ def test_base_url_config(app_client_base_url_prefix, path): # If this has been made absolute it may start http://localhost/ if href.startswith(""http://localhost/""): href = href[len(""http://localost/"") :] - assert href.startswith(""/prefix/""), { + assert href.startswith(""/prefix/""), json.dumps({ ""path"": path, + ""path_to_get"": path_to_get, ""href_or_src"": href, ""element_parent"": str(el.parent), - } + }, indent=4, default=repr) def test_base_url_affects_metadata_extra_css_urls(app_client_base_url_prefix): ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058790545,base_url is omitted in JSON and CSV views, https://github.com/simonw/datasette/issues/1519#issuecomment-974561593,https://api.github.com/repos/simonw/datasette/issues/1519,974561593,IC_kwDOBm6k_c46FqE5,9599,simonw,2021-11-20T00:53:19Z,2021-11-20T00:53:19Z,OWNER,Adding that test found (I hope!) all of the remaining `base_url` bugs. There were a bunch! I think I finally get to close #838 too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058790545,base_url is omitted in JSON and CSV views, https://github.com/simonw/datasette/issues/1519#issuecomment-974562942,https://api.github.com/repos/simonw/datasette/issues/1519,974562942,IC_kwDOBm6k_c46FqZ-,9599,simonw,2021-11-20T00:59:32Z,2021-11-20T00:59:32Z,OWNER,"Ouch a nasty bug crept through there - https://datasette-apache-proxy-demo-j7hipcg4aq-uc.a.run.app/prefix/fixtures/compound_three_primary_keys says > 500: name 'ds' is not defined","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058790545,base_url is omitted in JSON and CSV views, https://github.com/simonw/datasette/issues/1522#issuecomment-974564712,https://api.github.com/repos/simonw/datasette/issues/1522,974564712,IC_kwDOBm6k_c46Fq1o,9599,simonw,2021-11-20T01:07:49Z,2021-11-20T01:10:48Z,OWNER,https://apache-proxy-demo.datasette.io/prefix/fixtures/compound_three_primary_keys has broken suggested facet links - they go to `https://localhost:8001/prefix/fixtures/compound_three_primary_keys?_facet=pk1#facet-pk1` - but I think that's because I'm missing the `ProxyPreserveHost On` setting.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974565392,https://api.github.com/repos/simonw/datasette/issues/1522,974565392,IC_kwDOBm6k_c46FrAQ,9599,simonw,2021-11-20T01:11:20Z,2021-11-20T01:11:20Z,OWNER,"Yup, that fixed it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974565816,https://api.github.com/repos/simonw/datasette/issues/1522,974565816,IC_kwDOBm6k_c46FrG4,9599,simonw,2021-11-20T01:13:39Z,2021-11-20T01:13:39Z,OWNER,"I have a hunch that running `httpd -D FOREGROUND` doesn't show error logs, which would explain why I can't use the Cloud Run logs to figure out the reason for the 503s.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974572505,https://api.github.com/repos/simonw/datasette/issues/1522,974572505,IC_kwDOBm6k_c46FsvZ,9599,simonw,2021-11-20T01:50:48Z,2021-11-20T01:51:41Z,OWNER,"I figured out a recipe to run `httpd` as a service inside Alpine - works great on my laptop, here's my new `Dockerfile`: ```dockerfile FROM python:3-alpine # openrc gives us rc-service RUN apk add --no-cache \ openrc \ apache2 \ apache2-proxy \ bash ARG DATASETTE_REF RUN pip install https://github.com/simonw/datasette/archive/${DATASETTE_REF}.zip # Append this to the end of the default httpd.conf file RUN echo -e 'ServerName localhost\n\ \n\ \n\ Order deny,allow\n\ Allow from all\n\ \n\ \n\ ProxyPreserveHost On\n\ ProxyPass /prefix/ http://127.0.0.1:8001/\n\ Header add X-Proxied-By ""Apache2""' >> /etc/apache2/httpd.conf RUN echo 'Datasette' > /var/www/localhost/htdocs/index.html WORKDIR /app ADD https://latest.datasette.io/fixtures.db /app/fixtures.db EXPOSE 80 # RUN echo -e ""#!/bin/bash\nopenrc default\nrc-service apache2 start;\ndatasette /app/fixtures.db --setting base_url '/prefix/' --version-note '${DATASETTE_REF}' -h 0.0.0.0 -p 8001"" > /app/start.sh RUN echo ""#!/bin/bash"" >> start.sh RUN echo ""openrc default"" >> start.sh RUN echo ""rc-service apache2 start"" >> start.sh RUN echo ""datasette /app/fixtures.db --setting base_url '/prefix/' --version-note '${DATASETTE_REF}' -h 0.0.0.0 -p 8001"" >> /app/start.sh RUN chmod +x /app/start.sh CMD /app/start.sh ``` I'm going to try this on Cloud Run and see if it fixes the 503s One annoying thing about this: Ctrl+C on my laptop no longer stops the container, I have to `docker ps` and then `docker kill xxx` instead.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974573616,https://api.github.com/repos/simonw/datasette/issues/1522,974573616,IC_kwDOBm6k_c46FtAw,9599,simonw,2021-11-20T01:58:44Z,2021-11-20T01:58:44Z,OWNER,"Deploy to Cloud Run appears to hang here: ``` Deploying container to Cloud Run service [datasette-apache-proxy-demo] in project [datasette-222320] region [us-central1] ⠧ Deploying... Revision deployment finished. Waiting for health check to begin. ⠧ Creating Revision... . Routing traffic... ✓ Setting IAM Policy... ``` **Waiting for health check to begin** makes it sound like the container didn't start properly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974575512,https://api.github.com/repos/simonw/datasette/issues/1522,974575512,IC_kwDOBm6k_c46FteY,9599,simonw,2021-11-20T02:09:20Z,2021-11-20T02:09:20Z,OWNER,"> **Waiting for health check to begin** makes it sound like the container didn't start properly. That eventually failed, but I did get these in the build logs: ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974576436,https://api.github.com/repos/simonw/datasette/issues/1522,974576436,IC_kwDOBm6k_c46Fts0,9599,simonw,2021-11-20T02:14:45Z,2021-11-20T02:14:45Z,OWNER,I'm going to try running Apache with `httpd -D FOREGROUND` while running `datasette &`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974576624,https://api.github.com/repos/simonw/datasette/issues/1522,974576624,IC_kwDOBm6k_c46Ftvw,9599,simonw,2021-11-20T02:16:12Z,2021-11-20T02:16:12Z,OWNER,"Again, that approach worked on my laptop but when deployed to Cloud Run mostly gave me 503 errors for the `/prefix/` page, with the occasional 200. I did this: ```Dockerfile RUN echo ""#!/bin/bash"" >> start.sh # Start Datasette running in background with & RUN echo ""datasette /app/fixtures.db --setting base_url '/prefix/' --version-note '${DATASETTE_REF}' -h 0.0.0.0 -p 8001 &"" >> /app/start.sh RUN echo ""httpd -D FOREGROUND"" >> /app/start.sh RUN chmod +x /app/start.sh CMD /app/start.sh ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974577082,https://api.github.com/repos/simonw/datasette/issues/1522,974577082,IC_kwDOBm6k_c46Ft26,9599,simonw,2021-11-20T02:19:27Z,2021-11-20T02:19:27Z,OWNER,"https://docs.docker.com/config/containers/multi-service_container/ suggests `supervisord` as a last resort. https://stackoverflow.com/a/49100302/6083 has a neat looking recipe for than in Alpine: > **1.** `Dockerfile` is: > > FROM alpine:latest > RUN apk update && apk add --no-cache supervisor openssh nginx > COPY supervisord.conf /etc/supervisord.conf > CMD [""/usr/bin/supervisord"", ""-c"", ""/etc/supervisord.conf""] > > **2.** `supervisord.conf` is: > > [supervisord] > nodaemon=true > > [program:sshd] > command=/usr/sbin/sshd -D > > [program:nginx] > command=nginx -c /etc/nginx/nginx.conf ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974577565,https://api.github.com/repos/simonw/datasette/issues/1522,974577565,IC_kwDOBm6k_c46Ft-d,9599,simonw,2021-11-20T02:23:07Z,2021-11-20T02:23:07Z,OWNER,"OK, that works on my laptop - and Ctrl+C quits it, which is nice: ``` apache-proxy % docker run -p 5000:80 --rm datasette-build-arg-demo 2021-11-20 02:22:13,925 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message. 2021-11-20 02:22:13,927 INFO supervisord started with pid 1 2021-11-20 02:22:14,931 INFO spawned: 'datasette' with pid 7 2021-11-20 02:22:14,934 INFO spawned: 'httpd' with pid 8 2021-11-20 02:22:16,484 INFO success: datasette entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2021-11-20 02:22:16,484 INFO success: httpd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) ^C 2021-11-20 02:22:26,285 WARN received SIGINT indicating exit request 2021-11-20 02:22:26,286 INFO waiting for datasette, httpd to die 2021-11-20 02:22:26,315 INFO stopped: httpd (exit status 0) 2021-11-20 02:22:26,540 INFO stopped: datasette (exit status 0) ``` Here's my new Dockerfile: ```dockerfile FROM python:3-alpine RUN apk add --no-cache \ apache2 \ apache2-proxy \ supervisor \ bash ARG DATASETTE_REF RUN pip install https://github.com/simonw/datasette/archive/${DATASETTE_REF}.zip # Append this to the end of the default httpd.conf file RUN echo -e 'ServerName localhost\n\ \n\ \n\ Order deny,allow\n\ Allow from all\n\ \n\ \n\ ProxyPreserveHost On\n\ ProxyPass /prefix/ http://127.0.0.1:8001/\n\ Header add X-Proxied-By ""Apache2""' >> /etc/apache2/httpd.conf RUN echo 'Datasette' > /var/www/localhost/htdocs/index.html WORKDIR /app ADD https://latest.datasette.io/fixtures.db /app/fixtures.db EXPOSE 80 RUN echo ""[supervisord]"" >> /app/supervisord.conf RUN echo ""nodaemon=true"" >> /app/supervisord.conf RUN echo """" >> /app/supervisord.conf RUN echo ""[program:httpd]"" >> /app/supervisord.conf RUN echo ""command=httpd -D FOREGROUND"" >> /app/supervisord.conf RUN echo """" >> /app/supervisord.conf RUN echo ""[program:datasette]"" >> /app/supervisord.conf RUN echo ""command=datasette /app/fixtures.db --setting base_url '/prefix/' --version-note '${DATASETTE_REF}' -h 0.0.0.0 -p 8001"" >> /app/supervisord.conf CMD [""/usr/bin/supervisord"", ""-c"", ""/app/supervisord.conf""] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974577949,https://api.github.com/repos/simonw/datasette/issues/1522,974577949,IC_kwDOBm6k_c46FuEd,9599,simonw,2021-11-20T02:26:09Z,2021-11-20T02:26:17Z,OWNER,"So frustrating, that's giving me the same problem after being deployed! 503 errors for the first while, then it starts working.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974578141,https://api.github.com/repos/simonw/datasette/issues/1522,974578141,IC_kwDOBm6k_c46FuHd,9599,simonw,2021-11-20T02:27:23Z,2021-11-20T02:27:23Z,OWNER,"Aha! This could be the clue I was looking for: https://www.reddit.com/r/googlecloud/comments/fmkx63/comment/fl5csty/?utm_source=reddit&utm_medium=web2x&context=3 > Are you processing on a background thread in your container? If so, it's likely your problem, because cloud run will put your app into a low power state between http requests. For long running tasks in cloud run, you need to keep the http connection open, and not return until you are done. Maybe the `datasette &` process is being affected by that in some way?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974585374,https://api.github.com/repos/simonw/datasette/issues/1522,974585374,IC_kwDOBm6k_c46Fv4e,9599,simonw,2021-11-20T03:28:58Z,2021-11-20T03:28:58Z,OWNER,Based on https://medium.com/google-cloud/init-process-for-containers-d03a471fa0cc I might try s6.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974602459,https://api.github.com/repos/simonw/datasette/issues/1522,974602459,IC_kwDOBm6k_c46F0Db,9599,simonw,2021-11-20T06:15:58Z,2021-11-20T06:15:58Z,OWNER,First I'm going to try using Debian Buster as the base image instead of Alpine.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974605128,https://api.github.com/repos/simonw/datasette/issues/1522,974605128,IC_kwDOBm6k_c46F0tI,9599,simonw,2021-11-20T06:47:59Z,2021-11-20T06:47:59Z,OWNER,"I managed to port the whole thing over to Debian - which took a lot of work because their packaged Apache 2 works very differently from the Alpine one. Once again... I got it working fine on my laptop, but the image deployed to Cloud Run throws 503 errors! ```dockerfile FROM python:3.9.7-slim-bullseye RUN apt-get update && \ apt-get install -y apache2 supervisor && \ apt clean && \ rm -rf /var/lib/apt && \ rm -rf /var/lib/dpkg/info/* # Apache environment, copied from # https://github.com/ijklim/laravel-benfords-law-app/blob/e9bf385dcaddb62ea466a7b245ab6e4ef708c313/docker/os/Dockerfile ENV APACHE_DOCUMENT_ROOT=/var/www/html/public ENV APACHE_RUN_USER www-data ENV APACHE_RUN_GROUP www-data ENV APACHE_PID_FILE /var/run/apache2.pid ENV APACHE_RUN_DIR /var/run/apache2 ENV APACHE_LOCK_DIR /var/lock/apache2 ENV APACHE_LOG_DIR /var/log RUN ln -sf /dev/stdout /var/log/apache2-access.log RUN ln -sf /dev/stderr /var/log/apache2-error.log RUN mkdir -p $APACHE_RUN_DIR $APACHE_LOCK_DIR RUN a2enmod proxy RUN a2enmod proxy_http RUN a2enmod headers ARG DATASETTE_REF RUN pip install https://github.com/simonw/datasette/archive/${DATASETTE_REF}.zip # Append this to the end of the default httpd.conf file RUN echo '\n\ \n\ Options Indexes FollowSymLinks\n\ AllowOverride None\n\ Require all granted\n\ \n\ \n\ \n\ ServerName localhost\n\ DocumentRoot /app/html\n\ ProxyPreserveHost On\n\ ProxyPass /prefix/ http://127.0.0.1:8001/\n\ Header add X-Proxied-By ""Apache2""\n\ \n\ ' > /etc/apache2/sites-enabled/000-default.conf WORKDIR /app RUN mkdir -p /app/html RUN echo 'Datasette' > /app/html/index.html ADD https://latest.datasette.io/fixtures.db /app/fixtures.db EXPOSE 80 RUN echo ""[supervisord]"" >> /app/supervisord.conf RUN echo ""nodaemon=true"" >> /app/supervisord.conf RUN echo """" >> /app/supervisord.conf RUN echo ""[program:apache2]"" >> /app/supervisord.conf RUN echo ""command=apache2 -D FOREGROUND"" >> /app/supervisord.conf RUN echo """" >> /app/supervisord.conf RUN echo ""[program:datasette]"" >> /app/supervisord.conf RUN echo ""command=datasette /app/fixtures.db --setting base_url '/prefix/' --version-note '${DATASETTE_REF}' -h 0.0.0.0 -p 8001"" >> /app/supervisord.conf CMD [""/usr/bin/supervisord"", ""-c"", ""/app/supervisord.conf""] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974605529,https://api.github.com/repos/simonw/datasette/issues/1522,974605529,IC_kwDOBm6k_c46F0zZ,9599,simonw,2021-11-20T06:52:21Z,2021-11-20T06:52:21Z,OWNER,"I've now tried both Debian and Alpine, and I've tried both `tini` and `supervisord`. Each time I get the same result - I get 503 errors for the first dozen or so refreshes of `/prefix/` followed by it intermittently working. Absolutely stumped.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974607456,https://api.github.com/repos/simonw/datasette/issues/1522,974607456,IC_kwDOBm6k_c46F1Rg,17906,mrchrisadams,2021-11-20T07:10:11Z,2021-11-20T07:10:11Z,NONE,"As a a sanity check, would it be worth looking at trying to push the multi-process container on another provider of a knative / cloud run / tekton ? I have a somewhat similar use case for a future proejct, so i'm been very grateful to you sharing all the progress in this issue. As I understand it, Scaleway also offer a very similar offering using what appear to be many similar components that might at least see if it's an issue with more than one knative based FaaS provider https://www.scaleway.com/en/serverless-containers/ https://developers.scaleway.com/en/products/containers/api/#main-features ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974682507,https://api.github.com/repos/simonw/datasette/issues/1522,974682507,IC_kwDOBm6k_c46GHmL,9599,simonw,2021-11-20T17:24:13Z,2021-11-20T17:24:13Z,OWNER,"I'm going to leave this issue open, tag it as ""help wanted"" and cross my fingers that someone with Cloud Run deep expertise takes an interest in figuring out what's going wrong here!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974683220,https://api.github.com/repos/simonw/datasette/issues/1522,974683220,IC_kwDOBm6k_c46GHxU,9599,simonw,2021-11-20T17:29:12Z,2021-11-20T17:29:12Z,OWNER,"> As a a sanity check, would it be worth looking at trying to push the multi-process container on another provider of a knative / cloud run / tekton ? I have a somewhat similar use case for a future proejct, so i'm been very grateful to you sharing all the progress in this issue. That's a great idea. I'll try running on a non-Knative host too (probably Fly - though they actually run containers using Firecracker which ends up being completely different). Cloud Run are the only Knative host I've used, know of any others aside from Scaleway? They look like they're worth getting familiar with.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974685095,https://api.github.com/repos/simonw/datasette/issues/1522,974685095,IC_kwDOBm6k_c46GIOn,9599,simonw,2021-11-20T17:42:25Z,2021-11-20T17:42:25Z,OWNER,"I tried to deploy it to Fly - initially using `flyctl launch` but then switching to `flyctl deploy` so I could use the `--build-arg` option (posted [a feature request](https://community.fly.io/t/feature-request-flyctl-launch-build-arg/3209) here). Almost got it working, but it failed the health check: ``` % cd datasette/demos/apache-proxy apache-proxy % flyctl launch Creating app in /Users/simon/Dropbox/Development/datasette/demos/apache-proxy Scanning source code Detected Dockerfile app Automatically selected personal organization: Simon Willison ? Select region: sjc (Sunnyvale, California (US)) Created app floral-dust-4577 in organization personal Wrote config file fly.toml Your app is ready. Deploy with `flyctl deploy` ? Would you like to deploy now? Yes Deploying floral-dust-4577 ==> Validating app configuration --> Validating app configuration done Services TCP 80/443 ⇢ 8080 ==> Creating build context --> Creating build context done ==> Building image with Docker Sending build context to Docker daemon 8.704kB ... Error error building: executor failed running [/bin/sh -c pip install https://github.com/simonw/datasette/archive/${DATASETTE_REF}.zip]: exit code: 1 # I didn't pass the build argument, trying again with flyctl deploy apache-proxy % flyctl deploy --build-arg DATASETTE_REF=main Update available 0.0.229 -> v0.0.255 Run ""flyctl version update"" to upgrade Deploying floral-dust-4577 ==> Validating app configuration --> Validating app configuration done Services TCP 80/443 ⇢ 8080 ==> Creating build context --> Creating build context done ==> Building image with Docker Sending build context to Docker daemon 8.704kB [+] Building 15.7s (27/27) ... 0.0s ==> Pushing image to fly The push refers to repository [registry.fly.io/floral-dust-4577] 9bf88c92aa2a: Pushed 3d61728b8391: Pushed ... --> Pushing image done Image: registry.fly.io/floral-dust-4577:deployment-1637429501 Image size: 276 MB ==> Creating release Release v2 created You can detach the terminal anytime without stopping the deployment Monitoring Deployment 1 desired, 1 placed, 0 healthy, 0 unhealthy [health checks: 1 total, 1 critical] 1 desired, 1 placed, 0 healthy, 1 unhealthy [health checks: 1 total, 1 critical] v0 failed - Failed due to unhealthy allocations - no stable job version to auto revert to Failed Instances ==> Failure #1 Instance ID = 36adac86 Version = 0 Region = sjc Desired = run Status = running Health Checks = 1 total, 1 critical Restarts = 0 Created = 4m52s ago Recent Events TIMESTAMP TYPE MESSAGE 2021-11-20T17:32:52Z Received Task received by client 2021-11-20T17:32:52Z Task Setup Building Task Directory 2021-11-20T17:33:02Z Started Task started by client Recent Logs 2021-11-20T17:32:56Z [info] Unpacking image 2021-11-20T17:33:01Z [info] Preparing kernel init 2021-11-20T17:33:01Z [info] Configuring firecracker 2021-11-20T17:33:02Z [info] Starting virtual machine 2021-11-20T17:33:02Z [info] Starting init (commit: 7943db6)... 2021-11-20T17:33:02Z [info] Preparing to run: `/usr/bin/supervisord -c /app/supervisord.conf` as root 2021-11-20T17:33:02Z [info] 2021/11/20 17:33:02 listening on [fdaa:0:4ef:a7b:2295:36ad:ac86:2]:22 (DNS: [fdaa::3]:53) 2021-11-20T17:33:02Z [info] 2021-11-20 17:33:02,374 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message. 2021-11-20T17:33:02Z [info] 2021-11-20 17:33:02,376 INFO supervisord started with pid 510 2021-11-20T17:33:03Z [info] 2021-11-20 17:33:03,379 INFO spawned: 'apache2' with pid 515 2021-11-20T17:33:03Z [info] 2021-11-20 17:33:03,381 INFO spawned: 'datasette' with pid 516 2021-11-20T17:33:05Z [info] 2021-11-20 17:33:05,068 INFO success: apache2 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2021-11-20T17:33:05Z [info] 2021-11-20 17:33:05,068 INFO success: datasette entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2021-11-20T17:33:28Z [error] Health check status changed 'warning' => 'critical' ***v0 failed - Failed due to unhealthy allocations - no stable job version to auto revert to and deploying as v1 ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974692546,https://api.github.com/repos/simonw/datasette/issues/1522,974692546,IC_kwDOBm6k_c46GKDC,9599,simonw,2021-11-20T18:34:13Z,2021-11-20T18:34:13Z,OWNER,"Here's why it failed - the `fly.toml` file that was generated when I ran `flyctl launch` had this section: ```toml [[services]] http_checks = [] internal_port = 8080 processes = [""app""] protocol = ""tcp"" script_checks = [] ``` But I need `internal_port` to be 80 for Apache, so I changed that and ran `flyctl deploy --build-arg DATASETTE_REF=main` again - and it worked! https://floral-dust-4577.fly.dev/prefix/ - not seeing any 503 errors there.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974693350,https://api.github.com/repos/simonw/datasette/issues/1522,974693350,IC_kwDOBm6k_c46GKPm,9599,simonw,2021-11-20T18:39:27Z,2021-11-20T18:39:27Z,OWNER,"I'm going to go with Fly instead for this, especially as I can keep it within their free tier (and iI want to get more familiar with their platform).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1522#issuecomment-974695111,https://api.github.com/repos/simonw/datasette/issues/1522,974695111,IC_kwDOBm6k_c46GKrH,9599,simonw,2021-11-20T18:52:11Z,2021-11-20T18:52:11Z,OWNER,The demo is now live on https://datasette-apache-proxy-demo.fly.dev/prefix/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236,Deploy a live instance of demos/apache-proxy, https://github.com/simonw/datasette/issues/1519#issuecomment-974697824,https://api.github.com/repos/simonw/datasette/issues/1519,974697824,IC_kwDOBm6k_c46GLVg,9599,simonw,2021-11-20T19:11:21Z,2021-11-20T19:11:21Z,OWNER,"OK, i think I got all of them this time! The latest demo is now live at https://datasette-apache-proxy-demo.fly.dev/prefix/fixtures/sortable?_facet=pk2 I'm closing this issue, but feel free to re-open it if you spot any that I missed.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058790545,base_url is omitted in JSON and CSV views, https://github.com/simonw/datasette/issues/1519#issuecomment-974701788,https://api.github.com/repos/simonw/datasette/issues/1519,974701788,IC_kwDOBm6k_c46GMTc,9599,simonw,2021-11-20T19:42:29Z,2021-11-20T19:42:29Z,OWNER,"> I think what's happening here is Apache is actually making a request to `/fixtures` rather than making a request to `/prefix/fixtures` - and Datasette is replying to requests on both the prefixed and the non-prefixed paths. > > This is pretty confusing! I think Datasette should ONLY reply to `/prefix/fixtures` instead and return a 404 for `/fixtures` - this would make things a whole lot easier to debug. > > But shipping that change could break existing deployments. Maybe that should be a breaking change for 1.0. On further thought I'm not going to do this. Having Datasette work behind a proxy the way it does right now is clearly easy for people to deploy (now that I've fixed the bugs) and I trust my improved tests to catch problems in the future.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058790545,base_url is omitted in JSON and CSV views, https://github.com/simonw/datasette/issues/1524#issuecomment-974704254,https://api.github.com/repos/simonw/datasette/issues/1524,974704254,IC_kwDOBm6k_c46GM5-,9599,simonw,2021-11-20T20:03:51Z,2021-11-20T20:22:52Z,OWNER,"I'm also going to extract the Apache config files from https://github.com/simonw/datasette/blob/250db8192cb8aba5eb8cd301ccc2a49525bc3d24/demos/apache-proxy/Dockerfile into a separate file to make it easier to read. (The supervisor config needs to be dynamically constructed to include $DATASETTE_REF so I will leave it where it is.)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1059219106,"Improve Apache proxy documentation, link to demo", https://github.com/simonw/datasette/issues/1524#issuecomment-974707878,https://api.github.com/repos/simonw/datasette/issues/1524,974707878,IC_kwDOBm6k_c46GNym,9599,simonw,2021-11-20T20:34:51Z,2021-11-20T20:38:29Z,OWNER,"I pointed `CNAME` of `datasette-apache-proxy-demo.datasette.io` at `datasette-apache-proxy-demo.fly.dev.` using Vercel DNS: Then I asked Fly to issue a LetsEncrypt certificate for that: ``` % flyctl certs create datasette-apache-proxy-demo.datasette.io # About 53 seconds later: % flyctl certs show datasette-apache-proxy-demo.datasette.io The certificate for datasette-apache-proxy-demo.datasette.io has been issued. Hostname = datasette-apache-proxy-demo.datasette.io DNS Provider = constellix Certificate Authority = Let's Encrypt Issued = ecdsa,rsa Added to App = 53 seconds ago Source = fly ``` https://datasette-apache-proxy-demo.datasette.io/ works now - I'll use that in the documentation.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1059219106,"Improve Apache proxy documentation, link to demo", https://github.com/simonw/datasette/issues/1426#issuecomment-974711959,https://api.github.com/repos/simonw/datasette/issues/1426,974711959,IC_kwDOBm6k_c46GOyX,52649,tannewt,2021-11-20T21:11:51Z,2021-11-20T21:11:51Z,NONE,I think another thing would be to make `/pages/robots.txt` work. That way you can use jinja to generate a desired robots.txt. I'm using it to allow the main index and what it links to to be crawled (but not the database pages directly.),"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",964322136,"Manage /robots.txt in Datasette core, block robots by default", https://github.com/simonw/datasette/issues/1524#issuecomment-974721652,https://api.github.com/repos/simonw/datasette/issues/1524,974721652,IC_kwDOBm6k_c46GRJ0,9599,simonw,2021-11-20T22:41:03Z,2021-11-20T22:41:03Z,OWNER,New TIL: https://til.simonwillison.net/fly/custom-subdomain-fly,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1059219106,"Improve Apache proxy documentation, link to demo", https://github.com/simonw/datasette/issues/1524#issuecomment-974725814,https://api.github.com/repos/simonw/datasette/issues/1524,974725814,IC_kwDOBm6k_c46GSK2,9599,simonw,2021-11-20T23:24:01Z,2021-11-20T23:24:01Z,OWNER,"I noticed that `http://datasette-apache-proxy-demo.datasette.io/` wasn't redirecting to `https` so I built a new plugin: https://github.com/simonw/datasette-redirect-to-https ``` % curl -i 'http://datasette-apache-proxy-demo.datasette.io/prefix/fixtures/no_primary_key' HTTP/1.1 301 Moved Permanently date: Sat, 20 Nov 2021 23:22:50 GMT server: Fly/51d150d (2021-11-19) location: https://datasette-apache-proxy-demo.datasette.io/fixtures/no_primary_key x-proxied-by: Apache2 Debian transfer-encoding: chunked via: 1.1 fly.io fly-request-id: 01FMZTHTHVPC8BZY0625D7JV4B ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1059219106,"Improve Apache proxy documentation, link to demo",