issue_comments
7,931 rows where author_association = "OWNER" sorted by node_id
This data as json, CSV (advanced)
id | html_url | issue_url | node_id ▼ | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
907537366 | https://github.com/simonw/datasette/issues/1449#issuecomment-907537366 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42F-vW | simonw 9599 | 2021-08-28T00:29:16Z | 2021-08-28T00:29:29Z | OWNER | The closest plugin hook to this right now is [publish_subcommand](https://docs.datasette.io/en/stable/plugin_hooks.html#publish-subcommand-publish) - which looks like this: ```python @hookimpl def publish_subcommand(publish): @publish.command() @add_common_publish_arguments_and_options @click.option( "-k", "--api_key", help="API key for talking to my hosting provider", ) def my_hosting_provider(...): ``` But there are also several plugin hooks with `register_` prefixes, which may be a good naming convention to stick to here: `register_output_renderer(datasette)`, `register_routes(datasette)`, `register_facet_classes()`, `register_magic_parameters(datasette)`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
907537610 | https://github.com/simonw/datasette/issues/1449#issuecomment-907537610 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42F-zK | simonw 9599 | 2021-08-28T00:30:51Z | 2021-08-28T00:30:51Z | OWNER | There's also the option for plugins to muck around with existing registered commands - this could get a bit untidy if multiple plugins try to do it, but being able to replace `serve` with a fresh implementation that adds an additional command-line option before calling back to the original might open up some interesting possibilities. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
907542214 | https://github.com/simonw/datasette/issues/1449#issuecomment-907542214 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42F_7G | simonw 9599 | 2021-08-28T01:02:38Z | 2021-08-28T01:02:38Z | OWNER | Partial prototype of `datasette-verify`: ```python from datasette import hookimpl import click @hookimpl def register_commands(cli): from datasette.cli import sqlite_extensions @cli.command() @click.argument("files", type=click.Path(exists=True), nargs=-1) @sqlite_extensions def verify(files, sqlite_extensions): "Verify that files can be opened by Datasette" for file in files: print(file) ``` I had to move the `from datasette.cli import sqlite_extensions` inside the hook function to avoid a circular import. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
907538940 | https://github.com/simonw/datasette/issues/1449#issuecomment-907538940 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42F_H8 | simonw 9599 | 2021-08-28T00:39:28Z | 2021-08-28T00:39:28Z | OWNER | Here's the answer to that: ``` ~ % datasette --help Usage: datasette [OPTIONS] COMMAND [ARGS]... Datasette! Options: --version Show the version and exit. --help Show this message and exit. Commands: serve* Serve up specified SQLite database files with a web UI inspect install Install Python packages - e.g. package Package specified SQLite files into a new datasette Docker... plugins List currently available plugins publish Publish specified SQLite database files to the internet... uninstall Uninstall Python packages (e.g. ``` Since it's adding extra things that show up in `--help` under the "Commands:" heading, I should call them commands. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
907539065 | https://github.com/simonw/datasette/issues/1449#issuecomment-907539065 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42F_J5 | simonw 9599 | 2021-08-28T00:40:24Z | 2021-08-28T00:40:24Z | OWNER | I'm going to call the new hook `register_commands` - since it will allow ambitious plugins to register more than one command if they want to. That's also pleasingly similar to `register_routes`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
907539251 | https://github.com/simonw/datasette/issues/1449#issuecomment-907539251 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42F_Mz | simonw 9599 | 2021-08-28T00:41:37Z | 2021-08-28T00:41:50Z | OWNER | The first example plugin I'm going to build for this will be `datasette verify file.db file2.db` - it will take one or more paths to SQLite files and verify if they can be opened by Datasette or not. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
907539668 | https://github.com/simonw/datasette/issues/1449#issuecomment-907539668 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42F_TU | simonw 9599 | 2021-08-28T00:44:16Z | 2021-08-28T00:44:16Z | OWNER | Considering this piece of code: https://github.com/simonw/datasette/blob/a1a33bb5822214be1cebd98cd858b2058d91a4aa/datasette/cli.py#L122-L142 I think the hook itself gets called with a single argument, `cli`, which it can then use in the standard Click way to register extra stuff. I can't pass it `datasette` (like I do with `register_routes()`) because the Datasette object itself is instantiated by the `serve` command, which will not have been called. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
907540240 | https://github.com/simonw/datasette/issues/1450#issuecomment-907540240 | https://api.github.com/repos/simonw/datasette/issues/1450 | IC_kwDOBm6k_c42F_cQ | simonw 9599 | 2021-08-28T00:48:30Z | 2021-08-28T00:48:30Z | OWNER | I'll go with this: ``` % datasette --help Usage: datasette [OPTIONS] COMMAND [ARGS]... Datasette is an open source multi-tool for exploring and publishing data About Datasette: https://datasette.io/ Full documentation: https://docs.datasette.io/ Options: --version Show the version and exit. --help Show this message and exit. ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Datasette --help should show something more useful than "Datasette!" 981681138 | |
907540790 | https://github.com/simonw/datasette/issues/1449#issuecomment-907540790 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42F_k2 | simonw 9599 | 2021-08-28T00:52:32Z | 2021-08-28T00:52:32Z | OWNER | I don't think I can get this new hook to support the handy [--plugins-dir= mechanism](https://docs.datasette.io/en/stable/plugins.html#one-off-plugins-using-plugins-dir) for loading plugins from Python files as opposed to registering them with setuptools - that mechanism is itself implemented inside of code called by `datasette serve` so I don't have a way of taking advantage of it from outside that command. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
907540928 | https://github.com/simonw/datasette/issues/1449#issuecomment-907540928 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42F_nA | simonw 9599 | 2021-08-28T00:53:37Z | 2021-08-28T00:53:37Z | OWNER | I'll probably have to use this mechanism for the tests then: https://til.simonwillison.net/pytest/registering-plugins-in-tests | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
907543982 | https://github.com/simonw/datasette/issues/1449#issuecomment-907543982 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42GAWu | simonw 9599 | 2021-08-28T01:14:27Z | 2021-08-28T01:14:27Z | OWNER | Writing the test for this is proving difficult, because the `cli` module has already been imported when I attempt to register a new plugin - so it doesn't pick up on the additional command registrations. Trying to work around that with `importlib.reload(cli)`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
907547624 | https://github.com/simonw/datasette/issues/1449#issuecomment-907547624 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42GBPo | simonw 9599 | 2021-08-28T01:44:57Z | 2021-08-28T01:58:35Z | OWNER | Documentation: https://docs.datasette.io/en/latest/plugin_hooks.html#register-commands-cli | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
907547736 | https://github.com/simonw/datasette/issues/1449#issuecomment-907547736 | https://api.github.com/repos/simonw/datasette/issues/1449 | IC_kwDOBm6k_c42GBRY | simonw 9599 | 2021-08-28T01:45:36Z | 2021-08-28T01:45:36Z | OWNER | I need to push this out as an alpha so I can release a demo plugin that uses it. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `register_commands()` plugin hook to register extra CLI commands 981676832 | |
908832938 | https://github.com/simonw/datasette/issues/1446#issuecomment-908832938 | https://api.github.com/repos/simonw/datasette/issues/1446 | IC_kwDOBm6k_c42K7Cq | simonw 9599 | 2021-08-31T01:54:59Z | 2021-08-31T01:54:59Z | OWNER | I used the sticky footer mechanism in `datasette.app`: https://github.com/simonw/datasette.app/issues/3 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Modify base.html template to support optional sticky footer 978357984 | |
913001298 | https://github.com/simonw/datasette/pull/1455#issuecomment-913001298 | https://api.github.com/repos/simonw/datasette/issues/1455 | IC_kwDOBm6k_c42a0tS | simonw 9599 | 2021-09-04T16:31:32Z | 2021-09-04T16:31:32Z | OWNER | Great idea! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add scientists to target groups 988325628 | |
913001416 | https://github.com/simonw/datasette/pull/1455#issuecomment-913001416 | https://api.github.com/repos/simonw/datasette/issues/1455 | IC_kwDOBm6k_c42a0vI | simonw 9599 | 2021-09-04T16:32:21Z | 2021-09-04T16:32:21Z | OWNER | I'll add researchers too. | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add scientists to target groups 988325628 | |
913218494 | https://github.com/simonw/datasette/issues/1459#issuecomment-913218494 | https://api.github.com/repos/simonw/datasette/issues/1459 | IC_kwDOBm6k_c42bpu- | simonw 9599 | 2021-09-05T19:58:51Z | 2021-09-05T19:59:15Z | OWNER | This idea makes sense to me. There's actually an existing option that takes a path, called `--get` - it returns the HTML or JSON for that oath directly to the console, eg `datasette my.db --get /mydb/mytable.json` So... one option would be to allow combining that with `-o` to open that URL in the browser: datasette my.db -o --get /mydb So some options here are: - `datasette my.db --open-url /mydb` - `datasette my.db --open-path /mydb` - `datasette my.db --open --get /mydb` I quite like that last combination option, mainly to avoid adding even more command options. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | suggestion: allow `datasette --open` to take a relative URL 988556488 | |
914439356 | https://github.com/simonw/datasette/issues/1461#issuecomment-914439356 | https://api.github.com/repos/simonw/datasette/issues/1461 | IC_kwDOBm6k_c42gTy8 | simonw 9599 | 2021-09-07T16:11:37Z | 2021-09-07T16:11:37Z | OWNER | ``` (datasette) datasette % blacken-docs docs/*.rst docs/authentication.rst: Rewriting... docs/internals.rst:169: code block parse error Cannot parse: 14:0: <line number missing in source> docs/plugin_hooks.rst:251: code block parse error Cannot parse: 6:4: ] docs/plugin_hooks.rst:312: code block parse error Cannot parse: 38:0: <line number missing in source> docs/spatialite.rst: Rewriting... docs/testing_plugins.rst:135: code block parse error Cannot parse: 5:0: <line number missing in source> docs/writing_plugins.rst: Rewriting... ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Try blacken-docs 989986586 | |
914440282 | https://github.com/simonw/datasette/issues/1461#issuecomment-914440282 | https://api.github.com/repos/simonw/datasette/issues/1461 | IC_kwDOBm6k_c42gUBa | simonw 9599 | 2021-09-07T16:12:57Z | 2021-09-07T16:12:57Z | OWNER | Here's the diff it produced from that first run: ```diff diff --git a/docs/authentication.rst b/docs/authentication.rst index 0d98cf8..8008023 100644 --- a/docs/authentication.rst +++ b/docs/authentication.rst @@ -381,11 +381,7 @@ Authentication plugins can set signed ``ds_actor`` cookies themselves like so: .. code-block:: python response = Response.redirect("/") - response.set_cookie("ds_actor", datasette.sign({ - "a": { - "id": "cleopaws" - } - }, "actor")) + response.set_cookie("ds_actor", datasette.sign({"a": {"id": "cleopaws"}}, "actor")) Note that you need to pass ``"actor"`` as the namespace to :ref:`datasette_sign`. @@ -412,12 +408,16 @@ To include an expiry, add a ``"e"`` key to the cookie value containing a `base62 expires_at = int(time.time()) + (24 * 60 * 60) response = Response.redirect("/") - response.set_cookie("ds_actor", datasette.sign({ - "a": { - "id": "cleopaws" - }, - "e": baseconv.base62.encode(expires_at), - }, "actor")) + response.set_cookie( + "ds_actor", + datasette.sign( + { + "a": {"id": "cleopaws"}, + "e": baseconv.base62.encode(expires_at), + }, + "actor", + ), + ) The resulting cookie will encode data that looks something like this: diff --git a/docs/spatialite.rst b/docs/spatialite.rst index d1b300b..556bad8 100644 --- a/docs/spatialite.rst +++ b/docs/spatialite.rst @@ -58,19 +58,22 @@ Here's a recipe for taking a table with existing latitude and longitude columns, .. code-block:: python import sqlite3 - conn = sqlite3.connect('museums.db') + + conn = sqlite3.connect("museums.db") # Lead the spatialite extension: conn.enable_load_extension(True) - conn.load_extension('/usr/local/lib/mod_spatialite.dylib') + conn.load_extension("/usr/local/lib/mod_spatialite.dylib") # Initialize spatial metadata for thi… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Try blacken-docs 989986586 | |
914441037 | https://github.com/simonw/datasette/issues/1461#issuecomment-914441037 | https://api.github.com/repos/simonw/datasette/issues/1461 | IC_kwDOBm6k_c42gUNN | simonw 9599 | 2021-09-07T16:13:59Z | 2021-09-07T16:13:59Z | OWNER | I don't think I'll adopt it for this project. For example, here: ```diff response = Response.redirect("/") - response.set_cookie("ds_actor", datasette.sign({ - "a": { - "id": "cleopaws" - } - }, "actor")) + response.set_cookie("ds_actor", datasette.sign({"a": {"id": "cleopaws"}}, "actor")) ``` I chose to use the multi-line version to help emphasize the structure - the single-line replacement loses that. I think I'll continue to make my own editorial choices about how the code examples are laid out. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Try blacken-docs 989986586 | |
914644260 | https://github.com/simonw/datasette/issues/1462#issuecomment-914644260 | https://api.github.com/repos/simonw/datasette/issues/1462 | IC_kwDOBm6k_c42hF0k | simonw 9599 | 2021-09-07T21:34:32Z | 2021-09-07T21:34:32Z | OWNER | I think this is a setting. There are two relevant settings at the moment: ``` "template_debug": false, "trace_debug": false, ``` For consistence then this should be called `something_debug` - but do I want a single setting that exposes the `_internal` database and adds those debug options to the menu, or do I want those as two separate settings? - `internal_debug` to enable access to that `_internal` database - `menu_debug` for those menu options? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Separate out "debug" options from "root" options 990367646 | |
917839801 | https://github.com/simonw/datasette/issues/1468#issuecomment-917839801 | https://api.github.com/repos/simonw/datasette/issues/1468 | IC_kwDOBm6k_c42tR-5 | simonw 9599 | 2021-09-13T04:54:17Z | 2021-09-13T04:54:17Z | OWNER | Here's a already open issue for this: #972 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Faceting for custom SQL queries 994390593 | |
917839507 | https://github.com/simonw/datasette/issues/1468#issuecomment-917839507 | https://api.github.com/repos/simonw/datasette/issues/1468 | IC_kwDOBm6k_c42tR6T | simonw 9599 | 2021-09-13T04:53:22Z | 2021-09-13T04:53:22Z | OWNER | At the moment this isn't possible - though there's a workaround which is to define a SQL view for the query, at which point facets will be displayed again. I did a lot of the work required to support this when I refactored how facets worked a while back - but to finally implement this I need to refactor the table view and the arbitrary query view to share much more logic than they do at the moment. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Faceting for custom SQL queries 994390593 | |
917839062 | https://github.com/simonw/datasette/issues/1469#issuecomment-917839062 | https://api.github.com/repos/simonw/datasette/issues/1469 | IC_kwDOBm6k_c42tRzW | simonw 9599 | 2021-09-13T04:52:01Z | 2021-09-13T04:52:01Z | OWNER | Here's the code at fault: https://github.com/simonw/datasette/blob/b28b6cd2fe97f7e193a235877abeec2c8eb0a821/datasette/static/table.js#L137-L146 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Column cog shows "facet by this" when already default faceted 994450961 | |
917840012 | https://github.com/simonw/datasette/issues/1466#issuecomment-917840012 | https://api.github.com/repos/simonw/datasette/issues/1466 | IC_kwDOBm6k_c42tSCM | simonw 9599 | 2021-09-13T04:54:59Z | 2021-09-13T04:54:59Z | OWNER | Especially relevant now that 0.2.0 is out which is a much higher quality release. https://github.com/simonw/datasette-app/releases/tag/0.2.0 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add Datasette Desktop to installation documentation 991467558 | |
939074818 | https://github.com/simonw/datasette/pull/1481#issuecomment-939074818 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c43-SUC | simonw 9599 | 2021-10-08T19:40:23Z | 2021-10-08T19:40:23Z | OWNER | Then I created myself a temporary 3.10 environment using `pipenv` like so: cd /tmp mkdir py310 cd py310 pipenv shell --python /Users/simon/.pyenv/versions/3.10.0/bin/python And used that with my Datasette checkout like so: cd ~/.../datasette pip install -e '.[test]' pytest | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
939075686 | https://github.com/simonw/datasette/pull/1481#issuecomment-939075686 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c43-Shm | simonw 9599 | 2021-10-08T19:42:00Z | 2021-10-08T19:42:00Z | OWNER | Running `pytest -x --pdb` helped me see this error: ``` File "/Users/simon/Dropbox/Development/datasette/datasette/views/base.py", line 122, in dispatch_request await self.ds.refresh_schemas() File "/Users/simon/Dropbox/Development/datasette/datasette/app.py", line 344, in refresh_schemas await self._refresh_schemas() File "/Users/simon/Dropbox/Development/datasette/datasette/app.py", line 349, in _refresh_schemas await init_internal_db(internal_db) File "/Users/simon/Dropbox/Development/datasette/datasette/utils/internal_db.py", line 5, in init_internal_db await db.execute_write( File "/Users/simon/Dropbox/Development/datasette/datasette/database.py", line 102, in execute_write return await self.execute_write_fn(_inner, block=block) File "/Users/simon/Dropbox/Development/datasette/datasette/database.py", line 113, in execute_write_fn reply_queue = janus.Queue() File "/Users/simon/.local/share/virtualenvs/py310-Z8fTATkJ/lib/python3.10/site-packages/janus/__init__.py", line 39, in __init__ self._async_not_empty = asyncio.Condition(self._async_mutex) File "/Users/simon/.pyenv/versions/3.10.0/lib/python3.10/asyncio/locks.py", line 234, in __init__ raise ValueError("loop argument must agree with lock") ValueError: loop argument must agree with lock ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
939076399 | https://github.com/simonw/datasette/pull/1481#issuecomment-939076399 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c43-Ssv | simonw 9599 | 2021-10-08T19:43:33Z | 2021-10-08T19:43:33Z | OWNER | So maybe this is an issue with Janus? I'm using https://pypi.org/project/janus/ 0.6.1 which is the latest release, from October 2020. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
939078095 | https://github.com/simonw/datasette/pull/1481#issuecomment-939078095 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c43-THP | simonw 9599 | 2021-10-08T19:47:29Z | 2021-10-08T19:47:29Z | OWNER | Only mention I can find of that "loop argument must agree with lock" error is here - which doesn't have any tips for a workaround yet: https://giters.com/django/channels_redis/issues/278 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
939078872 | https://github.com/simonw/datasette/pull/1481#issuecomment-939078872 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c43-TTY | simonw 9599 | 2021-10-08T19:49:08Z | 2021-10-08T19:49:08Z | OWNER | Here's the code that raises that error: https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/Lib/asyncio/locks.py#L219-L234 ```python class Condition(_ContextManagerMixin, mixins._LoopBoundMixin): """Asynchronous equivalent to threading.Condition. This class implements condition variable objects. A condition variable allows one or more coroutines to wait until they are notified by another coroutine. A new Lock object is created and used as the underlying lock. """ def __init__(self, lock=None, *, loop=mixins._marker): super().__init__(loop=loop) if lock is None: lock = Lock() elif lock._loop is not self._get_loop(): raise ValueError("loop argument must agree with lock") ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
939079727 | https://github.com/simonw/datasette/pull/1481#issuecomment-939079727 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c43-Tgv | simonw 9599 | 2021-10-08T19:50:52Z | 2021-10-08T19:50:52Z | OWNER | And here's the relevant Janus code: https://github.com/aio-libs/janus/blob/d7970f8b76bcac2e087067ca4575ac845e481874/janus/__init__.py#L24-L42 ```python class Queue(Generic[T]): def __init__(self, maxsize: int = 0) -> None: self._loop = current_loop() self._maxsize = maxsize self._init(maxsize) self._unfinished_tasks = 0 self._sync_mutex = threading.Lock() self._sync_not_empty = threading.Condition(self._sync_mutex) self._sync_not_full = threading.Condition(self._sync_mutex) self._all_tasks_done = threading.Condition(self._sync_mutex) self._async_mutex = asyncio.Lock() # "loop argument must agree with lock" exception is raised here: self._async_not_empty = asyncio.Condition(self._async_mutex) self._async_not_full = asyncio.Condition(self._async_mutex) self._finished = asyncio.Event() self._finished.set() ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
939100803 | https://github.com/simonw/datasette/pull/1481#issuecomment-939100803 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c43-YqD | simonw 9599 | 2021-10-08T20:33:42Z | 2021-10-08T20:33:42Z | OWNER | There's a tiny chance this could be a bug in Python 3.10 itself - I filed an issue here: https://bugs.python.org/issue45416 - in which I said: > In Python 3.10 it is not possible to instantiate an asyncio.Condition that wraps an asyncio.Lock without raising a "loop argument must agree with lock" exception. > > This code raises that exception: > > asyncio.Condition(asyncio.Lock()) > > This worked in previous Python versions. > > Note that the error only occurs if an event loop is running. Here's a simple script that replicates the problem: > > import asyncio > > # This runs without an exception: > print(asyncio.Condition(asyncio.Lock())) > > # This does not work: > async def example(): > print(asyncio.Condition(asyncio.Lock())) > > # This raises "ValueError: loop argument must agree with lock": > asyncio.run(example()) | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
939180313 | https://github.com/simonw/datasette/pull/1481#issuecomment-939180313 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c43-sEZ | simonw 9599 | 2021-10-08T23:41:39Z | 2021-10-08T23:41:39Z | OWNER | I submitted a PR to Janus with a workaround for this: https://github.com/aio-libs/janus/pull/359 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
939185319 | https://github.com/simonw/datasette/pull/1481#issuecomment-939185319 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c43-tSn | simonw 9599 | 2021-10-09T00:04:54Z | 2021-10-09T00:04:54Z | OWNER | I applied my PR against Janus to my local copy of Datasette like so: pip uninstall janus pip install https://github.com/aio-libs/janus/archive/9e13d3fb74e2c93d7501443b370a455d1b302b1f.zip Then I ran the Datasette tests and got a much happier pass rate. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
939191311 | https://github.com/simonw/datasette/issues/1482#issuecomment-939191311 | https://api.github.com/repos/simonw/datasette/issues/1482 | IC_kwDOBm6k_c43-uwP | simonw 9599 | 2021-10-09T00:35:04Z | 2021-10-09T00:35:04Z | OWNER | I think that SQLite error message difference was caused by https://github.com/python/cpython/commit/a50e28377bcf37121b55c2de70d95a5386c478f8 or related work. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Support Python 3.10 1021550542 | |
938124652 | https://github.com/simonw/datasette/issues/1470#issuecomment-938124652 | https://api.github.com/repos/simonw/datasette/issues/1470 | IC_kwDOBm6k_c436qVs | simonw 9599 | 2021-10-07T20:17:53Z | 2021-10-07T20:18:55Z | OWNER | Here's the exception: ``` -> params[f"p{len(params)}"] = components[0] (Pdb) list 603 604 # Figure out the SQL for next-based-on-primary-key first 605 next_by_pk_clauses = [] 606 if use_rowid: 607 next_by_pk_clauses.append(f"rowid > :p{len(params)}") 608 -> params[f"p{len(params)}"] = components[0] 609 else: 610 # Apply the tie-breaker based on primary keys 611 if len(components) == len(pks): 612 param_len = len(params) 613 next_by_pk_clauses.append( ``` Debugger shows that `components` is an empty array, so `components[0]` cannot be resolved: ``` -> params[f"p{len(params)}"] = components[0] (Pdb) params {'search': 'hello'} (Pdb) components [] ``` So the bug is in this code: https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/views/table.py#L604-L617 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ?_sort=rowid with _next= returns error 995098231 | |
938131806 | https://github.com/simonw/datasette/issues/1470#issuecomment-938131806 | https://api.github.com/repos/simonw/datasette/issues/1470 | IC_kwDOBm6k_c436sFe | simonw 9599 | 2021-10-07T20:28:30Z | 2021-10-07T20:28:30Z | OWNER | On further investigation this isn't related to `_search` at all - it happens when you explicitly sort by `_sort=rowid` and apply a `_next` - https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_next=200 works without an error (currently) - https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_next=200&_sort=rowid shows that error | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ?_sort=rowid with _next= returns error 995098231 | |
938134038 | https://github.com/simonw/datasette/issues/1480#issuecomment-938134038 | https://api.github.com/repos/simonw/datasette/issues/1480 | IC_kwDOBm6k_c436soW | simonw 9599 | 2021-10-07T20:31:46Z | 2021-10-07T20:31:46Z | OWNER | I've had this problem too - my solution was to not use Cloud Run for databases larger than about 2GB, but the way you describe it here makes me think that maybe there is a workaround here which could get it to work. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Exceeding Cloud Run memory limits when deploying a 4.8G database 1015646369 | |
938142436 | https://github.com/simonw/datasette/pull/1481#issuecomment-938142436 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c436urk | simonw 9599 | 2021-10-07T20:44:43Z | 2021-10-07T20:44:43Z | OWNER | The 3.10 tests failed a lot. Trying to run this locally: ``` /tmp % pyenv install 3.10 python-build: definition not found: 3.10 The following versions contain `3.10' in the name: 3.10.0a6 3.10-dev miniconda-3.10.1 miniconda3-3.10.1 See all available versions with `pyenv install --list'. If the version you need is missing, try upgrading pyenv: brew update && brew upgrade pyenv ``` So trying: brew update && brew upgrade pyenv Then did this: ``` /tmp % brew upgrade pyenv ==> Upgrading 1 outdated package: pyenv 1.2.24.1 -> 2.1.0 ``` This decided to upgrade everything by downloaded everything on the internet. Aah, Homebrew. But it looks like I have `3.10.0` available to `pyenv` now. ``` /tmp % pyenv install 3.10.0 python-build: use openssl@1.1 from homebrew python-build: use readline from homebrew Downloading Python-3.10.0.tar.xz... -> https://www.python.org/ftp/python/3.10.0/Python-3.10.0.tar.xz Installing Python-3.10.0... ... ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
923106887 | https://github.com/simonw/datasette/issues/111#issuecomment-923106887 | https://api.github.com/repos/simonw/datasette/issues/111 | IC_kwDOBm6k_c43BX5H | simonw 9599 | 2021-09-20T16:58:39Z | 2021-09-20T16:58:39Z | OWNER | Still a good idea today too! Would be great for https://cdc-vaccination-history.datasette.io/ for example. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add “updated” to metadata 274615452 | |
924432643 | https://github.com/simonw/datasette/issues/111#issuecomment-924432643 | https://api.github.com/repos/simonw/datasette/issues/111 | IC_kwDOBm6k_c43GbkD | simonw 9599 | 2021-09-21T22:23:23Z | 2021-09-21T22:23:23Z | OWNER | I'm going to use https://github.com/dateutil/dateutil for this - it's been maintained constantly (by an evolving team of contributors) [since 2003](https://github.com/dateutil/dateutil/commit/68ae2757ae15c84bf947d47a82a314b3b975bc9b) and is a very trustworthy dependency. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add “updated” to metadata 274615452 | |
924437942 | https://github.com/simonw/datasette/issues/111#issuecomment-924437942 | https://api.github.com/repos/simonw/datasette/issues/111 | IC_kwDOBm6k_c43Gc22 | simonw 9599 | 2021-09-21T22:32:59Z | 2021-09-21T22:47:07Z | OWNER | Side-note: Django 4.0 [will switch](https://docs.djangoproject.com/en/dev/releases/4.0/#zoneinfo-default-timezone-implementation) from using `pytz` to using the standard library `zoneinfo` module introduced in Python 3.9, which has a backport that works as far back as 3.6: https://github.com/pganssle/zoneinfo (https://pypi.org/project/backports.zoneinfo/) If I need to handle timezones I'll use that, but I think I can get away without it? Django does this: https://github.com/django/django/blob/b0ed619303d2fb723330ca9efa3acf23d49f1d19/setup.cfg#L39-L43 ``` install_requires = asgiref >= 3.3.2 backports.zoneinfo; python_version<"3.9" sqlparse >= 0.2.2 tzdata; sys_platform == 'win32' ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add “updated” to metadata 274615452 | |
924435971 | https://github.com/simonw/datasette/issues/111#issuecomment-924435971 | https://api.github.com/repos/simonw/datasette/issues/111 | IC_kwDOBm6k_c43GcYD | simonw 9599 | 2021-09-21T22:29:15Z | 2021-09-21T22:29:49Z | OWNER | So this is a metadata key called `updated` which can be applied at the table, database or instance level. It is represented as a `.isoformat()` timestamp. Question: should I support just the date - `yyyy-mm-dd` - in addition to the datetime? I think so. I can easily imagine situations where the exact time of day that a change was made hasn't been recorded, but the overall date is known. But in that case, should the `updated` key sometimes be `yyyy-mm-dd` and sometimes be the full isoformat datetime? Or should there be an `updated_date` key that's used for just the date? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add “updated” to metadata 274615452 | |
924438481 | https://github.com/simonw/datasette/issues/111#issuecomment-924438481 | https://api.github.com/repos/simonw/datasette/issues/111 | IC_kwDOBm6k_c43Gc_R | simonw 9599 | 2021-09-21T22:34:03Z | 2021-09-21T22:34:21Z | OWNER | The simplest possible version of this: it's always represented as a UTC ISO datetime, like this: "updated": "2020-10-31T12:00:00+00:00" Later versions of Datasette could extend this to handle other timezones or support just the date (though that's a backwards incompatible change so probably better to decide on the date thing right now). | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add “updated” to metadata 274615452 | |
924443089 | https://github.com/simonw/datasette/issues/111#issuecomment-924443089 | https://api.github.com/repos/simonw/datasette/issues/111 | IC_kwDOBm6k_c43GeHR | simonw 9599 | 2021-09-21T22:45:14Z | 2021-09-21T22:45:26Z | OWNER | The audiences I care about here are: - Producers of this timestamp - generally that will be users who are using `datasette publish` to share their data - Human consumers of this timestamp - end users who look at a Datasette site and want to know how recent the data is - Machine consumers of this timestamp - API integrations that might want to check if a Datasette instance has been updated before downloading new data For producers I think there are going to be two categories. The first is users who run "publish" and want the site to reflect when they did so (probably using `--updated=now` when they publish). The second are users who are willing to spend more time thinking about this - for example my various git scraping projects where I want to use a date derived from the git history. For humans... I'd like to be able to calculate a relative time (3 hours ago) in addition to showing the display time, because that helps avoid confusion over timezones. For machine consumers, it might be nice to offer the option of a calculated Unix timestamp-since-1970, since those can be easier to work with in some languages than running a full ISO date parser. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add “updated” to metadata 274615452 | |
939386591 | https://github.com/simonw/datasette/issues/1470#issuecomment-939386591 | https://api.github.com/repos/simonw/datasette/issues/1470 | IC_kwDOBm6k_c43_ebf | simonw 9599 | 2021-10-10T01:17:34Z | 2021-10-10T01:17:34Z | OWNER | I'll open a separate issue for removing `_next=` when running a search. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ?_sort=rowid with _next= returns error 995098231 | |
932807859 | https://github.com/simonw/datasette/issues/1479#issuecomment-932807859 | https://api.github.com/repos/simonw/datasette/issues/1479 | IC_kwDOBm6k_c43mYSz | simonw 9599 | 2021-10-02T19:22:35Z | 2021-10-02T19:22:35Z | OWNER | I'm pretty sure this is a Windows issue, not a Fly issue. I imagine it affects other forms of `datasette publish` too. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Win32 "used by another process" error with datasette publish 1010112818 | |
932808043 | https://github.com/simonw/datasette/issues/1479#issuecomment-932808043 | https://api.github.com/repos/simonw/datasette/issues/1479 | IC_kwDOBm6k_c43mYVr | simonw 9599 | 2021-10-02T19:23:52Z | 2021-10-02T19:23:52Z | OWNER | I suspect the root cause of this may be in this code: https://github.com/simonw/datasette/blob/63886178a649586b403966a27a45881709d2b868/datasette/utils/__init__.py#L673-L677 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Win32 "used by another process" error with datasette publish 1010112818 | |
932808216 | https://github.com/simonw/datasette/issues/1479#issuecomment-932808216 | https://api.github.com/repos/simonw/datasette/issues/1479 | IC_kwDOBm6k_c43mYYY | simonw 9599 | 2021-10-02T19:25:09Z | 2021-10-02T19:25:09Z | OWNER | Actually no, from that stack trace you provided: ``` File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke return callback(*args, **kwargs) File "c:\users\grott\anaconda3\lib\site-packages\datasette\cli.py", line 283, in package call(args) File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in __exit__ next(self.gen) File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py", line 451, in temporary_docker_directory tmp.cleanup() ``` It looks like the problem occurs here: https://github.com/simonw/datasette/blob/b1fed48a95516ae84c0f020582303ab50ab817e2/datasette/utils/__init__.py#L449-L452 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Win32 "used by another process" error with datasette publish 1010112818 | |
953508979 | https://github.com/simonw/datasette/issues/1497#issuecomment-953508979 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c441WRz | simonw 9599 | 2021-10-28T05:13:49Z | 2021-10-28T05:13:49Z | OWNER | Wrote about this in my weeknotes: https://simonwillison.net/2021/Oct/28/weeknotes-kubernetes-web-components/ | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
942722595 | https://github.com/simonw/datasette/pull/1487#issuecomment-942722595 | https://api.github.com/repos/simonw/datasette/issues/1487 | IC_kwDOBm6k_c44MM4j | simonw 9599 | 2021-10-13T21:08:53Z | 2021-10-13T21:08:53Z | OWNER | Thanks for this! | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Added instructions for installing plugins via pipx, #1486 1023245060 | |
942725632 | https://github.com/simonw/datasette/issues/1469#issuecomment-942725632 | https://api.github.com/repos/simonw/datasette/issues/1469 | IC_kwDOBm6k_c44MNoA | simonw 9599 | 2021-10-13T21:13:30Z | 2021-10-13T21:13:30Z | OWNER | The core problem here is treating the `?_facet=` query string parameters as the point of truth for which facets are currently enabled. Instead, I could use a `data-` attribute on the displayed facets. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Column cog shows "facet by this" when already default faceted 994450961 | |
942779926 | https://github.com/simonw/datasette/issues/1488#issuecomment-942779926 | https://api.github.com/repos/simonw/datasette/issues/1488 | IC_kwDOBm6k_c44Ma4W | simonw 9599 | 2021-10-13T22:59:05Z | 2021-10-13T22:59:05Z | OWNER | This is weird - as far as I can tell `httpx` has included the query string in `raw_path` for well over a year: https://github.com/encode/httpx/commit/8e4a8a1c73f60fe5754f95b308beaa725cb8791d#diff-c9a78eb3b5f5c4fac4e5552165fbdd5320c7e3fadf9eedabcb5461393466c090R235 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Upgrade to httpx 0.20.0 (request() got an unexpected keyword argument 'allow_redirects') 1025754125 | |
942777414 | https://github.com/simonw/datasette/issues/1488#issuecomment-942777414 | https://api.github.com/repos/simonw/datasette/issues/1488 | IC_kwDOBm6k_c44MaRG | simonw 9599 | 2021-10-13T22:52:40Z | 2021-10-13T22:52:40Z | OWNER | Upgrading to 0.20.0 gives me lots of the following errors: '{"ok": false, "error": "Database not found: .json?_sort=relationships", "status": 404, "title": null}' It looks like the full query string is now being treated as the name of the database. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Upgrade to httpx 0.20.0 (request() got an unexpected keyword argument 'allow_redirects') 1025754125 | |
942778382 | https://github.com/simonw/datasette/issues/1488#issuecomment-942778382 | https://api.github.com/repos/simonw/datasette/issues/1488 | IC_kwDOBm6k_c44MagO | simonw 9599 | 2021-10-13T22:55:01Z | 2021-10-13T22:55:01Z | OWNER | I think the issue is in `route_path()`: ``` > /Users/simon/Dropbox/Development/datasette/datasette/app.py(1182)route_path() -> response = await view(request, send) (Pdb) path '/_memory.json?sql=select+sqlite_version()' ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Upgrade to httpx 0.20.0 (request() got an unexpected keyword argument 'allow_redirects') 1025754125 | |
942778673 | https://github.com/simonw/datasette/issues/1488#issuecomment-942778673 | https://api.github.com/repos/simonw/datasette/issues/1488 | IC_kwDOBm6k_c44Makx | simonw 9599 | 2021-10-13T22:55:44Z | 2021-10-13T22:55:44Z | OWNER | ``` (Pdb) request.scope['path'] '/_memory.json' (Pdb) request.scope['raw_path'] b'/_memory.json?sql=select+sqlite_version()' ``` So `raw_path` now includes the query string. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Upgrade to httpx 0.20.0 (request() got an unexpected keyword argument 'allow_redirects') 1025754125 | |
942782673 | https://github.com/simonw/datasette/issues/1488#issuecomment-942782673 | https://api.github.com/repos/simonw/datasette/issues/1488 | IC_kwDOBm6k_c44MbjR | simonw 9599 | 2021-10-13T23:04:54Z | 2021-10-13T23:04:54Z | OWNER | I think this is the change in `httpx` which is causing the bug for me: https://github.com/encode/httpx/commit/ff9813e84dab56f0f3c4ef3a159a4cce8c644a91#diff-0d0cbe9ebcd03cc8c780b0407762540a082f70cc64257f2fcd588cc30f43c15cR96 <img width="706" alt="Transport_API_as_plain_`request_-__response`_method____1840__·_encode_httpx_ff9813e" src="https://user-images.githubusercontent.com/9599/137224339-0aed171f-b351-4d67-8d6c-ad22f2301f18.png"> Previously it was using `path` from `path, _, query = full_path.partition(b"?")` to populate the `raw_path` key - but it changed to instead using `request.url.raw_path` which presumably implements the logic that includes the query string. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Upgrade to httpx 0.20.0 (request() got an unexpected keyword argument 'allow_redirects') 1025754125 | |
943594712 | https://github.com/simonw/datasette/pull/1489#issuecomment-943594712 | https://api.github.com/repos/simonw/datasette/issues/1489 | IC_kwDOBm6k_c44PhzY | simonw 9599 | 2021-10-14T18:04:11Z | 2021-10-14T18:04:11Z | OWNER | @dependabot recreate | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Update pyyaml requirement from ~=5.3 to >=5.3,<7.0 1026379132 | |
943620649 | https://github.com/simonw/datasette/pull/1458#issuecomment-943620649 | https://api.github.com/repos/simonw/datasette/issues/1458 | IC_kwDOBm6k_c44PoIp | simonw 9599 | 2021-10-14T18:38:58Z | 2021-10-14T18:38:58Z | OWNER | This is a great idea, thanks. | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Rework the `--static` documentation a bit 988555009 | |
943623246 | https://github.com/simonw/datasette/pull/1467#issuecomment-943623246 | https://api.github.com/repos/simonw/datasette/issues/1467 | IC_kwDOBm6k_c44PoxO | simonw 9599 | 2021-10-14T18:42:19Z | 2021-10-14T18:42:19Z | OWNER | This looks like a good fix to me. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Add Authorization header when CORS flag is set 991575770 | |
943632697 | https://github.com/simonw/datasette/pull/1467#issuecomment-943632697 | https://api.github.com/repos/simonw/datasette/issues/1467 | IC_kwDOBm6k_c44PrE5 | simonw 9599 | 2021-10-14T18:54:18Z | 2021-10-14T18:54:18Z | OWNER | The test there failed because it turns out there's a whole bunch of places that set the `Access-Control-Allow-Origin` header. I'm going to close this PR and ship a fix that refactors those places to use the same code. | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0} | Add Authorization header when CORS flag is set 991575770 | |
944986367 | https://github.com/simonw/datasette/pull/1481#issuecomment-944986367 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c44U1j_ | simonw 9599 | 2021-10-16T19:07:38Z | 2021-10-16T19:09:02Z | OWNER | This is blocking an upgrade for the Homebrew Datasette package: https://github.com/Homebrew/homebrew-core/pull/86932 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
945020210 | https://github.com/simonw/datasette/pull/1481#issuecomment-945020210 | https://api.github.com/repos/simonw/datasette/issues/1481 | IC_kwDOBm6k_c44U90y | simonw 9599 | 2021-10-16T23:19:51Z | 2021-10-16T23:19:51Z | OWNER | Since that Janus PR hasn't been merged yet, one temporary option for a fix would be to entirely vendor the fixed Janus - https://github.com/aio-libs/janus/blob/9e13d3fb74e2c93d7501443b370a455d1b302b1f/janus/__init__.py - since it's only a single module. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Fix compatibility with Python 3.10 1020436713 | |
946097058 | https://github.com/simonw/datasette/issues/1470#issuecomment-946097058 | https://api.github.com/repos/simonw/datasette/issues/1470 | IC_kwDOBm6k_c44ZEui | simonw 9599 | 2021-10-18T19:30:15Z | 2021-10-18T19:30:15Z | OWNER | https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_next=200&_sort=rowid is fixed now. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ?_sort=rowid with _next= returns error 995098231 | |
946360891 | https://github.com/simonw/datasette/issues/1493#issuecomment-946360891 | https://api.github.com/repos/simonw/datasette/issues/1493 | IC_kwDOBm6k_c44aFI7 | simonw 9599 | 2021-10-19T04:37:27Z | 2021-10-19T04:37:27Z | OWNER | I renamed `/:memory:` to `/_memory` in version 0.55 - https://docs.datasette.io/en/stable/changelog.html#v0-55 But... in 0.59 I stopped following HTTP redirects by default, which is why this used to work and no longer does! So the fix is to update the Homebrew regression test to use this instead: datasette --get '/_memory.json?sql=select+3*5' Thanks for catching this! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `--get '/:memory:.json?sql=select+3*5'` error with datasette 0.59 1028115674 | |
949912718 | https://github.com/simonw/datasette/issues/1496#issuecomment-949912718 | https://api.github.com/repos/simonw/datasette/issues/1496 | IC_kwDOBm6k_c44noSO | simonw 9599 | 2021-10-22T19:38:23Z | 2021-10-22T19:38:23Z | OWNER | https://docs.datasette.io/en/latest/sql_queries.html#named-parameters | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Named parameters docs should include an example of a cast 1033864602 | |
950402273 | https://github.com/simonw/datasette/issues/1482#issuecomment-950402273 | https://api.github.com/repos/simonw/datasette/issues/1482 | IC_kwDOBm6k_c44pfzh | simonw 9599 | 2021-10-24T22:00:29Z | 2021-10-24T22:00:29Z | OWNER | Janus 0.6.2 is out now and should have the fix. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Support Python 3.10 1021550542 | |
950403521 | https://github.com/simonw/datasette/pull/1495#issuecomment-950403521 | https://api.github.com/repos/simonw/datasette/issues/1495 | IC_kwDOBm6k_c44pgHB | simonw 9599 | 2021-10-24T22:09:18Z | 2021-10-24T22:09:18Z | OWNER | This is a great idea - I've wanted this myself before, but never spent any time thinking about how to achieve it. I think your design here is exactly right - an optional third item in the tuple consisting of a dictionary of options to pass to the view function. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Allow routes to have extra options 1033678984 | |
950403692 | https://github.com/simonw/datasette/pull/1495#issuecomment-950403692 | https://api.github.com/repos/simonw/datasette/issues/1495 | IC_kwDOBm6k_c44pgJs | simonw 9599 | 2021-10-24T22:10:43Z | 2021-10-24T22:10:43Z | OWNER | To land this change we'll need a unit test that demonstrates the new capability - I suggest putting that next to this test: https://github.com/simonw/datasette/blob/15a9d4abfff0c45dee2a9f851326e1d61b1c678c/tests/test_plugins.py#L648-L659 It will also need documentation, which should be added here: https://github.com/simonw/datasette/blob/15a9d4abfff0c45dee2a9f851326e1d61b1c678c/docs/plugin_hooks.rst#register-routes-datasette | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Allow routes to have extra options 1033678984 | |
950410554 | https://github.com/simonw/datasette/issues/1497#issuecomment-950410554 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44ph06 | simonw 9599 | 2021-10-24T23:01:20Z | 2021-10-24T23:01:28Z | OWNER | I can replicate locally by running: ``` docker build -f Dockerfile \ -t datasetteproject/datasette:0.59.1 \ --build-arg VERSION=0.59.1 . ``` This gives me the same error. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950410718 | https://github.com/simonw/datasette/issues/1497#issuecomment-950410718 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44ph3e | simonw 9599 | 2021-10-24T23:02:30Z | 2021-10-24T23:02:30Z | OWNER | I got the same error publishing 0.59: https://github.com/simonw/datasette/actions/runs/1343251945 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950415129 | https://github.com/simonw/datasette/issues/1497#issuecomment-950415129 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44pi8Z | simonw 9599 | 2021-10-24T23:21:33Z | 2021-10-24T23:21:33Z | OWNER | That fixed it! Resulting image is 249MB which is a very slight size reduction (I think previous was 259MB (uncompressed). | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950411320 | https://github.com/simonw/datasette/issues/1497#issuecomment-950411320 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44piA4 | simonw 9599 | 2021-10-24T23:06:05Z | 2021-10-24T23:06:05Z | OWNER | Right now the base image is: https://github.com/simonw/datasette/blob/e6e44372b34414eac2f36a4c1120af4f755aa423/Dockerfile#L1 I'm going to try `python:3.9.7-slim-buster` instead: https://hub.docker.com/layers/python/library/python/3.9.7-slim-buster/images/sha256-290b95e4b379762a9bd3d72644598e0972f4e2b5442bba60592c018fadcc744d?context=explore | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950411417 | https://github.com/simonw/datasette/issues/1497#issuecomment-950411417 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44piCZ | simonw 9599 | 2021-10-24T23:06:45Z | 2021-10-24T23:11:14Z | OWNER | Same errors with `3.9.7`: ``` #5 41.46 /usr/bin/perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory #5 41.46 dpkg: error processing package libc6:amd64 (--configure): #5 41.46 installed libc6:amd64 package post-installation script subprocess returned error exit status 127 #5 41.47 Errors were encountered while processing: #5 41.47 libc6:amd64 #5 41.50 E: Sub-process /usr/bin/dpkg returned an error code (1) ``` I'm suspicious of this part of the `Dockerfile`: https://github.com/simonw/datasette/blob/e6e44372b34414eac2f36a4c1120af4f755aa423/Dockerfile#L1-L18 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950411808 | https://github.com/simonw/datasette/issues/1497#issuecomment-950411808 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44piIg | simonw 9599 | 2021-10-24T23:08:59Z | 2021-10-24T23:08:59Z | OWNER | Looks like it's this bug, reported on the Debian mailing list: https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1818037.html No obvious workaround there though. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950411912 | https://github.com/simonw/datasette/issues/1497#issuecomment-950411912 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44piKI | simonw 9599 | 2021-10-24T23:09:41Z | 2021-10-24T23:09:41Z | OWNER | Here that is in the Debian bug tracker: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993755 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950412628 | https://github.com/simonw/datasette/issues/1497#issuecomment-950412628 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44piVU | simonw 9599 | 2021-10-24T23:13:20Z | 2021-10-24T23:13:27Z | OWNER | I think the root cause here is that I'm using a Debian Buster base image and then installing SpatiaLite from Debian unstable (sid) - as described in this comment: https://github.com/simonw/datasette/issues/1249#issuecomment-804309510 That's has worked fine in the past, but Sid is unstable - and this seems to be one of those instabilities. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950413185 | https://github.com/simonw/datasette/issues/1497#issuecomment-950413185 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44pieB | simonw 9599 | 2021-10-24T23:16:25Z | 2021-10-24T23:18:30Z | OWNER | Debian stable these days is "bullseye" - https://www.debian.org/releases/ - which has the version of SpatiaLite that I was previously pulling in from Sid: https://packages.debian.org/bullseye/libsqlite3-mod-spatialite So upgrading to the 3.9.7-slim-bullseye base image may help. https://hub.docker.com/layers/python/library/python/3.9.7-slim-bullseye/images/sha256-67af5f544115124dc6d6da1d9d2815aa9825f6fd4aa6710adb0ec1725280fb89?context=explore | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950415822 | https://github.com/simonw/datasette/issues/1497#issuecomment-950415822 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44pjHO | simonw 9599 | 2021-10-24T23:25:45Z | 2021-10-24T23:25:45Z | OWNER | I'm going to attempt to publish `0.59` to Docker Hub using https://github.com/simonw/datasette/blob/2c31d1cd9cd3b63458ccbe391866499fa3f44978/.github/workflows/push_docker_tag.yml - if that works I'll push `0.59.1` as well. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950416061 | https://github.com/simonw/datasette/issues/1497#issuecomment-950416061 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44pjK9 | simonw 9599 | 2021-10-24T23:27:18Z | 2021-10-24T23:27:18Z | OWNER | That worked: https://hub.docker.com/layers/datasetteproject/datasette/0.59/images/sha256-038decc28e0ea84b281ecc0058fe8eba7aa99596e5a2177ff714092ad03294ed?context=explore | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950416460 | https://github.com/simonw/datasette/issues/1497#issuecomment-950416460 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44pjRM | simonw 9599 | 2021-10-24T23:30:10Z | 2021-10-24T23:30:10Z | OWNER | Testing that newly published image: ``` % docker run -p 8002:8001 -v `pwd`:/mnt \ datasetteproject/datasette:0.59 datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db Unable to find image 'datasetteproject/datasette:0.59' locally 0.59: Pulling from datasetteproject/datasette 7d63c13d9b9b: Already exists 6ad2a11ca37b: Already exists e9edbe81a001: Already exists 36629b83aba2: Already exists 7338abefe51c: Already exists 6d71b6b88b82: Pull complete 8c4da3c56bdc: Pull complete Digest: sha256:038decc28e0ea84b281ecc0058fe8eba7aa99596e5a2177ff714092ad03294ed Status: Downloaded newer image for datasetteproject/datasette:0.59 INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit) ``` and `http://localhost:8002/versions.json` returns: ```json { "python": { "version": "3.9.7", "full": "3.9.7 (default, Oct 12 2021, 02:43:43) \n[GCC 10.2.1 20210110]" }, "datasette": { "version": "0.59" }, "asgi": "3.0", "uvicorn": "0.15.0", "sqlite": { "version": "3.34.1" ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950416659 | https://github.com/simonw/datasette/issues/1497#issuecomment-950416659 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44pjUT | simonw 9599 | 2021-10-24T23:31:41Z | 2021-10-24T23:31:41Z | OWNER | Published `0.59.1` as well: https://github.com/simonw/datasette/runs/3991214225?check_suite_focus=true Result: https://hub.docker.com/layers/datasetteproject/datasette/0.59.1/images/sha256-dc134f65bec40ed4ea7049188fe1e3915b8e6c3fd999b17effe8ec24868b979c?context=explore | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950416682 | https://github.com/simonw/datasette/issues/1497#issuecomment-950416682 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44pjUq | simonw 9599 | 2021-10-24T23:31:51Z | 2021-10-24T23:31:51Z | OWNER | One catch: the `latest` tag on Docker Hub is still three months old. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950416802 | https://github.com/simonw/datasette/issues/1497#issuecomment-950416802 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44pjWi | simonw 9599 | 2021-10-24T23:32:39Z | 2021-10-24T23:32:39Z | OWNER | That's because the `publish.yml` workflow ends with this, which isn't in the `push_docker_tag.yml` workflow: https://github.com/simonw/datasette/blob/2c31d1cd9cd3b63458ccbe391866499fa3f44978/.github/workflows/publish.yml#L117-L119 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
950417375 | https://github.com/simonw/datasette/issues/1497#issuecomment-950417375 | https://api.github.com/repos/simonw/datasette/issues/1497 | IC_kwDOBm6k_c44pjff | simonw 9599 | 2021-10-24T23:36:54Z | 2021-10-24T23:36:54Z | OWNER | Tried fixing this by pushing a new `latest` tag from my laptop: ``` (datasette) datasette % docker pull datasetteproject/datasette:0.59.1 0.59.1: Pulling from datasetteproject/datasette 7d63c13d9b9b: Already exists 6ad2a11ca37b: Already exists e9edbe81a001: Already exists 36629b83aba2: Already exists 7338abefe51c: Already exists 6b825daddc6c: Pull complete d7508b065a21: Pull complete Digest: sha256:dc134f65bec40ed4ea7049188fe1e3915b8e6c3fd999b17effe8ec24868b979c Status: Downloaded newer image for datasetteproject/datasette:0.59.1 docker.io/datasetteproject/datasette:0.59.1 (datasette) datasette % docker tag datasetteproject/datasette:0.59.1 datasetteproject/datasette:latest (datasette) datasette % docker push datasetteproject/datasette:latest The push refers to repository [docker.io/datasetteproject/datasette] d668c99b6ff1: Layer already exists aa20c9013575: Layer already exists c97eebf2b227: Layer already exists 284a6c64b82c: Layer already exists 388eedeb736e: Layer already exists 2feece0964b8: Layer already exists e8b689711f21: Layer already exists errors: denied: requested access to the resource is denied unauthorized: authentication required ``` So I logged in with `docker login`: ``` (datasette) datasette % docker login Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. Username: datasetteproject Password: ``` And ran the push again and it worked: ``` (datasette) datasette % docker push datasetteproject/datasette:latest The push refers to repository [docker.io/datasetteproject/datasette] d668c99b6ff1: Layer already exists aa20c9013575: Layer already exists c97eebf2b227: Layer already exists 284a6c64b82c: Layer already exists 388eedeb736e: Layer already exists 2feece0964b8: Layer already exists e8b689711f21: Layer already exists latest: digest: sha256:dc134f65bec40ed4ea7049188fe1e3915b8e6c3fd999b17effe8ec24868b979c size: 1793 ``` https://hub.do… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" 1034535001 | |
970712713 | https://github.com/simonw/datasette/issues/878#issuecomment-970712713 | https://api.github.com/repos/simonw/datasette/issues/878 | IC_kwDOBm6k_c452-aJ | simonw 9599 | 2021-11-16T21:54:33Z | 2021-11-16T21:54:33Z | OWNER | I'm going to continue working on this in a PR. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
970673085 | https://github.com/simonw/datasette/issues/878#issuecomment-970673085 | https://api.github.com/repos/simonw/datasette/issues/878 | IC_kwDOBm6k_c4520u9 | simonw 9599 | 2021-11-16T20:58:24Z | 2021-11-16T20:58:24Z | OWNER | New test: ```python class Complex(AsyncBase): def __init__(self): self.log = [] async def d(self): await asyncio.sleep(random() * 0.1) print("LOG: d") self.log.append("d") async def c(self): await asyncio.sleep(random() * 0.1) print("LOG: c") self.log.append("c") async def b(self, c, d): print("LOG: b") self.log.append("b") async def a(self, b, c): print("LOG: a") self.log.append("a") async def go(self, a): print("LOG: go") self.log.append("go") return self.log @pytest.mark.asyncio async def test_complex(): result = await Complex().go() # 'c' should only be called once assert tuple(result) in ( # c and d could happen in either order ("c", "d", "b", "a", "go"), ("d", "c", "b", "a", "go"), ) ``` And this code passes it: ```python import asyncio from functools import wraps import inspect try: import graphlib except ImportError: from . import vendored_graphlib as graphlib class AsyncMeta(type): def __new__(cls, name, bases, attrs): # Decorate any items that are 'async def' methods _registry = {} new_attrs = {"_registry": _registry} for key, value in attrs.items(): if inspect.iscoroutinefunction(value) and not value.__name__ == "resolve": new_attrs[key] = make_method(value) _registry[key] = new_attrs[key] else: new_attrs[key] = value # Gather graph for later dependency resolution graph = { key: { p for p in inspect.signature(method).parameters.keys() if p != "self" and not p.startswith("_") } for key, method in _registry.items() } new_attrs["_graph"] = graph return super().__new__(cls, name, bases, new_attrs) def make_met… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
970705738 | https://github.com/simonw/datasette/issues/878#issuecomment-970705738 | https://api.github.com/repos/simonw/datasette/issues/878 | IC_kwDOBm6k_c4528tK | simonw 9599 | 2021-11-16T21:44:31Z | 2021-11-16T21:44:31Z | OWNER | Wrote a TIL about what I learned using `TopologicalSorter`: https://til.simonwillison.net/python/graphlib-topologicalsorter | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
970544733 | https://github.com/simonw/datasette/issues/1509#issuecomment-970544733 | https://api.github.com/repos/simonw/datasette/issues/1509 | IC_kwDOBm6k_c452VZd | simonw 9599 | 2021-11-16T18:22:32Z | 2021-11-16T18:22:32Z | OWNER | This is mainly happening here: - https://github.com/simonw/datasette/issues/782 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Datasette 1.0 JSON API (and documentation) 1054243511 | |
970554697 | https://github.com/simonw/datasette/issues/782#issuecomment-970554697 | https://api.github.com/repos/simonw/datasette/issues/782 | IC_kwDOBm6k_c452X1J | simonw 9599 | 2021-11-16T18:32:03Z | 2021-11-16T18:32:03Z | OWNER | I'm going to take another look at this: - https://github.com/simonw/datasette/issues/878 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Redesign default .json format 627794879 | |
970553780 | https://github.com/simonw/datasette/issues/782#issuecomment-970553780 | https://api.github.com/repos/simonw/datasette/issues/782 | IC_kwDOBm6k_c452Xm0 | simonw 9599 | 2021-11-16T18:30:51Z | 2021-11-16T18:30:58Z | OWNER | OK, I'm ready to start working on this today. I'm going to go with a default representation that looks like this: ```json { "rows": [ {"id": 1, "name": "One"}, {"id": 2, "name": "Two"} ], "next_url": null } ``` Note that there's no `count` - all it provides is the current selection of results and an indication as to how the next can be retrieved (`null` if there are no more results). I'll implement `?_extra=` to provide everything else. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Redesign default .json format 627794879 | |
970718337 | https://github.com/simonw/datasette/pull/1512#issuecomment-970718337 | https://api.github.com/repos/simonw/datasette/issues/1512 | IC_kwDOBm6k_c452_yB | simonw 9599 | 2021-11-16T22:02:30Z | 2021-11-16T22:02:30Z | OWNER | I've decided to make the clever `asyncio` dependency injection opt-in - so you can either decorate with `@inject` or you can set `inject_all = True` on the class - for example: ```python import asyncio from datasette.utils.asyncdi import AsyncBase, inject class Simple(AsyncBase): def __init__(self): self.log = [] @inject async def two(self): self.log.append("two") @inject async def one(self, two): self.log.append("one") return self.log async def not_inject(self, one, two): return one + two class Complex(AsyncBase): inject_all = True def __init__(self): self.log = [] async def b(self): self.log.append("b") async def a(self, b): self.log.append("a") async def go(self, a): self.log.append("go") return self.log ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | New pattern for async view classes 1055402144 | |
970624197 | https://github.com/simonw/datasette/issues/878#issuecomment-970624197 | https://api.github.com/repos/simonw/datasette/issues/878 | IC_kwDOBm6k_c452ozF | simonw 9599 | 2021-11-16T19:49:05Z | 2021-11-16T19:49:05Z | OWNER | Here's the latest version of my weird dependency injection async class: ```python import inspect class AsyncMeta(type): def __new__(cls, name, bases, attrs): # Decorate any items that are 'async def' methods _registry = {} new_attrs = {"_registry": _registry} for key, value in attrs.items(): if inspect.iscoroutinefunction(value) and not value.__name__ == "resolve": new_attrs[key] = make_method(value) _registry[key] = new_attrs[key] else: new_attrs[key] = value # Topological sort of _registry by parameter dependencies graph = { key: { p for p in inspect.signature(method).parameters.keys() if p != "self" and not p.startswith("_") } for key, method in _registry.items() } new_attrs["_graph"] = graph return super().__new__(cls, name, bases, new_attrs) def make_method(method): @wraps(method) async def inner(self, **kwargs): parameters = inspect.signature(method).parameters.keys() # Any parameters not provided by kwargs are resolved from registry to_resolve = [p for p in parameters if p not in kwargs and p != "self"] missing = [p for p in to_resolve if p not in self._registry] assert ( not missing ), "The following DI parameters could not be found in the registry: {}".format( missing ) results = {} results.update(kwargs) results.update(await self.resolve(to_resolve)) return await method(self, **results) return inner bad = [0] class AsyncBase(metaclass=AsyncMeta): async def resolve(self, names): print(" resolve({})".format(names)) results = {} # Resolve them in the correct order ts = TopologicalSorter() ts2 = TopologicalSorter() print(" names = ", names) print(" s… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
970655304 | https://github.com/simonw/datasette/issues/878#issuecomment-970655304 | https://api.github.com/repos/simonw/datasette/issues/878 | IC_kwDOBm6k_c452wZI | simonw 9599 | 2021-11-16T20:32:16Z | 2021-11-16T20:32:16Z | OWNER | This code is really fiddly. I just got to this version: ```python import asyncio from functools import wraps import inspect try: import graphlib except ImportError: from . import vendored_graphlib as graphlib class AsyncMeta(type): def __new__(cls, name, bases, attrs): # Decorate any items that are 'async def' methods _registry = {} new_attrs = {"_registry": _registry} for key, value in attrs.items(): if inspect.iscoroutinefunction(value) and not value.__name__ == "resolve": new_attrs[key] = make_method(value) _registry[key] = new_attrs[key] else: new_attrs[key] = value # Gather graph for later dependency resolution graph = { key: { p for p in inspect.signature(method).parameters.keys() if p != "self" and not p.startswith("_") } for key, method in _registry.items() } new_attrs["_graph"] = graph return super().__new__(cls, name, bases, new_attrs) def make_method(method): @wraps(method) async def inner(self, _results=None, **kwargs): print("inner - _results=", _results) parameters = inspect.signature(method).parameters.keys() # Any parameters not provided by kwargs are resolved from registry to_resolve = [p for p in parameters if p not in kwargs and p != "self"] missing = [p for p in to_resolve if p not in self._registry] assert ( not missing ), "The following DI parameters could not be found in the registry: {}".format( missing ) results = {} results.update(kwargs) if to_resolve: resolved_parameters = await self.resolve(to_resolve, _results) results.update(resolved_parameters) return_value = await method(self, **results) if _results is not None: _res… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
970655927 | https://github.com/simonw/datasette/issues/878#issuecomment-970655927 | https://api.github.com/repos/simonw/datasette/issues/878 | IC_kwDOBm6k_c452wi3 | simonw 9599 | 2021-11-16T20:33:11Z | 2021-11-16T20:33:11Z | OWNER | What should be happening here instead is it should resolve the full graph and notice that `c` is depended on by both `b` and `a` - so it should run `c` first, then run the next ones in parallel. So maybe the algorithm I'm inheriting from https://docs.python.org/3/library/graphlib.html isn't the correct algorithm? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
970657874 | https://github.com/simonw/datasette/issues/878#issuecomment-970657874 | https://api.github.com/repos/simonw/datasette/issues/878 | IC_kwDOBm6k_c452xBS | simonw 9599 | 2021-11-16T20:36:01Z | 2021-11-16T20:36:01Z | OWNER | My goal here is to calculate the most efficient way to resolve the different nodes, running them in parallel where possible. So for this class: ```python class Complex(AsyncBase): async def d(self): pass async def c(self): pass async def b(self, c, d): pass async def a(self, b, c): pass async def go(self, a): pass ``` A call to `go()` should do this: - `c` and `d` in parallel - `b` - `a` - `go` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
970660299 | https://github.com/simonw/datasette/issues/878#issuecomment-970660299 | https://api.github.com/repos/simonw/datasette/issues/878 | IC_kwDOBm6k_c452xnL | simonw 9599 | 2021-11-16T20:39:43Z | 2021-11-16T20:42:27Z | OWNER | But that does seem to be the plan that `TopographicalSorter` provides: ```python graph = {"go": {"a"}, "a": {"b", "c"}, "b": {"c", "d"}} ts = TopologicalSorter(graph) ts.prepare() while ts.is_active(): nodes = ts.get_ready() print(nodes) ts.done(*nodes) ``` Outputs: ``` ('c', 'd') ('b',) ('a',) ('go',) ``` Also: ```python graph = {"go": {"d", "e", "f"}, "d": {"b", "c"}, "b": {"c"}} ts = TopologicalSorter(graph) ts.prepare() while ts.is_active(): nodes = ts.get_ready() print(nodes) ts.done(*nodes) ``` Gives: ``` ('e', 'f', 'c') ('b',) ('d',) ('go',) ``` I'm confident that `TopologicalSorter` is the way to do this. I think I need to rewrite my code to call it once to get that plan, then `await asyncio.gather(*nodes)` in turn to execute it. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
970738130 | https://github.com/simonw/datasette/issues/1513#issuecomment-970738130 | https://api.github.com/repos/simonw/datasette/issues/1513 | IC_kwDOBm6k_c453EnS | simonw 9599 | 2021-11-16T22:32:19Z | 2021-11-16T22:32:19Z | OWNER | I came up with the following query which seems to work! ```sql with cte as ( select rowid, country, country_long, name, owner, primary_fuel from [global-power-plants] ), truncated as ( select null as _facet, null as facet_name, null as facet_count, rowid, country, country_long, name, owner, primary_fuel from cte order by rowid limit 4 ), country_long_facet as ( select 'country_long' as _facet, country_long as facet_name, count(*) as facet_count, null, null, null, null, null, null from cte group by facet_name order by facet_count desc limit 3 ), owner_facet as ( select 'owner' as _facet, owner as facet_name, count(*) as facet_count, null, null, null, null, null, null from cte group by facet_name order by facet_count desc limit 3 ), primary_fuel_facet as ( select 'primary_fuel' as _facet, primary_fuel as facet_name, count(*) as facet_count, null, null, null, null, null, null from cte group by facet_name order by facet_count desc limit 3 ) select * from truncated union all select * from country_long_facet union all select * from owner_facet union all select * from primary_fuel_facet ``` (Limits should be 101, 31, 31, 31 but I reduced size to get a shorter example table). Results [look like this](https://global-power-plants.datasettes.com/global-power-plants?sql=with+cte+as+%28%0D%0A++select+rowid%2C+country%2C+country_long%2C+name%2C+owner%2C+primary_fuel%0D%0A++from+%5Bglobal-power-plants%5D%0D%0A%29%2C%0D%0Atruncated+as+%28%0D%0A++select+null+as+_facet%2C+null+as+facet_name%2C+null+as+facet_count%2C+rowid%2C+country%2C+country_long%2C+name%2C+owner%2C+primary_fuel%0D%0A++from+cte+order+by+rowid+limit+4%0D%0A%29%2C%0D%0Acountry_long_facet+as+%28%0D%0A++select+%27country_long%27+as+_facet%2C+country_long+as+facet_name%2C+count%28*%29+as+facet_count%2C%0D%0A++null%2C+null%2C+null%2C+null%2C+null%2C+null%0D%0A++from+cte+group+by+facet_name+order+by+facet_count+desc+limit+3%0D%0A%29%2C%0D%0Aowner_facet+as+%28%0D%0A++select+%27owner%27+as+_facet%2C+owner+as+fa… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Research: CTEs and union all to calculate facets AND query at the same time 1055469073 | |
970742415 | https://github.com/simonw/datasette/issues/1513#issuecomment-970742415 | https://api.github.com/repos/simonw/datasette/issues/1513 | IC_kwDOBm6k_c453FqP | simonw 9599 | 2021-11-16T22:37:14Z | 2021-11-16T22:37:14Z | OWNER | The query takes 42.794ms to run. Here's the equivalent page using separate queries: https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_facet_size=3&_size=2&_nocount=1 Annoyingly I can't disable facet suggestions but keep facets. I'm going to turn on tracing so I can see how long the separate queries took. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Research: CTEs and union all to calculate facets AND query at the same time 1055469073 | |
970758179 | https://github.com/simonw/datasette/issues/1513#issuecomment-970758179 | https://api.github.com/repos/simonw/datasette/issues/1513 | IC_kwDOBm6k_c453Jgj | simonw 9599 | 2021-11-16T22:47:38Z | 2021-11-16T22:47:38Z | OWNER | Trace now enabled: https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_facet_size=3&_size=2&_nocount=1&_trace=1 Here are the relevant traces: ```json [ { "type": "sql", "start": 31.214430154, "end": 31.214817089, "duration_ms": 0.3869350000016425, "traceback": [ " File \"/usr/local/lib/python3.8/site-packages/datasette/views/base.py\", line 262, in get\n return await self.view_get(\n", " File \"/usr/local/lib/python3.8/site-packages/datasette/views/base.py\", line 477, in view_get\n response_or_template_contexts = await self.data(\n", " File \"/usr/local/lib/python3.8/site-packages/datasette/views/table.py\", line 705, in data\n results = await db.execute(sql, params, truncate=True, **extra_args)\n" ], "database": "global-power-plants", "sql": "select rowid, country, country_long, name, gppd_idnr, capacity_mw, latitude, longitude, primary_fuel, other_fuel1, other_fuel2, other_fuel3, commissioning_year, owner, source, url, geolocation_source, wepp_id, year_of_capacity_data, generation_gwh_2013, generation_gwh_2014, generation_gwh_2015, generation_gwh_2016, generation_gwh_2017, generation_data_source, estimated_generation_gwh from [global-power-plants] order by rowid limit 3", "params": {} }, { "type": "sql", "start": 31.215234586, "end": 31.220110342, "duration_ms": 4.875756000000564, "traceback": [ " File \"/usr/local/lib/python3.8/site-packages/datasette/views/table.py\", line 760, in data\n ) = await facet.facet_results()\n", " File \"/usr/local/lib/python3.8/site-packages/datasette/facets.py\", line 212, in facet_results\n facet_rows_results = await self.ds.execute(\n", " File \"/usr/local/lib/python3.8/site-packages/datasette/app.py\", line 634, in execute\n return await self.databases[db_name].execute(\n" ], "database": "global-power-plants", "sql": "select countr… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Research: CTEs and union all to calculate facets AND query at the same time 1055469073 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
author_association 1 ✖