github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/2079#issuecomment-1563565407 | https://api.github.com/repos/simonw/datasette/issues/2079 | 1563565407 | IC_kwDOBm6k_c5dMh1f | 9599 | 2023-05-25T22:09:53Z | 2023-05-25T22:09:53Z | OWNER | Updated docs: https://docs.datasette.io/en/latest/json_api.html#enabling-cors | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726531350 | |
https://github.com/simonw/datasette/issues/2079#issuecomment-1563563438 | https://api.github.com/repos/simonw/datasette/issues/2079 | 1563563438 | IC_kwDOBm6k_c5dMhWu | 9599 | 2023-05-25T22:08:28Z | 2023-05-25T22:08:28Z | OWNER | I ran this on https://www.example.com/ twice using the console: ```javascript fetch( `https://latest.datasette.io/ephemeral/foo/1/-/update`, { method: "POST", mode: "cors", headers: { Authorization: `Bearer tok`, "Content-Type": "application/json", }, body: JSON.stringify({update: {blah: 1}}), } ) .then((r) => r.json()) .then((data) => { console.log(data); }); ``` And got this in the network pane: <img width="945" alt="image" src="https://github.com/simonw/datasette/assets/9599/cc767917-388b-4616-b35d-efaa1591948a"> | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726531350 | |
https://github.com/simonw/datasette/issues/2079#issuecomment-1563558915 | https://api.github.com/repos/simonw/datasette/issues/2079 | 1563558915 | IC_kwDOBm6k_c5dMgQD | 9599 | 2023-05-25T22:04:41Z | 2023-05-25T22:04:41Z | OWNER | I'm going with 3600 for 1 hour instead of 600 for 10 minutes. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726531350 | |
https://github.com/simonw/datasette/issues/2079#issuecomment-1563547097 | https://api.github.com/repos/simonw/datasette/issues/2079 | 1563547097 | IC_kwDOBm6k_c5dMdXZ | 9599 | 2023-05-25T21:51:38Z | 2023-05-25T21:51:38Z | OWNER | Also need to update this documentation: https://github.com/simonw/datasette/blob/9584879534ff0556e04e4c420262972884cac87b/docs/json_api.rst?plain=1#L453-L465 Or maybe make that automated via `cog`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726531350 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563522011 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563522011 | IC_kwDOBm6k_c5dMXPb | 9599 | 2023-05-25T21:22:30Z | 2023-05-25T21:22:30Z | OWNER | This is bad: ```python async def __call__(self, request, datasette): try: handler = getattr(self, request.method.lower()) return await handler(request, datasette) except AttributeError: return await self.method_not_allowed(request) ``` Because it hides any `AttributeError` exceptions that might occur in the view code. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563511171 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563511171 | IC_kwDOBm6k_c5dMUmD | 9599 | 2023-05-25T21:11:20Z | 2023-05-25T21:13:05Z | OWNER | I'm going to call this `VerbView` for the moment. Might even rename it to `View` later. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563498048 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563498048 | IC_kwDOBm6k_c5dMRZA | 9599 | 2023-05-25T20:57:52Z | 2023-05-25T20:58:13Z | OWNER | Here's a new `BaseView` class that automatically populates `OPTIONS` based on available methods: ```python class BaseView: async def head(self, *args, **kwargs): try: response = await self.get(*args, **kwargs) response.body = b"" return response except AttributeError: raise async def method_not_allowed(self, request): if ( request.path.endswith(".json") or request.headers.get("content-type") == "application/json" ): response = Response.json( {"ok": False, "error": "Method not allowed"}, status=405 ) else: response = Response.text("Method not allowed", status=405) return response async def options(self, request, *args, **kwargs): response = Response.text("ok") response.headers["allow"] = ", ".join( method.upper() for method in ("head", "get", "post", "put", "patch", "delete") if hasattr(self, method) ) return response async def __call__(self, request, datasette): try: handler = getattr(self, request.method.lower()) return await handler(request, datasette) except AttributeError: return await self.method_not_allowed(request) class DemoView(BaseView): async def get(self, datasette, request): return Response.text("Hello there! {} - {}".format(datasette, request)) post = get ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563488929 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563488929 | IC_kwDOBm6k_c5dMPKh | 9599 | 2023-05-25T20:48:12Z | 2023-05-25T20:48:39Z | OWNER | Actually no need for that extra level of parameter detection: `BaseView.__call__` should _always_ take `datasette, request` - `scope` and `receive` are both available on `request`, and `send` is only needed if you're not planning on returning a `Response` object. So the `get` and `post` and suchlike methods should take `datasette` and `request` too. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563444296 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563444296 | IC_kwDOBm6k_c5dMERI | 9599 | 2023-05-25T20:06:08Z | 2023-05-25T20:06:08Z | OWNER | This prototype seems to work well: ```diff diff --git a/datasette/app.py b/datasette/app.py index d7dace67..ed0edf28 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -17,6 +17,7 @@ import secrets import sys import threading import time +import types import urllib.parse from concurrent import futures from pathlib import Path @@ -1266,6 +1267,8 @@ class Datasette: # TODO: /favicon.ico and /-/static/ deserve far-future cache expires add_route(favicon, "/favicon.ico") + add_route(wrap_view(DemoView, self), '/demo') + add_route( asgi_static(app_root / "datasette" / "static"), r"/-/static/(?P<path>.*)$" ) @@ -1673,8 +1676,46 @@ def _cleaner_task_str(task): return _cleaner_task_str_re.sub("", s) -def wrap_view(view_fn, datasette): - @functools.wraps(view_fn) +class DemoView: + async def __call__(self, datasette, request): + return Response.text("Hello there! {} - {}".format(datasette, request)) + +def wrap_view(view_fn_or_class, datasette): + is_function = isinstance(view_fn_or_class, types.FunctionType) + if is_function: + return wrap_view_function(view_fn_or_class, datasette) + else: + if not isinstance(view_fn_or_class, type): + raise ValueError("view_fn_or_class must be a function or a class") + return wrap_view_class(view_fn_or_class, datasette) + + +def wrap_view_class(view_class, datasette): + async def async_view_for_class(request, send): + instance = view_class() + if inspect.iscoroutinefunction(instance.__call__): + return await async_call_with_supported_arguments( + instance.__call__, + scope=request.scope, + receive=request.receive, + send=send, + request=request, + datasette=datasette, + ) + else: + return call_with_supported_arguments( + instance.__call__, … | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563419066 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563419066 | IC_kwDOBm6k_c5dL-G6 | 9599 | 2023-05-25T19:42:16Z | 2023-05-25T19:43:08Z | OWNER | Maybe what I want here is the ability to register classes with the router - and have the router know that if it's a class it should instantiate it via its constructor and then await `__call__` it. The neat thing about it is that it can reduce the risk of having a class instance that accidentally shares state between requests. It also encourages that each class only responds based on the `datasette, request, ...` objects that are passed to its methods. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563359114 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563359114 | IC_kwDOBm6k_c5dLveK | 9599 | 2023-05-25T18:47:57Z | 2023-05-25T18:47:57Z | OWNER | Oops, that broke everything: ``` @documented async def await_me_maybe(value: typing.Any) -> typing.Any: "If value is callable, call it. If awaitable, await it. Otherwise return it." > if callable(value): E TypeError: 'module' object is not callable ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563329245 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563329245 | IC_kwDOBm6k_c5dLoLd | 9599 | 2023-05-25T18:26:47Z | 2023-05-25T18:28:08Z | OWNER | With type hints and a namedtuple: ```python import asyncio import types from typing import NamedTuple, Any class CallableStatus(NamedTuple): is_callable: bool is_async_callable: bool def check_callable(obj: Any) -> CallableStatus: if not callable(obj): return CallableStatus(False, False) if isinstance(obj, type): # It's a class return CallableStatus(True, False) if isinstance(obj, types.FunctionType): return CallableStatus(True, asyncio.iscoroutinefunction(obj)) if hasattr(obj, "__call__"): return CallableStatus(True, asyncio.iscoroutinefunction(obj.__call__)) assert False, "obj {} is somehow callable with no __call__ method".format(repr(obj)) ``` ```python for thing in ( async_func, non_async_func, AsyncClass(), NotAsyncClass(), ClassNoCall(), AsyncClass, NotAsyncClass, ClassNoCall, ): print(thing, check_callable(thing)) ``` ``` <function async_func at 0x1073d5120> CallableStatus(is_callable=True, is_async_callable=True) <function non_async_func at 0x1073d5080> CallableStatus(is_callable=True, is_async_callable=False) <__main__.AsyncClass object at 0x106ba7490> CallableStatus(is_callable=True, is_async_callable=True) <__main__.NotAsyncClass object at 0x106740150> CallableStatus(is_callable=True, is_async_callable=False) <__main__.ClassNoCall object at 0x10676d910> CallableStatus(is_callable=False, is_async_callable=False) <class '__main__.AsyncClass'> CallableStatus(is_callable=True, is_async_callable=False) <class '__main__.NotAsyncClass'> CallableStatus(is_callable=True, is_async_callable=False) <class '__main__.ClassNoCall'> CallableStatus(is_callable=True, is_async_callable=False) ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563326000 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563326000 | IC_kwDOBm6k_c5dLnYw | 9599 | 2023-05-25T18:23:38Z | 2023-05-25T18:23:38Z | OWNER | I don't like that `is_callable()` implies a single boolean result but actually returns a pair. I'll call it `check_callable(obj)` instead. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563318598 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563318598 | IC_kwDOBm6k_c5dLllG | 9599 | 2023-05-25T18:17:03Z | 2023-05-25T18:21:25Z | OWNER | I think I want that to return `(is_callable, is_async)` - so I can both test if the thing can be called AND if it should be awaited in the same operation (without any exceptions). I tried this: ```python def is_callable(obj): "Returns (is_callable, is_async_callable)" if not callable(obj): return False, False if isinstance(obj, types.FunctionType): return True, asyncio.iscoroutinefunction(obj) if hasattr(obj, '__call__'): return True, asyncio.iscoroutinefunction(obj.__call__) return False, False ``` ```python for thing in ( async_func, non_async_func, AsyncClass(), NotAsyncClass(), ClassNoCall(), AsyncClass, NotAsyncClass, ClassNoCall ): print(thing, is_callable(thing)) ``` And got: ``` <function async_func at 0x1073d5120> (True, True) <function non_async_func at 0x1073d5080> (True, False) <__main__.AsyncClass object at 0x106cce490> (True, True) <__main__.NotAsyncClass object at 0x106ccf710> (True, False) <__main__.ClassNoCall object at 0x106ccc810> (False, False) <class '__main__.AsyncClass'> (True, True) <class '__main__.NotAsyncClass'> (True, False) <class '__main__.ClassNoCall'> (True, False) ``` Which is almost right, but I don't like that `AsyncClass` is shown as callable (which it is, since it's a class) and awaitable (which it is not - the `__call__` method may be async but calling the class constructor is not). So I'm going to detect classes using `isinstance(obj, type)`. ```python def is_callable(obj): "Returns (is_callable, is_async_callable)" if not callable(obj): return False, False if isinstance(obj, type): # It's a class return True, False if isinstance(obj, types.FunctionType): return True, asyncio.iscoroutinefunction(obj) if hasattr(obj, '__call__'): return True, asyncio.iscoroutinefunction(obj.__call__) assert False, "obj {} somehow is callable with no __call__ method".format(obj) ``` I am re… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563308919 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563308919 | IC_kwDOBm6k_c5dLjN3 | 9599 | 2023-05-25T18:08:34Z | 2023-05-25T18:08:34Z | OWNER | After much fiddling this seems to work: ```python import asyncio, types def is_async_callable(obj): if not callable(obj): raise ValueError("Object is not callable") if isinstance(obj, types.FunctionType): return asyncio.iscoroutinefunction(obj) if hasattr(obj, '__call__'): return asyncio.iscoroutinefunction(obj.__call__) raise ValueError("Not a function and has no __call__ attribute") ``` Tested like so: ```python class AsyncClass: async def __call__(self): pass class NotAsyncClass: def __call__(self): pass class ClassNoCall: pass async def async_func(): pass def non_async_func(): pass for thing in (AsyncClass(), NotAsyncClass(), ClassNoCall(), async_func, non_async_func): try: print(thing, is_async_callable(thing)) except Exception as ex: print(thing, ex) ``` ``` <__main__.AsyncClass object at 0x106c32150> True <__main__.NotAsyncClass object at 0x106c32390> False <__main__.ClassNoCall object at 0x106c32750> Object is not callable <function async_func at 0x1073d5120> True <function non_async_func at 0x1073d5080> False ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563294669 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563294669 | IC_kwDOBm6k_c5dLfvN | 9599 | 2023-05-25T17:57:06Z | 2023-05-25T17:57:06Z | OWNER | I may need to be able to detect if a class instance has an `async def __call__` method - I think I can do that like so: ```python def iscoroutinefunction(obj): if inspect.iscoroutinefunction(obj): return True if hasattr(obj, '__call__') and inspect.iscoroutinefunction(obj.__call__): return True return False ``` From https://github.com/encode/starlette/issues/886#issuecomment-606585152 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563292373 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563292373 | IC_kwDOBm6k_c5dLfLV | 9599 | 2023-05-25T17:55:12Z | 2023-05-25T17:55:30Z | OWNER | So I think subclasses of `BaseView` need to offer a callable which accepts all five of the DI arguments - `datasette`, `request`, `scope`, `send`, `receive` - and then makes a decision based on the HTTP verb as to which method of the class to call. Those methods themselves can accept a subset of those parameters and will only be sent on to them. Having two layers of parameter detection feels a little bit untidy, but I think it will work. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/pull/2053#issuecomment-1563285150 | https://api.github.com/repos/simonw/datasette/issues/2053 | 1563285150 | IC_kwDOBm6k_c5dLdae | 9599 | 2023-05-25T17:48:50Z | 2023-05-25T17:49:52Z | OWNER | Uncommitted experimental code: ```diff diff --git a/datasette/views/database.py b/datasette/views/database.py index 455ebd1f..85775433 100644 --- a/datasette/views/database.py +++ b/datasette/views/database.py @@ -909,12 +909,13 @@ async def query_view( elif format_ in datasette.renderers.keys(): # Dispatch request to the correct output format renderer # (CSV is not handled here due to streaming) + print(data) result = call_with_supported_arguments( datasette.renderers[format_][0], datasette=datasette, - columns=columns, - rows=rows, - sql=sql, + columns=data["rows"][0].keys(), + rows=data["rows"], + sql='', query_name=None, database=db.name, table=None, @@ -923,7 +924,7 @@ async def query_view( # These will be deprecated in Datasette 1.0: args=request.args, data={ - "rows": rows, + "rows": data["rows"], }, # TODO what should this be? ) result = await await_me_maybe(result) diff --git a/docs/index.rst b/docs/index.rst index 5a9cc7ed..254ed3da 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -57,6 +57,7 @@ Contents settings introspection custom_templates + template_context plugins writing_plugins plugin_hooks ``` Where `docs/template_context.rst` looked like this: ```rst .. _template_context: Template context ================ .. currentmodule:: datasette.context This page describes the variables made available to templates used by Datasette to render different pages of the application. .. autoclass:: QueryContext :members: ``` And `datasette/context.py` had this: ```python from dataclasses import dataclass @dataclass class QueryContext: """ Used by the ``/database`` page when showing the results of a SQL query """ id: int "… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1656432059 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563283939 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563283939 | IC_kwDOBm6k_c5dLdHj | 9599 | 2023-05-25T17:47:38Z | 2023-05-25T17:47:38Z | OWNER | The idea behind `wrap_view()` is dependency injection - it's mainly used by plugins: https://docs.datasette.io/en/0.64.3/plugin_hooks.html#register-routes-datasette But I like the pattern so I started using it for some of Datasette's own features. I should use it for _all_ of Datasette's own features. But I still like the way `BaseView` helps with running different code for GET/POST/etc verbs. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/datasette/issues/2078#issuecomment-1563282327 | https://api.github.com/repos/simonw/datasette/issues/2078 | 1563282327 | IC_kwDOBm6k_c5dLcuX | 9599 | 2023-05-25T17:46:05Z | 2023-05-25T17:46:05Z | OWNER | Here's what `wrap_view()` does: https://github.com/simonw/datasette/blob/49184c569cd70efbda4f3f062afef3a34401d8d5/datasette/app.py#L1676-L1700 It's used e.g. here: https://github.com/simonw/datasette/blob/49184c569cd70efbda4f3f062afef3a34401d8d5/datasette/app.py#L1371-L1375 The `BaseView` thing meanwhile works like this: https://github.com/simonw/datasette/blob/d97e82df3c8a3f2e97038d7080167be9bb74a68d/datasette/views/base.py#L56-L157 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1726236847 | |
https://github.com/simonw/sqlite-utils/issues/554#issuecomment-1557607516 | https://api.github.com/repos/simonw/sqlite-utils/issues/554 | 1557607516 | IC_kwDOCGYnMM5c1zRc | 1231935 | 2023-05-22T17:18:33Z | 2023-05-22T17:18:33Z | NONE | Oh and for context - this goes away if I use `.upsert` instead of `insert(..., ignore=True)`, but I don't want to update the value if it's written, just do an insert if it's new. The code is basically: ```py def save_items(table, items): db["users"].insert(build_user(items[0]), pk="id",ignore=True) db[table].insert_all(items) if comments := fetch_comments(): save_items('comments', comments) if posts := fetch_posts(): save_items('posts', posts) ``` So either `comments` or `post` could create the relevant user if those items exist. In cases where they _both_ exist, I get this error. I need the `pk` because either call could create the table. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1720096994 | |
https://github.com/simonw/datasette/pull/2077#issuecomment-1557289070 | https://api.github.com/repos/simonw/datasette/issues/2077 | 1557289070 | IC_kwDOBm6k_c5c0lhu | 22429695 | 2023-05-22T14:08:33Z | 2023-06-29T14:40:35Z | NONE | ## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2077?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch and project coverage have no change. > Comparison is base [(`ede6203`)](https://app.codecov.io/gh/simonw/datasette/commit/ede62036180993dbd9d4e5d280fc21c183cda1c3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.40% compared to head [(`9785c4f`)](https://app.codecov.io/gh/simonw/datasette/pull/2077?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.40%. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #2077 +/- ## ======================================= Coverage 92.40% 92.40% ======================================= Files 39 39 Lines 5803 5803 ======================================= Hits 5362 5362 Misses 441 441 ``` </details> [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2077?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1719759468 | |
https://github.com/simonw/sqlite-utils/issues/552#issuecomment-1556292204 | https://api.github.com/repos/simonw/sqlite-utils/issues/552 | 1556292204 | IC_kwDOCGYnMM5cwyJs | 9599 | 2023-05-21T21:05:15Z | 2023-05-21T21:05:15Z | OWNER | Now live at https://sqlite-utils.datasette.io/en/latest/installation.html#setting-up-shell-completion | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718612569 | |
https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556291915 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | 1556291915 | IC_kwDOCGYnMM5cwyFL | 9599 | 2023-05-21T21:04:03Z | 2023-05-21T21:04:03Z | OWNER | Now live at https://sqlite-utils.datasette.io/en/latest/cli.html | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718607907 | |
https://github.com/simonw/sqlite-utils/pull/553#issuecomment-1556288300 | https://api.github.com/repos/simonw/sqlite-utils/issues/553 | 1556288300 | IC_kwDOCGYnMM5cwxMs | 9599 | 2023-05-21T20:48:01Z | 2023-05-21T20:48:01Z | OWNER | If https://sqlite-utils--553.org.readthedocs.build/en/553/cli.html#running-sql-queries looks good I can merge this. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718635018 | |
https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556288270 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | 1556288270 | IC_kwDOCGYnMM5cwxMO | 9599 | 2023-05-21T20:47:51Z | 2023-05-21T20:47:51Z | OWNER | This page has all of the changes: https://sqlite-utils--553.org.readthedocs.build/en/553/cli.html#running-sql-queries | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718607907 | |
https://github.com/simonw/sqlite-utils/pull/553#issuecomment-1556287870 | https://api.github.com/repos/simonw/sqlite-utils/issues/553 | 1556287870 | IC_kwDOCGYnMM5cwxF- | 22429695 | 2023-05-21T20:45:58Z | 2023-05-21T20:57:08Z | NONE | ## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/553?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch and project coverage have no change. > Comparison is base [(`e240133`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/e240133b11588d31dc22c632f7a7ca636c72978d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.36% compared to head [(`0b81794`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/553?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.36%. > :exclamation: Current head 0b81794 differs from pull request most recent head 21036a5. Consider uploading reports for the commit 21036a5 to get more accurate results <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #553 +/- ## ======================================= Coverage 96.36% 96.36% ======================================= Files 6 6 Lines 2726 2726 ======================================= Hits 2627 2627 Misses 99 99 ``` </details> [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/sqlite-utils/pull/553?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718635018 | |
https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556287599 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | 1556287599 | IC_kwDOCGYnMM5cwxBv | 9599 | 2023-05-21T20:44:55Z | 2023-05-21T20:44:55Z | OWNER | Put this in a PR so I can preview it: - #553 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718607907 | |
https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556269616 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | 1556269616 | IC_kwDOCGYnMM5cwsow | 9599 | 2023-05-21T19:33:13Z | 2023-05-21T19:33:13Z | OWNER | Now released: https://sqlite-utils.datasette.io/en/stable/changelog.html#v3-32 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718517882 | |
https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556265772 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | 1556265772 | IC_kwDOCGYnMM5cwrss | 9599 | 2023-05-21T19:16:15Z | 2023-05-21T19:16:15Z | OWNER | Another option: <img width="593" alt="image" src="https://github.com/simonw/sqlite-utils/assets/9599/a9d819a0-89e3-4b67-b2f0-1052b0a7a7c0"> That's using this markup: ``` Newline-delimited JSON ~~~~~~~~~~~~~~~~~~~~~~ Use ``--nl`` to get back newline-delimited JSON objects: .. code-block:: bash sqlite-utils dogs.db "select * from dogs" --nl .. code-block:: output {"id": 1, "age": 4, "name": "Cleo"} {"id": 2, "age": 2, "name": "Pancakes"} ``` And this extra CSS: ```css .highlight-output .highlight { border-left: 9px solid #30c94f; } ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718607907 | |
https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556263182 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | 1556263182 | IC_kwDOCGYnMM5cwrEO | 9599 | 2023-05-21T19:06:48Z | 2023-05-21T19:06:48Z | OWNER | I could split them up into two blocks like this: <img width="771" alt="image" src="https://github.com/simonw/sqlite-utils/assets/9599/9b042ac8-db30-49d5-8882-1ea7be272a49"> I do miss the visual indication that one of these is the command and one is the output though. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718607907 | |
https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556262574 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | 1556262574 | IC_kwDOCGYnMM5cwq6u | 9599 | 2023-05-21T19:04:59Z | 2023-05-21T19:04:59Z | OWNER | I wrote the docs like this because early examples include both the command and its output: https://sqlite-utils.datasette.io/en/stable/cli.html#returning-json <img width="766" alt="image" src="https://github.com/simonw/sqlite-utils/assets/9599/a5d05f24-ca0f-4718-a0a8-d85adc736141"> | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718607907 | |
https://github.com/simonw/sqlite-utils/issues/550#issuecomment-1556255309 | https://api.github.com/repos/simonw/sqlite-utils/issues/550 | 1556255309 | IC_kwDOCGYnMM5cwpJN | 9599 | 2023-05-21T18:42:25Z | 2023-05-21T18:42:25Z | OWNER | Tests passed here: https://github.com/simonw/sqlite-utils/actions/runs/5039119716 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718595700 | |
https://github.com/simonw/sqlite-utils/issues/550#issuecomment-1556250236 | https://api.github.com/repos/simonw/sqlite-utils/issues/550 | 1556250236 | IC_kwDOCGYnMM5cwn58 | 9599 | 2023-05-21T18:25:26Z | 2023-05-21T18:25:26Z | OWNER | Relevant issues: - https://github.com/python/importlib_metadata/issues/406 - https://github.com/PyCQA/flake8/issues/1701 It looks to me like this is only a problem for `flake8` on Python 3.7 - 3.8 and higher work OK. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718595700 | |
https://github.com/simonw/sqlite-utils/issues/550#issuecomment-1556249984 | https://api.github.com/repos/simonw/sqlite-utils/issues/550 | 1556249984 | IC_kwDOCGYnMM5cwn2A | 9599 | 2023-05-21T18:24:48Z | 2023-05-21T18:24:48Z | OWNER | This is blocking: - #549 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718595700 | |
https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556247818 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | 1556247818 | IC_kwDOCGYnMM5cwnUK | 9599 | 2023-05-21T18:17:46Z | 2023-05-21T18:17:46Z | OWNER | Draft documentation: https://sqlite-utils--549.org.readthedocs.build/en/549/cli.html#cli-tui | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718517882 | |
https://github.com/simonw/sqlite-utils/pull/549#issuecomment-1556242262 | https://api.github.com/repos/simonw/sqlite-utils/issues/549 | 1556242262 | IC_kwDOCGYnMM5cwl9W | 9599 | 2023-05-21T18:00:05Z | 2023-05-21T18:00:05Z | OWNER | Failing `mypy` test: https://github.com/simonw/sqlite-utils/actions/runs/5038983349/jobs/9036828465 ``` sqlite_utils/cli.py:37: error: Skipping analyzing "trogon": module is installed, but missing library stubs or py.typed marker [import] sqlite_utils/cli.py:37: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports Found 1 error in 1 file (checked 52 source files) ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718586377 | |
https://github.com/simonw/sqlite-utils/pull/549#issuecomment-1556241812 | https://api.github.com/repos/simonw/sqlite-utils/issues/549 | 1556241812 | IC_kwDOCGYnMM5cwl2U | 9599 | 2023-05-21T17:58:25Z | 2023-05-21T17:58:25Z | OWNER | Documentation: https://sqlite-utils--549.org.readthedocs.build/en/549/cli.html#cli-tui | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718586377 | |
https://github.com/simonw/sqlite-utils/pull/549#issuecomment-1556241555 | https://api.github.com/repos/simonw/sqlite-utils/issues/549 | 1556241555 | IC_kwDOCGYnMM5cwlyT | 22429695 | 2023-05-21T17:57:24Z | 2023-05-21T18:28:44Z | NONE | ## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/549?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage: **`83.33`**% and project coverage change: **`+0.06`** :tada: > Comparison is base [(`b3b100d`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/b3b100d7f5b2a76ccd4bfe8b0301a29e321d0375?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.30% compared to head [(`948692a`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/549?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.36%. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #549 +/- ## ========================================== + Coverage 96.30% 96.36% +0.06% ========================================== Files 6 6 Lines 2707 2726 +19 ========================================== + Hits 2607 2627 +20 + Misses 100 99 -1 ``` | [Impacted Files](https://app.codecov.io/gh/simonw/sqlite-utils/pull/549?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/549?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.22% <83.33%> (-0.03%)` | :arrow_down: | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/simonw/sqlite-utils/pull/549/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) </details> [:umbrella: View fu… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718586377 | |
https://github.com/simonw/sqlite-utils/issues/548#issuecomment-1556231832 | https://api.github.com/repos/simonw/sqlite-utils/issues/548 | 1556231832 | IC_kwDOCGYnMM5cwjaY | 9599 | 2023-05-21T17:24:13Z | 2023-05-21T17:24:13Z | OWNER | Oh, I see why that is now: https://github.com/simonw/sqlite-utils/blob/6027f3ea6939a399aeef2578fca17efec0e539df/sqlite_utils/cli.py#L2670-L2679 This is because of the following command: sqlite-utils analyze-tables table1 table2 --column x Since you can pass multiple tables AND multiple columns, the tool currently assumes that the column(s) you specify may be available on a subset of the provided tables. I'm going to change this so if the column is not on ANY of those tables you get an error. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718576761 | |
https://github.com/simonw/sqlite-utils/issues/547#issuecomment-1556228395 | https://api.github.com/repos/simonw/sqlite-utils/issues/547 | 1556228395 | IC_kwDOCGYnMM5cwikr | 9599 | 2023-05-21T17:11:15Z | 2023-05-21T17:11:15Z | OWNER | This will be a cosmetic change to the CLI output only - the options to save data to the database and the Python API function will continue to return `[(None, 158)]`. I can add an optimization though to avoid running the SQL count query if we know that it's all `null`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718572201 | |
https://github.com/simonw/sqlite-utils/issues/544#issuecomment-1556225788 | https://api.github.com/repos/simonw/sqlite-utils/issues/544 | 1556225788 | IC_kwDOCGYnMM5cwh78 | 9599 | 2023-05-21T17:02:05Z | 2023-05-21T17:02:05Z | OWNER | New docs: - https://sqlite-utils.datasette.io/en/latest/cli.html#cli-analyze-tables - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#analyze-tables - https://sqlite-utils.datasette.io/en/latest/python-api.html#analyzing-a-column - https://sqlite-utils.datasette.io/en/latest/reference.html#sqlite_utils.db.Table.analyze_column New help output: ``` % sqlite-utils analyze-tables --help Usage: sqlite-utils analyze-tables [OPTIONS] PATH [TABLES]... Analyze the columns in one or more tables Example: sqlite-utils analyze-tables data.db trees Options: -c, --column TEXT Specific columns to analyze --save Save results to _analyze_tables table --common-limit INTEGER How many common values --no-most Skip most common values --no-least Skip least common values --load-extension TEXT Path to SQLite extension, with optional :entrypoint -h, --help Show this message and exit. ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718515590 | |
https://github.com/simonw/sqlite-utils/pull/546#issuecomment-1556213396 | https://api.github.com/repos/simonw/sqlite-utils/issues/546 | 1556213396 | IC_kwDOCGYnMM5cwe6U | 9599 | 2023-05-21T15:58:12Z | 2023-05-21T16:18:46Z | OWNER | Documentation preview: - https://sqlite-utils--546.org.readthedocs.build/en/546/cli.html#cli-analyze-tables - https://sqlite-utils--546.org.readthedocs.build/en/546/cli-reference.html#analyze-tables - https://sqlite-utils--546.org.readthedocs.build/en/546/python-api.html#analyzing-a-column - https://sqlite-utils--546.org.readthedocs.build/en/546/reference.html#sqlite_utils.db.Table.analyze_column | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718550688 | |
https://github.com/simonw/sqlite-utils/pull/546#issuecomment-1556213031 | https://api.github.com/repos/simonw/sqlite-utils/issues/546 | 1556213031 | IC_kwDOCGYnMM5cwe0n | 22429695 | 2023-05-21T15:56:05Z | 2023-05-21T16:18:03Z | NONE | ## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/546?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage: **`93.75`**% and no project coverage change. > Comparison is base [(`b3b100d`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/b3b100d7f5b2a76ccd4bfe8b0301a29e321d0375?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.30% compared to head [(`9f23e68`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/546?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 96.31%. > :exclamation: Current head 9f23e68 differs from pull request most recent head 2eca17d. Consider uploading reports for the commit 2eca17d to get more accurate results <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #546 +/- ## ======================================= Coverage 96.30% 96.31% ======================================= Files 6 6 Lines 2707 2712 +5 ======================================= + Hits 2607 2612 +5 Misses 100 100 ``` | [Impacted Files](https://app.codecov.io/gh/simonw/sqlite-utils/pull/546?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/546?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.37% <90.90%> (+<0.01%)` | :arrow_up: | | [sqlite\_utils/cli.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/546?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718550688 | |
https://github.com/simonw/sqlite-utils/issues/544#issuecomment-1556211643 | https://api.github.com/repos/simonw/sqlite-utils/issues/544 | 1556211643 | IC_kwDOCGYnMM5cwee7 | 9599 | 2023-05-21T15:48:17Z | 2023-05-21T15:48:17Z | OWNER | I generated the commit message in https://github.com/simonw/sqlite-utils/commit/1c1991b447a1ddd3d61d9d4a8a1d6a9da47ced20 using `git diff | llm --system 'describe this change'`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718515590 | |
https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556210844 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | 1556210844 | IC_kwDOCGYnMM5cweSc | 9599 | 2023-05-21T15:44:10Z | 2023-05-21T15:44:10Z | OWNER | It looks like `nargs=-1` on a positional argument isn't yet supported - opened an issue here: - https://github.com/Textualize/trogon/issues/4 | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0} | 1718517882 | |
https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556191894 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | 1556191894 | IC_kwDOCGYnMM5cwZqW | 9599 | 2023-05-21T14:20:14Z | 2023-05-21T14:20:14Z | OWNER | Opened a feature request for customizing the help and command name: - https://github.com/Textualize/trogon/issues/2 | {"total_count": 2, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0} | 1718517882 | |
https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556190531 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | 1556190531 | IC_kwDOCGYnMM5cwZVD | 9599 | 2023-05-21T14:13:43Z | 2023-05-21T14:13:43Z | OWNER | OK, this works! ![trogon](https://github.com/simonw/sqlite-utils/assets/9599/2ae194c5-ec82-471a-9d1b-b01b3f2632f3) To try it out, install that branch from GitHub: pip install https://github.com/simonw/sqlite-utils/archive/refs/heads/trogon.zip Then run this: sqlite-utils install trogon And this: sqlite-utils tui | {"total_count": 5, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 3, "eyes": 0} | 1718517882 | |
https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556189823 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | 1556189823 | IC_kwDOCGYnMM5cwZJ_ | 9599 | 2023-05-21T14:09:59Z | 2023-05-21T14:09:59Z | OWNER | I don't want to add `trogon` as a default dependency because it's a little heavy - it pulls in all of Rich and Textual as well. People who use `sqlite-utils` just for its Python API won't benefit from this - it's a CLI feature only. But I have a `sqlite-utils install ...` command for helping people to install packages into the same virtual environment as `sqlite-utils` no matter how they installed that tool: https://sqlite-utils.datasette.io/en/stable/cli.html#cli-install So I can treat Trogon as an optional dependency and add the `sqlite-utils tui` command only if that package is also installed. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1718517882 | |
https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1548913065 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | 1548913065 | IC_kwDOCGYnMM5cUomp | 433780 | 2023-05-16T03:11:03Z | 2023-05-16T03:11:52Z | NONE | Using this thread and some [other resources](https://sqlite-utils.datasette.io/en/stable/cli.html#spatialite-helpers) I managed to cobble together a couple of sqlite-utils lines to add a geometry column for a table that already has a lat/lng column. ``` # add a geometry column sqlite-utils add-geometry-column [db name] [table name] geometry --type POINT --srid 4326 # add a point for each row to geometry column sqlite-utils --load-extension=spatialite [db name] 'update [table name] SET Geometry=MakePoint(longitude, latitude, 4326);' ``` | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1124731464 | |
https://github.com/simonw/datasette/pull/2052#issuecomment-1548617257 | https://api.github.com/repos/simonw/datasette/issues/2052 | 1548617257 | IC_kwDOBm6k_c5cTgYp | 193185 | 2023-05-15T21:32:20Z | 2023-05-15T21:32:20Z | CONTRIBUTOR | > Were you picturing that the whole plugin config object could be returned as a promise, or that the individual hooks (like makeColumnActions or makeAboveTablePanelConfigs supported returning a promise of arrays instead only returning plain arrays? The latter - that you could return a promise of arrays, so it parallels the ["await me maybe" pattern in Datasette](https://simonwillison.net/2020/Sep/2/await-me-maybe/), where you can return either a value, a callable or an awaitable. > I have a hunch that what you're describing might be achievable without adding Promises to the API with something Oops, I did a poor job explaining. Yes, this would work - but it requires me to continue to communicate the column names out of band (in order to fetch the facet data per-column before registering my plugin), vs being able to re-use them from the plugin implementation. This isn't that big of a deal - it'd be a nice ergonomic improvement, but nowhere near as a big of an improvement as having an officially sanctioned way to add stuff to the column menus in the first place. This could also be layered on in a future commit without breaking v1 users, too, so it's not at all urgent. > especially if those lines are encapsulated by a function we provide (maybe something that's available on the window provided by Datasette as an inline script tag Ah, this is maybe the the key point. Since it's all hosted inside Datasette, Datasette can provide some arbitrary sugar to make it easier to work with. My experience with async scripts in JS is that people sometimes don't understand the race conditions inherent to them. If they copy/paste from a tutorial, it does just work. But then they'll delete half the code, and by chance it still works on their machine/Datasette templates, and now someone's headed for an annoying debugging session -- maybe them, maybe someone else who tries to re-use their plugin. Again, a fairly minor thing, though. | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1651082214 | |
https://github.com/simonw/datasette/pull/2075#issuecomment-1547944971 | https://api.github.com/repos/simonw/datasette/issues/2075 | 1547944971 | IC_kwDOBm6k_c5cQ8QL | 22429695 | 2023-05-15T14:12:20Z | 2023-05-15T14:12:20Z | NONE | ## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2075?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch and project coverage have no change. > Comparison is base [(`49184c5`)](https://app.codecov.io/gh/simonw/datasette/commit/49184c569cd70efbda4f3f062afef3a34401d8d5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.40% compared to head [(`b99e1d3`)](https://app.codecov.io/gh/simonw/datasette/pull/2075?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.40%. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #2075 +/- ## ======================================= Coverage 92.40% 92.40% ======================================= Files 38 38 Lines 5751 5751 ======================================= Hits 5314 5314 Misses 437 437 ``` </details> [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2075?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1710164693 | |
https://github.com/simonw/datasette/pull/2068#issuecomment-1547911570 | https://api.github.com/repos/simonw/datasette/issues/2068 | 1547911570 | IC_kwDOBm6k_c5cQ0GS | 49699333 | 2023-05-15T13:59:35Z | 2023-05-15T13:59:35Z | CONTRIBUTOR | Superseded by #2075. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1690842199 | |
https://github.com/simonw/datasette/pull/2052#issuecomment-1546362374 | https://api.github.com/repos/simonw/datasette/issues/2052 | 1546362374 | IC_kwDOBm6k_c5cK54G | 9020979 | 2023-05-12T22:09:03Z | 2023-05-12T22:09:03Z | NONE | Hey @cldellow , thanks for the thoughtful feedback and describing the "lazy facets" feature! It sounds like the [postTask](https://developer.mozilla.org/en-US/docs/Web/API/Scheduler/postTask) API might be relevant for the types of network request scheduling you have in mind. Addressing your points inline below: > It might also be nice if the plugins could return Promises. Were you picturing that the whole plugin config object could be returned as a promise, or that the individual hooks (like `makeColumnActions` or `makeAboveTablePanelConfigs` supported returning a promise of arrays instead only returning plain arrays? I think what you're describing can be achievable, but I want to make sure I do so in a way that addresses your need / keeps the complexity of the plugin core system at a level this is approachable . I have a hunch that what you're describing might be achievable without adding Promises to the API with something like ``` fetch('/api/with-custom-facets').then(myFacets => { // reusing the go() idiom go(manager, myFacets); }) ``` but I'd like to confirm if that's the case before investigating adding support. > bulletproof plugin registration code that is robust against the order in which the script tags load Yes, I think what you wrote looks right to me! While it looks a little bit verbose compared to the second example, I'm hoping we can mitigate the cost of that during this API incubation phase by making it an easy-to-copy paste code snippet. I haven't heard of the GA queing pattern before, thanks for the example. I won't have time to implement of proof of concept in the next few weeks, but I took some time to think through the pros/cons to decide whether we may want to add this in a future release: I can see that this approach brings advantages - Plugin developers don't need to know the name of the datasette initialization event to start their plugin - Pushing a function to an array probably is easier (definitely more concise) than adding a docume… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1651082214 | |
https://github.com/simonw/datasette/issues/2073#issuecomment-1546119773 | https://api.github.com/repos/simonw/datasette/issues/2073 | 1546119773 | IC_kwDOBm6k_c5cJ-pd | 9599 | 2023-05-12T18:24:07Z | 2023-05-12T18:24:07Z | OWNER | Here's a demo of this breaking in Datasette Lite: https://lite.datasette.io/?sql=https://gist.github.com/simonw/261564c0ca01567df6eeb9b222b8be84&json=https%3A%2F%2Fcdn.jsdelivr.net%2Fnpm%2Fweb-features%2Findex.json#/data/baseline?_filter_column_1=is_baseline&_filter_op_1=exact&_filter_value_1=1&_filter_column_2=&_filter_op_2=notnull__1&_filter_value_2=1&_filter_column=&_filter_op=exact&_filter_value=&_sort=&_facet=is_baseline <img width="609" alt="image" src="https://github.com/simonw/datasette/assets/9599/3c23ae6e-7fb0-41d4-92b8-421c400a731a"> Here's a SQL query that demonstrates the underlying issue: ```sql select 'working', count(*) from baseline where is_baseline = 1 union all select 'broken', count(*) from baseline where is_baseline = '1' ``` https://lite.datasette.io/?sql=https://gist.github.com/simonw/261564c0ca01567df6eeb9b222b8be84&json=https%3A%2F%2Fcdn.jsdelivr.net%2Fnpm%2Fweb-features%2Findex.json#/data?sql=select+%27working%27%2C+count%28*%29+from+baseline+where+is_baseline+%3D+1%0Aunion+all%0Aselect+%27broken%27%2C+count%28*%29+from+baseline+where+is_baseline+%3D+%271%27 <img width="673" alt="image" src="https://github.com/simonw/datasette/assets/9599/867914f8-8cef-432c-a518-2739d1caf964"> | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1708030220 | |
https://github.com/simonw/datasette/issues/2073#issuecomment-1546117538 | https://api.github.com/repos/simonw/datasette/issues/2073 | 1546117538 | IC_kwDOBm6k_c5cJ-Gi | 9599 | 2023-05-12T18:21:38Z | 2023-05-12T18:21:38Z | OWNER | https://latest.datasette.io/fixtures doesn't currently have a view with any integer columns in it, making this bug harder to demonstrate there. I can't replicate the bug using https://datasette.io/content/plugins?_facet=stargazers_count&stargazers_count=3 - I would expect that not to work correctly. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1708030220 | |
https://github.com/simonw/sqlite-utils/issues/527#issuecomment-1540900733 | https://api.github.com/repos/simonw/sqlite-utils/issues/527 | 1540900733 | IC_kwDOCGYnMM5b2Ed9 | 167893 | 2023-05-09T21:15:05Z | 2023-05-09T21:15:05Z | CONTRIBUTOR | Sorry, I completely missed your first comment whilst on Easter break. This looks like a good practical compromise before v4. Thanks! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1578790070 | |
https://github.com/simonw/datasette/issues/2070#issuecomment-1540494121 | https://api.github.com/repos/simonw/datasette/issues/2070 | 1540494121 | IC_kwDOBm6k_c5b0hMp | 9599 | 2023-05-09T16:25:00Z | 2023-05-09T16:25:00Z | OWNER | Can now be used here: https://github.com/simonw/datasette/actions/workflows/deploy-branch-preview.yml | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1702354223 | |
https://github.com/simonw/datasette/issues/2070#issuecomment-1540491751 | https://api.github.com/repos/simonw/datasette/issues/2070 | 1540491751 | IC_kwDOBm6k_c5b0gnn | 9599 | 2023-05-09T16:23:12Z | 2023-05-09T16:23:12Z | OWNER | Added a actions `BRANCH_PREVIEW_VERCEL_TOKEN` secret to this repository. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1702354223 | |
https://github.com/simonw/sqlite-utils/pull/537#issuecomment-1539157643 | https://api.github.com/repos/simonw/sqlite-utils/issues/537 | 1539157643 | IC_kwDOCGYnMM5bva6L | 9599 | 2023-05-08T22:45:09Z | 2023-05-08T22:45:21Z | OWNER | Here's an example from the new tests: https://github.com/simonw/sqlite-utils/blob/a75abeb61b91a28650d3b9933e7ec80ad0d92529/tests/test_create.py#L291-L307 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1665200812 | |
https://github.com/simonw/sqlite-utils/issues/448#issuecomment-1539109816 | https://api.github.com/repos/simonw/sqlite-utils/issues/448 | 1539109816 | IC_kwDOCGYnMM5bvPO4 | 9599 | 2023-05-08T22:01:00Z | 2023-05-08T22:01:00Z | OWNER | This is being handled in: - #520 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1279144769 | |
https://github.com/simonw/sqlite-utils/issues/520#issuecomment-1539109587 | https://api.github.com/repos/simonw/sqlite-utils/issues/520 | 1539109587 | IC_kwDOCGYnMM5bvPLT | 9599 | 2023-05-08T22:00:46Z | 2023-05-08T22:00:46Z | OWNER | > Hey, isn't this essentially the same issue as #448 ? Yes it is, good catch! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1516644980 | |
https://github.com/simonw/sqlite-utils/issues/525#issuecomment-1539108140 | https://api.github.com/repos/simonw/sqlite-utils/issues/525 | 1539108140 | IC_kwDOCGYnMM5bvO0s | 9599 | 2023-05-08T21:59:41Z | 2023-05-08T21:59:41Z | OWNER | That original example passes against `main` now. | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1575131737 | |
https://github.com/simonw/sqlite-utils/issues/525#issuecomment-1539101853 | https://api.github.com/repos/simonw/sqlite-utils/issues/525 | 1539101853 | IC_kwDOCGYnMM5bvNSd | 9599 | 2023-05-08T21:52:44Z | 2023-05-08T21:52:44Z | OWNER | I like the `lambda-{uuid}` idea. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1575131737 | |
https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539100300 | https://api.github.com/repos/simonw/sqlite-utils/issues/514 | 1539100300 | IC_kwDOCGYnMM5bvM6M | 9599 | 2023-05-08T21:50:51Z | 2023-05-08T21:50:51Z | OWNER | Seeing as `sqlite-utils` doesn't currently provide mechanisms for adding `check` constraints like this I'm going to leave this - I'm happy with the fix I put in for the `not null` constraints. | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1465194249 | |
https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539099703 | https://api.github.com/repos/simonw/sqlite-utils/issues/514 | 1539099703 | IC_kwDOCGYnMM5bvMw3 | 9599 | 2023-05-08T21:50:06Z | 2023-05-08T21:50:06Z | OWNER | Applying the fix from the PR here doesn't fix the above problem either: - https://github.com/simonw/sqlite-utils/pull/515 So it looks like these kinds of `check` constraints currently aren't compatible with how `upsert()` works. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1465194249 | |
https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539094287 | https://api.github.com/repos/simonw/sqlite-utils/issues/514 | 1539094287 | IC_kwDOCGYnMM5bvLcP | 9599 | 2023-05-08T21:44:11Z | 2023-05-08T21:44:11Z | OWNER | OK, this fails silently: ```python import sqlite_utils db = sqlite_utils.Database(memory=True) db.execute('''CREATE TABLE employees ( id INTEGER PRIMARY KEY, name TEXT, age INTEGER, salary REAL, CHECK (salary is not null and salary > 0) );''') db["employees"].upsert({"id": 1, "name": "Bob"}, pk="id") list(db["employees"].rows) ```` It outputs: ```python [] ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1465194249 | |
https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539079507 | https://api.github.com/repos/simonw/sqlite-utils/issues/514 | 1539079507 | IC_kwDOCGYnMM5bvH1T | 9599 | 2023-05-08T21:28:37Z | 2023-05-08T21:28:37Z | OWNER | > This means that a table with NON NULL (or other constraint) columns that aren't part of the pkey can't have new rows upserted. Huh... on that basis, it's possible my fix in https://github.com/simonw/sqlite-utils/commit/2376c452a56b0c3e75e7ca698273434e32945304 is incomplete. I only covered the 'not null' case. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1465194249 | |
https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539078429 | https://api.github.com/repos/simonw/sqlite-utils/issues/514 | 1539078429 | IC_kwDOCGYnMM5bvHkd | 9599 | 2023-05-08T21:27:40Z | 2023-05-08T21:27:40Z | OWNER | Dupe of: - #538 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1465194249 | |
https://github.com/simonw/sqlite-utils/pull/515#issuecomment-1539077777 | https://api.github.com/repos/simonw/sqlite-utils/issues/515 | 1539077777 | IC_kwDOCGYnMM5bvHaR | 9599 | 2023-05-08T21:27:10Z | 2023-05-08T21:27:10Z | OWNER | I should have spotted this PR before I shipped my own fix! https://github.com/simonw/sqlite-utils/commit/2376c452a56b0c3e75e7ca698273434e32945304 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1465194930 | |
https://github.com/simonw/sqlite-utils/pull/519#issuecomment-1539058795 | https://api.github.com/repos/simonw/sqlite-utils/issues/519 | 1539058795 | IC_kwDOCGYnMM5bvCxr | 9599 | 2023-05-08T21:12:52Z | 2023-05-08T21:12:52Z | OWNER | This is a really neat fix, thank you. | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0} | 1505568103 | |
https://github.com/simonw/sqlite-utils/pull/537#issuecomment-1539055393 | https://api.github.com/repos/simonw/sqlite-utils/issues/537 | 1539055393 | IC_kwDOCGYnMM5bvB8h | 9599 | 2023-05-08T21:10:06Z | 2023-05-08T21:10:06Z | OWNER | Thanks! | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1665200812 | |
https://github.com/simonw/sqlite-utils/pull/528#issuecomment-1539053230 | https://api.github.com/repos/simonw/sqlite-utils/issues/528 | 1539053230 | IC_kwDOCGYnMM5bvBau | 9599 | 2023-05-08T21:08:23Z | 2023-05-08T21:08:23Z | OWNER | I fixed this in: - #527 Will fully remove this misfeature in: - #542 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1578793661 | |
https://github.com/simonw/sqlite-utils/issues/542#issuecomment-1539052467 | https://api.github.com/repos/simonw/sqlite-utils/issues/542 | 1539052467 | IC_kwDOCGYnMM5bvBOz | 9599 | 2023-05-08T21:07:41Z | 2023-05-08T21:07:41Z | OWNER | Relevant commits (will mostly revert these): - https://github.com/simonw/sqlite-utils/commit/455c35b512895c19bf922c2b804d750d27cb8dbd - https://github.com/simonw/sqlite-utils/commit/e0ec4c345129996011951e400388fd74865f65a2 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1700936245 | |
https://github.com/simonw/sqlite-utils/issues/527#issuecomment-1539051724 | https://api.github.com/repos/simonw/sqlite-utils/issues/527 | 1539051724 | IC_kwDOCGYnMM5bvBDM | 9599 | 2023-05-08T21:07:04Z | 2023-05-08T21:07:04Z | OWNER | Updated documentation: - https://sqlite-utils.datasette.io/en/latest/python-api.html#converting-data-in-columns - https://sqlite-utils.datasette.io/en/latest/cli.html#converting-data-in-columns - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#convert | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1578790070 | |
https://github.com/simonw/sqlite-utils/issues/527#issuecomment-1539035838 | https://api.github.com/repos/simonw/sqlite-utils/issues/527 | 1539035838 | IC_kwDOCGYnMM5bu9K- | 9599 | 2023-05-08T20:55:00Z | 2023-05-08T20:55:00Z | OWNER | I'm going to go with `--no-skip-false` as the CLI option. It's ugly, but this whole thing is ugly. I'm going to make a note to remove this misfeature in `sqlite-utils` 4. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1578790070 | |
https://github.com/simonw/sqlite-utils/issues/527#issuecomment-1539033736 | https://api.github.com/repos/simonw/sqlite-utils/issues/527 | 1539033736 | IC_kwDOCGYnMM5bu8qI | 9599 | 2023-05-08T20:52:51Z | 2023-05-08T20:52:51Z | OWNER | OK, I implemented that at the Python API level. I need to decide how it should work for the `sqlite-utils convert` command too: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#convert | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1578790070 | |
https://github.com/simonw/sqlite-utils/issues/530#issuecomment-1539018912 | https://api.github.com/repos/simonw/sqlite-utils/issues/530 | 1539018912 | IC_kwDOCGYnMM5bu5Cg | 9599 | 2023-05-08T20:39:00Z | 2023-05-08T20:39:00Z | OWNER | I think the natural place to add these in the Python library API would be https://sqlite-utils.datasette.io/en/stable/python-api.html#adding-foreign-key-constraints - maybe something like this: ```python db["books"].add_foreign_key("author_id", "authors", "id", on_delete=RESTRICT) ``` Then for the CLI tool could be added to https://sqlite-utils.datasette.io/en/stable/cli-reference.html#add-foreign-key ``` sqlite-utils add-foreign-key my.db books author_id authors id --on-update SET_NULL ``` I wouldn't support these for the other methods of adding foreign keys - `foreign_keys(...)` for the various `.insert()` etc methods and the `add_foreign_keys(...)` bulk menthod. | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 1, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1595340692 | |
https://github.com/simonw/sqlite-utils/issues/530#issuecomment-1539015064 | https://api.github.com/repos/simonw/sqlite-utils/issues/530 | 1539015064 | IC_kwDOCGYnMM5bu4GY | 9599 | 2023-05-08T20:35:07Z | 2023-05-08T20:35:07Z | OWNER | Wow, this is a neat feature I didn't know about. Looks like there are a bunch of options: - NO ACTION (default) - RESTRICT: application is prohibited from deleting a parent key when there exists one or more child keys mapped to it - SET NULL: when a parent key is deleted the child key columns of all rows in the child table that mapped to the parent key are set to contain SQL NULL values - SET DEFAULT: set a specific default - CASCADE: propagates the delete or update operation on the parent key to each dependent child key | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1595340692 | |
https://github.com/simonw/sqlite-utils/issues/532#issuecomment-1539009453 | https://api.github.com/repos/simonw/sqlite-utils/issues/532 | 1539009453 | IC_kwDOCGYnMM5bu2ut | 9599 | 2023-05-08T20:30:29Z | 2023-05-08T20:30:42Z | OWNER | Here's an improvement: ``` % sqlite-utils insert /tmp/b.db blah /tmp/blah.txt [####################################] 100% Error: Invalid JSON - use --csv for CSV or --tsv for TSV files JSON error: Expecting value: line 1 column 1 (char 0) ``` | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1620254998 | |
https://github.com/simonw/sqlite-utils/issues/532#issuecomment-1539006509 | https://api.github.com/repos/simonw/sqlite-utils/issues/532 | 1539006509 | IC_kwDOCGYnMM5bu2At | 9599 | 2023-05-08T20:28:56Z | 2023-05-08T20:28:56Z | OWNER | Was this a newline-delimited JSON file perhaps? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1620254998 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538975545 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538975545 | IC_kwDOCGYnMM5buuc5 | 1231935 | 2023-05-08T20:06:35Z | 2023-05-08T20:06:35Z | NONE | perfect, thank you! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1695428235 | |
https://github.com/simonw/sqlite-utils/issues/541#issuecomment-1538963959 | https://api.github.com/repos/simonw/sqlite-utils/issues/541 | 1538963959 | IC_kwDOCGYnMM5burn3 | 9599 | 2023-05-08T19:59:34Z | 2023-05-08T19:59:34Z | OWNER | There are 8 failing tests left: ``` ==== short test summary info ==== FAILED tests/test_cli_memory.py::test_memory_csv[False-test] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_csv[False-t] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_csv[False-t1] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_tsv[False] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_dump[extra_args0] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_two_files_with_same_stem - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_recipes.py::test_dateparse_errors[None-parsedate] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function Table.convert.<locals>.convert_value at 0x106bcaca0> FAILED tests/test_recipes.py::test_dateparse_errors[None-parsedatetime] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function Table.convert.<locals>.convert_value at 0x106bc9620> ERROR tests/test_cli.py::test_upsert_analyze - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> ==== 8 failed, 894 passed, 4 skipped, 1 error in 6.27s ==== ``` Full traceback here: https://gist.github.com/simonw/b40b3e814729d6c02a0302a84ce54d9e | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1700840265 | |
https://github.com/simonw/sqlite-utils/issues/534#issuecomment-1538933540 | https://api.github.com/repos/simonw/sqlite-utils/issues/534 | 1538933540 | IC_kwDOCGYnMM5bukMk | 9599 | 2023-05-08T19:34:37Z | 2023-05-08T19:34:37Z | OWNER | On macOS this shows the same warning: ``` % python -Wdefault $(which sqlite-utils) insert dogs.db dogs dogs.csv --csv [############------------------------] 35% [####################################] 100%/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py:1187: ResourceWarning: unclosed file <_io.TextIOWrapper name='dogs.csv' encoding='utf-8-sig'> insert_upsert_implementation( ResourceWarning: Enable tracemalloc to get the object allocation traceback ``` The file itself is a `click.File` which is automatically closed - https://click.palletsprojects.com/en/8.1.x/api/#click.File - but it looks like it's the `_io.TextIOWrapper` which is not being closed: https://github.com/simonw/sqlite-utils/blob/2376c452a56b0c3e75e7ca698273434e32945304/sqlite_utils/cli.py#L949-L956 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1622640374 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538921774 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538921774 | IC_kwDOCGYnMM5buhUu | 9599 | 2023-05-08T19:24:41Z | 2023-05-08T19:24:41Z | OWNER | That fix seems to work! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538910894 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538910894 | IC_kwDOCGYnMM5buequ | 9599 | 2023-05-08T19:16:52Z | 2023-05-08T19:17:00Z | OWNER | How about if I had logic which checked that all not-null columns were provided in the call to `upsert_all()` - and if they were, modified the `INSERT OR IGNORE INTO` to include a placeholder value for those columns that would then be fixed by the later `UPDATE`? Something like this: ```python [ ('INSERT OR IGNORE INTO [comments]([id], name) VALUES(?, ?);', [1, '']), ('UPDATE [comments] SET [name] = ? WHERE [id] = ?', ['Cleo', 1]) ] ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538903556 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538903556 | IC_kwDOCGYnMM5buc4E | 9599 | 2023-05-08T19:11:24Z | 2023-05-08T19:13:23Z | OWNER | I could detect if this happens using `cursor.rowcount` - not sure how I would recover from it though. This would also require some major re-engineering, since currently it all works by generating a list of SQL queries in advance and applying them inside a loop in `.insert_chunk()`: https://github.com/simonw/sqlite-utils/blob/80763edaa2bdaf1113717378b8d62075c4dcbcfb/sqlite_utils/db.py#L2839-L2878 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538893329 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538893329 | IC_kwDOCGYnMM5buaYR | 9599 | 2023-05-08T19:04:47Z | 2023-05-08T19:04:47Z | OWNER | This feels like a fundamental flaw in the way upserts are implemented by `sqlite-utils`. One fix would be to switch to using the `UPSERT` feature in SQLite: https://www.sqlite.org/lang_UPSERT.html But... > UPSERT syntax was added to SQLite with version 3.24.0 (2018-06-04). I still want to support SQLite versions earlier than that. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538889482 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538889482 | IC_kwDOCGYnMM5buZcK | 9599 | 2023-05-08T19:02:38Z | 2023-05-08T19:02:38Z | OWNER | Here's the code at fault: https://github.com/simonw/sqlite-utils/blob/80763edaa2bdaf1113717378b8d62075c4dcbcfb/sqlite_utils/db.py#L2774-L2788 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538887361 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538887361 | IC_kwDOCGYnMM5buY7B | 9599 | 2023-05-08T19:01:20Z | 2023-05-08T19:01:20Z | OWNER | Here's the problem: ```python import sqlite3 db = sqlite3.connect(":memory:") db.execute('create table foo (id integer primary key, name not null)') db.execute('insert into foo (id) values (1)') ``` Produces: ``` IntegrityError: NOT NULL constraint failed: foo.name ``` But this: ```python db.execute('insert or ignore into foo (id) values (1)') ``` Completes without an exception. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538801855 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538801855 | IC_kwDOCGYnMM5buEC_ | 9599 | 2023-05-08T18:00:17Z | 2023-05-08T18:00:17Z | OWNER | From time in the debugger, after creating the table it ends up doing this: ``` (Pdb) queries_and_params [ ('INSERT OR IGNORE INTO [comments]([id]) VALUES(?);', [1]), ('UPDATE [comments] SET [name] = ? WHERE [id] = ?', ['Cleo', 1]) ] ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538793817 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538793817 | IC_kwDOCGYnMM5buCFZ | 9599 | 2023-05-08T17:55:10Z | 2023-05-08T17:55:10Z | OWNER | Confirmed - I added this test and it fails: ```python def test_upsert_all_not_null(fresh_db): # https://github.com/simonw/sqlite-utils/issues/538 fresh_db["comments"].upsert_all( [{"id": 1, "name": "Cleo"}], pk="id", not_null=["name"], ) assert list(fresh_db["comments"].rows) == [{"id": 1, "name": "Cleo"}] ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1695428235 | |
https://github.com/simonw/sqlite-utils/issues/540#issuecomment-1537744000 | https://api.github.com/repos/simonw/sqlite-utils/issues/540 | 1537744000 | IC_kwDOCGYnMM5bqByA | 42327 | 2023-05-08T04:56:12Z | 2023-05-08T04:56:12Z | NONE | Hey @simonw, urllib3 maintainer here :wave: Sorry for breaking your CI. I understand you may prefer to pin the Python version, but note that specifying just `python: "3"` will get you the latest. We use that in urllib3: https://github.com/urllib3/urllib3/blob/main/.readthedocs.yml I can open PRs to sqlite-utils / datasette if you're interested | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1699184583 | |
https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537514610 | https://api.github.com/repos/simonw/sqlite-utils/issues/539 | 1537514610 | IC_kwDOCGYnMM5bpJxy | 9599 | 2023-05-07T18:43:24Z | 2023-05-07T18:43:24Z | OWNER | Documentation: - https://sqlite-utils.datasette.io/en/latest/cli.html#returning-raw-data-such-as-binary-content - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#query - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#memory | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1699174055 | |
https://github.com/simonw/sqlite-utils/issues/540#issuecomment-1537514069 | https://api.github.com/repos/simonw/sqlite-utils/issues/540 | 1537514069 | IC_kwDOCGYnMM5bpJpV | 9599 | 2023-05-07T18:40:18Z | 2023-05-07T18:40:18Z | OWNER | https://docs.readthedocs.io/en/stable/config-file/v2.html suggests: ```yaml build: os: ubuntu-22.04 tools: python: "3.11" ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1699184583 | |
https://github.com/simonw/sqlite-utils/issues/540#issuecomment-1537513912 | https://api.github.com/repos/simonw/sqlite-utils/issues/540 | 1537513912 | IC_kwDOCGYnMM5bpJm4 | 9599 | 2023-05-07T18:39:29Z | 2023-05-07T18:39:29Z | OWNER | https://readthedocs.org/projects/sqlite-utils/builds/20513034/ said: > Problem in your project's configuration. Invalid "python.version": expected one of (2, 2.7, 3, 3.5, 3.6, 3.7, 3.8, pypy3.5), got 3.11 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1699184583 | |
https://github.com/simonw/sqlite-utils/issues/540#issuecomment-1537513653 | https://api.github.com/repos/simonw/sqlite-utils/issues/540 | 1537513653 | IC_kwDOCGYnMM5bpJi1 | 9599 | 2023-05-07T18:37:59Z | 2023-05-07T18:38:19Z | OWNER | Useful comment here: - https://github.com/urllib3/urllib3/issues/2168#issuecomment-1537360928 > I faced the same issue. I switched to a different Python kernel (3.11.2) and it worked. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1699184583 | |
https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537507676 | https://api.github.com/repos/simonw/sqlite-utils/issues/539 | 1537507676 | IC_kwDOCGYnMM5bpIFc | 9599 | 2023-05-07T18:09:43Z | 2023-05-07T18:09:54Z | OWNER | This worked: ```bash sqlite-utils memory /tmp/books3.json:nl \ 'select name from books3' --raw-lines > titles.txt ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1699174055 | |
https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537507525 | https://api.github.com/repos/simonw/sqlite-utils/issues/539 | 1537507525 | IC_kwDOCGYnMM5bpIDF | 9599 | 2023-05-07T18:09:09Z | 2023-05-07T18:09:09Z | OWNER | I'm tempted to upgrade `--raw` to do this instead, but that would be a breaking change. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1699174055 | |
https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537507394 | https://api.github.com/repos/simonw/sqlite-utils/issues/539 | 1537507394 | IC_kwDOCGYnMM5bpIBC | 9599 | 2023-05-07T18:08:44Z | 2023-05-07T18:08:44Z | OWNER | Prototype: ```diff diff --git a/docs/cli-reference.rst b/docs/cli-reference.rst index 153e5f9..c830518 100644 --- a/docs/cli-reference.rst +++ b/docs/cli-reference.rst @@ -124,6 +124,7 @@ See :ref:`cli_query`. --json-cols Detect JSON cols and output them as JSON, not escaped strings -r, --raw Raw output, first column of first row + --raw-lines Raw output, first column of each row -p, --param <TEXT TEXT>... Named :parameters for SQL query --functions TEXT Python code defining one or more custom SQL functions @@ -192,6 +193,7 @@ See :ref:`cli_memory`. --json-cols Detect JSON cols and output them as JSON, not escaped strings -r, --raw Raw output, first column of first row + --raw-lines Raw output, first column of each row -p, --param <TEXT TEXT>... Named :parameters for SQL query --encoding TEXT Character encoding for CSV input, defaults to utf-8 diff --git a/sqlite_utils/cli.py b/sqlite_utils/cli.py index d25b1df..da0e4b6 100644 --- a/sqlite_utils/cli.py +++ b/sqlite_utils/cli.py @@ -1653,6 +1653,7 @@ def drop_view(path, view, ignore, load_extension): ) @output_options @click.option("-r", "--raw", is_flag=True, help="Raw output, first column of first row") +@click.option("--raw-lines", is_flag=True, help="Raw output, first column of each row") @click.option( "-p", "--param", @@ -1677,6 +1678,7 @@ def query( fmt, json_cols, raw, + raw_lines, param, load_extension, functions, @@ -1700,7 +1702,19 @@ def query( _register_functions(db, functions) _execute_query( - db, sql, param, raw, table, csv, tsv, no_headers, fmt, nl, arrays, json_cols + db, + sql, + par… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1699174055 | |
https://github.com/simonw/datasette/issues/2069#issuecomment-1537277919 | https://api.github.com/repos/simonw/datasette/issues/2069 | 1537277919 | IC_kwDOBm6k_c5boP_f | 31861128 | 2023-05-07T03:17:35Z | 2023-05-07T03:17:35Z | NONE | Some updates: I notice that there is an option in the CLI where we can explicitly set `immutable` mode when spinning up the server ```console Options: -i, --immutable PATH Database files to open in immutable mode ``` Then, the question is - how can I disable immutable mode in the deployed instance on Vercel? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 1698865182 |