issue_comments
7,931 rows where author_association = "OWNER" sorted by id descending
This data as json, CSV (advanced)
id ▲ | html_url | issue_url | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
1563565407 | https://github.com/simonw/datasette/issues/2079#issuecomment-1563565407 | https://api.github.com/repos/simonw/datasette/issues/2079 | IC_kwDOBm6k_c5dMh1f | simonw 9599 | 2023-05-25T22:09:53Z | 2023-05-25T22:09:53Z | OWNER | Updated docs: https://docs.datasette.io/en/latest/json_api.html#enabling-cors | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Datasette should serve Access-Control-Max-Age 1726531350 | |
1563563438 | https://github.com/simonw/datasette/issues/2079#issuecomment-1563563438 | https://api.github.com/repos/simonw/datasette/issues/2079 | IC_kwDOBm6k_c5dMhWu | simonw 9599 | 2023-05-25T22:08:28Z | 2023-05-25T22:08:28Z | OWNER | I ran this on https://www.example.com/ twice using the console: ```javascript fetch( `https://latest.datasette.io/ephemeral/foo/1/-/update`, { method: "POST", mode: "cors", headers: { Authorization: `Bearer tok`, "Content-Type": "application/json", }, body: JSON.stringify({update: {blah: 1}}), } ) .then((r) => r.json()) .then((data) => { console.log(data); }); ``` And got this in the network pane: <img width="945" alt="image" src="https://github.com/simonw/datasette/assets/9599/cc767917-388b-4616-b35d-efaa1591948a"> | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Datasette should serve Access-Control-Max-Age 1726531350 | |
1563558915 | https://github.com/simonw/datasette/issues/2079#issuecomment-1563558915 | https://api.github.com/repos/simonw/datasette/issues/2079 | IC_kwDOBm6k_c5dMgQD | simonw 9599 | 2023-05-25T22:04:41Z | 2023-05-25T22:04:41Z | OWNER | I'm going with 3600 for 1 hour instead of 600 for 10 minutes. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Datasette should serve Access-Control-Max-Age 1726531350 | |
1563547097 | https://github.com/simonw/datasette/issues/2079#issuecomment-1563547097 | https://api.github.com/repos/simonw/datasette/issues/2079 | IC_kwDOBm6k_c5dMdXZ | simonw 9599 | 2023-05-25T21:51:38Z | 2023-05-25T21:51:38Z | OWNER | Also need to update this documentation: https://github.com/simonw/datasette/blob/9584879534ff0556e04e4c420262972884cac87b/docs/json_api.rst?plain=1#L453-L465 Or maybe make that automated via `cog`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Datasette should serve Access-Control-Max-Age 1726531350 | |
1563522011 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563522011 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dMXPb | simonw 9599 | 2023-05-25T21:22:30Z | 2023-05-25T21:22:30Z | OWNER | This is bad: ```python async def __call__(self, request, datasette): try: handler = getattr(self, request.method.lower()) return await handler(request, datasette) except AttributeError: return await self.method_not_allowed(request) ``` Because it hides any `AttributeError` exceptions that might occur in the view code. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563511171 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563511171 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dMUmD | simonw 9599 | 2023-05-25T21:11:20Z | 2023-05-25T21:13:05Z | OWNER | I'm going to call this `VerbView` for the moment. Might even rename it to `View` later. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563498048 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563498048 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dMRZA | simonw 9599 | 2023-05-25T20:57:52Z | 2023-05-25T20:58:13Z | OWNER | Here's a new `BaseView` class that automatically populates `OPTIONS` based on available methods: ```python class BaseView: async def head(self, *args, **kwargs): try: response = await self.get(*args, **kwargs) response.body = b"" return response except AttributeError: raise async def method_not_allowed(self, request): if ( request.path.endswith(".json") or request.headers.get("content-type") == "application/json" ): response = Response.json( {"ok": False, "error": "Method not allowed"}, status=405 ) else: response = Response.text("Method not allowed", status=405) return response async def options(self, request, *args, **kwargs): response = Response.text("ok") response.headers["allow"] = ", ".join( method.upper() for method in ("head", "get", "post", "put", "patch", "delete") if hasattr(self, method) ) return response async def __call__(self, request, datasette): try: handler = getattr(self, request.method.lower()) return await handler(request, datasette) except AttributeError: return await self.method_not_allowed(request) class DemoView(BaseView): async def get(self, datasette, request): return Response.text("Hello there! {} - {}".format(datasette, request)) post = get ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563488929 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563488929 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dMPKh | simonw 9599 | 2023-05-25T20:48:12Z | 2023-05-25T20:48:39Z | OWNER | Actually no need for that extra level of parameter detection: `BaseView.__call__` should _always_ take `datasette, request` - `scope` and `receive` are both available on `request`, and `send` is only needed if you're not planning on returning a `Response` object. So the `get` and `post` and suchlike methods should take `datasette` and `request` too. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563444296 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563444296 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dMERI | simonw 9599 | 2023-05-25T20:06:08Z | 2023-05-25T20:06:08Z | OWNER | This prototype seems to work well: ```diff diff --git a/datasette/app.py b/datasette/app.py index d7dace67..ed0edf28 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -17,6 +17,7 @@ import secrets import sys import threading import time +import types import urllib.parse from concurrent import futures from pathlib import Path @@ -1266,6 +1267,8 @@ class Datasette: # TODO: /favicon.ico and /-/static/ deserve far-future cache expires add_route(favicon, "/favicon.ico") + add_route(wrap_view(DemoView, self), '/demo') + add_route( asgi_static(app_root / "datasette" / "static"), r"/-/static/(?P<path>.*)$" ) @@ -1673,8 +1676,46 @@ def _cleaner_task_str(task): return _cleaner_task_str_re.sub("", s) -def wrap_view(view_fn, datasette): - @functools.wraps(view_fn) +class DemoView: + async def __call__(self, datasette, request): + return Response.text("Hello there! {} - {}".format(datasette, request)) + +def wrap_view(view_fn_or_class, datasette): + is_function = isinstance(view_fn_or_class, types.FunctionType) + if is_function: + return wrap_view_function(view_fn_or_class, datasette) + else: + if not isinstance(view_fn_or_class, type): + raise ValueError("view_fn_or_class must be a function or a class") + return wrap_view_class(view_fn_or_class, datasette) + + +def wrap_view_class(view_class, datasette): + async def async_view_for_class(request, send): + instance = view_class() + if inspect.iscoroutinefunction(instance.__call__): + return await async_call_with_supported_arguments( + instance.__call__, + scope=request.scope, + receive=request.receive, + send=send, + request=request, + datasette=datasette, + ) + else: + return call_with_supported_arguments( + instance.__call__, … | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563419066 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563419066 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dL-G6 | simonw 9599 | 2023-05-25T19:42:16Z | 2023-05-25T19:43:08Z | OWNER | Maybe what I want here is the ability to register classes with the router - and have the router know that if it's a class it should instantiate it via its constructor and then await `__call__` it. The neat thing about it is that it can reduce the risk of having a class instance that accidentally shares state between requests. It also encourages that each class only responds based on the `datasette, request, ...` objects that are passed to its methods. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563359114 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563359114 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dLveK | simonw 9599 | 2023-05-25T18:47:57Z | 2023-05-25T18:47:57Z | OWNER | Oops, that broke everything: ``` @documented async def await_me_maybe(value: typing.Any) -> typing.Any: "If value is callable, call it. If awaitable, await it. Otherwise return it." > if callable(value): E TypeError: 'module' object is not callable ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563329245 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563329245 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dLoLd | simonw 9599 | 2023-05-25T18:26:47Z | 2023-05-25T18:28:08Z | OWNER | With type hints and a namedtuple: ```python import asyncio import types from typing import NamedTuple, Any class CallableStatus(NamedTuple): is_callable: bool is_async_callable: bool def check_callable(obj: Any) -> CallableStatus: if not callable(obj): return CallableStatus(False, False) if isinstance(obj, type): # It's a class return CallableStatus(True, False) if isinstance(obj, types.FunctionType): return CallableStatus(True, asyncio.iscoroutinefunction(obj)) if hasattr(obj, "__call__"): return CallableStatus(True, asyncio.iscoroutinefunction(obj.__call__)) assert False, "obj {} is somehow callable with no __call__ method".format(repr(obj)) ``` ```python for thing in ( async_func, non_async_func, AsyncClass(), NotAsyncClass(), ClassNoCall(), AsyncClass, NotAsyncClass, ClassNoCall, ): print(thing, check_callable(thing)) ``` ``` <function async_func at 0x1073d5120> CallableStatus(is_callable=True, is_async_callable=True) <function non_async_func at 0x1073d5080> CallableStatus(is_callable=True, is_async_callable=False) <__main__.AsyncClass object at 0x106ba7490> CallableStatus(is_callable=True, is_async_callable=True) <__main__.NotAsyncClass object at 0x106740150> CallableStatus(is_callable=True, is_async_callable=False) <__main__.ClassNoCall object at 0x10676d910> CallableStatus(is_callable=False, is_async_callable=False) <class '__main__.AsyncClass'> CallableStatus(is_callable=True, is_async_callable=False) <class '__main__.NotAsyncClass'> CallableStatus(is_callable=True, is_async_callable=False) <class '__main__.ClassNoCall'> CallableStatus(is_callable=True, is_async_callable=False) ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563326000 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563326000 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dLnYw | simonw 9599 | 2023-05-25T18:23:38Z | 2023-05-25T18:23:38Z | OWNER | I don't like that `is_callable()` implies a single boolean result but actually returns a pair. I'll call it `check_callable(obj)` instead. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563318598 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563318598 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dLllG | simonw 9599 | 2023-05-25T18:17:03Z | 2023-05-25T18:21:25Z | OWNER | I think I want that to return `(is_callable, is_async)` - so I can both test if the thing can be called AND if it should be awaited in the same operation (without any exceptions). I tried this: ```python def is_callable(obj): "Returns (is_callable, is_async_callable)" if not callable(obj): return False, False if isinstance(obj, types.FunctionType): return True, asyncio.iscoroutinefunction(obj) if hasattr(obj, '__call__'): return True, asyncio.iscoroutinefunction(obj.__call__) return False, False ``` ```python for thing in ( async_func, non_async_func, AsyncClass(), NotAsyncClass(), ClassNoCall(), AsyncClass, NotAsyncClass, ClassNoCall ): print(thing, is_callable(thing)) ``` And got: ``` <function async_func at 0x1073d5120> (True, True) <function non_async_func at 0x1073d5080> (True, False) <__main__.AsyncClass object at 0x106cce490> (True, True) <__main__.NotAsyncClass object at 0x106ccf710> (True, False) <__main__.ClassNoCall object at 0x106ccc810> (False, False) <class '__main__.AsyncClass'> (True, True) <class '__main__.NotAsyncClass'> (True, False) <class '__main__.ClassNoCall'> (True, False) ``` Which is almost right, but I don't like that `AsyncClass` is shown as callable (which it is, since it's a class) and awaitable (which it is not - the `__call__` method may be async but calling the class constructor is not). So I'm going to detect classes using `isinstance(obj, type)`. ```python def is_callable(obj): "Returns (is_callable, is_async_callable)" if not callable(obj): return False, False if isinstance(obj, type): # It's a class return True, False if isinstance(obj, types.FunctionType): return True, asyncio.iscoroutinefunction(obj) if hasattr(obj, '__call__'): return True, asyncio.iscoroutinefunction(obj.__call__) assert False, "obj {} somehow is callable with no __call__ method".format(obj) ``` I am re… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563308919 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563308919 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dLjN3 | simonw 9599 | 2023-05-25T18:08:34Z | 2023-05-25T18:08:34Z | OWNER | After much fiddling this seems to work: ```python import asyncio, types def is_async_callable(obj): if not callable(obj): raise ValueError("Object is not callable") if isinstance(obj, types.FunctionType): return asyncio.iscoroutinefunction(obj) if hasattr(obj, '__call__'): return asyncio.iscoroutinefunction(obj.__call__) raise ValueError("Not a function and has no __call__ attribute") ``` Tested like so: ```python class AsyncClass: async def __call__(self): pass class NotAsyncClass: def __call__(self): pass class ClassNoCall: pass async def async_func(): pass def non_async_func(): pass for thing in (AsyncClass(), NotAsyncClass(), ClassNoCall(), async_func, non_async_func): try: print(thing, is_async_callable(thing)) except Exception as ex: print(thing, ex) ``` ``` <__main__.AsyncClass object at 0x106c32150> True <__main__.NotAsyncClass object at 0x106c32390> False <__main__.ClassNoCall object at 0x106c32750> Object is not callable <function async_func at 0x1073d5120> True <function non_async_func at 0x1073d5080> False ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563294669 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563294669 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dLfvN | simonw 9599 | 2023-05-25T17:57:06Z | 2023-05-25T17:57:06Z | OWNER | I may need to be able to detect if a class instance has an `async def __call__` method - I think I can do that like so: ```python def iscoroutinefunction(obj): if inspect.iscoroutinefunction(obj): return True if hasattr(obj, '__call__') and inspect.iscoroutinefunction(obj.__call__): return True return False ``` From https://github.com/encode/starlette/issues/886#issuecomment-606585152 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563292373 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563292373 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dLfLV | simonw 9599 | 2023-05-25T17:55:12Z | 2023-05-25T17:55:30Z | OWNER | So I think subclasses of `BaseView` need to offer a callable which accepts all five of the DI arguments - `datasette`, `request`, `scope`, `send`, `receive` - and then makes a decision based on the HTTP verb as to which method of the class to call. Those methods themselves can accept a subset of those parameters and will only be sent on to them. Having two layers of parameter detection feels a little bit untidy, but I think it will work. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563285150 | https://github.com/simonw/datasette/pull/2053#issuecomment-1563285150 | https://api.github.com/repos/simonw/datasette/issues/2053 | IC_kwDOBm6k_c5dLdae | simonw 9599 | 2023-05-25T17:48:50Z | 2023-05-25T17:49:52Z | OWNER | Uncommitted experimental code: ```diff diff --git a/datasette/views/database.py b/datasette/views/database.py index 455ebd1f..85775433 100644 --- a/datasette/views/database.py +++ b/datasette/views/database.py @@ -909,12 +909,13 @@ async def query_view( elif format_ in datasette.renderers.keys(): # Dispatch request to the correct output format renderer # (CSV is not handled here due to streaming) + print(data) result = call_with_supported_arguments( datasette.renderers[format_][0], datasette=datasette, - columns=columns, - rows=rows, - sql=sql, + columns=data["rows"][0].keys(), + rows=data["rows"], + sql='', query_name=None, database=db.name, table=None, @@ -923,7 +924,7 @@ async def query_view( # These will be deprecated in Datasette 1.0: args=request.args, data={ - "rows": rows, + "rows": data["rows"], }, # TODO what should this be? ) result = await await_me_maybe(result) diff --git a/docs/index.rst b/docs/index.rst index 5a9cc7ed..254ed3da 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -57,6 +57,7 @@ Contents settings introspection custom_templates + template_context plugins writing_plugins plugin_hooks ``` Where `docs/template_context.rst` looked like this: ```rst .. _template_context: Template context ================ .. currentmodule:: datasette.context This page describes the variables made available to templates used by Datasette to render different pages of the application. .. autoclass:: QueryContext :members: ``` And `datasette/context.py` had this: ```python from dataclasses import dataclass @dataclass class QueryContext: """ Used by the ``/database`` page when showing the results of a SQL query """ id: int "… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | WIP new JSON for queries 1656432059 | |
1563283939 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563283939 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dLdHj | simonw 9599 | 2023-05-25T17:47:38Z | 2023-05-25T17:47:38Z | OWNER | The idea behind `wrap_view()` is dependency injection - it's mainly used by plugins: https://docs.datasette.io/en/0.64.3/plugin_hooks.html#register-routes-datasette But I like the pattern so I started using it for some of Datasette's own features. I should use it for _all_ of Datasette's own features. But I still like the way `BaseView` helps with running different code for GET/POST/etc verbs. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1563282327 | https://github.com/simonw/datasette/issues/2078#issuecomment-1563282327 | https://api.github.com/repos/simonw/datasette/issues/2078 | IC_kwDOBm6k_c5dLcuX | simonw 9599 | 2023-05-25T17:46:05Z | 2023-05-25T17:46:05Z | OWNER | Here's what `wrap_view()` does: https://github.com/simonw/datasette/blob/49184c569cd70efbda4f3f062afef3a34401d8d5/datasette/app.py#L1676-L1700 It's used e.g. here: https://github.com/simonw/datasette/blob/49184c569cd70efbda4f3f062afef3a34401d8d5/datasette/app.py#L1371-L1375 The `BaseView` thing meanwhile works like this: https://github.com/simonw/datasette/blob/d97e82df3c8a3f2e97038d7080167be9bb74a68d/datasette/views/base.py#L56-L157 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Resolve the difference between `wrap_view()` and `BaseView` 1726236847 | |
1556292204 | https://github.com/simonw/sqlite-utils/issues/552#issuecomment-1556292204 | https://api.github.com/repos/simonw/sqlite-utils/issues/552 | IC_kwDOCGYnMM5cwyJs | simonw 9599 | 2023-05-21T21:05:15Z | 2023-05-21T21:05:15Z | OWNER | Now live at https://sqlite-utils.datasette.io/en/latest/installation.html#setting-up-shell-completion | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Document how to setup shell auto-completion 1718612569 | |
1556291915 | https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556291915 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | IC_kwDOCGYnMM5cwyFL | simonw 9599 | 2023-05-21T21:04:03Z | 2023-05-21T21:04:03Z | OWNER | Now live at https://sqlite-utils.datasette.io/en/latest/cli.html | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Make as many examples in the CLI docs as possible copy-and-pastable 1718607907 | |
1556288300 | https://github.com/simonw/sqlite-utils/pull/553#issuecomment-1556288300 | https://api.github.com/repos/simonw/sqlite-utils/issues/553 | IC_kwDOCGYnMM5cwxMs | simonw 9599 | 2023-05-21T20:48:01Z | 2023-05-21T20:48:01Z | OWNER | If https://sqlite-utils--553.org.readthedocs.build/en/553/cli.html#running-sql-queries looks good I can merge this. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Reformatted CLI examples in docs 1718635018 | |
1556288270 | https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556288270 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | IC_kwDOCGYnMM5cwxMO | simonw 9599 | 2023-05-21T20:47:51Z | 2023-05-21T20:47:51Z | OWNER | This page has all of the changes: https://sqlite-utils--553.org.readthedocs.build/en/553/cli.html#running-sql-queries | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Make as many examples in the CLI docs as possible copy-and-pastable 1718607907 | |
1556287599 | https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556287599 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | IC_kwDOCGYnMM5cwxBv | simonw 9599 | 2023-05-21T20:44:55Z | 2023-05-21T20:44:55Z | OWNER | Put this in a PR so I can preview it: - #553 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Make as many examples in the CLI docs as possible copy-and-pastable 1718607907 | |
1556269616 | https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556269616 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | IC_kwDOCGYnMM5cwsow | simonw 9599 | 2023-05-21T19:33:13Z | 2023-05-21T19:33:13Z | OWNER | Now released: https://sqlite-utils.datasette.io/en/stable/changelog.html#v3-32 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Try out Trogon for a tui interface 1718517882 | |
1556265772 | https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556265772 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | IC_kwDOCGYnMM5cwrss | simonw 9599 | 2023-05-21T19:16:15Z | 2023-05-21T19:16:15Z | OWNER | Another option: <img width="593" alt="image" src="https://github.com/simonw/sqlite-utils/assets/9599/a9d819a0-89e3-4b67-b2f0-1052b0a7a7c0"> That's using this markup: ``` Newline-delimited JSON ~~~~~~~~~~~~~~~~~~~~~~ Use ``--nl`` to get back newline-delimited JSON objects: .. code-block:: bash sqlite-utils dogs.db "select * from dogs" --nl .. code-block:: output {"id": 1, "age": 4, "name": "Cleo"} {"id": 2, "age": 2, "name": "Pancakes"} ``` And this extra CSS: ```css .highlight-output .highlight { border-left: 9px solid #30c94f; } ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Make as many examples in the CLI docs as possible copy-and-pastable 1718607907 | |
1556263182 | https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556263182 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | IC_kwDOCGYnMM5cwrEO | simonw 9599 | 2023-05-21T19:06:48Z | 2023-05-21T19:06:48Z | OWNER | I could split them up into two blocks like this: <img width="771" alt="image" src="https://github.com/simonw/sqlite-utils/assets/9599/9b042ac8-db30-49d5-8882-1ea7be272a49"> I do miss the visual indication that one of these is the command and one is the output though. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Make as many examples in the CLI docs as possible copy-and-pastable 1718607907 | |
1556262574 | https://github.com/simonw/sqlite-utils/issues/551#issuecomment-1556262574 | https://api.github.com/repos/simonw/sqlite-utils/issues/551 | IC_kwDOCGYnMM5cwq6u | simonw 9599 | 2023-05-21T19:04:59Z | 2023-05-21T19:04:59Z | OWNER | I wrote the docs like this because early examples include both the command and its output: https://sqlite-utils.datasette.io/en/stable/cli.html#returning-json <img width="766" alt="image" src="https://github.com/simonw/sqlite-utils/assets/9599/a5d05f24-ca0f-4718-a0a8-d85adc736141"> | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Make as many examples in the CLI docs as possible copy-and-pastable 1718607907 | |
1556255309 | https://github.com/simonw/sqlite-utils/issues/550#issuecomment-1556255309 | https://api.github.com/repos/simonw/sqlite-utils/issues/550 | IC_kwDOCGYnMM5cwpJN | simonw 9599 | 2023-05-21T18:42:25Z | 2023-05-21T18:42:25Z | OWNER | Tests passed here: https://github.com/simonw/sqlite-utils/actions/runs/5039119716 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | AttributeError: 'EntryPoints' object has no attribute 'get' for flake8 on Python 3.7 1718595700 | |
1556250236 | https://github.com/simonw/sqlite-utils/issues/550#issuecomment-1556250236 | https://api.github.com/repos/simonw/sqlite-utils/issues/550 | IC_kwDOCGYnMM5cwn58 | simonw 9599 | 2023-05-21T18:25:26Z | 2023-05-21T18:25:26Z | OWNER | Relevant issues: - https://github.com/python/importlib_metadata/issues/406 - https://github.com/PyCQA/flake8/issues/1701 It looks to me like this is only a problem for `flake8` on Python 3.7 - 3.8 and higher work OK. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | AttributeError: 'EntryPoints' object has no attribute 'get' for flake8 on Python 3.7 1718595700 | |
1556249984 | https://github.com/simonw/sqlite-utils/issues/550#issuecomment-1556249984 | https://api.github.com/repos/simonw/sqlite-utils/issues/550 | IC_kwDOCGYnMM5cwn2A | simonw 9599 | 2023-05-21T18:24:48Z | 2023-05-21T18:24:48Z | OWNER | This is blocking: - #549 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | AttributeError: 'EntryPoints' object has no attribute 'get' for flake8 on Python 3.7 1718595700 | |
1556247818 | https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556247818 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | IC_kwDOCGYnMM5cwnUK | simonw 9599 | 2023-05-21T18:17:46Z | 2023-05-21T18:17:46Z | OWNER | Draft documentation: https://sqlite-utils--549.org.readthedocs.build/en/549/cli.html#cli-tui | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Try out Trogon for a tui interface 1718517882 | |
1556242262 | https://github.com/simonw/sqlite-utils/pull/549#issuecomment-1556242262 | https://api.github.com/repos/simonw/sqlite-utils/issues/549 | IC_kwDOCGYnMM5cwl9W | simonw 9599 | 2023-05-21T18:00:05Z | 2023-05-21T18:00:05Z | OWNER | Failing `mypy` test: https://github.com/simonw/sqlite-utils/actions/runs/5038983349/jobs/9036828465 ``` sqlite_utils/cli.py:37: error: Skipping analyzing "trogon": module is installed, but missing library stubs or py.typed marker [import] sqlite_utils/cli.py:37: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports Found 1 error in 1 file (checked 52 source files) ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | TUI powered by Trogon 1718586377 | |
1556241812 | https://github.com/simonw/sqlite-utils/pull/549#issuecomment-1556241812 | https://api.github.com/repos/simonw/sqlite-utils/issues/549 | IC_kwDOCGYnMM5cwl2U | simonw 9599 | 2023-05-21T17:58:25Z | 2023-05-21T17:58:25Z | OWNER | Documentation: https://sqlite-utils--549.org.readthedocs.build/en/549/cli.html#cli-tui | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | TUI powered by Trogon 1718586377 | |
1556231832 | https://github.com/simonw/sqlite-utils/issues/548#issuecomment-1556231832 | https://api.github.com/repos/simonw/sqlite-utils/issues/548 | IC_kwDOCGYnMM5cwjaY | simonw 9599 | 2023-05-21T17:24:13Z | 2023-05-21T17:24:13Z | OWNER | Oh, I see why that is now: https://github.com/simonw/sqlite-utils/blob/6027f3ea6939a399aeef2578fca17efec0e539df/sqlite_utils/cli.py#L2670-L2679 This is because of the following command: sqlite-utils analyze-tables table1 table2 --column x Since you can pass multiple tables AND multiple columns, the tool currently assumes that the column(s) you specify may be available on a subset of the provided tables. I'm going to change this so if the column is not on ANY of those tables you get an error. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | analyze-tables should validate provide --column names 1718576761 | |
1556228395 | https://github.com/simonw/sqlite-utils/issues/547#issuecomment-1556228395 | https://api.github.com/repos/simonw/sqlite-utils/issues/547 | IC_kwDOCGYnMM5cwikr | simonw 9599 | 2023-05-21T17:11:15Z | 2023-05-21T17:11:15Z | OWNER | This will be a cosmetic change to the CLI output only - the options to save data to the database and the Python API function will continue to return `[(None, 158)]`. I can add an optimization though to avoid running the SQL count query if we know that it's all `null`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | No need to show common values if everything is null 1718572201 | |
1556225788 | https://github.com/simonw/sqlite-utils/issues/544#issuecomment-1556225788 | https://api.github.com/repos/simonw/sqlite-utils/issues/544 | IC_kwDOCGYnMM5cwh78 | simonw 9599 | 2023-05-21T17:02:05Z | 2023-05-21T17:02:05Z | OWNER | New docs: - https://sqlite-utils.datasette.io/en/latest/cli.html#cli-analyze-tables - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#analyze-tables - https://sqlite-utils.datasette.io/en/latest/python-api.html#analyzing-a-column - https://sqlite-utils.datasette.io/en/latest/reference.html#sqlite_utils.db.Table.analyze_column New help output: ``` % sqlite-utils analyze-tables --help Usage: sqlite-utils analyze-tables [OPTIONS] PATH [TABLES]... Analyze the columns in one or more tables Example: sqlite-utils analyze-tables data.db trees Options: -c, --column TEXT Specific columns to analyze --save Save results to _analyze_tables table --common-limit INTEGER How many common values --no-most Skip most common values --no-least Skip least common values --load-extension TEXT Path to SQLite extension, with optional :entrypoint -h, --help Show this message and exit. ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | New options for analyze-tables --common-limit --no-most and --no-least 1718515590 | |
1556213396 | https://github.com/simonw/sqlite-utils/pull/546#issuecomment-1556213396 | https://api.github.com/repos/simonw/sqlite-utils/issues/546 | IC_kwDOCGYnMM5cwe6U | simonw 9599 | 2023-05-21T15:58:12Z | 2023-05-21T16:18:46Z | OWNER | Documentation preview: - https://sqlite-utils--546.org.readthedocs.build/en/546/cli.html#cli-analyze-tables - https://sqlite-utils--546.org.readthedocs.build/en/546/cli-reference.html#analyze-tables - https://sqlite-utils--546.org.readthedocs.build/en/546/python-api.html#analyzing-a-column - https://sqlite-utils--546.org.readthedocs.build/en/546/reference.html#sqlite_utils.db.Table.analyze_column | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Analyze tables options: --common-limit, --no-most, --no-least 1718550688 | |
1556211643 | https://github.com/simonw/sqlite-utils/issues/544#issuecomment-1556211643 | https://api.github.com/repos/simonw/sqlite-utils/issues/544 | IC_kwDOCGYnMM5cwee7 | simonw 9599 | 2023-05-21T15:48:17Z | 2023-05-21T15:48:17Z | OWNER | I generated the commit message in https://github.com/simonw/sqlite-utils/commit/1c1991b447a1ddd3d61d9d4a8a1d6a9da47ced20 using `git diff | llm --system 'describe this change'`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | New options for analyze-tables --common-limit --no-most and --no-least 1718515590 | |
1556210844 | https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556210844 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | IC_kwDOCGYnMM5cweSc | simonw 9599 | 2023-05-21T15:44:10Z | 2023-05-21T15:44:10Z | OWNER | It looks like `nargs=-1` on a positional argument isn't yet supported - opened an issue here: - https://github.com/Textualize/trogon/issues/4 | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0} | Try out Trogon for a tui interface 1718517882 | |
1556191894 | https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556191894 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | IC_kwDOCGYnMM5cwZqW | simonw 9599 | 2023-05-21T14:20:14Z | 2023-05-21T14:20:14Z | OWNER | Opened a feature request for customizing the help and command name: - https://github.com/Textualize/trogon/issues/2 | {"total_count": 2, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0} | Try out Trogon for a tui interface 1718517882 | |
1556190531 | https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556190531 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | IC_kwDOCGYnMM5cwZVD | simonw 9599 | 2023-05-21T14:13:43Z | 2023-05-21T14:13:43Z | OWNER | OK, this works! ![trogon](https://github.com/simonw/sqlite-utils/assets/9599/2ae194c5-ec82-471a-9d1b-b01b3f2632f3) To try it out, install that branch from GitHub: pip install https://github.com/simonw/sqlite-utils/archive/refs/heads/trogon.zip Then run this: sqlite-utils install trogon And this: sqlite-utils tui | {"total_count": 5, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 3, "eyes": 0} | Try out Trogon for a tui interface 1718517882 | |
1556189823 | https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556189823 | https://api.github.com/repos/simonw/sqlite-utils/issues/545 | IC_kwDOCGYnMM5cwZJ_ | simonw 9599 | 2023-05-21T14:09:59Z | 2023-05-21T14:09:59Z | OWNER | I don't want to add `trogon` as a default dependency because it's a little heavy - it pulls in all of Rich and Textual as well. People who use `sqlite-utils` just for its Python API won't benefit from this - it's a CLI feature only. But I have a `sqlite-utils install ...` command for helping people to install packages into the same virtual environment as `sqlite-utils` no matter how they installed that tool: https://sqlite-utils.datasette.io/en/stable/cli.html#cli-install So I can treat Trogon as an optional dependency and add the `sqlite-utils tui` command only if that package is also installed. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Try out Trogon for a tui interface 1718517882 | |
1546119773 | https://github.com/simonw/datasette/issues/2073#issuecomment-1546119773 | https://api.github.com/repos/simonw/datasette/issues/2073 | IC_kwDOBm6k_c5cJ-pd | simonw 9599 | 2023-05-12T18:24:07Z | 2023-05-12T18:24:07Z | OWNER | Here's a demo of this breaking in Datasette Lite: https://lite.datasette.io/?sql=https://gist.github.com/simonw/261564c0ca01567df6eeb9b222b8be84&json=https%3A%2F%2Fcdn.jsdelivr.net%2Fnpm%2Fweb-features%2Findex.json#/data/baseline?_filter_column_1=is_baseline&_filter_op_1=exact&_filter_value_1=1&_filter_column_2=&_filter_op_2=notnull__1&_filter_value_2=1&_filter_column=&_filter_op=exact&_filter_value=&_sort=&_facet=is_baseline <img width="609" alt="image" src="https://github.com/simonw/datasette/assets/9599/3c23ae6e-7fb0-41d4-92b8-421c400a731a"> Here's a SQL query that demonstrates the underlying issue: ```sql select 'working', count(*) from baseline where is_baseline = 1 union all select 'broken', count(*) from baseline where is_baseline = '1' ``` https://lite.datasette.io/?sql=https://gist.github.com/simonw/261564c0ca01567df6eeb9b222b8be84&json=https%3A%2F%2Fcdn.jsdelivr.net%2Fnpm%2Fweb-features%2Findex.json#/data?sql=select+%27working%27%2C+count%28*%29+from+baseline+where+is_baseline+%3D+1%0Aunion+all%0Aselect+%27broken%27%2C+count%28*%29+from+baseline+where+is_baseline+%3D+%271%27 <img width="673" alt="image" src="https://github.com/simonw/datasette/assets/9599/867914f8-8cef-432c-a518-2739d1caf964"> | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Faceting doesn't work against integer columns in views 1708030220 | |
1546117538 | https://github.com/simonw/datasette/issues/2073#issuecomment-1546117538 | https://api.github.com/repos/simonw/datasette/issues/2073 | IC_kwDOBm6k_c5cJ-Gi | simonw 9599 | 2023-05-12T18:21:38Z | 2023-05-12T18:21:38Z | OWNER | https://latest.datasette.io/fixtures doesn't currently have a view with any integer columns in it, making this bug harder to demonstrate there. I can't replicate the bug using https://datasette.io/content/plugins?_facet=stargazers_count&stargazers_count=3 - I would expect that not to work correctly. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Faceting doesn't work against integer columns in views 1708030220 | |
1540494121 | https://github.com/simonw/datasette/issues/2070#issuecomment-1540494121 | https://api.github.com/repos/simonw/datasette/issues/2070 | IC_kwDOBm6k_c5b0hMp | simonw 9599 | 2023-05-09T16:25:00Z | 2023-05-09T16:25:00Z | OWNER | Can now be used here: https://github.com/simonw/datasette/actions/workflows/deploy-branch-preview.yml | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Mechanism for deploying a preview of a branch using Vercel 1702354223 | |
1540491751 | https://github.com/simonw/datasette/issues/2070#issuecomment-1540491751 | https://api.github.com/repos/simonw/datasette/issues/2070 | IC_kwDOBm6k_c5b0gnn | simonw 9599 | 2023-05-09T16:23:12Z | 2023-05-09T16:23:12Z | OWNER | Added a actions `BRANCH_PREVIEW_VERCEL_TOKEN` secret to this repository. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Mechanism for deploying a preview of a branch using Vercel 1702354223 | |
1539157643 | https://github.com/simonw/sqlite-utils/pull/537#issuecomment-1539157643 | https://api.github.com/repos/simonw/sqlite-utils/issues/537 | IC_kwDOCGYnMM5bva6L | simonw 9599 | 2023-05-08T22:45:09Z | 2023-05-08T22:45:21Z | OWNER | Here's an example from the new tests: https://github.com/simonw/sqlite-utils/blob/a75abeb61b91a28650d3b9933e7ec80ad0d92529/tests/test_create.py#L291-L307 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Support self-referencing FKs in `Table.create` 1665200812 | |
1539109816 | https://github.com/simonw/sqlite-utils/issues/448#issuecomment-1539109816 | https://api.github.com/repos/simonw/sqlite-utils/issues/448 | IC_kwDOCGYnMM5bvPO4 | simonw 9599 | 2023-05-08T22:01:00Z | 2023-05-08T22:01:00Z | OWNER | This is being handled in: - #520 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Reading rows from a file => AttributeError: '_io.StringIO' object has no attribute 'readinto' 1279144769 | |
1539109587 | https://github.com/simonw/sqlite-utils/issues/520#issuecomment-1539109587 | https://api.github.com/repos/simonw/sqlite-utils/issues/520 | IC_kwDOCGYnMM5bvPLT | simonw 9599 | 2023-05-08T22:00:46Z | 2023-05-08T22:00:46Z | OWNER | > Hey, isn't this essentially the same issue as #448 ? Yes it is, good catch! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | rows_from_file() raises confusing error if file-like object is not in binary mode 1516644980 | |
1539108140 | https://github.com/simonw/sqlite-utils/issues/525#issuecomment-1539108140 | https://api.github.com/repos/simonw/sqlite-utils/issues/525 | IC_kwDOCGYnMM5bvO0s | simonw 9599 | 2023-05-08T21:59:41Z | 2023-05-08T21:59:41Z | OWNER | That original example passes against `main` now. | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Repeated calls to `Table.convert()` fail 1575131737 | |
1539101853 | https://github.com/simonw/sqlite-utils/issues/525#issuecomment-1539101853 | https://api.github.com/repos/simonw/sqlite-utils/issues/525 | IC_kwDOCGYnMM5bvNSd | simonw 9599 | 2023-05-08T21:52:44Z | 2023-05-08T21:52:44Z | OWNER | I like the `lambda-{uuid}` idea. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Repeated calls to `Table.convert()` fail 1575131737 | |
1539100300 | https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539100300 | https://api.github.com/repos/simonw/sqlite-utils/issues/514 | IC_kwDOCGYnMM5bvM6M | simonw 9599 | 2023-05-08T21:50:51Z | 2023-05-08T21:50:51Z | OWNER | Seeing as `sqlite-utils` doesn't currently provide mechanisms for adding `check` constraints like this I'm going to leave this - I'm happy with the fix I put in for the `not null` constraints. | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | upsert of new row with check constraints fails 1465194249 | |
1539099703 | https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539099703 | https://api.github.com/repos/simonw/sqlite-utils/issues/514 | IC_kwDOCGYnMM5bvMw3 | simonw 9599 | 2023-05-08T21:50:06Z | 2023-05-08T21:50:06Z | OWNER | Applying the fix from the PR here doesn't fix the above problem either: - https://github.com/simonw/sqlite-utils/pull/515 So it looks like these kinds of `check` constraints currently aren't compatible with how `upsert()` works. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | upsert of new row with check constraints fails 1465194249 | |
1539094287 | https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539094287 | https://api.github.com/repos/simonw/sqlite-utils/issues/514 | IC_kwDOCGYnMM5bvLcP | simonw 9599 | 2023-05-08T21:44:11Z | 2023-05-08T21:44:11Z | OWNER | OK, this fails silently: ```python import sqlite_utils db = sqlite_utils.Database(memory=True) db.execute('''CREATE TABLE employees ( id INTEGER PRIMARY KEY, name TEXT, age INTEGER, salary REAL, CHECK (salary is not null and salary > 0) );''') db["employees"].upsert({"id": 1, "name": "Bob"}, pk="id") list(db["employees"].rows) ```` It outputs: ```python [] ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | upsert of new row with check constraints fails 1465194249 | |
1539079507 | https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539079507 | https://api.github.com/repos/simonw/sqlite-utils/issues/514 | IC_kwDOCGYnMM5bvH1T | simonw 9599 | 2023-05-08T21:28:37Z | 2023-05-08T21:28:37Z | OWNER | > This means that a table with NON NULL (or other constraint) columns that aren't part of the pkey can't have new rows upserted. Huh... on that basis, it's possible my fix in https://github.com/simonw/sqlite-utils/commit/2376c452a56b0c3e75e7ca698273434e32945304 is incomplete. I only covered the 'not null' case. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | upsert of new row with check constraints fails 1465194249 | |
1539078429 | https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539078429 | https://api.github.com/repos/simonw/sqlite-utils/issues/514 | IC_kwDOCGYnMM5bvHkd | simonw 9599 | 2023-05-08T21:27:40Z | 2023-05-08T21:27:40Z | OWNER | Dupe of: - #538 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | upsert of new row with check constraints fails 1465194249 | |
1539077777 | https://github.com/simonw/sqlite-utils/pull/515#issuecomment-1539077777 | https://api.github.com/repos/simonw/sqlite-utils/issues/515 | IC_kwDOCGYnMM5bvHaR | simonw 9599 | 2023-05-08T21:27:10Z | 2023-05-08T21:27:10Z | OWNER | I should have spotted this PR before I shipped my own fix! https://github.com/simonw/sqlite-utils/commit/2376c452a56b0c3e75e7ca698273434e32945304 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | upsert new rows with constraints, fixes #514 1465194930 | |
1539058795 | https://github.com/simonw/sqlite-utils/pull/519#issuecomment-1539058795 | https://api.github.com/repos/simonw/sqlite-utils/issues/519 | IC_kwDOCGYnMM5bvCxr | simonw 9599 | 2023-05-08T21:12:52Z | 2023-05-08T21:12:52Z | OWNER | This is a really neat fix, thank you. | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0} | Fixes breaking DEFAULT values 1505568103 | |
1539055393 | https://github.com/simonw/sqlite-utils/pull/537#issuecomment-1539055393 | https://api.github.com/repos/simonw/sqlite-utils/issues/537 | IC_kwDOCGYnMM5bvB8h | simonw 9599 | 2023-05-08T21:10:06Z | 2023-05-08T21:10:06Z | OWNER | Thanks! | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Support self-referencing FKs in `Table.create` 1665200812 | |
1539053230 | https://github.com/simonw/sqlite-utils/pull/528#issuecomment-1539053230 | https://api.github.com/repos/simonw/sqlite-utils/issues/528 | IC_kwDOCGYnMM5bvBau | simonw 9599 | 2023-05-08T21:08:23Z | 2023-05-08T21:08:23Z | OWNER | I fixed this in: - #527 Will fully remove this misfeature in: - #542 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Enable `Table.convert()` on falsey values 1578793661 | |
1539052467 | https://github.com/simonw/sqlite-utils/issues/542#issuecomment-1539052467 | https://api.github.com/repos/simonw/sqlite-utils/issues/542 | IC_kwDOCGYnMM5bvBOz | simonw 9599 | 2023-05-08T21:07:41Z | 2023-05-08T21:07:41Z | OWNER | Relevant commits (will mostly revert these): - https://github.com/simonw/sqlite-utils/commit/455c35b512895c19bf922c2b804d750d27cb8dbd - https://github.com/simonw/sqlite-utils/commit/e0ec4c345129996011951e400388fd74865f65a2 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Remove `skip_false=True` and `--no-skip-false` in `sqlite-utils` 4.0 1700936245 | |
1539051724 | https://github.com/simonw/sqlite-utils/issues/527#issuecomment-1539051724 | https://api.github.com/repos/simonw/sqlite-utils/issues/527 | IC_kwDOCGYnMM5bvBDM | simonw 9599 | 2023-05-08T21:07:04Z | 2023-05-08T21:07:04Z | OWNER | Updated documentation: - https://sqlite-utils.datasette.io/en/latest/python-api.html#converting-data-in-columns - https://sqlite-utils.datasette.io/en/latest/cli.html#converting-data-in-columns - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#convert | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `Table.convert()` skips falsey values 1578790070 | |
1539035838 | https://github.com/simonw/sqlite-utils/issues/527#issuecomment-1539035838 | https://api.github.com/repos/simonw/sqlite-utils/issues/527 | IC_kwDOCGYnMM5bu9K- | simonw 9599 | 2023-05-08T20:55:00Z | 2023-05-08T20:55:00Z | OWNER | I'm going to go with `--no-skip-false` as the CLI option. It's ugly, but this whole thing is ugly. I'm going to make a note to remove this misfeature in `sqlite-utils` 4. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `Table.convert()` skips falsey values 1578790070 | |
1539033736 | https://github.com/simonw/sqlite-utils/issues/527#issuecomment-1539033736 | https://api.github.com/repos/simonw/sqlite-utils/issues/527 | IC_kwDOCGYnMM5bu8qI | simonw 9599 | 2023-05-08T20:52:51Z | 2023-05-08T20:52:51Z | OWNER | OK, I implemented that at the Python API level. I need to decide how it should work for the `sqlite-utils convert` command too: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#convert | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `Table.convert()` skips falsey values 1578790070 | |
1539018912 | https://github.com/simonw/sqlite-utils/issues/530#issuecomment-1539018912 | https://api.github.com/repos/simonw/sqlite-utils/issues/530 | IC_kwDOCGYnMM5bu5Cg | simonw 9599 | 2023-05-08T20:39:00Z | 2023-05-08T20:39:00Z | OWNER | I think the natural place to add these in the Python library API would be https://sqlite-utils.datasette.io/en/stable/python-api.html#adding-foreign-key-constraints - maybe something like this: ```python db["books"].add_foreign_key("author_id", "authors", "id", on_delete=RESTRICT) ``` Then for the CLI tool could be added to https://sqlite-utils.datasette.io/en/stable/cli-reference.html#add-foreign-key ``` sqlite-utils add-foreign-key my.db books author_id authors id --on-update SET_NULL ``` I wouldn't support these for the other methods of adding foreign keys - `foreign_keys(...)` for the various `.insert()` etc methods and the `add_foreign_keys(...)` bulk menthod. | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 1, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | add ability to configure "on delete" and "on update" attributes of foreign keys: 1595340692 | |
1539015064 | https://github.com/simonw/sqlite-utils/issues/530#issuecomment-1539015064 | https://api.github.com/repos/simonw/sqlite-utils/issues/530 | IC_kwDOCGYnMM5bu4GY | simonw 9599 | 2023-05-08T20:35:07Z | 2023-05-08T20:35:07Z | OWNER | Wow, this is a neat feature I didn't know about. Looks like there are a bunch of options: - NO ACTION (default) - RESTRICT: application is prohibited from deleting a parent key when there exists one or more child keys mapped to it - SET NULL: when a parent key is deleted the child key columns of all rows in the child table that mapped to the parent key are set to contain SQL NULL values - SET DEFAULT: set a specific default - CASCADE: propagates the delete or update operation on the parent key to each dependent child key | {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | add ability to configure "on delete" and "on update" attributes of foreign keys: 1595340692 | |
1539009453 | https://github.com/simonw/sqlite-utils/issues/532#issuecomment-1539009453 | https://api.github.com/repos/simonw/sqlite-utils/issues/532 | IC_kwDOCGYnMM5bu2ut | simonw 9599 | 2023-05-08T20:30:29Z | 2023-05-08T20:30:42Z | OWNER | Here's an improvement: ``` % sqlite-utils insert /tmp/b.db blah /tmp/blah.txt [####################################] 100% Error: Invalid JSON - use --csv for CSV or --tsv for TSV files JSON error: Expecting value: line 1 column 1 (char 0) ``` | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Show more information when JSON can't be imported with sqlite-utils insert 1620254998 | |
1539006509 | https://github.com/simonw/sqlite-utils/issues/532#issuecomment-1539006509 | https://api.github.com/repos/simonw/sqlite-utils/issues/532 | IC_kwDOCGYnMM5bu2At | simonw 9599 | 2023-05-08T20:28:56Z | 2023-05-08T20:28:56Z | OWNER | Was this a newline-delimited JSON file perhaps? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Show more information when JSON can't be imported with sqlite-utils insert 1620254998 | |
1538963959 | https://github.com/simonw/sqlite-utils/issues/541#issuecomment-1538963959 | https://api.github.com/repos/simonw/sqlite-utils/issues/541 | IC_kwDOCGYnMM5burn3 | simonw 9599 | 2023-05-08T19:59:34Z | 2023-05-08T19:59:34Z | OWNER | There are 8 failing tests left: ``` ==== short test summary info ==== FAILED tests/test_cli_memory.py::test_memory_csv[False-test] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_csv[False-t] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_csv[False-t1] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_tsv[False] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_dump[extra_args0] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_cli_memory.py::test_memory_two_files_with_same_stem - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> FAILED tests/test_recipes.py::test_dateparse_errors[None-parsedate] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function Table.convert.<locals>.convert_value at 0x106bcaca0> FAILED tests/test_recipes.py::test_dateparse_errors[None-parsedatetime] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function Table.convert.<locals>.convert_value at 0x106bc9620> ERROR tests/test_cli.py::test_upsert_analyze - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> ==== 8 failed, 894 passed, 4 skipped, 1 error in 6.27s ==== ``` Full traceback here: https://gist.github.com/simonw/b40b3e814729d6c02a0302a84ce54d9e | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Get tests to pass with `pytest -Werror` 1700840265 | |
1538933540 | https://github.com/simonw/sqlite-utils/issues/534#issuecomment-1538933540 | https://api.github.com/repos/simonw/sqlite-utils/issues/534 | IC_kwDOCGYnMM5bukMk | simonw 9599 | 2023-05-08T19:34:37Z | 2023-05-08T19:34:37Z | OWNER | On macOS this shows the same warning: ``` % python -Wdefault $(which sqlite-utils) insert dogs.db dogs dogs.csv --csv [############------------------------] 35% [####################################] 100%/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py:1187: ResourceWarning: unclosed file <_io.TextIOWrapper name='dogs.csv' encoding='utf-8-sig'> insert_upsert_implementation( ResourceWarning: Enable tracemalloc to get the object allocation traceback ``` The file itself is a `click.File` which is automatically closed - https://click.palletsprojects.com/en/8.1.x/api/#click.File - but it looks like it's the `_io.TextIOWrapper` which is not being closed: https://github.com/simonw/sqlite-utils/blob/2376c452a56b0c3e75e7ca698273434e32945304/sqlite_utils/cli.py#L949-L956 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ResourceWarning: unclosed file 1622640374 | |
1538921774 | https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538921774 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | IC_kwDOCGYnMM5buhUu | simonw 9599 | 2023-05-08T19:24:41Z | 2023-05-08T19:24:41Z | OWNER | That fix seems to work! | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `table.upsert_all` fails to write rows when `not_null` is present 1695428235 | |
1538910894 | https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538910894 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | IC_kwDOCGYnMM5buequ | simonw 9599 | 2023-05-08T19:16:52Z | 2023-05-08T19:17:00Z | OWNER | How about if I had logic which checked that all not-null columns were provided in the call to `upsert_all()` - and if they were, modified the `INSERT OR IGNORE INTO` to include a placeholder value for those columns that would then be fixed by the later `UPDATE`? Something like this: ```python [ ('INSERT OR IGNORE INTO [comments]([id], name) VALUES(?, ?);', [1, '']), ('UPDATE [comments] SET [name] = ? WHERE [id] = ?', ['Cleo', 1]) ] ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `table.upsert_all` fails to write rows when `not_null` is present 1695428235 | |
1538903556 | https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538903556 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | IC_kwDOCGYnMM5buc4E | simonw 9599 | 2023-05-08T19:11:24Z | 2023-05-08T19:13:23Z | OWNER | I could detect if this happens using `cursor.rowcount` - not sure how I would recover from it though. This would also require some major re-engineering, since currently it all works by generating a list of SQL queries in advance and applying them inside a loop in `.insert_chunk()`: https://github.com/simonw/sqlite-utils/blob/80763edaa2bdaf1113717378b8d62075c4dcbcfb/sqlite_utils/db.py#L2839-L2878 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `table.upsert_all` fails to write rows when `not_null` is present 1695428235 | |
1538893329 | https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538893329 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | IC_kwDOCGYnMM5buaYR | simonw 9599 | 2023-05-08T19:04:47Z | 2023-05-08T19:04:47Z | OWNER | This feels like a fundamental flaw in the way upserts are implemented by `sqlite-utils`. One fix would be to switch to using the `UPSERT` feature in SQLite: https://www.sqlite.org/lang_UPSERT.html But... > UPSERT syntax was added to SQLite with version 3.24.0 (2018-06-04). I still want to support SQLite versions earlier than that. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `table.upsert_all` fails to write rows when `not_null` is present 1695428235 | |
1538889482 | https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538889482 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | IC_kwDOCGYnMM5buZcK | simonw 9599 | 2023-05-08T19:02:38Z | 2023-05-08T19:02:38Z | OWNER | Here's the code at fault: https://github.com/simonw/sqlite-utils/blob/80763edaa2bdaf1113717378b8d62075c4dcbcfb/sqlite_utils/db.py#L2774-L2788 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `table.upsert_all` fails to write rows when `not_null` is present 1695428235 | |
1538887361 | https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538887361 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | IC_kwDOCGYnMM5buY7B | simonw 9599 | 2023-05-08T19:01:20Z | 2023-05-08T19:01:20Z | OWNER | Here's the problem: ```python import sqlite3 db = sqlite3.connect(":memory:") db.execute('create table foo (id integer primary key, name not null)') db.execute('insert into foo (id) values (1)') ``` Produces: ``` IntegrityError: NOT NULL constraint failed: foo.name ``` But this: ```python db.execute('insert or ignore into foo (id) values (1)') ``` Completes without an exception. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `table.upsert_all` fails to write rows when `not_null` is present 1695428235 | |
1538801855 | https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538801855 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | IC_kwDOCGYnMM5buEC_ | simonw 9599 | 2023-05-08T18:00:17Z | 2023-05-08T18:00:17Z | OWNER | From time in the debugger, after creating the table it ends up doing this: ``` (Pdb) queries_and_params [ ('INSERT OR IGNORE INTO [comments]([id]) VALUES(?);', [1]), ('UPDATE [comments] SET [name] = ? WHERE [id] = ?', ['Cleo', 1]) ] ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `table.upsert_all` fails to write rows when `not_null` is present 1695428235 | |
1538793817 | https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538793817 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | IC_kwDOCGYnMM5buCFZ | simonw 9599 | 2023-05-08T17:55:10Z | 2023-05-08T17:55:10Z | OWNER | Confirmed - I added this test and it fails: ```python def test_upsert_all_not_null(fresh_db): # https://github.com/simonw/sqlite-utils/issues/538 fresh_db["comments"].upsert_all( [{"id": 1, "name": "Cleo"}], pk="id", not_null=["name"], ) assert list(fresh_db["comments"].rows) == [{"id": 1, "name": "Cleo"}] ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `table.upsert_all` fails to write rows when `not_null` is present 1695428235 | |
1537514610 | https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537514610 | https://api.github.com/repos/simonw/sqlite-utils/issues/539 | IC_kwDOCGYnMM5bpJxy | simonw 9599 | 2023-05-07T18:43:24Z | 2023-05-07T18:43:24Z | OWNER | Documentation: - https://sqlite-utils.datasette.io/en/latest/cli.html#returning-raw-data-such-as-binary-content - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#query - https://sqlite-utils.datasette.io/en/latest/cli-reference.html#memory | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `--raw-lines` option, like `--raw` for multiple lines 1699174055 | |
1537514069 | https://github.com/simonw/sqlite-utils/issues/540#issuecomment-1537514069 | https://api.github.com/repos/simonw/sqlite-utils/issues/540 | IC_kwDOCGYnMM5bpJpV | simonw 9599 | 2023-05-07T18:40:18Z | 2023-05-07T18:40:18Z | OWNER | https://docs.readthedocs.io/en/stable/config-file/v2.html suggests: ```yaml build: os: ubuntu-22.04 tools: python: "3.11" ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sphinx.builders.linkcheck build error 1699184583 | |
1537513912 | https://github.com/simonw/sqlite-utils/issues/540#issuecomment-1537513912 | https://api.github.com/repos/simonw/sqlite-utils/issues/540 | IC_kwDOCGYnMM5bpJm4 | simonw 9599 | 2023-05-07T18:39:29Z | 2023-05-07T18:39:29Z | OWNER | https://readthedocs.org/projects/sqlite-utils/builds/20513034/ said: > Problem in your project's configuration. Invalid "python.version": expected one of (2, 2.7, 3, 3.5, 3.6, 3.7, 3.8, pypy3.5), got 3.11 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sphinx.builders.linkcheck build error 1699184583 | |
1537513653 | https://github.com/simonw/sqlite-utils/issues/540#issuecomment-1537513653 | https://api.github.com/repos/simonw/sqlite-utils/issues/540 | IC_kwDOCGYnMM5bpJi1 | simonw 9599 | 2023-05-07T18:37:59Z | 2023-05-07T18:38:19Z | OWNER | Useful comment here: - https://github.com/urllib3/urllib3/issues/2168#issuecomment-1537360928 > I faced the same issue. I switched to a different Python kernel (3.11.2) and it worked. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sphinx.builders.linkcheck build error 1699184583 | |
1537507676 | https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537507676 | https://api.github.com/repos/simonw/sqlite-utils/issues/539 | IC_kwDOCGYnMM5bpIFc | simonw 9599 | 2023-05-07T18:09:43Z | 2023-05-07T18:09:54Z | OWNER | This worked: ```bash sqlite-utils memory /tmp/books3.json:nl \ 'select name from books3' --raw-lines > titles.txt ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `--raw-lines` option, like `--raw` for multiple lines 1699174055 | |
1537507525 | https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537507525 | https://api.github.com/repos/simonw/sqlite-utils/issues/539 | IC_kwDOCGYnMM5bpIDF | simonw 9599 | 2023-05-07T18:09:09Z | 2023-05-07T18:09:09Z | OWNER | I'm tempted to upgrade `--raw` to do this instead, but that would be a breaking change. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `--raw-lines` option, like `--raw` for multiple lines 1699174055 | |
1537507394 | https://github.com/simonw/sqlite-utils/issues/539#issuecomment-1537507394 | https://api.github.com/repos/simonw/sqlite-utils/issues/539 | IC_kwDOCGYnMM5bpIBC | simonw 9599 | 2023-05-07T18:08:44Z | 2023-05-07T18:08:44Z | OWNER | Prototype: ```diff diff --git a/docs/cli-reference.rst b/docs/cli-reference.rst index 153e5f9..c830518 100644 --- a/docs/cli-reference.rst +++ b/docs/cli-reference.rst @@ -124,6 +124,7 @@ See :ref:`cli_query`. --json-cols Detect JSON cols and output them as JSON, not escaped strings -r, --raw Raw output, first column of first row + --raw-lines Raw output, first column of each row -p, --param <TEXT TEXT>... Named :parameters for SQL query --functions TEXT Python code defining one or more custom SQL functions @@ -192,6 +193,7 @@ See :ref:`cli_memory`. --json-cols Detect JSON cols and output them as JSON, not escaped strings -r, --raw Raw output, first column of first row + --raw-lines Raw output, first column of each row -p, --param <TEXT TEXT>... Named :parameters for SQL query --encoding TEXT Character encoding for CSV input, defaults to utf-8 diff --git a/sqlite_utils/cli.py b/sqlite_utils/cli.py index d25b1df..da0e4b6 100644 --- a/sqlite_utils/cli.py +++ b/sqlite_utils/cli.py @@ -1653,6 +1653,7 @@ def drop_view(path, view, ignore, load_extension): ) @output_options @click.option("-r", "--raw", is_flag=True, help="Raw output, first column of first row") +@click.option("--raw-lines", is_flag=True, help="Raw output, first column of each row") @click.option( "-p", "--param", @@ -1677,6 +1678,7 @@ def query( fmt, json_cols, raw, + raw_lines, param, load_extension, functions, @@ -1700,7 +1702,19 @@ def query( _register_functions(db, functions) _execute_query( - db, sql, param, raw, table, csv, tsv, no_headers, fmt, nl, arrays, json_cols + db, + sql, + par… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `--raw-lines` option, like `--raw` for multiple lines 1699174055 | |
1524709988 | https://github.com/simonw/datasette/issues/2065#issuecomment-1524709988 | https://api.github.com/repos/simonw/datasette/issues/2065 | IC_kwDOBm6k_c5a4Tpk | simonw 9599 | 2023-04-27T05:09:36Z | 2023-04-27T05:09:36Z | OWNER | That fixed it - after installing `main.zip` again I ran this and it worked: ~/.rye/tools/main-zip/bin/datasette install datasette-graphql ``` % ~/.rye/tools/main-zip/bin/datasette plugins [ { "name": "datasette-graphql", "static": true, "templates": true, "version": "2.2", "hooks": [ "database_actions", "extra_template_vars", "menu_links", "register_routes", "startup", "table_actions" ] } ] ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Datasette cannot be installed with Rye 1686033652 | |
1524707628 | https://github.com/simonw/datasette/issues/2065#issuecomment-1524707628 | https://api.github.com/repos/simonw/datasette/issues/2065 | IC_kwDOBm6k_c5a4TEs | simonw 9599 | 2023-04-27T05:06:44Z | 2023-04-27T05:06:44Z | OWNER | I need `pip` as a dependency too: ``` % ~/.rye/tools/main-zip/bin/datasette install datasette-graphql Traceback (most recent call last): File "/Users/simon/.rye/tools/main-zip/bin/datasette", line 8, in <module> sys.exit(cli()) ^^^^^ File "/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/datasette/cli.py", line 365, in install run_module("pip", run_name="__main__") File "<frozen runpy>", line 222, in run_module File "<frozen runpy>", line 142, in _get_module_details ImportError: No module named pip ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Datasette cannot be installed with Rye 1686033652 | |
1524699863 | https://github.com/simonw/datasette/issues/2065#issuecomment-1524699863 | https://api.github.com/repos/simonw/datasette/issues/2065 | IC_kwDOBm6k_c5a4RLX | simonw 9599 | 2023-04-27T04:56:22Z | 2023-04-27T04:56:22Z | OWNER | Turned this into a TIL: https://til.simonwillison.net/python/rye | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Datasette cannot be installed with Rye 1686033652 | |
1524680160 | https://github.com/simonw/datasette/issues/2066#issuecomment-1524680160 | https://api.github.com/repos/simonw/datasette/issues/2066 | IC_kwDOBm6k_c5a4MXg | simonw 9599 | 2023-04-27T04:27:50Z | 2023-04-27T04:27:50Z | OWNER | That fixed it. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Failing test: httpx.InvalidURL: URL too long 1686042269 | |
1524675817 | https://github.com/simonw/datasette/issues/2066#issuecomment-1524675817 | https://api.github.com/repos/simonw/datasette/issues/2066 | IC_kwDOBm6k_c5a4LTp | simonw 9599 | 2023-04-27T04:21:03Z | 2023-04-27T04:21:03Z | OWNER | I went with this to generate the long string: https://github.com/simonw/datasette/blob/0b0c5cd7a94fe3f151a3e10261b5c84ee64f2f18/tests/test_csv.py#L157-L176 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Failing test: httpx.InvalidURL: URL too long 1686042269 | |
1524669124 | https://github.com/simonw/datasette/issues/2066#issuecomment-1524669124 | https://api.github.com/repos/simonw/datasette/issues/2066 | IC_kwDOBm6k_c5a4JrE | simonw 9599 | 2023-04-27T04:10:44Z | 2023-04-27T04:10:52Z | OWNER | I need an alternative way of generating a long string with a shorter URL. ```sql select group_concat('abcabcabc', '') from json_each(json_array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)) ``` https://latest.datasette.io/_memory?sql=select+group_concat%28%27abcabcabc%27%2C+%27%27%29+from+json_each%28json_array%281%2C+2%2C+3%2C+4%2C+5%2C+6%2C+7%2C+8%2C+9%2C+10%29%29 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Failing test: httpx.InvalidURL: URL too long 1686042269 | |
1524666049 | https://github.com/simonw/datasette/issues/2066#issuecomment-1524666049 | https://api.github.com/repos/simonw/datasette/issues/2066 | IC_kwDOBm6k_c5a4I7B | simonw 9599 | 2023-04-27T04:06:18Z | 2023-04-27T04:06:18Z | OWNER | Most recent `httpx` release is 0.24 a couple of weeks ago. Here's what changed in that version: https://github.com/encode/httpx/compare/0.23.3...0.24.0 It looks like that URL limit is new: https://github.com/encode/httpx/commit/57daabf673705954afa94686c0002801c93d31f3#diff-78d8d93b5dd4c77d99c3e2b46b7286ba71a8fd60e92d8bd4eee5fb200b4f87bfR149-R155 ```python def urlparse(url: str = "", **kwargs: typing.Optional[str]) -> ParseResult: # Initial basic checks on allowable URLs. # --------------------------------------- # Hard limit the maximum allowable URL length. if len(url) > MAX_URL_LENGTH: raise InvalidURL("URL too long") ``` https://github.com/encode/httpx/blob/32e25497a36e6222cc64a758c98275b450dac28d/httpx/_urlparse.py#L153-L155 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Failing test: httpx.InvalidURL: URL too long 1686042269 | |
1524660603 | https://github.com/simonw/datasette/issues/2066#issuecomment-1524660603 | https://api.github.com/repos/simonw/datasette/issues/2066 | IC_kwDOBm6k_c5a4Hl7 | simonw 9599 | 2023-04-27T04:02:55Z | 2023-04-27T04:02:55Z | OWNER | In the debugger: ``` (Pdb) MAX_URL_LENGTH 65536 ``` Weird this only seems to be a problem with `httpx` on Python 3.7 though. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Failing test: httpx.InvalidURL: URL too long 1686042269 | |
1524659084 | https://github.com/simonw/datasette/issues/2066#issuecomment-1524659084 | https://api.github.com/repos/simonw/datasette/issues/2066 | IC_kwDOBm6k_c5a4HOM | simonw 9599 | 2023-04-27T04:02:07Z | 2023-04-27T04:02:07Z | OWNER | This is the failing test: https://github.com/simonw/datasette/blob/249fcf8e3e2a90e763f41b080c1b9ec8017f5005/tests/test_csv.py#L156-L167 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Failing test: httpx.InvalidURL: URL too long 1686042269 | |
1524655203 | https://github.com/simonw/datasette/issues/2066#issuecomment-1524655203 | https://api.github.com/repos/simonw/datasette/issues/2066 | IC_kwDOBm6k_c5a4GRj | simonw 9599 | 2023-04-27T03:59:56Z | 2023-04-27T03:59:56Z | OWNER | In a fresh Datasette checkout I ran: pipenv shell --python /Users/simon/.pyenv/versions/3.7.16/bin/python That gave me a virtual environment with 3.7.16 Python. Then I ran: pip install -e '.[test]' Weirdly that gave me a `which pytest` of `/opt/homebrew/bin/pytest` which ran the tests on 3.11. I figured out the location of the virtual environment with `which python` and then ran this: ``` % /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/bin/pytest tests/test_csv.py ============================================================================================== test session starts =============================================================================================== platform darwin -- Python 3.7.16, pytest-7.3.1, pluggy-1.0.0 SQLite: 3.39.5 rootdir: /private/tmp/datasette configfile: pytest.ini plugins: asyncio-0.21.0, timeout-2.1.0, xdist-3.2.1, anyio-3.6.2 asyncio: mode=strict collected 15 items tests/test_csv.py ..........F.... [100%] ==================================================================================================== FAILURES ==================================================================================================== ________________________________________________________________________________________________ test_max_csv_mb _________________________________________________________________________________________________ app_client_csv_max_mb_one = <datasette.utils.testing.TestClient object at 0x107ab5c08> def test_max_csv_mb(app_client_csv_max_mb_one): response = app_client_csv_max_mb_one.get( ( "/fixtures.csv?sql=selec… | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Failing test: httpx.InvalidURL: URL too long 1686042269 | |
1524648995 | https://github.com/simonw/datasette/issues/2066#issuecomment-1524648995 | https://api.github.com/repos/simonw/datasette/issues/2066 | IC_kwDOBm6k_c5a4Ewj | simonw 9599 | 2023-04-27T03:56:42Z | 2023-04-27T03:57:11Z | OWNER | I don't have 3.7 locally. Trying to install it with `pyenv`. brew install pyenv Then: pyenv install --list | grep 3.7 Installing: pyenv install 3.7.16 Output: Installed Python-3.7.16 to /Users/simon/.pyenv/versions/3.7.16 So the executable is `/Users/simon/.pyenv/versions/3.7.16/bin/python`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Failing test: httpx.InvalidURL: URL too long 1686042269 | |
1524638233 | https://github.com/simonw/datasette/issues/2066#issuecomment-1524638233 | https://api.github.com/repos/simonw/datasette/issues/2066 | IC_kwDOBm6k_c5a4CIZ | simonw 9599 | 2023-04-27T03:50:51Z | 2023-04-27T03:50:51Z | OWNER | Failure was on 3.7. I'll try that. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Failing test: httpx.InvalidURL: URL too long 1686042269 | |
1524637376 | https://github.com/simonw/datasette/issues/2066#issuecomment-1524637376 | https://api.github.com/repos/simonw/datasette/issues/2066 | IC_kwDOBm6k_c5a4B7A | simonw 9599 | 2023-04-27T03:50:19Z | 2023-04-27T03:50:19Z | OWNER | Having trouble replicating this on my laptop. I tried a fresh virtual environment with fresh packages (in case this is a `httpx` change) but this passed: pytest tests/test_csv.py | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Failing test: httpx.InvalidURL: URL too long 1686042269 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
updated_at (date) >30 ✖