html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/datasette/issues/567#issuecomment-590667545,https://api.github.com/repos/simonw/datasette/issues/567,590667545,MDEyOklzc3VlQ29tbWVudDU5MDY2NzU0NQ==,9599,simonw,2020-02-25T03:40:49Z,2020-02-25T03:40:49Z,OWNER,Closing this in favour of #682 - a mechanism for writing to databases from a clue which is being developed in pull request #683.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",476573875,Datasette Edit, https://github.com/simonw/datasette/pull/683#issuecomment-590679273,https://api.github.com/repos/simonw/datasette/issues/683,590679273,MDEyOklzc3VlQ29tbWVudDU5MDY3OTI3Mw==,9599,simonw,2020-02-25T04:37:21Z,2020-02-25T04:37:21Z,OWNER,I'm happy with this now. I'm going to merge to master.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",570101428,.execute_write() and .execute_write_fn() methods on Database, https://github.com/simonw/datasette/issues/684#issuecomment-590681375,https://api.github.com/repos/simonw/datasette/issues/684,590681375,MDEyOklzc3VlQ29tbWVudDU5MDY4MTM3NQ==,9599,simonw,2020-02-25T04:47:01Z,2020-02-25T04:47:01Z,OWNER,This page here: https://datasette.readthedocs.io/en/latest/internals.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",570301333,Add documentation on Database introspection methods to internals.rst, https://github.com/simonw/datasette/pull/683#issuecomment-590681676,https://api.github.com/repos/simonw/datasette/issues/683,590681676,MDEyOklzc3VlQ29tbWVudDU5MDY4MTY3Ng==,9599,simonw,2020-02-25T04:48:29Z,2020-02-25T04:48:29Z,OWNER,Documentation: https://datasette.readthedocs.io/en/latest/internals.html#await-db-execute-write-sql-params-none-block-false,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",570101428,.execute_write() and .execute_write_fn() methods on Database, https://github.com/simonw/datasette/issues/685#issuecomment-590682210,https://api.github.com/repos/simonw/datasette/issues/685,590682210,MDEyOklzc3VlQ29tbWVudDU5MDY4MjIxMA==,9599,simonw,2020-02-25T04:50:53Z,2020-02-25T04:50:53Z,OWNER,"Current implementations: https://github.com/simonw/datasette/blob/a093c5f79fa034a97d2ad8b606745dd3b80365af/datasette/database.py#L103-L168 At the very least the method name `execute_against_connection_in_thread()` should be updated to something that's more similar to the new (and documented) `.execute_write_fn()` method.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",570309546,Document (and reconsider design of) Database.execute() and Database.execute_against_connection_in_thread(), https://github.com/simonw/datasette/issues/668#issuecomment-590687137,https://api.github.com/repos/simonw/datasette/issues/668,590687137,MDEyOklzc3VlQ29tbWVudDU5MDY4NzEzNw==,9599,simonw,2020-02-25T05:11:35Z,2020-02-25T05:11:35Z,OWNER,I can imitate how `shapefile-to-SQLite` does it: https://github.com/simonw/shapefile-to-sqlite/blob/5e3fb00eb1eee6ae0103a9ce4167a5e5ec17cf9f/shapefile_to_sqlite/utils.py#L105,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",563347679,Make it easier to load SpatiaLite, https://github.com/simonw/datasette/issues/676#issuecomment-590697977,https://api.github.com/repos/simonw/datasette/issues/676,590697977,MDEyOklzc3VlQ29tbWVudDU5MDY5Nzk3Nw==,9599,simonw,2020-02-25T05:57:24Z,2020-02-25T05:57:24Z,OWNER,"OK, this is in master now. You can install master using: pip install https://github.com/simonw/datasette/archive/master.zip Documentation here: https://datasette.readthedocs.io/en/latest/json_api.html#special-table-arguments","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",568091133,?_searchmode=raw option for running FTS searches without escaping characters, https://github.com/simonw/datasette/pull/683#issuecomment-590610180,https://api.github.com/repos/simonw/datasette/issues/683,590610180,MDEyOklzc3VlQ29tbWVudDU5MDYxMDE4MA==,9599,simonw,2020-02-25T00:00:07Z,2020-02-25T00:00:07Z,OWNER,"Basic stuff to cover in unit tests: - Exercise `.execute_write(sql)` - both with block=True and block=False - Exercise `.execute_write_fn(fn)` in the same way - Throw 10 updates in the queue, block on just the last one, check it worked correctly I'm going to write these tests directly against a `Database()` object rather than booting up an entire Datasette instance.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",570101428,.execute_write() and .execute_write_fn() methods on Database, https://github.com/simonw/datasette/pull/683#issuecomment-590614896,https://api.github.com/repos/simonw/datasette/issues/683,590614896,MDEyOklzc3VlQ29tbWVudDU5MDYxNDg5Ng==,9599,simonw,2020-02-25T00:16:51Z,2020-02-25T00:16:51Z,OWNER,"The other problem with the poll-for-UUID-completion idea: how long does this mean Datasette needs to keep holding onto the `WriteTask` objects? Maybe we say you only get to ask ""is this UUID still in the queue"" and if the answer is ""no"" then you assume the task has been completed.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",570101428,.execute_write() and .execute_write_fn() methods on Database, https://github.com/simonw/datasette/pull/683#issuecomment-590617822,https://api.github.com/repos/simonw/datasette/issues/683,590617822,MDEyOklzc3VlQ29tbWVudDU5MDYxNzgyMg==,9599,simonw,2020-02-25T00:26:48Z,2020-02-25T00:26:48Z,OWNER,"This failing test is a nasty one - the whole thing just hangs (so I imagine Travis will run for a while before hopefully giving up). Here's what happens if I add `--full-trace` and then hit Ctrl+C to cancel a test run: ``` $ pytest -k test_execute_write_fn_block_true --full-trace =================================================================== test session starts =================================================================== platform darwin -- Python 3.7.5, pytest-5.2.4, py-1.8.1, pluggy-0.13.1 rootdir: /Users/simonw/Dropbox/Development/datasette, inifile: pytest.ini plugins: asyncio-0.10.0 collected 410 items / 409 deselected / 1 selected tests/test_database.py ^C^C ================================================================= 409 deselected in 4.45s ================================================================= Traceback (most recent call last): File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/main.py"", line 193, in wrap_session session.exitstatus = doit(config, session) or 0 File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/main.py"", line 237, in _main config.hook.pytest_runtestloop(session=session) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/hooks.py"", line 286, in __call__ return self._hookexec(self, self.get_hookimpls(), kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 93, in _hookexec return self._inner_hookexec(hook, methods, kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 87, in firstresult=hook.spec.opts.get(""firstresult"") if hook.spec else False, File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 208, in _multicall return outcome.get_result() File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 80, in get_result raise ex[1].with_traceback(ex[2]) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 187, in _multicall res = hook_impl.function(*args) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/main.py"", line 258, in pytest_runtestloop item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/hooks.py"", line 286, in __call__ return self._hookexec(self, self.get_hookimpls(), kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 93, in _hookexec return self._inner_hookexec(hook, methods, kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 87, in firstresult=hook.spec.opts.get(""firstresult"") if hook.spec else False, File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 208, in _multicall return outcome.get_result() File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 80, in get_result raise ex[1].with_traceback(ex[2]) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 187, in _multicall res = hook_impl.function(*args) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/runner.py"", line 80, in pytest_runtest_protocol runtestprotocol(item, nextitem=nextitem) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/runner.py"", line 95, in runtestprotocol reports.append(call_and_report(item, ""call"", log)) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/runner.py"", line 176, in call_and_report call = call_runtest_hook(item, when, **kwds) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/runner.py"", line 201, in call_runtest_hook lambda: ihook(item=item, **kwds), when=when, reraise=reraise File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/runner.py"", line 229, in from_call result = func() File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/runner.py"", line 201, in lambda: ihook(item=item, **kwds), when=when, reraise=reraise File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/hooks.py"", line 286, in __call__ return self._hookexec(self, self.get_hookimpls(), kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 93, in _hookexec return self._inner_hookexec(hook, methods, kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 87, in firstresult=hook.spec.opts.get(""firstresult"") if hook.spec else False, File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 208, in _multicall return outcome.get_result() File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 80, in get_result raise ex[1].with_traceback(ex[2]) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 187, in _multicall res = hook_impl.function(*args) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/runner.py"", line 125, in pytest_runtest_call item.runtest() File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/python.py"", line 1429, in runtest self.ihook.pytest_pyfunc_call(pyfuncitem=self) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/hooks.py"", line 286, in __call__ return self._hookexec(self, self.get_hookimpls(), kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 93, in _hookexec return self._inner_hookexec(hook, methods, kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 87, in firstresult=hook.spec.opts.get(""firstresult"") if hook.spec else False, File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 208, in _multicall return outcome.get_result() File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 80, in get_result raise ex[1].with_traceback(ex[2]) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 187, in _multicall res = hook_impl.function(*args) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pytest_asyncio/plugin.py"", line 158, in pytest_pyfunc_call pyfuncitem.obj(**testargs), loop=event_loop)) File ""/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py"", line 566, in run_until_complete self.run_forever() File ""/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py"", line 534, in run_forever self._run_once() File ""/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py"", line 1735, in _run_once event_list = self._selector.select(timeout) File ""/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/selectors.py"", line 558, in select kev_list = self._selector.control(None, max_ev, timeout) KeyboardInterrupt During handling of the above exception, another exception occurred: Traceback (most recent call last): File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/bin/pytest"", line 8, in sys.exit(main()) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/config/__init__.py"", line 90, in main return config.hook.pytest_cmdline_main(config=config) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/hooks.py"", line 286, in __call__ return self._hookexec(self, self.get_hookimpls(), kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 93, in _hookexec return self._inner_hookexec(hook, methods, kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 87, in firstresult=hook.spec.opts.get(""firstresult"") if hook.spec else False, File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 208, in _multicall return outcome.get_result() File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 80, in get_result raise ex[1].with_traceback(ex[2]) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 187, in _multicall res = hook_impl.function(*args) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/main.py"", line 230, in pytest_cmdline_main return wrap_session(config, _main) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/main.py"", line 209, in wrap_session config.hook.pytest_keyboard_interrupt(excinfo=excinfo) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/hooks.py"", line 286, in __call__ return self._hookexec(self, self.get_hookimpls(), kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 93, in _hookexec return self._inner_hookexec(hook, methods, kwargs) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/manager.py"", line 87, in firstresult=hook.spec.opts.get(""firstresult"") if hook.spec else False, File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 208, in _multicall return outcome.get_result() File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 80, in get_result raise ex[1].with_traceback(ex[2]) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/pluggy/callers.py"", line 187, in _multicall res = hook_impl.function(*args) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/terminal.py"", line 680, in pytest_keyboard_interrupt self._keyboardinterrupt_memo = excinfo.getrepr(funcargs=True) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/_code/code.py"", line 598, in getrepr return fmt.repr_excinfo(self) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/_code/code.py"", line 830, in repr_excinfo reprtraceback = self.repr_traceback(excinfo) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/_code/code.py"", line 778, in repr_traceback reprentry = self.repr_traceback_entry(entry, einfo) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/_code/code.py"", line 737, in repr_traceback_entry reprargs = self.repr_args(entry) if not short else None File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/_code/code.py"", line 656, in repr_args args.append((argname, saferepr(argvalue))) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/_io/saferepr.py"", line 67, in saferepr return SafeRepr(maxsize).repr(obj) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/site-packages/_pytest/_io/saferepr.py"", line 36, in repr s = super().repr(x) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/reprlib.py"", line 52, in repr return self.repr1(x, self.maxlevel) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/reprlib.py"", line 60, in repr1 return getattr(self, 'repr_' + typename)(x, level) File ""/Users/simonw/.local/share/virtualenvs/datasette-oJRYYJuA/lib/python3.7/reprlib.py"", line 112, in repr_dict for key in islice(_possibly_sorted(x), self.maxdict): KeyboardInterrupt Task was destroyed but it is pending! task: wait_for=()]>> ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",570101428,.execute_write() and .execute_write_fn() methods on Database, https://github.com/simonw/datasette/pull/666#issuecomment-591049024,https://api.github.com/repos/simonw/datasette/issues/666,591049024,MDEyOklzc3VlQ29tbWVudDU5MTA0OTAyNA==,9599,simonw,2020-02-25T20:19:09Z,2020-02-25T20:19:09Z,OWNER,"This is great, thanks!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",562085508,"Use inspect-file, if possible, for total row count", https://github.com/simonw/datasette/issues/684#issuecomment-591062343,https://api.github.com/repos/simonw/datasette/issues/684,591062343,MDEyOklzc3VlQ29tbWVudDU5MTA2MjM0Mw==,9599,simonw,2020-02-25T20:50:32Z,2020-02-25T20:50:32Z,OWNER,More important is to document `execute()` - see #685 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",570301333,Add documentation on Database introspection methods to internals.rst,