issue_comments
30 rows where "created_at" is on date 2021-08-09
This data as json, CSV (advanced)
Suggested facets: reactions, updated_at (date)
| id ▼ | html_url | issue_url | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 894881016 | https://github.com/simonw/datasette/issues/1425#issuecomment-894881016 | https://api.github.com/repos/simonw/datasette/issues/1425 | IC_kwDOBm6k_c41Vsz4 | simonw 9599 | 2021-08-09T00:21:53Z | 2021-08-09T00:21:53Z | OWNER | Still one test failure: ``` def test_hook_render_cell_link_from_json(app_client): sql = """ select '{"href": "http://example.com/", "label":"Example"}' """.strip() path = "/fixtures?" + urllib.parse.urlencode({"sql": sql}) response = app_client.get(path) td = Soup(response.body, "html.parser").find("table").find("tbody").find("td") a = td.find("a") > assert a is not None, str(a) E AssertionError: None E assert None is not None ``` The weird thing about this one is that I can't replicate it on my laptop - but it happens in CI every time, including when I shell in and try to run that single test. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | render_cell() hook should support returning an awaitable 963528457 | |
| 894881448 | https://github.com/simonw/datasette/issues/1425#issuecomment-894881448 | https://api.github.com/repos/simonw/datasette/issues/1425 | IC_kwDOBm6k_c41Vs6o | simonw 9599 | 2021-08-09T00:24:25Z | 2021-08-09T00:24:39Z | OWNER | My hunch is that the "skip this `render_cell()` result if it returns `None`" logic isn't working correctly, ever since I added the `await_me_maybe` line. Could that be because Pluggy handles the "do the next if `None` is returned" logic itself, but I'm no-longer returning `None`, I'm returning an awaitable which when awaited returns `None`. This would suggest that all of the `await_me_maybe()` plugin hooks have the same bug. That's definitely possible - it may well be that no-one has yet stumbled across a bug caused by a plugin returning an awaitable and hence not being skipped, because plugin hooks that return awaitable are rare enough that no-one has tried two plugins which both use that trick. Still don't see why it would pass on my laptop but fail in CI though. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | render_cell() hook should support returning an awaitable 963528457 | |
| 894882123 | https://github.com/simonw/datasette/issues/1425#issuecomment-894882123 | https://api.github.com/repos/simonw/datasette/issues/1425 | IC_kwDOBm6k_c41VtFL | simonw 9599 | 2021-08-09T00:27:43Z | 2021-08-09T00:27:43Z | OWNER | Good news: `render_cell()` is the only hook to use `firstresult=True`: https://github.com/simonw/datasette/blob/f3c9edb376a13c09b5ecf97c7390f4e49efaadf2/datasette/hookspecs.py#L62-L64 https://pluggy.readthedocs.io/en/latest/#first-result-only | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | render_cell() hook should support returning an awaitable 963528457 | |
| 894882642 | https://github.com/simonw/datasette/issues/1425#issuecomment-894882642 | https://api.github.com/repos/simonw/datasette/issues/1425 | IC_kwDOBm6k_c41VtNS | simonw 9599 | 2021-08-09T00:29:57Z | 2021-08-09T00:29:57Z | OWNER | Here's the code in `pluggy` that implements this: https://github.com/pytest-dev/pluggy/blob/0a064fe275060dbdb1fe6e10c888e72bc400fb33/src/pluggy/callers.py#L31-L43 ```python if hook_impl.hookwrapper: try: gen = hook_impl.function(*args) next(gen) # first yield teardowns.append(gen) except StopIteration: _raise_wrapfail(gen, "did not yield") else: res = hook_impl.function(*args) if res is not None: results.append(res) if firstresult: # halt further impl calls break ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | render_cell() hook should support returning an awaitable 963528457 | |
| 894883664 | https://github.com/simonw/datasette/issues/1425#issuecomment-894883664 | https://api.github.com/repos/simonw/datasette/issues/1425 | IC_kwDOBm6k_c41VtdQ | simonw 9599 | 2021-08-09T00:33:56Z | 2021-08-09T00:33:56Z | OWNER | I could extract that code out and write my own function which implements the equivalent of calling `pm.hook.render_cell(...)` but runs `await_me_maybe()` before checking if `res is not None`. That's pretty nasty. Could I instead call the plugin hook normally, but then have additional logic which says "if I await it and it returns `None` then try calling the hook again but skip this one" - not sure if there's a way to do that either. I could remove the `firstresult=True` from the hookspec - which would cause it to call and return ALL hooks - but then in my own code use only the first one. This is slightly less efficient (since it calls all the hooks and then discards all-but-one value) but it's the least unpleasant in terms of the code I would have to write - plus I don't think it's going to be THAT common for someone to have multiple expensive `render_cell()` hooks installed at once (they are usually pretty cheap). | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | render_cell() hook should support returning an awaitable 963528457 | |
| 894884874 | https://github.com/simonw/datasette/issues/1425#issuecomment-894884874 | https://api.github.com/repos/simonw/datasette/issues/1425 | IC_kwDOBm6k_c41VtwK | simonw 9599 | 2021-08-09T00:38:20Z | 2021-08-09T00:38:20Z | OWNER | I'm trying the version where I remove `firstresult=True`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | render_cell() hook should support returning an awaitable 963528457 | |
| 894893319 | https://github.com/simonw/datasette/issues/1425#issuecomment-894893319 | https://api.github.com/repos/simonw/datasette/issues/1425 | IC_kwDOBm6k_c41Vv0H | simonw 9599 | 2021-08-09T01:08:56Z | 2021-08-09T01:09:12Z | OWNER | Demo: https://latest.datasette.io/fixtures/simple_primary_key shows `RENDER_CELL_ASYNC_RESULT` where the CSV version shows `RENDER_CELL_ASYNC`: https://latest.datasette.io/fixtures/simple_primary_key.csv - because of this test plugin code: https://github.com/simonw/datasette/blob/a390bdf9cef01d8723d025fc3348e81345ff4856/tests/plugins/my_plugin.py#L98-L122 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | render_cell() hook should support returning an awaitable 963528457 | |
| 894900267 | https://github.com/simonw/datasette/issues/1425#issuecomment-894900267 | https://api.github.com/repos/simonw/datasette/issues/1425 | IC_kwDOBm6k_c41Vxgr | simonw 9599 | 2021-08-09T01:31:22Z | 2021-08-09T01:31:22Z | OWNER | I used this to build a new plugin: https://github.com/simonw/datasette-query-links Demo here: https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++%27select+*+from+[facetable]%27+as+query%0D%0Aunion%0D%0Aselect%0D%0A++%27select+sqlite_version()%27%0D%0Aunion%0D%0Aselect%0D%0A++%27select+this+is+invalid+SQL+so+will+not+be+linked%27 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | render_cell() hook should support returning an awaitable 963528457 | |
| 894921512 | https://github.com/simonw/datasette/issues/1421#issuecomment-894921512 | https://api.github.com/repos/simonw/datasette/issues/1421 | IC_kwDOBm6k_c41V2so | simonw 9599 | 2021-08-09T03:05:26Z | 2021-08-09T03:05:26Z | OWNER | I may have a way to work around this, using `explain`. Consider this query: ```sql select * from facetable where state = :state and on_earth = :on_earth and neighborhood not like '00:04' ``` Datasette currently gets confused and shows three form fields: https://latest.datasette.io/fixtures?sql=select+*+from+facetable%0D%0Awhere+state+%3D+%3Astate%0D%0Aand+on_earth+%3D+%3Aon_earth%0D%0Aand+neighborhood+not+like+%2700%3A04%27&state=&on_earth=&04= <img width="698" alt="fixtures__select___from_facetable_where_state____state_and_on_earth____on_earth_and_neighborhood_not_like__00_04__and_pyinfra_pip_py_at_current_·_Fizzadar_pyinfra" src="https://user-images.githubusercontent.com/9599/128656369-8af860bf-f7c1-4c9c-beba-1eb6887a8336.png"> But... if I run `explain` [against that](https://latest.datasette.io/fixtures?sql=explain+select+*+from+facetable%0D%0Awhere+state+%3D+%3Astate%0D%0Aand+on_earth+%3D+%3Aon_earth%0D%0Aand+neighborhood+not+like+%2700%3A04%27&state=&on_earth=&04=) I get this (truncated): addr | opcode | p1 | p2 | p3 | p4 | p5 | comment -- | -- | -- | -- | -- | -- | -- | -- 20 | ResultRow | 6 | 10 | 0 | | 0 | 21 | Next | 0 | 3 | 0 | | 1 | 22 | Halt | 0 | 0 | 0 | | 0 | 23 | Transaction | 0 | 0 | 35 | 0 | 1 | 24 | Variable | 1 | 2 | 0 | :state | 0 | 25 | Variable | 2 | 3 | 0 | :on_earth | 0 | 26 | String8 | 0 | 4 | 0 | 00:04 | 0 | 27 | Goto | 0 | 1 | 0 | | 0 | Could it be as simple as pulling out those `Variable` rows to figure out the names of the variables in the query? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | "Query parameters" form shows wrong input fields if query contains "03:31" style times 959999095 | |
| 894922145 | https://github.com/simonw/datasette/issues/1421#issuecomment-894922145 | https://api.github.com/repos/simonw/datasette/issues/1421 | IC_kwDOBm6k_c41V22h | simonw 9599 | 2021-08-09T03:07:38Z | 2021-08-09T03:07:38Z | OWNER | I hoped this would work: ```sql with foo as ( explain select * from facetable where state = :state and on_earth = :on_earth and neighborhood not like '00:04' ) select p4 from foo where opcode = 'Variable' ``` But sadly [it returns an error](https://latest.datasette.io/fixtures?sql=with+foo+as+%28%0D%0A++explain+select+*+from+facetable%0D%0A++where+state+%3D+%3Astate%0D%0A++and+on_earth+%3D+%3Aon_earth%0D%0A++and+neighborhood+not+like+%2700%3A04%27%0D%0A%29%0D%0Aselect+p4+from+foo+where+opcode+%3D+%27Variable%27&state=&on_earth=&04=): > near "explain": syntax error | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | "Query parameters" form shows wrong input fields if query contains "03:31" style times 959999095 | |
| 894922703 | https://github.com/simonw/datasette/issues/1421#issuecomment-894922703 | https://api.github.com/repos/simonw/datasette/issues/1421 | IC_kwDOBm6k_c41V2_P | simonw 9599 | 2021-08-09T03:09:29Z | 2021-08-09T03:09:29Z | OWNER | Relevant code: https://github.com/simonw/datasette/blob/ad90a72afa21b737b162e2bbdddc301a97d575cd/datasette/views/database.py#L225-L231 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | "Query parameters" form shows wrong input fields if query contains "03:31" style times 959999095 | |
| 894925437 | https://github.com/simonw/datasette/issues/1421#issuecomment-894925437 | https://api.github.com/repos/simonw/datasette/issues/1421 | IC_kwDOBm6k_c41V3p9 | simonw 9599 | 2021-08-09T03:19:00Z | 2021-08-09T03:19:00Z | OWNER | This may not work: > `ERROR: sql = 'explain select 1 + :one + :two', params = None: You did not supply a value for binding 1.` The `explain` queries themselves want me to pass them parameters. I could try using the regex to pull out candidates and passing `None` for each of those, including incorrect ones like `:31`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | "Query parameters" form shows wrong input fields if query contains "03:31" style times 959999095 | |
| 894925914 | https://github.com/simonw/datasette/issues/1421#issuecomment-894925914 | https://api.github.com/repos/simonw/datasette/issues/1421 | IC_kwDOBm6k_c41V3xa | simonw 9599 | 2021-08-09T03:20:42Z | 2021-08-09T03:20:42Z | OWNER | I think this works! ```python _re_named_parameter = re.compile(":([a-zA-Z0-9_]+)") async def derive_named_parameters(db, sql): explain = 'explain {}'.format(sql.strip().rstrip(";")) possible_params = _re_named_parameter.findall(sql) try: results = await db.execute(explain, {p: None for p in possible_params}) return [row["p4"].lstrip(":") for row in results if row["opcode"] == "Variable"] except sqlite3.DatabaseError: return [] ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | "Query parameters" form shows wrong input fields if query contains "03:31" style times 959999095 | |
| 894927185 | https://github.com/simonw/datasette/issues/1421#issuecomment-894927185 | https://api.github.com/repos/simonw/datasette/issues/1421 | IC_kwDOBm6k_c41V4FR | simonw 9599 | 2021-08-09T03:25:01Z | 2021-08-09T03:25:01Z | OWNER | One catch with this approach: if the SQL query is invalid, the parameters will not be extracted and shown as form fields. Maybe that's completely fine? Why display a form if it's going to break when the user actually runs the query? But it does bother me. I worry that someone who is still iterating on and editing their query before actually starting to use it might find the behaviour confusing. So maybe if the query raises an exception it could fall back on the regular expression results? | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | "Query parameters" form shows wrong input fields if query contains "03:31" style times 959999095 | |
| 894929080 | https://github.com/simonw/datasette/issues/1421#issuecomment-894929080 | https://api.github.com/repos/simonw/datasette/issues/1421 | IC_kwDOBm6k_c41V4i4 | simonw 9599 | 2021-08-09T03:33:02Z | 2021-08-09T03:33:02Z | OWNER | Fixed! Fantastic, this one has been bothering me for *years*. https://latest.datasette.io/fixtures?sql=select+*+from+facetable%0D%0Awhere+state+%3D+%3Astate%0D%0Aand+on_earth+%3D+%3Aon_earth%0D%0Aand+neighborhood+not+like+%2700%3A04%27 <img width="707" alt="fixtures__select___from_facetable_where_state____state_and_on_earth____on_earth_and_neighborhood_not_like__00_04__and_pyinfra_pip_py_at_current_·_Fizzadar_pyinfra" src="https://user-images.githubusercontent.com/9599/128657807-80ac818d-6fd9-4f70-ad26-900cec6a7482.png"> | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | "Query parameters" form shows wrong input fields if query contains "03:31" style times 959999095 | |
| 894929769 | https://github.com/simonw/datasette/issues/1421#issuecomment-894929769 | https://api.github.com/repos/simonw/datasette/issues/1421 | IC_kwDOBm6k_c41V4tp | simonw 9599 | 2021-08-09T03:36:49Z | 2021-08-09T03:36:49Z | OWNER | SQLite carries a warning about using `EXPLAIN` like this: https://www.sqlite.org/lang_explain.html > The output from EXPLAIN and EXPLAIN QUERY PLAN is intended for interactive analysis and troubleshooting only. The details of the output format are subject to change from one release of SQLite to the next. Applications should not use EXPLAIN or EXPLAIN QUERY PLAN since their exact behavior is variable and only partially documented. I think that's OK here, because of the regular expression fallback. If the format changes in the future in a way that breaks the query the error should be caught and the regex-captured parameters should be returned instead. Hmmm... actually that's not entirely true: https://github.com/simonw/datasette/blob/b1fed48a95516ae84c0f020582303ab50ab817e2/datasette/utils/__init__.py#L1084-L1091 If the format changes such that the same columns are returned but the `[row["p4"].lstrip(":") for row in results if row["opcode"] == "Variable"]` list comprehension returns an empty array it will break Datasette! I'm going to take that risk for the moment, but I'll actively watch out for problems in the future. If this does turn out to be bad I can always go back to the pure regular expression mechanism. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | "Query parameters" form shows wrong input fields if query contains "03:31" style times 959999095 | |
| 894930013 | https://github.com/simonw/datasette/issues/1421#issuecomment-894930013 | https://api.github.com/repos/simonw/datasette/issues/1421 | IC_kwDOBm6k_c41V4xd | simonw 9599 | 2021-08-09T03:38:06Z | 2021-08-09T03:38:06Z | OWNER | Amusing edge-case: if you run this against a `explain ...` query it falls back to using regular expressions, because `explain explain select ...` is invalid SQL. https://latest.datasette.io/fixtures?sql=explain+select+*+from+facetable%0D%0Awhere+state+%3D+%3Astate%0D%0Aand+on_earth+%3D+%3Aon_earth%0D%0Aand+neighborhood+not+like+%2700%3A04%27&state=&on_earth= | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | "Query parameters" form shows wrong input fields if query contains "03:31" style times 959999095 | |
| 895003796 | https://github.com/simonw/datasette/issues/1425#issuecomment-895003796 | https://api.github.com/repos/simonw/datasette/issues/1425 | IC_kwDOBm6k_c41WKyU | abdusco 3243482 | 2021-08-09T07:14:35Z | 2021-08-09T07:14:35Z | CONTRIBUTOR | I believe this also provides a workaround for the problem I face in https://github.com/simonw/datasette/issues/1300. Now I should be able to get table PKs and generate a row URL. I'll test this out and report my findings. ```py from datasette.utils import path_from_row_pks pks = await db.primary_keys(table) url = self.ds.urls.row_blob( database, table, path_from_row_pks(row, pks, not pks), column, ) ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | render_cell() hook should support returning an awaitable 963528457 | |
| 895500565 | https://github.com/simonw/datasette/issues/1426#issuecomment-895500565 | https://api.github.com/repos/simonw/datasette/issues/1426 | IC_kwDOBm6k_c41YEEV | simonw 9599 | 2021-08-09T20:00:04Z | 2021-08-09T20:00:04Z | OWNER | A few options for how this would work: - `datasette ... --robots allow` - `datasette ... --setting robots allow` Options could be: - `allow` - allow all crawling - `deny` - deny all crawling - `limited` - allow access to the homepage and the index pages for each database and each table, but disallow crawling any further than that The "limited" mode is particularly interesting. Could even make it the default, but I think that may be a bit too confusing. Idea would be to get the key pages indexed but use `nofollow` to discourage crawlers from indexing individual row pages or deep pages like `https://datasette.io/content/repos?_facet=owner&_facet=language&_facet_array=topics&topics__arraycontains=sqlite#facet-owner`. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Manage /robots.txt in Datasette core, block robots by default 964322136 | |
| 895509536 | https://github.com/simonw/datasette/issues/1426#issuecomment-895509536 | https://api.github.com/repos/simonw/datasette/issues/1426 | IC_kwDOBm6k_c41YGQg | simonw 9599 | 2021-08-09T20:12:57Z | 2021-08-09T20:12:57Z | OWNER | I could try out the `X-Robots` HTTP header too: https://developers.google.com/search/docs/advanced/robots/robots_meta_tag#xrobotstag | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Manage /robots.txt in Datasette core, block robots by default 964322136 | |
| 895510773 | https://github.com/simonw/datasette/issues/1426#issuecomment-895510773 | https://api.github.com/repos/simonw/datasette/issues/1426 | IC_kwDOBm6k_c41YGj1 | simonw 9599 | 2021-08-09T20:14:50Z | 2021-08-09T20:19:22Z | OWNER | https://twitter.com/mal/status/1424825895139876870 > True pinging google should be part of the build process on a static site :) That's another aspect of this: if you DO want your site crawled, teaching the `datasette publish` command how to ping Google when a deploy has gone out could be a nice improvement. Annoyingly it looks like you need to configure an auth token of some sort in order to use their API though, which is likely too much hassle to be worth building into Datasette itself: https://developers.google.com/search/apis/indexing-api/v3/using-api ``` curl -X POST https://indexing.googleapis.com/v3/urlNotifications:publish -d '{ "url": "https://careers.google.com/jobs/google/technical-writer", "type": "URL_UPDATED" }' -H "Content-Type: application/json" { "error": { "code": 401, "message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.", "status": "UNAUTHENTICATED" } } ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Manage /robots.txt in Datasette core, block robots by default 964322136 | |
| 895522818 | https://github.com/simonw/datasette/issues/1426#issuecomment-895522818 | https://api.github.com/repos/simonw/datasette/issues/1426 | IC_kwDOBm6k_c41YJgC | simonw 9599 | 2021-08-09T20:34:10Z | 2021-08-09T20:34:10Z | OWNER | At the very least Datasette should serve a blank `/robots.txt` by default - I'm seeing a ton of 404s for it in the logs. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | Manage /robots.txt in Datasette core, block robots by default 964322136 | |
| 895571420 | https://github.com/simonw/sqlite-utils/issues/310#issuecomment-895571420 | https://api.github.com/repos/simonw/sqlite-utils/issues/310 | IC_kwDOCGYnMM41YVXc | simonw 9599 | 2021-08-09T21:44:38Z | 2021-08-09T21:44:38Z | OWNER | When I ship this I should update the TILs at https://til.simonwillison.net/cloudrun/tailing-cloud-run-request-logs and https://til.simonwillison.net/jq/flatten-nested-json-objects-jq to reference it. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `sqlite-utils insert --flatten` option to flatten nested JSON 964400482 | |
| 895572309 | https://github.com/simonw/sqlite-utils/issues/310#issuecomment-895572309 | https://api.github.com/repos/simonw/sqlite-utils/issues/310 | IC_kwDOCGYnMM41YVlV | simonw 9599 | 2021-08-09T21:46:15Z | 2021-08-09T21:46:15Z | OWNER | Documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#flattening-nested-json-objects | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | `sqlite-utils insert --flatten` option to flatten nested JSON 964400482 | |
| 895577012 | https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895577012 | https://api.github.com/repos/simonw/sqlite-utils/issues/309 | IC_kwDOCGYnMM41YWu0 | simonw 9599 | 2021-08-09T21:55:52Z | 2021-08-09T21:59:03Z | OWNER | Yeah this error message could certainly be more helpful. I thought `OverflowError` might be one of the SQLite exceptions: https://docs.python.org/3/library/sqlite3.html#exceptions - but it turns out it's actually reusing the Python built-in `OverflowError` class: ```python import sqlite3 db = sqlite3.connect(":memory:") caught = [] try: db.execute("create table foo (number integer)") db.execute("insert into foo (number) values (?)", [34223049823094832094802398430298048240]) except Exception as e: print(e) caught.append(e) isinstance(caught[0], OverflowError) ``` Here's where that happens in the Python `sqlite3` module code: https://github.com/python/cpython/blob/058fb35b57ca8c5063d16ec818e668b3babfea65/Modules/_sqlite/util.c#L123-L124 | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils insert errors should show SQL and parameters, if possible 963897111 | |
| 895581038 | https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895581038 | https://api.github.com/repos/simonw/sqlite-utils/issues/309 | IC_kwDOCGYnMM41YXtu | simonw 9599 | 2021-08-09T22:03:54Z | 2021-08-09T23:39:53Z | OWNER | Steps to reproduce: echo '{"v": 34223049823094832094802398430298048240}' | sqlite-utils insert /tmp/blah.db row - | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils insert errors should show SQL and parameters, if possible 963897111 | |
| 895587282 | https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895587282 | https://api.github.com/repos/simonw/sqlite-utils/issues/309 | IC_kwDOCGYnMM41YZPS | simonw 9599 | 2021-08-09T22:15:25Z | 2021-08-09T22:15:25Z | OWNER | I'm going to use a bit of a dirty trick for this one: I'm going to recursively inspect the stack on an error and try to find the `sql` and `params` variables. That way I can handle this all at the CLI layer without changing the exceptions that are being raised by the Python library. | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils insert errors should show SQL and parameters, if possible 963897111 | |
| 895587441 | https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895587441 | https://api.github.com/repos/simonw/sqlite-utils/issues/309 | IC_kwDOCGYnMM41YZRx | simonw 9599 | 2021-08-09T22:15:45Z | 2021-08-09T22:15:45Z | OWNER | ``` OverflowError: Python int too large to convert to SQLite INTEGER >>> import sys >>> def find_variables(tb, vars): to_find = list(vars) found = {} for var in to_find: if var in tb.tb_frame.f_locals: vars.remove(var) found[var] = tb.tb_frame.f_locals[var] if vars and tb.tb_next: found.update(find_variables(tb.tb_next, vars)) return found ... >>> find_variables(sys.last_traceback, ["sql", "params"]) {'params': [34223049823094832094802398430298048240], 'sql': 'INSERT INTO [row] ([v]) VALUES (?);'} ``` | {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils insert errors should show SQL and parameters, if possible 963897111 | |
| 895592507 | https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895592507 | https://api.github.com/repos/simonw/sqlite-utils/issues/309 | IC_kwDOCGYnMM41Yag7 | simonw 9599 | 2021-08-09T22:26:28Z | 2021-08-09T22:33:48Z | OWNER | Demo: ``` $ echo '{"v": 34223049823094832094802398430298048240}' | sqlite-utils insert /tmp/blah.db row - Error: Python int too large to convert to SQLite INTEGER sql = INSERT INTO [row] ([v]) VALUES (?); parameters = [34223049823094832094802398430298048240] ``` | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils insert errors should show SQL and parameters, if possible 963897111 | |
| 895622908 | https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895622908 | https://api.github.com/repos/simonw/sqlite-utils/issues/309 | IC_kwDOCGYnMM41Yh78 | simonw 9599 | 2021-08-09T23:40:29Z | 2021-08-09T23:40:29Z | OWNER | TIL about how the stack inspection works: https://til.simonwillison.net/python/find-local-variables-in-exception-traceback | {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | sqlite-utils insert errors should show SQL and parameters, if possible 963897111 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] (
[html_url] TEXT,
[issue_url] TEXT,
[id] INTEGER PRIMARY KEY,
[node_id] TEXT,
[user] INTEGER REFERENCES [users]([id]),
[created_at] TEXT,
[updated_at] TEXT,
[author_association] TEXT,
[body] TEXT,
[reactions] TEXT,
[issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
ON [issue_comments] ([user]);
created_at (date) 1 ✖