issue_comments

Table actions

15 rows where issue = 314665147

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

id ▼ html_url issue_url node_id user created_at updated_at author_association body reactions issue performed_via_github_app
381643173 https://github.com/simonw/datasette/issues/216#issuecomment-381643173 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTY0MzE3Mw== simonw 9599 2018-04-16T15:21:17Z 2018-04-16T15:21:17Z OWNER Yikes, definitely a bug. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381644355 https://github.com/simonw/datasette/issues/216#issuecomment-381644355 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTY0NDM1NQ== simonw 9599 2018-04-16T15:24:38Z 2018-04-16T15:24:38Z OWNER So there are two tricky problems to solve here: * I need a way of encoding `null` into that `_next=` that is unambiguous from the string `None` or `null`. This means introducing some kind of escaping mechanism in those strings. I already use URL encoding as part of the construction of those components here, maybe that can help here? * I need to figure out what the SQL should be for the "next" set of results if the previous value was null. Thankfully we use the primary key as a tie-breaker so this shouldn't be impossible. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381645274 https://github.com/simonw/datasette/issues/216#issuecomment-381645274 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTY0NTI3NA== simonw 9599 2018-04-16T15:27:16Z 2018-04-16T15:27:16Z OWNER Relevant code: https://github.com/simonw/datasette/blob/904f1c75a3c17671d25c53b91e177c249d14ab3b/datasette/app.py#L828-L832 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381645973 https://github.com/simonw/datasette/issues/216#issuecomment-381645973 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTY0NTk3Mw== simonw 9599 2018-04-16T15:29:11Z 2018-04-16T15:29:11Z OWNER I could use `$null` as a magic value that means None. Since I'm applying `quote_plus()` to actual values, any legit strings that look like this will be encoded as `%24null`: ``` >>> urllib.parse.quote_plus('$null') '%24null' ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381648053 https://github.com/simonw/datasette/issues/216#issuecomment-381648053 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTY0ODA1Mw== simonw 9599 2018-04-16T15:35:17Z 2018-04-16T15:35:17Z OWNER I think the correct SQL is this: https://datasette-issue-189-demo-3.now.sh/salaries-7859114-7859114?sql=select+rowid%2C+*+from+%5B2017+Maryland+state+salaries%5D%0D%0Awhere+%28middle_initial+is+not+null+or+%28middle_initial+is+null+and+rowid+%3E+%3Ap0%29%29%0D%0Aorder+by+middle_initial+limit+101&p0=391 ``` select rowid, * from [2017 Maryland state salaries] where (middle_initial is not null or (middle_initial is null and rowid > :p0)) order by middle_initial limit 101 ``` Though this will also need to be taken into account for #198 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381649140 https://github.com/simonw/datasette/issues/216#issuecomment-381649140 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTY0OTE0MA== simonw 9599 2018-04-16T15:38:29Z 2018-04-16T15:38:29Z OWNER But what would that SQL look like for `_sort_desc`? {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381649437 https://github.com/simonw/datasette/issues/216#issuecomment-381649437 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTY0OTQzNw== simonw 9599 2018-04-16T15:39:21Z 2018-04-16T15:39:21Z OWNER Here's where that SQL gets constructed at the moment: https://github.com/simonw/datasette/blob/10a34f995c70daa37a8a2aa02c3135a4b023a24c/datasette/app.py#L761-L771 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381786522 https://github.com/simonw/datasette/issues/216#issuecomment-381786522 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTc4NjUyMg== simonw 9599 2018-04-16T23:58:45Z 2018-04-16T23:59:13Z OWNER Weird... tests are failing in Travis, despite passing on my local machine. https://travis-ci.org/simonw/datasette/builds/367423706 {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381788051 https://github.com/simonw/datasette/issues/216#issuecomment-381788051 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTc4ODA1MQ== simonw 9599 2018-04-17T00:07:48Z 2018-04-17T00:07:48Z OWNER Still failing. This is very odd. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381794744 https://github.com/simonw/datasette/issues/216#issuecomment-381794744 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTc5NDc0NA== simonw 9599 2018-04-17T00:51:41Z 2018-04-17T00:51:41Z OWNER I'm reverting this out of master until I can figure out why the tests are failing. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381798786 https://github.com/simonw/datasette/issues/216#issuecomment-381798786 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTc5ODc4Ng== simonw 9599 2018-04-17T01:18:25Z 2018-04-17T01:18:25Z OWNER Here's the test that's failing: https://github.com/simonw/datasette/blob/59a3aa859c0e782aeda9a515b1b52c358e8458a2/tests/test_api.py#L437-L470 I got Travis to spit out the `fetched` and `expected` variables. `expected` has 201 items in it and is identical to what I get on my local laptop. `fetched` has 250 items in it, so it's clearly different from my local environment. I've managed to replicate the bug in production! I created a test database like this: python tests/fixtures.py sortable.db Then deployed that database like so: datasette publish now sortable.db \ --extra-options="--page_size=50" --branch=debug-travis-issue-216 And... if you click "next" on this page https://datasette-issue-216-pagination.now.sh/sortable-5679797/sortable?_sort_desc=sortable_with_nulls five times you get back 250 results, when you should only get back 201. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381799267 https://github.com/simonw/datasette/issues/216#issuecomment-381799267 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTc5OTI2Nw== simonw 9599 2018-04-17T01:21:35Z 2018-04-17T01:21:35Z OWNER The version that I deployed which exhibits the bug is running SQLite `3.8.7.1` - https://datasette-issue-216-pagination.now.sh/sortable-5679797?sql=select+sqlite_version%28%29 The version that I have running locally which does NOT exhibit the bug is running SQLite `3.23.0` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381799408 https://github.com/simonw/datasette/issues/216#issuecomment-381799408 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTc5OTQwOA== simonw 9599 2018-04-17T01:22:30Z 2018-04-17T01:22:30Z OWNER ... which is VERY surprising, because `3.23.0` only came out on 2nd April this year: https://www.sqlite.org/changes.html - I have no idea how I came to be running that version on my laptop. {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381801302 https://github.com/simonw/datasette/issues/216#issuecomment-381801302 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTgwMTMwMg== simonw 9599 2018-04-17T01:33:43Z 2018-04-17T01:33:43Z OWNER This is the SQL that returns differing results in production and on my laptop: https://datasette-issue-216-pagination.now.sh/sortable-5679797?sql=select+%2A+from+sortable+where+%28sortable_with_nulls+is+null+and+%28%28pk1+%3E+%3Ap0%29%0A++or%0A%28pk1+%3D+%3Ap0+and+pk2+%3E+%3Ap1%29%29%29+order+by+sortable_with_nulls+desc+limit+51&p0=b&p1=t ``` select * from sortable where (sortable_with_nulls is null and ((pk1 > :p0) or (pk1 = :p0 and pk2 > :p1))) order by sortable_with_nulls desc limit 51 ``` I think that `order by sortable_with_nulls desc` bit is at fault - the primary keys should be included in that order by as well. Sure enough, changing the query to this one returns the same results across both environments: ``` select * from sortable where (sortable_with_nulls is null and ((pk1 > :p0) or (pk1 = :p0 and pk2 > :p1))) order by sortable_with_nulls desc, pk1, pk2 limit 51 ``` {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  
381803157 https://github.com/simonw/datasette/issues/216#issuecomment-381803157 https://api.github.com/repos/simonw/datasette/issues/216 MDEyOklzc3VlQ29tbWVudDM4MTgwMzE1Nw== simonw 9599 2018-04-17T01:45:24Z 2018-04-17T01:45:24Z OWNER Fixed! {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} Bug: Sort by column with NULL in next_page URL 314665147  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);