{"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419357290", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1419357290, "node_id": "IC_kwDOCGYnMM5Umaxq", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2023-02-06T16:21:44Z", "updated_at": "2023-02-06T16:21:44Z", "author_association": "CONTRIBUTOR", "body": "SQLite doesn't have a native `DATETIME` type. It stores dates internally as strings and then has [functions](https://www.sqlite.org/lang_datefunc.html) to work with date-like strings. Yes it's weird.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419390560", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1419390560, "node_id": "IC_kwDOCGYnMM5Umi5g", "user": {"value": 21095447, "label": "4l1fe"}, "created_at": "2023-02-06T16:43:47Z", "updated_at": "2023-02-06T16:43:47Z", "author_association": "NONE", "body": "> SQLite doesn't have a native `DATETIME` type. It stores dates internally as strings and then has [functions](https://www.sqlite.org/lang_datefunc.html) to work with date-like strings. Yes it's weird.\r\n\r\nThat's correct. But my issue is about the application level libraries that, i suppose, have better data understanding if see a specific type such as `DATETIME`. \r\n\r\nI'm writing data with **dataset** i've mentioned. The lib changes its behavior depending on a type. I saw different behavior with types `DATETIME, FLOAT, TEXT`. Dataset, for their part, is built upon Sqlalchemy, you know what it is.\r\n\r\nTo be honest, i didn't dive into the details of why the behavior changes, but when i altered manually by other util a type of column to `DATETIME` things got back to normal.\r\n\r\nOn the matter, can i achieve it with Sqlite Utils at the moment?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419734229", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1419734229, "node_id": "IC_kwDOCGYnMM5Un2zV", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-02-06T20:53:28Z", "updated_at": "2023-02-06T21:16:29Z", "author_association": "NONE", "body": "I think it's not currently possible: sqlite-utils requires that it be one of `integer`, `text`, `float`, `blob` ([see code](https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/cli.py#L2266))\r\n\r\nIMO, this is a bit of friction and it would be nice if it was more permissive. SQLite permits developers to use any data type when creating a table. For example, this is a perfectly cromulent sqlite session that creates a table with columns of type `baz` and `bar`:\r\n\r\n```\r\nsqlite> create table foo(column1 baz, column2 bar);\r\nsqlite> .schema foo\r\nCREATE TABLE foo(column1 baz, column2 bar);\r\nsqlite> select * from pragma_table_info('foo');\r\ncid         name        type        notnull     dflt_value  pk        \r\n----------  ----------  ----------  ----------  ----------  ----------\r\n0           column1     baz         0                       0         \r\n1           column2     bar         0                       0   \r\n```\r\n\r\nThe idea is that the application developer will know what meaning to ascribe to those types. For example, I'm working on a plugin to Datasette. Dates are tricky to handle. If you have some existing rows, you can look at the values in them to know how a user is serializing the dates -- as an ISO 8601 string? An RFC 3339 string? With millisecond precision? With timezone offset? But if you don't yet have any rows, you have to guess. If the column is of type `TEXT`, you don't even know that it's meant to hold a date! In this case, my plugin will look to see if the column is of type `DATE` or `DATETIME`, and assume a certain representation when writing.\r\n\r\nPerhaps there is an argument that sqlite-utils is trying to conform to SQLite's strict mode, and that is why it limits the choices. In strict mode, SQLite requires that the data type be one of `INT`, `INTEGER`, `REAL`, `TEXT`, `BLOB`, `ANY`. But that can't be the case -- sqlite-utils supports `FLOAT`, which is not one of the valid types in strict mode, and it rejects `INT`, `REAL` and `ANY`, which _are_ valid.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419740776", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1419740776, "node_id": "IC_kwDOCGYnMM5Un4Zo", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-02-06T20:59:01Z", "updated_at": "2023-02-06T20:59:01Z", "author_association": "NONE", "body": "That said, it looks like the check is only enforced at the CLI level. If you use the API directly, I think it'll work.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419916684", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419916684, "node_id": "IC_kwDOBm6k_c5UojWM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:09:51Z", "updated_at": "2023-02-06T23:10:13Z", "author_association": "OWNER", "body": "The inputs and outputs for this are pretty complex.\r\n\r\nInputs:\r\n\r\n- `?_next=` from the query string\r\n- `is_view` - is this for a table or view? If it's a view it uses offset/limit pagination - which could actually work for arbitrary queries too. Also views could have keyset pagination if they are known to be sorted by a particular column.\r\n- `sort` and `sort_desc` reflecting the current sort order\r\n- `use_rowid` for if the table is a rowid table with no primary key of its own\r\n- `pks` - the primary keys for the table\r\n- `params` - the current set of parameters, I think used just to count their length so new params can be added as `p5` etc without collisions. This could be handled with a `s0`, `s1` etc naming convention instead.\r\n\r\nOutputs:\r\n\r\n- `where_clauses` - a list of where clauses to add to the query\r\n- `params` - additional parameters to use with the query due to the new where clauses\r\n- `order_by` - the order by clause", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419917661", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419917661, "node_id": "IC_kwDOBm6k_c5Uojld", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:10:51Z", "updated_at": "2023-02-06T23:10:51Z", "author_association": "OWNER", "body": "I should turn `sort` and `sort_desc` into an object representing the sort order earlier in the code.\r\n\r\nI should also create something that bundles together `pks` and `use_rowid` and maybe `is_view` as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419921228", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419921228, "node_id": "IC_kwDOBm6k_c5UokdM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:14:15Z", "updated_at": "2023-02-06T23:14:15Z", "author_association": "OWNER", "body": "Crucial utility function: https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/utils/__init__.py#L137-L160", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419928455", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419928455, "node_id": "IC_kwDOBm6k_c5UomOH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:21:50Z", "updated_at": "2023-02-06T23:21:50Z", "author_association": "OWNER", "body": "Found more logic relating to this:\r\n\r\nhttps://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L684-L732", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419953256", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419953256, "node_id": "IC_kwDOBm6k_c5UosRo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:42:56Z", "updated_at": "2023-02-06T23:43:10Z", "author_association": "OWNER", "body": "Relevant issue:\r\n- https://github.com/simonw/datasette/issues/1773\r\n\r\nExplains this comment: https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L697\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null}