{"html_url": "https://github.com/simonw/sqlite-utils/issues/125#issuecomment-664062546", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/125", "id": 664062546, "node_id": "MDEyOklzc3VlQ29tbWVudDY2NDA2MjU0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-27T00:33:03Z", "updated_at": "2020-07-27T00:33:03Z", "author_association": "OWNER", "body": "I'm going to imitate how Datasette solves this problem:\r\n```json\r\n[\r\n  {\r\n    \"name\": \"lorem.txt\",\r\n    \"mode\": 33188,\r\n    \"mtime\": 1595805965,\r\n    \"sz\": 16984,\r\n    \"data\": {\r\n      \"$base64\": true,\r\n      \"encoded\": \"eJzt0c1xAyEMBeC7q1ABHleR3HxNAQrIjmb4M0gelx+RTY7p4N2WBYT0vmufUknH8kq5lz5pqRFXsTOl3pYkE/NJnHXoStruJEVjc0mOCyTqq/ZMJnXEZW1Js2ZvRm5U+DPKk9hRWqjyvTFx0YfzhT6MpGmN2lR1fzxjyfVMD9dFrS+bnkleMpMam/ZGXgrX1I/K+5Au3S/9lNQRh0k4Gq/RUz8GiKfsQm+7JLsJ6fTo5JhVG00ZU76kZZkxePx49uIjnpNoJyYlWUsoaSl/CcVATje/Kxu13RANnrHweaH3V5Jh4jvGyKCnxJLiXPKhmW3fiCnG7Jql7RR3UvFo8jJ4z039dtOkTFmWzL1be9lt8A5II471m6vXy+l0BR/4wAc+8IEPfOADH/jABz7wgQ984AMf+MAHPvCBD3zgAx/4wAc+8IEPfOADH/jABz7wgQ984AMf+MAHPvCBD3zgAx/4wAc+8IEPfOADH/jABz7wgQ984PuP7xubBoN9\"\r\n    }\r\n  }\r\n]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 665817570, "label": "Output binary columns in \"sqlite-utils query\" JSON"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/125#issuecomment-664065341", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/125", "id": 664065341, "node_id": "MDEyOklzc3VlQ29tbWVudDY2NDA2NTM0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-27T00:49:41Z", "updated_at": "2020-07-27T00:49:41Z", "author_association": "OWNER", "body": "Documentation: https://github.com/simonw/sqlite-utils/commit/20e543e9a492f2e764caae73c38e87f18eaec444?short_path=7240b7c#diff-7240b7c71b1a8194da0c001c64fc8d40", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 665817570, "label": "Output binary columns in \"sqlite-utils query\" JSON"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/126#issuecomment-664065597", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/126", "id": 664065597, "node_id": "MDEyOklzc3VlQ29tbWVudDY2NDA2NTU5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-27T00:51:11Z", "updated_at": "2020-07-27T00:51:11Z", "author_association": "OWNER", "body": "I'm going to implement this as the reverse of #125 - binary columns in JSON are now output like this:\r\n```json\r\n  {\r\n    \"name\": \"lorem.txt\",\r\n    \"mode\": 33188,\r\n    \"mtime\": 1595805965,\r\n    \"sz\": 16984,\r\n    \"data\": {\r\n      \"$base64\": true,\r\n      \"encoded\": \"eJzt0c1xAyEMBeC7q1ABHleR3HxNAQrIjmb4M0gelx+RTY7p4N2WBYT0vmufUknH8kq5lz5pqRFXsTOl3pYkE/NJnHXoStruJEVjc0mOCyTqq/ZMJnXEZW1Js2ZvRm5U+DPKk9hRWqjyvTFx0YfzhT6MpGmN2lR1fzxjyfVMD9dFrS+bnkleMpMam/ZGXgrX1I/K+5Au3S/9lNQRh0k4Gq/RUz8GiKfsQm+7JLsJ6fTo5JhVG00ZU76kZZkxePx49uIjnpNoJyYlWUsoaSl/CcVATje/Kxu13RANnrHweaH3V5Jh4jvGyKCnxJLiXPKhmW3fiCnG7Jql7RR3UvFo8jJ4z039dtOkTFmWzL1be9lt8A5II471m6vXy+l0BR/4wAc+8IEPfOADH/jABz7wgQ984AMf+MAHPvCBD3zgAx/4wAc+8IEPfOADH/jABz7wgQ984AMf+MAHPvCBD3zgAx/4wAc+8IEPfOADH/jABz7wgQ984PuP7xubBoN9\"\r\n    }\r\n  }\r\n]\r\n```\r\nSo the `sqlite-utils insert` command should learn to spot `{\"$base64\": true...}` values and base64 decode them before inserting them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 665819048, "label": "Ability to insert binary data on the CLI using JSON"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/126#issuecomment-664106405", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/126", "id": 664106405, "node_id": "MDEyOklzc3VlQ29tbWVudDY2NDEwNjQwNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-27T04:00:08Z", "updated_at": "2020-07-27T04:00:33Z", "author_association": "OWNER", "body": "```\r\n$ echo '[\r\n    {\r\n        \"name\": \"transparent.gif\",\r\n        \"content\": {\r\n            \"$base64\": true,\r\n            \"encoded\": \"R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\"\r\n        }\r\n    }\r\n]' | sqlite-utils insert trans.db files - --pk=name\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 665819048, "label": "Ability to insert binary data on the CLI using JSON"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/114#issuecomment-664106621", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/114", "id": 664106621, "node_id": "MDEyOklzc3VlQ29tbWVudDY2NDEwNjYyMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-27T04:01:13Z", "updated_at": "2020-07-27T04:01:13Z", "author_association": "OWNER", "body": "Work in progress in `transform` branch here: https://github.com/simonw/sqlite-utils/tree/transform", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621989740, "label": "table.transform() method for advanced alter table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/122#issuecomment-663931279", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/122", "id": 663931279, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MzkzMTI3OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-26T03:33:23Z", "updated_at": "2020-07-27T04:30:49Z", "author_association": "OWNER", "body": "One idea: `sqlite-utils insert-files`\r\n\r\nIt could work something like this:\r\n\r\n    sqlite-utils insert-files files.db /tmp/blah.jpg /tmp/foo.gif \\\r\n      --table files \\\r\n      -c key:filename -c hash:sha256 -c body:content \\\r\n      --pk key\r\n\r\nThis would insert those two image files into the database in a table called `files` with a schema that looks something like this:\r\n\r\n```sql\r\nCREATE TABLE files (\r\n    key text primary key,\r\n    hash text,\r\n    body blob\r\n);\r\n```\r\n\r\nThe `-c key:filename` options here are the most interesting: they let you create the table with a specific layout. The bit before the `:` is the column name. The bit after the `:` can be a range of different things:\r\n\r\n- `filename` - just the filename\r\n- `filepath` - the full filepath (provided on the command-line)\r\n- `absolutepath` - the filepath expanded to start with `/home/...` or whatever\r\n- `sha256` - the SHA256 of the contents\r\n- `md5` - the MD5\r\n- `content` - the binary content itself\r\n- `mtime` - the mtime (floating point timestamp)\r\n- `ctime` - the ctime (floating point timestamp)\r\n- `mtime_iso` - the mtime as an ISO datetime\r\n- `ctime_iso` - the mtime as an ISO datetime\r\n- `size` - the size of the file in bytes", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 665700495, "label": "CLI utility for inserting binary files into SQLite"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/122#issuecomment-664127741", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/122", "id": 664127741, "node_id": "MDEyOklzc3VlQ29tbWVudDY2NDEyNzc0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-27T05:29:48Z", "updated_at": "2020-07-27T05:29:48Z", "author_association": "OWNER", "body": "Test command:\r\n```\r\nsqlite-utils insert-files gifs.db *.gif \\\r\n    -c filename:filename \\\r\n    -c filepath:filepath \\\r\n    -c absolutepath:absolutepath \\\r\n    -c sha256:sha256 \\\r\n    -c md5:md5 \\\r\n    -c content:content \\\r\n    -c mtime:mtime \\\r\n    -c ctime:ctime \\\r\n    -c mtime_iso:mtime_iso \\\r\n    -c ctime_iso:ctime_iso \\\r\n    -c size:size \\\r\n    --pk absolutepath\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 665700495, "label": "CLI utility for inserting binary files into SQLite"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/122#issuecomment-664128071", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/122", "id": 664128071, "node_id": "MDEyOklzc3VlQ29tbWVudDY2NDEyODA3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-27T05:30:54Z", "updated_at": "2020-07-27T05:30:54Z", "author_association": "OWNER", "body": "Inserting files by piping them in should work - but since a filename cannot be derived this will need a `--name blah.gif` option.\r\n\r\n    cat blah.gif | sqlite-utils insert-files files.db files - --name=blah.gif", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 665700495, "label": "CLI utility for inserting binary files into SQLite"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/127#issuecomment-664163206", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/127", "id": 664163206, "node_id": "MDEyOklzc3VlQ29tbWVudDY2NDE2MzIwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-27T07:10:05Z", "updated_at": "2020-07-27T07:10:05Z", "author_association": "OWNER", "body": "I tried to get this working but it was a bit tricky because `-` doesn't behave like a regular `pathlib.Path` - needs a bit more thought on how the implementation would work.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 666040390, "label": "Ability to insert files piped to insert-files stdin"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/122#issuecomment-664163524", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/122", "id": 664163524, "node_id": "MDEyOklzc3VlQ29tbWVudDY2NDE2MzUyNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-27T07:10:41Z", "updated_at": "2020-07-27T07:10:41Z", "author_association": "OWNER", "body": "Docs: https://github.com/simonw/sqlite-utils/blob/ebc802f7ff0e640b6ae11ea525290fea0115228c/docs/cli.rst#inserting-binary-data-from-files", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 665700495, "label": "CLI utility for inserting binary files into SQLite"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/128#issuecomment-664683608", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/128", "id": 664683608, "node_id": "MDEyOklzc3VlQ29tbWVudDY2NDY4MzYwOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-27T23:09:22Z", "updated_at": "2020-07-27T23:09:22Z", "author_association": "OWNER", "body": "This seems to work, but needs more tests:\r\n```diff\r\ndiff --git a/sqlite_utils/db.py b/sqlite_utils/db.py\r\nindex d6b9ecf..ee26433 100644\r\n--- a/sqlite_utils/db.py\r\n+++ b/sqlite_utils/db.py\r\n@@ -7,6 +7,7 @@ import itertools\r\n import json\r\n import os\r\n import pathlib\r\n+import uuid\r\n \r\n SQLITE_MAX_VARS = 999\r\n \r\n@@ -40,11 +41,13 @@ COLUMN_TYPE_MAPPING = {\r\n     str: \"TEXT\",\r\n     bytes.__class__: \"BLOB\",\r\n     bytes: \"BLOB\",\r\n+    memoryview: \"BLOB\",\r\n     datetime.datetime: \"TEXT\",\r\n     datetime.date: \"TEXT\",\r\n     datetime.time: \"TEXT\",\r\n     decimal.Decimal: \"FLOAT\",\r\n     None.__class__: \"TEXT\",\r\n+    uuid.UUID: \"TEXT\",\r\n     # SQLite explicit types\r\n     \"TEXT\": \"TEXT\",\r\n     \"INTEGER\": \"INTEGER\",\r\n@@ -1336,6 +1339,8 @@ def jsonify_if_needed(value):\r\n         return json.dumps(value, default=repr)\r\n     elif isinstance(value, (datetime.time, datetime.date, datetime.datetime)):\r\n         return value.isoformat()\r\n+    elif isinstance(value, uuid.UUID):\r\n+        return str(value)\r\n     else:\r\n         return value\r\n ```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 666639051, "label": "Support UUID and memoryview types"}, "performed_via_github_app": null}