{"html_url": "https://github.com/simonw/sqlite-utils/pull/333#issuecomment-979345527", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/333", "id": 979345527, "node_id": "IC_kwDOCGYnMM46X6B3", "user": {"value": 2118708, "label": "Florents-Tselai"}, "created_at": "2021-11-25T16:31:47Z", "updated_at": "2021-11-25T16:31:47Z", "author_association": "NONE", "body": "Thanks for your reply @simonw . Tbh, my first attempt was actually the `parquet-to-sqlite` package but I already had Makefiles that relied on `SQLite-utils` and it was less intrusive to my workflow. Maybe I'll revisit that decision.\r\nFYI: there's a `[sqlite-parquet-vtable](https://github.com/cldellow/sqlite-parquet-vtable)`\r\n\r\nI don't think plugins make much sense either. Probably defeats the purpose of simplicity: simple database along with a pip-able package.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1039037439, "label": "Add functionality to read Parquet files."}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/333#issuecomment-979442854", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/333", "id": 979442854, "node_id": "IC_kwDOCGYnMM46YRym", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-25T19:47:26Z", "updated_at": "2021-11-25T19:47:26Z", "author_association": "OWNER", "body": "I just remembered that there's one other place that this could fit: as a Datasette \"insert\" plugin.\r\n\r\nThis is vaporware at the moment, but the idea is that Datasette itself could grow a mechanism for importing data, that's driven by plugins.\r\n\r\nOut of the box Datasette would be able to import CSV and CSV files, similar to `sqlite-utils insert ... --csv` - but plugins would then be able to add support for additional format such as GeoJSON or - in this case - Parquet.\r\n\r\nThe neat thing about having it as a Datasette plugin is that one plugin would enable three different ways of importing data:\r\n\r\n1. Via a new `datasette insert ...` CLI option (similar to `sqlite-utils`)\r\n2. Via a web form upload interface, where authenticated Datasette users would be able to upload files\r\n3. Via an API interface, where files could be programatically submitted to a running Datasette server\r\n\r\nI started fleshing out this idea quite a while ago but didn't make much concrete progress, maybe I should revisit it:\r\n\r\n- https://github.com/simonw/datasette/issues/1160", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1039037439, "label": "Add functionality to read Parquet files."}, "performed_via_github_app": null}