{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/17#issuecomment-624284539", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/17", "id": 624284539, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNDI4NDUzOQ==", "user": {"value": 41546558, "label": "RhetTbull"}, "created_at": "2020-05-05T20:20:05Z", "updated_at": "2020-05-05T20:20:05Z", "author_association": "CONTRIBUTOR", "body": "FYI, I've got an [issue](https://github.com/RhetTbull/osxphotos/issues/25) to make osxphotos cross-platform but it's low on my priority list. About 90% of the functionality could be done cross-platform but right now the MacOS specific stuff is embedded throughout and would take some work. Though I try to minimize it, there's sprinklings of ObjC & Applescript throughout osxphotos.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612860531, "label": "Only install osxphotos if running on macOS"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/18#issuecomment-624364557", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/18", "id": 624364557, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNDM2NDU1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-05T23:49:18Z", "updated_at": "2020-05-05T23:49:18Z", "author_association": "MEMBER", "body": "Label is `macos-latest`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612860758, "label": "Switch CI solution to GitHub Actions with a macOS runner"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/19#issuecomment-624406285", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/19", "id": 624406285, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNDQwNjI4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-06T02:10:03Z", "updated_at": "2020-05-06T02:10:03Z", "author_association": "MEMBER", "body": "Most annoying part of this is the difficulty of actually showing a photo.\r\n\r\nMaybe I need to run a local proxy that I can link to? A custom Datasette plugin perhaps?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613002220, "label": "apple-photos command should work even if upload has not run"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/2#issuecomment-615931488", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/2", "id": 615931488, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTkzMTQ4OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T19:24:02Z", "updated_at": "2020-04-18T19:24:02Z", "author_association": "MEMBER", "body": "I made a start on this last week with a https://github.com/simonw/heic-to-jpeg proxy.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533352, "label": "Ability to convert HEIC images to JPEG"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-624408220", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 624408220, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNDQwODIyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-06T02:18:47Z", "updated_at": "2020-05-06T02:18:47Z", "author_association": "MEMBER", "body": "The `apple_photos` table has an indexed `uuid` column and a `path` column which stores the full path to that photo file on disk.\r\n\r\nI can write a custom Datasette plugin which takes the `uuid` from the URL, looks up the path, then serves up a thumbnail of the jpeg or heic image file.\r\n\r\nI'll prototype this is a one-off plugin first, then package it on PyPI for other people to install.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-624408370", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 624408370, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNDQwODM3MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-06T02:19:27Z", "updated_at": "2020-05-06T02:19:27Z", "author_association": "MEMBER", "body": "The plugin can be generalized: it can be configured to know how to take the URL path, look it up in ANY table (via a custom SQL query) to get a path on disk and then serve that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-624408738", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 624408738, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNDQwODczOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-06T02:21:05Z", "updated_at": "2020-05-06T02:21:32Z", "author_association": "MEMBER", "body": "Here's rendering code from my hacked-together not-yet-released S3 image proxy:\r\n```python\r\nfrom starlette.responses import Response\r\nfrom PIL import Image, ExifTags\r\nimport pyheif\r\n\r\nfor ORIENTATION_TAG in ExifTags.TAGS.keys():\r\n if ExifTags.TAGS[ORIENTATION_TAG] == \"Orientation\":\r\n break\r\n ...\r\n # Load it into Pillow\r\n if ext == \"heic\":\r\n heic = pyheif.read_heif(image_response.content)\r\n image = Image.frombytes(mode=heic.mode, size=heic.size, data=heic.data)\r\n else:\r\n image = Image.open(io.BytesIO(image_response.content))\r\n\r\n # Does EXIF tell us to rotate it?\r\n try:\r\n exif = dict(image._getexif().items())\r\n if exif[ORIENTATION_TAG] == 3:\r\n image = image.rotate(180, expand=True)\r\n elif exif[ORIENTATION_TAG] == 6:\r\n image = image.rotate(270, expand=True)\r\n elif exif[ORIENTATION_TAG] == 8:\r\n image = image.rotate(90, expand=True)\r\n except (AttributeError, KeyError, IndexError):\r\n pass\r\n\r\n # Resize based on ?w= and ?h=, if set\r\n width, height = image.size\r\n w = request.query_params.get(\"w\")\r\n h = request.query_params.get(\"h\")\r\n if w is not None or h is not None:\r\n if h is None:\r\n # Set h based on w\r\n w = int(w)\r\n h = int((float(height) / width) * w)\r\n elif w is None:\r\n h = int(h)\r\n # Set w based on h\r\n w = int((float(width) / height) * h)\r\n w = int(w)\r\n h = int(h)\r\n image.thumbnail((w, h))\r\n\r\n # ?bw= converts to black and white\r\n if request.query_params.get(\"bw\"):\r\n image = image.convert(\"L\")\r\n\r\n # ?q= sets the quality - defaults to 75\r\n quality = 75\r\n q = request.query_params.get(\"q\")\r\n if q and q.isdigit() and 1 <= int(q) <= 100:\r\n quality = int(q)\r\n\r\n # Output as JPEG or PNG\r\n output_image = io.BytesIO()\r\n image_type = \"JPEG\"\r\n kwargs = {\"quality\": quality}\r\n if image.format == \"PNG\":\r\n image_type = \"PNG\"\r\n kwargs = {}\r\n\r\n image.save(output_image, image_type, **kwargs)\r\n return Response(\r\n output_image.getvalue(),\r\n media_type=\"image/jpeg\",\r\n headers={\"cache-control\": \"s-maxage={}, public\".format(365 * 24 * 60 * 60)},\r\n )\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-625947133", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 625947133, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNTk0NzEzMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-08T18:13:06Z", "updated_at": "2020-05-08T18:13:06Z", "author_association": "MEMBER", "body": "`datasette-media` will be able to handle this once I implement https://github.com/simonw/datasette-media/issues/3", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633234781", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 633234781, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMzIzNDc4MQ==", "user": {"value": 41439, "label": "dmd"}, "created_at": "2020-05-24T13:56:13Z", "updated_at": "2020-05-24T13:56:13Z", "author_association": "NONE", "body": "As that seems to be closed, can you give a hint on how to make this work?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633626741", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 633626741, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMzYyNjc0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-25T15:38:55Z", "updated_at": "2020-05-25T15:38:55Z", "author_association": "MEMBER", "body": "Sure, I should absolutely document this!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633629944", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 633629944, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMzYyOTk0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-25T15:47:42Z", "updated_at": "2020-05-25T15:47:42Z", "author_association": "MEMBER", "body": "I'll add a proper section to the README, but for the moment here's how I do this.\r\n\r\nFirst, install `datasette` and the `datasette-media` plugin.\r\n\r\nCreate a `metadata.yaml` file with the following content:\r\n\r\n```yaml\r\nplugins:\r\n datasette-media:\r\n photo:\r\n sql: |-\r\n select path as filepath, 200 as resize_height from apple_photos where uuid = :key\r\n photo-big:\r\n sql: |-\r\n select path as filepath, 1024 as resize_height from apple_photos where uuid = :key\r\n```\r\nNow run `datasette -m metadata.yaml photos.db` - thumbnails will be served at http://127.0.0.1:8001/-/media/photo/F4469918-13F3-43D8-9EC1-734C0E6B60AD and larger sizes of the image at http://127.0.0.1:8001/-/media/photo-big/A8B02C7D-365E-448B-9510-69F80C26304D\r\n\r\nI also made myself two custom pages, one showing recent images and one showing random images.\r\n\r\nTo do this, install the `datasette-template-sql` plugin and then create a `templates/pages` directory and add these files:\r\n\r\n`recent-photos.html`\r\n```html\r\n

Recent photos

\r\n\r\n
\r\n{% for photo in sql(\"select * from apple_photos order by date desc limit 100\") %}\r\n \r\n{% endfor %}\r\n
\r\n```\r\n\r\n`random-photos.html`\r\n```html\r\n

Random photos

\r\n\r\n
\r\n{% for photo in sql(\"with foo as (select * from apple_photos order by date desc limit 5000) select * from foo order by random() limit 100\") %}\r\n \r\n{% endfor %}\r\n
\r\n```\r\n\r\nNow run `datasette -m metadata.yaml photos.db --template-dir=templates/`\r\n\r\nVisit http://127.0.0.1:8001/random-photos to see some random photos or http://127.0.0.1:8002/recent-photos for recent photos.\r\n\r\nThis is using this mechanism: https://datasette.readthedocs.io/en/stable/custom_templates.html#custom-pages", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633643921", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 633643921, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMzY0MzkyMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-25T16:29:44Z", "updated_at": "2020-05-25T16:29:44Z", "author_association": "MEMBER", "body": "https://github.com/dogsheep/dogsheep-photos/blob/dc43fa8653cb9c7238a36f52239b91d1ec916d5c/README.md#serving-photos-locally-with-datasette-media", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633644225", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 633644225, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMzY0NDIyNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-25T16:30:44Z", "updated_at": "2020-05-25T16:30:44Z", "author_association": "MEMBER", "body": "I'll add docs on using `datasette-json-html` too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633704127", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 633704127, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMzcwNDEyNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-25T20:14:22Z", "updated_at": "2020-05-25T20:14:22Z", "author_association": "MEMBER", "body": "https://github.com/dogsheep/dogsheep-photos/blob/0.4.1/README.md#serving-photos-locally-with-datasette-media", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626388764", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21", "id": 626388764, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjM4ODc2NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-10T20:58:52Z", "updated_at": "2020-05-10T20:58:52Z", "author_association": "MEMBER", "body": "More from the debugger:\r\n```\r\n> /Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/photoinfo.py(614)place()\r\n-> self._place = PlaceInfo5(self._info[\"reverse_geolocation\"])\r\n```\r\nAnd:\r\n```\r\n> /Users/simon/Dropbox/Development/photos-to-sqlite/photos_to_sqlite/utils.py(91)osxphoto_to_row()\r\n-> place = photo.place\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615474990, "label": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626388837", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21", "id": 626388837, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjM4ODgzNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-10T20:59:32Z", "updated_at": "2020-05-10T20:59:32Z", "author_association": "MEMBER", "body": "So it appears it's possible for `photo.place` to raise that exception. A workaround could be to catch that and treat those photos as not having a place.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615474990, "label": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626390317", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21", "id": 626390317, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjM5MDMxNw==", "user": {"value": 41546558, "label": "RhetTbull"}, "created_at": "2020-05-10T21:11:24Z", "updated_at": "2020-05-10T21:50:58Z", "author_association": "CONTRIBUTOR", "body": "Ugh....Yeah, I think easiest is to catch the exception and return no place as you suggest. This particular bit of code involves un-archiving a serialized NSKeyedArchiver which uses an object table and it is certainly possible to create a circular reference that way. Because this is happening in the decode, the circular reference must be in the original data. Does Photos show valid reverse geolocation info for the photo in question? If so, Photos may be doing something beyond a simple decode of the binary plist. For now, I'll push a patch to catch the exception.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615474990, "label": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626394989", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21", "id": 626394989, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjM5NDk4OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-10T21:50:36Z", "updated_at": "2020-05-10T21:50:36Z", "author_association": "MEMBER", "body": "https://github.com/Marketcircle/bpylist/pull/2 looks relevant here.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615474990, "label": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626395103", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21", "id": 626395103, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjM5NTEwMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-10T21:51:36Z", "updated_at": "2020-05-10T21:51:36Z", "author_association": "MEMBER", "body": "@RhetTbull I tried that workaround and it turns out I'm getting this error on ALL of my photos now!\r\n\r\nIt's weird: a few day ago this wasn't happening. Now it's happening to everything. I'm not sure what I might have changed. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615474990, "label": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626395209", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21", "id": 626395209, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjM5NTIwOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-10T21:52:42Z", "updated_at": "2020-05-10T21:52:42Z", "author_association": "MEMBER", "body": "Aha! It looks like I accidentally installed the old bplist into the same environment:\r\n```\r\n$ pip freeze | grep bpylist\r\nbpylist==0.1.4\r\nbpylist2==3.0.0\r\n```", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615474990, "label": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626395507", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21", "id": 626395507, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjM5NTUwNw==", "user": {"value": 41546558, "label": "RhetTbull"}, "created_at": "2020-05-10T21:54:45Z", "updated_at": "2020-05-10T21:54:45Z", "author_association": "CONTRIBUTOR", "body": "@simonw does Photos show valid reverse geolocation info? Are you sure you're using [bpylist2](https://github.com/xa4a/bpylist2) and not bpylist? They're both unfortunately imported as \"bpylist\" so if you somehow got the wrong (original bpylist) version installed, it could be the issue. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615474990, "label": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626395641", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21", "id": 626395641, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjM5NTY0MQ==", "user": {"value": 41546558, "label": "RhetTbull"}, "created_at": "2020-05-10T21:55:54Z", "updated_at": "2020-05-10T21:55:54Z", "author_association": "CONTRIBUTOR", "body": "Did removing old bpylist solve the original problem or do you still have a photo that throws circular reference?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615474990, "label": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626395781", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21", "id": 626395781, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjM5NTc4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-10T21:57:09Z", "updated_at": "2020-05-10T21:57:09Z", "author_association": "MEMBER", "body": "Yes, I just recreated my virtual environment from scratch and the error went away.\r\n\r\nThe problem occurred when I ran `pip install datasette-bplist` in the same virtual environment - https://github.com/simonw/datasette-bplist/blob/master/setup.py depends on `bpylist` which is incompatible with `bpylist2`.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615474990, "label": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626396379", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21", "id": 626396379, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjM5NjM3OQ==", "user": {"value": 41546558, "label": "RhetTbull"}, "created_at": "2020-05-10T22:01:48Z", "updated_at": "2020-05-10T22:01:48Z", "author_association": "CONTRIBUTOR", "body": "Frustrates me when package authors create a \"drop in\" replacement with the same import name...this kind of thing has bitten me more than once! Would've been nicer I think for bpylist2 to do \"import bpylist2 as bpylist\"", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615474990, "label": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-748436195", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21", "id": 748436195, "node_id": "MDEyOklzc3VlQ29tbWVudDc0ODQzNjE5NQ==", "user": {"value": 8573886, "label": "nickvazz"}, "created_at": "2020-12-19T07:44:32Z", "updated_at": "2020-12-19T07:44:49Z", "author_association": "NONE", "body": "I have also run into this a bit, would it be possible to post your `requirements.txt` so I can try and reproduce your [blog post](https://simonwillison.net/2020/May/21/dogsheep-photos/)?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615474990, "label": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/22#issuecomment-626667235", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/22", "id": 626667235, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjY2NzIzNQ==", "user": {"value": 41546558, "label": "RhetTbull"}, "created_at": "2020-05-11T12:20:34Z", "updated_at": "2020-05-11T12:20:34Z", "author_association": "CONTRIBUTOR", "body": "@simonw FYI, osxphotos includes a built in ExifTool class that uses [exiftool](https://exiftool.org/) to read and write exif data. It's not exposed yet in the docs because I really only use it right now in the osphotos command line interface to write tags when exporting. In v0.28.16 (just pushed) I added an ExifTool.as_dict() method which will give you a dict with all the exif tags in a file. For example:\r\n\r\n```python\r\nimport osxphotos\r\nphotos = osxphotos.PhotosDB().photos()\r\nexiftool = osxphotos.exiftool.ExifTool(photos[0].path)\r\nexifdata = exiftool.as_dict()\r\ntags = exifdata[\"IPTC:Keywords\"]\r\n```\r\n\r\nNot as elegant perhaps as a python only implementation because ExifTool has to make subprocess calls to an external tool but exiftool is by far the best tool available for reading and writing EXIF data and it does support HEIC.\r\n\r\nAs for implementation, ExifTool uses a singleton pattern so the first time you instantiate it, it spawns an IPC to exiftool but then keeps it open and uses the same process for any subsequent calls (even on different files). ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615626118, "label": "Try out ExifReader"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/22#issuecomment-626941278", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/22", "id": 626941278, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNjk0MTI3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-11T20:25:58Z", "updated_at": "2020-05-11T20:25:58Z", "author_association": "MEMBER", "body": "Interesting - do you know if there's anything the `exiftool` process handles that `ExifReader` doesn't?\r\n\r\nI'm actually just going to extract a subset of the EXIF data at first - since the original photo files will always be available I don't feel the need to get everything out for the first step.\r\n\r\nMy plan is to use EXIF to help support photo collections that aren't in Apple Photos - I'm going to build a database table keyed by the `sha256` of each photo that extracts the camera make, lens, a few settings (ISO, aperture etc) and the GPS lat/lon.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615626118, "label": "Try out ExifReader"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/22#issuecomment-627007458", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/22", "id": 627007458, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNzAwNzQ1OA==", "user": {"value": 41546558, "label": "RhetTbull"}, "created_at": "2020-05-11T22:51:52Z", "updated_at": "2020-05-11T22:52:26Z", "author_association": "CONTRIBUTOR", "body": "I'm not familiar with `ExifReader`. I wrote my own wrapper around `exiftool` because I wanted a simple way to write EXIF data when exporting photos (e.g. writing out to PersonInImage and keywords to IPTC:Keywords) and the existing python packages like [pyexiftool](https://github.com/smarnach/pyexiftool) didn't do quite what I wanted. If all you're after is the camera and shot info, that's available in `ZEXTENDEDATTRIBUTES` table. I've got an open issue [#11](https://github.com/RhetTbull/osxphotos/issues/11) to add this to osxphotos but it hasn't bubbled to the top of my backlog yet. \r\n\r\nosxphotos will give you the location info: `PhotoInfo.location` returns a tuple of (lat, lon) though this info is in ZEXTENDEDATTRIBUTES too (though it might not be correct as I believe Photos creates this table at import and the user might have changed the location of a photo, e.g. if camera didn't have GPS).\r\n\r\n```sql\r\nCREATE TABLE ZEXTENDEDATTRIBUTES (\r\n Z_PK INTEGER PRIMARY KEY, Z_ENT INTEGER, \r\n Z_OPT INTEGER, ZFLASHFIRED INTEGER, \r\n ZISO INTEGER, ZMETERINGMODE INTEGER, \r\n ZSAMPLERATE INTEGER, ZTRACKFORMAT INTEGER, \r\n ZWHITEBALANCE INTEGER, ZASSET INTEGER, \r\n ZAPERTURE FLOAT, ZBITRATE FLOAT, ZDURATION FLOAT, \r\n ZEXPOSUREBIAS FLOAT, ZFOCALLENGTH FLOAT, \r\n ZFPS FLOAT, ZLATITUDE FLOAT, ZLONGITUDE FLOAT, \r\n ZSHUTTERSPEED FLOAT, ZCAMERAMAKE VARCHAR, \r\n ZCAMERAMODEL VARCHAR, ZCODEC VARCHAR, \r\n ZLENSMODEL VARCHAR\r\n);\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615626118, "label": "Try out ExifReader"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/22#issuecomment-628405453", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/22", "id": 628405453, "node_id": "MDEyOklzc3VlQ29tbWVudDYyODQwNTQ1Mw==", "user": {"value": 41546558, "label": "RhetTbull"}, "created_at": "2020-05-14T05:59:53Z", "updated_at": "2020-05-14T05:59:53Z", "author_association": "CONTRIBUTOR", "body": "I've added support for the above exif data to [v0.28.17](https://github.com/RhetTbull/osxphotos/releases/tag/v0.28.17) of osxphotos. `PhotoInfo.exif_info` will return an `ExifInfo` [dataclass](https://docs.python.org/3/library/dataclasses.html) object with the following properties:\r\n\r\n```python\r\n flash_fired: bool\r\n iso: int\r\n metering_mode: int\r\n sample_rate: int\r\n track_format: int\r\n white_balance: int\r\n aperture: float\r\n bit_rate: float\r\n duration: float\r\n exposure_bias: float\r\n focal_length: float\r\n fps: float\r\n latitude: float\r\n longitude: float\r\n shutter_speed: float\r\n camera_make: str\r\n camera_model: str\r\n codec: str\r\n lens_model: str\r\n```\r\n\r\nIt's not all the EXIF data available in most files but is the data Photos deems important to save. Of course, you can get all the exif_data\r\n\r\nNote: this only works in Photos 5. As best as I can tell, EXIF data is not stored in the database for earlier versions. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 615626118, "label": "Try out ExifReader"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/23#issuecomment-631120771", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/23", "id": 631120771, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTEyMDc3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-19T22:32:48Z", "updated_at": "2020-05-19T22:32:48Z", "author_association": "MEMBER", "body": "Documentation: https://github.com/dogsheep/photos-to-sqlite/blob/e2fab012551eed05278040b5d57e7373a1b9a0bf/README.md#creating-a-subset-database", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621280529, "label": "create-subset command for creating a publishable subset of a photos database"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/24#issuecomment-631255206", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/24", "id": 631255206, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTI1NTIwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T06:00:25Z", "updated_at": "2020-05-20T06:00:25Z", "author_association": "MEMBER", "body": "This needs documentation.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621323348, "label": "Configurable URL for images"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/25#issuecomment-631127454", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25", "id": 631127454, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTEyNzQ1NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-19T22:48:00Z", "updated_at": "2020-05-21T15:58:32Z", "author_association": "MEMBER", "body": "I built #23 to help with this.\r\n\r\n $ dogsheep-photos create-subset photos.db public.db \\\r\n \"select sha256 from apple_photos where albums like '%Public%'\"\r\n\r\nAnd publish with Vercel:\r\n\r\n $ datasette publish now public.db --project dogsheep-photos \\\r\n --about=dogsheep/dogsheep-photos \\\r\n --about_url=\"https://github.com/dogsheep/dogsheep-photos\" \\\r\n --install=datasette-json-html \\\r\n --install=datasette-cluster-map", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621332242, "label": "Create a public demo"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/25#issuecomment-631251707", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25", "id": 631251707, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTI1MTcwNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T05:49:27Z", "updated_at": "2020-05-21T15:58:42Z", "author_association": "MEMBER", "body": "Renaming this demo to `dogsheep-photos.dogsheep.net`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621332242, "label": "Create a public demo"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/25#issuecomment-631253136", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25", "id": 631253136, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTI1MzEzNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T05:53:58Z", "updated_at": "2020-05-20T05:53:58Z", "author_association": "MEMBER", "body": "Updated deploy command:\r\n```\r\ndatasette publish now public.db --project dogsheep-photos \\\r\n --about=dogsheep/dogsheep-photos \\\r\n --about_url=\"https://github.com/dogsheep/dogsheep-photos\" \\\r\n --install=datasette-json-html \\\r\n --install=datasette-cluster-map \\\r\n --title \"Dogsheep Photos demo\"\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621332242, "label": "Create a public demo"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/25#issuecomment-631253248", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25", "id": 631253248, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTI1MzI0OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T05:54:18Z", "updated_at": "2020-05-20T05:54:18Z", "author_association": "MEMBER", "body": "https://dogsheep-photos.dogsheep.net/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621332242, "label": "Create a public demo"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/25#issuecomment-631253852", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25", "id": 631253852, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTI1Mzg1Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T05:56:17Z", "updated_at": "2020-05-21T22:26:16Z", "author_association": "MEMBER", "body": "I have a `deploy-demo.sh` script now:\r\n```bash\r\n#!/bin/bash\r\nif [ -f public.db ]; then\r\n rm public.db\r\nfi\r\npipenv run dogsheep-photos create-subset photos.db public.db \\\r\n \"select sha256 from apple_photos where albums like '%Public%'\"\r\npipenv run sqlite-utils create-view public.db photos_on_a_map \\\r\n \"select\r\n date,\r\n latitude,\r\n longitude,\r\n apple_photos.sha256,\r\n uploads.ext,\r\n json_object(\r\n 'title',\r\n 'Taken on ' || date,\r\n 'image',\r\n 'https://photos.simonwillison.net/i/' || uploads.sha256 || '.' || uploads.ext || '?w=400',\r\n 'link',\r\n 'https://photos.simonwillison.net/i/' || uploads.sha256 || '.' || uploads.ext || '?w=1200'\r\n ) as popup\r\n from\r\n apple_photos\r\n join uploads on apple_photos.sha256 = uploads.sha256\r\n where\r\n latitude is not null\r\n order by\r\n date desc\" \\\r\n --replace\r\npipenv run datasette publish now public.db --project dogsheep-photos \\\r\n --about=dogsheep/dogsheep-photos \\\r\n --about_url=\"https://github.com/dogsheep/dogsheep-photos\" \\\r\n --install=datasette-json-html \\\r\n --install=datasette-pretty-json \\\r\n --install=datasette-cluster-map>=0.10 \\\r\n --title \"Dogsheep Photos demo\"\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621332242, "label": "Create a public demo"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631226481", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26", "id": 631226481, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTIyNjQ4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T04:18:29Z", "updated_at": "2020-05-20T04:18:29Z", "author_association": "MEMBER", "body": "I just renamed the repository.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621444763, "label": "Rename project to dogsheep-photos"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631226572", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26", "id": 631226572, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTIyNjU3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T04:18:52Z", "updated_at": "2020-05-20T04:18:52Z", "author_association": "MEMBER", "body": "Need to reconfigure Circle CI.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621444763, "label": "Rename project to dogsheep-photos"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631226953", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26", "id": 631226953, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTIyNjk1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T04:20:34Z", "updated_at": "2020-05-20T04:20:34Z", "author_association": "MEMBER", "body": "Huh, it looks like Circle CI picked up the name change automatically. https://app.circleci.com/pipelines/github/dogsheep/dogsheep-photos", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621444763, "label": "Rename project to dogsheep-photos"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631227020", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26", "id": 631227020, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTIyNzAyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T04:20:48Z", "updated_at": "2020-05-20T04:21:16Z", "author_association": "MEMBER", "body": "Next time I push a release it will create `dogsheep-photos` on PyPI.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621444763, "label": "Rename project to dogsheep-photos"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631227105", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26", "id": 631227105, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTIyNzEwNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T04:21:06Z", "updated_at": "2020-05-20T04:21:06Z", "author_association": "MEMBER", "body": "Then I just need to push a final photos-to-sqlite release that updates the README to tell people about the name change.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621444763, "label": "Rename project to dogsheep-photos"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631227245", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26", "id": 631227245, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTIyNzI0NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T04:21:38Z", "updated_at": "2020-05-20T04:21:38Z", "author_association": "MEMBER", "body": "I'm going to release 0.4 now.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621444763, "label": "Rename project to dogsheep-photos"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631229409", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26", "id": 631229409, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTIyOTQwOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T04:30:40Z", "updated_at": "2020-05-20T04:30:40Z", "author_association": "MEMBER", "body": "https://pypi.org/project/photos-to-sqlite/ now links to dogsheep-photos.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621444763, "label": "Rename project to dogsheep-photos"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631229485", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26", "id": 631229485, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMTIyOTQ4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-20T04:31:02Z", "updated_at": "2020-05-20T04:31:02Z", "author_association": "MEMBER", "body": "https://pypi.org/project/dogsheep-photos/ is live.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 621444763, "label": "Rename project to dogsheep-photos"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/28#issuecomment-751125270", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/28", "id": 751125270, "node_id": "MDEyOklzc3VlQ29tbWVudDc1MTEyNTI3MA==", "user": {"value": 129786, "label": "jmelloy"}, "created_at": "2020-12-24T22:26:22Z", "updated_at": "2020-12-24T22:26:22Z", "author_association": "NONE", "body": "This comes around if you\u2019ve run the photo export without running an s3 upload. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 624490929, "label": "Invalid SQL no such table: main.uploads"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/pull/29#issuecomment-739058820", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/29", "id": 739058820, "node_id": "MDEyOklzc3VlQ29tbWVudDczOTA1ODgyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-12-04T22:32:35Z", "updated_at": "2020-12-04T22:32:35Z", "author_association": "MEMBER", "body": "Thanks for this!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638375985, "label": "Fixed bug in SQL query for photo scores"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/3#issuecomment-934207940", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/3", "id": 934207940, "node_id": "IC_kwDOD079W843ruHE", "user": {"value": 1751612, "label": "jratike80"}, "created_at": "2021-10-05T08:57:41Z", "updated_at": "2021-10-05T08:57:41Z", "author_association": "NONE", "body": "Maybe the exif-loader from the SpatiaLite project could be useful as a reference even it is written in C and it also saves images as blobs https://www.gaia-gis.it/fossil/spatialite-tools/file?name=exif_loader.c&ci=tip. The tool is also integrated into the spatialite-gui application.\r\n\r\nI found some user documentation from the web archive http://web.archive.org/web/20180629041238/https://www.gaia-gis.it/spatialite-2.3.1/spatialite-exif-2.3.1.html.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533481, "label": "Import EXIF data into SQLite - lens used, ISO, aperture etc"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/3#issuecomment-934372104", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/3", "id": 934372104, "node_id": "IC_kwDOD079W843sWMI", "user": {"value": 41546558, "label": "RhetTbull"}, "created_at": "2021-10-05T12:38:24Z", "updated_at": "2021-10-05T12:38:24Z", "author_association": "CONTRIBUTOR", "body": "As dogsheep-photos already uses [osxphotos](https://github.com/RhetTbull/osxphotos) to load photos you can access the EXIF data via osxphotos. Apple Photos imports a small subset of EXIF data at the time the photo is imported and osxphotos provides this via the [exif_info](https://github.com/RhetTbull/osxphotos#exifinfo) property. If you want the full EXIF data, osxphotos also provides a wrapper around [exiftool](https://github.com/RhetTbull/osxphotos#exiftool).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533481, "label": "Import EXIF data into SQLite - lens used, ISO, aperture etc"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-748562330", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31", "id": 748562330, "node_id": "MDEyOklzc3VlQ29tbWVudDc0ODU2MjMzMA==", "user": {"value": 41546558, "label": "RhetTbull"}, "created_at": "2020-12-20T04:45:08Z", "updated_at": "2020-12-20T04:45:08Z", "author_association": "CONTRIBUTOR", "body": "Fixes the issue mentioned here: https://github.com/dogsheep/dogsheep-photos/issues/15#issuecomment-748436115", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 1, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 771511344, "label": "Update for Big Sur"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-811362316", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31", "id": 811362316, "node_id": "MDEyOklzc3VlQ29tbWVudDgxMTM2MjMxNg==", "user": {"value": 871250, "label": "PabloLerma"}, "created_at": "2021-03-31T19:14:39Z", "updated_at": "2021-03-31T19:14:39Z", "author_association": "NONE", "body": "\ud83d\udc4b could I help somehow for this to be merged? As Big Sur is going to be more used as the time goes I think it would be nice to merge and publish a new version. Nice work!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 771511344, "label": "Update for Big Sur"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-1035717429", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31", "id": 1035717429, "node_id": "IC_kwDOD079W849u8s1", "user": {"value": 18504, "label": "harperreed"}, "created_at": "2022-02-11T01:55:38Z", "updated_at": "2022-02-11T01:55:38Z", "author_association": "NONE", "body": "I would love this merged! ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 771511344, "label": "Update for Big Sur"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-1190995982", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31", "id": 1190995982, "node_id": "IC_kwDOD079W85G_SgO", "user": {"value": 19231792, "label": "jakewilkins"}, "created_at": "2022-07-21T03:26:38Z", "updated_at": "2023-04-14T22:41:31Z", "author_association": "NONE", "body": "\ud83d\udc4b Any update on getting this merged?\r\n\r\nAlternatively, is there a work around for this issue to unblock myself?\r\n\r\nedit to add: huge fan of both this project and `osxphotos`, thanks so much for your work here \ud83d\ude4f If I had any experience with Python I would offer to help but somehow I've managed to not write any Python in 10+ years of programming \ud83d\ude05 \r\n\r\nEdit again to add:\r\n\r\n> Alternatively, is there a work around for this issue to unblock myself?\r\n\r\nYes, there is. I was able to apply the patch of this PR and it applies (mostly) cleanly and works.\r\n\r\n- verified I have a high enough version of `osxphotos`\r\n- downloaded the .patch of this (by appending `.patch` to the URL)\r\n- edited the patch to remove the `setup.py` changes\r\n- `cd` to the directory containing `dogsheep-photos` and `git apply 31.patch`\r\n", "reactions": "{\"total_count\": 3, \"+1\": 3, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 771511344, "label": "Update for Big Sur"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-1382655354", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31", "id": 1382655354, "node_id": "IC_kwDOD079W85SaaV6", "user": {"value": 2704860, "label": "fidiego"}, "created_at": "2023-01-14T04:08:36Z", "updated_at": "2023-01-14T04:08:36Z", "author_association": "NONE", "body": "I just tried this branch and saw some errors. \r\n\r\nI installed this PR locally with:\r\n\r\n```bash\r\npip install https://github.com/RhetTbull/dogsheep-photos/archive/update_for_bigsur.zip\r\n```\r\n\r\n
\r\nSystem Details\r\n\r\n**OS:** MacOS Monterey\r\n**Python Version:** Python 3.10.8\r\n\r\n
\r\n\r\n
\r\nStacktrace\r\n\r\n```python\r\nTraceback (most recent call last):\r\n File \"/Users/df/.venvs/photo-experiments/bin/dogsheep-photos\", line 8, in \r\n sys.exit(cli())\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py\", line 1130, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py\", line 1055, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py\", line 1657, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py\", line 1404, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py\", line 760, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/dogsheep_photos/cli.py\", line 254, in apple_photos\r\n sha256 = calculate_hash(pathlib.Path(photo.path))\r\n File \"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py\", line 960, in __new__\r\n self = cls._from_parts(args)\r\n File \"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py\", line 594, in _from_parts\r\n drv, root, parts = self._parse_args(args)\r\n File \"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py\", line 578, in _parse_args\r\n a = os.fspath(a)\r\nTypeError: expected str, bytes or os.PathLike object, not NoneType\r\n```\r\n
", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 771511344, "label": "Update for Big Sur"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/32#issuecomment-791053721", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/32", "id": 791053721, "node_id": "MDEyOklzc3VlQ29tbWVudDc5MTA1MzcyMQ==", "user": {"value": 6213, "label": "dsisnero"}, "created_at": "2021-03-05T00:31:27Z", "updated_at": "2021-03-05T00:31:27Z", "author_association": "NONE", "body": "I am getting the same thing for US West (N. California) us-west-1", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 803333769, "label": "KeyError: 'Contents' on running upload"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/32#issuecomment-882091516", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/32", "id": 882091516, "node_id": "IC_kwDOD079W840k6X8", "user": {"value": 10793464, "label": "aaronyih1"}, "created_at": "2021-07-18T17:29:39Z", "updated_at": "2021-07-18T17:33:02Z", "author_association": "NONE", "body": "Same here for US West (N. California) us-west-1. Running on Catalina.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 803333769, "label": "KeyError: 'Contents' on running upload"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/32#issuecomment-884688833", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/32", "id": 884688833, "node_id": "IC_kwDOD079W840u0fB", "user": {"value": 10793464, "label": "aaronyih1"}, "created_at": "2021-07-22T06:40:25Z", "updated_at": "2021-07-22T06:40:25Z", "author_association": "NONE", "body": "The solution here is to upload an image to the bucket first. It is caused because it does not properly handle the case when there are no images in the bucket.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 803333769, "label": "KeyError: 'Contents' on running upload"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/33#issuecomment-777951854", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/33", "id": 777951854, "node_id": "MDEyOklzc3VlQ29tbWVudDc3Nzk1MTg1NA==", "user": {"value": 675335, "label": "leafgarland"}, "created_at": "2021-02-12T03:54:39Z", "updated_at": "2021-02-12T03:54:39Z", "author_association": "NONE", "body": "I think that is a typo in the docs, you can use\r\n\r\n > dogsheep-photos apple-photos photos.db", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 803338729, "label": "photo-to-sqlite: command not found"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/33#issuecomment-778002092", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/33", "id": 778002092, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODAwMjA5Mg==", "user": {"value": 11855322, "label": "robmarkcole"}, "created_at": "2021-02-12T06:19:32Z", "updated_at": "2021-02-12T06:19:32Z", "author_association": "NONE", "body": "hi @leafgarland that results in a new error:\r\n```\r\n(venv) (base) Robins-MacBook:datasette robin$ dogsheep-photos apple-photos photos.db\r\nTraceback (most recent call last):\r\n File \"/Users/robin/datasette/venv/bin/dogsheep-photos\", line 8, in \r\n sys.exit(cli())\r\n File \"/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py\", line 1259, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/Users/robin/datasette/venv/lib/python3.8/site-packages/dogsheep_photos/cli.py\", line 206, in apple_photos\r\n db.conn.execute(\r\nsqlite3.OperationalError: no such table: attached.ZGENERICASSET\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 803338729, "label": "photo-to-sqlite: command not found"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/33#issuecomment-778014990", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/33", "id": 778014990, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODAxNDk5MA==", "user": {"value": 675335, "label": "leafgarland"}, "created_at": "2021-02-12T06:54:14Z", "updated_at": "2021-02-12T06:54:14Z", "author_association": "NONE", "body": "Ahh, that might be because macOS Big Sur has changed the structure of the photos db. Might need to wait for a later release, there is a PR which adds support for Big Sur. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 803338729, "label": "photo-to-sqlite: command not found"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/33#issuecomment-778246347", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/33", "id": 778246347, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODI0NjM0Nw==", "user": {"value": 41546558, "label": "RhetTbull"}, "created_at": "2021-02-12T15:00:43Z", "updated_at": "2021-02-12T15:00:43Z", "author_association": "CONTRIBUTOR", "body": "Yes, Big Sur Photos database doesn't have `ZGENERICASSET` table. PR #31 will fix this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 803338729, "label": "photo-to-sqlite: command not found"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/35#issuecomment-813249000", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/35", "id": 813249000, "node_id": "MDEyOklzc3VlQ29tbWVudDgxMzI0OTAwMA==", "user": {"value": 1151557, "label": "ligurio"}, "created_at": "2021-04-05T07:37:57Z", "updated_at": "2021-04-05T07:37:57Z", "author_association": "NONE", "body": "There are trained ML models used in Photoprism:\r\n- https://dl.photoprism.org/tensorflow/nasnet.zip\r\n- https://dl.photoprism.org/tensorflow/nsfw.zip", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 842695374, "label": "Support to annotate photos on other than macOS OSes"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/pull/36#issuecomment-1006708046", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/36", "id": 1006708046, "node_id": "IC_kwDOD079W848ASVO", "user": {"value": 71983, "label": "scoates"}, "created_at": "2022-01-06T16:04:46Z", "updated_at": "2022-01-06T16:04:46Z", "author_association": "NONE", "body": "This one got me, today, too. \ud83d\udc4d", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 988493790, "label": "Correct naming of tool in readme"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932007", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615932007, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTkzMjAwNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T19:27:55Z", "updated_at": "2020-04-18T19:27:55Z", "author_association": "MEMBER", "body": "Research thread: https://twitter.com/simonw/status/1249049694984011776\r\n> I want to build some software that lets people store their own data in their own S3 bucket, but if possible I'd like not to have to teach people the incantations needed to get their bucket setup and minimum-permission credentials figures out\r\n\r\nhttps://testdriven.io/blog/storing-django-static-and-media-files-on-amazon-s3/ looks useful", "reactions": "{\"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932204", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615932204, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTkzMjIwNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T19:29:22Z", "updated_at": "2020-04-18T19:34:44Z", "author_association": "MEMBER", "body": "I'm going to call my bucket `dogsheep-photos-simon`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615933273", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615933273, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTkzMzI3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T19:37:33Z", "updated_at": "2020-04-18T19:37:33Z", "author_association": "MEMBER", "body": "https://console.aws.amazon.com/s3/bucket/create?region=us-west-1\r\n\r\n![S3_Management_Console](https://user-images.githubusercontent.com/9599/79669552-33e2a380-8171-11ea-9ab5-5785d34f652a.png)\r\n\r\nI created it with no public read-write access. I plan to use signed URLs via a transforming proxy to access images for display on the web.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615935577", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615935577, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTkzNTU3Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T19:54:59Z", "updated_at": "2020-04-18T19:55:30Z", "author_association": "MEMBER", "body": "Creating IAM groups called `dogsheep-photos-simon-read-write` and `dogsheep-photos-simon-read`: https://console.aws.amazon.com/iam/home#/groups - I created them with no attached policies.\r\n\r\nNow I can attach an \"inline policy\" to each one. For the read-write group I go here:\r\n\r\nhttps://console.aws.amazon.com/iam/home#/groups/dogsheep-photos-simon-read-write\r\n\r\n![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669703-2d086080-8172-11ea-9597-83e0b155193e.png)\r\n\r\nExample policies are here: https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html\r\n\r\nFor the read-write one I went with:\r\n```json\r\n{\r\n \"Version\": \"2012-10-17\",\r\n \"Statement\": [\r\n {\r\n \"Effect\": \"Allow\",\r\n \"Action\": \"s3:*\",\r\n \"Resource\": [\r\n \"arn:aws:s3:::dogsheep-photos-simon/*\"\r\n ]\r\n }\r\n ]\r\n}\r\n```\r\nFor the read-only policy I'm going to guess that this is appropriate:\r\n\r\n```json\r\n{\r\n \"Version\": \"2012-10-17\",\r\n \"Statement\": [\r\n {\r\n \"Effect\": \"Allow\",\r\n \"Action\": [\r\n \"s3:GetObject*\",\r\n \"s3:ListBucket\"\r\n ],\r\n \"Resource\": [\r\n \"arn:aws:s3:::dogsheep-photos-simon/*\"\r\n ]\r\n }\r\n ]\r\n}\r\n```\r\nI tried the policy simulator to test this out: https://policysim.aws.amazon.com/home/index.jsp?#groups/dogsheep-photos-simon-read - this worked:\r\n\r\n![IAM_Policy_Simulator](https://user-images.githubusercontent.com/9599/79669893-cd12b980-8173-11ea-8dfb-5660ce3652da.png)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615936880", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615936880, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTkzNjg4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T20:04:31Z", "updated_at": "2020-04-18T20:04:31Z", "author_association": "MEMBER", "body": "Next step: create two IAM users, one for each of those groups.\r\n\r\nhttps://console.aws.amazon.com/iam/home#/users$new?step=details\r\n\r\n![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669931-1bc05380-8174-11ea-9657-0e0c6a692d42.png)\r\n\r\n![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669941-27137f00-8174-11ea-8ce7-249f0d4f96f6.png)\r\n\r\nI copied the keys into a secure note in 1password.\r\n\r\nCouldn't get into Transmit with them though! https://library.panic.com/transmit/transmit5/iam-roles/ may help.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615941746", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615941746, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk0MTc0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T20:29:36Z", "updated_at": "2020-04-18T20:29:36Z", "author_association": "MEMBER", "body": "I'm going to create another user just for Transmit, with full S3 access.\r\n\r\nname: `dogsheep-photos-simon-s3-all-access`\r\n\r\nRather than creating a group for that user, I'm trying the \"Attach existing policies directly\" option:\r\n\r\n![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79670182-03513880-8176-11ea-811a-c80aefb4538a.png)\r\n\r\nThat user DID work with Transmit. I uploaded a test HEIC image. I used Transmit to copy a signed URL for it.\r\n\r\n```\r\n~ $ curl -i 'https://dogsheep-photos-simon.s3.us-west-1.amazonaws.com/IMG_7195.HEIC?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAWXFXAI...' | head -n 100\r\n % Total % Received % Xferd Average Speed Time Time Time Current\r\n Dload Upload Total Spent Left Speed\r\n 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 200 OK\r\nx-amz-id-2: gBOCYqZfbNAnv0R/uJ++qm2NbW5SgD4TapgF9RQjzzeDIThcCz/BkKU+YoxlG4NJHlcmMgAHyh4=\r\nx-amz-request-id: C2FE7FCC3BD53A84\r\nDate: Sat, 18 Apr 2020 20:28:54 GMT\r\nLast-Modified: Sat, 18 Apr 2020 20:13:49 GMT\r\nETag: \"fe3e081239a123ef745517878c53b854\"\r\nAccept-Ranges: bytes\r\nContent-Type: image/heic\r\nContent-Length: 1913097\r\nServer: AmazonS3\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615942116", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615942116, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk0MjExNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T20:30:56Z", "updated_at": "2020-04-18T20:30:56Z", "author_association": "MEMBER", "body": "Next step: attempt a programmatic upload using the `dogsheep-photos-simon-read-write` credentials from a Jupyter notebook.\r\n\r\nAlso attempt a programmatic bucket listing and read using `dogsheep-photos-simon-read` credentials.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615944806", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615944806, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk0NDgwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T20:41:39Z", "updated_at": "2020-04-18T20:41:39Z", "author_association": "MEMBER", "body": "This worked!\r\n\r\n![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670712-d868e380-8179-11ea-82a5-5dfd17356113.png)\r\n\r\nAnd this worked:\r\n\r\n![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670777-50370e00-817a-11ea-83cd-18ebf5702878.png)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615945056", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615945056, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk0NTA1Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T20:42:41Z", "updated_at": "2020-04-18T20:42:41Z", "author_association": "MEMBER", "body": "But... `list_objects` failed for both of my keys (read and write):\r\n\r\n![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670798-75c41780-817a-11ea-9907-2cbc4a2e497c.png)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615946537", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615946537, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk0NjUzNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T20:48:13Z", "updated_at": "2020-04-18T20:48:13Z", "author_association": "MEMBER", "body": "How about generating a signed URL?\r\n```python\r\nread_client.generate_presigned_url(\r\n \"get_object\",\r\n Params={\r\n \"Bucket\": \"dogsheep-photos-simon\",\r\n \"Key\": \"this_is_fine.jpg\",\r\n },\r\n ExpiresIn=600\r\n)\r\n```\r\nGave me https://dogsheep-photos-simon.s3.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398\r\n\r\nWhich does this:\r\n\r\n```\r\n~ $ curl -i 'https://dogsheep-photos-simon.s3.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398'\r\nHTTP/1.1 307 Temporary Redirect\r\nx-amz-bucket-region: us-west-1\r\nx-amz-request-id: E78CD859AEE21D33\r\nx-amz-id-2: 648mx+1+YSGga7NDOU7Q6isfsKnEPWOLC+DI4+x2o9FCc6pSCdIaoHJUbFMI8Vsuh1ADtx46ymU=\r\nLocation: https://dogsheep-photos-simon.s3-us-west-1.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398\r\nContent-Type: application/xml\r\nTransfer-Encoding: chunked\r\nDate: Sat, 18 Apr 2020 20:47:21 GMT\r\nServer: AmazonS3\r\n\r\n\r\nTemporaryRedirectPlease re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.dogsheep-photos-simon.s3-us-west-1.amazonaws.comdogsheep-photos-simonE78CD859AEE21D33648mx+1+YSGga7NDOU7Q6isfsKnEPWOLC+DI4+x2o9FCc6pSCdIaoHJUbFMI8Vsuh1ADtx46ymU=~ $ \r\n```\r\nSo it redirects to another URL... which returns this:\r\n```\r\n~ $ curl -i 'https://dogsheep-photos-simon.s3-us-west-1.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398'\r\nHTTP/1.1 200 OK\r\nx-amz-id-2: XafOl6mswj3yz0GJC9+Ptot1ll5sROVwqsMc10CUUfgpaUANTdIx2GhnONb5d1GVFJ6wlS2j3UY=\r\nx-amz-request-id: 258387C180411AFE\r\nDate: Sat, 18 Apr 2020 20:47:52 GMT\r\nLast-Modified: Sat, 18 Apr 2020 20:37:35 GMT\r\nETag: \"ee04081c3182a44a1c6944e94012e977\"\r\nAccept-Ranges: bytes\r\nContent-Type: binary/octet-stream\r\nContent-Length: 53072\r\nServer: AmazonS3\r\n\r\n????JFIF??C\r\n```\r\nSo that worked! It did come back with `Content-Type: binary/octet-stream` though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615947229", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615947229, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk0NzIyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T20:51:26Z", "updated_at": "2020-04-18T20:51:26Z", "author_association": "MEMBER", "body": "Running the upload again like this resulted in the correct content-type:\r\n```python\r\nclient.upload_file(\r\n \"/Users/simonw/Desktop/this_is_fine.jpg\",\r\n \"dogsheep-photos-simon\",\r\n \"this_is_fine.jpg\",\r\n ExtraArgs={\r\n \"ContentType\": \"image/jpeg\"\r\n }\r\n)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615947370", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615947370, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk0NzM3MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T20:52:13Z", "updated_at": "2020-04-18T20:52:13Z", "author_association": "MEMBER", "body": "This is great! I now have a key that can upload photos, and a separate key that can download photos OR generate signed URLs to access those photos.\r\n\r\nNext step: a script that starts uploading my photos.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615948102", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615948102, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk0ODEwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T20:56:59Z", "updated_at": "2020-04-18T20:56:59Z", "author_association": "MEMBER", "body": "I'm going to start with this:\r\n\r\n`photos-to-sqlite upload photos.db ~/path/to/directory`\r\n\r\nThis will scan the provided directory (and all sub-directories) for image files. It will then:\r\n\r\n* Calculate a sha256 of the contents of that file\r\n* Upload the file to a key that's `sha256.jpg` or `.heic`\r\n* Upload a `sha256.json` file with the original path to the image\r\n* Add that image to a `uploads` table in `photos.db`\r\n\r\nStretch goal: grab the EXIF data and include that in the `.json` upload AND the `uploads` database table.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615957385", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4", "id": 615957385, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk1NzM4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T21:56:16Z", "updated_at": "2020-04-18T21:58:11Z", "author_association": "MEMBER", "body": "Got this working! I'll do EXIF in a separate ticket #3.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602533539, "label": "Upload all my photos to a secure S3 bucket"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/5#issuecomment-615949574", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/5", "id": 615949574, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk0OTU3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T21:06:07Z", "updated_at": "2020-04-18T21:06:07Z", "author_association": "MEMBER", "body": "```\r\n$ photos-to-sqlite s3-auth\r\nCreate S3 credentials and paste them here:\r\n\r\nAccess key ID: xxx\r\nSecret access key: yyy\r\n$ cat auth.json \r\n{\r\n \"access_key_id\": \"xxx\",\r\n \"secret_access_key\": \"yyy\"\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602551638, "label": "photos-to-sqlite s3-auth command"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/6#issuecomment-615979923", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/6", "id": 615979923, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk3OTkyMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T23:36:02Z", "updated_at": "2020-04-18T23:36:02Z", "author_association": "MEMBER", "body": "I'll use a Click progress bar. To do this I need to first calculate the sum number of bytes in the photos that are going to be uploaded, then run the upload.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602575575, "label": "Add progress bar to upload command"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/6#issuecomment-615983393", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/6", "id": 615983393, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk4MzM5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T23:53:10Z", "updated_at": "2020-04-18T23:53:10Z", "author_association": "MEMBER", "body": "```\r\n$ photos-to-sqlite upload photos3.db ~/Pictures/Photos\\ Library.photoslibrary/Masters/2020\r\nUploading 2.09 GB [##----------------------------------] 6% 00:36:37\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602575575, "label": "Add progress bar to upload command"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/7#issuecomment-615993178", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/7", "id": 615993178, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTk5MzE3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-19T00:37:08Z", "updated_at": "2020-04-19T00:37:08Z", "author_association": "MEMBER", "body": "https://pypi.org/project/ImageHash/ Is one option.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602585497, "label": "Integrate image content hashing"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/7#issuecomment-906015471", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/7", "id": 906015471, "node_id": "IC_kwDOD079W842ALLv", "user": {"value": 18232, "label": "dkam"}, "created_at": "2021-08-26T02:01:01Z", "updated_at": "2021-08-26T02:01:01Z", "author_association": "NONE", "body": "Perceptual hashes might be what you're after : http://phash.org", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 602585497, "label": "Integrate image content hashing"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/8#issuecomment-618100434", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/8", "id": 618100434, "node_id": "MDEyOklzc3VlQ29tbWVudDYxODEwMDQzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-23T00:02:53Z", "updated_at": "2020-04-23T00:02:53Z", "author_association": "MEMBER", "body": "I don't think it matters one way or the other - I'm storing the sha256 in the filename, so the fact that I could read the MD5 back from the list bucket operation doesn't give me any benefits.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 605147638, "label": "Should I have used MD5 instead of SHA256?"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/8#issuecomment-618100658", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/8", "id": 618100658, "node_id": "MDEyOklzc3VlQ29tbWVudDYxODEwMDY1OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-23T00:03:35Z", "updated_at": "2020-04-23T00:03:35Z", "author_association": "MEMBER", "body": "Also MD5 isn't guaranteed for the ETag:\r\n> If an object is created by either the Multipart Upload or Part Copy operation, the ETag is not an MD5 digest, regardless of the method of encryption.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 605147638, "label": "Should I have used MD5 instead of SHA256?"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/9#issuecomment-618724149", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/9", "id": 618724149, "node_id": "MDEyOklzc3VlQ29tbWVudDYxODcyNDE0OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-23T23:35:29Z", "updated_at": "2020-04-23T23:35:29Z", "author_association": "MEMBER", "body": "```\r\n% photos-to-sqlite upload photos.db ~/Pictures/Photos\\ Library.photoslibrary/originals\r\nFetching existing keys from S3...\r\nGot 22,446 existing keys\r\nCalculating hashes [####--------------------------------] 13% 00:04:14\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 605938063, "label": "upload command should be resumable, should only upload photos not already uploaded"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/9#issuecomment-618725155", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/9", "id": 618725155, "node_id": "MDEyOklzc3VlQ29tbWVudDYxODcyNTE1NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-23T23:39:14Z", "updated_at": "2020-04-23T23:39:14Z", "author_association": "MEMBER", "body": "A few minutes later...\r\n```\r\nFetching existing keys from S3...\r\nGot 22,446 existing keys\r\nCalculating hashes [####################################] 100% \r\n22,441 hashed files, 610 are not yet in S3\r\nUploading 0.99 GB\r\nUploading 610 photos [------------------------------------] 1/610 03:10:35\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 605938063, "label": "upload command should be resumable, should only upload photos not already uploaded"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep.github.io/pull/6#issuecomment-1021264135", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/6", "id": 1021264135, "node_id": "IC_kwDODMzF1s4830EH", "user": {"value": 1151557, "label": "ligurio"}, "created_at": "2022-01-25T14:52:40Z", "updated_at": "2022-01-25T14:52:40Z", "author_association": "NONE", "body": "@simonw, could you review?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 842765105, "label": "Add testres-db tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/1#issuecomment-706775706", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/1", "id": 706775706, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjc3NTcwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-11T22:14:00Z", "updated_at": "2020-10-11T22:14:00Z", "author_association": "MEMBER", "body": "A live demo would be good too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718934942, "label": "Documentation on how to use this with Datasette"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/pull/10#issuecomment-777839351", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/10", "id": 777839351, "node_id": "MDEyOklzc3VlQ29tbWVudDc3NzgzOTM1MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-11T22:37:55Z", "updated_at": "2021-02-11T22:37:55Z", "author_association": "MEMBER", "body": "I've merged these changes by hand now, thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 770712149, "label": "BugFix for encoding and not update info."}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/11#issuecomment-777690332", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/11", "id": 777690332, "node_id": "MDEyOklzc3VlQ29tbWVudDc3NzY5MDMzMg==", "user": {"value": 3613583, "label": "dskrad"}, "created_at": "2021-02-11T18:16:01Z", "updated_at": "2021-02-11T18:16:01Z", "author_association": "NONE", "body": "I solved this issue by modifying line 31 of utils.py\r\n\r\n content = ET.tostring(ET.fromstring(content_xml.strip())).decode(\"utf-8\")", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 792851444, "label": "XML parse error"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/11#issuecomment-777798330", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/11", "id": 777798330, "node_id": "MDEyOklzc3VlQ29tbWVudDc3Nzc5ODMzMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-11T21:18:58Z", "updated_at": "2021-02-11T21:18:58Z", "author_association": "MEMBER", "body": "Thanks for the fix!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 792851444, "label": "XML parse error"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-905203570", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13", "id": 905203570, "node_id": "IC_kwDOEhK-wc419E9y", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-25T05:51:22Z", "updated_at": "2021-08-25T05:53:27Z", "author_association": "MEMBER", "body": "The debugger showed me that it broke on a string that looked like this:\r\n```xml\r\n\r\n\r\n\r\n

\r\n \r\n Q3 2018 Reflection & Development\r\n \r\n

\r\n ...\r\n```\r\nYeah that is not valid XML!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 978743426, "label": "xml.etree.ElementTree.ParseError: not well-formed (invalid token)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-905206234", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13", "id": 905206234, "node_id": "IC_kwDOEhK-wc419Fna", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-25T05:58:42Z", "updated_at": "2021-08-25T05:58:42Z", "author_association": "MEMBER", "body": "https://github.com/dogsheep/evernote-to-sqlite/blob/36a466f142e5bad52719851c2fbda0c05cd35b99/evernote_to_sqlite/utils.py#L34-L42\r\n\r\nNot sure why I was round-tripping the `content_xml` like that - I will try not doing that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 978743426, "label": "xml.etree.ElementTree.ParseError: not well-formed (invalid token)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-906635938", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13", "id": 906635938, "node_id": "IC_kwDOEhK-wc42Ciqi", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-26T18:18:27Z", "updated_at": "2021-08-26T18:18:27Z", "author_association": "MEMBER", "body": "It looks like I was using the round-trip to dump the `` and `` content - it's just HTML, not even trying to be XML.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 978743426, "label": "xml.etree.ElementTree.ParseError: not well-formed (invalid token)"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/14#issuecomment-911772943", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/14", "id": 911772943, "node_id": "IC_kwDOEhK-wc42WI0P", "user": {"value": 46968, "label": "step21"}, "created_at": "2021-09-02T14:53:11Z", "updated_at": "2021-09-02T14:53:11Z", "author_association": "NONE", "body": "Additionally, assuming the line numbers match up with the provided enenx file, the mentioned line plus one before and after is as follows:\r\n```\r\n]]>\r\n\r\n

\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 986829194, "label": "xml.etree.ElementTree.Parse Error - mismatched tag"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/4#issuecomment-706784028", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/4", "id": 706784028, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjc4NDAyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-11T23:20:32Z", "updated_at": "2020-10-11T23:20:32Z", "author_association": "MEMBER", "body": "I haven't done the FTS on OCR yet. I'm going to move that to another ticket because it requires more thought.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718938508, "label": "Configure FTS + add an index on the date columns"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/4#issuecomment-706786548", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/4", "id": 706786548, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjc4NjU0OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-11T23:39:46Z", "updated_at": "2020-10-11T23:39:46Z", "author_association": "MEMBER", "body": "Should have used porter stemming for this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718938508, "label": "Configure FTS + add an index on the date columns"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776180", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5", "id": 706776180, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjc3NjE4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-11T22:17:55Z", "updated_at": "2020-10-11T22:17:55Z", "author_association": "MEMBER", "body": "We could even do server-side thumbnailing for some of these images, but I'm inclined to serve up the full size ones and set a width on the image element based on the `width` attribute on ``.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718938889, "label": "Figure out how to display images from tags inline in Datasette"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776242", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5", "id": 706776242, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjc3NjI0Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-11T22:18:30Z", "updated_at": "2020-10-11T22:19:48Z", "author_association": "MEMBER", "body": "Alternatively, rather than relying on `datasette-media` this could base64-embed the images. `evernote-to-sqlite` could register itself as a Datasette plugin that knows how to do this.\r\n\r\nMaybe rename the column to `evernote_content` and register a render cell hook that knows how to rewrite those note bodies so that they are visible?\r\n\r\nMight need to feed them through Bleach too, just in case any nasty code can get into them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718938889, "label": "Figure out how to display images from tags inline in Datasette"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776447", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5", "id": 706776447, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjc3NjQ0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-11T22:20:32Z", "updated_at": "2020-10-11T22:20:32Z", "author_association": "MEMBER", "body": "Or... I could do this client-side. JavaScript that looks for `` tags and fetches the data using `fetch()` wouldn't be too hard to write.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718938889, "label": "Figure out how to display images from tags inline in Datasette"}, "performed_via_github_app": null}