issues
24 rows where repo = 197882382
This data as json, CSV (advanced)
Suggested facets: user, state, comments, updated_at, closed_at, author_association, type, created_at (date), updated_at (date), closed_at (date)
| id ▼ | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 470637068 | MDU6SXNzdWU0NzA2MzcwNjg= | 1 | Use XML Analyser to figure out the structure of the export XML | simonw 9599 | closed | 0 | 1 | 2019-07-20T05:19:02Z | 2019-07-20T05:20:09Z | 2019-07-20T05:20:09Z | MEMBER | https://github.com/simonw/xml_analyser | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
| 470637152 | MDU6SXNzdWU0NzA2MzcxNTI= | 2 | Import workouts | simonw 9599 | closed | 0 | 1 | 2019-07-20T05:20:21Z | 2019-07-20T06:21:41Z | 2019-07-20T06:21:41Z | MEMBER | From #1 | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
| 470637206 | MDU6SXNzdWU0NzA2MzcyMDY= | 3 | Import ActivitySummary | simonw 9599 | closed | 0 | 0 | 2019-07-20T05:21:00Z | 2019-07-20T05:58:07Z | 2019-07-20T05:58:07Z | MEMBER | From #1 ```python 'ActivitySummary': {'attr_counts': {'activeEnergyBurned': 980, 'activeEnergyBurnedGoal': 980, 'activeEnergyBurnedUnit': 980, 'appleExerciseTime': 980, 'appleExerciseTimeGoal': 980, 'appleStandHours': 980, 'appleStandHoursGoal': 980, 'dateComponents': 980}, 'child_counts': {}, 'count': 980, 'parent_counts': {'HealthData': 980}}, ``` | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
| 470640505 | MDU6SXNzdWU0NzA2NDA1MDU= | 4 | Import Records | simonw 9599 | closed | 0 | 1 | 2019-07-20T06:11:20Z | 2019-07-20T06:21:41Z | 2019-07-20T06:21:41Z | MEMBER | From #1: ```python 'Record': {'attr_counts': {'creationDate': 2672233, 'device': 2665111, 'endDate': 2672233, 'sourceName': 2672233, 'sourceVersion': 2671779, 'startDate': 2672233, 'type': 2672233, 'unit': 2650012, 'value': 2672232}, 'child_counts': {'HeartRateVariabilityMetadataList': 2318, 'MetadataEntry': 287974}, 'count': 2672233, 'parent_counts': {'Correlation': 2, 'HealthData': 2672231}}, ``` | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
| 470691622 | MDU6SXNzdWU0NzA2OTE2MjI= | 5 | Add progress bar | simonw 9599 | closed | 0 | 2 | 2019-07-20T16:29:07Z | 2019-07-22T03:30:13Z | 2019-07-22T02:49:22Z | MEMBER | Showing a progress bar would be nice, using Click. The easiest way to do this would probably be be to hook it up to the length of the compressed content, and update it as this code pushes more XML bytes through the parser: https://github.com/dogsheep/healthkit-to-sqlite/blob/d64299765064501f4efdd9a0b21dbdba9ec4287f/healthkit_to_sqlite/utils.py#L6-L10 | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
| 470856782 | MDU6SXNzdWU0NzA4NTY3ODI= | 6 | Break up records into different tables for each type | simonw 9599 | closed | 0 | 1 | 2019-07-22T01:54:59Z | 2019-07-22T03:28:55Z | 2019-07-22T03:28:50Z | MEMBER | I don't think there's much benefit to having all of the different record types stored in the same enormous table. Here's what I get when I use `_facet=type`: <img width="358" alt="hello2__records__2_672_233_rows" src="https://user-images.githubusercontent.com/9599/61601118-e2f54d00-abe8-11e9-8bf6-3df2ef969112.png"> I'm going to try splitting these up into separate tables - so `HKQuantityTypeIdentifierBodyMassIndex` becomes a table called `rBodyMassIndex` - and see if that's nicer to work with. | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
| 472097220 | MDU6SXNzdWU0NzIwOTcyMjA= | 7 | Script uses a lot of RAM | simonw 9599 | closed | 0 | 3 | 2019-07-24T06:11:11Z | 2019-07-24T06:35:52Z | 2019-07-24T06:35:52Z | MEMBER | I'm using an XML pull parser which should avoid the need to slurp the whole XML file into memory, but it's not working - the script still uses over 1GB of RAM when it runs according to Activity Monitor. I think this is because I'm still causing the full root element to be incrementally loaded into memory just in case I try and access it later. http://effbot.org/elementtree/iterparse.htm says I should use `elem.clear()` as I go. It also says: > The above pattern has one drawback; it does not clear the root element, so you will end up with a single element with lots of empty child elements. If your files are huge, rather than just large, this might be a problem. To work around this, you need to get your hands on the root element. So I will try that recipe and see if it helps. | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
| 472104705 | MDExOlB1bGxSZXF1ZXN0MzAwNTgwMjIx | 8 | Use less RAM | simonw 9599 | closed | 0 | 0 | 2019-07-24T06:35:01Z | 2019-07-24T06:35:52Z | 2019-07-24T06:35:52Z | MEMBER | dogsheep/healthkit-to-sqlite/pulls/8 | Closes #7 | healthkit-to-sqlite 197882382 | pull | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | |||||
| 472429048 | MDU6SXNzdWU0NzI0MjkwNDg= | 9 | Too many SQL variables | tholo 166463 | closed | 0 | 4 | 2019-07-24T18:24:17Z | 2019-07-26T10:01:05Z | 2019-07-26T10:01:05Z | NONE | Decided to try importing my data, and ran into this: ``` Traceback (most recent call last): File "/Users/tholo/Source/health/bin/healthkit-to-sqlite", line 10, in <module> sys.exit(cli()) File "/Users/tholo/Source/health/lib/python3.7/site-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/Users/tholo/Source/health/lib/python3.7/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/Users/tholo/Source/health/lib/python3.7/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/tholo/Source/health/lib/python3.7/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/Users/tholo/Source/health/lib/python3.7/site-packages/healthkit_to_sqlite/cli.py", line 50, in cli convert_xml_to_sqlite(fp, db, progress_callback=bar.update) File "/Users/tholo/Source/health/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py", line 41, in convert_xml_to_sqlite write_records(records, db) File "/Users/tholo/Source/health/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py", line 80, in write_records column_order=["startDate", "endDate", "value", "unit"], File "/Users/tholo/Source/health/lib/python3.7/site-packages/sqlite_utils/db.py", line 911, in insert_all result = self.db.conn.execute(sql, values) sqlite3.OperationalError: too many SQL variables ``` Added some debug output in sqlite_utils/db.py, which resulted in: ``` INSERT INTO [rBodyMassIndex] ([creationDate], [endDate], [metadata_HKWasUserEntered], [metadata_Health Mate App Version], [metadata_Modified Date], [metadata_Withings Link], [metadata_Withings User Identifier], [sourceName], [sourceVersion], [startDate], [unit], [value]) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) , (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) , (?, ?, ?, ?, ?, … | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
| 519038979 | MDU6SXNzdWU1MTkwMzg5Nzk= | 10 | Failed to import workout points | simonw 9599 | closed | 0 | 4 | 2019-11-07T04:50:22Z | 2019-11-08T01:18:37Z | 2019-11-08T01:18:37Z | MEMBER | I just ran the script and it failed to import any `workout_points`, though it did import `workouts`. | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
| 723838331 | MDU6SXNzdWU3MjM4MzgzMzE= | 11 | export.xml file name varies with different language settings | jarib 572 | closed | 0 | 7 | 2020-10-17T20:07:18Z | 2020-10-17T21:39:15Z | 2020-10-17T21:14:10Z | NONE | The XML file exported from my phone has a Norwegian file name – `eksport.xml` 🙄 I can work around this by unpacking the zip and using `--xml`, but then I lose the workout points. Perhaps this could be solved by `--localized-xml eksport.xml`? Alternatively just fall back to the first XML file in the root folder of the zip. | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
| 727848625 | MDU6SXNzdWU3Mjc4NDg2MjU= | 12 | Some workout columns should be float, not text | simonw 9599 | open | 0 | 4 | 2020-10-23T02:47:02Z | 2022-06-23T04:35:02Z | MEMBER | Columns `duration`, `totalDistance` and `totalEnergyBurned` should be converted to float. https://github.com/dogsheep/healthkit-to-sqlite/blob/71e36e1cf034b96de2a8e6652265d782d3fdf63b/healthkit_to_sqlite/utils.py#L50-L57 | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
| 743071410 | MDExOlB1bGxSZXF1ZXN0NTIxMDU0NjEy | 13 | SQLite does not have case sensitive columns | tomaskrehlik 1689944 | open | 0 | 1 | 2020-11-14T20:12:32Z | 2021-08-24T13:28:26Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/13 | This solves a weird issue when there is record with metadata key that is only different in letter cases. See the test for details. | healthkit-to-sqlite 197882382 | pull | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||||
| 771608692 | MDU6SXNzdWU3NzE2MDg2OTI= | 14 | UNIQUE constraint failed: workouts.id | n8henrie 1234956 | open | 0 | 5 | 2020-12-20T15:11:20Z | 2023-07-10T14:46:52Z | NONE | I'm getting an error on my initial attempt to import data: ```console $ healthkit-to-sqlite 20201119\ healthkit\ export.zip healthkit.db Importing from HealthKit [###################################-] 98% 00:00:01 Traceback (most recent call last): File "venv/bin/healthkit-to-sqlite", line 8, in <module> sys.exit(cli()) File "venv/lib/python3.9/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "venv/lib/python3.9/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "venv/lib/python3.9/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "venv/lib/python3.9/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "venv/lib/python3.9/site-packages/healthkit_to_sqlite/cli.py", line 57, in cli convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf) File "venv/lib/python3.9/site-packages/healthkit_to_sqlite/utils.py", line 34, in convert_xml_to_sqlite workout_to_db(el, db, zipfile) File "venv/lib/python3.9/site-packages/healthkit_to_sqlite/utils.py", line 57, in workout_to_db pk = db["workouts"].insert(record, alter=True, hash_id="id").last_pk File "venv/lib/python3.9/site-packages/sqlite_utils/db.py", line 1660, in insert return self.insert_all( File "venv/lib/python3.9/site-packages/sqlite_utils/db.py", line 1778, in insert_all self.insert_chunk( File "venv/lib/python3.9/site-packages/sqlite_utils/db.py", line 1588, in insert_chunk result = self.db.execute(query, params) File "venv/lib/python3.9/site-packages/sqlite_utils/db.py", line 213, in execute return self.conn.execute(sql, parameters) sqlite3.IntegrityError: UNIQUE constraint failed: workouts.id ``` | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/14/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
| 793907673 | MDExOlB1bGxSZXF1ZXN0NTYxNTEyNTAz | 15 | added try / except to write_records | ryancheley 9857779 | open | 0 | 0 | 2021-01-26T03:56:21Z | 2021-01-26T03:56:21Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/15 | to keep the data write from failing if it came across an error during processing. In particular when trying to convert my HealthKit zip file (and that of my wife's) it would consistently error out with the following: ``` db.py 1709 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: too many SQL variables --------------------------------------------------------------------------------------------------------------------------------------------------------------------- db.py 1709 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: too many SQL variables --------------------------------------------------------------------------------------------------------------------------------------------------------------------- db.py 1709 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: table rBodyMass has no column named metadata_HKWasUserEntered --------------------------------------------------------------------------------------------------------------------------------------------------------------------- healthkit-to-sqlite 8 <module> sys.exit(cli()) core.py 829 __call__ return self.main(*args, **kwargs) core.py 782 main rv = self.invoke(ctx) core.py 1066 invoke return ctx.invoke(self.callback, **ctx.params) core.py 610 invoke return callback(*args, **kwargs) cli.py 57 cli convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf) utils.py 42 convert_xml_to_sqlite write_records(records, db) utils.py 143 write_records db[table].insert_all( db.py 1899 insert_all self.insert_chunk( db.py 1720 insert_chunk self.insert_chunk( db.py 1720 insert_chunk self.insert_chunk( db.py 1714 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.co… | healthkit-to-sqlite 197882382 | pull | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/15/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||||
| 830901133 | MDExOlB1bGxSZXF1ZXN0NTkyMzY0MjU1 | 16 | Add a fallback ID, print if no ID found | n8henrie 1234956 | open | 0 | 0 | 2021-03-13T13:38:29Z | 2021-03-13T14:44:04Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/16 | Fixes https://github.com/dogsheep/healthkit-to-sqlite/issues/14 | healthkit-to-sqlite 197882382 | pull | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||||
| 836063389 | MDU6SXNzdWU4MzYwNjMzODk= | 17 | Datetime columns are not properly formatted to be recognizes as datetime | n8henrie 1234956 | open | 0 | 0 | 2021-03-19T14:33:04Z | 2021-03-19T14:33:04Z | NONE | Currently, the datetimes are formatted in a way that is not recognized by datasette-vega for plotting with a `Date/time` type for the axis. For example, if you have datasette running locally with `datasette-vega` installed and have a database that includes resting heart rate: ``` http://localhost:8001/healthkit/rRestingHeartRate#g.mark=line&g.x_column=startDate&g.x_type=temporal&g.y_column=value&g.y_type=quantitative ``` The plot is blank unless you choose `Label` as the type for the date data. The `startDate` (and `creationDate` and `endDate`) columns appear like: `2019-11-14 18:22:18 -0700` If instead the format for this column is changed slightly: `2019-11-14T18:22:18-07:00` they are recognized as proper dates and the charting works as expected. I have a PR that addresses this issue, will submit shortly. | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
| 836064851 | MDExOlB1bGxSZXF1ZXN0NTk2NjI3Nzgw | 18 | Add datetime parsing | n8henrie 1234956 | open | 0 | 0 | 2021-03-19T14:34:22Z | 2021-03-19T14:34:22Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/18 | Parses the datetime columns so they are subsequently properly recognized as datetime. Fixes https://github.com/dogsheep/healthkit-to-sqlite/issues/17 | healthkit-to-sqlite 197882382 | pull | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||||
| 975158266 | MDU6SXNzdWU5NzUxNTgyNjY= | 19 | table activity_summary has no column named appleMoveTime | simonw 9599 | closed | 0 | 0 | 2021-08-20T00:46:44Z | 2021-08-20T00:54:34Z | 2021-08-20T00:54:34Z | MEMBER | Got this error today against a fresh export: table activity_summary has no column named appleMoveTime | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | completed | ||||||
| 975166271 | MDU6SXNzdWU5NzUxNjYyNzE= | 20 | Add index on workout_points.date | simonw 9599 | open | 0 | 2 | 2021-08-20T01:08:04Z | 2021-08-20T01:12:48Z | MEMBER | Sorting that by date makes sense for seeing most recent points, and my DB has 2.5m points in so it's an expensive sort! | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
| 977128935 | MDU6SXNzdWU5NzcxMjg5MzU= | 21 | Duplicate Column | FabianHertwig 32016596 | open | 0 | 1 | 2021-08-23T15:00:44Z | 2021-08-23T17:00:59Z | NONE | Hey, thank you for this repo! When I try to convert my export, I get a multiple column error. Here is the stack trace: ```sh (.venv) (base) computer:bodyweight_app user$ healthkit-to-sqlite ./data/Health_export.zip ./data/healthkit.db Importing from HealthKit [###############################-----] 87% 00:00:22 Traceback (most recent call last): File "/MyProject/.venv/bin/healthkit-to-sqlite", line 10, in <module> sys.exit(cli()) File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/cli.py", line 57, in cli convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf) File "/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py", line 41, in convert_xml_to_sqlite write_records(records, db) File "/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py", line 146, in write_records batch_size=50, File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 2579, in insert_all extracts=extracts, File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 1246, in create extracts=extracts, File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 767, in create_table self.execute(sql) File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 421, in execute return self.conn.execute(sql) sqlite3.OperationalError: duplicate column name: metadata_Meal ``` | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/21/reactions", "total_count": 4, "+1": 4, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | ||||||||
| 978086284 | MDExOlB1bGxSZXF1ZXN0NzE4NzM0MTkx | 22 | Make sure that case-insensitive column names are unique | FabianHertwig 32016596 | open | 0 | 1 | 2021-08-24T13:13:38Z | 2021-08-24T13:26:20Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/22 | This closes #21. When there are metadata entries with the same case insensitive string, then there is an error when trying to create a new column for that metadata entry in the database table, because a column with that case insensitive name already exists. ```xml <Record type="HKQuantityTypeIdentifierDietaryCholesterol" sourceName="MyFitnessPal" sourceVersion="35120" unit="mg" creationDate="2021-07-04 20:55:27 +0200" startDate="2021-07-04 20:55:00 +0200" endDate="2021-07-04 20:55:00 +0200" value="124"> <MetadataEntry key="meal" value="Dinner"/> <MetadataEntry key="Meal" value="Dinner"/> </Record> ``` The code added in this PR checks if a key already exists in a record and if so adds a number at its end. The resulting column names look like the example below then. Interestingly, the column names viewed with Datasette are not case insensitive. ```text startDate, endDate, value, unit, sourceName, sourceVersion, creationDate, metadata_meal, metadata_Meal_2, metadata_Mahlzeit ``` | healthkit-to-sqlite 197882382 | pull | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/22/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||||
| 1515717718 | PR_kwDOC8tyDs5Gc-VH | 23 | Include workout statistics | badboy 2129 | open | 0 | 0 | 2023-01-01T17:29:57Z | 2023-01-01T17:29:57Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/23 | Not sure when this changed (iOS 16 maybe?), but the `WorkoutStatistics` now has a whole bunch of information about workouts, e.g. for runs it contains the distance (as a `<WorkoutStatistics type="HKQuantityTypeIdentifierDistanceWalkingRunning ...>` element). Adding it as another column at leat allows me to pull these out (using SQLite's JSON support). I'm running with this patch on my own data now. | healthkit-to-sqlite 197882382 | pull | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/23/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} | 0 | ||||||
| 1515883470 | I_kwDOC8tyDs5aWovO | 24 | DOC: xml.etree.ElementTree.ParseError due to healthkit version 12 | mmngreco 6231413 | open | 0 | 2 | 2023-01-01T23:00:38Z | 2023-03-30T10:17:31Z | NONE | Hi @simonw I hope you find this issue ok, the idea is provide some documentation to other users like me about how to solve this problem and save some time. Following the instructions from the `README.md` I've faced this error: ```bash (venv) mgreco@pop-os apple-health master* (23:44|0s) $ healthkit-to-sqlite apple_health_export/export.xml healthkit.db --xml Importing from HealthKit [------------------------------------] 0% Traceback (most recent call last): File "/home/mgreco/github/mmngreco/apple-health/venv/bin/healthkit-to-sqlite", line 33, in <module> sys.exit(load_entry_point('healthkit-to-sqlite', 'console_scripts', 'healthkit-to-sqlite')()) File "/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) File "/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/cli.py", line 57, in cli convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf) File "/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/utils.py", line 25, in convert_xml_to_sqlite for tag, el in find_all_tags( File "/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/utils.py", line 12, in find_all_tags for event, el in parser.read_events(): File "/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/xml/etree/ElementTree.py", line 1324, in read_events raise event File "/home/mgreco/gith… | healthkit-to-sqlite 197882382 | issue | {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/24/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issues] (
[id] INTEGER PRIMARY KEY,
[node_id] TEXT,
[number] INTEGER,
[title] TEXT,
[user] INTEGER REFERENCES [users]([id]),
[state] TEXT,
[locked] INTEGER,
[assignee] INTEGER REFERENCES [users]([id]),
[milestone] INTEGER REFERENCES [milestones]([id]),
[comments] INTEGER,
[created_at] TEXT,
[updated_at] TEXT,
[closed_at] TEXT,
[author_association] TEXT,
[pull_request] TEXT,
[body] TEXT,
[repo] INTEGER REFERENCES [repos]([id]),
[type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
ON [issues] ([user]);