Fixing FITs?

[markbulk]

I have a garbage HRM (iFit) that will report non-integer values for BPM. Garmin stores those values without truncating or rounding, despite it not being compatible with the data type. As a result, many of my activity files are abandoned when going through processing due to:

Failed to processes file /Users/some_name/HealthData/FitFiles/Activities/5929293279_ACTIVITY.tcx: invalid literal for int() with base 10: '151.5'

Given that this is a known issue with Garmin's behavior (they will write bad data into the FIT files), it seems like this should not be a reason to ignore the FIT activity file?

[tcgoetz]

Can you share a file with this data so I can debug it and see the best way to fix it?

[markbulk]

Apologies for wasting your time here. Here is the file in case you care.

It seems as if I am blaming the wrong entity entirely. I hadn't noted that it was failing on a TCX file (assumed because the directory was ../FitFiles/Activities, it was a FIT file). The TCX files that crapped out on your system were created by iFit (a brand of Icon Fitness). They create garbage files that include fractional heart rate values. This is entirely their fault.

I had been going through the trouble of uploading them because otherwise I don't get power information in Garmin. I wrote an R script to merge the best information from the available CSV and TCX (and deals with the non-integer heartrate issue). See here, if somebody finds this and cares.

[markbulk]

So despite my stupidity about the TCX file, I do have a number of FIT files that fail to parse due to tuple index out of range. I'm not a Python programmer, so I don't really know what would cause this.

I have explored the files a little and there doesn't seem to be consistency between device (some were my Edge 1030 others were my FR 945) or event type (some were biking and some were running). They typically have extra fields (I have a Humon Hex from which I get the ConnectIQ data), but many workouts that have the extra fields didn't have this problem.

Here are three example files:
File One
File Two
File Three

I should note that for my 1275 FIT activity files, this error occurred for 78 of them, or roughly 6%, so pretty frequently.

[tcgoetz]

Fix for TCX issue on develop branch 6af707e will be merged to master for next release.

[markbulk]

Thanks for tackling this! Looking at these activities again - some were recorded with my Humon Hex and some were not. Odd that they would intermittently be causing problems.

[tcgoetz]

I dumped Fit file one and it has badly formatted developer fields from the Humon:

DeveloperFieldDefinition(dev_total_hemoglobin_conc[currHemoConc]: scale None offset None units g/dl, 1 of FLOAT32) for <MessageType.record: 20> field 54 DeveloperFieldDefinition(dev_saturated_hemoglobin_percent[currHemoPerc]: scale None offset None units %, 1 of FLOAT32) for <MessageType.record: 20> field 57 DeveloperFieldDefinition(dev_humonhexzone[humonhexzone]: scale None offset None units Zone, 1 of INT8) for <MessageType.record: 20> field None DeveloperFieldDefinition(dev_avg_total_hemoglobin_conc[lapHemoConc]: scale None offset None units g/dl, 0 of FLOAT32) for <MessageType.lap: 19> field 84

[tcgoetz]

Zero length fields. Developer field indexes that don't map to developer fields definitions, ...