How Accurate Are Activity Trackers? EE Data From Omron, Fitbit, Jawbone & Other Devices Reveals 10% Error & More

Even though the study doesn't provide a straight-forward answer to the question "Which is the best activity tracker?", it is still revealing.

I hope you don't rely on the data from your activity tracker as a basis to decide how much you can, should or may eat on a daily basis. Why? Well, the first and most important result of a recent study from the Human Performance Laboratory at the Ball State University is that "consumer-based PA [physical activity] monitors should be used cautiously for estimating EE [energy expenditure]" (Nelson. 2016) - and this goes for the data from all the devices that were tested by Nelson et al.: The BodyMedia FIT and the NikeFuel armband, the DirectLife monitor, the Omron HJ-720IT, the Fitbit One, the Fitbit Zip, the Fitbit Flex, the Jawbone UP24, the Basis B1 Band Monitor and the ActiGraph.

Don't tell me you use an activity tracker, but don't periodize your training!

30% More on the Big Three: Squat, DL, BP!

Mix Things Up to Make Extra-Gains

Linear vs. Undulating Periodizationt

12% Body Fat in 12 Weeks W/ Periodizatoin

Detraining + Periodization - How to?

Tapering 101 - Learn How It's Done!

In view of the fact that tracking your energy expenditure is only one of the functions activity trackers are supposed to fulfill and considering the fact that you probably use them only to see if you have gotten more or less active (I do at least hope that you don't use them to guide your appetite ;-), it is still worth to take a look at the detailed results of this recent study.

As you will have guessed, the study was designed to "examine the validity of EE estimates from a variety of consumer-based, physical activity monitors under free-living conditions" (Nelson. 2016). To this ends, sixty (26.4 ± 5.7 yr) healthy men (n = 30) and women (n = 30) wore eight different types of activity monitors simultaneously while completing a 69-min protocol.

If you work out to be able to allow yourself to eat, you know you have a serious problem | learn why!

"The monitors included the BodyMedia FIT armband worn on the left arm, the DirectLife monitor around the neck, the Fitbit One, the Fitbit Zip, and the ActiGraph worn on the belt, as well as the Jawbone Up and Basis B1 Band monitor on the wrist.

The validity of the EE estimates from each monitor was evaluated relative to criterion values concurrently obtained from a portable metabolic system (i.e., Oxycon Mobile) [which is obviously in itself not 100% exact]. Differences from criterion measures were expressed as a mean absolute percent error and were evaluated using 95% equivalence testing" (Nelson. 2016).

A brief glance at Figure 2 reveals that the accuracy was surprisingly similar among the devices. To be more precise, the mean absolute percent error values (computed as the average absolute value of the group-level errors) were 9.3%, 10.1%, 10.4%, 12.2%, 12.6%, 12.8%, 13.0%, and 23.5% for the BodyMedia FIT, Fitbit Zip, Fitbit One, Jawbone Up, ActiGraph, DirectLife, NikeFuel Band, and Basis B1 Band, respectively (unfortunately, not all data appears to be fully reported in the manuscript version of the study I had access to, so don't ask me about missing data, please ;-).

What did the test protocol look like? Subjects took part in a structured activity protocol consisting of 11 activities (three sedentary, four household, and four ambulatory/exercise) chosen by researchers from a list of 21 activities ranging from lying around on the couch to treadmill jogging. Activities were counterbalanced so that sex and age categories had approximately equal participation in the activities. All subjects began by lying quietly on a bed for 10 min. All other activities were performed for 5 min each, in order of generally increasing intensity. All activities were performed at a self-selected intensity by the subject. Subjects chosen to perform the jogging activity had the option of participating in a brisk walk if unable to jog for 5 min.

As the scientists point out, of all tested devices, only "[t]he results from the equivalence testing showed that the estimates from the BodyMedia FIT, Fitbit Zip, and NikeFuel Band (90% confidence interval = 341.1-359.4) were within the 10% equivalence zone around the indirect calorimetry estimate. If you still insist on trying to match your energy intake "exactly" to your energy expenditure, you should plan for a 10% + X% difference from your actual energetic demands - after all, even the indirect calorimetry that was used as a yardstick to judge the accuracy of the devices is not 100% accurate.

Figure 1: Mean absolute percent error when estimating energy expenditure for selected devices (Nelson. 2016).

In that, it is also worth mentioning that the accuracy of the devices was activity and device dependent. The Fitbit One, for example, produces the least error for stair climbing. For the Jawbone UP24, however, the "activity" for which it predicts your energy expenditure best is sitting around.

Accordingly, you could argue that you'd have to wear a certain device for a certain activity, e.g. (a) the Fitbit One, when sitting around (13%), working in the household (27%), taking the stairs (11%), jogging (22%) or cycling (43%) [note: on absolute terms, the error of the Fitbit for being sedentary is still lower than with the device from Jawbone], and (b) the Jawbone UP24, when you're simply walking around... but let's be honest: Since even that wouldn't be 100% accurate, it would be dumb to buy multiple fitness / activity trackers, wouldn't it?

Figure 2: With the exception of data from cycling and housework, the step count data (this graph) is sign. more accurate than the EE data in Figure 1 | If you want to learn more about what activity trackers are good / not good for and what you can / should make of the results of the study at hand, listen to me discuss this study on Monday's installment Super HumanRadio | click here to download the complete podcast that also includes discussions of the links NSAIDs and satellite cells and BPC-157 for muscle and tendon repair!

With an error of 10% you will always lose or gain weight involuntarily: The idea that a tiny technical device on your arm or belt could exactly tell you how much energy you need is in itself hilarious. And that's not just because the study at hand shows that even the best devices are on average +/-10% off (remember: that's +/-10% off another rough estimate that's never 100% exact). If you were dumb enough to match your diet blindly to the data your activity tracker provides, you would thus never achieve reliable results.

With that being said, our body is no biological machine that works according to a set of several (complex) equations. Therefore, the whole idea of a "quantified self" - as awesome as it may seem for the average control freak - must be seen as a tool to hold yourself accountable; a qualitative or semi-quantitative tool in the sense of "oh, I have been roughly 20% less active this week than last week, maybe I should..."

If the previously described rationale is behind the way you use the data from your activity tracker, congratulations! If not, I have to warn you: The margin between "quantifying yourself" and suffering from obsessive-compulsive disorder (OCD) and/or using the devices to fuel your exercise addiction is narrower than you may think | Do you agree, disagree? Let's discuss. Leave a comment on Facebook!

References

• Nelson, Benjamin N; et al. "Validity of Consumer-Based Physical Activity Monitors for Specific Activity Types ED." Med Sci Sports Exerc (2016): Ahead of print.