Abstract
Key Points
Question
How accurate are the step counts obtained from Apple Watch?
Findings
In this validation study, video steps vs. Apple Watch steps (mean ± SD) were 2965 ± 144 vs. 2964 ± 145 steps; P < 0.001. Lin’s concordance correlation coefficient showed a strong correlation (r = 0.96; P < 0.001) between the two measurements. There was a total error of 0.034% (1.07 steps) for the Apple Watch steps when compared with the manual counts obtained from video recordings.
Meaning
Our study is one of the initial studies to objectively validate the accuracy of the step counts obtained from Apple watch at different walking speeds. Apple Watch tested to be an extremely accurate device for measuring daily step counts for adults.
KEY WORDS: Apple watch, step counts, tracking
Introduction
Accuracy of smartphone applications and various wearable devices for tracking physical activity have been reported.1,2 The rise in popularity of Apple Watch as a health and fitness tracker warrants validation as validation data at different walking speeds are limited.3,4 This might be important to the general population with the growing reliance on wearable technology devices for measuring daily step counts. The objective of this study was to assess the accuracy of Apple Watch to track steps by comparing it to the step counts obtained using video recordings.
Methods
Participants who were able to walk on a treadmill were recruited at the Pennsylvania State University Berks campus. Before each test, an Apple Watch was calibrated, paired to an iPhone to obtain step counts, and applied to fit snugly on top of the participant’s left wrist. Apple watch data were aggregated for analysis in 1-min epochs. Each participant completed a treadmill protocol with four different speeds [slow (1.9mph), moderate (3.0mph), brisk (4.0mph), and jogging (5.2mph)] with 6 min duration/speed.5 Apple watch data were appropriately synchronized to the video counts by noting the start and end time for each speed. The entire protocol was video recorded using a Canon VIXIA HF500 camcorder placed perpendicular to the treadmill to observe the steps. Video recordings were manually assessed for step counts by two independent observers from the recorded video clips. Intra class correlation coefficient between the two observers was 0.949 with a 95% CI (0.918, 0.968).
Results
Seventy-one participants (24F/47M) with wide range of body mass indices (19.5–40.7 kg/m2, mean 25.7 ± 3.8 kg/m2) between the age groups of 18-55 years (< 20, 20–29, 30–39, 40–49, > 50 years; 13, 69, 8, 4, and 6%, respectively) completed the study. Total video steps vs. total Apple Watch steps (mean ± SD) were 2965 ± 144 vs. 2964 ± 145 steps; P < 0.001. Lin’s concordance correlation coefficient showed a strong correlation (r = 0.96; P < 0.001) between the two measurements. There was a total error of 0.034% (1.07 steps) for the Apple Watch across all stages and across all participants.
Table 1 shows the mean ± SD and minimum-maximum range of step counts for the video and the Apple Watch with the reported relative percent error for Apple Watch steps at different speeds. Apple Watch consistently overestimated the step counts for the slow, moderate, and brisk paces (relative error − 2.6, − 0.9, and − 1.6%) and underestimated (relative error 3.4%) for the jogging pace of the treadmill protocol. The least amount of relative error (− 0.9%) was observed for the moderate intensity pace (3.0 mph). There were no statistically significant differences in step counts between genders, across age groups, and with different BMI categories.
Table 1.
Step Counts and Errors for Different Speeds
| Variable | Slow (1.9 mph) | Moderate (3.0 mph) | Brisk (4.0 mph) | Jogging (5.2 mph) | ||||
|---|---|---|---|---|---|---|---|---|
| Mean ± SD | Min-max | Mean ± SD | Min-max | Mean ± SD | Min-max | Mean ± SD | Min-max | |
| Video counts, steps | 560 ± 38 | 443–722 | 680 ± 32 | 601–755 | 783 ± 63 | 695–998 | 941 ± 47 | 830–1073 |
| Apple Watch, steps | 574 ± 38 | 469–669 | 686 ± 41 | 574–760 | 795 ± 69 | 631–1023 | 908 ± 54 | 781–1050 |
| Relative error in Apple watch, steps % | − 2.6 ± 5.6 | − 22–17.8 | − 0.9 ± 5.1 | − 23.4–16.9 | − 1.6 ± 4.2 | − 13.7–15.4 | 3.4 ± 4.8 | − 15.9–16.2 |
| Absolute error in Apple watch, steps | − 13.8 ± 30.5 | − 98.5–101.50 | − 6.0 ± 34.7 | − 143–123 | − 12.1 ± 33.2 | − 106–132 | 33.0 ± 45.5 | − 144–159.5 |
Figure 1 reports a Bland-Altman plot comparing the video steps and the Apple Watch steps. Comparison of the plots suggests good absolute mean-level differences in video steps and Apple Watch step estimates (mean difference = 1.07 steps, 95% CI − 82.62 to 84.76; P = 0.93). There was an overall negligible underestimation (1.07steps) of step count by Apple Watch. There was no proportional bias (b = − 0.03; t = − 0.083; P = 0.93) between the two measurements.
Fig. 1.
Bland Altman Plot comparing step counts on the treadmill between the Apple Watch and video steps.
Discussion
Our study is one of the initial studies to objectively validate the accuracy of the step counts obtained from Apple watch at different walking speeds on the treadmill. Apple Watch tested to be an extremely accurate device for measuring daily step counts for adults with different BMI categories and across different age groups. Although Apple Watch minimally overestimated steps at lower, moderate, and brisk speeds of walking and underestimated steps at a faster pace, it seemed to be most accurate at the moderate intensity pace. This might have a practical application to the general population, per American College of Sports Medicine recommendations on moderate intensity exercise.6 Our study is limited by being conducted with relatively healthy volunteers in a controlled setting, and results should be confirmed in other settings. Our findings may help reinforce individuals’ trust in using wearable devices such as Apple Watch to track health behaviors, which might impact their overall fitness levels.
Acknowledgements
We thank our study participants, Mr. Michael Briggs for statistical guidance, and Ms. Filomena Kilar for research assistance.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they do not have a conflict of interest.
References
- 1.Case MA, Burwick HA, Volpp KG, Patel MS. Accuracy of smartphone applications and wearable devices for tracking physical activity data. JAMA. 2015;313(6):625–626. doi: 10.1001/jama.2014.17841. [DOI] [PubMed] [Google Scholar]
- 2.Fokkema T, Kooiman TJ, Krijnen WP, et al. Reliability and validity of ten consumer activity trackers depend on walking speed. Medicine & Science in Sports & Exercise. 2017;49(4):793–800. doi: 10.1249/MSS.0000000000001146. [DOI] [PubMed] [Google Scholar]
- 3.Wallen MP, Gomersall SR, Keating SE, et al. Accuracy of Heart Rate Watches: Implications for Weight Management. PLoS ONE11. 2016;5:e0154420. doi: 10.1371/journal.pone.0154420. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.El-Amrawy F, Nounou MI. Are currently available wearable devices for activity tracking and heart rate monitoring accurate, precise, and medically beneficial? Healthcare informatics research. 2015;21(4):315–320. doi: 10.4258/hir.2015.21.4.315. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Diaz Keith M., Krupka David J., Chang Melinda J., Peacock James, Ma Yao, Goldsmith Jeff, Schwartz Joseph E., Davidson Karina W. Fitbit®: An accurate and reliable device for wireless physical activity tracking. International Journal of Cardiology. 2015;185:138–140. doi: 10.1016/j.ijcard.2015.03.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Garber Carol Ewing, Blissmer Bryan, Deschenes Michael R., Franklin Barry A., Lamonte Michael J., Lee I-Min, Nieman David C., Swain David P. Quantity and Quality of Exercise for Developing and Maintaining Cardiorespiratory, Musculoskeletal, and Neuromotor Fitness in Apparently Healthy Adults. Medicine & Science in Sports & Exercise. 2011;43(7):1334–1359. doi: 10.1249/MSS.0b013e318213fefb. [DOI] [PubMed] [Google Scholar]