Abstract
Daily step counts from the Withings Activite were validated against those collected concurrently from the PiezoRxD Pedometer and the wGT3X-BT Actigraph worn on the waist and on the wrist in free-living conditions from 10 older adult volunteers. The Withings Activite underestimated step counts but showed good correlations with the other devices (Pearson correlation coefficient: 0.850 – 0.891).
I. Introduction
Studies have suggested that there is a high correlation between activity levels and overall health status[1, 2]. A widely used, simple proxy for activity levels is step counts. For example, a study showed that number of recorded steps per day is a predictor of cognitive performance in older adults[3]. With the rise of wearables, it has become increasingly more feasible to capture steps from people in free-living conditions with many devices ranging from so-called “research-grade” actigraphs to commercial or “consumer-grade” smartwatches available. However, there has been a lack of standardization or, in the case of many commercial wearables, lack of access to the primary data or description of the step-counting algorithm[4]. The commercial watch-like devices, in particular, offer advantages (e.g., very long battery life, appealing form factors) that make them attractive for conducting longitudinal research in older or chronically ill patients. However, using the estimated step counts in free-living conditions from these devices often raises the question of how accurate the step counts may be.
In this context, for the last couple years as part of the Oregon Center for Aging & Technology (ORCATECH) platform[5] the consumer-grade Withings (Issy-les-Moulineaux, France) Activite has been used to monitor participants’ daily step counts who are participating in studies such as the national Collaborative Aging Research using Technology (CART) initiative[6, 7] and the Ecologically Valid Ambient Longitudinal and Unbiased Assessment of Treatment Efficacy in Alzheimer’s Disease (EVALUATE-AD)[8]. These studies consist of a network of homes in which residents go about their daily lives while researchers at ORCATECH measure and observe their interactions with sensing technology. Due to the longitudinal nature of these studies and the concern over data loss if devices needed to be charged every few days, the Withings Activite was chosen for its appealing form factor and its long battery life (6 months or more).
While many other consumer grade activity watches have been validated for step count accuracy, there is no published study that has measured specifically how well the Withings Activite is able to estimate steps. Given the familiar form factor and long battery life, the Withings Activite is a prime candidate for studies monitoring participant activity long-term, especially in the older population. However, for wider adoption and sharing of data of this device in research, the step count accuracy needs to be validated.
In this paper, we present the validation results of step counts from the Withings Activite against other extensively used and validated step estimation devices in older adults in free-living conditions.
II. Methods
A. Participants
Participants were recruited from the cohort already enrolled in the CART initiative. Oregon Health & Science University (OHSU) institutional review board (IRB) approved all experimental procedures involving human subjects (Study #18634). Each subject signed a consent form to enroll in the study.
Eleven subjects enrolled in the study, but data were collected from only 10 subjects due to technical difficulties. Table 1 shows the characteristics of the 10 study participants from whom data were successfully collected. Their mean age was 72.4 with a standard deviation of 5.50.
Table I.
Participant characteristics
| Demographic features | Count (%) or Mean (SD) | |
|---|---|---|
| Age group | 60–69 years | 2 (20%) |
| 70–79 years | 6 (60%) | |
| 80–89 years | 2 (20%) | |
| Sex | Male | 3 (30%) |
| Female | 7 (70%) | |
| Body mass index (BMI) | 31.1 (3.79) | |
| Cumulative Illness Rating Scale (CIRS) | 21.4 (3.14) | |
B. Devices
The devices used in this study included the Withings Activite, the wGT3X-BT from ActiGraph (Pensacola, Florida, United States of America), and the PiezoRxD from StepsCount (Deep River, Ontario, Canada).
The Withings Activite records step count at a per minute level. The ActiGraph wGT3X-BT captures high-resolution raw acceleration data. This device has been well-validated and used for assessing physical activity (PA) under free-living conditions[9]. The wGT3X-BT (referred to throughout the rest of the paper as the Actigraph) and other similar devices from ActiGraph have also been used in several large-scale epidemiological studies[10, 11]. The Actigraphs were set to have a sampling rate of 30Hz during deployment. The PiezoRxD (referred to throughout the rest of the paper as the Pedometer) from StepsCount is a medical-grade PA monitor and has been validated for measuring step count and moderate-to-vigorous PA in both laboratory settings and free-living conditions[12, 13].
C. Study Design and Data Collection
Each enrolled participant wore four devices in total: the (1) Pedometer and the (2) Actigraph on the waist; and the (3) Withings Activite and the (4) Actigraph on the non-dominant wrist for a maximum duration of two weeks under free-living conditions. Fig. 1 shows a photo of how the four devices were worn during the study. The four devices monitored each participant’s PA simultaneously. The participants were instructed to take off the devices only when they would be exposed to water, such as during showers. Since this was a free-living experiment and counting the actual number of steps taken by participants was impossible, we used the steps measured by the validated Pedometer as the ground truth.
Figure 1.
Experimental Setup. The (1) Pedometer and the (2) Actigraph on the waist; and the (3) Withings Activite and the (4) Actigraph on the non-dominant wrist.
D. Data Treatment
The Withings Activite and the Pedometer use proprietary algorithms to calculate step numbers on the respective devices. Alternatively, the Actigraph provides access to raw accelerometer data. The ActiLife 6 platform (companion software purchased from ActiGraph) was used to derive steps from the acceleration signals collected using the Actigraphs. When downloading the data from the Actigraphs using the ActiLife 6 platform, the low frequency extension was not selected in order to avoid overestimation of steps[14].
E. Data Analysis
The daily steps obtained from the four devices were summarized using descriptive statistics. The correlations of daily steps collected using the four devices were examined using Pearson correlation coefficients. The daily steps were compared pairwise across two devices at a time using linear regression models, scatter plots, and Bland-Altman plots (Withings Activite vs the Pedometer, Withings Activite vs the Actigraph on the waist, Withings Activite vs the Actigraph on the wrist).
III. Results
Table 2 shows the descriptive statistics of the daily steps collected from the 10 participants. Some data loss for the Actigraph worn on the waist occurred due to one participant losing the device during the data collection period. There were also data loss for the Actigraph worn on the wrist due to water damage during data collection for one participant.
Table II.
Descriptive statistics of daily steps collected from the 10 participants using the 4 devices
| Withings | Actigraph on the waist | Actigraph on the wrist | Pedometer | |
|---|---|---|---|---|
| Count | 97 | 84 | 91 | 97 |
| Mean | 2950 | 3889 | 8374 | 5048 |
| Standard Deviation | 2218 | 2606 | 3886 | 3635 |
| Minimum | 92 | 455 | 1030 | 226 |
| 25 percentile | 1099 | 1699 | 5390 | 2369 |
| Median | 2725 | 3569 | 8040 | 4184 |
| 75 percentile | 3844 | 5248 | 10649 | 6717 |
| Maximum | 11388 | 13262 | 21143 | 16612 |
From Table 2, it can be observed that the Withings Activite estimated the fewest number of steps, followed by the Actigraph on the waist, and then the Pedometer. The Actigraph on the wrist estimated the most number of steps.
Table 3 shows the Pearson correlation coefficients for pairwise comparisons between the devices. The daily steps from Withings Activite showed high correlation with the other three devices (Pearson correlation coefficients: 0.850 – 0.891). Interestingly the pair of devices with the lowest observed correlation were the Actigraph worn on the wrist and the Pedometer (Pearson correlation coefficient: 0.823).
Table III.
Pearson correlation coefficient between devices
| Withings | Actigraph on the waist | Actigraph on the wrist | Pedometer | |
|---|---|---|---|---|
| Withings | ╲ | 0.891 | 0.850 | 0.856 |
| Actigraph on the waist | ╲ | ╲ | 0.861 | 0.908 |
| Actigraph on the wrist | ╲ | ╲ | ╲ | 0.823 |
| Pedometer | ╲ | ╲ | ╲ | ╲ |
Table 4 shows the slope and intercept of the respective linear regression models. The slopes indicate the relative number of steps the Withings Activite would output per each step count output for the respective other devices. The intercepts can be interpreted as the general offset or overall difference between the Withings Activite and the respective other devices. The slope being one with the intercept being zero means perfect agreement. It can be observed that the Withings Activite aligns the most with the Actigraph worn on the waist and the least with the Actigraph worn on the wrist.
Table IV.
Linear regression model
| Slope (95% CI) | Intercept (95% CI) | |
|---|---|---|
| Withings vs Pedometer | 0.522 (0.458, 0.586) | 315 (−84.8, 714) |
| Withings vs Actigraph on the waist | 0.782 (0.695, 0.870) | −111 (−521, 298) |
| Withings vs Actigraph on the wrist | 0.474 (0.412, 0.536) | −1180 (−1750, −608) |
Scatter plots and Bland-Altman plots for each of the pairwise comparisons are shown in Figs. 2, 3, and 4. From the scatter plots, it is readily observed that the Withings Activite tends to measure fewer steps compared to the other three devices.
Figure 2.
Left: Scatter plot of the Withings Activite vs the Pedometer with regression line. Right: Bland-Altman plot of the Withings Activite and the Pedometer.
Figure 3.
Left: Scatter plot of the Withings Activite vs the Actigraph worn on the waist with regression line. Right: Bland-Altman plot of the Withings Activite and the Actigraph worn on the waist.
Figure 4.
Left: Scatter plot of the Withings Activite vs the Actigraph worn on the wrist with regression line. Right: Bland-Altman plot of the Withings Activite and the Actigraph worn on the wrist.
From the Bland-Altman plots, the difference between the step counts from the devices vs the mean of the step counts from the devices can be more readily observed. The mean difference is the smallest between the Withings Activite and the Actigraph worn on the waist (958.8) while the mean difference is the largest between the Withings Activite and the Actigraph worn on the wrist (5583.7). From these Bland-Altman plots, it also can be observed that as the mean daily steps from the two devices gets larger, the difference between the daily steps from the two devices increases as well. Such differences, however, could be corrected by performing linear transformations using the parameters from Table 4.
IV. Discussion
Although the Withings Activite tended to underestimate step counts as compared to the other well-validated devices, the Withings Activite should be considered a viable option for measuring the relative PA levels of study participants as we have shown that estimated steps from the Withings Activite are highly correlated with those from devices regularly used as the gold standards for step and activity estimation[9–13]. In addition, the linear regression parameters presented in Table 4 could be used to transform the step counts estimated by the Withings Activite to generate step counts more aligned with the aforementioned gold standards. Judging from the scatterplots presented in Figs. 2–4, linear regression is a reasonable method to relate the step estimation from these devices. Future work will examine further if other types of transformation are more suitable. Given the high correlations with the gold standards and the advantages of a long battery life (6+ months) and a desirable form factor, the Withings Activite is a good candidate for long-term studies interested in the relative activity levels of their participants. This is especially true for studies in populations (e.g. aging studies) where the frequent charging and non-conventional appearance of traditionally available consumer activity trackers can be a barrier.
It should be noted that the user’s manual for the Actigraph recommends that the Actigraph should be worn as close to the center of mass of the participant as possible and that the step algorithms for wrist-worn Actigraph have not been validated. The results presented herein show that the wrist-worn Actigraphs recorded many more daily steps as compared to the waist-worn Actigraphs. This finding is consistent with that from a previous study[15]. It is assumed that the wrist-worn actigraphy devices recorded “steps” even when the participants were only sitting and moving their hands. We also speculate that the consistent underestimate of steps by the Withings Activite may be the result of an effort by the proprietary algorithm to filter out steps caused by extraneous hand/arm movements. It is interesting to note from Table 3 that the Withings Activite was more highly correlated with the Pedometer than the Actigraph worn on the wrist.
To conclude, our results validate the Withings Activite for measuring relative PA levels of the participants despite a consistent underestimation of absolute number of steps compared to the other devices commonly used in research. This opens up opportunities for the informed use of the Withings Activite as a PA monitor in future clinical studies.
Clinical Relevance —
Although the Withings Activite underestimated steps, they may be used in studies to estimate relative level of physical activity in free-living conditions since they have good correlations with other well-validated devices. Underestimation of steps may be corrected using linear transformation.
Acknowledgment
We thank the participants for taking part in the study. We also thank the technical staff at ORCATECH for their efforts in recruitment of subjects and deployment of the devices.
References
- [1].Kraus WE et al. , “Daily step counts for measuring physical activity exposure and its relation to health,” Medicine & Science in Sports & Exercise, vol. 51, no. 6, pp. 1206–1212, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2].Ewald B, McEvoy M, and Attia J, “Pedometer counts superior to physical activity scale for identifying health markers in older adults,” British Journal of Sports Medicine, vol. 44, no. 10, pp. 756–761, 2010. [DOI] [PubMed] [Google Scholar]
- [3].Calamia M, De Vito A, Bernstein JP, Weitzner DS, Carmichael OT, and Keller JN, “Pedometer-assessed steps per day as a predictor of cognitive performance in older adults,” Neuropsychology, vol. 32, no. 8, p. 941, 2018. [DOI] [PubMed] [Google Scholar]
- [4].Bassett DR, Toth LP, LaMunion SR, and Crouter SE, “Step counting: a review of measurement considerations and health-related applications,” Sports Medicine, vol. 47, no. 7, pp. 1303–1315, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5].Kaye J et al. , “Methodology for establishing a community-wide life laboratory for capturing unobtrusive and continuous remote activity and health data,” JoVE (Journal of Visualized Experiments), no. 137, p. e56942, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Kaye J et al. , “THE COLLABORATIVE AGING RESEARCH USING TECHNOLOGY (CART) INITIATIVE: BUILDING SCALABLE, ECOLOGICALLY VALID INFORMATION COMMUNICATION TECHNOLOGY RESEARCH CAPACITY,” Alzheimer’s & Dementia: The Journal of the Alzheimer’s Association, vol. 13, no. 7, p. P1217, 2017. [Google Scholar]
- [7].Kaye J et al. , “COLLABORATIVE AGING RESEARCH USING TECHNOLOGY: NEW PATHWAYS FORWARD,” Innovation in Aging, vol. 3, no. Suppl 1, p. S832, 2019. [Google Scholar]
- [8].Thomas NW, Lindauer A, and Kaye J, “EVALUATE-AD and Tele-STAR: novel methodologies for assessment of caregiver burden in a telehealth caregiver intervention–a case study,” Dementia and geriatric cognitive disorders, vol. 47, no. 3, pp. 176–184, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [9].Aadland E and Ylvisåker E, “Reliability of the Actigraph GT3X+ accelerometer in adults under free-living conditions,” PloS one, vol. 10, no. 8, p. e0134606, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [10].Lee I-M and Shiroma EJ, “Using accelerometers to measure physical activity in large-scale epidemiological studies: issues and challenges,” Br J Sports Med, vol. 48, no. 3, pp. 197–201, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [11].Boyer WR, Wolff-Hughes DL, Bassett DR, Churilla JR, and Fitzhugh EC, “Accelerometer-Derived Total Activity Counts, Bouted Minutes of Moderate to Vigorous Activity, and Insulin Resistance: NHANES 2003–2006,” Preventing chronic disease, vol. 13, pp. E146–E146, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [12].O’Brien MW, Wojcik WR, and Fowles JR, “Medical-grade physical activity monitoring for measuring step count and moderate-to-vigorous physical activity: Validity and reliability study,” JMIR mHealth and uHealth, vol. 6, no. 9, p. e10706, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [13].O’BRIEN MW, Wojcik WR, D’ENTREMONT L, and Fowles JR, “Validation of the PiezoRx® step count and moderate to vigorous physical activity times in free living conditions in adults: a pilot study,” International journal of exercise science, vol. 11, no. 7, p. 541, 2018. [PMC free article] [PubMed] [Google Scholar]
- [14].Feito Y, Hornbuckle LM, Reid LA, and Crouter SE, “Effect of ActiGraph’s low frequency extension for estimating steps and physical activity intensity,” PloS one, vol. 12, no. 11, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [15].Tudor-Locke C, Barreira TV, and Schuna JJ, “Comparison of step outputs for waist and wrist accelerometer attachment sites,” Medicine and science in sports and exercise, vol. 47, no. 4, pp. 839–842, 2015. [DOI] [PubMed] [Google Scholar]