Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Nov 1.
Published in final edited form as: Med Sci Sports Exerc. 2013 Nov;45(11):2105–2112. doi: 10.1249/MSS.0b013e318299d2eb

A Comparison of Energy Expenditure Estimation of Several Physical Activity Monitors

Kathryn L Dannecker 1, Nadezhda A Sazonova 2, Edward L Melanson 3,4, Edward S Sazonov 2, Raymond C Browning 1
PMCID: PMC3800491  NIHMSID: NIHMS487833  PMID: 23669877

Abstract

Accurately and precisely estimating free-living energy expenditure (EE) is important for monitoring energy balance and quantifying physical activity. Recently, single and multi-sensor devices have been developed that can classify physical activities, potentially resulting in improved estimates of EE.

PURPOSE

To determine the validity of EE estimation of a footwear-based physical activity monitor and to compare this validity against a variety of research and consumer physical activity monitors.

METHODS

Nineteen healthy young adults (10 male, 9 female), completed a four-hour stay in a room calorimeter. Participants wore a footwear-based physical activity monitor, as well as Actical, Actigraph, IDEEA, DirectLife and Fitbit devices. Each individual performed a series of postures/activities. We developed models to estimate EE from the footwear-based device, and we used the manufacturer's software to estimate EE for all other devices.

RESULTS

Estimated EE using the shoe-based device was not significantly different than measured EE (476(20) vs. 478(18) kcal) (Mean (SE)), respectively, and had a root mean square error (RMSE) of (29.6 kcal (6.2%)). The IDEEA and DirectLlife estimates of EE were not significantly different than the measured EE but the Actigraph and Fitbit devices significantly underestimated EE. Root mean square errors were 93.5 (19%), 62.1 kcal (14%), 88.2 kcal (18%), 136.6 kcal (27%), 130.1 kcal (26%), and 143.2 kcal (28%) for Actical, DirectLife, IDEEA, Actigraph and Fitbit respectively.

CONCLUSIONS

The shoe based physical activity monitor provides a valid estimate of EE while the other physical activity monitors tested have a wide range of validity when estimating EE. Our results also demonstrate that estimating EE based on classification of physical activities can be more accurate and precise than estimating EE based on total physical activity.

Keywords: Room calorimeter, oxygen consumption, free-living physical activity, shoe-based physical activity monitor

INTRODUCTION

Over 60% of the United States population is currently overweight, and concerns of the health risks associated with overweight and obesity are pervasive (8). The benefits of regular physical activity for weight maintenance and weight loss are well known (15), and recent data shows that prolonged sitting and inactive lifestyles may increase the risk of common chronic diseases (25, 33). Furthermore, caloric restriction when combined with physical activity improves metabolic and aerobic fitness (19). As a result, individuals attempting to lose or maintain weight are recommended to modify their diets to reduce energy intake, sit less and increase physical activity to increase energy expenditure (EE).

Most methods to estimate free-living EE have limitations that may prevent weight management success. Subjective measures of energy intake and EE (i.e. self-report surveys) can increase energy balance awareness but individuals typically under-report energy intake and over-report physical activity (4, 38). The gold-standard methods of indirect calorimetry and doubly labeled water are only feasible in the research settings because they are expensive and require specialized, technical equipment. Furthermore, doubly labeled water is limited in that it does not provide minute-by-minute EE data and thus can't provide details regarding physical activity EE. Therefore the best option for estimating total EE (TEE) is to use objective, minimally obtrusive devices that accurately and precisely quantify non-exercise activity thermogenesis (NEAT) and exercise EE.

Accelerometers are a common sensor used to measure the duration and intensity of PA (3). New technology has resulted in small, relatively unobtrusive accelerometers that are appealing to both researchers and consumers. Accelerometers typically use validated algorithms to estimate EE, achieving moderate to good validity in estimating physical activity energy expenditure (PAEE) in a research setting (standard error (SE) between 7.4% and 48.1% (1, 5)). However, accelerometers tend to underestimate PAEE and TEE when used in non-weight bearing activities and/or free-living environments (5, 14, 21, 26). While there are several brands of accelerometers that are currently used in research or available to consumers, no single study has compared the EE estimation validity of these devices against a gold-standard measure such as indirect room calorimetry.

To further improve estimates of EE using an objective measuring tool, new devices and algorithms that have the ability to detect posture and type of activity have recently been developed. These devices/algorithms are able to more accurately and precisely estimate EE as they can distinguish between activities that have different metabolic rates (e.g. stand vs. walk) and use activity specific EE relationships (2, 30, 34, 35). For instance, a neural network developed by Staudenmayer et al. improved the activity specific root mean squared error of the Actigraph accelerometer by up to 1.19 MET compared to the Freedson regression equation (30).

We have recently developed a footwear-based physical activity monitor that is intended to do three things: classify activity, measure weight, and estimate EE. In previous work, we demonstrated that this device is able to classify 6 major postures and activities (sitting, standing, walking, ascending stairs, descending stairs and cycling) with 98% accuracy (27). In a follow-up study, activity classification was used to develop accurate activity-specific EE estimation (28) but other physical activity monitoring devices were not tested simultaneously. Additionally, to improve the practicality of this device for weight management, a revised prototype has been developed that has updated accelerometry hardware and a new method of wireless communication with a smartphone. Therefore, the purpose of this study was to validate the use of this footwear-based physical activity monitor to estimate EE, and to compare the accuracy with EE estimated using other accelerometry based devices. We hypothesized that the EE estimation from the footwear-based device would not be significantly different from the measured EE via room calorimetry. We also hypothesized that other research and consumer devices that do not use activity classification would be less accurate and precise in estimating EE compared to the foot-wear based device.

METHODS

Subjects

Nineteen subjects (10 male, 9 female) were recruited from the Fort Collins and Denver communities to participate in this study (Table 1). The protocol was approved by the Colorado State University Institutional Review Board and participants gave written informed consent prior to beginning the study. Subjects completed a physical activity and health-history questionnaire (9), and were determined to be in good health by a physician. Based on self-report, subjects were inactive to moderately active (less than six hours of physical exercise per week), not taking any medications known to alter metabolism, and weight stable over the past six months.

Table 1.

Physical characteristics of participants

Subject Characteristics (N=19, 10M, 9F) Age (yrs) Height (m) Weight (kg) BMI (kg/m2)
26.9 (6.6) 1.73 (.10) 75.1 (17.1) 25.1 (4.6)
Open in a new tab

Values are mean (SD).

Study design

Each subject completed one 4-hour stay in a room calorimeter following a 4-hour fast. Prior to data collection we measured each subject's height and weight. Subjects wore six physical activity monitoring devices: one prototype shoe device (pair of shoes), three devices used in research, and two consumer devices. Prior to entering the room calorimeter, subjects were familiarized with the equipment in the room (e.g. cycle ergometer, treadmill). We recorded metabolic data while each individual performed a series of randomly assigned postures and activities (Table 2). The last hour of data collection consisted of free-living activities of the individual's choice. Walking activities were performed on a treadmill (Trainer 480 Treadmill, Gold's Gym Merchandising Inc., Irving, TX), cycling was performed on a stationary bicycle (Lode, Groningen, Netherlands), and stepping was performed by stepping up and down on a single eight-inch step (Reebok Step, Reebok Intl., Canton, MA).

Table 2.

Description of Protocol

Activity Description Time
Equilibration Quiet resting, data excluded 30 min
Supine laying on bed 20 min
Sitting watching TV 20 min
performing computer work 20 min
Standing Quiet 10 min
Active 10 min
Random assignment; 6 of 8 possible activities Walking, 2.5mph 10 min each; 60 min total
Walking, 3.5mph
Uphill, 2.5%, 2.5mph
Stepping
Sweeping
Cycling, 75W
Standing
Sitting
Free-living Any of the above activities, self-selected pace and posture 60 min, or until completion of 4 hours of data collection
Open in a new tab

Metabolic measurements

Oxygen consumption and carbon dioxide production were measured using the whole-room indirect calorimeter located in the Clinical Translational Research Center at the University of the Colorado Anschutz Medical Campus (23). The accuracy and precision of the system is tested monthly using propane combustion tests. The average O2 and CO2 recoveries during the period when the study was performed were 98.7 (0.7%) and 99.3 (0.1%) (mean(SD)), respectively. EE and substrate oxidation were calculated using the non-protein RQ based on the equations of Jequier et al. (16).

Prototype shoe device

Participants were fitted with the appropriately sized recreational walking shoes, equipped with a pressure sensing insole and accelerometer (Figure 1). Technical specifications of the prototype device have been explained in detail in previous work (27, 28), although the device used in the current study was equipped with a different accelerometer. The hardware included an insole that had five pressure sensors (force sensitive resistors) and a heel-mounted tri-axial accelerometer. Pressure and acceleration data were collected at 25 Hz from eight channels (five pressure and three acceleration)/shoe. Data were transmitted using a Bluetooth transmitter to a smart phone. The sensor system is lightweight (<40g) and non-obtrusive. We used a previously developed classification algorithm to classify activities into one of four posture/activity groups which were applicable to the activities performed in this study: “Sit”, “Stand”, “Walk”, and “Cycle”, (27).

Figure 1.

Figure 1

Open in a new tab

a) A pair of shoes equipped with sensors, wireless transmitter and batteries. Arrows show Anterior-Posterior (AAP), Medial-Lateral (AML) and Superior-Inferior (ASI) axes of accelerometer. b) A pressure sensitive insole with force sensitive resistors. PH is heel pressure sensor, PMO,PMM,PMI are 5th, 3rd and 1st metatarsal head sensors, respectively, and PHX is the hallux sensor.

Development and Validation of EE Model for Shoe Device

EE models were developed for each posture/activity using data from this experiment and methods described in Sazonova, et al. (28). A lag time of two minutes between the activity that the subject performed and the room calorimeter data was used as it produced the least error in the EE estimation. The EE models used anthropometric measurements, accelerometer and pressure sensor signals as predictors for an ordinary least squares linear regression. Metrics included: coefficient of variation (cv); standard deviation (std); number of zero crossings (zc); and entropy H of the distribution X of signal values. The median value of each of the four metrics combined from all five pressure sensors was used to form a single pressure sensor metric (med(metric)). The complete set of potential predictors consisted of 16 metrics: twelve (3×4) metrics from accelerometer sensors and four metrics from pressure sensors.

We used the “leave-one-out” approach for cross-validation of the footwear device when training and estimating the EE for each type of activity for every subject. The criteria for determining the best set of predictors was the model that provided the best fit (by producing the maximum adjusted coefficient of determination, R2adj and the minimum Akaike Information Criterion, AIC) in the training step and the best predictive performance (the minimum mean squared error, MSE and the minimum mean absolute error, MAE ) in the validation step.

Activity monitoring devices

Participants were equipped with three physical activity monitoring devices that are used in research: Actical (Phillips Respironics, Inc., Bend, OR), Actigraph GT3X (Actigraph, LLC., Pensacoloa, FL), and IDEEA (MiniSun, Fresno, CA). The Actical was set to record one minute epochs and we used the manufacturer's software to estimate EE, which is based on the work of Heil et al. (13). The Actigraph was set to record an epoch length of one second and we used manufacturer's software (Actilife version 5.10) and the Work/Energy and Freedson et al. algorithm to estimate EE (10). Both devices were worn on an elastic belt directly over either the right or left anterior superior iliac spine. The IDEEA has five sensors, which were placed under the sole of each foot, on each thigh and over the sternum. We used the estimated EE (per second) using the manufacturer's software with EE estimates based on the activity being performed.

Participants also wore two devices currently marketed to consumers: The Directlife activity monitor (Philips Electronics, Andover, MA) and Fitbit Tracker (Fitbit, Inc., San Francisco, CA). The Directlife activity monitor is a triaxial accelerometer. Data from the device was downloaded and EE (per hour) was estimated using the proprietary web-based software. The Fitbit Tracker is an accelerometer device that also uses a web-based software application to provide estimated EE to the user. We downloaded data and used the software to estimate EE (5-minute intervals). Both devices were worn on the same elastic belt that held the research activity monitors.

Device Comparison

TEE was calculated from the room calorimeter for the time period that corresponded with data collected from each device. The first 30 minutes of data from the room calorimeter were not used, as minute-to-minute readings during this time are not accurate due to time required for adequate respiratory gas mixture in the room. Thus, We compared measured EE to estimated EE from the Actical, Actigraph, IDEEA and Fitbit devices over a three and a half hour time period. Because the Directlife software only estimated hourly EE, we compared measured and estimated EE over a 3 hour period. Because some device software only calculated PAEE while other software estimated TEE, we adjusted for the difference by estimating resting metabolic rate (RMR). RMR was estimated using the Harris-Benedict equation (11) and then added to the EE estimated from Actical, Actigraph and Directlife to permit a comparison of TEE across devices.

EE estimated by the shoe-based device during validation was compared to estimations made from the five other devices. The shoe-based device used the previously described algorithms to calculate EE on a minute-to-minute basis over the entire three and a half hours of activity. We also developed two group-specific linear regression equations using the measured EE and Actical data so that we could compare of one of the research devices to the prototype device using the same participants. For each subject, we calculated the mean measured EE during the last three minutes of each activity (e.g. standing, walking at 2.5 MPH) and the mean Actical count for that same time period. We then used linear regression to determine the relationship between Actical counts and measured EE. The first linear regression equation included all activities, while the second excluded cycling, as EE associated with this activity was not well estimated by the accelerometer (EE increased but counts remained near zero). We also used the Actical group specific linear regression (no cycling) to estimate the EE associated with sitting, standing, walking and cycling. An estimate of EE was computed for each minute the activity of interest was performed during the protocol (excluding the free-living period) and averaged over the duration of the activity for each participant. We then compared the mean measured and estimated EE values for each activity. A further comparison was made with the Fitbit device to see if manually classifying activities via the web-based software would improve the EE validity of the device. The activity labeling works by classifying each activity performed during the wearing of the device, which then allows the software to apply to that time period a MET equivalent based on a compendium of physical activities.

Statistical Analysis

Mean standard error (SE), root mean squared error (RMSE) and the percentage of the RMSE with respect to the measured value (%RMSE) were calculated for each device. Because the equivalence of variance assumption was not met we used a Kruskal-Wallis one-way analysis of variance on ranks to test for significance between the measured and estimated values for each device and between the shoes and the other devices. To determine if there was a significant difference between measured and shoe/Actical estimated EE for each activity, we used a paired t-test. If the Shapiro-Wilk test of normality failed, we used a Mann-Whitney rank sum test. A p-value < .05 was considered significant.

RESULTS

Participants who experienced multiple sensor failures or incomplete data were excluded from the shoe device analysis, leaving 17 subjects with complete metabolic and sensor data. Because Our previous study demonstrated that only one shoe is required to obtain valid EE estimation, subjects with data from at least one shoe were included in the analysis (27). If data from both shoes were available, the average estimated EE is presented. Due to randomization of the activity protocol (see Table 2), only 12 subjects performed the cycling activity. Table 3 reports the mean EE of all subjects analyzed for each device (shoe EE model results for each activity are presented in the supplementary data, Table 1).

Table 3.

Mean measured and estimated EE for each device

Device (N) Measured EE (kcal) Estimated EE (kcal) RMSE (kcal) RMSE (%)
Shoes (17) 478.1 (20.0) 476.5 (18.4) 29.6 6.2
Actical* (19) 503.3 (19.2) 383.2 (16.9)a,b 130.2 25.9
Actical† (19) 503.3 (19.2) 558.2 (29.4) 101.7 20.2
Actical‡ (19) 503.3 (19.2) 527.9 (28.6) 89.7 17.8
Actigraph* (16) 494.2 (20.0) 375.0 (20.6)a,b 132.6 26.8
IDEEA (18) 504.2 (20.3) 445.3 (23.2) 88.2 17.5
Directlife*^ (19) 455.4 (17.8) 448.5 (13.1) 62.1 13.6
Fitbit (16) 499.0 (23.8) 362.8 (18.9)a,b 143.2 28.7
Fitbit-CL (16) 499.0 (23.8) 515.8 (13.0) 64.3 12.9
Open in a new tab

Mean room-measured and device-estimated EE, standard error (SE) of the estimate, root mean squared error (RMSE) and %RMSE. Measured EE values differ due to different sample sizes and participants for each device.

*

Harris-Benedict equation adjustment.

†

Estimated EE from subject-specific linear regression model (Actical only, all activities).

‡

Estimated EE from subject specific linear regression model without cycling (Actical only).

^

3-hour comparison.

a

Significant difference from measured.

b

Significant difference from Shoes.

Fitbit-CL: Fitbit using manual classification.

Device Comparison

EE estimation accuracy and precision varied according to the device (Table 3). The estimate of EE using the shoes was not significantly different than the measured EE (p=.955) and had the smallest RMSE of all devices. Out of the five research and consumer devices, the IDEEA and Directlife were not significantly different than the mean measured EE (p=.06 and .76, respectively). When we used regression models developed from the Actical data using the participants from this study to estimate EE (Figure 2), estimates of EE improved. Mean predicted EE was 558.2 kcal and 527.9 kcal using the equation including all activities and without cycling, respectively. In addition, RMSE values improved from 130.2 kcal (25.9%) using the manufacturer's software to 101.7 kcal (20.2%) using the all activities regression and 89.7 kcal (17.8%) using the regression that did not include cycling. Shoe and Actical estimates of EE during sitting, standing and walking were not significantly different than measured values but the Actical significantly underestimated the EE of cycling activity (Table 4). Fitbit had the largest RMSE of 143.2 kcal (28.7%, p<.001). However after labeling activities (Fitbit-CL), the mean RMSE was reduced to 64.3 kcal (12.9%). The unlabeled estimates always underestimated EE, while the classified values were underestimated about half of the time, and were more accurate in all but two subjects.

Figure 2.

Figure 2

Open in a new tab

EE vs. Actical counts. Linear regression lines are shown using data from all activities and data from all activities except cycling. All activity regression equation: y = 0.0018x + 2.1581 (R2 = 0.5666) (solid line); No cycling regression equation: y = 0.0019x + 1.8228 (R2 = 0.7244) (dashed line).

Table 4.

Activity specific measured and estimated EE for Shoes and Actical device

Activity Shoes (17) Actical (19)
Meas EE (kcal) Est EE (kcal) RMSE (%) Meas EE (kcal) Est EE (kcal) RMSE (%)
Sit 198.8 (13.1) 197.6 (11.7) 10.4 80.6 (3.4) 95.2 (5.0) 28.5
Stand 53.0 (8.6) 52.8 (8.1) 12.6 66.5 (4.8) 65.2 (4.9) 16.9
Walk 181.6 (14.9) 181.9 (14.3) 8.8 123.8 (11.0) 143.6 (14.9) 31.6
Cycle 67.7 (8.0) 67.2 (7.7) 14.8 59.1 (3.5) 21.9 (2.1)a 54.5
Open in a new tab

Mean (SE) of room-measured (Meas) and device-estimated (Est) EE for the shoe and Actical devices. Estimated EE for the Actical was from the group-specific regression without cycling and include specified sit, stand (standing and sweeping), walk (level and uphill) and cycling, but do not include the free-living time period. Measured EE values for each activity differ due to different protocols (not all participants completed all activities), sample sizes and participants for each device.

a

Significant difference from measured.

DISCUSSION

In this study, determined the validity of a shoe-based physical activity monitor which incorporates insole pressure sensors and triaxial accelerometry to classify major postures/activities and estimate EE. We hypothesized that this device would provide a valid estimate of EE compared to room calorimetry. Additionally, we hypothesized that consumer and research devices would be less accurate/precise when estimating EE. Our results demonstrate that the shoe-based device accurately estimated TEE (478.1(20.0) vs. 476.5(18.4) kcal, measured vs. estimated, respectively) with a %RMSE of 6.2%. Furthermore, of the five consumer and research devices, the DirectLife and IDEEA were also not significantly different than the measured value, but we observed greater %RMSE values of 13.6% and 17.5%, respectively compared to the shoe-based device.

This study demonstrates that an unobtrusive shoe-based physical activity monitoring device that combines plantar pressure and accelerometry can accurately and precisely estimate EE. The accuracy and precision of this device is likely due to the activity-specific EE models and the ability to detect changes in posture (e.g. sitting vs. standing). The activities with the best EE estimation validity were sitting (10.4 %RMSE) and walking (8.8 %RMSE), while standing and cycling were only slightly less accurate/precise (12.6 and 14.8 %RMSE respectively). The decreased accuracy and precision of predicting EE of standing may be attributed to the wide range of activities that were included in this classification, such as transitioning, active standing, quiet standing, and lifestyle activities that requires only arm movement (e.g. sweeping).

Each of the models developed to estimate EE utilized a different combination of the 14 possible metrics in the linear regressions (see supplementary data Table 1). The subject's weight and the log(BMI) were understandably predictive characteristics of EE in all four models, owing to the fact that an individual's weight and body composition are predictive of both resting and activity metabolic rate. Pressure and acceleration sensors at the foot allow the device to extract important information from the movement of the legs which relate to specific activities. For instance, during walking activity, the number of zero crossings (zc) of the acceleration in the anterior-posterior direction (Acc3) contributed to the prediction of EE. As the step frequency increases with an increase in the speed of ambulation, the number of zero crossing of the anterior-posterior acceleration will increase and thus contribute to the accurate prediction of EE during walking.

Overall, these results suggest that placing multiple sensor types at the foot is an effective method for estimating EE given that it allows for accurate classification of typical postures/activities. Generally, algorithms developed to estimate EE through classification have less bias, standard error and RMSE than estimations made by regression equations alone (30). However, a limitation of accelerometers is that movements with little or no trunk movement, such as standing and cycling, are most likely to be misclassified (35). The shoe-based device, with its capability of classifying standing and cycling activities with good accuracy, will therefore more accurately estimate the EE of these activities.

Currently, there is no consensus as to how many classes need to be used to distinguish between postures/activities with distinctly different metabolic demands. Our prototype shoe-based device only made estimations of EE based on four activity classes (supine, sit, stand and walk). Others have used single accelerometers and pattern recognition techniques to classify as many as 15 activities with relatively good classification accuracy (~90%) (2, 17, 20, 30). However, there is likely to be a balance between the number of necessary activity classifications and maintaining high classification accuracy. Recent attempts at classification have been employed in order to identify and distinguish low-to-moderate intensity activities (24, 36), and also to classify a wide range of activities from sedentary to those which are common for exercise (2, 30, 34). Future research should continue to examine which activities are necessary for accurate and precise EE estimation models and also practical for the function of the device as a weight management tool (e.g., to allow real-time activity feedback via a smartphone).

In addition to classifying activities, the relatively good validity of the footwear device may be due to the nature of our leave-one-out validation technique which used the same subjects to calibrate and validate the device. It is well known that group-specific models are most accurate and precise when they are applied to the same group from which they were created (7). For this reason, we elected to develop two group-based regression equations using the Actical to make a comparison of the shoe-based device with another device that used a group specific model. The Actical regressions using all activities and without cycling resulted in estimates of EE that were not significantly different from the mean measured EE value (p=.07 and p=0.42, respectively). Both group-specific regressions were more accurate/precise than using the manufacturer's software to estimate EE, but it should be noted that these regressions were not cross-validated in an independent sample and likely overestimate the ability to predict EE. We also examined activity-specific EE estimations of the shoes and Actical device (group specific linear regression without cycling) and found that, with the exception of the Actical during cycling, both devices estimates of EE were not significantly different than the measured values. However, the %RMSE was greater for the Actical compared to the shoes. A partial explanation for the relatively poor estimates of cycling EE using the Actical is that we used the regression equation that did not include cycling, but given the similarities in the linear regressions, this would only improve the predictive ability slightly. More importantly, these results highlight the challenge of using a hip-mounted accelerometer to estimate cycling EE given the small and workload independent accelerations experienced at the hip during this activity. In general, the activity-specific results suggest that both devices can provide reasonable estimates of EE during typical activities if a group-specific calibration is used. Software that uses group-based models are a current limitation in the field of physical activity monitoring because manufacturers typically supply the user with a regression based on a population of healthy, lean individuals, yet the device may be used by individuals who do not match this group (37). Future work should will need to determine whether the current algorithms are valid on a variety of populations (i.e. physically active, obese, children and elderly).

The use of commercially available physical activity monitors is becoming increasingly popular in research to objectively quantify physical activity at the individual and group level, as well as for personal use to monitor physical activity levels related to weight management and/or fitness goals. The validity of these devices is critical to quantifying current and changing levels in physical activity. Of the three research-based devices, only IDEEA was not significantly different than the mean EE, yet had a greater error than previously reported (39), and a moderately high RMSE. The mean EE estimated by the device had an 11.7% error against the room calorimeter, and the RMSE was 88.2 kcal during three and a half hours of data collection. The IDEEA device is unique among the commercially available devices validated in this study because it uses multiple sensors and sensor types, and more sophisticated algorithms to estimate EE. While being impractical for use outside of a research lab, the success of the IDEEA device illustrates the effectiveness of multiple sensors to provide a valid estimate of EE.

Like previous investigations, we found the Actical and Actigraph devices significantly underestimated EE during a protocol of sedentary to moderately vigorous activities (5, 6, 21, 22, 29). The Actical and Actigraph devices also had large %RMSE values of 25.9 and 26.8% respectively. One explanation for the limited performance of these devices is that we used a range of activities, including cycling, uphill walking and stepping, in our protocol. These activities are a challenge for hip mounted accelerometers because the acceleration magnitude and/or frequency doesn't scale with the metabolic demand (14, 31, 32). However, there are two limitations to our approach used to quantify EE using the Actical and Actigraph devices. First, we estimated resting metabolic rate, rather than using a subject-specific value. This likely introduced some error in the estimates of total EE. Second, we did not utilize pattern-recognition techniques to estimate EE. Recent studies that have used artificial neural networks to estimate EE from an Actigraph device have reported improvements in estimating EE of ~30-60% (30, 34). Therefore it is possible that the accuracy and precision of the Actical and/or Actigraph could be significantly improved using this approach and future studies are needed to confirm this possibility.

This study was the first to compare the validity of several consumer and research activity monitoring devices together against room calorimetry. Furthermore, to our knowledge it was the first EE validation of the Fitbit tracker, a device marketed for consumer use. One of the most accurate and precise devices overall was the Directlife consumer device, with an RMSE of 62.1 kcal (14%). Bonomi et al reported the Directlife device to provide a valid estimate of EE over a 14-day period using double labeled water with a standard error (SE) of the estimated TEE to be .9MJ per day, (8.96 kcal per hour) or 7.4% of the measured TEE (1). Our results determined that the SE was .44MJ per day, or 2.9%. The main disadvantage of Directlife device is the web-based software which only allows a user to determine EE on an hourly basis. While the hourly resolution may be sufficient for monitoring EE patterns over the course of several days, it may be inconvenient for individuals attempting to track changes in EE during specific period of the day (e.g. after work only). The time resolution also likely contributed to the error in a shorter study such as ours. Additionally individuals see only physical activity EE, so that RMR needed to be estimated from a prediction equation to make similar comparisons of total EE among all devices. With respect to the Fitbit device, it was accurate only after manual activity classification, a process that is very time consuming.

A shoe-based physical activity monitor that can provide a valid estimate of EE may be a practical tool for weight management. This device is minimally obtrusive as it would fit into an existing shoe, and the software could be accessed using a smartphone. Individuals would be able to track their EE as well as be able to see how they are spending their time. For instance, this device is able to detect changes in posture (e.g. time spent lying, sitting or standing) and could alert an individual to make more transitions to standing; which research shows may have health benefits (1212, 18). This device could therefore be implemented into existing weight management programs. Future research includes the development of a smartphone interface and quantifying changes in activities and associated EE during an intervention period.

In conclusion, a device that utilizes an instrumented insole and foot-mounted accelerometer can accurately and precisely estimate EE during typical free-living tasks. Other research and consumer physical activity monitors had a wide range of accuracy and precision when estimating EE. Collectively, these results support the use of multi-sensor devices that can accurately classify activity and use the activity classification to estimate EE, particularly in weight management applications.

Supplementary Material

1

ACKNOWLEDGEMENTS

The research would not have been possible without the assistance of the University of Colorado Clinical and Translational Research staff. This work was supported in part by NIH UL1 RR025789 and NIH SBIR grant R43DK083229 awarded to Physical Activity Innovations, Inc.

Funding: This work was funded by NIH grant R43DK083229 awarded to Physical Activity Innovations, Inc and NIH UL1 RR025789

Footnotes

CONFLICT OF INTEREST

Drs. Sazonov and Browning have an equity interest in Physical Activity Innovations, Inc., which sponsored the research. As a result, Drs. Sazonov and Browning have a potential conflict of interest as Physical Activity Innovations may benefit from the results of the present study. Both Drs. Sazonov and Browning have conflict of interest management plans in place at their respective institutions.

The results of this study do not constitute endorsement by the American College of Sports Medicine.

Disclosure of Professional Relationships: Ms Dannecker, Drs. Sazonova and Melanson have no relationships to disclose.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

REFERENCES

  • 1.Bonomi AG, Plasqui G, Goris AH, Westerterp KR. Estimation of free-living energy expenditure using a novel activity monitor designed to minimize obtrusiveness. Obesity. 2010;18(9):1845–51. doi: 10.1038/oby.2010.34. [DOI] [PubMed] [Google Scholar]
  • 2.Bonomi AG, Plasqui G, Goris AHC, Westerterp KR. Improving assessment of daily energy expenditure by identifying types of physical activity with a single accelerometer. J Appl Physiol. 2009;107(3):655–61. doi: 10.1152/japplphysiol.00150.2009. [DOI] [PubMed] [Google Scholar]
  • 3.Butte NF, Ekelund U, Westerterp KR. Assessing physical activity using wearable monitors: measures of physical activity. Med Sci Sports Exerc. 2012;44(Suppl 1):S5–12. doi: 10.1249/MSS.0b013e3182399c0e. [DOI] [PubMed] [Google Scholar]
  • 4.Corder K, Ekelund U, Steele RM, Wareham NJ, Brage S. Assessment of physical activity in youth. J Appl Physiol. 2008;105(3):977–87. doi: 10.1152/japplphysiol.00094.2008. [DOI] [PubMed] [Google Scholar]
  • 5.Crouter SE, Churilla JR, Bassett DR. Estimating energy expenditure using accelerometers. Eur J Appl Physiol. 2006;98(6):601–12. doi: 10.1007/s00421-006-0307-5. [DOI] [PubMed] [Google Scholar]
  • 6.Crouter SE, Clowers KG, Bassett DR. A novel method for using accelerometer data to predict energy expenditure. J Appl Physiol. 2006;100(4):1324–31. doi: 10.1152/japplphysiol.00818.2005. [DOI] [PubMed] [Google Scholar]
  • 7.Edwards AG, Hill JO, Byrnes WC, Browning RC. Accuracy of Optimized Branched Algorithms to Assess Activity-Specific PAEE. Med Sci Sports Exerc. 2010;42(4):672–82. doi: 10.1249/MSS.0b013e3181bd196d. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Flegal KM, Carroll MD, Ogden CL, Curtin LR. Prevalence and trends in obesity among US adults, 1999-2008. JAMA. 2010;303(3):235–41. doi: 10.1001/jama.2009.2014. [DOI] [PubMed] [Google Scholar]
  • 9.Franklin BH, Whaley GP, Howley ET. ACSM's Guidelines for Exercise Testing and Prescription. 6th ed. Lippincott Williams & Wilkins; Philadelphia, PA: 2000. pp. 22–27. [Google Scholar]
  • 10.Freedson PS, Melanson E, Sirard J. Calibration of the Computer Science and Applications, Inc. accelerometer. Med Sci Sports Exerc. 1998;30(5):777–81. doi: 10.1097/00005768-199805000-00021. [DOI] [PubMed] [Google Scholar]
  • 11.Harris J, Benedict F. A biometric study of basal metabolism in man. Carnegie Institute of Washington; Washington D.C.: 1919. pp. 1–12. [Google Scholar]
  • 12.Healy GN, Dunstan DW, Salmon J, Cerin E, Shaw JE, Zimmet PZ, et al. Breaks in sedentary time: beneficial associations with metabolic risk. Diabetes Care. 2008;31(4):661–6. doi: 10.2337/dc07-2046. [DOI] [PubMed] [Google Scholar]
  • 13.Heil DP. Predicting activity energy expenditure using the Actical (R) activity monitor. Res Q Exerc Sport. 2006;77(1):64–80. doi: 10.1080/02701367.2006.10599333. [DOI] [PubMed] [Google Scholar]
  • 14.Hendelman D, Miller K, Baggett C, Debold E, Freedson P. Validity of accelerometry for the assessment of moderate intensity physical activity in the field. Med Sci Sports Exerc. 2000;32(9 Suppl):S442–9. doi: 10.1097/00005768-200009001-00002. [DOI] [PubMed] [Google Scholar]
  • 15.Hill JO, Wyatt HR. Role of physical activity in preventing and treating obesity. J Appl Physiol. 2005;99(2):765–70. doi: 10.1152/japplphysiol.00137.2005. [DOI] [PubMed] [Google Scholar]
  • 16.Jequier E, Schutz Y. Long-term measurements of energy expenditure in humans using a respiration chamber. Am J Clin Nutr. 1983;38(6):989–98. doi: 10.1093/ajcn/38.6.989. [DOI] [PubMed] [Google Scholar]
  • 17.Khan AM, Lee YK, Lee SY, Kim TS. A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Trans Inf Technol Biomed. 2010;14(5):1166–72. doi: 10.1109/TITB.2010.2051955. [DOI] [PubMed] [Google Scholar]
  • 18.Kokkinos P, Sheriff H, Kheirbek R. Physical inactivity and mortality risk. Cardiol Res Pract. 2011;2011:924945. doi: 10.4061/2011/924945. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Larson-Meyer DE, Redman L, Heilbronn LK, Martin CK, Ravussin E. Caloric restriction with or without exercise: the fitness versus fatness debate. Med Sci Sports Exerc. 2010;42(1):152–9. doi: 10.1249/MSS.0b013e3181ad7f17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Lee MW, Khan AM, Kim JH, Cho YS, Kim TS. A single tri-axial accelerometer-based real-time personal life log system capable of activity classification and exercise information generation. Conf Proc IEEE Eng Med Biol Soc. 2010;2010:1390–3. doi: 10.1109/IEMBS.2010.5626729. [DOI] [PubMed] [Google Scholar]
  • 21.Leenders N, Sherman MW, Nagaraja HN, Kien CL. Evaluation of methods to assess physical activity in free-living conditions. Med Sci Sports Exerc. 2001;33(7):1233–40. doi: 10.1097/00005768-200107000-00024. [DOI] [PubMed] [Google Scholar]
  • 22.Lyden K, Kozey SL, Staudenmeyer JW, Freedson PS. A comprehensive evaluation of commonly used accelerometer energy expenditure and MET prediction equations. Eur J Appl Physiol. 2011;111(2):187–201. doi: 10.1007/s00421-010-1639-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Melanson EL, Ingebrigtsen JP, Bergouignan A, Ohkawara K, Kohrt WM, Lighton JR. A new approach for flow-through respirometry measurements in humans. Am J Physiol Regul Integr Comp Physiol. 2010;298(6):R1571–9. doi: 10.1152/ajpregu.00055.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Midorikawa T, Tanaka S, Kaneko K, Koizumi K, Ishikawa-Takata K, Futami J, et al. Evaluation of low-intensity physical activity by triaxial accelerometry. Obesity. 2007;15(12):3031–8. doi: 10.1038/oby.2007.361. [DOI] [PubMed] [Google Scholar]
  • 25.Owen N, Sparling PB, Healy GN, Dunstan DW, Matthews CE. Sedentary behavior: emerging evidence for a new health risk. Mayo Clin Proc. 2010;85(12):1138–41. doi: 10.4065/mcp.2010.0444. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Plasqui G, Westerterp KR. Physical activity assessment with accelerometers: An evaluation against doubly labeled water. Obesity. 2007;15(10):2371–9. doi: 10.1038/oby.2007.281. [DOI] [PubMed] [Google Scholar]
  • 27.Sazonov E, Fulk G, Hill J, Schutz Y, Browning R. Monitoring of Posture Allocations and Activities by a Shoe-Based Wearable Sensor. Trans Biomed Eng. 2011;58(4):983–90. doi: 10.1109/TBME.2010.2046738. [DOI] [PubMed] [Google Scholar]
  • 28.Sazonov N, Browning R, Sazonov E. Accurate prediction of energy expenditure using a shoe-based activity monitor. Med Sci Sports Exerc. 2011;43(7):1312–1321. doi: 10.1249/MSS.0b013e318206f69d. [DOI] [PubMed] [Google Scholar]
  • 29.Spierer DK, Hagins M, Rundle A, Pappas E. A comparison of energy expenditure estimates from the Actiheart and Actical physical activity monitors during low intensity activities, walking, and jogging. Eur J Appl Physiol. 2011;111(4):659–67. doi: 10.1007/s00421-010-1672-7. [DOI] [PubMed] [Google Scholar]
  • 30.Staudenmayer J, Pober D, Crouter S, Bassett D, Freedson P. An artificial neural network to estimate physical activity energy expenditure and identify physical activity type from an accelerometer. J Appl Physiol. 2009;107(4):1300–7. doi: 10.1152/japplphysiol.00465.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Swan PD, Byrnes WC, Haymes EM. Energy expenditure estimates of the Caltrac accelerometer for running, race walking, and stepping. Br J Sports Med. 1997;31(3):235–9. doi: 10.1136/bjsm.31.3.235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Terrier P, Aminian K, Schutz Y. Can accelerometry accurately predict the energy cost of uphill/downhill walking? Ergonomics. 2001;44(1):48–62. doi: 10.1080/00140130118289. [DOI] [PubMed] [Google Scholar]
  • 33.Tremblay MS, Colley RC, Saunders TJ, Healy GN, Owen N. Physiological and health implications of a sedentary lifestyle. Appl Physiol Nutr Metab. 2010;35(6):725–40. doi: 10.1139/H10-079. [DOI] [PubMed] [Google Scholar]
  • 34.Trost SG, Wong WK, Pfeiffer KA, Zheng Y. Artificial neural networks to predict activity type and energy expenditure in youth. Med Sci Sports Exerc. 2012;44(9):1801–9. doi: 10.1249/MSS.0b013e318258ac11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.van Hees V, Ekelund U. Novel daily energy expenditure estimation by using objective activity type classification: where do we go from here? J Appl Physiol. 2009;107(3):639–40. doi: 10.1152/japplphysiol.00793.2009. [DOI] [PubMed] [Google Scholar]
  • 36.van Hees VT, van Lummel RC, Westerterp KR. Estimating activity-related energy expenditure under sedentary conditions using a tri-axial seismic accelerometer. Obesity. 2009;17(6):1287–92. doi: 10.1038/oby.2009.55. [DOI] [PubMed] [Google Scholar]
  • 37.Van Remoortel H, Giavedoni S, Raste Y, Burtin C, Louvaris Z, Gimeno-Santos E, et al. Validity of activity monitors in health and chronic disease: a systematic review. Int J Behav Nutr Physical Activity. 2012;9(1):84. doi: 10.1186/1479-5868-9-84. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Walsh MC, Hunter GR, Sirikul B, Gower BA. Comparison of self-reported with objectively assessed energy expenditure in black and white women before and after weight loss. Am J Clin Nutr. 2004;79(6):1013–9. doi: 10.1093/ajcn/79.6.1013. [DOI] [PubMed] [Google Scholar]
  • 39.Zhang K, Pi-Sunyer FX, Boozer CN. Improving energy expenditure estimation for physical activity. Med Sci Sports Exerc. 2004;36(5):883–9. doi: 10.1249/01.mss.0000126585.40962.22. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS487833-supplement-1.docx (15.4KB, docx)

RESOURCES