Comparisons of prediction equations for estimating energy expenditure in youth

Abstract

Objectives

The purpose of this study was to compare the validity of Actigraph 2-regression models (2RM) and 1-regression models (1RM) for estimation of EE in children.

Design

The study used a cross-sectional design with criterion estimates from a metabolic cart

Methods

A total of 59 children (7–13yrs) performed 12 activities (randomly selected from a set of 24 activities) for 5min each, while being concurrently measured with an Actigraph GT3X and indirect calorimetry. MET_RMR (MET considering one's resting metabolic rate) for the GT3X was estimated applying 2RM with vector magnitude (VM2RM) and vertical axis (VA2RM), and four standard 1RMs. The validity of the 2RMs and 1RMs was evaluated using 95% equivalence testing and mean absolute percent error (MAPE).

Results

For the group-level comparison, equivalence testing revealed that the 90% confidence intervals for all 2RMs and 1RMs were outside of the equivalence zone (range: 3.63, 4.43) for indirect calorimetry. When comparing the individual activities, VM2RM produced smaller MAPEs (range: 14.5 to 45.3%) than VA2RM (range, 15.5 to 58.1%) and 1RMs (range, 14.5 to 75.1%) for most of the light and moderate activities.

Conclusions

None of the 2RMs and 1RMs were equivalent to indirect calorimetry. The 2RMs showed smaller individual-level errors than the 1RMs.

Introduction

The Actigraph (Actigraph, Pensacola, FL) is the most widely used activity monitoring device but there is still debate about the most appropriate way to reduce and process the data.¹ A particularly challenging issue is to convert accelerometer counts into more useable units (a process typically referred to as calibration). Calibration equations are typically developed using regression analyses that link counts with corresponding energy expenditure (EE) values (metabolic equivalents: METs) measured from a criterion measure.² Four independent calibration studies^3-6 have proposed equations to estimate children's EE but cross-validation studies^{7, 8} have not supported their accuracy for assessing free living physical activity behavior.

The inherent limitation of these standard single regression models (referred to as 1RM hereafter) is that they are specific for the activities used in the calibration.⁹ Previous research by Crouter et al.^{10, 11} demonstrated the utility of a two-regression model (2RM) that allows for the use of separate regression equations for locomotive (i.e. running, walking) and non-locomotive activities. This enables separate predictions for these two distinct patterns. In subsequent independent validation studies, this method was shown to provide comparable EE values relative to doubly-labeled water¹² and indirect calorimetry¹³ in adults. A set of 2RMs was developed to estimate EE in youth using the latest, tri-axial Actigraph GT3X model,¹⁴ but a subsequent cross-validation study⁷ yielded equivocal findings. This study⁷ reported significant underestimation of EE for the seven structured activities (performed for 5 min each) and considerable error across the more extended, free-living measurement session. The discrepant findings are difficult to reconcile considering the promising findings in adults¹³ and the favorable preliminary validation results.¹⁴ It is widely recommended that validity of accelerometry-based monitors should be examined using independent samples^{15, 16} and under various settings.² The present study systematically evaluates the relative validity of two 2RM models compared to four different 1RM equations in an independent sample of youth.

Method

A total of 59 children between the ages of 7 and 13 volunteered to participate in the study. All of the participants and their parents signed an assent form and written informed consent, respectively, prior to participation. In Table 1, the demographic profiles of the participants were summarized. This study was approved by the Institutional Review Board at our institution.

Table 1. Physical characteristics of the participants (n=59).

Variables	Values
Age (M ± SD, range)	9.9 ± 1.8 yrs, 7-13 yrs
Gender
Female (n=18)	30.5%
Male (n=41)	69.5%
Height (M ± SD)	146.0 ± 12.9 cm
Weight (M ± SD)	38.9 ± 13.8 kg
Body Mass Index (BMI) a
Normal weight (n=51)	79.7%
Overweight (n=6)	13.6%
Obese (n=3)	6.8%
Race
Caucasian (n=56)	93.2%
Asian (n=2)	3.4%
African American (n=1)	1.7%
Hispanic (n=1)	1.7%
Resting Metabolic Rate
All (n=59)	4.89 ml⋅kg-1·min-1
Female (n=18)	4.75 ml⋅kg-1·min-1
Male (n=41)	4.96 ml⋅kg-1·min-1

^aBMI; 5^th – 85^th percentile as normal Weight, 85^th – 95^th percentile as overweight, ≥95^th as obese

Each participant performed 12 activities randomly selected from a set of 24 activities designed to mimic free-living activities in children. The 24 activities were classified into four distinct intensity categories according to MET-derived criteria based on criterion MET values from the OM. The criteria were MET_RMR≤2.0 for sedentary, 2.0< MET_RMR ≤3.9 for light, 4.0≤MET_RMR ≤5.9 for moderate, and 6.0≤MET_RMR for vigorous intensity. We used adjusted MET_RMR values of 2.0 and 4.0 for the sedentary and moderate intensity, respectively, instead of traditional MET_RMR values of 1.5 and 3.0. The higher threshold of 4 METS for MVPA⁸ is well established based on the established evidence for higher RMR values in youth (∼5.0ml·kg^-1·min^-1) for children compared with adults (∼3.5ml·kg^-1·min^-1).¹⁷ The lower threshold of 2 METS to define Sedentary behavior preserves the same 2:1 ratio between MVPA and Sedentary in adults (i.e. 3.0 METS: 1.5 METS). Recent findings from our lab (manuscript in review) have supported the high classification accuracy of the 2.0 and 4.0 MET_RMR for classifying sedentary and moderate intensity in children.

The activities were completed in random order and in a semi-structured manner to capture natural variation in physical activity. The characteristics of the included activities by intensity along with corresponding means and standard deviations are presented (see Supplementary Table 1). Each activity was performed for 5 min, with a 1-min resting period between them. Participants wore an Actigraph GT3X for measurement of physical activity and were fitted with a portable metabolic analyzer (i.e. Oxycon Mobile (OM); CareFusion Corp, San Diego, CA) for measurement of pulmonary ventilation and gas exchange (for determination of oxygen uptake). The funding organization (i.e. National Institutes of Health) did not play any role in collecting data (including the analyses and interpretation) and in approving or disapproving the publication of this study.

The Actigraph GT3X is a compact (3.8×3.7×1.8cm) and lightweight (27g) tri-axial accelerometer that records acceleration ranging from 0.05g to 2.00g. The Actigraph GT3X was initialized at 30Hz and placed on the right hip at the level of iliac crest. Data were downloaded at 1s epoch and then reintegrated into 10s epochs and 60s for analyses. The ActiLife software (firmware version 6.5.1) was used for data management.

The OM, the criterion measure for determination of EE, is a portable gas analyzer that measures breath-by-breath respiratory gas exchange under free-living conditions. Volume and gas calibration were performed before each trial following the manufacturer's recommendations. The OM has been shown to be valid and reliable.^{18, 19}

The Actigraph data were processed using 2 different 2RM approaches,¹⁴ one developed with vertical axis data (VA2RM) and one developed using vector magnitude data (VM2RM). The Actigraph data were also processed for the 1RM equations developed by Freedson/Trost et al. (FT1RM),³ Trost et al. (TR1RM),⁶ Puyau et al. (PU1RM),⁴ and Treuth et al. (TH1RM)⁵ to provide comparisons with previous results (see Supplementary Table 2). To ensure effective comparisons, all data were presented using equivalent outcome units of MET_RMR. MET_RMR is a MET value that incorporates one's resting metabolic rate (RMR). In this study, the Schofield equation²⁰ was used to predict RMR of the children, which is the same methodology used in previous validation studies.^{21, 22, 8, 23} The specific MET_RMR value was calculated by dividing the measured activity VO₂ value (i.e. ml·kg^-1·min^-1) by the predicted (Schofield²⁰) RMR value (i.e. ml·kg^-1·min^-1).

The 2RM method by Crouter et al.¹⁴ predicts EE in MET_RMR. However, FT1RM and TH1RM yield MET_3.5, which is a MET value based on a resting metabolic rate of 3.5ml·kg^-1·min^-1. Therefore, MET_RMR values for the FT1RM and TH1RM were obtained by first multiplying MET values predicted from the equations by 3.5ml·kg^-1·min^-1, then dividing by a RMR estimated using the Schofield equation.²⁰ The TR1RM and PU1RM provide estimates of EE in Kcal·min^-1 and Kcal·kg·min^-1, respectively. Therefore, different conversions were needed to compute MET_RMR. An established conversion factor (1 L O₂ = 4.825 kcal) was used to convert EE estimates into estimates of oxygen consumption in ml·kg^-1·min^-1. This was then divided by the weight (kg) and the estimated RMR value for TR1RM, and by the estimated RMR value for PU1RM to compute MET_RMR.²³ For the OM data, MET_RMR values were calculated by dividing VO₂ values (ml·kg^-1·min^-1) of performed activities by the estimated RMR value.²⁰

A statistical “equivalence test”^{24, 25} was used to examine measurement agreement between the Actigraph equations (i.e. 2RMs and 1RMs) and the OM. In traditional analytic approaches, the null hypothesis is that the two methods are equal. Failure to reject the null suggests “no evidence of a difference”; however, this does not necessarily imply that there is “evidence of equivalence”.²⁶ With an equivalence test, the null hypothesis is flipped to specify a difference between two means, thereby allowing a direct test of equivalence. In the context of the present study, a 95% equivalence test (i.e. α=5%) would conclude that Actigraph EE models (i.e. 2RMs and 1RMs) and the OM are considered to be significantly equivalent to each other if a 90% of confidence interval (CI) for a mean of the Actigraph EE model fell into a proposed equivalence zone (i.e. ±10% of the mean) of the OM.

To evaluate overall agreement between the Actigraph EE models and OM across all the activities, the 90%CIs for means of each Actigraph EE model were estimated from a mixed model ANOVA in which participants are included as a random effect to account for potential correlation of measurements from the same participant. Whether or not the Actigraph measurement for each of the 24 activities was significantly equivalent to the corresponding OM measurement was tested using an equivalence test with an alpha-level of 5%. The null hypothesis of non-equivalence was rejected when the 90% confidence interval for the mean Actigraph EE measurement fell inside the equivalence zone, which we defined as ±10% of the mean for OM. The equivalence testing and Pearson correlation coefficient were used to examine group-level agreement.

To examine individual-level measurement errors, mean absolute percent errors (MAPE) and Bland Altman plots were used. MAPE was calculated by dividing an absolute difference between a regression model and the OM by the OM, multiplied by 100%. Bland-Altman plots with corresponding 95% limits of agreement and fitted lines (from regression analyses between mean and difference) with their corresponding parameters (i.e. intercept, slope) were presented. Perfect agreement between the Actigraph and OM would be identified with a fitted line that provides a slope of 0 and an intercept of 0.

Results

Overall, none of the regression models (i.e. VM2RM, VA2RM, FT1RM, TR1RM, PU1RM, TH1RM) were significantly equivalent to the OM for the group-level comparisons as none of the 90%CIs of the 6 regression models were completely included within the equivalence zone of the OM (lower bound: 3.63, upper bound: 4.43). (See Table 2) Correlations for the VM2RM (r=0.53) and VA2RM (r=0.58) with the OM were all significant (P<0.0001), and higher than for the FT1RM (r=0.37), TR1RM (r=0.30), PU1RM (r=0.37) and TH1RM (r=0.35). MAPEs for the VM2RM, VA2RM, FT1RM, TR1RM, PU1RM and TH1RM were 28.2%, 34.2%, 31.5%, 40.2%, 61.7% and 36.6%, respectively.

Table 2.

Overall measurement agreement between Actigraph regression models and Oxycon Mobile in energy expenditure measurement.

Method			90% CI of Regression Models		Equivalence zone of OM		Correlations		MAPE (%)
Type	M	SE	Lower bound	Upper bound	Lower bound	Upper bound	r	P-value	M	SD
Criterion measure
OMa	4.03	0.06	-	-	3.63	4.43	-	-	-	-
Regression Models
VM2RMb	2.88	0.05	2.79	2.97	-	-	0.53	<0.0001	28.2	9.3
VA2RMc	2.64	0.04	2.57	2.71	-	-	0.58	<0.0001	34.2	7.2
FT1RMd	2.76	0.06	2.65	2.86	-	-	0.37	0.0040	31.5	11.0
TR1RMe	2.46	0.10	2.29	2.62	-	-	0.30	0.0221	40.2	15.5
PU1RMf	1.54	0.04	1.47	1.60	-	-	0.37	0.0036	61.7	7.2
TH1RMg	2.55	0.06	2.44	2.65	-	-	0.35	0.0068	36.6	11.3

Note: Values are MET_RMR (metabolic equivalents taking into account resting metabolic rate) unless otherwise noted.

^aOxycon Mobile;

^bVector magnitude 2 regression model;

^cVertical axis 2 regression model;

^dFreedson/Trost 1 regression model;

^eTrost 1 regression model;

^fPuyau 1 regression model;

^gTreuth 1 regression model

Regarding the agreement between the 2RMs and the OM, both VM2RM and VA2RM yielded non-equivalent underestimations for all 48 possible activity-specific comparisons (i.e., 2 methods×24 activities) (See Figure 1.(a)). Of the 96 possible comparisons (i.e. 4 methods×24 activities) between the 1RMs and OM, none of them were significantly equivalent with one another, except for the two comparisons between the FT1RM (i.e. 90%CI: 4.22, 4.84) and the OM (i.e. equivalence zone: 3.97, 4.85) for brisk walking, and between the PU1RM (i.e. 90%CI: 4.91, 5.64) and the OM (i.e. equivalence zone: 4.88, 5.96) for dribbling a basketball.

Figure 1.

Means and standard errors of (a) two 2-regression models (VM2RM - Vector magnitude 2 regression model; VA2RM -Vertical axis 2 regression model) and (b) four 1-regression models (FT1RM - Freedson/Trost 1 regression model; TR1RM - Trost 1 regression model; PU1RM - Puyau 1 regression model; TH1RM - Treuth 1 regression model) and Oxycon Mobile for each activity type; Note - MET_RMR (metabolic equivalents taking into account resting metabolic rate).

The MAPE values for the 2RMs ranged from 32.5% to 39.4% for sedentary activities, from 14.5% to 42.9% for light intensity, from 15.0% to 58.1% for moderate intensity, and from 19.6% to 61.5% for vigorous intensity activities (See Supplementary Table 3). The MAPE values for the four 1RMs ranged from 17.3% to 67.2% for sedentary activities, from 14.5% to 70.3% for light intensity, from 17.3% to 75.1% for moderate intensity, and from 16.3% to 78.0% for vigorous intensity activities. The 2RMs, in particular VM2RM, produced relatively smaller (and/or comparable) MAPE values in comparison with (all or most of) the 1RMs for most of the 15 LPA (light physical activity) and MPA (moderate physical activity), the exception being “Hand weight exercises” compared with TH1RM.

The Bland-Altman plots showed that VM2RM (range of 95% limits of agreement = 1.68MET_RMR) (i.e., -1.99 to -0.31MET_RMR) and VA2RM (range: 1.50MET_RMR) showed relatively narrower ranges of 95% limits of agreement in comparison with FT1RM (range: 2.11MET_RMR), TR1RM (range: 3.05MET_RMR), PU1RM (range: 1.77MET_RMR), and TH1RM (range: 2.12MET_RMR). (See Supplementary Figure 1 and Supplementary Table 4) The slopes of the fitted line were not significant for VM2RM (P=0.286), FT1RM (P=0.655) and TH1RM (P=0.716). This implies that patterns of systematic bias were not significant in the VM2RM, FT1RM and TH1RM. However, significant patterns of systematic errors were found for VA2RM (P=0.001), TR1RM (P<0.001) and PU1RM (P=0.002).

Discussion

The present study examined the validity of two recently developed 2RMs and four standard 1RMs for physical activity EE estimation relative to the indirect calorimetry criterion measure in children. Overall, none of the six regression models were significantly equivalent to the criterion measure. All regression models provided considerably large underestimation of EE values measured by indirect calorimetry. Given that the 2RMs produced narrower ranges of 95% limits of agreement, less biased patterns of Bland-Altman plots and smaller MAPEs, the use of the 2RMs may result in less individual variation in measuring EE in comparison with the 1RMs.

We did not detect any particular pattern in the errors that may help to determine the underlying cause of the discrepancies. However, comparison of our results with prior related studies offers some insights into sources of error. To be specific, the findings on the validity of the 2RMs in children contrast with the original cross validation study¹⁴ but are consistent with the more recent independent validation study.⁷ In the original cross validation study of the 2RMs,¹⁴ the VM2RM and VA2RM yielded MET_RMR values comparable with indirect calorimetry (i.e. mean differences<0.8MET_RMR) for most of the performed activities. However, the favorable findings results may be attributable to the fact that the number, types, and duration of the activities performed were equivalent to those used to develop the VM2RM and VA2RM models. It should not be surprising to find good validity when a cross-validation study uses the exactly same general methodology that is used in a calibration study. The previous study¹⁴ also used relatively crude statistical methods (i.e. one-way ANOVA) to test for agreement, and had a relatively small sample size for the comparison group (n=39). In the current study, the 2RM method was evaluated in an independent, larger sample of children (n=59) and employed a more rigorous methodology (i.e. equivalence testing) than previous studies. A subsequent independent validation study by Crouter et al.⁷ evaluated the VM2RM and VA2RM using a more variable set of activities (i.e. 8 structured activities for 8 minutes each+1 free-living measurement session for approximately 2 hours) and a larger number of participants (i.e. 32 girls and 40 boys ages 11-15yrs) in comparison to their original calibration study.¹⁴ Consistent with the present study, that study⁷ demonstrated that the VM2RM and VA2RM underestimated EE measured with indirect calorimetry for the 8 structured activities (range of mean percent errors, 1.0%–23.1%) and free-living activity measurement (range of mean percent error, 26.3% and 27.2%, respectively). The convergent findings from the two independent validation studies (i.e. the current study and the study by Crouter et al.⁷), the 2RM method appears to underestimate EE values in children.

From a public health standpoint, it is important to establish procedures to more accurately estimate EE in youth. Because LPA and MPA are common outcome measures in contemporary physical activity research for youth,²⁷ accurate calibration of EE equations is critical for accurately estimating EE in these two intensities. In this study, the resulting MAPE values for the 2RMs were comparable and/or relatively smaller in comparison with MAPEs for the 1RMs for the majority of the LPA and MPA. This suggests that the 2RMs may provide more accurate estimates of EE and/or activity time for LPA and MPA in comparison with the 1RMs. A similar result was reported by Crouter et al.⁷ who compared the VM2RM, VA2RM, and 5 standard 1RMs (i.e. by Freedson/Trost et al.,³ Treuth et al.,⁵ Trost et al.,⁶ Puyau et al.,⁴ and Evenson et al.²⁸) against indirect calorimetry for estimating minutes spent during 4 categories of physical activity (i.e. sedentary, LPA, MPA, VPA). In the study by Crouter et al.,⁷ the VM2RM and VA2RM produced smaller ranges of 95% limits of agreement in comparison with the 5 1RMs in classifying LPA and MPA. This indicates that in measuring EE on LPA and MPA, the 2 RMs would have relatively smaller individual bias in comparison with the 1RMs. Another study conducted by Trost et al.⁸ evaluated activity classification accuracy of five 1RMs (Freedson/Trost et al.,³ Puyau et al.,⁴ Treuth et al.,⁵ Mattocks et al.,²⁹ Evenson et al.²⁸) using Receiver Operating Characteristic-area under the curve (ROC-AUC). In the Trost et al. study,⁸ all the five 1RMs demonstrated relatively low classification accuracy for LPA (ROC-AUC ranges from 0.43 to 0.70) and MPA (ROC-AUC ranges from 0.56 to 0.79) time in comparison to classification accuracy for sedentary (ROC-AUC ranges from 0.80 to 0.90) and VPA (ROC-AUC ranges from 0.54 to 0.84). This suggests that LPA and MPA are more difficult to evaluate using 1RM approaches. The present results do not directly support the 2RM approach but it is clear that it provides advantages over the traditional 1RM approach.

There are some key strengths of the present study. To our knowledge, this study is the first study to employ the novel method of equivalence testing in validation studies on physical activity measurement tools in children. Given that equivalence testing is intended to determine how closely one method equates to another method, it is a more appropriate analytic approach for this type of validation research. Another strength is that we included a wide variety of activities (i.e. a total of 24) designed to mimic the range of children's free-living activities.

However, the results of the current study still need to be interpreted with some caution. The current study used slightly different methodologies than that employed in the original 2RM development study.¹⁴ Most importantly, the activities of this study were performed in a random fashion whereas in the validation study,¹⁴ the 2RMs were developed and validated with activities being performed from the lowest to the highest intensity category. We view the random order of activities as an important design feature since it more directly captures free-living behavior. However, it is also possible that this fully randomized order could lead to overestimation of the “criterion” EE values for lower intensity activities if performed after more vigorous activities. Moreover, children's RMR was not measured, but estimated using the Schofield's equation.²⁰ However, the Schofield's equation accurately estimated RMR values for pre-school children,³⁰ and has been widely used in previous validation studies^{21, 22, 8, 23} to evaluate different sets of Actigraph EE equations for youth.

Conclusion

None of the two 2RMs and four 1RMs were significantly equivalent to the indirect calorimetry for both overall group comparisons and activity-specific comparisons (only except for 1 activity with the TH1RM). Relatively smaller individual variations were identified for the 2RMs in comparison with the 1RMs. In measuring estimates of EE for LPA and MPA, the 2RMs showed relatively smaller individual errors in comparison with the 1RMs. Future studies using the Actigraph to measure EE should be informed of the smaller potential biases inherent in the use of the 2RMs than the 1RMs.

Supplementary Material

1

Supplementary Figure 1. Bland-Altman plots with 95% limits of agreement for two 2-regression models (VM2RM - Vector magnitude 2 regression model; VA2RM - Vertical axis 2 regression model) and four 1-regression models (FT1RM - Freedson/Trost 1 regression model; TR1RM - Trost 1 regression model; PU1RM - Puyau 1 regression model; TH1RM - Treuth 1 regression model) in relation to Oxycon Mobile (OM); Note - MET_RMR (metabolic equivalents taking into account resting metabolic rate)

Practical Implications.

• Researchers would be informed of the relative validity of different types of prediction equations for Actigraph accelerometers to estimate energy expenditure in youth.

• Epidemiological surveillance and/or intervention studies would utilize prediction equations (i.e. two-regression models) in order to obtain accurate estimates of energy expenditure levels for youth.

• Relatively more accurate estimation of energy expenditure for light and moderate activities could be achieved with the use of the two regression models.