Wrist Acceleration Cut-points for Moderate-to-Vigorous Physical Activity in Youth

Abstract

Purpose

To examine the validity of wrist acceleration cut-points for classifying moderate (MPA), vigorous (VPA) and moderate-to-vigorous (MVPA) physical activity.

Methods

Fifty-seven children (5-12y) completed 15 semi-structured activities. Three sets of wrist cut-points (>192mg, >250mg, >314mg), previously developed using Euclidian norm minus one (ENMO₁₉₂₊), GENEActiv software (GENEA₂₅₀₊) and Bandpass Filtered followed by Euclidian Norm (BFEN₃₁₄₊), were evaluated against indirect calorimetry. Analyses included classification accuracy, equivalence testing and Bland-Altman procedures.

Results

All cut-points classified MPA, VPA and MVPA with substantial accuracy (ENMO₁₉₂₊: κ = 0.72 [95% confidence interval: 0.72 – 0.73], MVPA: area under the receiver operating characteristic curve (ROC-AUC) = 0.85 [0.85 – 0.86]; GENEA₂₅₀₊: κ = 0.75 [0.74 – 0.76], MVPA: ROC-AUC = 0.85 [0.85 – 0.86]; BFEN₃₁₄₊: κ = 0.73 [0.72 – 0.74], MVPA: ROC-AUC = 0.86 [0.86 – 0.87]). BFEN₃₁₄₊ misclassified 19.7% non-MVPA epochs as MPA, whereas ENMO₁₉₂₊ and GENEA₂₅₀₊ misclassified 32.6% and 26.5% of MPA epochs as non-MVPA, respectively. Group estimates of MPA time were equivalent (p<0.01) to indirect calorimetry for the BFEN₃₁₄₊ MPA cut-point (mean bias: -1.5%, limits of agreement [LoA]: -57.5 - 60.6%), while estimates of MVPA time were equivalent (p<0.01) to indirect calorimetry for the ENMO₁₉₂₊ (mean bias: -1.1% [LoA: - 53.7% – 55.9%]) and GENEA₂₅₀₊ (mean bias: 2.2% [LoA: -56.5% – 52.2%]) cut-points. Individual variability (LoAs) was large for MPA (min: BFEN₃₁₄₊, -60.6% – 57.5%; max: GENEA₂₅₀₊, -42.0% – 104.1%), VPA (min: BFEN₃₁₄₊, -238.9% – 54.6%; max: ENMO₁₉₂₊, -244.5% – 127.4%) and MVPA (min: ENMO₁₉₂₊, -53.7% – 55.0%; max: BFEN₃₁₄₊, -83.9% – 25.3%).

Conclusion

Wrist acceleration cut-points misclassified a considerable proportion of non-MVPA and MVPA. Group level estimates of MVPA were acceptable; however, error for individual level prediction was larger.

Introduction

Accurate measurement of physical activity (PA) in children is of critical importance to monitor prevalence and trends, establish associations with health outcomes, identify determinants, and to evaluate the effectiveness of interventions to promote PA (1). Hip-mounted accelerometers have commonly been used to objectively quantify habitual PA in children (2). However, low participant compliance with accelerometry protocols have resulted in considerable non-wear time and, subsequently, loss of data (3). National biobanks such as U.K. Biobank (4), and large population surveys (5) including the National Health and Nutrition Examination Study (NHANES) 2011-2014 (6) in the U.S. incorporated wrist-worn accelerometers. Recent evidence indicates that wrist-placement results in increased wear time due to greater compliance (6–8), which has consequently caused a shift from hip-placement to wrist-placement.

Traditionally, accelerometer-based PA monitoring devices have provided proprietary units referred to as “counts” from which cut-points have been developed to classify moderate (MPA), vigorous (VPA) and moderate-to-vigorous physical activity (MVPA) and estimate time spent in MVPA. However, more recently, commonly used accelerometer-based motion sensors such as the GENEActiv (ActivInsights Ltd., Cambridge, UK) and ActiGraph GT3X+ and GT9X (ActiGraph Corporation, Pensacola Beach, FL) provide access to high frequency tri-axial acceleration data, and therefore cut-points to define PA intensity have been developed for these data collected from wrist devices. The existence of multiple cut-points makes comparisons of PA outcomes from studies that have used different cut-points challenging, and inconsistencies between studies may affect conclusions about PA prevalence, health benefits, determinants and the effectiveness of interventions. Therefore, studies are needed that simultaneously compare the validity of multiple cut-points to provide evidence upon which consensus can be reached for consistent data reduction approaches, which could increase the comparability of PA outcomes between studies.

Recent laboratory-based calibration studies (9–11) have developed three sets of PA intensity thresholds for raw acceleration output from wrist-worn devices in 6-14 year-old children using indirect calorimetry as the criterion measure. The cut-points were cross-validated and demonstrated acceptable classification accuracy. However, two studies (10, 11) applied the leave-one out cross-validation approach in the calibration sample and evaluated classification accuracy for the same MPA and VPA activities, which were predominantly ambulatory (e.g., treadmill walking and running). As such, generalizability to free living scenarios may be limited. One set of cut-points (9) was cross-validated in an independent sample of 5-8 year-olds (12), however, the sample size was small (n=15), the protocol included a limited range of activities, and the cut-points were not cross-validated in children older than 8 years.

These independent calibration studies used different data processing methodologies and have resulted in different cut-points, ranging from 192mg (11) to 314mg (10) and 696mg (11) to 998mg (10) for MPA and VPA, respectively; thus providing different PA estimates, which makes it difficult to compare outcomes between studies. Therefore, additional studies are needed to adequately cross-validate cut-points. A recent study (13) validated various data processing approaches for the wrist-worn ActiGraph in children and concluded that differences in PA estimates were caused by the use of different methods. However, because Kim et al. (13) did not include a valid criterion measure, the most accurate approach could not be determined. In agreement with best practice recommendations from Welk et al. (14), the authors suggested that the validity of different methods, along with their corresponding cut-points, should be evaluated simultaneously, relative to gold standard methods. Therefore, the aim of this study was to simultaneously evaluate the performance of three sets of wrist acceleration cut-points for classifying MPA, VPA and MVPA and estimating time spent in PA intensities, under consistent conditions, using portable indirect calorimetry as the criterion measure in 5-12 year-old children.

Methods

Participants

Fifty-seven children aged 5-12y who were without physical or health conditions that would affect participation in PA were recruited as part of an activity monitor validation study. The study was approved by the University of Wollongong Health and Medical Human Research Ethics Committee. Descriptive characteristics of participants are presented in Table 1. Written parental consent and participant assent were obtained prior to participation.

Table 1. Participant characteristics.

Characteristics
Age (y)	9.2 ± 2.3
Sex
Boys (n)	28 (49.1%)
Girls (n)	29 (50.9%)
Height (cm)	135.9 ± 14.6
Body mass (kg)	32.7 ± 10.9
BMI percentile	53.2 ± 28.6
Overweight (n)	7 (12.3%)
Obese (n)	2 (3.5%)
Age distribution
5-7 (n=19)	33.3%
8-18 (n=24)	42.1%
11-12 (n=14)	24.6%
Race
Caucasian (n)	54 (94.7%)
Asian (n)	3 (5.3%)

Characteristics of the participants are presented as mean ± SD, distributions of the sample are presented in percentages. Weight status was classified according to the 2000 Centers for Disease Control and Prevention Growth Charts for the United States (11).

Procedures

Participants were required to visit the laboratory on two occasions. Anthropometric measures were completed during the first visit using standardised procedures while children were wearing light clothing and with shoes removed. BMI (kg/m²) was calculated to categorize participants as normal weight or overweight/obese, according to the 2000 CDC Growth Charts for the United States (15). Children completed a protocol of 15 semi-structured activities (Table 2) from sedentary (lying down, TV viewing, handheld e-game, writing/coloring, computer game), light-intensity PA (LPA: getting ready for school, standing class activity, slow walk, dancing), and MVPA (tidy up, brisk walk, soccer, basketball, running, locomotor course). Activities were equally divided over 2 visits and completed in a structured order of increasing intensity for 5 min (except for lying down which was done for 10 min).

Table 2. Activity Protocol.

Activity Type	Activity Trial	Intensity	Description of Activity Trial
Resting	Lying down	Sedentary	Lying down awake on a mattress in supine position - arms at sides - rest for 10 min.
Sitting	TV viewing	Sedentary	Watching a movie in a comfortable chair. Instructed to minimize body movements.
	Handheld e-game	Sedentary	Sitting on a chair at a desk playing an e-game on a handheld device.
	Writing/coloring	Sedentary	Sitting on a chair at a desk, 5-8 y: coloring on paper using pencils, 9-12 y: copying words on a pad of paper using a pencil.
	Computer game	Sedentary	Sitting on a chair at a desk playing an educational computer game.
Lifestyle	Getting ready for school	Light	Get dressed, set table, pour food, pack up, brush teeth, pack bag, leave for school.
	Standing class activity	Light	Standing activities with minimal movement such as writing/drawing on a white board.
	Dancing	Light	Following a video with dance step instructions (Zumba® fitness).
	Tidy up	Moderate	Tidying up a 4x5 m area: pick up clothes, towels, toys and sport equipment and return them into boxes.
	Basketball	Moderate	Shooting a basketball using a 2.29 m adjustable hoop, chase the ball within a 4.9x4.6 m area and bounce back to the start position at the boundary line apposite from the hoop.
	Soccer	Vigorous	Kicking a foam soccer ball on a 5 m distance between a 1 m wide goal after dodging between a straight line of 5 cones (1 m apart). Instructed to jog back to start position after kicking the ball.
	Locomotor course	Vigorous	Continuously completing a course including 4x 2-foot jump, jogging and sliding between cones around a 4x9.5 m area.
Ambulatory	Slow walk	Light	Walking slowly at a self-selected comfortable speed around a 45 m indoor track. Examiner regulates constant speed by recording lap times.
	Brisk walk	Moderate	Walking briskly at a self-selected brisk comfortable speed around a 45 m indoor track. Examiner regulates constant speed by recording lap times.
	Running	Vigorous	Run at a self-selected comfortable speed around a 45 m indoor track. Examiner regulates constant pace by speed lap times.

All activities are completed for 5 min, except from lying down (10 min)

Instrumentation

At each visit, children were fitted with a portable respiratory gas analysis system (MetaMax^® 3B, Cortex, Biophysics, Leipzig, Germany) to provide the criterion assessment of PA energy expenditure. Children were also fitted with a GENEActiv dorsally on the non-dominant wrist.

Indirect calorimetry

Oxygen consumption (O₂) was assessed using the MetaMax^® 3B portable breath-by-breath respiratory gas analysis system to provide the criterion assessment of energy expenditure. The participants wore a facemask (Hans Rudolph, Kansas City, MO) covering their nose and mouth, which was held in place by a head harness. Prior to every measurement, the analyser was calibrated according to the manufacturer’s guidelines. Breath-by-breath data from indirect calorimetry were downloaded and exported using MetaSoft (version 4.3.2).

Activity monitor

The GENEActiv has a waterproof design and measures tri-axial accelerations ranging in magnitude ±8g at a sample frequency ranging from 10-100Hz. Acceleration values are digitized by a 12-bit analog-to-digital converter. Accelerometers were initialised with a sample frequency of 100Hz.

Data reduction

Energy expenditure

Volume of O₂ uptake and CO₂ production were averaged per 10s for every entire activity bout of 5min and converted into units of energy expenditure (kcal·min^-1) using the Weir equation (16). For analytical purposes, and for consistency with the calibration studies of the cut-points (9–11), the activities were categorised in the primary analyses as non-MVPA (<3 METs), MPA (≥3 to <6 METs) or VPA (≥6 METs) based on average measured energy expenditure values. MPA and VPA were subsequently combined and classified as MVPA (≥3 METs). The participants’ measured resting energy expenditure (REE) from the lying down trial was used to define 1 MET in order to calculate MET-values for all activities. Breath-by-breath samples from the data collected between minutes 7.0 and 9.0 during the lying down trial were averaged to calculate mean REE. Metabolic data (10s epochs) from the activities were scaled to the children’s REE and converted into youth METs using customized software. Although 3 METs has widely been used as an intensity threshold to distinguish MPA from LPA, there is considerable evidence that 4 METs is more accurate for classifying MPA in children and adolescents (17) and that brisk walking, a key behavioral indicator of MPA, is associated with an energy cost of approximately 4 METs (18). It should be noted that researchers have based these estimates on either predicted REE or measured REE. As such, studies have demonstrated that MET levels for walking and other activities are somewhat contingent on the choice of the denominator (19, 20). In our sample, the larger value results in ~3 METs for brisk walking as the behavioural indicator, when based on measured REE (slow walking = 2.9 ± 0.5 METs; brisk walking = 3.4 ± 0.6 METs) (see Table, Supplemental Digital Content 1, metabolic data by activities for indirect calorimetry, http://links.lww.com/MSS/B67). However, when based on predicted REE, the value was closer to 4 METs (slow walking = 4.0 ± 0.6 METs; brisk walking = 4.7 ± 0.7 METs), which was consistent with a previous study (comfortable walking = 3.9 ± 0.6 METs; brisk walking = 4.7 ± 0.6 METs) (21). Therefore, supplementary analyses were conducted testing the consistency of the findings using a threshold of 4 METs, for which METs were calculated by dividing mean energy expenditure values by REE predicted from the participant’s sex, age, body mass, and height using Schofield’s (22) equation for children aged 3–10 or 10–18 yr.

Accelerometry

Data reduction approaches were performed according to the methods reported in calibration studies by Hildebrand et al. (11), Phillips et al. (9) and Schaefer et al. (10) for the development of the three cut-points evaluated. Raw wrist data were downloaded using the GENEActiv software version 2.2. Signal processing codes from Hildebrand et al. (11) were downloaded and applied to convert raw acceleration data into 1s epochs according to the Euclidian norm minus one (ENMO) approach. This method subtracted 1g from the Euclidian norm (EN = sqrt (x² + y² +z²)), after which negative values were rounded up to zero. According to the methods described by Phillips et al. (9), raw acceleration data was converted into 1s epochs using the GENEActiv post processing software, in order to create gravity-subtracted signal vector magnitude (SVMgs) data. Customized software was developed using the statistical computing language R (v.3.1.2) in order to apply a band-pass filter to the raw acceleration data (4th order Butterworth filter with ω₀ = 0.2-15Hz) to remove the gravitational acceleration component as well as high-frequency sensor noise, as described by Schaefer et al. (10). EN was taken from the three resulting signals and averaged per 1s epoch. This method is referred to as Bandpass Filtered followed by Euclidian Norm (BFEN). The methods of the calibration studies resulted in sets of cut-points as described below in order of increasing acceleration magnitude, and hereafter referred to as:

• Hildebrand et al. (11), ENMO₁₉₂₊: non-dominant wrist; MPA, 192-695 mg; VPA, ≥696 mg.

• Phillips et al. (9), GENEA₂₅₀₊: right wrist; MPA, >275 to ≤700 mg; VPA, >700 mg, left wrist; MPA, >250 to ≤750 mg; VPA, >750 mg. Calibration procedures for these cut-points were based on the cumulative sum of gravity-based accelerations measured with a sample frequency of 80Hz, making the original cut-points frequency dependent (11). For presentation purposes, the cut-point values were converted from a time dependent unit (g.seconds) to the time independent unit mg in order to compare with values of other cut-points.

• Schaefer et al. (10), BFEN₃₁₄₊: non-dominant wrist; MPA, 314-998 mg; VPA, ≥998 mg.

The 1s epochs for accelerometry data of all methods were averaged over 10s windows in order to align with indirect calorimetry data.

Data synchronization

At the beginning of each laboratory visit, the activity monitors and indirect calorimetry were synchronized with an internal computer clock. After applying the cut-points, predicted intensity classification for the wrist acceleration data was aligned with the ground truth energy expenditure data in order to examine classification accuracy. All valid epochs from each activity trial were included in analyses to reflect how activity monitors are applied under free-living conditions. Estimated time spent in each PA intensity using indirect calorimetry or wrist accelerometry was established by summing the 10s epochs classified for each intensity.

Statistical analyses

Normality of the data was confirmed prior to analyses. Classification accuracy for each set of cut-points (MPA, VPA, non-MVPA) was examined by calculating weighted κ statistics. Kappa coefficients were interpreted using the ratings suggested by Landis and Koch (23): poor (0 – 0.20), fair (0.21 – 0.40), moderate (0.41 – 0.60), substantial (0.61 – 0.80), and almost perfect (0.81 – 1.0). Contingency tables were applied to summarize classification accuracy and percentage of misclassified epochs for each intensity. Because of the public health focus on MVPA, the intensities of MPA and VPA were combined as one dichotomous variable MVPA and the classification accuracy was evaluated using sensitivity, specificity and area under the receiver operating characteristic curve (ROC-AUC). ROC-AUC values were defined as excellent (≥0.90), good (0.80-0.89), fair (0.70-0.79), or poor (<0.70) (24). The equivalence of time estimates between the cut-points and indirect calorimetry for each intensity was examined at the group level using the 95% paired equivalence test. In order to reject the null-hypothesis of the equivalence test, the 90% confidence interval (CI) of time spent in the intensity predicted by the monitors should fall entirely within the predefined equivalence region of ±10% (25). Measurement agreement and systematic bias for estimated time spent in intensities were evaluated at the individual level using Bland-Altman procedures (26). Analyses were performed using the statistical computing language R v.3.1.2 (The R Foundation for Statistical Computing) and SPSS v.21.0 (IBM Corporation, Armonk NY).

Results

All participants completed the protocol. For one of the visits, wrist acceleration data were unavailable for 3 children. Data from one child were entirely excluded from the analyses and data from 3 participants for a total of 8 activities were excluded because of indirect calorimetry failure. A total of 25,452 PA intensity annotated 10s epochs (94.4% of the total data) from 57 children were available for analyses.

Applying the contingency tables for classification accuracy (Table 3), ENMO₁₉₂₊ (κ = 0.72 [95% confidence interval (CI): 0.72 to 0.73]), GENEA₂₅₀₊ (κ = 0.75 [95% CI: 0.74 to 0.76]) and BFEN₃₁₄₊ (κ = 0.73 [95% CI: 0.72 to 0.74]) exhibited substantial agreement. The proportion of correctly classified epochs for the BFEN₃₁₄₊ MPA and VPA cut-points (52.0% and 93.6%, respectively) was higher than for the ENMO₁₉₂₊ cut-points (46.5% and 70.0%, respectively) and the GENEA₂₅₀₊ cut-points (45.4% and 79.9%, respectively). However, ENMO₁₉₂₊ and GENEA₂₅₀₊ classified non-MVPA (90.5% and 89.2%, respectively) more accurately than BFEN (81.7%). BFEN misclassified 19.7% of non-MVPA as MPA and 39.4% of MPA as VPA. The highest proportions of misclassification for ENMO₁₉₂₊ and GENEA₂₅₀₊ on the other hand were found for MPA misclassified as non-MVPA (ENMO₁₉₂₊: 32.6% epochs; GENEA₂₅₀₊: 26.5% epochs) and VPA misclassified as MPA (ENMO₁₉₂₊: 20.8% epochs; GENEA₂₅₀₊: 28.1% epochs). ENMO₁₉₂₊ and GENEA₂₅₀₊ misclassified 25.0% and 19.4% of VPA as MPA. Classification accuracy for MVPA was good for all cut-points (ROC-AUC: ENMO₁₉₂₊, 0.85 [95% CI: 0.85 to 0.86]; GENEA₂₅₀₊, 0.85 [95% CI: 0.85 to 0.86]; BFEN₃₁₄₊, 0.86 [95% CI: 0.86 to 0.87]). Although the true-positive rate (sensitivity) for BFEN₃₁₄₊ (0.94) was higher than for ENMO₁₉₂₊ (0.80) and GENEA₂₅₀₊ (0.81), specificity for BFEN₃₁₄₊ was lower (0.78) compared to ENMO₁₉₂₊ (0.90) and GENEA₂₅₀₊ (0.89).

Table 3. Contingency tables for classification accuracy of raw wrist acceleration cut-points.

Actual Intensity	Cut-points classification of intensity
	1	2	3
	ENMO192+
1. non-MVPA	14418 (90.5)	1312 (8.2)	193 (1.2)
2. MPA	2217 (32.6)	3160 (46.5)	1416 (20.8)
3. VPA	138 (5.0)	684 (25.0)	1914 (70.0)
	GENEA250+
1. non-MVPA	14208 (89.2)	1493 (9.4)	222 (1.4)
2. MPA	1802 (26.5)	3081 (45.4)	1910 (28.1)
3. VPA	20 (0.7)	531 (19.4)	2185 (79.9)
	BFEN314+
1. non-MVPA	12448 (78.2)	3130 (19.7)	345 (2.2)
2. MPA	580 (8.5)	3535 (52.0)	2678 (39.4)
3. VPA	8 (0.3)	167 (6.1)	2561 (93.6)

The presented values indicate the proportion of epochs classified for each intensity, with percentages presented between brackets. The values in boldface indicate the proportion of epochs correctly classified for the physical activity intensity. MPA: moderate physical activity; VPA: vigorous physical activity; MVPA: moderate-to-vigorous physical activity; ENMO: cut-points developed using Euclidian norm minus one; GENEA: cut-points developed using the GENEActiv post processing software; BFEN: cut-points developed using Bandpass Filtered followed by Euclidian Norm.

At the group level, estimated time spent in MPA was equivalent (p<0.01) to indirect calorimetry for BFEN₃₁₄₊ and estimated time spent in MVPA was equivalent for ENMO₁₉₂₊ and GENEA₂₅₀₊ (Figure 1). Outcomes of the Bland-Altman analyses are presented in Table 4. BFEN₃₁₄₊ overestimated time spent in MPA by a small margin of 1.5% (limits of agreement [LoA]: -57.5% – 60.6%), whereas ENMO₁₉₂₊ and GENEA₂₅₀₊ overestimated time spent in MPA by 30.1% (LoA: -99.6% – 39.4%) and 31.0% (LoA: -104.1% – 42.0%), respectively. Overestimation of time spent in VPA was larger for BFEN₃₁₄₊ (92.2% [LoA: -54.6% – 238.9%]) compared to ENMO₁₉₂₊ (58.5% [LoA: -127.4% – 244.5%]) and GENEA₂₅₀₊ (75.2% [LoA: -91.8% – 242.2%]). Mean bias for time spent in MVPA was small for ENMO₁₉₂₊ (-1.1% [LoA: -55.9% – 53.7%]) and GENEA₂₅₀₊ (2.2% [LoA: -52.2% – 56.5%]), whereas time spent MVPA was overestimated by BFEN₃₁₄₊ to a larger extent (29.3% [LoA: -25.3% – 83.9%]). At the individual level, LoAs were wide for all cut-points and for all intensities, especially for VPA estimates from all cut-points and for MPA estimates from the ENMO₁₉₂₊ and GENEA₂₅₀₊. Systematic bias (p<0.05) was found for time spent in all intensities estimated by all cut-points, with the exceptions of time spent in MPA estimated by BFEN₃₁₄₊ and GENEA₂₅₀₊, indicating that errors increased with increasing time spent in the intensities.

Figure 1. 95% equivalence test for raw wrist acceleration-based estimated time spent in physical activity intensities.

Times estimated by wrist-worn cut-points are equivalent to indirect calorimetry if 90% confidence intervals lie entirely within the equivalence region of indirect calorimetry. MPA: moderate physical activity; VPA: vigorous physical activity; MVPA: moderate-to-vigorous physical activity; ENMO: cut-points developed using Euclidian norm minus one; GENEA: cut-points developed using the GENEActiv post processing software; BFEN: cut-points developed using Bandpass Filtered followed by Euclidian Norm.

Table 4. Agreement analysis of raw wrist acceleration-based estimations of physical activity intensities compared to indirect calorimetry.

Intensity	Mean bias (%)	Limits of agreement	p-value slope
ENMO192+
MPA	30.1	-39.4 – 99.6	0.04
VPA	-58.5	-244.5 – 127.4	0.00
MVPA	1.1*	-53.7 – 55.9	0.00
GENEA250+
MPA	31.0	-42.0 – 104.1	0.05
VPA	-75.2	-242.2 – 91.8	0.00
MVPA	-2.2*	-56.5 – 52.2	0.00
BFEN314+
MPA	-1.5*	-60.6 – 57.5	0.28
VPA	-92.2	-238.9 – 54.6	0.00
MVPA	-29.3	-83.9 – 25.3	0.00

MPA: moderate physical activity; VPA: vigorous physical activity; MVPA: moderate-to-vigorous physical activity; ENMO: cut-points developed using Euclidian norm minus one; GENEA: cut-points developed using the GENEActiv post processing software; BFEN: cut-points developed using Bandpass Filtered followed by Euclidian Norm. Mean bias was calculated as: measured intensity time – estimated intensity time; a positive value indicates underestimation; a negative value indicates overestimation.

^*Significantly equivalent to indirect calorimetry (p < 0.05).

Supplementary analyses (see Tables and Figure, Supplemental Digital Content 2, Supplementary analyses for the raw wrist acceleration cut-points using a ≥4-MET MVPA definition, http://links.lww.com/MSS/B68) indicated that classification accuracy for MPA, VPA and non-MVPA remained similar when 1 MET was defined using predicted REE and a 4-MET threshold for MPA was applied to the data (ENMO₁₉₂₊, κ = 0.65 [95% CI: 0.64 to 0.66], GENEA_250+, κ = 0.71 [95% CI: 0.70 to 0.72], BFEN_314+, κ = 0.75 [95% CI: 0.74 to 0.76]). Although ROC-AUC values for MVPA (ENMO₁₉₂₊, 0.85 [95% CI: 0.85 to 0.86]; GENEA₂₅₀₊, 0.86 [95% CI: 0.85 to 0.86]; BFEN₃₁₄₊, 0.87 [95% CI: 0.87 to 0.88]) were similar to the primary analyses, slightly more non-MVPA epochs were correctly classified (see Table, Supplemental Digital Content 2, 2.1: Contingency tables for classification accuracy of raw wrist acceleration cut-points using a ≥4-MET MVPA definition, http://links.lww.com/MSS/B68). Although time spent in MVPA estimated by ENMO₁₉₂₊ and GENEA₂₅₀₊ using the ≥4-MET MVPA definition was not equivalent to indirect calorimetry as they were in the primary analyses, the means and/or 90% CIs for estimated time spent in MPA and MVPA for ENMO₁₉₂₊ and GENEA₂₅₀₊ overlapped the equivalence region and thus approached equivalence. BFEN₃₁₄₊ overestimated time spent in MVPA for both the 3-MET (1 MET = measured REE) approach (29.3% [LoA: -25.3% – 83.9%]) and the 4-MET (1 MET = predicted REE) approach (18.3% [LoA: -13.5% - 50.2%]) (see Table, Supplemental Digital Content 2, 2.2: Agreement analysis of raw wrist acceleration-based estimations of physical activity intensities compared to indirect calorimetry using a ≥4-MET MVPA definition, http://links.lww.com/MSS/B68). Time spent in MPA estimated by BFEN₃₁₄₊ was no longer equivalent to the criterion measure, whereas time spent in VPA was (p<0.01) (see Figure, Supplemental Digital Content 2, 2.3: 95% equivalence test for raw wrist acceleration-based estimated time spent in physical activity intensities using a ≥4-MET MVPA definition, http://links.lww.com/MSS/B68). In contrast, when defining MVPA as ≥4-METs, fewer MPA epochs were misclassified by BFEN₃₁₄₊ as VPA compared to the 3-MET approach, however more VPA epochs were misclassified as MPA. The overestimation of time spent in VPA from BFEN₃₁₄₊ was small for the 4-MET approach (0.5% [LoA: -39.7% – 40.6%]), whereas overestimation of time spent in MPA for BFEN₃₁₄₊ was larger (34.4% [LoA: -20.4% – 89.1%]). At the individual level, errors for all cut-points were decreased for time spent in VPA when using the 4-MET approach, but increased for time spent in MPA, compared to outcomes from the 3-MET approach.

Discussion

Current international PA guidelines specify that children should accumulate a minimum of 60 minutes per day of MVPA (27). Therefore, the accurate measurement of MVPA is central to understanding the prevalence and patterns of PA, the dose of PA required to achieve health benefits, the determinants of PA, and the effect of PA interventions for children, which typically target MVPA. This study simultaneously cross-validated three previously published wrist acceleration cut-points for the classification of MVPA in children. ENMO₁₉₂₊, GENEA₂₅₀₊ and BFEN₃₁₄₊ demonstrated good classification accuracy for MVPA. However, while time spent in MVPA estimated by ENMO₁₉₂₊ and GENEA₂₅₀₊ were equivalent to indirect calorimetry, misclassification of non-MVPA as MVPA resulted in an overestimation of time spent in MVPA for BFEN₃₁₄₊. Although ENMO₁₉₂₊ and GENEA₂₅₀₊ classified non-MVPA more accurately than BFEN₃₁₄₊, these cut-points still misclassified a significant proportion of MVPA epochs as non-MVPA (37.6% and 27.2%, respectively). Findings were relatively consistent in supplementary analyses, where predicted REE was used to define 1 MET and MVPA was defined as ≥4METs. The classification accuracy of MPA, VPA and MVPA remained relatively similar for all cut-points compared to previous analyses and, although time spent in MVPA estimated by ENMO₁₉₂₊ and GENEA₂₅₀₊ were no longer equivalent to indirect calorimetry, estimates approached equivalence.

Findings from the current study were similar to findings in previous independent cross-validation studies, which demonstrated good classification accuracy for MVPA estimates from raw acceleration wrist cut-points (10, 12), and that classification for VPA is generally higher than for MPA (10–12). Even though classification of MPA, VPA and MVPA was most accurate for BFEN₃₁₄₊, ENMO₁₉₂₊ and GENEA₂₅₀₊ estimated time spent in MVPA more accurately than BFEN₃₁₄₊. Time spent in MVPA was overestimated by BFEN₃₁₄₊ because a relatively large proportion (19.7%) of non-MVPA was misclassified as MPA, which was in agreement with Schaefer et al.’s (10) application in free-living individuals. This misclassification could be explained by activities of light intensity that involve vigorous wrist movements. For example, BFEN₃₁₄₊ misclassified 66.4% of non-MVPA as MPA during the non-MVPA activity “Getting ready for school” (see Table, Supplemental Digital Content 3, Confusion matrices for the raw wrist acceleration cut-points using a ≥3-MET MVPA definition, http://links.lww.com/MSS/B69), an activity of low intensity that involved relatively high wrist motion (e.g., while getting dressed, packing a schoolbag, brushing hair etc.) The opposite effect may occur when MVPA activities involve limited wrist movement. As such, the ENMO₁₉₂₊ and GENEA₂₅₀₊ misclassified 82.3% and 77.1%, respectively, of MPA as non-MVPA during “Tidy up”, an activity of MPA intensity that may have involved limited upper body and wrist motions due to carrying objects while walking. Because of the public health focus on MVPA, misclassification by wrist cut-points of MPA as VPA and vice-versa may not represent a major measurement limitation. However, increased interest among researchers in the influence of sedentary behaviors, defined as any waking behaviors in a sitting or reclining position that require an energy expenditure of ≤1.5 METs (28), and light physical activity (1.5 to <3.0 METs), on health makes it critical to discriminate between these behaviors and MVPA. Previous studies indicate that accurate assessment of sedentary behaviors and the number of breaks in sedentary time based on a lack of wrist movement is challenging (11, 29, 30). The findings from this study confirm that the use of the magnitude of acceleration only might not be effective in distinguishing MVPA from non-MVPA. This finding is relatively consistent with previous studies using cut-points based on proprietary activity “counts” (31–33). This is likely because the association between counts or raw acceleration and energy expenditure, whether on the hip or wrist, differs for different types of physical activities, resulting in cut-points performing well for some activities and demonstrating considerable misclassification during other activities. It should be noted that the benefit of using raw acceleration-based cut-points over using count-based cut-points remains unclear, as in general cut-points result in misclassification, which was also demonstrated by the results in this study for all cut-points. Therefore, progress on alternative approaches, such as those utilizing machine learning (29, 33, 34), may be required. However, similar to the inconsistencies that occur because of the existence of multiple cut-points, the existence of different machine learning approaches and models, such as artificial neural networks (35), decision trees (36) and hidden Markov models (37), presents further challenges and evidence to reach consensus on the most accurate approach for categorizing physical activity intensities in children is required.

An additional limitation of the wrist cut-points validated in the current study is that calibration studies used different processing methodologies. While Schaefer et al. (10) used a filtering approach to remove static accelerations from the tri-axial data, Hildebrand et al. (11) and Phillips et al. (9) subtracted the value of gravity from the vector magnitude, in order to focus the outcome variable on dynamic rather than static accelerations. Hildebrand et al. (11) used the ENMO method, which rounds negative values, resulting from subtracting the vector magnitude by 1g, up to zero. Phillips et al. (9) on the other hand, replaced the negative values with their absolute values and summed the resulting values, which creates a dependency on sample frequency, and thus the cut-points should be converted when using different sample frequencies in order to compare results across studies. The ENMO₁₉₂₊ and BFEN₃₁₄₊ were developed using averaged acceleration magnitudes and can be used for different sample frequencies and epoch lengths. The different processing methods also resulted in different units for the outcomes; Hildebrand et al. (11) and Schaefer et al. (10) used gravity units in g and mg, respectively, whereas Phillips et al. (9) used gravity-based acceleration seconds. Taking all of this into account makes it complicated to compare results from the different cut-points and, as the field progresses, it is important that procedures are standardized based on evidence. Furthermore, some data indicate that raw acceleration output from the GENEActiv and ActiGraph may differ in children during common activities (11). This is likely because manufacturer specific transformations (e.g. filtering) are applied to the raw acceleration data, resulting in different outputs from different devices that may not be a representation of the actual raw acceleration signals (38). As such, our findings may only apply to the GENEActiv monitor and further evaluation across different monitor brands is required.

A strength of this study was that three recently developed sets of raw wrist acceleration cut-points were evaluated simultaneously, against a criterion measure. The study included a broad age range and an equal distribution of age and sex across the sample. Additionally, a range of tasks, beyond treadmill-based ambulatory activities, that are likely to resemble children’s free-living behaviors were included in the protocol. Although these activities reflect daily activities that children typically engage in, the findings of the present study should be confirmed under free-living conditions. A potential limitation of this study is that validation focused on MVPA and did not include light PA or sedentary behavior. Our previous cross-validation study (29) of sedentary cut-points demonstrated that, while hip-based cut-points typically misclassify light activities (e.g. standing still) as sedentary postures, wrist cut-points exhibit some misclassification of non-sedentary behaviors as sedentary and vice-versa. Therefore, it is essential to apply the most accurate intensity specific cut-points for accurate estimates of sedentary behaviors and light intensity PA. However, in order to investigate the accuracy of cut-points for distinguishing sedentary behaviors from light intensity PA, postures such as sitting and standing should be evaluated. This is typically performed using alternative criterion measures, such as direct observation, as described in our previous work (39). Another potential limitation is that acceleration signals were not calibrated to local gravity before analysis in order to minimize sensor calibration errors, as described by van Hees et al. (40). Furthermore, body accelerations and metabolic rate during the exercise bouts may not have been aligned due to lags in oxygen consumption, and true classification accuracy may have been underestimated. However, this data reduction approach reflects how cut-points are used in free-living population studies and, because the approach was applied consistently across cut-points, one cut-point was not biased over the other.

In conclusion, although raw acceleration wrist cut-points exhibited good accuracy for classifying MVPA in children, all cut-points misclassified a significant proportion of MVPA epochs as non-MVPA. While the cut-points demonstrated acceptable estimates of time spent in MPA, VPA, and MVPA at the group level, their application was less accurate for individual measures. When combined with the practical advantages of wrist worn placement, surveillance application of the raw wrist acceleration cut-points would be acceptable for group level estimates of MVPA, although alternative data processing approaches such as machine learning methods may be needed to achieve a generally higher accuracy for the assessment of PA intensities among individual children.