Validation of the Actical Accelerometer in Multiethnic Preschoolers: The Children's Healthy Living (CHL) Program

Abstract

This study aimed to determine the validity and reliability of the Actical accelerometer for measuring physical activity (PA) in preschool children of mixed ethnicity, compared with direct observation via a modified System for Observing Fitness Instruction Time (SOFIT) protocol and proxy parental reports (PA Logs). Fifty children in Hawai‘i wore wrist-mounted accelerometers for two 7-day periods with a washout period between each week. Thirty children were concurrently observed using SOFIT. Parents completed PA Logs for three days. Reliability and validity were measured by intra-class correlation coefficient and proportions of agreement concurrently. There was slight agreement (proportion of agreement: 82%; weighted Kappa=.17, P <.001) between the accelerometer and SOFIT as well as between the accelerometer and the PA Logs (proportions of agreement: 40%; weighted Kappa=0.15, P <.001). PA logs underestimated the PA levels of the children, while the Actical was found to be valid and reliable for estimating PA levels of multiethnic, mixed ethnicity preschoolers. These findings suggest that accelerometers can be objective, valid, and accurate physical activity assessment tools compared to conventional PA logs and subjective reports of activity for preschool children of mixed ethnicity.

Introduction

The benefits of regular physical activity (PA) have been well documented in the public health literature. However there remains a paucity of literature on the PA behaviors of Pacific Islanders, notably multiethnic Pacific Islander children. In order to determine whether PA recommendations are being met, researchers have conducted studies on measuring and quantifying PA in children,^1–3 although few studies have focused on preschool children.^4–7 Accelerometers are popular, reliable, and objective devices for measuring PA. Their small size, reusability, and ability to measure accelerations produced by bodily movement may make them ideal for assessing the PA levels of children.^8–10 The use of accelerometry in assessing the PA of free living children has been well documented.^5,11 The Actical accelerometer (Respironics, Koninklijke Philips Electronics N.V.) in particular, utilizes an omnidirectional sensor, which may aid in capturing activities not typically assessed via accelerometry.^6,12 This accelerometer, one of the smallest available, is also water-resistant. The ideal location for the wearing of accelerometers has not been agreed upon; however, Heil and colleagues have shown that wrist or ankle-mounted monitors can accurately assess free-living PA.¹²

To ensure that techniques, instruments, and data on PA are reliable, they are usually validated or calibrated against other similar (usually criterion standard) sources of information.¹⁹ Direct observation, doubly labeled water, and indirect calorimetry are considered criteria for measuring PA and energy expenditure in young children, and have several advantages over other techniques.¹¹ Proxy measures may also serve as sources of validation; however, some have cautioned against their use as they may introduce a source of bias dependent on the “characteristics and perceptions of the proxy respondent.”¹¹

Although studies have utilized accelerometers in Pacific Island children,^13,14 none to date have been validated in the preschool age group. As such, this research sought to evaluate and provide the validity and reliability of a wrist-mounted Actical accelerometer for assessing the PA behaviors of 2–4 year old multiethnic, predominately Pacific Islander children, compared against direct observation and proxy parental reports. To achieve this aim, the following hypotheses were explored:

The results derived from the accelerometer are comparable for assessing PA during a preschool day to results obtained from direct observation.

The results derived from the accelerometer are comparable for assessing children's PA during all day activity to results of activity logs completed by parents.

The accelerometer is a reliable tool for measuring PA in multiethnic preschool children.

Methods

Participants and Procedures

Preschoolers were recruited from three Head Start sites in Hawai‘i. Eligibility criteria for sites included preschools with a high proportion of Native Hawaiian and other Pacific Islander children, as forthcoming studies would also take place in communities with higher proportions of indigenous people. All children at the selected sites were eligible for the study; no children were excluded. Study purposes and procedures were approved by the Human Studies Program at the University of Hawai‘i at Manoa. Written informed consent was obtained from parents. Preschoolers provided verbal assent before being anthropometrically measured and fitted with an accelerometer. Compensation was provided to parents in two equal payments at the end of each study phase.

Study Design

This cross sectional study took place in phases. The parents were instructed that the child was to wear the accelerometer daily (without removal) until the study staff removed it seven days later.

Phase I. During the first week, accelerometers were placed on the child's non-dominant wrist. Parents were asked to record, on the provided PA Logs, all their child's activities for three days (two week days and one weekend day). Five children (from each of the three sites) were observed by trained study staff during the preschool day utilizing a modified SOFIT protocol (detailed below). Accelerometers were removed from the children's wrist on day 8 of the study, with the data downloaded to a computer for analysis. A one-to-two week washout period then occurred.

Phase II. During the second phase, the same sample of preschoolers were again fitted with accelerometers to wear for another week. Five more children were randomly chosen and observed during the preschool day by staff and assessed via the modified SOFIT. Parents were not asked to complete PA Logs during the second phase of the study. Accelerometers were removed from the children's wrists on day 8, with the generated data downloaded to a computer for analysis.

Measures

For each child, date of birth and sex information was obtained from parents. Details on the race and ethnicity of the child's biological parents were also obtained. Ethnic ancestry had to be provided in (approximate) percentages to sum to 100%. Each child's ancestry was then determined from the sum of the mother's and father's reported ancestry. This method, the BLEND methodology (to characterize admixed ancestries), has been detailed elsewhere.¹⁵

Each child's weight and height was measured (in light clothing, without shoes) by trained measurers on the first day of the study using a calibrated digital scale, (Precision UC-300, A&D Company Limited, Tokyo, Japan), and a portable height board (Shorr, Olney, MD). Measures were obtained three times and the median of the three values used in analysis to calculate body mass index (BMI; kg/m²) for age percentiles.

Assessment of Physical Activity

The Actical Accelerometer. The Actical accelerometer is a small, lightweight, water-resistant, omni-directional device, able to measure movement in multiple planes. The devices were initialized to save data in 15-second intervals to identify the spontaneous movements of children. The use of this accelerometer has been well documented in the literature.^6,16 Parents were assured the device could be worn while sleeping, bathing or swimming, and were provided bands if the accelerometer came off and needed to be replaced at home.

Activity logs. Parents were provided three PA logs. One PA log corresponded to an 18-hour day (5:00 a.m. to 11:00 p.m.) of observation. The logs asked the following be recorded: the time of the activity (in thirty-minute increments), where the activity took place, the activity, and the intensity level of the activity. Intensity levels provided on the logs, were categorized as low, medium, or high, with examples of corresponding activities provided, ie, low = walking slowly, medium = hiking, high = running. Parents could report how long the child did an activity which fell within the 30 minute reporting increment. Once the logs were returned, Metabolic Equivalent (MET) values and activity categories were assigned based on the Pacific Tracker (PacTrac) tool.¹⁷ PacTrac utilizes data from the compendium of energy expenditures developed by Ainsworth and colleagues and contains both adult and youth MET values.^18,19 If parents did not provide, or provided inadequate activity information for a time period, the default category “Child default for additional minutes,” was assigned and was excluded from further analyses.

SOFIT. SOFIT is a three phase, momentary time sampling, interval recording, validated direct observation method designed to measure student MVPA levels, lesson context, and teacher behaviors.^20,21 SOFIT scores activities using a 5-point scale (1 = lying down, 2 = sitting, 3 = standing, 4 = walking, 5 = very active) at a 20-second interval.²⁰ Only the first phase of SOFIT (student PA levels) was used for this study, as the other phases were not relevant to the purposes of the research. Each selected child was observed for one morning at their respective Head Start class (classes are typically half-day), during the same period as the 7-day accelerometer-recording phase. This is a modification of the SOFIT protocol, to allow us to compare a preschool day of SOFIT with a corresponding preschool day of accelerometry data.

Data Analysis

Statistical analyses were conducted using SAS version 9.3 (SAS Institute Inc., Cary, NC).

Accelerometers. Counts per minute (cpm) were calculated by adding data corresponding to four 15-second intervals. A total of 1440 minutes, or 1 full (24-hour) day, was collected per child. Levels of activity were categorized as Sedentary, if cpm ≤ 40 (eg, sitting watching TV), Light, if cpm were ≥ 41 or ≤ 2295 (eg, slow walking), Moderate, if cpm were ≥ 2296 or ≤ 6815 (eg, brisk walking) or Vigorous, if cpm ≥ 6816 (eg, running).⁷ It is important to note that accelerometer counts have no inherent meaning until they are converted, based on level of intensity to relevant constructs.²² If the total number of minutes for one single activity level during a day was greater than 1300 minutes (90% of a 24 hour day), then the data for this day were excluded from the data analysis to minimize extreme outliers.

Testing the reliability of the accelerometers. Intra-class correlation coefficient (ICC) has been widely implemented to determine the test-retest reliability of accelerometer-measured PA.^23–28 ICC is the ratio of between participants' variability and total variability. Hence the higher the ICC, the more reliable the instrument. To quantify the test-retest reliability using data from the two seven-day periods, a mixed effect model was run where subject, week, week day are treated as random effects and week and weekday are nested within participants. No distinction of sites (no site effect) is expected for each activity level (sedentary, light, moderate, and vigorous). For each of the mixed effect models, the ICC was defined as such:

ICC = between-subject variance / (between-subject variance + within-subject variance)

ICCs were estimated based on the number of days of assessment: all data (8 days), day 2 to day 7 (6 days), day 2 to day 6 (5 days), day 2 to day 5 (4 days), and day 2 to day 4 (3 days). Between-subject variance and within-subject variance were obtained through the estimated variance components in random effects models via restricted maximum likelihood methods using a compound symmetric covariance structure in SAS PROC MIXED.

Comparing Accelerometers and SOFIT. To align SOFIT and accelerometer data for agreement comparison, we examined eight different combinations of models for sedentary + light + moderate + vigorous activity. To align with the accelerometer data, the SOFIT rating was summed at the minute level by either taking the closest integer of the mean of the ratings for the three 20-second intervals (MEAN), taking the mode of the ratings for the three 20-second intervals (MODE), taking the maximum of the ratings for the three 20-second intervals (MAX), or taking the minimum of the ratings for the three 20-second intervals (MIN). The model that best fit the assumptions of accelerometer was [1 = sedentary, 2 + 3 = light, 4 + 5 = moderate + vigorous] when taking the mean of the ratings for the three 20-second intervals. This model was utilized for the analysis. SOFIT data was then summed at the minute level by taking the closest integer of the mean of the ratings for the three 20-intervals and were categorized as: 1 = sedentary, 2 + 3 = light, 4 + 5 = moderate + vigorous.

Comparing Accelerometers and PA Logs. Energy expenditures for data obtained via PA logs were coded as follows: sleeping at 0.9 METS, sedentary at 1.1–1.5 METS, light activity at 1.6–2.9 METS, moderate activity at 3.0–5.9 METS, and vigorous activity at 6.0 METS and above.^18,19 For the purpose of comparison, sleeping and sedentary MET categories were combined into one category (sedentary). In order to align accelerometer data with PA log data, which recorded activity by 30-minute intervals, accelerometer data were summarized every 30 minutes into 4 activity levels using the following rules; sedentary, if counts per 30 minutes ≤ 1200, light, if counts per 30 minutes ≥ 1201 but ≤ 68,850, moderate, if counts per 30 minutes ≥ 68,851 but ≤ 204,450, and vigorous, if counts per 30 minutes ≥ 204,451.

The Kappa statistic was used to calculate the level of agreement between the two sets of measures compared. Weighting was done using the WTKAP option, in the PROC FREQ TEST statement. By default, PROC FREQ uses Cicchetti-Allison weights when computing the weighted Kappa coefficients.

Results

Among the 50 children who wore accelerometers for phase 1 of the study, after downloading three accelerometers were found to not contain data, leaving 47 children with accelerometer data. In phase 2 of the study, 46 children returned and wore the accelerometers; however, five accelerometers contained no data. As a result, there were a total of 49 children with accelerometer data, of which 39 had data for both phases, 8 had data only for phase 1 and two had data only for phase 2. Those 49 children constituted a total of 689 days of accelerometer data. Among those 49 children, 23 had at least one day of data deleted because the total number of minutes for sedentary activities was greater than 1300 minutes for that day. This means that over 21 hours for that day was recorded as sedentary, which we interpreted as the accelerometer being removed for that day and thus did not include it in the analysis. In total, 86 (12.5%) days of data were removed, leaving a total of 603 days of data for further analysis.

The mean age of the 47 children (phase 1) with valid accelerometer data was 3.6 ± 0.5 years (Table 1). The majority (64%) were male, 46% were part Native Hawaiian (all of mixed ethnicity); and 14% were part Other Pacific Islander (all of mixed ethnicity)-(data not shown). Mean BMI percentile was 61.1 ± 33.2.

Table 1.

Demographic characteristics among children in the sample in week one

	Accelerometer n=47a	SOFITb n=30	PA Logsc n=45
	Mean ± Standard deviation
Age (years)	3.5 ± 0.5	3.5 ± 0.6	3.5 ± 0.6
Weight (kg)d	16.9 ± 2.7	16.5 ± 2.8	16.7 ± 2.8
Height (cm)d	101.2 ± 4.3	100.6 ± 4.6	101.0 ± 4.4
BMI percentiled	61.1 ± 33.2	59.9 ± 32.8	59.2 ± 32.4
	Percentage (%)
Sex
Female	36	43	40

^a47 children with valid accelerometer data during week 1.

^bSystem for Observing Fitness Instruction Time (SOFIT).

^cPhysical Activity (PA) Logs.

^dOne child did not have height or weight measured.

Validating accelerometer data with SOFIT

Of the 47 phase I preschoolers with accelerometer data, 30 have corresponding SOFIT data. The maximum observation time per child was 176 minutes (around 3 hours) and the minimum observation time was 76 minutes. The 30 children completed a total of 3976 activities, with each minute assigned for a type of activity. Overall, the proportion of agreement between SOFIT and accelerometers was 0.74. Table 2 represents the agreement in 1-minute comparison for the categorization of minutes of activity, between the accelerometer and SOFIT. Kappa index was used to quantify how much the agreement exceeds chance levels. The weighted Kappa coefficient was 0.17 (P<.001), indicating slight agreement between the SOFIT and accelerometers-data not shown in tables. Challenges in “agreement” occurred where the accelerometer classified 391 minutes (accelerometer row “MVPA” and SOFIT column “Light”) of activity as moderate/vigorous, however this same time frame was categorized as light by SOFIT. The same occurred when the SOFIT categorized 406 activity minutes as moderate/vigorous, which the accelerometer categorized as light (under the SOFIT column “MVPA” and accelerometer row “Light”).

Table 2.

Categorization of activities per minute of Physical Activity and intensity level (Sedentary, Light, Moderate and Vigorous Physical Activity [MVPA]) as identified by the Accelerometer and the (modified) SOFITa (n=30)^a

		SOFITb
		Sedentary	Light	MVPA	Total
Accelerometerc	Sedentary	6	205	16	227
	Light	12	2744	406	3162
	MVPA	3	391	193	587
	Total	21	3340	615	3976

^aPercentage of agreement was 74% (6 + 2744 + 193) /3976.

^bSOFIT: System for Observing Fitness Instruction Time. SOFIT scores activities at a 20-second interval using a 5-point scale (1 = lying down, 2 = sitting, 3 = standing, 4 = walking, 5 = very active). In this study, activities were summed at a 1-minute interval using mean scale of the three 20-second intervals and were categorized as: 1 = sedentary, 2 + 3 = light, 4 + 5 = moderate + vigorous.

^cAccelerometer counts per minute (cpm) were calculated for the accelerometer data by adding data corresponding to four 15-second intervals. Levels of activity were categorized as Sedentary, if cpm ≤ 40, Light, if cpm were ≥41 or ≤ 2295, Moderate, if cpm were ≥ 2296 or ≤ 6815 or Vigorous, if cpm ≥ 6816.

Validating accelerometer-derived data with PA logs

Of the 47 phase I preschoolers with accelerometer data, 45 children have corresponding data collected through PA logs. Those 45 children contributed a total of 6287 activities, with each activity representing a 30-minute interval. A total of 719 (11.4%) activities were categorized as “child default for additional minutes” and were excluded from the analyses, leaving a total of 5568 activities for further analysis. The proportion of agreement for PA logs and accelerometers was 0.40 [(1666 + 493 + 63 + 30) / 5568] (Table 3). The corresponding weighted Kappa coefficient was 0.15 (P<.001), representing slight agreement between the logs and accelerometers. Table 3 also suggests that the PA log, compared to accelerometers, often underestimated moderate and vigorous types of activity levels. For example, from Table 3, the accelerometer identified a total of 405 minutes of activities as moderate and 538 minutes of as vigorous activities. Under PA log, of the 405 accelerometer-identified moderate minutes of activities, 206 were coded as sedentary; of the 538 accelerometer-identified minutes of vigorous activities, 383 were coded as sedentary. Hence, a large quantity (62% [(206 + 383) / (405 + 538)] of the non-sedentary activities were treated as sedentary.

Table 3.

Categorization of physical activities per 30-minute intervals at the different intensity level (Sedentary, Light, Moderate and Vigorous Physical Activity (MVPA)) as identified by the Accelerometer and the PA Log (n=45)^a

		Physical Activity Log
		Sedentary	Light	Moderate	Vigorous	Total
Accelerometerb	Sedentary	1666	41	34	35	1776
	Light	1893	493	260	203	2849
	Moderate	206	80	63	56	405
	Vigorous	383	70	55	30	538
	Total	4148	684	412	324	5568

^aPercentage of agreement was 40% (1666 + 493 + 63 + 30) /5568)

^bAccelerometer Counts per minute (cpm) were calculated for the accelerometer data by adding data corresponding to four 15-second intervals. Levels of activity were categorized as Sedentary, if cpm ≤ 40, Light, if cpm were ≥41 or ≤ 2295, Moderate, if cpm were ≥ 2296 or ≤ 6815 or Vigorous, if cpm ≥ 6816.

Discussion and Conclusion

The aim of this study was to validate a wrist-mounted Actical accelerometer by comparing against direct observation (SOFIT) and proxy reports (parental logs) for multiethnic preschool children. The results from the Actical accelerometer had “slight agreement” with the SOFIT/ direct observation (our objective measure) and the subjective measure (PA logs). In other studies conducted among free living children, Burdette and colleagues,⁸ as well as Trost and others,²⁹ found slight agreement (r=0.20) and (r=0.19) respectively with their accelerometers (Tritrac-R3D and CSA) parental recall (Burdette) and a self-report questionnaire (Trost). Children in the Trost study however were much older than the ones in both our sample and that of Burdette's. Others who have used the Actical in researching PA in infants and toddlers found stronger levels of agreement (r=0.42) relating the subjective measure to a proxy report of activity from caregivers.³⁰ However, participants in this study,³⁰ were younger (4–17 months) than those in our study and wore the accelerometer for four days, compared to our study which was conducted over two, 7-day wearing periods. Higher levels of agreement than reported by our study were found by Van Cauwenberge and colleagues³¹ (r=0.66) in correlating accelerometer outputs with objective measures of activity; however, Van Cauwenberghe and colleagues³¹ derived their results utilizing an Actigraph accelerometer, which at study time was not water-proof and therefore not appropriate for use in Hawai‘i, where water-based activities may be more common. Our results also differ from other literature³² which indicated over-reporting by surrogates of children's activity; by contrast, our (parental proxy/surrogate reports) PA logs underreported their preschoolers' activities.

Possible reasons for the slight agreement we found between the subjective (PA Logs) and objective (Actical) measures as well as between the two objective measures (SOFIT and Actical) may include the small sample size in both phases of the study. We also posit that surrogate reporting by parents and caregivers who may not have been aware of activities their children engaged in at school or other times they are away from home. The required data collection instrument may have been a burden to busy parents, as they were asked to keep detailed reports of their children's varying activities within half-hour increments. Utilizing PA logs with surrogate reporters requires special preparation and effort and may not be the most appropriate validation tool because of these requirements. Also, utilizing SOFIT which employs momentary sampling (estimates of behavior as having occurred throughout an observation interval anchored by the end of that interval) may have played a part in the results obtained and is a limitation. Momentary sampling methods may over or under estimate activity when compared to methods that measure continuous behavior (in this case, the accelerometer). Compared to SOFIT the accelerometer tracked movement (activity) continuously, therefore requiring no estimating, generalizing or scoring of activity and activity level.

The accelerometers were novel for children this young. Wearing of the device required their cooperation and compliance. Parents and guardians ensured wearing compliance for the duration of the study, and also, a number of strategies were devised to make the wearing of the devices desirable and fun including providing a colorful assortments of wristbands to choose from. Despite these efforts, a few children removed the device.

Notwithstanding, the unique contribution of this study is the utilization of the Actical accelerometer for measuring PA in understudied, underrepresented children (Native Hawaiians and Pacific Islanders) of mixed ethnicity in a not often studied age group, preschool age. There is a paucity of PA studies in low-income, free-living, preschool children, especially children of Native Hawaiian or Pacific Islander ancestry and mixed ethnicity. Research on the growing segment of the mixed race population is warranted and timely. Overall, these results can provide researchers with useful information that supports physical activity research in Native Hawaiian and Pacific Islander preschool children. As the prevalence of overweight and obesity are increasing in these populations, researchers should employ valid and reliable methodologies and tools aimed at measuring PA. This study is one of the first to evaluate a wrist-mounted accelerometer in Native Hawaiian, Pacific Islander and mixed ethnicity preschool children. Results indicate that wrist-mounted accelerometers may be a viable indicator of PA and, with further validation using larger samples, could be used for measurement evaluations, interventions or longitudinal investigations. Future directions are to identify pattern recognition for specific activities, investigate shorter time sampling with accelerometers, as well as utilize accelerometry for PA prevalence studies in larger samples of this population.

Conflict of Interest

None of the authors identify any conflicts of interest.

Disclosure Statement

The CHL program is supported by the Agriculture and Food Research Initiative Grant no 2011-68001-30335 from the USDA National Institute of Food and Agricultural Science Enhancement Coordinated Agricultural Program. The findings and conclusions are those of the authors and the contents of this publication do not necessarily reflect the views or policies of the United States Department of Agriculture. The mention of trade names, commercial products, or organizations is for identification only and does not imply endorsement by any of the groups listed above.