Compliance With Wearing Physical Activity Accelerometers in High School Students

Abstract

Background

Accelerometer use in physical activity research has become increasingly popular but is prone to problems with missing data, which complicate the data reduction and analysis process. The purpose of this study was to determine the effect of hypothesized compliance strategies on improving compliance with wearing a physical activity accelerometer in high school students.

Methods

Each of four local high schools was assigned to one of four compliance strategies: (1) receiving three phone calls, (2) completing a daily journal, (3) compensation contingent on number of complete (≥ 10 hours) days of data, and (4) control condition. Participants wore ActiGraph accelerometers for seven days to determine compliance and physical activity.

Results

The contingent group had the highest level of compliance with 96% of the participants acquiring at least four of seven complete days of data. After controlling for grade level, school level percent minority students, and school level socioeconomic status (SES), the contingent group’s compliance remained significantly higher (P = .04) than the journal (85%), phone (72%), and control (70%) participants.

Conclusions

The contingent compliance strategy improved the amount of time the students wore the monitor each day and, thus, the total number of days with ≥ 10 hours of data.

There is an increasing dependence on objective monitoring for measuring physical activity. Given the use of accelerometers in increasingly large scale studies, such as the National Health and Examination Survey (NHANES) 2003 to 2004,¹ improving subject compliance is an important issue so that the amount of missing accelerometer data can be reduced.

Traditional methods of physical activity assessment include self-reports, interviews, and activity diaries.² More recently, accelerometers have become increasingly popular to provide a more objective measure of physical activity intensity.³ In general, accelerometers use sophisticated electronics to detect accelerations in single or multiple planes of motion. For overall physical activity, the accelerometer is typically worn at the waist and, thus, provides an estimate of whole body movement.

Although accelerometers are typically viewed as superior to self-report measures,⁴ there remain several unresolved issues with the data reduction process.⁵ If the accelerometer is set to collect data in 1-minute intervals for one week, there will be 10,080 data points to reduce to meaningful summary variables. In early field studies, if the subject met a certain level of counts per day, every data point was used, assuming that the subject was wearing the monitor as instructed.⁶ More recent studies are attempting to account for the times when investigators believe the subject was not wearing the accelerometer (missing data).^7-9 The process by which these missing data points are identified has not been standardized, and there is still considerable disagreement on the best procedures for dealing with such data. Missing data have been operationalized in the data reduction process by identifying consecutive minutes of zero output from the accelerometer. One review found that studies have ranged from 10 to 30 minutes of consecutive zeros⁵ and more recent studies have used 60 minutes.¹⁰

Reducing the amount of missing data will place less of a burden on the data reduction process and the assumptions and decisions made in this process, which will have less impact on the calculated summary variables and results. There have been no experimental studies investigating the efficacy of different compliance strategies conducted before this study although several strategies have been proposed.¹¹ It is currently not known which strategy or strategies will ultimately lead to the highest levels of compliance among subjects wearing accelerometers.¹² Therefore, the purpose of this study was to determine the effect of hypothesized strategies on compliance with wearing a physical activity accelerometer in high school students. A secondary purpose was to perform a sensitivity analysis to determine the effect of data reduction procedures on levels of compliance.

Methods

Design

Using a quasi-experimental design, 3 compliance strategies¹¹ were tested against a control condition using four local public high schools. The three strategies were (1) receiving three phone calls during the week as reminders; (2) completing a daily journal to record times when the monitor was on, removed, and why the monitor was removed; and (3) compensation amount contingent on number of complete (≥10 hours) days of data. To avoid cross contamination among subjects, only one condition was administered to the students at a given school. Schools were randomly assigned to one of the four conditions following school agreement to participate in the study.

Subjects

A convenience sample of students from four Minneapolis, MN high schools was invited to participate in this study. At each school, 3 to 4 physical education, health, or science classrooms were identified by school principles and faculty. Students within those classes were invited to participate in the study following brief classroom presentations. Students received an introductory letter and consent form to take home to their parent/guardian and returned the signed forms to their teacher. Students that were ≥ 18 years old were allowed to sign the consent form for themselves, although their age was verified by teachers or administrators. A total of 89 students were recruited into the study, resulting in a 35% participation rate using the total number of students in the classrooms as the denominator.

We did not want to influence student compliance by telling them the true purpose of the study as this may have artificially improved compliance. Therefore, the students were told that they would be testing out a new device (the accelerometer) for measuring physical activity in high school students. Upon completion of the study protocol, students were told the true purpose of the study, and the student and a parent/guardian were given the opportunity to withhold their data. Two students (no parents) withheld their data. This study was approved by the university’s Institutional Review Board.

Measures

The ActiGraph (Model 7164; Pensacola, FL) accelerometer was used to determine times when the monitor was being worn and when it was not, in addition to physical activity. The ActiGraph has been previously validated for use with children in laboratory and field settings.^13-15 It is a small (5.1 × 3.8 × 1.5 cm), light-weight (42.6 g), single plane (vertical) accelerometer that collects and stores accelerations from 0.05 to 2.00 G with a frequency response of 0.25 to 2.50 Hz. These settings capture normal human motion but will filter out high frequency vibrations from mechanical sources (eg, operating a lawn mower).¹⁶ The analog acceleration is filtered and converted to a digital signal and this value (count) is stored in user-specified time intervals. One-minute intervals were used for this study. ActiGraph monitors were initialized for a common time for all children at a particular school although this could not be standardized across schools due to different school and classroom schedules. After data collection, each monitor was downloaded to a computer for subsequent data reduction and analysis.

Upon return of the accelerometer, participating students also completed a brief questionnaire to obtain demographics and feedback regarding their experience with wearing the accelerometer and the specific condition to which they were assigned. For the students in the journal condition, journals were collected at the end of the study but were not used in the data reduction process. Rather, the journals were used as a means of, possibly, keeping the students “on task” and as a physical reminder to wear the accelerometer.

Procedures

All data were collected during the Spring of 2007 (March-May). Due to a limited number of accelerometers and school schedules, data collection at different schools did not overlap. The control condition received their accelerometer along with brief verbal instructions and a one-page summary of the instructions. Students received a telephone call the night before they were scheduled to return the accelerometer. All students in the experimental conditions also received the same verbal and written instructions and a phone call the day before the scheduled return of the monitors. The journal condition received additional instructions to complete a daily journal for one week to record the times when the monitor was worn, when it was removed, and the reason for removing the monitor. The phone condition received two additional telephone calls on the 3rd (Monday) and 5th day (Wednesday) of monitoring. At the time of recruitment, the contingent condition viewed a graphical representation of some typical data downloaded from the activity monitor to illustrate compliant versus noncompliant days of data. They were then told they would be compensated for each day with at least 10 hours of data ($5.00 per day) and would receive an additional $10.00 upon the return of their physical activity monitor. All other conditions were told they would receive $35.00 for participating and $10.00 upon return of the monitor.

Data Reduction

ActiGraph data were reduced using a custom developed software program.⁹ All data contained within the time frame from when the monitor was initialized (ie, when the students received their monitors) until the same time the following week was processed. For example, if students received their monitors at 10:00 AM on Friday, the data from Friday at 10:00 until the next Friday at 10:00 would be processed through the program. For days 2 to 7, all data from 00:01 until midnight was reduced to summary variables. Day one and day eight were combined to form a composite seventh day of data.

Daily inclusion criteria were established to determine days and times with acceptable accelerometer data. Blocks of time incorporating at least 20 continuous minutes of “0” output from the ActiGraph were considered to be times when the subject was not wearing the monitor.⁸ These data points were eliminated and not used in any calculations. Following these deletions, days with less than 10 hours of data were eliminated from data reduction to account for unrepresentative days of physical activity.

The reduced data were placed into three data sets (usual, weekdays, weekend). Previously, it has been shown that four days of activity monitoring are needed to provide a reliable estimate (r = 0.80) of usual physical activity.¹⁷ Students with at least four out of seven days of ActiGraph data were retained for the usual data set. To be included in the weekday and weekend data sets, students needed to have 3 to 5 weekdays and at least one weekend day that met the daily inclusion criteria.

After processing the data through the inclusion criteria, summary compliance and physical activity variables were calculated. Compliance was assessed by calculating the number of days with at least 10 hours of data. To examine the average number of hours of data per day, participants were divided into three compliance categories based on the number of days of complete data they recorded: <4 days, 4 to 6 days, or 7 days. For these analyses, we included all days in the dataset, including those days with less than 10 hours of data. Average ActiGraph counts per minute was calculated as the total counts for all included days summed and divided by the total number of minutes the monitor was worn for all included days. Time spent in moderate to vigorous physical activity (MVPA) was calculated in two ways: (1) the average number of minutes per day spent in MVPA and (2) the average percent of time spent in MVPA each day. The time spent in MVPA was calculated using age-specific count cutoffs based on using 3.0 and 6.0 as the metabolic equivalent (MET) cutoffs for moderate and vigorous intensity activity, respectively.¹⁸

Sensitivity Analysis

A sensitivity analysis was performed to determine if changing the off-bout criteria (the number of continuous zero counts) would alter the compliance or physical activity results. Accelerometer data were reduced using the 20-, 30-, 45-, and 60-minute off-bout criteria. If an off-bout spanned midnight (eg, 9 PM to 6 AM), two off-bouts were computed: one for the off time before midnight and another for the off time after midnight.

Statistical Analysis

Demographic and physical activity descriptive data were analyzed using one-way general linear models and Mantel-Haenszel chi-square tests. Although randomization was at the school level, the unit of analysis is the individual since there was only one school per condition. One-way general linear models and Duncan’s multiple range tests were used to identify differences in compliance between demographic groups defined within the following categories: gender, race, race category (White vs. minority), and grade level. Among the four treatment groups, differences in the number of days the accelerometers were worn were identified using mixed model analyses while controlling for grade level, school level percent minority students, and school level SES. In addition, one-way general linear models and Duncan’s tests were used to identify possible physical activity differences for different levels of compliance. For these analyses, three compliance groups were defined by the number of complete days of data (0–3, 4–6, or 7 days). For the sensitivity analysis, a repeated measures general linear model was used to determine the effect of the different off-bout criteria for compliance and physical activity summary variables.

Results

Overall school characteristics and student participant characteristics are presented in Table 1. The school receiving the contingent compliance strategy included students that were significantly younger than the other three treatment groups (P < .001). There were no significant differences in age group distribution, grade level, percent minority, or percent male among the treatment groups. There was, however, a possible trend for percent minority (P = 0.08), with a greater percentage of minority or mixed race students in the contingent and journal conditions.

Table 1.

School and Student Participant Characteristics

	Group
	Control	Phone	Journal	Contingent
School level
Enrollment	1618	1837	1345	1419
% non-White	43.8	44.3	80.9	84.5
% male	50.7	46.4	52.5	47.4
% elligible for free / reduced lunch	36.5	35.5	76.4	75.7
Student level
n	20	21	20	26
Age mean (SD); yr	17+1.5	18+0.9	17+1.4	16+1.0*
Age distribution (n)
15 yrs	2	0	1	13
16 yrs	6	2	5	8
17 yrs	7	7	5	4
18+ yrs	5	12	9	1
Grade level (%)
9–10	65.0	14.3	40.0	96.2**
11–12	35.0	85.7	60.0	3.8
% non-White or mixed	30.0	61.9	95.0	80.8**
% male	55.0	42.9	15.0	19.2

^*P = 0.001;

^**M-H chi square P = 0.08.

Compliance

Compliance data by demographic characteristics are presented in Table 2. There were no significant differences in compliance based on gender or race. Participants in grades 9 to 10 had significantly higher compliance rates than those in grades 11 to 12. Ninety percent of all 9 to 10th graders had at least four days of complete data compared with only 71% of the 11 to 12th graders (P = 0.0007). Similarly, 9 to 10th graders wore the accelerometers 13 hours per day, while 11 to 12th graders wore them 11 hours per day on average (P < .001). Participants were divided into three compliance categories based on the number of days of complete data they recorded: <4 days, 4 to 6 days, or 7 days. As expected, each of these groups was significantly different than the other two, based on the average number of hours worn per day (P < .0001). The <4-day group only wore the accelerometers for 7 hours per day on average compared with 12 and 15 hours per day for the 4 to 6 and 7-day groups, respectively.

Table 2.

Compliance Statistics by Gender, Race/Ethnicity, and Grade Level

	Gender			Race/ethnicity					Grade level
	All students (n = 87)	Female (n = 59)	Male (n = 28)	Caucasian (n = 28)	Hispanic (n = 12)	African-American (n = 20)	Asian (n = 16)	Other/mix (n = 11)	9–10 (n = 49)	11–12 (n = 38)
% of students with
0–3 days	18.4	18.7	17.8	25.0	25.0	15.0	6.3	20.0	10.2	29.0
4–6 days	47.1	47.5	46.4	50.0	33.3	60.0	56.3	20.0	42.9	52.7
7 days	34.5	33.9	35.7	25.0	41.7	25.0	37.5	60.0	46.9	18.4
At least 4 days (%)*	81.6	81.4	82.1	75.0	75.0	85.0	93.8	80.0	89.8	71.1
Hours worn /day*	12.1	12.1	11.9	11.4	12.7	12.1	12.1	13	12.9	10.9
Mean (sd)	(3.38)	(3.26)	(3.84)	(3.21)	(3.88)	(3.83)	(2.81)	(4.03)	(2.60)	(4.07)

^*significant grade level difference P ≤ .008.

Table 3 provides compliance data for the four treatment groups. Ninety-six percent of participants within the contingent group had at least four out of seven complete days of data. After controlling for grade level, school level percent minority students, and school level SES, this remained significantly greater (P = 0.04) than the 85% of journal participants, 72% of phone participants, and 70% of control participants with at least four days of complete data.

Table 3.

Compliance Statistics by Experimental Group

	Group
	Control (n = 20)	Phone (n = 22)	Journal (n = 25)	Contingent (n = 26)
% of students with
0–3 days	30.0	28.6	15.0	3.9
4–6 days	40.0	57.2	50.0	42.3
7 days	30.0	14.3	35.0	53.9
At least 4 days*	70.0	71.5	85.0	96.2

^*P = .04 controlling for grade level and school level percent minority students and school level SES (free and reduced school lunch).

There was no support for the hypothesis that more compliant participants were more active when comparing the three compliance groups (<4 days, 4 to 6 days, 7 days) for the average counts per minute (P = 0.69), absolute minutes (P = 0.17 to 0.71) or percent of time spent in activity intensity categories (P = 0.16 to 0.97). The only exception was a significant difference in the average number of minutes per day of sedentary behavior between the <4-day and the 7-day compliance groups. The least compliant group recorded 490 minutes per day of sedentary activity, while the most compliant group recorded 555 minutes per day (P = 0.038).

Physical Activity

Descriptive physical activity data are presented in Figure 1. In general, males spent a greater percent of their time in MVPA compared with females (P < .001). In addition, males had a significantly higher average number of counts per minute than females on all days (518.7 ± 236.81 vs 340.1 ± 106.64, P < .0001) and when days were divided into weekdays (P < .0001) and weekends (P = 0.0028). There were no significant differences between males and females in the minutes or percentages of time spent performing sedentary or light physical activity.

Figure 1.

Percent of time spent in moderate-and-vigorous physical activity (MVPA) and vigorous physical activity (VPA) by gender, race, and grade level; Mean + SD.

Caucasians spent a significantly greater percent of their time in MVPA (P = 0.0035) and VPA (P = 0.016) compared with all minorities combined. Caucasians had a significantly higher average number of counts per minute than minorities on all days (483.6 ± 242.99 vs 362.9 ± 137.05, P = 0.0084) and when days were divided into weekdays (P = 0.0034) and weekends (P = 0.044). There were no significant differences between Caucasians and minorities on measures of sedentary and light physical activity. Physical activity was also compared when minorities were categorized into the following groups: Hispanic, African-American, Asian, and Other/Mixed (Figure 1). Asians spent a significantly smaller percent of time performing MVPA compared with all other race groups (P = 0.031). No other significant differences were found among these groups.

When comparing 9 to 10th grade students with 11 to 12th grade students, the older group of students performed an average of 294 minutes per day of light activity, while the younger students performed 266 minutes per day (P = 0.044). No other significant differences were detected.

Sensitivity Analysis

The mean number of off-bouts per day calculated when applying the 20-, 30-, 45-, and 60-min off-bout criteria were 3.0 ± 1.50, 2.2 ± 1.01, 1.7 ± 0.69, and 1.4 ± 0.54, respectively (all significantly different from each other, P = 0.010). Mean hours worn per day and the number of days of data significantly increased with increasing off-bout criterion length from 12.1 ± 3.38 hours per day and 5.3 ± 1.76 days for the 20-min criterion to 13.0 ± 3.79 hours per day and 5.5 ± 1.69 days for the 60-min criterion (P < .001 for both). Going from the 20- to 30-min criterion and from the 30- to the 45-min criterion resulted in one and two additional subjects with at least four days of data, respectively. There was no further gain in subjects going from the 45- to the 60-min criterion. Minutes per day of sedentary activity also increased with increasing off-bout criterion length; 523 ± 91.4, 543 ± 102.8, 562 ± 113.5, and 576 ± 123.7, respectively (all significantly different from each other, P < .001). Light, MVPA, and VPA decreased slightly (1–2 minute difference) with increasing off-bout criterion length but they were not statistically significant (data not shown).

Discussion

The purpose of this study was to identify compliance strategies that significantly improve compliance with wearing a physical activity accelerometer in high school students. Our primary finding was that compliance was the highest in the group receiving contingent compensation. This group was also the youngest, as differences in grade level were significant; however, analyses controlled for grade level and other demographic characteristics, implying that the contingent compliance strategy improved the amount of time the students wore the monitor each day and, thus, the total number of days with at least 10 hours of data.

In a previous study of accelerometer compliance among middle-school students, there were no significant differences in compliance rates by gender, race, or grade level.¹⁹ Grade group differences in the current study indicate that older students may be less willing to participate in research projects, possibly due to time and social constraints. These findings may indicate a need for different compliance strategies for different age groups. While fewer monetary incentives may be required for elementary school students, older high school students may require additional monetary or other incentives such as class points, extra credit, or other rewards or privileges to improve compliance. There appeared to be no differences in the physical activity levels based on the number of days the student had at least 10 hours of data. The hypothesis was that less compliant students may also be less active; however, students may be compliant or noncompliant for a number of reasons. Qualitative data collected when the monitors were returned indicated that most students did not wear the monitor as instructed simply because they forgot. More girls indicated not wearing the monitor because of fashion/clothes or social conformity (not wanting to look different or be asked a lot of questions about the accelerometer).

Among all students, physical activity was found to be higher in males and in Caucasians. These results are similar to other studies that have found higher levels of physical activity for males and Caucasians compared with females and minority students, respectively.^6,20-24 There was no difference in physical activity between the younger and older students, contrary to several other studies.^6,22,24 This discrepancy, however, may be due to how the students were categorized, the method of physical activity assessment (survey versus accelerometer), or the procedures used to reduce the accelerometry data.

The sensitivity analysis indicated that while there were some changes in compliance statistics, there was little change in the physical activity variables, with the exception of minutes spent in sedentary behavior. An increasing off-bout criterion length allows more zero count values to be included in the data reduction process, therefore increasing the number of epochs (minutes) of sedentary behavior. Increasing the off-bout criterion resulted in fewer off-bouts per day (from about 3.0 to 1.4) and one to three additional subjects with at least four days of data. Based on the method used to calculate the off bouts, two off bouts per 24-hour period would be reasonable if the person removed the accelerometer only for sleeping. Therefore, the 30- or 45-minute off-bout criteria lengths (2.2 ± 1.0 and 1.7 ± 0.7 off bouts per day, respectively) may be the most appropriate to use with high school students. Although it is not possible to determine which off-bout criterion is the most valid, these longer criteria will increase sample size modestly without significantly affecting estimates of time spent in MVPA. The same is not true, however, for sedentary behavior measured with accelerometry. Changing from the 20-min to the 60-min off-bout criterion resulted in an additional 52 minutes of sedentary behavior.

A limitation of this study was that the higher level of economic deprivation (% receiving free or reduced school lunch) observed among students in the contingent school may have provided an especially strong incentive compared with students at a school with less school-wide deprivation, such as the control school. We were able to minimize this potential problem by controlling for school-level SES and percent minority in the analyses; however, the possibility remains that some other school-level factors not measured may be confounding this association. A more rigorous experimental design (eg, matching schools on demographic characteristics and then assigning conditions) is recommended for any future research in this area to further control for school-level differences. The sample was economically and racially diverse but was conducted in one midwestern city and may not be generalizable to other high school students in other geographies. Further testing is needed to follow up these findings in elementary and middle schools and other geographical settings. Lastly, the contingent condition only tested one level of compensation ($5.00 per day). Our results indicate that this level of compensation was adequate to increase compliance with wearing the accelerometer. It is not clear, however, what level of compensation would be adequate to achieve the same result and whether this would differ based on subject age (elementary school, middle school, adults, older adults). Not all studies will be able to afford such monetary incentives as considered in the contingent condition. The journal condition tested here, however, provides a lower cost alternative that was also effective in increasing the percent of students with usable data compared with the control condition.

Conclusion

Using a compensation strategy that was contingent on the number of complete days of data (at least 10 hours per day) resulted in at least four days of data for 96% of the students, significantly more than the other conditions tested. Only 70% of the students in the control condition had data for at least four days. The journal condition resulted in the second highest percent of students with at least four days of data (85%), providing a less costly alternative to the monetary incentive tested in the contingent condition. When possible, it is recommended that a contingent compensation strategy be used to improve compliance with wearing physical activity accelerometers in high school students.

Altering the number of consecutive minutes required to identify times when the accelerometer was not being worn (off bouts) had modest effects on compliance statistics and resulted in one to three additional subjects with at least four days of data. Effects of the different off-bout criteria on physical activity were minimal, although nearly one additional hour of sedentary behavior was identified using a 60-min criterion compared with the 20-min criterion. Investigators must consider the effects of data processing decisions on compliance, physical activity, and sedentary behavior outcomes. The intent of this sensitivity analysis was not to confirm the most appropriate criteria to determine nonwearing time for accelerometry research but to provide evidence that such data processing decisions may impact results. Therefore, further research in this area is warranted.