Process evaluation results from the HEALTHY physical education intervention

Abstract

Process evaluation is an assessment of the implementation of an intervention. A process evaluation component was embedded in the HEALTHY study, a primary prevention trial for Type 2 diabetes implemented over 3 years in 21 middle schools across the United States. The HEALTHY physical education (PE) intervention aimed at maximizing student engagement in moderate-to-vigorous physical activity through delivery of structured lesson plans by PE teachers. Process evaluation data collected via class observations and interventionist interviews assessed fidelity, dose delivered, implementor participation, dose received and barriers. Process evaluation results indicate a high level of fidelity in implementing HEALTHY PE activities and offering 225 min of PE every 10 school days. Concerning dose delivered, students were active for approximately 33 min of class, representing an average of 61% of the class time. Results also indicate that PE teachers were generally engaged in implementing the HEALTHY PE curriculum. Data on dose received showed that students were highly engaged with the PE intervention; however, student misbehavior was the most common barrier observed during classes. Other barriers included teacher disengagement, large classes, limited gym space and poor classroom management. Findings suggest that the PE intervention was generally implemented and received as intended despite several barriers.

Introduction

Background and rationale for HEALTHY

Physical inactivity is a primary contributing factor for several major diseases including Type 2 diabetes. Type 2 diabetes is a serious disease with complications that can include heart disease, kidney failure, blindness and limb amputation. Historically, Type 2 diabetes was rarely found in children and adolescents; however, there have been significant increases in the prevalence of overweight/obesity [1, 2] and incidence of Type 2 diabetes [3–5] in recent decades in the pediatric population of the United States.

Current research indicates that regular physical activity is essential to the prevention of Type 2 diabetes [6–8] and federal guidelines recommend that children and adolescents engage in 60 min of physical activity daily, most of which should be moderate or vigorous intensity aerobic physical activity [9]. Few youth meet this goal, however. Data from the National Survey of Children’s Health showed that only 30% of children and adolescents engaged in vigorous physical activity for at least 20 min each day [10]. In addition, evidence from schools, which are particularly influential environments regarding pediatric health, showed inadequate school-based physical activity. Findings from the School Health Policies and Programs Study, a national survey of school health policies and practices, indicated that 78% of schools required that students take physical education (PE) courses; however, only 22% of middle schools provided 45 min of PE each day for at least 18 weeks of the school year [11]. Moreover, according to findings from the Youth Risk Behavior Survey, 44% of adolescents indicated that they did not attend PE classes during a typical week [12].

In response to the considerable increase in the incidence of pediatric Type 2 diabetes, the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health sponsored a primary prevention trial to reduce the risk factors for Type 2 diabetes called HEALTHY [13]. The objectives of the HEALTHY study were to develop and test a multi-component school-based intervention aimed at reducing modifiable risk factors for Type 2 diabetes in youth by promoting physical activity and healthy nutrition.

HEALTHY study design

The HEALTHY study was a multi-site, cluster randomized controlled trial with 42 middle schools participating (21 intervention and 21 control). Seven field centers across the United States administered the study and were overseen by a coordinating center. The study participants included 4603 middle school students (2307 intervention and 2296 control) aged 9–14 years who were assessed at the beginning of their sixth grade year and again toward the end of eighth grade on weight-related measures, fasting insulin and fasting glucose. The HEALTHY intervention consisted of four integrated components: behavior, communications, nutrition and PE. The intervention began during the second semester of students’ sixth grade year and continued until the end of their eighth grade year. The details of the HEALTHY study research design and methods have been reported elsewhere [13] as well as details regarding each intervention component [14–17].

HEALTHY PE intervention

The goal of the PE intervention component was to increase students’ engagement in moderate-to-vigorous physical activity (MVPA) via the HEALTHY PE curriculum [17]. A study interventionist identified as the Physical Activity Coordinator (PAC) oversaw the implementation of the PE intervention at each field center, which involved training PE teachers at intervention schools each semester to deliver the HEALTHY PE curriculum. PACs also visited PE classes regularly to provide one-on-one feedback to teachers. Intervention school PE teachers were asked to deliver the HEALTHY PE curriculum in lieu of their preexisting PE curricula. In order to implement the HEALTHY PE lessons, the study provided each intervention school with a PE teaching assistant and necessary PE equipment. The curriculum included a variety of required units (basketball, soccer and team handball) and elective units (badminton, cooperative games, dance, fitness, football, frisbee, lacrosse, pickleball, softball, street hockey, table tennis, tennis, track and field, volleyball, and swimming). Each unit consisted of approximately 10 lessons arranged to build skills consecutively. The lesson plans were developed using middle school PE guidelines from the National Association for Sport and Physical Education [18]. Each lesson was structured and included three activity components: instant activities (IAs), health-related physical activities (HRPAs) and skill development and/or game play activities (SD/GPAs). IAs were short simple activities designed to get students moving at the beginning of class. HRPAs were gross motor or fitness activities aimed at producing MVPA and/or increasing muscle strength. SD/GPAs focused on building and practicing skills as well as group game play. Finally, intervention schools were asked to provide at least 225 min of PE class time every 10 school days during the study. A detailed description of the HEALTHY PE intervention component has been reported elsewhere [17], and the HEALTHY PE curriculum manuals are available to the public [19].

HEALTHY process evaluation

Process evaluation is an assessment of the implementation of an intervention and is useful in understanding the dynamics of a trial and optimizing study efficacy [20, 21]. The HEALTHY process evaluation was based on the conceptual framework outlined by Linnan and Steckler [20] and the design is similar to those used in comparable school-based physical activity intervention studies [i.e. Child and Adolescent Trial for Cardiovascular Health (CATCH), Lifestyle Education for Activity Program (LEAP) and Pathways] [22–24]. The process evaluation of HEALTHY monitored the implementation of the intervention to ascertain the extent to which components were delivered and received as intended. Monitoring was a critical element of the intervention and helped ensure that components were implemented successfully. HEALTHY was a complex multi-component intervention and process evaluation data were used to document the extent to which various components were actually implemented and how the intervention was received by the target group as well as those implementing it. This information can be used to help explain why an intervention is or is not effective. Furthermore, given that HEALTHY was a large multi-site study, process evaluation helped to ensure that intervention components were implemented equally and consistently across sites. Finally, process evaluation data were used to refine the implementation of the HEALTHY intervention using a rapid feedback system whereby process evaluation findings concerning intervention strengths, challenges and recommendations were reported to the intervention team twice per year during implementation.

A variety of components were assessed in the process evaluation of the HEALTHY PE intervention, including fidelity, dose delivered, implementor participation, dose received and barriers [20, 25]. ‘Fidelity’ is the extent to which intervention components were delivered as expected. ‘Dose delivered’ is the amount of each intervention component delivered. ‘Implementor participation’ is the extent to which those implementing the intervention prepared for and engaged in the delivery of components. ‘Dose received’ is the extent to which participants engaged with, interacted with, were receptive to or utilized intervention components. Finally, ‘barriers’ are problems encountered in implementing intervention components and reaching participants. The purpose of this article is to report the process evaluation findings concerning the extent to which the HEALTHY PE intervention was implemented during the main trial in terms of fidelity, dose delivered, implementor participation, dose received and barriers. These findings may be useful for those who are designing, modifying, adopting or evaluating school-based PE programs.

Methods

Process evaluation measures and procedures

Process evaluation data for the HEALTHY study were collected each intervention semester through the course of the intervention, which lasted five semesters as cohort students progressed through middle school. Process data were collected in the 21 intervention schools only. The HEALTHY process evaluation utilized a mixed methods approach, and implementation of the PE intervention was assessed via structured observations and interviews. A comprehensive description of the methods for process evaluation staff training, instrument development, data collection, data entry and data management has been reported elsewhere [25]. Additionally, a report detailing the process evaluation data collected by semester, method and intervention component has been previously published [25].

PE class observations

During the study, trained research assistants observed each PE teacher and their class at least three times each intervention semester. These research assistants were employed by the field centers and were excluded from participating in intervention activities. Observations were conducted randomly throughout each semester, and teachers were typically not informed of when observations would take place. The observation instrument included dichotomous scale items (i.e. implemented/not implemented or present/not present) and activities were timed to ascertain durations. Specifically, observations assessed (i) the class structure (i.e. one teacher with one class or team-taught combined classes), (ii) PE unit taught (e.g. basketball), (iii) completion of HEALTHY PE activities (IA, HRPA and SD/GPA), (iv) teacher engagement, (v) student engagement, (vi) length of time students were active, (vii) total class time and (viii) barriers observed during class. A total of 1101 PE observations were completed across all 21 intervention sites during the course of the study. Observations were spread evenly across the five intervention semesters; however, the number of observations varied by school as the number of PE teachers varied by school. Intervention schools differed in terms of size, student enrollment and number of teachers employed.

Physical activity coordinator interviews

Trained interviewers conducted structured interviews with all seven PACs near the end of each semester regarding PE implementation at the intervention schools. During the first three semesters, one interview was conducted per school; however, during the last two semesters, one interview was conducted regarding all three intervention schools. This change was made so that each PAC could discuss all three of the intervention schools they oversaw in one longer interview instead of three separate interviews, one for each school. Each interview lasted 30–60 min during which interviewers took notes and audio recorded the interview. The interview was scripted and involved Likert-type rating scale items and open-ended questions. Specifically, the interviews assessed (i) the usefulness of study trainings with teachers, (ii) teacher engagement with trainings and PAC interactions, (iii) receptivity of teachers to the PE intervention, (iv) students’ activity levels (v) and barriers encountered during the semester and resolution efforts. Demographic characteristics of PAC interview participants were not systematically collected during the study owing to concerns of identifiability. Not collecting identifiable characteristics may have reduced PAC response bias. A total of 77 interviews were conducted across all intervention sites during the study.

Data analysis

Due to the nature of the process data and the purpose of this article, we conducted a descriptive mixed-methods analysis. Observational data collected at each field center were entered and electronically transferred to a central database maintained by the study coordinating center. The SAS software program (version 9.2; SAS Institute, Cary, NC, USA) was used to analyze these data. Interview data were compressed into key point summaries by trained interviewers at each field center and subsequently transferred to the study’s qualitative data core to be aggregated and analyzed. A more detailed description of this process has been previously reported [25]. Summative interview data were open coded by topic with a grounded theory approach to identify emergent themes and trends using ATLAS.ti (version 5.2; Scientific Software Development GmbH, Berlin, Germany) [26].

Results

Fidelity

Using data from PE class observations, we calculated intervention fidelity values. HEALTHY PE activities were implemented 87.6% of the time (85.3% for IAs, 84.0% for HRPAs and 93.4% for SD/GPAs; Table I). Ideally, HEALTHY activities would have been implemented in all of the intervention school PE classes during the study. However, activity implementation was somewhat lower in the first semester of the intervention and then increased over time with the exception of HRPAs, which dropped slightly in the last intervention semester. According to PAC interviews, there was some initial resistance from teachers to implementing the HEALTHY PE curriculum: some teachers were ‘stuck in their ways’ and others were unwilling to work outside of their ‘comfort zone’. Generally, PACs noted a positive change in PE teachers’ attitudes toward HEALTHY implementation over time as they attained a greater understanding of HEALTHY’s mission and saw how the HEALTHY PE curriculum positively affected student behavior and activity levels. Finally, school records showed that 100.0% of intervention schools scheduled at least 225 min of PE class time every 10 school days through the course of intervention implementation.

Table I.

Fidelity of HEALTHY PE activity implementation

Implementation of PE activities	Intervention semester					Overall (%)
	Spring 2007 (%)	Fall 2007 (%)	Spring 2008 (%)	Fall 2008 (%)	Spring 2009 (%)
IAs	79.0	85.5	84.9	89.6	89.3	85.3
HRPAs	76.5	85.5	87.0	91.4	82.0	84.0
SD or GPAs	86.8	96.5	95.3	94.6	96.1	93.4

Dose delivered

The durations of PE activities were recorded during observations to ascertain dose delivered values. The average total time students were observed participating in HEALTHY activities was 33.1 min (SD = 15.6) per class, with IAs lasting 5.5 min on average (SD = 3.5), HRPAs lasting 8.0 min on average (SD = 6.4) and SD/GPAs lasting 19.6 min on average (SD = 12.0; Table II). Overall, students were active for 61.0% of the class time. Median activity times were slightly lower than mean values, indicating a slight positive skewness. This was due to some teachers extending activities far beyond typical durations as IAs were designed to last 4–7 min, HRPAs were designed to last 10–15 min and SD/GPAs were designed to last 15–25 min. Also, there was considerable variance in the length of PE activities. This variability is likely due to differences in scheduled PE class times across schools and the fact that teachers were given flexibility in terms of how many activities to deliver and how long activities should last.

Table II.

Durations of PE activities

	Median	Mean	SD	Range
IAs	5.0	5.5	3.5	0–18
HRPAs	7.0	8.0	6.4	0–46
SD or GPAs	17.0	19.6	12.0	0–60
Total HEALTHY activity timea	29.0	33.1	15.6	0–82
Other class timeb	20.0	21.1	8.4	0–51
Total PE class timec	48.0	54.3	15.5	26–90

All values are in minutes.

^aTotal HEALTHY activity time is the sum of the IAs, HRPAs and SD or GPAs.

^bOther class time was non-active time spent listening to instructions, transitioning between activities, sitting, resting and getting water.

^cTotal class time observed was the time between the beginning and end of class, not necessarily the scheduled PE class time.

Implementor participation

Each intervention semester, PACs led study trainings for the PE teachers. Ratings from PAC interviews on teacher engagement during these trainings showed that teachers were quite engaged through the course of the study (M = 4.5, SD = 0.5; rating scale of 1–5 with 5 being ‘very engaged’; Table III). Engagement during trainings related to teachers paying attention, asking questions and participating in activities. According to one PAC, ‘they loved the workshop trainings. It’s fun and it’is good content, so I think they get the benefits of both. And, it’s active—physical educators like to move and our trainings had lots of good content and lots of movement’. Based on PAC interview responses, the first training was very useful in preparing teachers for their role in the study and how to implement the HEALTHY PE curriculum. Over the course of the study, the trainings helped remind teachers of their role. According to one PAC, the middle semester training ‘served as a reminder of what HEALTHY expects from the PE teachers. They need to be reminded of the goals of the study, such as MVPA, IAs and how to keep students’ heart rates up’. Trainings also gave teachers an opportunity to discuss strategies for classroom management and how to keep student activity levels high. Toward the end of the study, a number of PACs reported that the trainings were ‘overkill’, especially for teachers who had been involved with the study from the beginning.

Table III.

Teacher and student engagement with the HEALTHY PE intervention

	Intervention semester
	Spring 2007	Fall 2007	Spring 2008	Fall 2008	Spring 2009	Overall
Implementor participation
Teacher engagement in study trainingsa	4.7 (0.5)	4.7 (0.5)	4.4 (0.7)	4.6 (0.5)	4.3 (0.5)	4.5 (0.5)
Teacher engagement in one-on-one interactions with PACsa	4.4 (0.8)	4.6 (0.5)	4.7 (0.6)	4.4 (0.6)	4.3 (0.5)	4.5 (0.6)
Teacher engagement in implementing the lessons as prescribedb	4.0 (1.0)	4.3 (1.0)	4.3 (0.7)	3.9 (0.7)	3.8 (0.7)	4.1 (0.8)
Teachers gave instructions (%)	97.5	99.0	100.0	98.6	100.0	98.9
Teachers were engaged during class (%)	92.5	99.0	98.4	98.6	97.6	96.9
Dose received
Students followed instructions (%)	93.6	98.0	97.4	96.8	97.2	96.4
Students were engaged and active during class (%)	91.8	96.5	95.8	93.0	94.8	94.2

Values are mean (SD) or percent.

^aBased on rating scale ranging from 1 (not engaged) to 5 (very engaged).

^bBased on rating scale ranging from 1 (low engagement) to 5 (high engagement).

PACs also regularly visited PE classes to observe and give teachers feedback on a one-on-one basis. PACs reported that teachers were quite engaged during these interactions (M = 4.5, SD = 0.6; rating scale of 1–5 with 5 being very engaged; Table III). According to PAC interview responses, the one-on-one interactions were most helpful to teachers when they focused on teacher strengths, and PACs believed that these in-person interactions helped sustain teacher motivation and involvement in implementing the intervention. One-on-one interactions in the early semesters focused more on increasing teacher self-efficacy to deliver the HEALTHY PE curriculum while other semesters focused on specific issues such as student discipline, use of the PE teaching assistant and classroom and time management.

Teachers were responsible for implementing the HEALTHY PE intervention, and PACs reported moderately high teacher engagement in implementing the lessons as prescribed (M = 4.1, SD = 0.8; rating scale of 1–5 with 5 being ‘high engagement’; Table III). In addition, classroom observational data showed that teachers delivered instructions in 98.9% of classes observed and were engaged during PE in 96.9% of the classes observed. A teacher was rated as engaged if they were actively supervising students, giving instructions or feedback, setting up or taking down equipment and enforcing rules for a majority of the class time. These values were slightly lower in the first semester but remained high over the course of the study. Overall, PAC interviews noted good teacher engagement in implementing the HEALTHY lessons. At some schools, there was a mix of motivated and disengaged teachers, which presented some challenges to PE implementation. According to one PAC, ‘several of the teachers are very professional, highly motivated and want to improve their skills while others need to be trained to transition from gym teacher to HEALTHY curriculum teacher’. Because many PE classes were team-taught, the highly engaged teachers would compensate for colleague disengagement. A number of PACs also mentioned that some of the veteran PE teachers were resistant to HEALTHY PE changes in terms of curriculum and teaching style.

Dose received

Observational data from PE classes were used to determine student engagement with the PE intervention. Over the course of the study, students followed instructions in 96.4% of classes observed, and students were engaged and active during PE in 94.2% of the classes observed (Table III). In order to meet the definition of being engaged/active during a PE class, a majority of students in a class had to be active for a majority of the class time. Responses from PAC interviews indicated that student engagement was tied to teacher engagement and class management. One PAC stated that ‘teacher enthusiasm was a huge part of student involvement in game activities. When the teachers are excited about doing the program, the kids feed off of that’. Another PAC observed that ‘when [student] participation went down, it wasn’t because of the units. It was because of class management’. PACs also noted that students tended to be more active during the basketball, fitness, football, handball, lacrosse, soccer, street hockey and swimming units and less active during the cooperative games, softball, tennis and volleyball units. Moreover, student participation and activity level was generally higher during activities that students liked. According to one PAC, ‘in the units the kids liked, activity levels were higher. In the units where students were unfamiliar or less skilled, they lost interest and were less active’. In general, PACs commented that students with fewer skills were less engaged in activity, and boys were generally more active than girls. During the initial intervention semesters, the students were excited about the new PE equipment and HEALTHY activities including IAs, HRPAs and GPAs. According to one PAC, ‘generally, the kids loved the increased activity levels and the newness of the equipment and curriculum’. As the intervention progressed, students became bored with the repetition of IAs and HRPAs and SD activities were less appealing. On the other hand, students continued to enjoy tag/chase games and FLOW (Fitness Lab on Wheels—a circuit training program) as HRPA activities as well as lead-up games and GPAs as part of SD/GPA. Some PACs also noted that some students seemed to ‘outgrow’ certain activities perhaps because they were bored or had a change in attitude. Finally, a number of PACs noted the importance of having ample PE equipment for student activity. Simply stated by one PAC, ‘if you really want heart rates up, you’ve gotta have lots of equipment’.

Barriers

Based on PE class observations, barriers that were significant enough to disrupt the functioning of the class occurred in almost 1 of 10 classes. The most common barrier was disruptive student behavior (60.0%) followed by teacher disengagement (17.0%) and school-wide interruptions (e.g. fire drill; 10.0%). Other barriers rarely observed were shortened PE classes (6.0%), student confusion over instructions (4.0%) and adverse weather (3.0%).

The most common barriers mentioned in the PAC interviews were large student to PE teacher ratios, limited PE facility space and poor classroom management. Other frequently mentioned barriers were disengaged and ‘burned-out teachers’ a lack of institutional support for PE and teacher absences due to coaching responsibilities, non-study trainings and ‘covering other classes’. One PAC recalled that ‘at times, the administration was pulling students out of PE to do some academic testing, and we had situations where students were working on writing assignments during PE because of pressures from administration’.

Discussion

Summary and interpretation of results

The process evaluation findings over the course of the study indicate a high level of intervention fidelity in terms of implementation of HEALTHY PE activities (87.6%). Therefore, the intervention was successful in implementing PE curriculum change whereby teachers delivered the HEALTHY PE lessons and activities. This level of fidelity of implementation is similar to the Pathways study in which PE was delivered 81.0% of the time as well as the LEAP study in which five of the seven PE elements were implemented in intervention schools [23, 24]. The successful implementation of the HEALTHY PE curriculum was fostered by securing teacher buy-in to the intervention. PE teachers were not only key stakeholders but also intervention implementors. In addition, intervention schools were able to schedule 225 min of PE class time every 10 school days with no significant problems despite contemporary trends to trim PE time from school schedules due to standardized testing constraints.

Data related to dose delivered show median and mean values of 29.0 and 33.1 min respectively of student activity time during PE, which is a substantial amount of active time. These durations are comparable to PE lesson lengths in similar studies of 30–33 min [22, 24]. The HEALTHY process evaluation data did not assess student activity time in terms of intensity, and therefore, it may have included light to vigorous physical activity. There was considerable variance in the length of PE activities suggesting inconsistency in the amount of physical activity students engaged in across PE classes and intervention schools, which may be attributed to varied PE class times and how teachers structured and managed class activities.

Data on implementor participation suggest that teachers were generally engaged during study trainings and consultation interactions with the PACs. Observational data also show that teachers were generally engaged in delivering the HEALTHY PE curriculum over the course of the study. The CATCH and Pathways studies also documented overall positive reactions from PE teachers concerning study-generated curriculum materials, activities that engage students and study-sponsored training and consultation [22, 24]. Alternatively, PACs noted that some teachers were initially resistant to HEALTHY PE changes at the beginning of the study and some remained disengaged over the 3 years of intervention implementation. Teacher disengagement may not have been specifically tied to HEALTHY as general teacher burnout is common in the US public education system [27].

Results relating to student engagement with the PE intervention, or dose received, show that the frequency of student engagement with the PE intervention was high during the study. Findings from PAC interviews indicated that student interest and engagement in PE activities dropped toward the end of the study. The Pathways study also reported that some teachers noticed that students were no longer challenged by certain games and other students seemed bored with activities in the final year of the intervention [24].

The frequency of barriers observed during PE classes was reasonably low. There were some differences and similarities between barriers noted during class observations and barriers reported by PACs. The most commonly observed barrier was student misbehavior followed by teacher disengagement. The most common barriers reported by PACs were large PE classes, limited gym space and poor classroom management. Teacher disengagement and a lack of institutional support for PE were other barriers mentioned in PAC interviews. Barriers noted by PACs included individual, classroom and school-level barriers, whereas observational data focused on barriers within PE classes. In comparison, barriers recorded in the Pathways study included lack of teacher motivation and commitment to delivering the intervention, lack of support from school administrators, using gym space and time for non-PE events (e.g. assemblies) and emphasis on other academic subjects over PE [28]. Many of these barriers may be due to the de-emphasis of PE in contemporary American schools with standardized testing requirements and school subject hierarchies.

In sum, these findings suggest that the HEALTHY PE intervention was generally implemented and received as intended. The primary study outcome results were recently reported [29], which documented significant differences between intervention and control schools in body mass index (BMI) z-score, the percentage of students with BMI at or above the 95th percentile, waist circumference at or above the 90th percentile and fasting insulin levels. However, no significant differences were found in the percentage of students with BMI at or above the 85th percentile, mean waist circumference or fasting glucose levels. The PE intervention was only one of the four HEALTHY components, and thus, it is beyond the scope of this article to draw conclusions about the effect of the PE intervention on study outcomes. Nonetheless, according to a Cochrane review of school-based physical activity interventions, these interventions are effective at increasing durations of physical activity, reducing blood cholesterol levels and increasing aerobic capacity [30]. On the other hand, these interventions are often not effective at reducing BMI or systolic and diastolic blood pressure.

Strengths and limitations

One of the strengths of the HEALTHY process evaluation was the mixed methods design. Such a design was helpful in understanding the implementation of an intervention program that was complex and multidimensional in terms of the number of deliverable intervention components and subcomponents, the number of different study group and school members involved and the variety of contexts and settings of the HEALTHY intervention. Standardizing process evaluation procedures and instruments across sites was also a strength. Research assistants and interviewers who collected process evaluation data were periodically trained at study group meetings on procedures for collecting, compiling and transferring data. Also, the amount of quantitative and qualitative process evaluation data elicited was an asset as this lends to an accurate assessment of intervention implementation. Finally, another strength was the means of rapidly entering and analyzing process evaluation data in order to provide feedback to interventionists and other study group members between intervention semesters.

On the other hand, there are a number of limitations of the HEALTHY process evaluation. First, data were not directly collected from students and PE teachers over the course of the study on the PE intervention. Although observations and interventionist interviews were valuable and minimized respondent burden for school members, collecting data from students and teachers would have provided an even more comprehensive assessment of PE intervention implementation. Process data was collected from students concerning the communications intervention component but not the PE intervention. And, teachers were interviewed at the conclusion of the study; however, these interviews focused primarily on intervention efficacy, efficiency and recommendations for the future of HEALTHY. Another limitation is that there may have been some social desirability response bias. Some teachers and students may have behaved differently during PE class observations than they would if a study observer was not present. Additionally, responses from PAC interviews might have been biased because they were reporting on an intervention, which they were responsible for successfully coordinating. Another limitation was that the process evaluation data did not assess the proportion of time students spent in MVPA, which was the primary objective of the PE intervention. Heart rate monitoring data were collected on three randomly selected consented students per PE class at baseline (fall 2006), interim (spring 2008) and end of study (spring 2009); however, these were outcome and not process evaluation data, and these data may not be representative measurements of MVPA durations during PE classes. A report of the heart rate monitoring data for both intervention and control schools is in preparation. Finally, process evaluation data were not collected in the control schools or in the intervention schools prior to implementation; thus, we have no data on to compare or contextualize other PE programs with the HEALTHY PE intervention.

Implications for practice and evaluation

These process evaluation findings have several implications for school-based PE programs. First, teacher resistance and disengagement may have been related to their limited role in the development of the PE intervention. It may be beneficial for future physical activity interventions to collaborate with interested PE teachers in intervention development as opposed to simply providing implementors with a manualized program created by school outsiders. Second, trainings for teachers on implementing activities aimed at maximizing MVPA may need to significantly focus on teacher skills and attitudes regarding classroom management as this was a prominent barrier. Also, including school administrators in trainings or providing some form of professional development for administrators on student health and physical activity to create more buy-in to PE programs may be beneficial. We also recommend that teacher training and consultation should be interactive, motivational and strengths-based in nature. Third, when implementing structured PE programs, it is important to focus on both activity completion and increasing time spent in MVPA. Concerning the HEALTHY PE activities, IAs were helpful in getting students active at the very beginning of class, and certain HRPAs (i.e. tag/chase games and FLOW) were effective at eliciting student activity, although repetition of these activities resulted in decreased student activity. Also, lead-up games and GPAs elicited higher student activity levels. These findings suggest that an effective PE curriculum should include activities that are fresh, relevant and interesting to students as they progress through adolescence, and caution should be taken in the repetition of activities. Another implication concerns resources for PE. Having ample PE equipment was key to maximizing student activity, and large student to teacher ratios detracted from student activity. Some PE classes had two teachers for 125 students. Finally, if the primary goal of PE is to increase student activity levels, we recommend implementing units that elicit higher activity levels (basketball, fitness, football, handball, lacrosse, soccer, street hockey and swimming) over those that result in lower levels (cooperative games, softball, tennis and volleyball).

There are also implications for the evaluation of school-based PE interventions. First, heart rate monitoring data collected on a larger sample of students each intervention semester would have been useful in assessing PE intervention fidelity in terms of time spent in MVPA. In addition, class observations could have assessed students’ activity level during each HEALTHY PE activity delivered in terms of slow, moderate and fast movement through space. Second, PE teaching assistants could have recorded PE units and lessons delivered as an assessment of dose delivered as well as student receptivity and engagement concerning PE units and activities in documentation logs. Third, although PE teachers were not periodically interviewed during the course of the study regarding implementation in order to minimize respondent burden, a short survey may have been helpful in capturing teachers’ thoughts and concerns regarding delivery of the HEALTHY PE intervention. Finally, having specific goals and objectives for intervention implementation are helpful in forming valid assessments of intervention fidelity. Certain HEALTHY intervention components had more specific goals and objectives outlined in the study protocol than others, which provided the process evaluation committee more or less specific parameters in evaluating fidelity for each component. We hope that these findings will be useful in the design, implementation and evaluation of effective school-based PE programs.

Funding

National Institute of Diabetes, Digestive and Kidney Diseases of the National Institutes of Health (U01-DK61230, U01-DK61249, U01-DK61231, and U01-DK61223).

Conflict of interest statement

None declared.

Appendix.

The following individuals and institutions constitute the HEALTHY Study Group (* indicates principal investigator or director): Study Chair: Childrens Hospital Los Angeles: F. R. Kaufman; Field Centers: Baylor College of Medicine: T. Baranowski*, L. Adams, J. Baranowski, A. Canada, K. T. Carter, K. W. Cullen, M. H. Dobbins, R. Jago, A. Oceguera, A. X. Rodriguez, C. Speich, L. T. Tatum, D. Thompson, M.A. White, C. G. Williams; Oregon Health & Science University: L. Goldberg*, D. Cusimano, L. DeBar, D. Elliot, H. M. Grund, S. McCormick, E. Moe, J. B. Roullet, D. Stadler; Temple University: G. Foster* (Steering Committee Chair), J. Brown, B. Creighton, M. Faith, E. G. Ford, H. Glick, S. Kumanyika, J. Nachmani, L. Rosen, S. Sherman, S. Solomon, A. Virus, S. Volpe, S. Willi; University of California at Irvine: D. Cooper*, S. Bassin, S. Bruecker, D. Ford, P. Galassetti, S. Greenfield, J. Hartstein, M. Krause, N. Opgrand, Y. Rodriguez, M. Schneider; University of North Carolina at Chapel Hill: J. Harrell*, A. Anderson, T. Blackshear, J. Buse, J. Caveness, A. Gerstel, C. Giles, W. Hall, A. Jessup, P. Kennel, R. McMurray, A-M. Siega-Riz, M. Smith, A. Steckler, A. Zeveloff; University of Pittsburgh: M. D. Marcus*, M. Carter, S. Clayton, B. Gillis, K. Hindes, J. Jakicic, R. Meehan, R. Noll, J. Vanucci, E. Venditti; University of Texas Health Science Center at San Antonio: R. Treviño*, A. Garcia, D. Hale, A. Hernandez, I. Hernandez, C. Mobley, T. Murray, J. Stavinoha, K. Surapiboonchai, Z. Yin; Coordinating Center: George Washington University: K. Hirst*, K. Drews, S. Edelstein, L. El ghormli, S. Firrell, M. Huang, P. Kolinjivadi, S. Mazzuto, T. Pham, A. Wheeler; Project Office: National Institute of Diabetes and Digestive and Kidney Diseases: B. Linder*, C. Hunter, M. Staten; Central Biochemistry Laboratory: University of Washington Northwest Lipid Metabolism and Diabetes Research Laboratories: S. M. Marcovina*.