Impact of Implementation and Conduct of the HEALTHY Primary Prevention Trial on Student Performance

Abstract

Purpose

To determine whether a school-wide intervention program to reduce risk factors for type 2 diabetes (T2D) affected student achievement, rates of disciplinary actions, and attendance rates.

Design

The HEALTHY primary prevention trial was designed to evaluate a comprehensive school-based intervention to reduce factors for T2D, especially overweight and obesity. Students were followed up from beginning of sixth grade (Fall 2006) through end of eighth grade (Spring 2009).

Setting

Forty-two middle schools at seven U.S. sites.

Subjects

Schools were randomized in equal numbers at each site to intervention (21 schools, 2307 students) or control (21 schools, 2296 students).

Intervention

An integrated school-wide program that focused on (1) foods and beverages, (2) physical education, (3) classroom-based behavior change and education, and (4) social marketing communication and promotional campaigns.

Measures

Aggregate (grade- and school-wide) test performance (passing rate), attendance, and referrals for disciplinary actions.

Analysis

Descriptive statistics and tests of intervention versus control using mixed linear models methods to adjust for the clustering of students within schools.

Results

There were no differences between intervention and control schools in test performance for mathematics (p = .7835) or reading (p = .6387), attendance (p = .5819), or referrals for disciplinary action (p = .8671).

Conclusion

The comprehensive HEALTHY intervention and associated research procedures did not negatively impact student achievement test scores, attendance, or referrals for disciplinary action.

PURPOSE

Rates of type 2 diabetes (T2D) in youth have increased dramatically.¹ Consequently, efforts to mitigate risk factors for T2D, especially overweight and obesity, are a public health priority. Schools offer ideal venues for opportunities to optimize nutrition and activity,^2–4 but access may be limited by concerns about possible deleterious effects of comprehensive interventions on student achievement,⁵ particularly standardized test performance.⁶ Finally, there may be concerns that noncurricular activities or the presence of nonschool research staff could impact student discipline or attendance.

HEALTHY was funded by the National Institute of Diabetes and Digestive and Kidney Diseases to evaluate a comprehensive intervention to reduce risk factors for T2D. Results documented equivalent reductions in the combined prevalence of overweight and obesity in control (−4.1%) and intervention (−4.5%) schools at the end of the 2.5-year program. However, intervention schools had greater decreases in obesity (−5.5% vs. −3.8% in control, p = .05) and rates of elevated waist circumference (−8.1% vs. −5.9%, p = .04), and smaller increases in fasting insulin (3.8 vs. 4.0 µU/dL, p = .04).⁷ In this article, we examine whether the intervention and on-site research procedures had a deleterious effect on standardized test performance, attendance, or reported disciplinary actions as a proxy for student behavior.

METHODS

Design and Sample

HEALTHY was a 3-year cluster-randomized, controlled primary prevention trial. Details of the HEALTHY protocol have been described.⁸ In brief, 42 U.S. middle schools with at least 50% of students eligible for free or reduced-price lunch, or belonging to a minority group, were recruited by the seven participating centers. The study was approved by institutional review boards at each site. Parent informed consent and child assent were obtained to permit data collection from students. Schools were randomized by site to intervention (3 schools per site) or control (3 schools per site) conditions.

Intervention

Details of HEALTHY have been published that provide in-depth information regarding recruitment and retention,⁹ each of the intervention components,^10–13 and process evaluation.¹⁴ Intervention schools were provided 2.5 years of a comprehensive program that targeted nutrition, physical activity, and the social environment. Much of the intervention was delivered by school staff with materials, equipment, training, and support provided by the study. Participation of control schools was limited to recruitment and data collection.

Several components of the HEALTHY intervention were implemented during instructional time. The physical education (PE) intervention component included lesson plans to ensure completion of at least 150 minutes of moderate to vigorous physical activity (MVPA) during a minimum of 225 minutes of total class time every 10 days of school; this level of MVPA was selected to influence fitness and body weight.¹¹ The behavior change intervention component was delivered by teachers in classrooms designated by participating schools. Weekly 30-minute sessions (Fun Learning Activities for Student Health [FLASH]) were designed to increase health knowledge and motivation, and capitalized on peer influence to enhance health behaviors. FLASH sessions were presented for 8 to 10 weeks for each of the five intervention semesters. The social marketing component used study and student-designed materials (students used instructional time for creating some of these materials). Promotional events and activities were held during lunch or at school assemblies.

Measures

Data to document school characteristics and the stability of the school environment, such as enrollment, number of days in the school year, student body racial/ethnic breakdown, supplementary academic programs, and changes in key administrative personnel, were extracted from school records annually. Similarly, HEALTHY captured academic performance (percentage passing state accountability math and reading tests) for each of the sixth, seventh, and eighth grades in the school year before the start of the study (2005–2006) and during the 3 years of the study. Full-day absences and disciplinary actions for the student body and the cohort grade were collected by using school-reported definitions. Annual absence rate per student was computed as the total number of full-day absences recorded by the school during the school year divided by the total number of students. Rate of disciplinary actions per student was computed as the total number of disciplinary actions recorded by the school during the school year divided by the total number of students.

Data Analysis

Descriptive data are presented as means, ranges, frequencies, and percentages. Mixed linear models were used to test the effect of intervention versus control both across and within year of study and student grade. The analytic method adjusted for sources of variability between school and also within school (between students clustered in a school).

RESULTS

School Characteristics

Control and intervention schools were equivalent in terms of overall enrollment (average 863 in control and 873 in intervention), mandated number of school days (average 179 days in both intervention and control), proportion of students who qualified for free or reduced meals (74% control vs. 79% intervention), and Hispanic and/or Black students (70% control vs. 77% intervention). Across the 3 study years, a similar number of control and intervention schools initiated academic programs (six versus four) or programs to address underachievement (15 versus 14). Approximately half the schools (10 control and 10 intervention) experienced a change in district superintendent, and more control than intervention schools changed school principals (nine versus five). Overall, school characteristics and administrative infrastructures were similar in intervention and control schools.

Standard Test Results

Table 1 shows the mean percentage of students reported as passing (state standard criteria) for math and reading tests administered state-wide in the year immediately before and in the 3 years during the study. Because school accountability measures are mandated at the state level, changes in testing standards or procedures from one year to the next impacted all schools in the state, so the control and intervention schools at each study site were in the same state and affected equally. Only one control school reported administering the same tests with more stringent passing criteria between the 2007–2008 and 2008–2009 school years.

Table 1.

Mean Percentage Passing State-Wide Math and Reading Accountability Tests in Control and Intervention Schools by School Year^*

Test and Grade	Control				Intervention
	2005– 2006 (Baseline), %	2006– 2007 (Sixth Grade), %	2007– 2008 (Seventh Grade), %	2008– 2009 (Eighth Grade), %	2005– 2006 (Baseline), %	2006– 2007 (Sixth Grade), %	2007– 2008 (Seventh Grade), %	2008– 2009 (Eighth Grade), %
Math
Sixth grade	57.4	57.8	60.2	62.5	54.2	54.2	56.9	58.3
Seventh grade	52.6	54.0	56.3	63.6	51.2	54.4	58.8	61.9
Eighth grade	49.5	51.5	58.4	59.5	51.3	52.4	60.7	58.9
Reading
Sixth grade	66.0	65.2	62.2	66.4	60.9	59.6	55.8	57.7
Seventh grade	62.5	64.8	59.0	64.5	59.9	62.2	60.2	60.1
Eighth grade	64.6	67.1	65.4	69.8	62.2	64.5	63.9	66.6

^*The 2005–2006 school year is considered “baseline,” i.e., before participation in the HEALTHY study. The bold values identify the study cohort grade across the 3 school years of the study.

There was no statistically significant difference between control and intervention schools in math (p = .7835) or reading (p = .6387). For the most part, percentage passing math tests increased year by year. For reading tests, in all cases but one class in an intervention school, percentage passing at end of study (2008–2009) was higher than the baseline value.

School Attendance

Table 2 shows mean annual rates of full-day absences. From beginning to end of study, annual absence rates of the HEALTHY cohort grade increased in control (from 8.7% to 9.9%) and intervention (from 8.9% to 10.2%) schools, but changes were not statistically significant (p = .5819). Absence rates for students in the cohort grade were lower than school-wide in sixth grade, but in general, rates across intervention and control and across year in study were stable, in the 9% to 10% range.

Table 2.

Mean Per Student Annual Rates of Full-Day Absence and Disciplinary Action

	Control		Intervention
	Cohort Grade	Entire School	Cohort Grade	Entire School
Absence rate per student*
2006–2007 (cohort in sixth grade)	8.7	10.2	8.9	9.6
2007–2008 (cohort in seventh grade)	9.1	9.4	9.6	9.7
2008–2009 (cohort in eighth grade)	9.9	9.4	10.2	9.6
Disciplinary action rate per student†
2006–2007 (cohort in sixth grade)	1.8	1.7	1.9	1.8
2007–2008 (cohort in seventh grade)	1.3	1.3	1.4	1.4
2008–2009 (cohort in eighth grade)	1.3	1.3	1.2	1.3

^*Computed as total number of full-day absences recorded by the school during the school year divided by total number of students.

^†Computed as total number of disciplinary actions recorded by the school during the school year divided by total number of students.

Disciplinary Actions

Table 2 also presents annual disciplinary actions. Some schools recorded only suspensions and expulsions, while others also recorded principal referrals or written complaints. Observed rates were highly variable, perhaps reflecting changes in definitions or recording requirements, but there were no apparent systematic differences in practices between intervention and control schools. There was a significant drop in annual rates of disciplinary action from sixth to eighth grades in both the study cohort (p = .0394) and school-wide population (p = .0201), but no difference between intervention and control schools (p = .8671).

DISCUSSION

Summary

During the planning phase of the HEALTHY study, stakeholders from the school community expressed concerns about the numerous demands on school resources for implementation of the program. The intervention was a comprehensive multiyear primary prevention program that relied on teachers and school staff to implement significant parts of the intervention, used instructional time to provide the educational components of the intervention, and affected PE class, the food environment, and the social milieu. Further, intervention schools were required to accommodate the presence of study staff and observers.

Available data document positive^15–18 or neutral effects of health promotion programs on measures of student health and academic performance,¹⁹ but there is little information about the possibility of deleterious effects. HEALTHY provided an opportunity to examine relative performance on academic achievement tests, school attendance, and school-reported disciplinary actions. Despite the considerable demands placed on schools by the HEALTHY intervention, the present data provide no evidence that incorporation of a comprehensive health promotion program had adverse effects on measures of concern to the schools. Specifically, there were no differences between intervention and control schools in yearly reading and math accountability test results or measures of student attendance or disciplinary actions.

Limitations

School reporting procedures were not standardized and may represent variable accuracy, completeness, and consistency. However, we applied similar procedures to both control and intervention schools within each district or site, permitting comparison of trends and measures of central tendency. Future research should include prospectively defined measures of adverse as well as positive effects of health promotion programs to provide a more complete picture of potential harms as well as benefits of implementing school-based programs.

Significance.

HEALTHY provides evidence that middle schools can undergo major environmental changes and the introduction of health programs delivered during classroom time without negatively affecting critical performance metrics. This evidence may contribute to our ability to test the important hypothesis that prolonged exposure to a healthier lifestyle throughout the school day leads to improved mental, physical, and behavioral performance.

So What? Implications for Health Promotion Practitioners and Researchers.

What is already known on this topic?

Available data indicate that health promotion programs are associated with positive effects on student health and academic performance, but little is known about potential negative effects of such programs.

What does this article add?

Data from the HEALTHY study, a middle school–based health promotion initiative to reduce risk for type 2 diabetes conducted in 42 schools across seven U.S. sites, provide evidence that neither research procedures nor implementation of a comprehensive program using school staff and instructional time were associated with negative effects on standardized test performance, attendance, or disciplinary actions.

What are the implications for health promotion practice or research?

Evidence from the HEALTHY study demonstrates that health promotion activities and research procedures need not affect critical performance metrics. Future school-based research using more detailed analyses of possible negative effects in addition to benefits of intervention is indicated.