Effectiveness and Spillover of an After-School Health Promotion Program for Hispanic Elementary School Children

Abstract

Objectives. We evaluated the effectiveness and spillover of an after-school health education and physical activity program among Hispanic elementary school children.

Methods. In fall 2008, students in third through fifth grades in 6 schools in El Paso, Texas (n = 901), were randomized to intervention (n = 292 participants) or control (n = 354) classrooms (4 unknown). Intervention classrooms also contained a spillover group (n = 251) that did not join the after-school program but that completed measurements and surveys. The intervention was a 12-week culturally tailored after-school program meeting twice a week. Four-month outcomes were body mass index, aerobic capacity, and dietary intentions and knowledge. We calculated intervention exposure as the proportion of after-school participants per classroom.

Results. Intervention exposure predicted lower body mass index (P = .045), higher aerobic capacity (P = .012), and greater intentions to eat healthy (P = .046) for the classroom at follow-up. Intervention effectiveness increased with increasing proportions of intervention participants in a classroom. Nonparticipants who had classroom contact with program participants experienced health improvements that could reduce their risk of obesity.

Conclusions. Spillover of beneficial intervention effects to nonparticipants is a valuable public health benefit and should be part of program impact assessments.

Over the past decade, obesity rates have increased dramatically among children and adults in the United States.¹ Childhood excess weight is a strong predictor of adulthood excess weight,² and is associated with increased risk of common chronic diseases such as diabetes and cardiovascular disease.³ Hispanic children and adolescents have higher incidences of overweight⁴ and type 2 diabetes⁵ than the national average and have lower rates of physical activity.⁴ Without effective interventions, current trends in obesity will likely continue unabated. A meta-analysis of 64 intervention studies of childhood obesity found that few included appreciable numbers of Hispanic participants,⁶ and none of these resulted in improvements in body mass index (BMI).⁶

The majority of interventions examined in the meta-analysis were school-based programs conducted during the school day.⁶ Schools have several well-known advantages for youth-targeted behavior change interventions. Almost all children attend school; schools have appropriate facilities for intervention activities and strong credibility within the community. However, school districts face increasing pressures for students to perform well on standardized examinations and are unable to release class time for health promotion activities. After-school activities provide a viable alternative for health promotion. An estimated 8.4 million youths in the United States participate in some form of after-school activity, and an additional 18.5 million families reported they would be interested in after-school activities if they were available.⁷ Hispanic children are more likely than the national average to participate in after-school programs (21% vs 15% of school-aged children).⁸

Despite the apparent need for and interest in after-school activities, few after-school health promotion programs have been evaluated and published.^9–18 In none of these studies were a majority of participants Hispanic. Therefore, we developed, implemented, and evaluated a culturally tailored health education and physical activity after-school program for a population of predominantly Hispanic elementary school children.

METHODS

Following a pilot study described elsewhere,¹⁹ we approached 9 schools in El Paso, Texas, in July and August 2008 by contacting the principal and the physical education (PE) teachers; 6 (67%) schools agreed to participate. Selection criteria were school location (for logistical purposes, half of those chosen were located within 5 miles of the University of Texas at El Paso campus), size, socioeconomic status, and percentage of children with limited English proficiency. Schools that did not participate either had a negative previous experience with a different after-school program (n = 1) or did not respond to several requests to arrange a meeting with the principal or PE teachers (n = 2). On average, nonparticipating schools did not differ substantially from participating schools in school characteristics.

We recruited students in third, fourth, and fifth grades by making announcements and passing out consent forms during PE classes. Children were eligible to participate if they were enrolled in 1 of the target grades and had no condition that would endanger their own or others' safety. For after-school participation, the parental consent form required a description of means of transportation home, emergency contact information, and health insurance (Medicaid was acceptable). If parents consented, we asked children for their assent before allowing them to participate. We composed the parental consent form (in English and Spanish) at an eighth-grade reading level and the children's assent form at a third-grade reading level.

Study Design and Sample

Randomization of the intervention occurred on a classroom level. Within each school, we randomly assigned 4 to 6 classrooms per grade level to intervention activities (2 or 3 classrooms) or to the control condition (2 or 3 classrooms). In the 6 schools, we randomized 85 classrooms (intervention, n = 44; control, n = 41; Figure 1). We assessed physical fitness and dietary indicators at baseline and after the semester-long intervention. Although the total number of students per school participating in the study ranged from 93 to 207 (mean = 150), we limited the number of students allowed in the after-school sessions at each school to approximately 50. The student-to-teacher ratio in the after-school sessions was no higher than approximately 17 to 1, well below the average maximum recommended for elementary school PE (28 to 1).²⁰

FIGURE 1.

After-school health education and physical activity program culturally tailored to Mexican American third- to fifth-grade students: El Paso, TX, 2008.

Children who were randomized to the intervention classrooms but declined an invitation to participate in the after-school program were asked to assent, after their parents gave consent, to be surveyed and measured. Intervention classrooms therefore had students who agreed to join the after-school program and others who did not participate but agreed to be surveyed and measured—the spillover group. Although we did not systematically assess the reasons students declined to join the after-school program, anecdotal reports mentioned time conflicts of our program with other activities, lack of interest, lack of transportation home after the program, and parents' refusal to allow children to participate.

After-School Intervention

The after-school program ran twice weekly for 12 weeks (September through November 2008), for a total of 24 sessions at each school. Each session took place in the schoolyard or in the multipurpose room and comprised a 20- to 30-minute health education component followed by 45 to 60 minutes of physical activity. Because resources were limited and because of previous reports of teacher turnover in daily after-school interventions,⁹ we decided in collaboration with school staff to hold the intervention twice a week for the duration of a regular PE session.

We selected a bilingual health education curriculum, Bienestar (well-being),^21–23 that is culturally targeted to Mexican Americans. The curriculum is grounded in social cognitive theory²⁴ and includes modules on healthy eating, exercise, diabetes, and self-esteem. It has been shown to increase appropriate dietary intake and physical fitness among students in third through fifth grades.^21–23 The intervention's 24 sessions at each school ensured that students would be exposed to all 16 modules of the curriculum. We hired bilingual community health workers through the human resources department of the University of Texas at El Paso to teach the health education curriculum.

We adapted the physical activity component of the after-school program from the Coordinated Approach To Children's Health^25–27 physical activity curriculum. The activities for the program emphasized cardiovascular activity and aerobic recreational games. To teach the physical activity component of the program, we recruited senior-level student teachers from the University of Texas at El Paso Physical Education Teacher Education program through announcements in several upper-level courses required for the PE teaching certification. During their PE teachers' training,²⁸ all student teachers had been exposed to the curriculum we used.

Children received small incentives for their continued participation (e.g., a small football or pedometer) every second week. Members of the control and spillover groups received fourth-grade health workbooks and incentives at pretest and follow-up measurements, but they did not attend the after-school sessions.

Measures

Measurements were completed just before and after the intervention, in August and December 2008. Project staff collected these measurements in collaboration with each school's PE teachers during PE classes. Physical fitness measurements consisted of 2 indicators that were part of the state-mandated fitness assessments: BMI (defined as weight in kg divided by height in m²) and aerobic capacity.²⁹ Nutrition indicators were dietary intentions and dietary knowledge, assessed through a self-administered survey that also covered demographic variables such as age, gender, grade level, and self-reported ethnicity. Many students were apparently not aware of their ethnicity because more than half marked don't know or other. Consequently, we did not include self-reported ethnicity in any of the analyses.

We measured height and weight with a Tanita-BF 215 portable electronic scale with height rod (Tanita Corporation of America, Arlington Heights, IL). We calculated BMI values and converted them to a population percentile for each participant according to the participant's age and gender. We measured aerobic capacity with the Progressive Aerobic Cardiovascular Endurance Run (PACER) test, which requires participants to run up and down a 20-meter court.²⁹ At each side of the court, a beep sounds to signal the student to turn around and run back. The test increases in speed every minute and is completed when a student fails to reach the other side in time for the signal for the second time.

We adapted nutrition indicators, which consisted of self-reported dietary intentions (8 items) and dietary knowledge (10 items) from the previously validated After School Student Questionnaire,⁹ derived from the Health Behavior Questionnaire²⁶ and the School-Based Nutrition Monitoring Student Questionnaire.³⁰ The translation department of the University of Texas at El Paso developed a Spanish version of the survey; 23.5% of the participants chose the Spanish version. The items consisted of 2 answer options accompanied by a graphical representation of the food choice: an unhealthier option and a healthier option. An example food intentions item was, “If you were at the movies, which one would you pick? a) popcorn with butter or b) popcorn without butter.” Internal consistency for the intentions to eat healthy scale was low at baseline (α = 0.59) but higher at follow-up (α = 0.69). Internal consistency for health knowledge was better (baseline α = 0.69; follow-up α = 0.74).

Analyses

According to Optimal Design Software (Survey Research Center, Institute of Social Research, Ann Arbor, MI),³¹ the target sample size (6 schools, ≥ 80 participants/school, for a total of 480 participants) enabled power to detect an effect size of a Cohen's d of 0.30, comparable with effect sizes observed in a previous study that used the same dietary outcome scales.⁹ Although the mean weighted effect size for BMI change found in a meta-analysis of programs addressing childhood obesity was smaller, effect sizes among programs that had positive effects exceeded a d of 0.30.⁶

We conducted all primary analyses with HLM 6.08 (Scientific Software International, Lincolnwood, IL)³² with a 3-level design: students (level 1) were nested within their classrooms (level 2), and classrooms were nested within their schools (level 3). Three-level multilevel analyses involved fitting a series of conditional random-effects regression models with maximum likelihood estimation.

Because randomization to intervention condition occurred on a classroom level, intervention exposure was a classroom-level variable. However, the study design presented the likelihood for spillover or ripple effects arising from the mixture within the same classrooms of students who participated in the after-school program and spillover students, who did not.

To better understand the effects of having both groups of students in the same classrooms and to more accurately reflect the average exposure to the intervention in a given classroom, we constructed a variable termed intervention exposure indicator. Classrooms containing only control group participants received a 0 for intervention exposure, and classrooms with only intervention participants received a 1. Classrooms with both after-school participants and spillover participants received a value indicating the proportion of after-school participants compared with the total number of study participants in that classroom. For example, if a classroom had 8 after-school participants and 2 nonparticipants (spillover students), its intervention exposure indicator variable was 0.80. Classroom size was the only other classroom-level indicator available.

We adopted this method in accordance with social network and diffusion-of-innovation theories.³³ In these approaches, the likelihood of adopting a behavior can be defined as the proportion of the network or social group (e.g., a classroom) that has adopted the behavior (e.g., participating in an after-school health program).³⁴

School-level variables consisted of school socioeconomic status (the percentage of socioeconomically disadvantaged students in the school) and language (the percentage of children with limited English proficiency).³⁵

RESULTS

All participating schools were similar in size (80–120 students/grade). The average percentage of children with low socioeconomic status was 72%, similar to the school district's average (70%). The average percentage of children with limited English proficiency across participating schools (47%) was higher than the district average (30%). Only the 2 schools with the highest socioeconomic status³⁵ did not reach the maximum number of allowed participants in the after-school project (50/school).

Participation

Approximately 1720 children were approached (Figure 1), of whom 901 (52.4%) participated. Of the participants, 354 (39.3%) were in classrooms assigned to the control condition. A total of 292 (32.4%) children participated in the after-school intervention, and 251 (27.9%) were in the spillover group whose parents gave consent for them to be surveyed. Allocation was unknown for 4 participants. Among the 44 intervention classrooms, the participation rate ranged from 0.22 (i.e., 22% of the children in that classroom were after-school participants, and 78% were not) to 1.00 (all children in that classroom were after-school participants, and none were in the spillover group). On average, intervention exposure was 54.9% (SD = 21.2%) in intervention classrooms.

At the 4-month follow-up, study retention was high, with 804 (89.6%) of the participants completing at least some of the follow-up measures. More after-school (n = 50) than control (n = 28) and spillover (n = 15) participants did not complete follow-up measurements (Figure 1). To assess whether certain characteristics were associated with increased likelihood of dropping out, we compared after-school participants who did not participate with those who did participate in the follow-up. In bivariate analyses, we detected no significant baseline differences in demographic characteristics or any of the dependent variables between dropouts and those who completed both baseline and follow-up measurements.

Baseline and Follow-Up Values

The mean age of the participants was 9.2 years ±1 year, and 53.0% were boys. According to Centers for Disease Control and Prevention growth charts,³⁶ 25.6% of participants were overweight (BMI > 95th percentile for their age and gender); another 21.6% were at risk for overweight (BMI > 85th percentile). At baseline, the intervention, control, and spillover groups had no significant differences on any of the demographic or outcome variables, although scores on the intentions to eat healthy scale were marginally higher among intervention than spillover participants (b = 0.45; t=1.740; P=.083).

Taking into account the nested structure of the data, we detected small reductions in BMI percentile from baseline to follow-up among all participants (Table 1). The BMI percentile reduction was largest (2.8%; P = .015) among participants in the after-school program. We observed smaller reductions that were not statistically significant among the spillover (2.0%; P = .085) and control (1.4%; P = .249) groups.

TABLE 1.

Baseline and Follow-Up Health Indicators Among Third- to Fifth-Grade Participants and Nonparticipants in an After-School Health Promotion Program Culturally Tailored to Mexican Americans: El Paso, TX, 2008

	Control Group (n = 326)	P	Spillover Groupa (n = 236)	P	Intervention Group (n = 242)	P
Boys, %	55.4		48.6		54.1
Age at baseline, y, mean (SD)	9.10 (1.08)		9.27 (0.84)		9.24 (0.87)
At risk for overweight,b %	20.0		21.5		19.4
Overweight,c %	25.6		22.3		28.1
BMI (kg/m2)
Baseline, mean (SD)	19.99 (4.37)		19.54 (4.19)		20.25 (4.43)
Follow-up, mean (SD)	19.87 (4.26)		19.43 (4.10)		20.08 (4.38)
Change	−0.12	.328	−0.11	.503	–0.17	.029
BMI percentile
Baseline, mean (SD)	73.19 (27.48)		69.12 (30.36)		73.26 (27.05)
Follow-up, mean (SD)	71.79 (27.34)		67.12 (30.95)		70.45 (28.46)
Change	−1.40	.249	−2.00	.085	−2.81	.015
Students who reduced risk,d No.	22		16		17
Students who increased risk, No.	10		8		5
Aerobic capacity,e no. laps
Baseline, mean (SD)	22.9 (12.3)		20.6 (11.5)		22.3 (12.7)
Follow-up, mean (SD)	25.2 (13.8)		22.9 (12.4)		25.0 (15.5)
Change	+2.3	.054	+2.3	.061	+2.7	.058
Dietary intentions,f no. correct
Baseline, mean (SD)	4.23 (1.79)		4.02 (2.02)		4.49 (2.09)
Follow-up, mean (SD)	4.40 (2.09)		4.31 (1.95)		4.93 (2.38)
Change	+0.17	.487	+0.29	.368	+0.44	.068
Health knowledge,g no. correct
Baseline, mean (SD)	6.39 (2.34)		6.40 (2.49)		6.68 (2.55)
Follow-up, mean (SD)	6.39 (2.50)		6.86 (2.65)		6.92 (2.56)
Change	0.00	.825	+0.46	.157	+0.34	.327

Note. BMI = body mass index. Means took into account nested data structure but were not adjusted for covariates.

^aSpillover group represents those who were invited but did not join the after-school program and consented to be measured and surveyed.

^bDefined as > 85th percentile for age and gender in national population.

^cDefined as > 95th percentile for age and gender in national population.

^dDefined as changing BMI percentile score from at risk for overweight (> 85th percentile) to healthy weight (< 85th percentile) or from overweight (> 95th percentile) to at risk.

^eMeasured by the Progressive Aerobic Cardiovascular Endurance Run test, no. of laps run on a 20-m court.

^fNumber of correct answers on an 8-item scale.

^gNumber of correct answers on a 10-item scale.

The intervention exposure indicator variable was a significant predictor of posttest BMI percentile (b = −2.98; t=−2.030; P = .045; Table 2), indicating that a proportionally larger group of after-school participants in a classroom was associated with lower BMI percentile at follow-up for all tested students in that classroom. We used the coefficient of the intervention exposure indicator to calculate the estimated BMI percentile difference between classrooms varying in proportion of after-school participants. For example, students in a classroom with 100% intervention group participants had a 2.98% lower mean BMI percentile at follow-up than did students in a classroom with only control participants (intervention exposure indicator of 0; 2.98% × [1.00 − 0.00]). Students in a classroom with 70% after-school participants had a 1.19% lower mean BMI percentile at follow-up than did students in a classroom with 30% after-school participants (2.98 × [0.70 − 0.30]).

TABLE 2.

Predictors of Health Indicators Among Third- to Fifth-Grade Participants in an After-School Health Promotion Program Culturally Tailored to Mexican Americans: El Paso, TX, 2008

	BMI Percentile, b (SE)	Aerobic Capacity,a b (SE)	Dietary Intentions,b b (SE)	Dietary Knowledge,c b (SE)
Intercept (fixed effect)	3.22 (2.09)	4.15 (3.22)	4.81 (0.50)	4.45 (1.12)
Individual-level effects
Age, y	−0.79* (0.37)	0.40 (0.39)	0.05 (0.10)	0.02 (0.11)
Girls	−0.24 (0.73)	−2.15** (0.79)	−0.03 (0.18)	0.48* (0.21)
Pretest score	0.97** (0.01)	0.81** (0.03)	0.50** (0.05)	0.46** (0.05)
Classroom-level effects
Intervention exposured	−2.98* (1.47)	3.87* (1.51)	0.81* (0.40)	0.71 (0.40)
Students in classroom, no.	0.12 (0.10)	0.09 (0.11)	0.02 (0.03)	−0.01 (0.03)
School-level effects
Socioeconomically disadvantaged students, %	−0.09 (0.02)	0.00 (0.06)	−0.01 (0.01)	0.00 (0.01)
Students with limited English proficiency, %	0.06 (0.03)	0.01 (0.07)	−0.01 (0.01)	−0.03 (0.01)

Note. BMI = body mass index.

^aMeasured by the Progressive Aerobic Cardiovascular Endurance Run test, no of laps run on a 20-m court.

^bNo. of correct answers on an 8-item scale.

^cNo. of correct answers on a 10-item scale.

^dProportion of students in classroom who participated in the after-school program.

*P < .05; **P < .01.

Aside from baseline value, only age was a significant predictor of follow-up BMI percentile. The intervention was more effective in reducing BMI percentile for older children; for each year older that participants were, BMI posttest percentile was 0.79% lower (b = −0.79; t=−2.11; P = .035).

All participants experienced marginal improvements in aerobic capacity. After-school program participants improved an average of 2.7 laps; the average improvement among the spillover and control groups was 2.3 laps. The intervention exposure indicator was a significant predictor of posttest aerobic capacity (PACER score; b = 3.87; t=2.565; P = .012; Table 2). Thus, for a classroom with 50% intervention participants, the PACER score was almost 2 laps higher (3.87 × 0.50) than it was for a classroom with only control participants, with adjustment for pretest score and covariates. The intervention was more effective in increasing PACER scores for boys than it was for girls (boys increased by 2.15 laps more; t = −2.744; P = .007).

Only participants in the after-school program showed marginal improvements on any of the dietary outcomes: their intentions to eat healthy food increased by 0.44 units (5.5%; P = .068). The intervention exposure indicator was the only significant predictor of posttest intentions to eat healthy (except for pretest value; b = 0.80; t= 2.02; P=0.046; Table 2). A classroom with only intervention participants had a 0.81-unit (10.1%) higher score than that of a control classroom in willingness to choose healthier food options at posttest, after control for baseline value and covariates.

We had substantial amounts of missing data on the composite scales of health knowledge and dietary intentions because children needed to answer 20 knowledge items and 16 intention items without missing 1 to be included in the analyses. We repeated analyses with missing data imputed,³⁷ but because this resulted in no changes in significance of coefficients, we present only the original analyses.

DISCUSSION

We developed and evaluated a culturally tailored after-school health education and physical activity program for a population of predominantly socioeconomically underserved Mexican American elementary school children. The risk profile of the participants was high: almost half were overweight or at risk for overweight at baseline, higher than national averages for all children and for Hispanic children.¹ We exceeded our initial recruitment targets, and the number of participants (901) was larger than in any other published study of an after-school program conducted in the United States to date.³⁸

Interest in participation in the project was greater in schools with a larger proportion of socioeconomically disadvantaged children. During the 12-week intervention, the teachers (student PE teachers and community health workers) successfully conducted the program without major incident at all 6 schools, suggesting that it would be feasible to replicate this program elsewhere. Increased intervention exposure was associated with improved physical fitness and dietary intentions. We also found that a higher proportion of after-school participants in a classroom was associated with reduced BMI percentile and increased aerobic capacity score and dietary intentions at follow-up for the classroom as a whole.

Our findings have important implications. They suggest that even children with 1 degree of separation from the intervention (i.e., sharing classroom contact with students who attended an after-school program) experienced improvements that over time could lower their risk of obesity. Without further qualitative work, we cannot determine with certainty how these effects occurred, but we can propose several potential pathways. First, intervention participants likely interacted with nonparticipants in the same classroom during the school day, during recess, and possibly outside of school. These interactions may have exposed the spillover group to some intervention components and a resulting improvement in their health status.

Second, previous research indicates that behavior is more likely to be adopted if a larger number of influential people close to a person have done so (e.g., classmates and close friends).³⁴ We found that a larger proportion of classmates in the after-school program was indeed associated with improved health outcomes. It is possible that classrooms with a large proportion of after-school program participants were more effective in establishing healthy behaviors (e.g., being physically active) as socially desirable, thereby encouraging healthy behaviors for the entire classroom. Finally, reciprocal reinforcement²⁴ may have occurred, in which members of the spillover group, observing their peers who participated in the after-school intervention being reinforced for healthy behaviors (e.g., the incentives offered), may have adopted these behaviors.

Although intervention spillover or ripple effects on nonparticipants were described in school-based settings more than half a century ago in interventions addressing classroom discipline,³⁹ public health interventions rarely address beneficial health effects for nonparticipants. In fact, school-based interventions may actually take precautions to prevent these effects from occurring (contamination). However, as we increasingly understand the necessity of effectiveness trials for real-world dissemination of interventions, the importance of assessing public health impact—rather than just efficacy—is evident.⁴⁰

Understanding and harnessing beneficial spillover effects is a relatively underexplored area of research and could lead to more comprehensive assessments of interventions' effects on households, schools, and communities. For example, recent evidence suggests that prevalence of obesity has been found to spread through social networks: individuals are increasingly at risk for obesity if their family members and friends become obese.⁴¹ The same might hold true for positive health outcomes and risk reduction. If we can assess to what extent our interventions have impact beyond the individual participant, we may find better ways to capitalize on these beneficial consequences. This is essential because in public health contexts, resources are generally limited, and interventions cannot be offered to every eligible participant.

Our results provide an illustration: 55 students (17 after-school program participants, 16 students in the spillover group, and 22 control participants) improved their health status from at risk to healthy weight or from overweight to at risk (17.6% of at-risk or overweight participants). Thus, although a larger proportion of children in the intervention group improved their health status, the majority of children who improved their weight status actually did not participate in the after-school project. These are important findings in light of the high-risk profile of the priority population and the fact that children in these age groups tend to increase, not decrease, their BMI.⁴²

Limitations

The unique characteristics of the study setting along the US–Mexico border, where Hispanics are the majority, have to be considered when interpreting our findings. Although all health education materials were bilingual (as were all but one of the teachers), the language of choice for teaching was often English, possibly limiting the ability of monolingual Spanish-speaking students to follow the health education portion of the after-school program.

Although previous research reported adequate internal consistency for all the scales we used, the nutritional intentions scale showed low internal consistency at baseline. A further limitation was that individual attendance rates were not recorded, and thus an assessment of a potentially more accurate dose–effect relation was not possible. Finally, although the project was inexpensive to implement, our limited resources made it impossible to conduct long-term follow-ups to assess whether the effects were maintained.

Conclusions

Our findings indicate that even when children are not part of an intervention, they can still benefit, through exposure to participating peers, from a program that is implemented in their surroundings. In the context of a high-risk population with limited access to care, spillover of positive health effects on nonparticipants presents a consequence of value and should be part of the evaluation of the public health impact of future programs.