School-community partnerships: A cluster-randomized trial of an after-school soccer program

Abstract

Objective

Identifying community-based programs that increase physical activity among diverse youth could yield sustainable tools to reduce obesity and obesity disparities. We sought to evaluate the impact of a community-based after-school soccer and youth development program, America SCORES, on students’ physical activity, weight status, and fitness.

Design

Cluster-randomized trial.

Setting

After-school programs in 6 schools within a large urban school district.

Participants

All 4^th and 5^th grade students in after-school programs at the study schools were eligible.

Intervention

3 schools were randomized to receive the SCORES after-school program, delivered via the train-the-trainer model.

Main Outcome Measures

Change in minutes of after-school moderate-to-vigorous physical activity (MVPA), fitness (VO2max), and body mass index (BMI) over one school year.

Results

Participants (n=156) were diverse (42% Latino, 32% Asian, 12% African American) and 76 (49%) had a BMI above the 85^th percentile. There were no significant group differences in change in physical activity, fitness or weight status among all students. However, among students with a BMI ≥ 85^th percentile, SCORES significantly increased MVPA after school (3.4 mins/day, 95% CI[0.3, 6.5]) and on Saturdays (18.5 mins, 95% CI [3.4, 33.6]]).

Conclusions

Existing community-based programs like SCORES can increase physical activity among low-income youth, particularly those most at risk for weight-related comorbidities. While evaluating existing programs presents special challenges, partnerships between communities, schools, and researchers are an important component of translational research to address obesity.

INTRODUCTION

The alarming increase in pediatric obesity^1,2 has been accompanied by a rise in the prevalence of diabetes and hypertension in youth.^3,4 Physical activity protects against the development of obesity and is closely linked to decreased cardiovascular risk,^5,6 yet youth are far from meeting recommended activity levels.^7–9 Low-income and minority youth are disproportionately less active and more obese than their high-income or white peers.^8–12

The after-school hours are a promising venue in which to increase physical activity for diverse youth.¹³ While recent demonstration projects in the after-school setting have shown promising trends in physical activity and fitness in youth of color,^14,15 when funding ends for these research-driven programs, new sources of financial support must be identified to continue programming. A train-the-trainer model, in which after-school staff are trained to implement a physical activity curriculum, offers greater promise for sustainability after research funding is gone. A handful of recent studies have employed a train-the-trainer model in the after-school setting with some promising results,^16–18 although none were randomized longitudinal trials.

Using a cluster-randomized controlled design, the present study investigates the impact of the America SCORES after-school program on physical activity levels, weight status, and cardiovascular fitness among diverse elementary school students. SCORES is an organization that provides after-school soccer and literacy programs in 15 cities across the country to promote child health and positive youth development. They have refined their curriculum based on 20 years of experience working in low-income schools. A pilot study of SCORES demonstrated that participants significantly improved their fitness, with a trend towards improved weight status among obese students after one school year.¹⁹ The present study examined the impact of SCORES, delivered via a train-the-trainer model.

METHODS

Design

This cluster-randomized controlled trial examined the impact of the SCORES program on physical activity levels, weight status, and cardiovascular fitness over one school year, among 4^th and 5^th grade students. The University of California, San Francisco’s Committee on Human Research approved this research. Parent or guardians provided written consent and students gave verbal assent.

Sample Size

Sample size calculations suggested 64 participants would be required to see a 5-minute difference in MVPA, assuming student-level randomization within 2 schools. Subsequently, it was decided to randomize at the school level, in response to principals’ concerns that randomizing students within schools would be unfair. Given expected clustering by school, the sample size was increased to 6 schools.

Setting and Participants

This study took place in a large, diverse, urban school district, with an enrollment of 56,000 students. Of 72 schools with grade K-5 enrollment, 60 schools that had not offered SCORES in the year prior to the study were eligible to participate. The study was presented at a regularly scheduled principals’ meeting, at which 14 eligible schools were represented, and 7 schools agreed to participate. At study schools, 61% of students were eligible for free or reduced-price (FRP) meals (range 44% to 89%). There were no statistically significant differences between the study schools and other eligible schools (n=53) in the percent FRP meals (p=0.960), percent non-white students (p=0.390), or school enrollment (p=0.227).

All fourth and fifth grade students enrolled in the after-school program at participating schools were eligible for the study. After-school programs can accommodate approximately 25% of the total student body and preferentially enroll students who qualify for FRP meals.

Intervention

America SCORES is an after-school soccer program started by an elementary school teacher in 1994 in Washington D.C. to provide a safe after-school environment and to promote literacy among her inner-city students. SCORES differs from typical after-school sports programs in that it uses soccer as a “hook” to engage youth, with a primary goal of building competencies and skills that will support students’ overall development, including teamwork, leadership, and academic commitment. The program, which has expanded to 15 major cities throughout the country, offers soccer, creative writing and service learning experiences to youth that would otherwise have limited access to extracurricular activities. SCORES employs soccer and writing coaches who are trained with a standard curriculum to lead the SCORES program in the after-school setting. Students spend 2–3 days per week in soccer drills or games for up to 2 hours each day. The 2 non-soccer days each week are dedicated to creative writing and performance in the 12-week fall session, and to community service projects in the 12-week spring session. Historically, schools pay up to half of the cost of operating the SCORES program, while SCORES has raised the balance through grants and private donations.

While the initial plan was to study the traditional SCORES model described above, due to significant budget cuts in the district in 2009–10, schools had fewer resources to contract staff from outside agencies like SCORES to deliver their programs. SCORES responded by moving to a train-the-trainer model in which SCORES trained the district’s after-school staff to operate the SCORES program. SCORES had already implemented this model with good success in several schools in the district that had a strong history with the SCORES program, but its impact and dissemination to new schools had not been tested.

Under the train-the-trainer model, the 3 SCORES schools received the SCORES curriculum, 6 hours of training in the fall before SCORES began, and a further 6-hour training in the spring. Training included lesson planning and execution, student soccer and poetry skill development, and behavior management. SCORES provided coaches with soccer coaching manuals with over 100 soccer practice activities and games and a writing program curriculum with examples and activities. Additionally, SCORES staff visited each of the three intervention school sites multiple times over the course of the study to provide technical assistance. After-school staff were encouraged to use the SCORES curriculum to provide a minimum of 1 hour of soccer on 2 afternoons each week and 1 hour of creative writing on 2 afternoons each week, during each 12-week session (fall and spring). SCORES organized game days on Saturdays, providing 1 additional hour of soccer each week. While SCORES activities were open to all 4^th and 5^th graders at intervention sites, students were not required to participate in the SCORES program and could, instead, participate in other scheduled activities.

Outcomes

Study measures were collected in the fall (baseline), winter (mid-point), and spring(end-point) of the 2009–10 school-year.

The primary outcome was change in after-school minutes of moderate-to-vigorous physical (MVPA) activity from baseline to both mid- and end-point. The after-school period was designated as the time from the end of the school day (based on each school’s after-school program schedule) until the end of the after-school program (mean 3.0 hrs, range 2.3 to 4.3 hrs). Students wore a GT1M or GT3X accelerometer (Actigraph, LLC, Fort Walton Beach, FL), which yield equivalent activity counts when programmed in uniaxial mode,²⁰ for 7 consecutive days (including 5 school days) at all three time points. Accelerometer data were collected from intervention and control schools in alternating weeks in an attempt to minimize differences of the weather’s impact on physical activity between the two groups. Accelerometer data were recorded in 15-second epochs. Time periods with more than 20 minutes (80 epochs) of continuous zero values were considered non-wear time²¹ and were omitted from analyses. A composite method of accelerometer data reduction, which uses more available accelerometer data compared to standard approaches and is more representative of student activity across days than imputation,²² was used to determine average minutes spent in MVPA. First, each epoch was categorized as MVPA if activity counts were ≥ 574, based on Evenson’s cutpoints.²³ Second, each time-stamped epoch was “averaged” across school days, yielding the proportion of days for which a given epoch was considered MVPA. For example, if, for the 15-second epoch ending at 3:00:15, 2 of 5 school days were considered MVPA, 40% (6 seconds) of the “composite” epoch for 3:00:15 would be considered MVPA. Finally, total time in MVPA was calculated by summing the number of seconds of MVPA accumulated in each epoch of the “composite” day.

Secondary outcomes were change in body mass index (BMI)z-score and cardiorespiratory fitness. With children in indoor clothes and shoes off, trained researchers measured height to the nearest tenth of a centimeter using the 420 Measure-All Portable Measuring Board (KWS Medical Supplies, LLC, North Bend, WA) and weight to the nearest tenth of a kilogram using the Tanita Model BWB 800 digital scale (Tanita Corporation of America, Arlington Heights, IL). BMI was calculated as weight(kg)/height(m)². BMI z-scores were calculated using the 2000 CDC Growth reference data.²⁴ Cardiorespiratory fitness was assessed using the validated 20-meter shuttle test.^2–5 A parent or guardian of each student provided parents’ highest level of education (used as a proxy for SES) and student’s race.

As part of a process evaluation, school staff reported their SCORES schedules and student attendance. Researchers conducted semi-structured interviews with after-school staff at each school (total n=11). Interview transcriptions were reviewed by two researchers and barriers to implementing the curriculum were identified and enumerated.

Randomization

The 7 study schools were randomly allocated to intervention or control status (Figure). Researchers informed schools and SCORES of the assignments. Neither schools nor researchers were blinded to assignment. Prior to the study start date, leadership at one of the intervention schools changed, and the new principal withdrew the school from the study.

Figure.

CONSORT diagram of after-school participant flow through the study

Statistical Methods

To accommodate the cluster design and repeated measures, linear mixed effect models (with random effects for school and student) were used to analyze differences between groups in change in after-school minutes of MPVA, BMI, and fitness, from baseline to mid- and end-point, adjusting for baseline values as well as sex and baseline weight status. A test for interaction demonstrated that weight status (BMI ≥ 85^th percentile) modified the impact of SCORES on after-school MVPA (p = 0.041), but no other outcomes. All analyses were performed using Stata/MP version 11 (StataCorp LP, College Station, TX).

RESULTS

Of 88 eligible students in the 3 intervention schools, 82 (93%) enrolled in the study, and 74 of 86 eligible students (86%) enrolled in the study in control schools. Follow-up data were available on all but 3 students in each group at end-point (96% retention - Figure); all 6 of these students had left the school. Students were diverse and the majority of parents had a high school education or less (Table 1). At baseline (Table 1), a smaller proportion of intervention students were overweight or obese than control students (40% vs. 58%, 95% CI for difference [2%, 34%]), with a trend towards lower BMI z-scores among intervention students (mean difference −0.28, 95% CI [−0.60, 0.05]). Students in intervention schools did more MVPA after school than students in control schools, both overall (mean difference 6.4 mins, 95% CI [3.3, 9.4]) and among students with a BMI ≥ 85^th percentile at baseline (mean difference 4.6 mins, 95% CI [0.8, 8.3]) with no other differences between groups. Overall, students did more MVPA on weekdays than on Saturdays (mean difference overall 14.5 mins, 95% CI [8.8, 20.3]) and overweight and obese students did less MVPA than non-overweight students (mean difference −4.5 mins, 95% CI [−7.6, −1.4]).

Table 1.

Baseline characteristics, mean ± SD or %

	All students (n=156)	Intervention (n = 82)	Control (n = 74)
Female (%)	40	38	42
Age (years)	9.8 ± 0.6	9.8 ± 0.6	9.8 ± 0.7
Race (%)
African American	12	14	11
Asian	32	36	27
Latino	42	38	45
White	0	0	1
Other	14	13	16
Mother’s Education Level (%)
High school or less	56	56	56
Some college	30	27	32
College grad	14	16	12
BMI (kg/m2)	20.5 ± 4.5	19.8 ± 4.3	21.2 ± 4.6
BMI z-score	1.0 ± 1.0	0.8 ± 1.0	1.1 ± 1.0
BMI ≥ 85th percentile (%)	49	40	58
VO2max (ml/kg*min)	47.1 ± 5.7	47.3 ± 6.0	46.9 ± 5.4
After-School Length (min)	180 ± 37.9	190 ± 49	172 ± 13
School-Day Length (min)	480 ± 0	480 ± 0	480 ± 0
Time in MVPA (min)
After School	17.3 ± 9.7	20.3 ± 11.0	14.0 ± 6.4
School-Day	19.4 ± 9.6	20.3 ± 9.0	18.4 ± 10.2
Weekday 6am – 10pm	52.6 ± 22.0	56.0 ± 21.9	48.7 ± 21.7
Saturday 6am – 10pm	32.9 ± 32.1	35.1 ± 34.7	30.5 ± 29.2

In adjusted models, there was no difference between groups in change in after-school MVPA(Table 2), nor were there differences in change in BMI z-score or fitness (or any changes within groups). Among students with a BMI above the 85^th percentile at baseline (n=76), SCORES significantly increased MVPA after school by 3.4 mins (95% CI [0.3, 6.5]) and on Saturdays by 18.5 mins (95% CI [3.4, 33.6]), though there were no differences in change in BMI z-score or fitness between groups.

Table 2.

Adjusted mean differences (SE) in MVPA from baseline by condition and pretest weight status

	Mid-point Δ		End-point Δ		Δ [95% CI] SCORES vs. controls*
	SCORES	Control	SCORES	Control
After-school MVPA (mins)‡
All Participants	0.6 (1.0)	−0.1 (1.0)	2.1 (1.0)	1.5 (1.1)	0.7 [−1.7, 3.1]
BMI <85th %tile	−1.7 (1.5)	0.0 (2.1)	0.9 (1.6)	2.6 (2.1)	−1.7 [−6.4, 3.0]
BMI ≥85 %tile	3.6 (1.4)	0.2 (1.2)	3.8 (1.4)	0.4 (1.2)	3.4 [0.3, 6.5]
Saturday MVPA (mins)
All Participants	10.3 (7.0)	−2.4 (7.0)	12.3 (7.0)	−0.4 (7.2)	12.7 [−5.3, 30.8]
BMI <85th %tile	13.7 (9.6)	3.8 (10.3)	13.1 (9.9)	3.2 (10.5)	10.0 [−15.6, 35.5]
BMI ≥85 %tile	10.6 (7.2)	−7.5 (6.0)	14.6 (6.8)	−3.4 (6.6)	18.5 [3.4, 33.6]

BMI: Body Mass Index

^*From mixed effects repeated measure model, coefficient regressing change from baseline on group (SCORES vs. control), adjusted for baseline value, baseline BMI z-score and sex.

^‡Group×weight status interaction was significant only for after-school MVPA.

Interviews with 8 after-school staff at intervention schools suggested that staff enjoyed leading SCORES. All interviewees (n=8) reported receiving excellent one-on-one help from SCORES staff during the school year, but noted they would have benefitted from longer trainings (n=4) and additional training mid-season (n=4). Staff at all 3 intervention schools noted barriers to implementing SCORES, including: not enough time to implement the curriculum because of competing priorities including homework time and other enrichment activities (3 schools); space limitations (2 schools); and lack of student motivation to participate (2 schools). Participation in SCORES was optional and overall attendance was 48%(range 0% to 100%), with overweight and obese students attending more than normal weight students (60% vs. 39%, 95% CI for difference [2%, 38%]). While staff almost met the goal of offering 12 weeks of SCORES programming in the fall (mean 11.3 weeks), in the spring only 7 weeks of SCORES were offered on average (this was driven by low-compliance at one school). Based on average attendance rates, students in intervention schools were exposed to an average of 1.4 hours of soccer each week.

DISCUSSION

There is a clear need for programs and policies that effectively increase physical activity among diverse youth. In the present study the SCORES program significantly increased MVPA among overweight and obese youth, both after school and on the weekend. A unique strength of this study is its focus on a program that has demonstrated portability and scalability under its traditional model, where the program is delivered by SCORES staff. Like other long-standing programs, SCORES has specific advantages in implementing the train-the-trainer model, including extensive expertise working with school districts and existing infrastructure to readily disseminate their curriculum.

While the adjusted relative increase in after-school MVPA of 3.4 minutes in SCORES schools was small, it is consistent with the mean effect (4 minutes) reported in a recent meta-analysis of physical activity interventions conducted across multiple settings.²⁶ Taken together, the increased activity demonstrated after-school and on Saturday among overweight and obese youth in the present study equate to an additional 35 minutes per week of MVPA.

Three recent studies have employed train the trainer models across different after-school settings. One cluster-randomized study, taking place in a single school district over 3 years, trained after-school staff to implement the Catch Kids Club curriculum²⁷ in intervention schools.¹⁶ While the nested cross-sectional design could not account for significant between-group differences in baseline MVPA, in the first year of the intervention overweight and obese students engaged in 6 more minutes of MVPA daily than did students in the baseline year.¹⁶ One quasi-experimental study implemented the Catch Kids Club curriculum in YMCAs and Boys and Girls Clubs, with no increase in MVPA over one school year.¹⁸ However, the training for staff was minimal and staff noted mixed reception of the program by youth. A third study offered multi-day trainings for YMCA staff, using a 10-unit curriculum to support the provision of at least 30 minutes of physical activity daily in YMCA after-school programs.¹⁷ This study demonstrated an increase in after-school MVPA of 10 minutes over one school-year. While promising, it is unclear if the increased MVPA seen in that quasi-experimental study, which had retention rates just under 50%, would be replicable in a randomized controlled trial.

To our knowledge, the present study of SCORES is the first randomized longitudinal study of after-school physical activity delivered via the train-the-trainer model. With retention rates of 96% in both arms and an intention-to-treat analysis, this study reflects real-world effects. Differences in study design make it difficult to compare effect sizes between the present study and other recent studies.^16–18 While randomization in the community setting is difficult, future randomized trials comparing the effectiveness of various curricula under the train-the-trainer model would be of value.

In the present study, the largest increase in MVPA was seen on Saturdays, when the program was delivered by SCORES staff, rather than after-school staff. Under the intended design for the present study, the program was to have been delivered entirely by SCORES staff. However, school district budget constraints precluded this approach. It would be of interest to compare students’ physical-activity levels in programs delivered by staff of existing programs (such as SCORES)versus via the train-the-trainer model, exploring cost-effectiveness as well. Both approaches, which take advantage of the expertise and infrastructure of existing community programs, could lead more quickly to solutions to the obesity epidemic than traditional clinical research models that may not be scalable or sustainable.

This study had several limitations. While we controlled for baseline characteristics, the significant between-group differences at baseline still pose a threat to internal validity. Accelerometers captured only 1 week of data at each time point, which may not represent activity at other times and we did not account for program attendance during accelerometer data collection nor for the time at which children left the after-school program (parents can pick their child up early). Thus, after-school MVPA may have occurred outside the after-school program, though this would affect both groups equally. Finally, the intervention was not delivered with as much fidelity as we would have expected if SCORES staff, as opposed to trained after-school staff, had led the program, which may underestimate the program’s effects.

America SCORES is a promising program that increases physical activity among overweight and obese youth. While research with community partners presents special challenges, it provides the most accurate assessment of intervention effectiveness in the real world setting where programs are actually delivered.