Impact of a Georgia elementary school-based intervention on physical activity opportunities: A quasi-experimental study

Abstract

Objectives:

To assess the impact of Power Up for 30, a flexible, Comprehensive School Physical Activity Program-based state-wide elementary school initiative, on school physical activity opportunities (physical education, recess, in-class physical activity, before-school physical activity, and after-school physical activity) one year after Power Up for 30 training.

Design:

Quasi-experimental.

Methods:

In 2013, all 1333 public Georgia elementary schools were invited to complete the School Physical Activity Survey which assessed school physical activity opportunities. Upon survey completion, schools were invited to attend training. Of the 719 (54%) schools meeting survey response criteria, 300 schools attended training by 9/2014 and 419 schools did not. Between 3/2015–5/2015, 79 trained and 80 untrained schools were randomly selected to receive a follow-up survey assessing the frequency and duration of physical activity opportunities. Analyses, adjusted for baseline physical activity opportunities and school characteristics, compared weekly minutes of physical activity opportunities at follow-up between trained and untrained schools.

Results:

In adjusted analyses at follow-up, trained schools provided 36 more minutes of weekly physical activity opportunities than untrained schools (99% confidence interval: 16–56), particularly during recess (mean difference: 8min per week; 99% confidence interval: 0–17), during in-class breaks (mean difference: 11 min per week, 99% confidence interval: 3–20), and before school (mean difference: 8 min per week, 99% confidence interval: 4–12).

Conclusions:

Flexible, multi-component interventions like Power Up for 30 increase physical activity opportunities. If future studies identify that school physical activity opportunities positively impact student physical activity, this model may be a feasible strategy for broad-scale implementation.

1. Introduction

Physical activity (PA) benefits children by improving cardiovascular and muscular fitness, and decreasing body fat.¹ However, only about half of U.S. youth achieve the recommended 60 min of moderate-to-vigorous PA (MVPA) per day.² At least 30 of the recommended daily 60 MVPA minutes should occur during regular school hours since the majority of school-aged children spend up to half of their waking hours in school.²

Prior studies have suggested that PA interventions effectively but modestly increase PA.^3–5 Many previously tested multi-component interventions, including programs based on the Comprehensive School PA Program (CSPAP), a whole-of-school approach recommended by the Centers for Disease Control and Prevention, are highly structured and require all intervention schools to complete the same components (e.g. hiring PA leaders, making curriculum changes, etc.).^4,6 Complex interventions limit program scalability, while interventions allowing for adaptation by schools facilitate integration of PA into the school. Only a few studies have explored interventions that can be modified at the school or teacher levels and have reported that these interventions may increase PA opportunities or PA during school.^5,7,8

Power Up for 30 (PU30) is a state-wide CSPAP-based initiative to increase PA in school which allows tailoring of the initiative at the school level to encourage 30 min of PA outside of physical education (PE) each school day.⁹ Using a quasi-experimental design, this study explored the impact of PU30 on the PA opportunities offered by elementary schools after PU30 training by reanalyzing data from a prior study and additionally adjusting for baseline school PA opportunities and other school characteristics.¹⁰

2. Methods

Georgia Shape, a state-wide multi-agency initiative addressing childhood obesity, along with the Georgia Department of Public Health, Georgia Department of Education (DOE), HealthMPowers (a non-profit organization aiming to increase health knowledge and promote health-enhancing behaviours among youth), and others created PU30 which involved: (1) a voluntary commitment by each school to increase PA outside of regular PE to 30 min or more per day; (2) a needs assessment (School PA Survey) of baseline PA opportunities and environmental characteristics completed by administrators, PE teachers, and grade level chairs for kindergarten through fifth grades; and (3) upon survey completion, a tailored full-day training based on evidence-based strategies for increasing PA before, during (recess, in-class PA), and after school. PU30 recommended at least one administrator or administrative designee, one PE teacher, and one grade level chair from each school attend the training at an area school, but not all schools met this goal (between 12/2013 and 12/2016, only 1898 administrators, PE teachers, and grade level chairs from 730 schools attended training). At the training, HealthMPowers: (1) discussed barriers and facilitators to PA, and the importance of PA; (2) provided strategies for integrating PA before, during, and after school; (3) shared and modeled the use of low-and no-cost resources including exercise DVDs, PowerPoint files, and an online resource guide containing links to web-based PA videos, PA curricula, and integrated PA-academic lessons; and (4) assisted school teams in developing an action plan to increase PA based on the results of the needs assessment. Throughout the year, participants received monthly emailed newsletters and telephone-based technical support assisting with the engagement of school administrators and staff and selection of appropriate resources.

We employed a quasi-experimental design to compare PA opportunities between PU30-trained and untrained groups at follow-up. In October 2013, the DOE, in partnership with HealthMPowers and the Department of Public Health, emailed an invitation to participate in PU30 and a baseline School PA Survey, which served as part of a needs assessment, to all 1333 Georgia public elementary schools (Supplementary Appendix A). We included schools in the current study if they provided baseline School PA Survey responses from at least one administrator, one PE teacher, and at least three grade level chairs by September 2014 (n = 719 schools; 54%). Upon survey completion, schools were eligible for training. Three hundred of these schools completed the School PA Survey and participated in PU30 training by September 2014, and 419 schools completed the School PA Survey but did not complete PU30 training by September 2014. From the 300 trained schools, we randomly selected 79 schools to form the PU30 arm, and from the 419 schools that did not participate in PU30 training, we randomly selected 80 schools to form the untrained comparison arm, while blinded to the baseline results. Between March and May 2015, we emailed an electronic survey to assess PA opportunities of both arms at follow-up. None of the untrained schools received training before follow-up survey administration. Based on pilot data, we determined a sample of 80 schools per study group to be sufficient to provide 80% power to detect a 0.2 SD change across the set of study outcomes using a significance level of 5% for all tests. This study was reviewed by the Emory University Institutional Review Board and determined to be exempt

Among the schools randomly selected for inclusion in the study, 71/79 (90%) PU30 and 62/80 (78%) untrained schools provided follow-up data between March 2015 and May 2015 from at least one PE teacher and at least one grade level chair. Schools included in this analytic dataset did not significantly differ from sampled schools by free/reduced lunch rate (FRL), race/ethnicity, school size, gender, or geographic setting (p < 0.05; data not shown).

A multidisciplinary team adapted the baseline School PA Survey from widely used school PA survey tools (follow-up administrator survey in Supplementary Appendix B).^11–13 Prior to data collection, the study team conducted focus groups with 117 elementary administrators, PE teachers, and grade level chairs in four Georgia school districts to assess survey clarity, ease of use, completeness, and conciseness (unpublished data, 2017). PE teachers provided data regarding PE, before-school, and after-school PA opportunities, while grade level chairs provided data on recess and in-class PA breaks. PE teacher and grade level chair surveys, which took 10–15 min to complete, included subsets of questions from the larger administrator survey which provided data not used in the current study. PA opportunity outcomes at follow-up included PE frequency (0,1, 2–3,4–5 days per week) and duration (<15,15–19, 20–29, 30–39, 40–49, ≥50 min per PE class); recess frequency (0, 1, 2, 3, 4, or 5 days per week) and duration (<15, 15–19, 20–29, ≥30 min per recess); in-class PA time (0,1–5, 6–10,11–15,16–20, 21–25, ≥25 min per day); before-school PA frequency (0, 1, 2, 3, 4, or 5 days per week) and duration (<10, 10–15, 16–20, 21–25, and ≥25 min per day); and after-school PA frequency (0,1, 2, 3, 4, or 5 days per week) and duration (<10, 10–15, 16–20, 21–25, and ≥25 min per day). To convert these categorical responses into a continuous measures of PA time in minutes/week, we used Monte Carlo techniques (Supplementary Appendices C–E).¹⁴ We defined total weekly PA time as the sum of PA time during school (PE, recess, and in-class PA breaks), before school, and after school.

Covariates included school baseline PAtime, school-level demographics (percent FRL, percent White, and percent Hispanic), and other school-level characteristics (school size and geographical setting). Answer choices to questions regarding PE frequency (0,1, 2, 3,4, 5 days) and before-and after-school PA (presence vs. absence) on the baseline School PA Survey differed from the follow-up survey, so prior to Monte Carlo simulation, responses to questions on PE frequency were re-coded to allow congruence at baseline and follow-up. With the exception of binary variables indicating the presence of before-school and after-school PA at baseline, all other baseline PA variables (weekly during-school PA time, PE time, recess time, and in-class PA time) underwent a Monte Carlo simulation described above. We defined during-school PA time at baseline as the sum of PA time during PE, recess, and in-class PA breaks.

We obtained demographic and school characteristics data from the Georgia DOE and National Center for Education Statistics.^15,16 We based FRL (≤25%, 25.1–50%, 50.1–75%, >75%) and geographical setting (city, suburb, town, rural) categorization on prior National Center for Educational Statistics publications.¹⁶ When no a priori categorizations were identified, we retained covariates as continuous variables (e.g. percent White, percent Hispanic, and school size). Race/ethnicity could be characterized almost completely by percent White, Black, and Hispanic because the sum of percent White, Black, and Hispanic was almost always 100%, so we excluded percent Black and included it as part of the referent group in the models.

To assess whether school demographics and other school characteristics differed between PU30-trained and untrained schools, we computed standardized differences (the difference between means/pooled standard deviation). Standardized differences are independent of sample size, and their absolute values can be interpreted as indicating a meaningful imbalance when greater than 0.1. Three models compared PA opportunities (total weekly PA, PE, recess, in-class PA, before-school PA, and after-school PA time per week) between trained and untrained schools at follow-up. First, the unadjusted model assessed the impact of PU30 on PA opportunities alone using linear regression and generalized estimating equations (GEE), which accounted for clustering of multiple respondents per school, and of schools within districts. Monte Carlo replicates were summarized using PROC MIANALYZE to obtain the mean difference and 99% confidence intervals (CIs). The second approach additionally accounted for baseline PA time (during-school PA, PE, recess, or in-class PA break) or baseline presence of PA (before-school PA or after-school PA). We assessed all baseline PA-exposure interaction terms, but no baseline PA-exposure terms were significant. In the third model, in addition to baseline PA time, we used inverse probability of treatment weights to account for the imbalance of baseline school demographics and characteristics. We included all school demographics and characteristics in the propensity score model. To account for multiple testing, we set alpha at 0.01. We performed all analyses using SAS version 9.4 (Cary, NC) and used the Transparent Reporting of Evaluations with Nonrandomized Designs guidelines to inform the reporting of the research.¹⁷

In a sensitivity analysis to assess the potential impact of unmeasured confounding due to participation in other PA programs, we repeated the analyses controlling for the covariates in model 3 as well as covariates indicating the involvement of schools in other PA programs during the prior (0, 1, or 2 PA programs in 2012–2013) and concurrent years of PU30 training (0,1, 2, or 3 or more PA programs in 2013–2014). We obtained data on prior and concurrent PA programming in each school from the Stong4Life School Programs database maintained by Children’s Healthcare of Atlanta. This database includes information on nutrition-and PA-related programs being implemented in schools throughout Georgia, although we included only PA program data in the current study. We believe this data set to include all programs with a broad reach in the state; however, information is submitted voluntarily, and therefore may not be equally comprehensive across all years.

3. Results

PU30-trained schools were more likely than untrained schoolsto be of high (>75%) FRL but less likely to be of mid-high FRL(50% < FRL ≤ 75%; Table 1). Trained schools had a lower proportionof white students, had a higher proportion of Hispanic students, andwere, on average, larger. A greater proportion of trained schoolswere in suburbs while more untrained schools were in rural areas.Trained schools were more likely to have at least one school PAprogram in the year prior to the study (2012–2013) and in theconcurrent year of study (2013–2014) compared with untrainedschools. At baseline, PU30-trained schools had 11 fewer minutesof recess, but 10 more minutes of in-class PA breaks each weekcompared to untrained schools (Table 2).

Table 1.

School demographics and other characteristics of 71 PowerUp for30-trained and 62 untrained Georgia elementary schools in 2013–2014.

Characteristic	Untrained (n = 62 schools) N (%) or mean (SD)	Trained (n = 71 schools) N (%) or mean (SD)	Absolute value of standardized difference
Free/reduced lunch rate
High (>75% FRL)a	22 (35.5%)	36 (50.7%)	0.31
Mid-high (50%<FRL ≤75%)a	18(29.0%)	11 (15.5%)	0.33
Mid-low (25%<FRL ≤50%)	13(21.0%)	12(16.9%)	0.10
Low (≤25% FRL)	9(14.5%)	12(16.9%)	0.07
Mean % Whitea	50.8 (27.8)	39.0 (27.0)	0.43
Mean % Hispanica	11.2(11.7)	19.8(18.4)	0.56
School size
Large (>735 students)a	17(27.4%)	28 (39.4%)	0.24
Medium (550–735 students)a	23 (37.1%)	22 (31.4%)	0.12
Small (<550 students)a	22 (35.5%)	21 (30.0%	0.12
School geographical location
City	9(14.5%)	9(12.7%)	0.05
Suburbana	21 (33.9%)	46 (64.8%)	0.65
Town	8(12.9%)	7 (9.9%)	0.10
Rurala	24 (38.7%)	9(12.7%)	0.62
Mean # school physical activity programs in 2012–2013
0a	51 (82.3%)	49 (69.0%)	0.31
1a	9(14.5%)	19(26.8%)	0.31
2	2 (3.2%)	3 (4.2%)	0.05
Mean # school physical activity programs in 2013–2014
0a	46 (74.2%)	20 (28.2%)	1.04
1a	10(16.1%)	22 (31.0%)	0.36
2a	4 (6.5%)	23 (32.4%)	0.69
3 or morea	2 (3.2%)	6 (8.5%)	0.22

SD = standard deviation; FRL= free/reduced lunch rate.

^aAbsolute value ofthe standardized difference >0.1 is considered a meaningful imbalance.

Table 2.

Crude mean (standard deviation) minutes of physical activity (PA) offered per week across 71 Power Up for 30-trained and 62 untrained Georgia elementary schools at baseline (2013–2014) and follow-up (2015).

	Baseline		Follow-up
	Untrained (n = 62) Mean (SD)	Trained (n = 71) Mean (SD)	Untrained (n = 62) Mean (SD)	Trained (n = 71) Mean (SD)
PE	105.6 (5.3)	107.7 (4.4)	105.5 (5.5)	104.9(4.3)
Recess	100.3 (3.9)	89.8 (4.2)	96.2 (3.7)	98.7 (3.6)
In-class PA	30.4 (2.3)	40.5 (2.6)	36.1 (2.6)	51.9(2.5)
Before-school PA	PA time not assessed at baseline		3.8 (1.4)	14.9 (3.3)
After-school PA	PA time not assessed at baseline		23.7(6.1)	32.5(5.4)

SD = standard deviation; PE = physical education.

At follow-up, after accounting for baseline PA opportunity time,and demographic and other school characteristics using inverseprobability of treatment weights, trained schools provided 36 moreminutes of total PA time per week than untrained schools (99% CI:16–56; Table 3; model 3). Most of this difference was due to moretime during recess (mean difference: 8 min/week; 99% CI: 0–17), in-class PA breaks (mean difference: 11 min/week; 99% Cl: 3–20), and before school PA opportunities (mean difference: 8 min/week; 99% Cl: 4–12) among trained schools. In the sensitivity analysis additionally adjusting for prior and concurrent PA program participation, results were attenuated but consistent with the main analysis (Supplementary Appendix F).

Table 3.

Mean difference (minutes/week) in physical activity (PA) time per week between Power Up for 30 (PU30)-trained schools and untrained schools, approximately one year after PU30 training (2015) among 71 PU30-trained and 62 untrained Georgia elementary schools.

	Model 1a Mean diff (99% CI)	Model 2b Mean diff (99% CI)	Model 3c Mean diff (99% CI)
Total PA timed	42.8 (22.7 to 62.9)	37.2 (16.5 to 58.0)	36.3 (16.2 to 56.4)
PEtime	−0.6 (−9.4 to 8.2)	−2.7 (−9.6 to 4.1)	−3.4 (−10.2 to 3.4)
Recess time	2.5 (−6.6 to 11.6)	6.5 (−1.8 to 14.8)	8.1 (−0.4 to 16.5)
In class PA time	15.8 (8.2 to 23.4)	12.0 (3.8 to 20.1)	11.4(3.0 to 19.8)
Before school PA time	11.2 (6.5 to 15.8)	10.0 (5.4 to 14.6)	7.9 (3.6 to 12.2)
After school PA time	8.8 (0.1 to 17.6)	6.9 (−0.9 to 14.8)	3.7 (−4.0 to 11.3)

CI = confidence interval; PE = physical education.

Boldface indicates statistical significance (P<0.01).

^aModel 1 was the unadjusted model.

^bModel2was adjusted forbaseline physical activity time (during-school PA time, PEtime, recess time, and in-class PA time) orbaseline physical activity presence/absence (before-school, after-school PA) ofthe outcome.

^cModel 3 was adjusted for baseline physical activity time of the outcome, demographics, and other school characteristics.

^dTotal PA time may not equal the exact sum of PE time, recess time, in-class PA time, before-school PA time, and after-school PA time because of missing values or adjustment for varying baseline PA opportunities.

4. Discussion

This quasi-experimental study found that PU30-trained schools reported 36 more minutes of PA opportunities each week compared with untrained schools even after controlling for baseline PA, demographics, and other school characteristics. This overall increase in PA time was attributable to small increases (8–11 min per week) in PA time offered throughout the week from recess, in-class PA breaks, and before-school PA.

The current study supports most but not all findings from a prior study assessing PU30 which did not control for school baseline PA opportunities and other characteristics.¹⁰ In contrast to the current study, Barrett-Williams et al. concluded that there was an increase in after-school PA program time.¹⁰ Some of the discrepancies between the current and prior study are likely due to methodological differences such as adjustment for covariates, use of combined PA frequency and duration data, and adjustment for the hierarchical nature of the data in the current study. Findings from this study are also comparable to other studies using multi-component PA interventions, which report more PA time provided by teachers after intervention, particularly during school and involving staff.^5,18

While prior and concurrent PA programming variables in the sensitivity analysis were conceptualized as indicators of school-level support for health and PA promotion, participation in prior or concurrent PA programming could also indicate increased opportunities for school administrators and staff to learn about and adopt school PA practices. Sensitivity analysis results must be interpreted with caution, as the Stong4Life School Programs database used to create the prior and concurrent PA programming variables is not comprehensive and further exploration of data completeness and construct validity is needed. The current study did not examine additional effects of implementation, such as the degree of attendance at training on PA opportunities, but a subsequent qualitative study evaluated barriers and facilitators to participation and implementation of Power Up for 30.¹⁹

This study had at least four strengths. First, the survey had a high response rate (84%). Second, data were provided by both grade level chairs (for questions involving in-classroom PA and recess) and PE teachers (for questions related to PE and before-and afterschool activity). This contrasts with prior studies which relied only on PE teachers, who may have less accurate knowledge than grade level chairs about in-class PA or recess time.⁵ Third, independent assessment of each PA opportunity allowed identification of which PA opportunities are most likely to be changed (e.g. recess, in-class breaks, and before school PA) when schools are given the flexibility to choose how to incorporate additional PA into the school day. Fourth, this study built on prior literature and additionally adjusted for confounders, combined duration and frequency measures into a continuous outcome, and used GEE to account for hierarchical nature of the data to produce more valid estimates of the effects of PU30 on PA opportunities in the elementary school setting.

Despite these strengths, there were important limitations to this study. First, the PU30 initiative was not randomized across schools, which increases the possibility of unmeasured or residual confounding because schools opting into PU30 training may differ in important yet unobserved ways from those not volunteering for PU30. However, adjusted analyses likely diminished much of the potential bias. Second, only 78% of untrained schools responded to the follow-up survey (compared with 90% among trained schools), and the loss to follow-up may further bias estimates if the analytic sample differed from the overall sample by unmeasured covariates. However, responding and non-responding schools did not differ on measured demographic or other school characteristics (data not shown), and the probability of untrained, non-responding schools providing more PA than trained schools at follow-up (which may attenuate the observed results), seems low. Third, similar to prior studies,^5,18 this study design only involved an approximately one-year follow-up time, which may be insufficient time for the effects of the PU30 initiative to occur. Further exploration indicated that schools with longer follow-up time had larger differences in total PA time between trained and untrained schools (data not shown), which is consistent with some,²⁰ but not all,⁷ prior publications. Fourth, assessment of PA opportunities relied on self-reported data and increased reporting accuracy among trained individuals as a result of being more familiar with the survey topics could have resulted in differential outcome misclassification. However, alternative forms of PA opportunity measurement, such as researcher observation, may have likely also biased the outcome data. The School PA Survey has not yet been validated, but PA policy assessments on which the survey was based have demonstrated moderate to almost perfect intra-rater reliability.¹³ Our estimates of inter-rater reliability for the School PA Survey using weighted and unweighted Kappa statistics were substantial or almost perfect for questions related to PE and before-school PA, moderate for questions related to recess and after-school PA, and fair for questions related to in-class PA (Supplementary Appendices G–H).²¹ The survey was adopted from widely used surveys by a large multidisciplinary team and our group is currently exploring a formal study assessing survey validity and reliability. Finally, the current study did not explore the quality of PA, developmental appropriateness of PA, PA literacy/skills of students, or PA intensity of students during each PA opportunity. Prior studies using accelerometers have reported that students spend only 33% of PE time in MVPA and even less time in other segments of the school day, and it is possible that just a fraction of the PA opportunity time reported in this study is spent in MVPA.^22,23 Further, not all students may engage in PA during recess or before-school programs. Future studies using activity monitoring devices may detail the amount and intensity of PA performed at the individual level.

5. Conclusion

Flexible and adaptable CSPAP-based initiatives can increase staff-reported PA opportunities, particularly during recess, in class, and before school. If future studies identify a positive impact of school PA opportunities on student PA, this adaptable CSPAP-based model may be a viable strategy for scale-up across diverse settings.

Practical implications.

• Programs aiming to increase physical activity during multiple parts of the elementary school day (i.e. before, during, and after school) can increase physical activity opportunities provided by schools.

• When given the flexibility to adapt physical activity programs at the school level, schools increase physical activity time during recess, in-class breaks, and before-school programs.

• In non-randomized intervention studies, accounting for differences between intervention and comparison schools may impact study results.

Abbreviations:

PA

physical activity

MVPA

moderate-to-vigorous physical activity

CSPAP

Comprehensive School Physical Activity Program

PU30

Power Up for 30

PE

physical education

DOE

Department of Education

FRL

free/reduced lunch rate.