Improving Nutrition and Physical Activity Policies in Afterschool Programs: Results from a Group-Randomized Controlled Trial

Abstract

OBJECTIVE

Afterschool programs can be health-promoting environments for children. Written policies positively influence nutrition and physical activity (PA) environments, but effective strategies for building staff capacity to write such policies have not been evaluated. This study measures the comprehensiveness of written nutrition, PA, and screen time policies in afterschool programs and assesses impact of the Out of School Nutrition and Physical Activity (OSNAP) intervention on key policies.

METHODS

Twenty afterschool programs in Boston, MA participated in a group-randomized, controlled trial from September 2010 to June 2011. Intervention program staff attended learning collaboratives focused on practice and policy change. The Out-of-School Time (OST) Policy Assessment Index evaluated written policies. Inter-rater reliability and construct validity of the measure and impact of the intervention on written policies were assessed.

RESULTS

The measure demonstrated moderate to excellent inter-rater reliability (Spearman’s r=0.53 to 0.97) and construct validity. OSNAP was associated with significant increases in standards-based policy statements surrounding snacks (+2.6, p=0.003), beverages (+2.3, p=0.008), screen time (+0.8, p=0.046), family communication (+2.2, p=0.002), and a summary index of OSNAP goals (+3.3, p=0.02).

CONCLUSIONS

OSNAP demonstrated success in building staff capacity to write health-promoting policy statements. Future research should focus on determining policy change impact on practices.

BACKGROUND

Afterschool programs hold promise as health-promoting environments for children. Afterschool programs serve 8.4 million children annually for an average of 8.1 hours per week (Afterschool Alliance, 2009), often providing snacks and physical activity (PA) opportunities (Coleman et al, 2008). Recent studies have found that interventions to modify PA practices at afterschool programs can increase children’s PA (Beets et al 2009; Dzewaltowski et al 2010; Gortmaker et al 2012). Interventions to modify food service practices have improved the nutritional quality of snacks served (Mozaffarian et al 2010; Nanney et al 2012; Giles et al 2012). Although mounting evidence supports an association between afterschool program practices and child health, less is known about the written policies that may direct afterschool program practices.

Written nutrition and PA policies have the potential to change staff practices and environmental attributes (Story et al 2009; Jaime et al 2009). Written policies in K-12 settings have been shown to modify food service (Cullen & Watson, 2009), increase access to physical education (Slater et al 2011), and improve dietary intake (Cradock et al 2011; Cullen et al 2008). Policies are defined in this study as “laws, regulations, and rules (both formal and informal) [CDC Division of Nutrition, Physical Activity, and Obesity, 2012],” that outline specific practices to be followed. This definition of policy refers to written rules that shape and direct organizational practices, facilitate communication of expectations and procedures to staff, families and children, and help make changes in practice sustainable over time by encoding them into documents (Fixsen et al 2005). Such policies can be written at several levels of influence for afterschool programs, including federal standards for snacks served as part of federal meals programs, state or municipal licensing regulations, school district wellness policies, rules from provider agencies (such as the YMCA or Boys and Girls Club), and program-level written documents such as family and staff handbooks, schedules, menus, or letters to parents.

Although recent reviews of national and state advisory organizations’ nutrition (Beets et al 2011) and PA (Beets et al 2010) guidelines for afterschool programs have suggested that some consensus exists over the key standards that should be included in health-promoting nutrition and PA policies, enforceable federal, state, and program-level policies, if they exist, tend to be non-specific. Federal regulations for snacks served as part of the National School Lunch Afterschool Program (NSLP) or the Child and Adult Care Food Program (CACFP) require that participating programs serve two of four food components (milk, fruit/vegetable, grains/breads, meat/meat alternate) to meet reimbursement requirements, but do not specify further nutrition standards (USDA, 2007; USDA, 2009). The current status of state licensing regulations surrounding PA, nutrition or screen time (ST) in afterschool programs is unknown. A recent study examining policies and related practices at the afterschool program level found that many programs do not have written policies related to nutrition or PA, and that existing PA policies are typically written in non-specific, non-measurable language (Ajja et al, 2012). Expanding and improving written policies may be an important component of lasting changes to afterschool programs’ nutrition and PA environments. However, despite the accumulating evidence on successful efforts to change afterschool environments and practices (Beets et al 2009; Dzewaltowski et al 2010; Gortmaker et al 2012; Mozaffarian et al 2010; Nanney et al 2012; Giles et al 2012; Huberty et al 2013), no interventions have engaged afterschool program staff in writing or adopting policy statements to align with health-promoting environmental and practice changes. Additionally, while a measure has been developed to survey whether or not programs have any written policies for nutrition and PA (Ajja et al 2012), there are no existing methods for a more detailed assessment of the actual language in written policies.

This study describes the development and psychometric properties of a measure designed to assess the language of written afterschool program policies related to nutrition, PA and ST standards. It then uses data from a group-randomized, controlled trial to test the hypothesis that a multilevel intervention focused on advancing specific policy, systems, and environmental changes in afterschool programs would result in improvement in related afterschool program-level policies.

METHODS

Study Design

Twenty afterschool programs in Boston, MA participated in the Out of School Nutrition and Physical Activity (OSNAP) group-randomized, controlled trial from September 2010 through June 2011. The unit of randomization and analysis was the program. Local community afterschool program providers identified programs that met the following study eligibility requirements: serving children ages 5–12, enrolling more than 39 students, and running continuously from at least mid-October through the end of May. Programs were matched on their program provider, snack provider, PA facilities, and school-level socio-demographic composition and randomized to intervention or control (delayed intervention) status following baseline data collection in the fall of 2010 (Figure 1). The study manager enrolled and matched programs; a colleague not involved in the study randomized programs to intervention or control status using a computer-based random number generator. Delayed intervention programs participated in OSNAP the following school year. The sample size was originally determined for other study outcomes; it was determined that the study would have 95% power to detect an increase of 0.2 servings of fruits and vegetables consumed per day. The study was approved by the Harvard School of Public Health Committee on Human Subjects and the Boston Public Schools (BPS) Research and Evaluation Department.

Figure 1.

Out of School Time Physical Activity and Nutrition (OSNAP) Initiative flow chart.

Intervention

OSNAP was a multi-level intervention with a community-based participatory approach focusing on policy, systems, and environmental changes to support several nutrition and PA-related standards in afterschool settings. The OSNAP standards were developed in 2008 and were based on available science at the time, with input from local OSNAP project partners in Boston, MA to ensure feasibility; they had evolved from a prior partnership with the YMCA of the USA (Wiecha et al 2010). The OSNAP standards were: 1) Include 30 minutes of moderate, fun, PA for every child every day (include outdoor activity if possible); 2) Offer 20 minutes of vigorous PA three times per week; 3) Ban sugar-sweetened beverages (SSBs) from snacks served; 4) Offer water as a beverage at snack every day; 5) Offer a fruit or vegetable option every day at snack; 6) Ban foods with trans fats; 7) Ban SSBs brought in from outside the snack program; 8) Eliminate use of commercial broadcast TV/movies; and 9) Limit recreational computer time to less than one hour per day. Giles et al (2012) have described the two core OSNAP intervention components: a series of three learning collaboratives (LCs) with teams of afterschool staff, focused on skill development and knowledge exchange (Wiecha et al 2010; Kilo 1998), and snack menu changes made in conjunction with the BPS Department of Food and Nutrition Services.

The LCs took place over six months during the school year. Each LC lasted three hours and was facilitated by study staff. At the first LC, afterschool program teams reviewed information about their nutrition, PA, and ST practices and written policies found in their program’s documents, such as handbooks, menus, schedules, joint-use agreements, and communications to families. Programs set practice and policy goals in the first LC to address unmet OSNAP standards and identified action steps for meeting those goals, such as specifying that they would write a letter to parents stating that water was the only beverage allowed from home. In the two subsequent LCs, staff focused more intently on policy, participating in skill-building sessions related to both writing and communicating policy statements, both for meeting the OSNAP standards and for meeting other related goals. The first policy-related skill-building session focused on crafting policy statements that were measurable, attainable, and related to specific goals and the practice implications of using verbs that require accountability (e.g. “will” rather than “should”). Participants then wrote policy statements to address the goals they had set in the first LC. At the third LC, participants shared their policy statements along with their tips and strategies for communication and implementation. The goals, and therefore the policy statements, were usually focused on meeting specific OSNAP standards, although programs did explore other, related policy statements, such as increasing nutrition education and reducing candy and gum brought into the program. Tools and resources provided to afterschool staff teams included a policy writing guide providing sample policy language related to each of the OSNAP standards and to other nutrition, PA, and ST goals for afterschool programs, sample snack menus and program schedules that met the standards and could be adapted to program use, and a poster outlining the OSNAP standards (all materials available online at www.osnap.org). Program staff could directly adopt the OSNAP standards using language suggested in the guide or write their own policy statements.

Measures

Out-of-School (OST) Policy Assessment Index: Measure Development

We developed the OST Policy Assessment Index to evaluate the content of written policies in OST settings related to the OSNAP standards and other selected indicators (Appendix 1). The index was modeled after existing instruments and document review methods used in school and child care settings (Benjamin et al 2007; Falbe et al 2011; Schwartz et al 2009) in order to compare the language in a program’s policy documentation to identified standards for healthy nutrition, PA, and ST environments. Researchers identified 61 main indicators for “best practice” standards within the domains of PA, snacks, beverages, ST, and family communication for OST and child care settings that were either suggested from research literature or promoted by national organizations at the time the instrument was developed (American Academy of Pediatrics 2011; Alliance for a Healthier Generation 2009; National Policy and Legal Analysis Network to Prevent Childhood Obesity (NPLAN) 2012; NPLAN 2013). Several indicators were linked with sub-items in order to capture more details about specific policy language. These items were consistent with suggested standards for nutrition and PA in afterschool programs released subsequent to the tool creation (Wiecha et al 2012; National Institute on Out of School Time, 2013). The study team used this index to determine whether programs had policy statements meeting the indicators in each domain. Because the primary goal for the policy component of this particular intervention was to write policy statements supporting the standards, we also used the index to determine how many of the OSNAP standards were reflected in the program’s policy language.

Policy Identification

Researchers requested copies of all written policy documents from program directors during baseline data collection in fall 2010 and following the intervention in spring 2011. These included student, staff, and/or family handbooks, joint-use agreements, and communications with families, as well as implementation and communication documents such as menus, schedules, curriculum materials, flyers/brochures, staff training materials, and posters highlighting program rules. This analysis focused solely on program-level policies since the intervention took place at the program level; provider-, district-, and state-level policies were not included. Trained data collectors, blinded to intervention status, photographed posted program rules and gathered copies of documents such as handbooks, schedules, menus, and letters to parents. Researchers assumed no change in policy at follow-up unless a new policy was submitted by the program or otherwise documented by on-site data collectors at follow-up; this assumption was verified by afterschool program directors.

Policy Coding

Two raters (one blinded to intervention status) independently reviewed each document, coding for the presence or absence (coded 0=absent, 1=present) of a policy statement in adherence with each indicator item in the index, noting the source. For example, if a program’s parent handbook included a statement specifying that the program provided at least an hour of moderate PA opportunities every day, a rater would code that a policy statement meeting the standard of providing at least 30 minutes of moderate to vigorous PA each day was “present.” After independent review, the two raters identified discrepancies and developed consensus coding. Using the index, total scores for each domain (Physical Activity, Nutrition, Beverages, Screen Time, Communication) were calculated by summing the present items. If a policy statement matching an indicator on the index was found in multiple documents, that item was only counted once in the total score. We calculated the “OSNAP score” by summing the number of indicators present that addressed specific OSNAP standards across the domains.

Program Characteristics

Program directors reported the number of children enrolled, the number of staff at the program, and the provider agency overseeing the program. Researchers obtained aggregated demographic variables and food service provider type from school administrative records.

Statistical Analysis

Inter-rater reliability of the OST Policy Assessment Index was assessed at baseline using Spearman’s correlation coefficients comparing the raters’ independent coding in each domain. It was hypothesized that YMCA of Greater Boston programs would have more extensive written policies supporting nutrition, PA, and reduced screen time given the existence of iPlay, an initiative in Boston YMCA programs committed to these issues (YMCA of Greater Boston 2012). Therefore, we evaluated the index’s construct validity by comparing these programs’ baseline policy scores to non-YMCA programs’ scores.

The primary outcome was change in OSNAP policy score from baseline to follow-up. Additional outcomes included score changes for each OST Policy Assessment Index domain. Linear regression models were estimated to test whether randomization to the OSNAP intervention resulted in increases in policy domain scores at follow-up compared to control programs, adjusting for baseline policy domain score. Estimated models included nine indicator variables to account for the matched pair study design. Intention-to-treat (ITT) analysis protocols were used for all analyses of intervention impact. Analyses were conducted using SAS 9.3 (SAS Institute, Cary, NC).

RESULTS

Sample Characteristics

At baseline, 71 total policy documents were received from 18 of the 20 programs. The two programs without policy documents reported having no written policies. At baseline, there were no differences in program characteristics (Table 1) or the number of documents submitted between intervention and control programs (p=0.17). At follow-up, 44 additional documents were submitted by 13 of the 20 programs; 31 documents came from intervention programs, and 13 from control programs.

Table 1.

Characteristics of Afterschool Programs Participating in the OSNAP Intervention Trial, Boston, MA, Fall 2010 (n=20).

	Intervention	Control	p-value
Average child age per site, mean (SD)	8.0 (0.6)	7.7 (0.7)	0.36
Percentage non-Hispanic white participants per site, mean (SD)	5.6 (6.6)	15.5 (17.5)	0.11
Percentage non-Hispanic black participants per site, mean (SD)	37.2 (36.2)	38.3 (25.4)	0.94
Percentage Hispanic participants per site, mean (SD)	43.1 (34.5)	32.7 (21.3)	0.43
Number of staff per site, mean (SD)	7.6 (6.8)	10.6 (11.4)	0.48
Average number of participants enrolled per site, mean (SD)	62.1 (36.7)	83.0 (91.1)	0.51
Program Snack Provider, n (%)			0.99
On-site Cafeteria	4 (40%)	3 (30%)
Outside Vendor	5 (50%)	6 (60%)
Program-Provided	1 (10%)	1 (10%)
Sponsoring agency, n (%)			0.66
YMCA	4 (40%)	4 (40%)
Boys & Girls Club	1 (10%)	3 (30%)
Boston Center for Youth and Families	2 (20%)	2 (20%)
Independent Program	3 (30%)	1 (10%)

Frequently Observed Policies

Baseline values for domain-specific and OSNAP scores are reported in Table 2. For PA, policy statements frequently observed in program policy documents at baseline addressed indicators for requiring at least 30 minutes of moderate PA every day (9 programs), requiring the program to encourage all children to participate in PA (15 programs), requiring variety in the types of PA offered (7 programs), and the use of a PA curriculum (8 programs). Policies referring to indicators for an optimal PA environment, including statements requiring the provision of space and equipment for PA, were not observed at baseline. Policies restricting trans fats (5 programs), promoting fruits and vegetables, (5 programs) and promoting a nutrition curriculum (8 programs) were the most frequently observed items in the snacks domain at baseline, while policies promoting water (6 programs) and restricting SSBs (8 programs) were most frequently observed in the beverage domain. Frequently observed policies regarding ST indicators included limitations on ST practices, though policies restricting the presence of televisions, computers, or video game apparatuses were not observed.

Table 2.

Baseline scores of Out-of-School (OST) Policy Assessment Index domains, mean (SD), in 20 after school programs in Boston, MA, Fall 2010.

Domain	Possible range	Mean baseline policy scores, all sites (n=20)	Inter-rater reliability (Spearman’s r)	Construct validity (comparison of means)
				YMCA-affiliated programs (n=8)	Non-YMCA programs (n=12)	p-value
OSNAP score	0–10	2.90 (3.28)	0.84 (p<0.001)	5.50 (3.82)	1.17 (1.03)	0.001
Physical Activity	0–13	3.35 (1.84)	0.89 (p<0.001)	4.63 (1.19)	2.50 (1.73)	0.007
Nutrition	0–16	1.90 (1.71)	0.94 (p<0.001)	3.13 (1.46)	1.08 (1.38)	0.005
Beverages	0–10	1.30 (1.53)	0.97 (p<0.001)	2.13 (1.36)	0.75 (1.42)	0.04
Screen Time	0–9	2.20 (0.83)	0.53 (p=0.04)	2.75 (0.46)	1.83 (0.83)	0.01
Communication	0–13	1.50 (1.24)	0.96 (p<0.001)	1.38 (1.06)	1.58 (1.38)	0.72

Psychometric Properties of OST Policy Assessment Index

Inter-rater reliability of the index was good to excellent, ranging from r=0.84 to r=0.97 for four of the five domains and for the overall OSNAP score, though lower for the Screen Time domain (r=0.53, p<0.04) (Table 2). As hypothesized, at baseline, programs affiliated with the YMCA of Greater Boston had significantly higher policy scores in the domains of physical activity, snacks, beverages, screen time and OSNAP standards, providing evidence for measure construct validity.

Primary and Secondary Outcomes

Table 3 depicts the number of programs with written policy statements supporting the OSNAP Standards at baseline and follow up and the number of programs that had either written their own new policy statements or directly adopted policy statements supporting the OSNAP standards. Chi-square tests (p<0.05) suggest that, at follow-up, more intervention than control programs had newly written policy statements supporting OSNAP standards, particularly: 1) offering 20 minutes of vigorous PA three times per week, 2) banning trans fats from program-provided snacks, 3) requiring that the program serve a fruit or vegetable every day at snack, 4) banning SSBs from program-provided snacks, 5) requiring water to be served as the primary beverage at snack, 6) banning the use of recreational television or movies, and 7) limiting recreational computer time. Some examples of policy statements written by the programs included statements prohibiting any beverage except water to be sent in by families, statements specifying that sugary drinks and “junk food” would not be served at the program, and changes to schedules demonstrating that short, 10-minute PA breaks would be incorporated into daily homework time.

Table 3.

Number of programs with specific policy statements related to OSNAP standards at baseline (Fall 2010 and follow-up (Spring 2011), among 20 Boston afterschool programs.

	BASELINE		FOLLOWUP		CHANGE
	Intervention (n=10)	Control (n=10)	Intervention (n=10)	Control (n=10)	Intervention (n=10)	Control (n=10)
Policy requiring provision of at least 30 minutes of physical activity every day	5	4	8	5	3	1
Policy requiring provision of at least 20 minutes of vigorous physical activity at least 3 times per week	3	2	8	2	5	0
Policy requiring or encouraging that all children participate in physical activity	9	6	10	7	1	1
Policy banning trans fats from snacks served	2	3	8	4	6	1
Policy requiring a fruit or vegetable to be served every day at snack	1	2	7	2	6	0
Policy banning sugar-sweetened beverages from snacks served	2	3	8	4	6	1
Policy requiring water to be served as the primary beverage at snack	2	2	8	3	6	1
Policy banning sugar-sweetened beverages brought from outside the snack program	0	1	3	1	3	0
Policy banning the use of recreational television or movies	2	3	8	3	6	0
Policy limiting recreational computer use to less than 1 hour per day	3	3	8	3	5	0

After adjusting for the matched pair design and for baseline policy scores, the intervention resulted in significant increases in OST Policy Assessment Index scores from baseline to follow-up in the domains of Nutrition (+2.6, p=0.003), Beverages (+2.3, p=0.008), Screen Time (+0.8, p=0.046), Communication (+2.2, p=0.002) and the overall OSNAP Score standards (+3.3, p=0.02) (Table 4). The overall Physical Activity domain scores were not significantly improved in intervention programs at follow-up when compared with control sites.

Table 4.

Changes in average overall policy scores in intervention and control afterschool programs, Boston, MA, Fall 2010 – Spring 2011 (n=20)

	Baseline mean (SD)	Follow-up mean (SD)	Crude change	Adjusted difference at follow-upa	95% CI for adjusted change	p-value
OSNAP
Intervention	2.9 (3.0)	6.7 (3.2)	+3.8	+3.3	0.83, 5.77	0.02
Control	2.9 (3.7)	3.4 (3.6)	+0.5
Physical Activity
Intervention	3.9 (1.7)	5.1 (0.9)	+1.2	+1.0	−0.6, 2.6	0.183
Control	2.8 (1.9)	3.0 (2.1)	+0.2
Nutrition
Intervention	1.8 (1.5)	4.6 (1.0)	+2.8	+2.6	1.2, 4.1	0.003
Control	1.9 (2.0)	2.1 (2.0)	+0.2
Beverages
Intervention	1.1 (1.4)	3.3 (1.5)	+2.2	+2.3	0.8, 3.8	0.008
Control	1.5 (1.6)	1.6 (1.7)	+0.1
Screen Time
Intervention	2.2 (0.9)	3.1 (0.7)	+0.9	+0.8	0.2, 1.6	0.046
Control	2.2 (0.8)	2.3 (0.9)	+0.1
Communication
Intervention	1.3 (1.3)	4.1 (0.9)	+2.8	+2.2	1.1, 3.4	0.002
Control	1.7 (1.2)	2.1 (1.4)	+0.4

^aAdjusted difference at follow-up represents the difference in outcomes in the intervention compared to the control sites adjusting for baseline policy scores and controlling for randomization pair indicator variables (9 indicators).

DISCUSSION

Afterschool programs are a relatively understudied setting for the promotion of healthy behaviors to reduce obesity risk in children, despite their potential to influence millions of U.S. children. This randomized, controlled trial of an intervention in afterschool programs to build staff capacity to improve written policies regarding nutrition, PA, and ST demonstrates that the OSNAP intervention was effective at increasing the number of policy statements promoting specific healthy practices. In particular, efforts promoting policy language specific to the OSNAP standards, the key focus of the intervention, were successful: at follow-up, intervention sites’ written program policies supported an increased number of OSNAP standards (+3.3; 95% CI 0.83–5.77) compared to control programs. This study also developed a reliable research tool to evaluate written policy language related to OSNAP. It complements existing research tools to evaluate environmental and practice indicators in afterschool environments (Ajja et al 2012) by measuring multiple areas of policy content related to PA, snacks, beverages, ST, and communication, thus informing a more complete understanding of policy structures in afterschool programs.

Written policies have the potential to direct afterschool program practices to promote healthier school environments for children and youth. For example, a written policy specifying that a program will not allow televisions to be on during program time, if followed in practice, has the potential to reduce children’s exposure to ST. Studies have examined the impact of policy change on student behavior (Jaime & Lock, 2009; Cullen & Watson, 2009; Cradock et al, 2011), surveyed the current status of written school and child care policies (Falbe et al 2011; Schwartz et al 2009), and examined the role policy plays in reported implementation practice in schools (Chriqui et al 2010). This research further demonstrates the ability of a targeted afterschool intervention to improve written policies. The ability to improve policies suggests an avenue for afterschool programs to make sustainable changes to promote healthy environments and practices. However, in this study, improvements were largely seen in implementation- and communication-focused policy documents such as schedules, letters to parents, and posted rules. We observed fewer changes in more time-stable documents such as handbooks or joint-use agreements. While implementation documents are critical to informing daily practice, integration of new policy statements into handbooks or joint-use agreements may promote sustainability. Program staff noted that change in the latter type of document often occurs in the summer and thus may have occurred after our study concluded. An additional sustainability concern is that these policy changes were initiated at the program-level; the intervention did not focus on changing state, district, or provider-level policies. Involving multiple spheres of influence, including state regulatory bodies and district-level policymakers, and ensuring consistency across these levels, may help promote sustainability (Fixsen et al 2005), a concern for afterschool settings given staff turnover (Dennehy & Noam, 2005; Kelder et al 2005) and changes in participants from year to year.

This study also introduces the development of a tool for researchers studying obesity prevention policies in afterschool programs. The structure of the OST Policy Assessment Index allows researchers to systematically compare written program policies to indicators of established nutrition, PA, and ST standards and identify potential areas for strengthening and expanding program policies. The measure demonstrated moderate to excellent inter-rater reliability and also provided preliminary evidence for construct validity. While the OST Policy Assessment Index provides a means of describing the scope of written policies, it did not assess indicators of the monitoring, evaluation, and enforcement of these policies within the programs, as some measures for policies found in school and child care settings do (Falbe et al 2011; Schwartz et al 2009). Because the tool was designed to assess the presence of specific policy standards in the areas of nutrition, PA and screen time, it does not capture other policy standards that may also be important quality indicators. Further tool development may be necessary to improve content validity and expand the utility of the measure as additional evidence-based policies are developed.

This study was conducted in one urban area of the United States, limiting generalizability to afterschool programs in other settings and states. Additionally, this study did not evaluate whether the observed changes in policy at a given program were associated with changes in relevant practices or children’s behavior. Ajja et al. (2012) demonstrated cross-sectional associations between the presence of any policy statement regarding nutrition and higher servings of fruits, vegetables and whole grains, as well as between the presence of any PA policy and a higher number of pedometer steps in girls, and Beets et al (2013) have found cross-sectional negative associations between the presence of any PA policy and boys’ MVPA (Beets et al 2013). However, it is yet unknown how changes in afterschool policies impact corresponding program practices and environments or child behaviors. Future research might explore whether changes in specific policy language (e.g. a written statement specifying that SSBs are never served) result in corresponding changes in practices (e.g. actually not serving SSBs) and student behaviors (e.g. reduced intake of SSBs).

Conclusions

The OST Policy Assessment Index is a reliable, valid research tool that can be used to evaluate OSNAP-related written afterschool policies and assess policy change over time. This randomized, controlled trial demonstrates that the OSNAP intervention was successful at increasing the breadth of afterschool policies within the domains of nutrition, beverages, screen time, family communication, and related to the specific OSNAP standards that were the focus of the intervention activities.

Appendix. Items on the OSNAP Policy Assessment Inventory

Physical activity

• 1Any policy on physical activity is present.
• 2
  Policy requires 30 minutes of PA every day.

  1. Sub-item: If another amount of PA is required, how many minutes per day?
• 3
  Policy requires 20 minutes of vigorous PA at least 3 times a week.

  1. Sub-item: If yes, how is vigorous activity defined and encouraged?
• 4Policy requires or encourages that every child participate in PA.
• 5Policy requires or encourages variety in PA.
• 6Policy requires or encourages outdoor play when possible.
• 7
  Policy specifies weather conditions under which staff must take children outside.

  1. Sub-item: Specify details of weather policy (high temperature, low temperature, air quality)
• 8Policy requires indoor space for physical activity.
• 9Policy requires outdoor space for physical activity.
• 10Policy requires adequate age-appropriate PA equipment.
• 11Policy requires a coordinator of PA policies and programs (including for afterschool).
• 12A physical activity curriculum is required for students.
• 13The organization has a joint-use agreement for using physical facilities.

Nutrition

• 14Any policy on nutrition/food service is present.
• 15
  Policy restricts trans fats.

  1. Sub-item: Note whether trans fats are banned or limited. If only limited, describe limit.
• 16
  Policy addresses serving fruits/vegetables (not 100% juice) at snack time.

  1. Sub-item: Note whether policy stipulates that a fruit or vegetable must be served every day. If other, describe.
• 17
  Policy addresses serving whole grains at snack time.

  1. Sub-item: Note whether policy stipulates that whole grains must always be served. If other, describe.
• 18Policy encourages child participation in food prep for program snacks.
• 19Policy encourages family-style serving.
• 20Policy encourages a variety of foods be served.
• 21
  Policy addresses snacks in vending machines.

  1. Sub-item: Note whether policy bans vending machines entirely, bans unhealthy foods from vending machine, or other policy.
• 22Policy addresses adequate space for food storage.
• 23Policy addresses adequate space for food prep.
• 24
  Policy encourages healthy food be served at celebrations.

  1. Sub-item: Note whether policy encourages healthy foods at celebrations, limits the number of celebrations, bans unhealthy foods or all foods at celebrations, or other.
• 25Policy requires coordinator of healthy eating policies (including menus) and/or programs.
• 26Nutrition policies are applied to snacks supplied from outside the program (parents, guest programs, staff)
• 27Policy restricts students from bringing in outside food.
• 28The organization has a joint-use agreement for using kitchen facilities.
• 29A nutrition education curriculum is required for students.

Beverages

• 30Any policy on beverages is present.
• 31Policy prohibits sugar sweetened beverages (SSBs).
• 32
  Policy addresses the serving of water.

  1. Sub-item: Note whether policy stipulates that water is served as the primary beverage, water must be available at all times, water must be available during outdoor play, water must be easily accessible to children for self-serve, or other.
• 33
  Policy addresses milk served to children.

  1. Sub-item: Note whether policy prohibits flavored milk or whole milk.
• 34
  Policy addresses 100% fruit juice.

  1. Sub-item: Note whether policy prohibits 100% juice from being served, explicitly encourages 100% juice to be served, restricts 100% juice to 4 oz serving of less, or restricts 100% juice to a serving size greater than 4 oz.
• 35
  Policy addresses beverages in vending machines.

  1. Sub-item: Note whether policy restricts unhealthy beverages, puts vending machines with unhealthy beverages on timers, or other.
• 36Policy addresses adequate space for beverage and beverage equip.
• 37Policy prohibits SSBs at celebrations.
• 38Policy prohibits SSBs brought in from outside the snack program by parents or guest programs.
• 39Policy restricts students from bringing in outside beverages.

Screen time

• 40Any policy on screen time is present.
• 41
  Policy restricts TV/movie viewing.

  1. Sub-item: Note whether all TV/movie viewing is prohibited or only commercial TV/movie viewing.
  2. Sub-item: Note whether the length of TV/movie viewing is restricted.
• 42Policy restricts the presence of TVs.
• 43Policy restricts the presence of computers.
• 44
  Policy restricts computer use.

  1. Sub-item: Note whether all computer use is prohibited or only recreational computer use.
  2. Sub-item: Note whether the length of computer use is restricted.
• 45Policy restricts the presence of video game systems.
• 46Policy restricts video game usage.
• 47Policy restricts which websites can be accessed by children during program time.
• 48Policy restricts student use of handheld devices (cell phones, texting, etc).

Communications

• 49Policy encourages visual promotion of healthy foods.
• 50Policy requires staff to post healthy eating policies.
• 51Policy requires staff to inform parents of healthy eating policies.
• 52Policy requires staff to inform parents of changes to the snack menu.
• 53Policy encourages visual promotion of healthy beverages.
• 54Policy requires staff to post healthy beverage policies.
• 55Policy requires staff to inform parents of healthy beverage policies.
• 56Policy encourages visual promotion of healthy screen time behaviors.
• 57Policy requires staff to post screen time policies.
• 58Policy requires staff to inform parents of healthy screen time policies.
• 59Policy encourages visual promotion of PA.
• 60Policy requires staff to post PA policies.
• 61Policy requires staff to inform parents of PA policies.

OSNAP Score (pulled from across domains)

1. Policy requires 30 minutes of PA every day.

2. Policy requires 20 minutes of vigorous PA at least 3 times a week.

3. Policy requires or encourages that every child participate in PA.^*

4. Policy bans trans fats.

5. Policy stipulates that fruits and vegetables are served every day.

6. Policy bans sugar sweetened beverages (SSBs).

7. Policy stipulates that water is served as the primary beverage.

8. Policy prohibits SSBs brought in from outside the snack program by parents or guest programs.

9. Policy prohibits commercial TV/movie viewing.

10. Policy limits recreational computer time to less than 1 hour per day.

^*Note: In the original OSNAP standards, the standard of providing physical activity to all children was not separate from the standard to provide 30 minutes of physical activity every day. However, these standards were considered separate constructs in the OST Policy Assessment Index. Therefore, the total OSNAP policy score is 10, although the original OSNAP standards were conceptualized as 9 standards.