Making Healthy Eating and Physical Activity Policy Practice: The Design and Overview of a Group Randomized Controlled Trial in Afterschool Programs

Abstract

National and state organizations have developed policies calling upon afterschool programs (ASPs, 3-6pm) to serve a fruit or vegetable (FV) each day for snack, while eliminating foods and beverages high in added-sugars, and to ensure children accumulate a minimum of 30 min/d of moderate-to-vigorous physical activity (MVPA). Few efficacious and cost-effective strategies exist to assist ASP providers in achieving these important public health goals. This paper reports on the design and conceptual framework of Making Healthy Eating and Physical Activity (HEPA) Policy Practice in ASPs, a 3-year group randomized controlled trial testing the effectiveness of strategies designed to improve snacks served and increase MVPA in children attending community-based ASPs. Twenty ASPs, serving over 1,800 children (6-12yrs) will be enrolled and match-paired based on enrollment size, average daily min/d MVPA, and days/week FV served, with ASPs randomized after baseline data collection to immediate intervention or a 1-year delayed group. The framework employed, STEPs (Strategies To Enhance Practice), focuses on intentional programming of HEPA in each ASPs’ daily schedule, and includes a grocery store partnership to reduce price barriers to purchasing FV, professional development training to promote physical activity to develop core physical activity competencies, as well as ongoing technical support/assistance. Primary outcome measures include children’s accelerometry-derived MVPA and time spend sedentary while attending an ASP, direct observation of staff HEPA promoting and inhibiting behaviors, types of snacks served, and child consumption of snacks, as well as, cost of snacks via receipts and detailed accounting of intervention delivery costs to estimate cost-effectiveness.

Introduction

Across the nation, afterschool programs (ASPs) serve more than 8.4 million youth, the majority of which are elementary-age (6-12yrs).¹ Programs run every day of the school year, and start immediately following the end of the regular school schedule, lasting for an average of 3 hours each day.² This widespread reach and length of contact has made them a focus point of childhood obesity prevention efforts, in recent years. As part of their daily schedule, ASPs offer a snack and opportunities for children to be physically active. Unfortunately, the snacks served in ASPs are characterized as high in sugar, salt, and/or fat, and are almost devoid of fruits or vegetables,^3-7 whereas the amount of physical activity children accumulate falls well below existing policy standards.^8-10

To address this gap between policy and practice, both state and national organizations have developed, adopted, or endorsed policies outlining the nutritional quality of snacks served and the amount of physical activity children should accumulate while attending an ASP.^{11, 12} While such policies are necessary and have the potential to make a major contribution to children’s total daily healthy eating and physical activity (HEPA), there are few effective and scalable strategies ASP providers can use to meet the HEPA policies.^{2, 5, 13-21} Previous interventions targeting healthy eating, physical activity, or both within the ASP setting have had mixed results. Policy-focused interventions^{3, 6} have shown that substantial changes in the quality of snacks served can be achieved from adopting a policy that defines the types of snacks ASPs should serve. However, these studies have failed to address a major barrier in serving healthier snacks – cost.^21-24 Another study^{5, 14} did not make changes in the quality of snacks served, with this largely due to cost and the lack of priority of serving a more healthful snack, like a fruit or vegetable. Three physical activity interventions in the ASP setting have reported modest increases in physical activity through the use of pre-packaged curricula.^{5, 13, 17} However, five studies reported that pre-packaged curricula do not increase children’s physical activity compared to ASPs not using the curricula.^{14-16, 18, 20} Moreover, pre-packaged curricula can be costly²⁵ or contain a large number of unfamiliar games that are difficult for unskilled staff to play.²⁰ Moreover, no physical activity interventions in the ASP setting have explicitly investigated the impact on clearly defined policy goals.

This paper describes the study design and conceptual approach of a large scale group randomized controlled trial, Making HEPA Policy Practice. The goal of this study is to evaluate the effectiveness of HEPA strategies, which consist of a multi-step, adaptive intervention²⁶ approach, that addresses price barriers to serving more healthful snacks and professional development training to develop core competencies to promote physical activity to meet the NAA Healthy Eating Standards and the California After School Resource Center Physical Activity Guidelines. Additionally, detailed information on the delivery of intervention will be collected to estimate cost-effectiveness.

Methods

Study Design

The study was designed and will be reported according to the CONSORT guidelines for cluster randomized controlled trials.

A total of twenty ASPs (cluster/group level) will be randomly selected and recruited fall 2012 from a pre-existing list of 535 program providers within a 1.5hr drive of the university. The list was provided by a state-level organization responsible for policy and resources for ASPs. For this study, ASPs are defined as child care programs operating immediately after the school day, every day of the school year for a minimum of 2 hours, serving a minimum of 30 children of elementary age (6-12yrs), operating in a school, community, or faith setting, and providing a snack, homework assistance and completion time, enrichment (e.g., arts-n-crafts), and opportunities for physical activity.²⁵ Programs that are singularly focused (e.g., dance, tutoring) and/or physical activity focused (e.g., sports, activity clubs), are not eligible for participation. We will measure the physical activity of at least 1,300 children enrolled across the 20 ASPs (65 per ASP). All children enrolled, staff, and ASP leaders in the ASPs are eligible to participate in the study. The only exclusion criterion for children to take part in the physical activity assessment (i.e., accelerometry) is the inability to be physically active without an assistive device (e.g., wheel chair, crutches). No other exclusion criteria will be imposed on any of the study procedures.

The design is a repeated cross-sectional group randomized controlled trial with a delayed treatment group. This design is appropriate when outcomes are tracked at a group level (e.g., ASPs), instead of at the individual level (e.g., children) ^{27, 28} and is consistent with recent large scale trials of site-level interventions for children and adolescents.^{5, 29-33} The study will take place over 3 years, with one year of baseline (i.e., year 1), and two years of intervention (i.e., year 2 and 3). The twenty ASPs will be randomized into one of two conditions: 1) immediate HEPA strategies or 2) 1-year delayed group. The immediate intervention group will receive the HEPA strategies (outline below) over 2 years (i.e., year 2 and 3), while the delayed group will receive the HEPA strategies during the last year (i.e., year 3) of the study. This design allows for the testing of the effects of the HEPA strategies compared to routine practice (i.e., between group differences from baseline to end of year 2 of the intervention) and the additional improvements achieved in HEPA from receiving 1 vs. 2 years of the intervention (i.e., between group differences from baseline to end of year 3 of the intervention). All outcomes will be modeled and expressed as changes occurring at the ASP level – the unit of randomization.

All measures will occur during the spring of each year. Across the study, we anticipate 24% of the children will leave the participating ASPs each year for reasons unrelated to the study (e.g., family relocating, transitioning from elementary to middle school). Further, based on our pilot work ³⁴ we anticipate that almost two thirds of the children will be present at two of the three measurement occasions and that an adventitious cohort of ~30% of the children will be present at all 3 measurements.

Informed Consent

Each ASP will provide parents information on the nature of the study and the child level measurements (physical activity and height and weight) to be collected, prior to enrolling a child in the ASP. Parents will be able to opt their child out of participation in the child-level assessment. A detailed list of these parents will be maintained by the ASP and provided to research staff prior to data collection. Each eligible child will be asked to verbally assent in front of ASP and research staff to participating in the data collection. Additionally, information regarding the study will be placed in parent handbooks, signup pamphlets, and posted on the ASPs’ websites.

Randomization and Pair Matching

Randomization of the 20 ASPs to immediate (n = 10) vs. delayed treatment (n = 10) groups will be performed after baseline data collection, summer 2013, using a random number generator. Programs will be match-paired based on enrollment size, average levels of MVPA/d, and number of days per week (out of a 5 day week) a FV is served. Enrollment size will be selected to ensure comparable group composition on a marker of organizational complexity (e.g., operating an ASP of 30 children is less complex than operating an ASP serving >150 children/day). Both MVPA and FV will be identified as pertinent matching variables because they serve as the primary outcomes of interest for HEPA and, based on prior work, are the most difficult (i.e., MVPA) ^{5, 13, 15, 16, 18, 20, 25, 34, 35} or costly (i.e., FV) ^{21, 22} outcomes to change in this setting.

Policy Benchmarks for Evaluation

Over the past 10 years, a number of healthy eating and physical activity policies for the ASP setting have been developed and endorsed at the state and national level.^{11, 12} Of these, the two most promising policies are the Healthy Eating Standards from the National Afterschool Association (NAA) and the Physical Activity Guidelines from the California After School Resource Center and California Department of Education. In 2011, the NAA developed the Healthy Eating Standards which call on ASPs to serve, on a daily basis, a fruit or vegetable, eliminate foods and beverages that are high in added sugar, and avoid foods and beverages containing artificial ingredients. In 2009, the California Department of Education and California After School Resource Center developed the California After School Physical Activity Guidelines which state that all ASPs ensure children engage in a minimum of 30 to 60 minutes of MVPA when the program is in session. The importance of these two policies (referred hereafter as the Healthy Eating and Physical Activity [HEPA] Policies) are reflected in the clearly defined programmatic (e.g., serve a FV every day) or behavioral (e.g., children accumulate 30min MVPA/d) goals and their ability to meaningfully contribute to children’s overall dietary intake and provide at least half of the recommended daily minutes of MVPA. These policy benchmarks will serve as primary outcome targets that the ASPs will work towards and be evaluated by in this 3 year study.

Intervention – Healthy Eating and Physical Activity Strategies

Strategies To Enhance Practice (STEPs) for HEPA

The design and delivery of the HEPA strategies will be based on the STEPs conceptual framework, developed by our research team, which involves a multi-step, adaptive intervention²⁶ approach to incorporating the HEPA strategies into daily routine practice. The approach consists of identifying essential ASP characteristics that represent fundamental building blocks which function as necessary programmatic components to achieving full integration of the HEPA strategies and eventual achievement of HEPA policies. The approach considers each individual ASP as a separate setting, even when an ASP might be part of a larger organization (e.g., YMCA, Boys and Girls Club). This approach is conceptually analogous to Maslow’s hierarchy of needs³⁶ and nonspecific hypothesis in psychotherapy.^{37, 38} Additionally, STEPs is informed from a systems framework for translating childhood obesity policies into practice in ASPs,² the extensive work within the ASP setting,^{8, 9, 11, 12, 24, 25, 34, 39-54}and is consistent with the growing literature on systems capacity building.^{2, 55, 56}

The overall framework of STEPs is informed by the hierarchy of needs from Maslow³⁶ and applied to the ASP organizational setting. Within this perspective,³⁶ basic human physiological needs serve as the first and primary need to be satisfied, without which, higher order needs such as self-actualization, cannot be achieved. Applying this to the ASP setting, the primary and initial need centers on daily programmatic structure, such as having a daily schedule of programmed activities, a weekly snack menu, and keeping to these schedules, must be met prior to addressing HEPA related issues and eventual compliance. Where ASPs do not have schedules or menus, or they exist yet fail to adhere to them, all resources and capacities of individuals within the setting – leaders and frontline staffers – will be directed towards occupying the attention of 30 to up to 200 children over a span of 3 hours each day, for roughly 180 days of the year (length of an average school year). In this scenario, other priorities and the ability to achieve them, such as creating a healthy eating and physical activity friendly environment, are pushed to the background or forgotten altogether. Attempting to intervene on the snacks served or the amount of physical activity children accumulate within an ASP such as this would be viewed as unimportant because the basic needs remain unmet.

The STEPs approach is also based on the concept of nonspecific factors from the field of psychotherapy.^{37, 38} In psychotherapeutic research, nonspecific factors are those elements of the therapist/patient relationship, such as interaction qualities, that are universal (all therapies rely on a patient interacting with a therapist) and therefore, not specific to any one therapeutic technique, yet account for a portion of the psychotherapies’ effectiveness.³⁷ Applied to the ASP organizational setting, nonspecific factors are the essential programmatic elements (e.g., schedules, menus, budget – see below for more detail), and are considered “universal” across all ASP settings – meaning they must exist for basic program delivery to occur. Their identification, evaluation, and where necessary, modification, within the STEPs continuum can lead to greater enhancement of the intervention – introduction of specific intervention techniques (e.g., professional development training, grocery store partnership).

The STEPs approach also departs from traditional intervention models that are based on a pre-defined package of intervention components all provided identically to those individuals or settings allocated to a treatment condition.⁵⁷ The field of intervention research is moving away from “static” interventions to ones that allow for more local/setting specific tailoring.^{26, 58} To this end, STEPs recognizes that each ASP is unique and, therefore, will require some similar and some different resources/strategies to achieve the HEPA policies (i.e., there is no “one size fits all” intervention). The approach taken in STEPs is one where some degree of local tailoring will occur that is both responsive and adaptive to the characteristics of each ASP.^{26, 58} This tailoring should assist with the local relevancy of the HEPA strategies, and subsequent uptake/integration of them into daily practice. STEPs is designed so that any one ASP can enter anywhere along the continuum, with the understanding that some ASPs will enter at a lower level indicating the need for greater technical assistance to achieve the HEPA policies versus those programs that enter at a higher level (see Figure 1 and 2). The conceptual framework for both STEPs for healthy eating and physical activity are presented in Figure 1 and 2 and described separately below.

Figure 1.

Strategies To Enhance Practice for Healthy Eating Conceptual Framework

Figure 2.

Strategies To Enhance Practice for Physical Activity Conceptual Framework

STEPs-Healthy Eating (STEPs-HE)

The STEPs-HE 1 through 4 represent the foundational building blocks of serving healthier snacks and focus solely on the program leader as the primary target of the process of integrating the HE Standards into routine practice. The first step of the healthy eating strategies begins with the identification of a schedule/menu of daily/weekly snack offerings. The menu serves as a guideline for the types of snacks to be purchased and served on a day-to-day basis and, therefore, represents the initial programmatic building block of intentionally serving foods and beverages that meet the HE Policies. The second step is to determine whether an ASP follows the provided snack menu. This represents an important indicator of program integrity whereby snacks are identified as served via the menu and are served within a given day or week. Thus the menu reflects practice. STEPs-HE 3 and 4 consist of identifying the budgeted amount for snack purchases, either daily in a cost/snack/child/day or annually (typically across the school year), and the location(s) where snacks are purchased. These two steps provide important decision-based information regarding cost (how much can be spent on snacks) and convenience (travel time to purchase snacks).

Technical assistance at STEPs-HE 1 through 4 focuses on developing a 2 or 4 week rotating snack menu that clearly defines the snacks to be served and their respective serving size. Research staff will work collaboratively with ASP leaders to modify existing snack menus to have them conform to the HE Policies. This will take place during June/July 2013 for approximately 2hrs. Where snack menus and the snacks served are incongruent, technical assistance will be provided to determine challenges associated with serving the menu-specified snacks. Additional technical assistance will include the collection of snack purchase receipts over a single month and subsequent computation of monthly and per snack/child/day costs. Where programs are identified as having a limited budget and/or are purchasing snacks from retailers where the retail cost of snacks that meet the HE Policies is prohibitive, support will be provided to link ASPs with an existing network of grocery stores that provide a discount on snacks that meet the HE Policies.^{24, 48} As described in our pilot studies,^{24, 48} the grocery store partnerships consist of grocery stores that provide a discount pricing structure (e.g., cost plus or percentage off retail price) on items that meet the HE Policies. The premise of the partnership is for programs to be able to purchase snacks from a convenient location, close to where a program operates, at a price that allows a program to maintain its current snack expenditures. Price is considered one of the major barriers to purchasing more healthful snacks.²¹ Through this partnership, ASPs can maintain current expenditures, minimize travel distance to purchase snacks, and improve snack quality to meet the HE Policies.^{24, 48} Where access to a partner grocery store is not feasible, most likely due to travel distance, alternative food purchasing outlets will be identified to assist ASPs in maintaining snack expenditures while meeting the HE policies.

STEPs-HE 5 focuses on the snacks served by categorizing them into existing classifications ^{6, 24, 48} and determining the extent to which a program meets the HE Standards. This information will be utilized to determine necessary changes to the snack menu, cost reductions through elimination of unnecessary or costly items (e.g., eliminate flavored beverages – serve only water),^{22, 59} and snacks already served that meet the HE Policies. STEPs-HE 6 represents the first step where direct delivery of the strategies will be focused on the frontline staff employed at the ASP. The frontline staff are those individuals who are directly responsible for interacting with the children on a daily basis, throughout the duration of the program. The HE Policies call upon the staff to serve as role models for healthy eating by refraining from eating or drinking inappropriate foods, such as fast food, candy, or sugar-sweetened beverages while the program operates. Nutrition education is an important part of creating an overall healthy eating environment in ASPs. Staff will be trained to delivery of nutrition education materials for a minimum of once per week. Staff will also be provided with pre-existing, freely available nutrition education materials and trained on their delivery. Example materials include My Plate from the USDA and the Food and Fun Curriculum. Staff will be encouraged to incorporate nutrition education during snack time on a designated day each week.

Trainings (healthy eating and physical activity), lasting a total of 3 hours, will occur at the beginning of the school year (August 2013) along with 4 booster sessions (occurring simultaneously with the physical activity boosters – see below for details) per ASP, each lasting the entirety of the program. Booster sessions will include a walkthrough with the program leader to review opportunities to meet the HE Policies. Research personnel, site leaders and staff will convene a 20 to 30 minute meeting immediately after the end of the ASP to discuss areas that are consistent and inconsistent with meeting the HE Policies. Strategies to address challenges will be agreed upon and implemented in subsequent days.

STEPs for Physical Activity (STEPs-PA)

The foundational building blocks of the achieving PA Standards are represented in STEPs-PA 1 through 4 (see Figure 2). These target the ASP leader, the individual directly responsible for day-to-day operations of the ASPs, and evaluate whether essential programmatic elements are in place. Initially, the first and second steps determine whether ASPs have a schedule of daily programming and the extent to which it is followed. Both of these elements signify important indicators of intentional programming at every level, beyond physical activity, while also providing an indication of program integrity whereby activities that are scheduled, such as enrichment, homework assistance, and physical activity, are scheduled and occur at a given time and day. Thus, the scheduled activities reflect practice. STEPs-PA 3 and 4 consist of evaluating program schedules to identify where, if any, physical activity opportunities are made available. If physical activity opportunities are made available then how often they occur during a typical week (never, some of the days, everyday), and the amount of time allotted for physical activity within a daily schedule is determined. As a basis for achieving the 30 minutes of MVPA/day during the ASPs, a program needs to ensure it schedules physical activity daily for a minimum of 60 minutes.⁶⁰

Technical assistance for STEPs-PA 1 through 4 will focus on professional development training targeting ASP leaders to develop high quality schedules that include daily offerings of physical activity. The workshop will consist of working with ASP leaders to develop a one to two week rotating schedule that incorporates the following descriptive information: time activity occurs, indication of scheduled activity, location activity takes place, equipment/materials required to conduct activity, and staff responsible for delivering activity. These workshops will focus both on scheduling physical activity and non-physical activity (e.g., enrichment) opportunities. The workshop will occur during summer 2013 and last approximately 1 to 2 hours, depending on the amount of assistance ASP staff and site leaders require. The physical activity workshop will occur in conjunction with healthy eating.

Once STEPs-PA 1-4 are achieved, or if an ASP already provides daily physical activity for the requisite amount of time, STEPs-PA 5 and 6 will be implemented. These steps will be delivered to all ASPs. STEPs-PA 5 focuses on working with the ASP leader to ensure the scheduled activity offerings appeal to both boys and girls by providing a girls-only physical activity opportunity and organized physical activities.^{43, 61} Both components are associated with higher overall activity levels, and associated with higher activity levels of girls.^{43, 45, 61} The final STEPs-PA focuses on professional development training for frontline staff, those individuals that directly interact with children on a daily basis. Staff will be provided with a 3 hour training in August 2013, prior to the beginning of the school year. The training will focus on the LET US Play principles of removing lines from games, eliminating elimination, reducing team size, getting uninvolved staff and children involved in activities, by modifying the space, equipment, and rules of the games/activities commonly played in the ASP setting. The trainings will consist of presentation on the LET US Play principles, video demonstrations of traditional and LET US Play modified games, and hands-on involvement in games presented in both the traditional and LET US Play format.^{61, 62} These staff behaviors and components of games will be identified as primary barriers to maximizing children’s physical activity and have been shown to be modifiable through professional development training.^{46, 51-53, 61, 62}

All of STEPs-PA 1-6 will occur prior to the beginning of the school year fall 2013. In addition, 4 booster sessions per ASP, conducted by a single person, each lasting for the entirety of a single ASP operating day (e.g., 3-6pm), will occur from September 2013 to February 2014. The booster session will include a walkthrough of the ASP with the site leader to identify physical activity opportunities and their consistency with LET US Play principles. Research personnel, site leaders, and staff will convene a 20 to 30 minute meeting immediately after the end of the ASP to discuss areas that are consistent and inconsistent with meeting the PA Standards. Strategies to address challenges will be agreed upon and implemented in subsequent days. It is important to note that the STEPs for PA intentionally and deliberately does not suggest games or activities for ASPs to schedule and play. This decision not to provide ASP leaders and staff with a suggested list of games is based on the following: 1) the majority of ASPs already play some types of games, even during free time when children organize their own games; 2) the number of unique games, not variations of the same game such as different types of tag games, is no more than 15 to 20 games; and 3) when ASPs are provided extensive game resources from pre-packaged physical activity curricula, few are ever incorporated into scheduled physical activity opportunities.²⁰ All ASPs will receive steps 4 thru 6 for the STEPs to HE and steps 5 and 6 of the STEPs to PA.

Primary Measures

A detailed timeline of major project events and measurements is presented in Table 1. All measurements will occur during the spring (March-April) of each year. Measures will take place on days when the weather is conducive for outdoor activities. This decision is deliberate since inclement weather, due to its infrequent occurrence, cannot be balanced across groups. Consistent with previously established protocols, each ASP will be visited for data collection on 4 non-consecutive, unannounced randomly selected days Monday through Thursday.^{8, 9, 34, 44} Fridays are not assessed because children typically do not have homework over the weekend, and therefore, the schedule of the ASPs is altered in comparison to the schedule of activities occurring on all other week days.

Table 1. Timeline of intervention events and measures.

Physical Activity

The primary physical activity and sedentary behavior outcome will be accelerometry derived. Children attending the ASPs on the days of measurement will have the opportunity to wear the ActiGraph GT3X+ (Shalimar, FL) for up to 4 days. The accelerometers will be distilled using 5 second epochs to account for the intermittent and sporadic nature of children’s physical activity ⁶³ and to improve the ability to capture the transitory physical activity patterns of children.^{64, 65} Upon arrival to the programs, children will be fitted with an accelerometer by research staff and the arrival time recorded (monitor time on). After affixing the accelerometer to the participants’ waist with an elastic belt, children will be allowed to participate in their normal ASP activities. Research staff will continuously monitored the entire ASP for compliance in wearing the accelerometers. Before a child departs from a program, research staff will remove the elastic belt and record the time of departure (monitor time off). Children will wear the monitors for their entire attendance at the ASPs. This procedure will be performed throughout the duration of the study and is consistent with prior studies evaluating physical activity in ASPs.^{8, 9, 34, 43, 44, 50} Physical activity data will be collected Mondays through Thursdays. Cutpoint thresholds associated with moderate and vigorous activity will be used to distill the physical activity intensity levels ⁶⁶ and sedentary behavior ⁶⁷. Children will be considered to have a valid day of accelerometer data if their total daily wear time (off time minus on time) is equal to or greater than 60 minutes.^{8, 9, 68} Children with a minimum of a single day will be included all analyses.^{9, 61} The primary physical activity outcome of interest will be expressed as a dichotomous variable for each day representing ≥30 min/d of MVPA versus <30 min/d of MVPA.⁶⁹ This is consistent with the California ASP policy goal of 30 min/d of MVPA and the guideline specifying each day, not the average of days.⁶¹ Of secondary interest will be time spent in MVPA and sedentary expressed a continuous variables (i.e., minutes).⁶¹

Snacks Served

The types of foods and beverages served as snack will be recorded via direct observation by trained research personnel. On each measurement day, immediately at the start of snack, a trained observer will record the brand name(s), size, and packaging, where appropriate, of the foods and beverages served as snack for that day. Foods and beverage items served as snacks will be classified according to existing categories for snacks and beverages:^{12, 70} sugar-sweetened beverages (e.g., soda, powered drink mixed, sport drinks), dairy food unsweetened (e.g., string cheese); dairy food sweetened (e.g., Trix yogurt); milk unsweetened (non-fat, 1%, 2%, and whole); milk sweetened (e.g., chocolate, strawberry); 100% fruit juice; salty flavored snacks (e.g., Doritos, Chex Mix), salty unflavored snacks (e.g., pretzels, plain corn tortilla chips); desserts (e.g., cookies, pop tarts); candy (e.g., chocolate, frozen treats); non-fruit fruit (e.g., fruit roll ups; fruit leather); prepackaged fruit (e.g., applesauce, fruit in syrup); cereal sugar-sweetened (e.g., Fruit Loops); cereal unsweetened (e.g., Cheerios); and fruits and vegetables (e.g., fresh, frozen, dried). Water will be recorded if programs provided water in cups or bottles during snack time. All snack items served will be expressed as days served per week, ranging from 0 to 5 days/wk.^{6, 23} Inter-rater agreement on the snacks served was ##% across ## observations of snacks

Snacks Consumed

Consumption of snacks will be collected using a modified direct observation protocol.^{23, 71} During snack, children sit in groups of three or more. At each unannounced site visit, trained research staff will randomly selected a group of children. Within this group, no more than five children will be randomly selected and observed for the entire duration of the snack time (approximately 15mins). During this time, a single observer will record what the children were served for snack and indicate whether each child consumed the snack. Consumption will be operationalized as observing a child eating 50% or more of an offered snack item. For instance, if children are provided a whole piece of fruit, a child will be classified as consuming the fruit if researchers observe that the child has eaten at least half of the fruit. Where children do not eat any of the snack or only take several bites, consumption will be recorded as zero (i.e., not consumed). Previously established inter-rater consumption reliability for 107 children served 217 snacks was high (κ= 0.89 and percent agreement 97%).²³

Snack Cost

Costs of the snacks will be estimated based on receipts ASPs will provide from March through May 2013 at baseline and during October, February, and April for intervention years 1 and 2.²³ For each individual snack item, cost per snack served will determined using standard serving sizes.¹²

Staff HEPA Behaviors

Context of the afterschool program, staff behaviors and the structure of the physical activity opportunities will be measured via direct observation using the System for Observing Staff Promotion of Activity and Nutrition (SOSPAN).^{72, 73} This instrument is based on a momentary time sampling and is designed to measure staff behaviors and the structure of physical activity opportunities that either promote (e.g. verbal promotion, modeling physical activity) or discourage (e.g. verbal discouragement of physical activity, staff leading elimination games) children’s physical activity.⁷³ The instrument also provides information on the context of the afterschool programs’ scheduled activity (i.e., physical activity, snack, enrichment, academics). Observation will occur during outcome data collection in the spring of each year on the unannounced nonconsecutive weekdays (Mon-Thurs). Additionally, SOSPAN scans will be performed on 4 unannounced nonconsecutive weekdays during the fall of intervention year 1 and 2 in both the immediate and delayed intervention groups. These fall measurements will serve as process evaluation indicators for integrating the HEPA strategies into routine practice.

A schedule of the daily activities will be collected at the beginning of each observation day. The SOSPAN scans will be completed continuously one after another from the beginning to the end of the program (~3pm-6pm). Prior to observation, each site will be visited to determine available spaces in which the program activities could occur. These spaces will be referred to as “target areas”.^73-75 Target areas that are occupied by ASP attendees and staff will be continuously scanned throughout the day. Target areas that are available to the ASP, yet unused during the program, will not be scanned. Five SOSPAN scans will be completed in each occupied target area prior to the observer moving to the next target area. A representative sample of all of the activities occurring over the course of one afterschool program day will be collected by systematically rotating through the occupied target areas and continuously completing SOSPAN scans.⁷³

Intervention Cost

Delivery costs (i.e., resource use) associated with implementing the HEPA strategies will be compiled over the duration of the project.⁶¹ This will include all costs incurred in delivery, such as supplies and materials utilization (e.g., printing of materials for parents, staff), equipment purchases, training costs (e.g., hourly wages for employees), and costs associated with providing ongoing technical support (e.g., travel for site visits, time allocated to weekly phone calls). Delivery costs will be compared to the usual costs associated with delivering the OST programs without the addition of the HEPA strategies and will be based on baseline costs (i.e., Standard Practice – no HEPA Strategies). The information for cost of standard practice will be provided by the ASPs. Costs related to the development of materials and the evaluation of the strategies will be excluded in order to capture the true cost of replicating the strategies across other ASPs.

Net costs associated with delivering the HEPA Strategies will be calculated by subtracting the costs for Standard Practice from the delivery costs of the HEPA Strategies. Note that the costs will be estimated from the point of view of the program rather than from societal perspective. The cost estimates will provide information on resource needs of the program so that costs of scaling up of the program can be calculated. One of the simple ways of expressing the cost numbers is to indicate the cost per child enrolled or per child-day of participation. Once the additional costs are known, policy makers will be able to decide what proportion of costs can be mobilized through user charges. The higher the user charge (amount of money to be paid by parents for participating in the program), the lower the acceptance is likely to be, especially from lower socioeconomic categories. Therefore, policy-makers need to carefully evaluate the alternative mechanisms of mobilizing the resources needed. To guide policy-making, this study will also conduct a willingness to pay survey among the participants. Standard downward or upward or mixed bidding process can be used to determine the level of willingness to pay of parents.⁷⁶ The effectiveness of the HEPA Policies with and without the HEPA Strategies will be measured using two outcomes: changes in MVPA (physical activity) and changes in the nutritional quality of foods consumed (healthy eating). We will calculate an Incremental Cost Effectiveness Ratio (ICER) using the following formula: ICER = ($ HEPA Strategies – $ Standard Practice)/(Effect HEPA Strategies – Effect of Current Practice). The cost-effectiveness ratio will be used to provide decision-makers some idea on the cost of the program per unit of outcome achieved. This ratio will be used for advocacy purposes.

Secondary Measures

ASP Policy Characteristics

The policy characteristics of the ASPs will be assessed using the Healthy Afterschool Activity and Nutrition Document tool (HAAND). The HAAND is a rubric-based index designed to quantify the physical activity and nutrition environment within ASPs.⁷⁷ The HAAND consists of two sub-indices and corresponding rating scales – Healthy Afterschool Program Index for Physical Activity and Nutrition (HAPI-PA and HAPI-N). Each sub-index consists of 6 elements (policies, training, child involvement, evaluation, curriculum, scheduling activity or quality of snacks served) identified through published quality rating scales for ASPs, standards for ASPs, core competencies, and indexes developed for other environments (child care setting).^78-80 Additionally, key stakeholders (ASP leaders, staffers) will be interviewed and contribute to the elements appearing on the existing HAPI-PA and HAPI-N. The scores on the individual elements of the HAPI-PA and HAPI-N can be used individually or can be summed to provide an overall rating ranging from 0 to 23 (HAPI-PA) or 30 (HAPI-N). These scores (overall and individual items) will be used as site-level indicators of supporting environments for physical activity and healthy eating. The inter-rater reliability of the HAAND elements is ≥90% agreement.

ASP Physical Environment

The size of the physical environment of each ASP will be collected using two methods. Based on the site directors self-report, all areas used for physical activity space (e.g. gym, open green space, courts etc.,) and non-physical activity space (e.g. classrooms, cafeteria, etc.,) will be identified, divided into target areas, and measured for physical size. Used indoor and outdoor space will be verified by the program site director and direct observation via SOSPAN. Indoor physical activity area (ft²) will be measured using a measuring wheel (RR112 4“ Keson RoadRunner, Aurora, Illinois). Google Earth software will be used to obtain aerial pictures (top down) of the outdoor area used for physical activity and polygon measurement tool will be used to the draw target area boundaries. Estimates of the outdoor spatial area (acre) will be calculated using Geographical Information Systems software (GIS).^{81, 82}

Anthropometric Measures

We will measure children’s height and weight using standardized, widely accepted protocols^83-90 during the spring measurement periods. Height and weight will be transformed into BMI age-sex specific percentiles⁹¹ – which will accommodate for differences in ages when comparing BMI of children.^{83, 84, 91}

Process Evaluation

Process measures will be collected in both the immediate and delayed intervention ASPs during the fall of intervention year 01 and 02 on 4 nonconsecutive unannounced days (Mon-Thur). As described previously, process measures will include SOSPAN⁷² and direct observation of snacks served and consumed by children.²³ Additionally, we will collect and maintain detailed records on the number of staff attending beginning of the year training and booster sessions, and technical assistant requests. We will also conduct structured interviews in both the immediate and delayed intervention ASPs with each ASP leader (the person who is responsible at the site level for staff and children – daily operations) and one frontline staffer (individual responsible for daily interactions with children attending the ASP). The frontline staffer will be randomly selected from the staffers employed at each ASP. Interviews of ASP staff members and leaders will be conducted by a trained research staff member. The interview will be semi-structured and include assessment of possible influences on adoption and implementation of strategies such as leadership support, engagement of ASP formal change mechanisms, and identifying and addressing organization-specific contextual factors. The interview will also include assessment of possible influences on implementation such as perceived tangible support from leadership, fit with schedule and routines, fit with job tasks, confidence and comfort with skills need to implement, and working environment/climate.

Analysis Plan

An important consideration for the proposed intervention elements that target physical activity is the evaluation of ASPs, rather than tracking individual children over time. Hence, hypothesized increases in child physical activity will reflect group-level changes of children who attend an ASP that implements the HEPA strategies versus the control standard practice ASPs. For the primary physical activity outcome, the design is a three-level model, where level 1 represents multiple accelerometer-derived PA measurements (4 days at baseline and 4 days at each successive post-test) nested within each child (level 2), and children nested within ASPs (level 3).⁶¹ The treatment effect will be estimated at level 3, the unit of randomization and treatment allocation (i.e., intervention). Power calculations were performed using Optimal Design HLM Software (v.2.0). With an average of 67 children per ASP, a total of 20 ASPs (estimated total sample size for children = 1300), a level 3 ICC of 0.09, a level 2 ICC of 0.63, with a level 3 covariate (HAAND, SO-SPAN scores) explaining 0.35 variance, the study has a power of 0.85 to detect a 14% increase in the dichotomized MVPA outcome (ES = 0.35). The variance estimates are based on cross-sectional study of 19 ASPs and 812 children.¹⁰

The MVPA outcome will be expressed as continuous (change in minutes) and binary (proportion of children meeting 30min MVPA daily policy goal). Both linear and non-linear terms will be modeled to account for differences in change from baseline to the end of the two years of intervention.⁶¹ Included in the models will be a time-varying covariate of daily time in attendance for each child. Analyses for changes in accelerometer-derived MVPA and time spent sedentary will be analyzed separately for boy and girls.⁶¹ Additionally, secondary analyses will be conducted on children that are present at baseline and post intervention assessments.⁶¹

Changes in the types of snacks served will be modeled at the ASP level (N = 20). Differences between baseline and Year 02 and 03 average days served per week for each food category will be evaluated using a mixed model regression that examines the change in average days served/wk between baseline and end of the two year intervention. The model will include a treatment variable and time variable and the treatment-x-time interaction. Power will be estimated using G-Power (v.3.1.0) with a 2 level model (time nested within sites) and a sample size of 20, a correlation among repeated measures of 0.5, has the power to detect an effect size of 0.25 (10% increase) at an alpha level of 0.05 with a power of 0.80.

For both changes in physical activity and types of snacks served, intent-to-treat (ITT) models will be estimated and serve as the main outcome analyses. This modeling approach will include all children present at baseline and post-assessments (for physical activity) and all snacks served across ASPs. Additionally, the ITT models will assume all intervention ASPs received and were compliant with the intervention as delivered and that control ASPs did not substantially change HEPA programming. A second series of models will be estimated for changes in children’s MVPA by using only those children present at all measurement occasions. Analyses will also investigate the influence of implementation on changes in the primary outcomes at the child-level (physical activity) and ASP-level (snacks). Implementation predictor variables will be changes in staff behaviors as measured via SOSPAN, changes in the policy environment via the HAAND tool, and attendance at the trainings and the amount of technical assistance received. Document reviews of menus and daily operating schedules will also be made to evaluate changes in programmatic structure as a result of the intervention.

Missing outcome data will be handled using full information maximum likelihood estimators to account for children not measured at baseline or at post-assessments. Missing covariate data will be handled using a complete case analysis and multiple imputation.^{92, 93} To determine if differential attrition bias exists, we will examine attrition between the cross-sectional and longitudinal measures for movers and stayers according to established contrasts.⁹⁴ All analyses will be performed using Stata (v.13.0 College Station, TX).

Discussion

In this paper we describe the design and conceptual approach of a group randomized controlled trial evaluating the effectiveness and cost of delivery of implementing a multi-step, adaptive intervention²⁶ to incorporating HEPA strategies into daily routine practice to achieve existing public health policies targeting the nutritional quality of snacks served and the amount of MVPA children accumulate while attending an ASP. Importantly, this study will be among the first large-scale randomized trials to evaluate the effectiveness of a HEPA intervention by framing the outcomes in a policy-relevant manner – proportion of children accumulating 30 minutes of more of MVPA during the ASP and the number of days fruits or vegetables are served per week. Again, this substantively departs from prior studies where outcomes are reported as changes in minutes of activity or servings of fruits/vegetables, without placing these into the context of whether policy goals were achieved.

This study substantively departs from past studies by focusing on developing structural components within an ASP, for instance developing quality daily schedules and weekly menus, which serve as essential and foundational building blocks for integrating strategies that can bring practice up to a level that meets HEPA Policies. Furthermore, the HEPA strategies evaluated have demonstrated prior effectiveness, both for cost and achieving the HEPA policies, in pilot studies.^{23, 61} The outcomes from the proposed study will provide evidence of the scalability of the strategies, as well as, their effectiveness within a diverse set of ASPs.

Conclusion

While numerous intervention studies targeting physical activity and/or healthy eating in the ASP setting have been conducted over the past 10 years, few have been successful. Current practice in ASPs suggests additional efforts are needed to identify efficacious and low-cost solutions that ASPs can readily integrate into routine practice to achieve HEPA policies. The outcomes from this study will provide evidence that ASPs can bring practice up to a level that meets existing policies, and do so without substantial costs incurred.