Effects of a Competency-Based Professional Development Training on Children's Physical Activity and Staff Physical Activity Promotion in Summer Day Camps

Abstract

Background

The YMCA of USA adopted Physical Activity (PA) Standards for summer day camps (SDC) in 2011. Standards call for increasing children's PA, as well as, increasing staff behaviors related to creating an activity-friendly environment, such as role modeling and verbally promoting PA. The objective of this study was to evaluate strategies designed to meet the YMCA PA Standards.

Methods

Four YMCA SDCs participated in this pre/multiple-post test study. Strategies to increase staff PA promotion included ongoing professional development training, workshops, and technical support. Changes in staff behaviors and child PA were measured via the System for Observing Staff Promotion of Activity and Nutrition and the System for Observing Play and Leisure Time in Youth, respectively.

Results

Nine of 13 staff PA promotion behaviors demonstrated statistically significant changes in the desired direction. For example, staff engagement in PA with children increased by 11.4% (25.4% vs. 36.8%), while idle-time fell by 42.4% (53.1% vs. 10.7%) from baseline to final assessment. The percentage of girls and boys observed sedentary during scheduled PA decreased by 16.9% and 17.4%, while moderate-to-vigorous physical activity increased 3.3% and 3.5%, respectively. Changes in activity levels varied by grade level.

Conclusions

Strategies herein show promise for impacting staff behaviors and, in-turn, child PA. Continued support is likely required if changes are to be sustained.

Introduction

With more than 5,000 summer day camps (SDCs) in operation across the nation¹ and 14.3 million children in attendance annually,² SDCs are one setting, outside of the school year, with the potential to impact the physical activity (PA) of children. Children's PA in SDCs is particularly important because children's body mass index gains are greater over the summer than during the school year. ³ Recently, the YMCA of USA adopted PA Standards for their day-long youth programs, including SDCs, to address children's activity levels.⁴ The adoption of these standards is noteworthy because the YMCA of USA is one of the largest SDC providers in the United States, serving more than 9 million youth nationwide. The PA Standards focus on creating a PA-friendly environment by outlining key behaviors staff should exhibit, such as role modeling PA or verbally encouraging PA, and other behaviors from which staff should refrain, such as withholding or prescribing PA as a consequence of misbehavior.⁵ However, PA Standards do not outline strategies SDCs can use to change staff behaviors. Initial evidence suggests staff and program leaders struggle to integrate behaviors that promote PA into routine practice⁶ and children are largely inactive while attending SDCs.^7-9 Staff and program leaders, therefore, require strategies to create SDC environments that align with PA Standards.

To date, several studies have attempted to increase children's PA in the school and afterschool setting,^10-16 but limited attention has been given to interventions in the SDC environment. Consequently, little is known about how to most effectively align SDC staff behaviors with PA Standards. One promising strategy for increasing desired behavior is competency-based training.¹⁷ Competencies can be defined as, “any individual characteristic that can be measured and that can be shown to differentiate significantly between superior and average performers, or between effective and ineffective performer.” ¹⁸ Competency-based training diverges from traditional training models by focusing on the demonstration of observable skills rather than knowledge.¹⁹ We hypothesize that this approach will be effective because staff and program leaders should be able to integrate skills learned in the competency based trainings into their existing program.

In the spring of 2011 the YMCA of Columbia, SC, with the support of the University of South Carolina adopted the competency-based training approach in an attempt to align staff behaviors with the YMCA of USA PA Standards for SDCs.^20,21 Outcomes were promising for both changing staff behaviors and increasing child activity levels at the midpoint of the partnership.⁹ However, it is important to continue to evaluate this training's effectiveness. Therefore, the purpose of this paper is to describe a three year partnership between the University and local YMCA's to provide competency-based professional development training ¹⁷ and the impact of the training on children's activity levels in participating SDCs.

Methods

Setting and Participants

The results in this paper represent the findings of a multi-year intervention and evaluation using a single group pre/multiple-post assessment design. Findings from the midpoint evaluation have been reported elsewhere.⁹ The methods reported herein closely reflect the methods of the midpoint evaluation. Passive consent was obtained from participants and their parents due to the observational nature of data collection. The university's institutional review board approved all procedures prior to the start of the study. Four serving approximately 600 children each day across the 4 sites participated in the study. Programs provided children various physical activity opportunities throughout the summer including, free-play opportunities; organized games, such as sports and tag; water-based activities such as swimming, or playing at a water park. In addition to physical activity opportunities SDCs provided, assemblies, typically to begin or end the day often included songs and dance and camp wide announcements, and enrichment activities, such as arts and crafts.

Each SDC employed a single site leader and approximately 30 staff members. Site leaders oversaw daily program operations by creating schedules, managing staff, interacting with parents, etc. Staff members were responsible for leading groups of children through scheduled activities each day. The SDCs provided scheduled activities from 9am-4:30pm daily. Children were grouped by grade level, and grade levels were divided into smaller groups of children with one staff member responsible for 10 children eachThe majority of children were elementary-aged and were enrolled in the program an average of 4 days a week, 8 hours per day, for 8 weeks during the summer.

Intervention

Competency based professional development training

Training was the primary strategy for aligning staff behaviors with PA Standards. Professional development training was delivered prior to the beginning of each post-assessment summer (May 2012 and 2013). All trainings were led by trained university personnel and were integrated into mandatory preexisting staff trainings. The preexisting, day long (i.e., 8 hours) trainings occurred prior to the start of the SDC (i.e., May) each year. The PA training lasted approximately 90 minutes and was one of many sessions which all staff were required to attend throughout the day.

The 5Ms training model—Mission, Manage, Motivate, Monitor, Maximize ¹⁷ is competency based guided all professional development training. Two elements of the model connect it to a competency based approach: emphasis on a set of core skills that differentiate between superior and poor performers, and experiential training to improve those skills.^19,22,23 Identification of the core skills is founded in health promotion theory,^24,25 “best practices” position statements,^26,27 literature on competencies for PA promotion,^28-30 and PA policy documents.^5,31,32 Embedded in the 5Ms model are the “LET US Play” principles which is an acronym for lines, elimination, team size, uninvolved staff/kids, and space, equipment and rules.²¹ Training consisted of participatory exercises and video demonstrations. Staff first viewed video demonstrations of specific games such as kickball or tag, which violate the LET US Play principles by including lines, elimination, large team sizes, etc. Strategies to alter those games in order to align them with the LET US Play principles were then presented to staff. After each video, staff were debriefed on the strategies presented, and asked for input on additional strategies. Following the video demonstrations, staff participated in the games and modifications presented in the videos and practiced strategies for managing children in PA environments such as: using countdowns to transition between activities quickly, actively supervising children, keeping all children in view. Practicing management strategies was included in order to help ensure staff could manage and modify games to comply with LET US Play principles adequately. We hypothesized that this would reduce child wait and instruction time, and reduce or eliminate discipline problems due to extended wait or instruction time. In addition to management strategies staff were also specifically trained to offer girls only physical activity opportunities. This strategy was developed collaboratively with site leaders in an effort to reduce the gap between girls and boys activity levels observed at baseline.

On-site booster trainings

A total of nine training “booster” sessions were conducted at each SDC over the two post-assessment summers (five boosters summer 2012 and four summer 2013). Each booster session consisted of a “walkthrough” during which site leaders and staff would receive real-time feedback from university personnel. Walkthroughs occurred during schedule PA opportunities and lasted approximately two hours per visit. Following each walkthrough, observation notes and suggestions for program enhancement were complied, and presented to site leaders and staff in a meeting following the conclusion of the SDC for that day. Meetings with staff and site leaders to review the notes from the walkthrough lasted approximately 45 minutes. Suggestions were aligned with competencies presented to staff in the professional development training, based on the LET US Play principles, and focused on modifying games to enhance child PA, managing PA environments effectively, and modeling and encouraging child PA.

Workshop – Schedule Modification

A lack of detailed schedules was identified by university personnel and YMCA site leaders as one of the barriers to quickly moving through scheduled activities in the SDC. Prior to the intervention, schedules created by the SDCs listed only general activities, such as only enrichment or field games, and failed to indicate the location, equipment needed, or staff roles within the scheduled activity. This led to extended times when children had no task in which to engage, or idle-time, while staff decided upon the game, organized children, and retrieved and set up necessary equipment for the activity. Prior to summer 2012 program leaders were provided a schedule template and attended a scheduling workshop about creating schedules with specific activities, activity location, equipment, and staff roles. Site leaders were also encouraged to designate certain times when girls only physical activity opportunities would be offered. Site leaders and the lead author then built program schedules collaboratively using the scheduling template. Prior to the summer of 2013 schedules from 2012 were returned to site leaders. Site leaders were encouraged to use the 2012 schedules as a template to build the 2013 schedule. Prior to the start of the 2013 summer program site leaders provided their completed schedules to the lead author for feedback.

Weekly feedback

During evaluation, site leaders and staff received feedback the morning following each site visit. Notes from the evaluation team were compiled and emailed to site leaders for distribution to staff. Feedback focused on modifying games, effective management of children during PAs, and staff PA modeling and encouragement. Feedback was aligned with the 5Ms model and the LET US Play principles.

Instrumentation

System for observing staff promotion of activity and nutrition (SOSPAN)

Staff PA promotion behaviors were collected via SOSPAN. This instrument utilizes momentary time sampling to record instances of staff PA promotion behaviors consistent with PA standards. SOSPAN captures 13 PA promotion behaviors and has been validated and found reliable in the SDC setting.⁶ The instrument is divided into three subsections, including a) staff PA promoting behaviors, b) staff PA discouraging behaviors, and c) SDC context. Staff PA promoting behaviors (n=6) include behaviors or contextual characteristics of the PA environment over which staff have direct control that are theoretically or empirically linked to increased child activity. Examples include: staff verbally promoting PA or providing a girls only PA opportunity. Staff PA discouraging behaviors (n=7) include behaviors or contextual characteristics of the PA environment over which staff have direct control that are theoretically or empirically linked to decreased child activity. Examples include, staff verbally discouraging PA and children standing and waiting in line for their turn. SDC context includes scheduled activity and activity location.

Systematic observation of physical and leisure activity in youth (SOPLAY)

Child PA levels were collected via SOPLAY ³³ concurrently with staff behaviors. SOPLAY captures activity levels of large groups of children, using momentary time sampling. The activity codes included in SOPLAY have been extensively used in prior research.^33-36 For this study the vigorous activity level of the SOPLAY instrument was considered moderate-to-vigorous-physical-activity (MVPA).³⁷

Observation Schedule and Protocol

Data were collected over 98 program days over the three measurement summers. Data collection occurred on unannounced nonconsecutive weekdays (Mon-Thurs) at each site throughout June, July and August 2011 (baseline); and July and August 2012-2013 (outcome). SOPLAY and SOSPAN scans were alternated continuously from the beginning to the end of each program day. The scan sequence was as follows: SOPLAY, SOSPAN, SOPLAY, SOSPAN. This protocol is different from traditional SOPLAY protocol where target areas are scanned at predetermined times across a day.³⁸ Rather, the protocol for this study was designed to hold the time of observations constant across all sites while following a single group of children, regardless of the target area utilized. This decision was made for 2 reasons 1) groups of children within grade levels could have differing daily schedules of activities occurring in different target areas across days and weeks of observation and 2) a given target area might be used by the camp on one day at 10am, but not used at 10am any other day. Thus, holding both target area and time of observation in the target area constant would fail to capture many of the daily scheduled activities taking place at the SDCs. Consistent with SOPLAY and SOSPAN protocol ^6,33 the size, boundaries, and locations of target areas in which the SDC operated at each site were identified prior to data collection in the Summer of 2011. Examples of target areas include pools, fields, gyms, playgrounds. The number of target areas at individual sites ranged from 17-28, with a total of 91 target areas identified across the four SDCs.

On observation days, trained observers arrived unannounced before the program began and followed a randomly selected group of children within a pre-selected grade-level. Grade levels were systematically selected prior to the site visit in order to ensure at least 75% of the groups within each grade level were observed at each site and that each grade level was observed at each site on at least 4 program days during each measurement summer. The randomly selected groups of children and staff were followed throughout the entire day while observers systematically and continuously scanned the target areas populated by the group. Scans of the children and the staff responsible for the target group started at the beginning of the scheduled program day (9am), and were made continuously (one-after-the-other) until the end (4:30p.m.) of the SDC. To prevent observer fatigue, observers took two 15-minute breaks and one 30-minute lunch break during the day.

Observer Training and SOSPAN/SOPLAY Reliability

Prior to data collection periods, observers were oriented to study instruments and protocols via classroom training, video analysis, and field practice. Classroom training lasted two days, for six hours each day, and included a review of study protocol, and orientation to the instrument. Video analysis included observing sample videos of SDCs and practicing entering appropriate codes according to SOSPAN and SOPLAY protocols. Observers completed 3 hours of training on at least six days at participant programs. This field training included familiarization with target areas at program sites and completing practice/reliability scans.

Reliability was collected prior to measurement and on at least 30% of measurement days, or 31 total days, across data collection periods.³⁹ Inter-rater agreement criteria were set at >80% using interval-by-interval agreement for each SOSPAN category.³⁹ Percent agreement between observers for SOSPAN behaviors ranged from 81.8% to 99.6%. Interval-by-interval reliability for SOPALY activity codes were estimated via one way random effects single and average measures intraclass correlations (ICCs). Single measures ICCs for SOPLAY categories ranged from 0.80 to 0.97 with average measures ranging from 0.89 to 0.98.

Data Analysis

Stata (v.12.0., College Station, TX) was used to complete all statistical analyses. Child activity levels were expressed as the percentage of children engaged in sedentary behavior or MVPA in each SOPLAY scan [(children sedentary, walking, or vigorous/total children in scan)*100], while staff behaviors were expressed as a percentage of total SOSPAN scans a behavior was observed [(scans with variable/total SOSPAN scans during PA opportunities)*100]. Changes in child activity and sedentary levels and staff behaviors were examined using random effects linear regression models with scans nested within groups of children nested within SDC sites. Models estimating the percent of children in MVPA and sedentary were conducted separately and controlled for the total number of children in each scan and daily high and low temperatures. Intervention effects were modeled at the site level. Where appropriate both linear and non-linear terms were included in models to account for the nonlinear change in staff behaviors and the percent of children in MVPA and sedentary over time. Secondary models were estimated by grade level during scheduled PA.

Results

Over the three measurement periods 12,803 SOSPAN and SOPLAY scans were completed during scheduled program time. A total of 8,348 SOSPAN and SOPLAY scans were completed during scheduled PA.

Changes in the percentage of children in MVPA and sedentary

High and low temperatures did not demonstrate statistical significance in the model and were removed from final analysis. Changes in the percentage of children sedentary and children engaged in MVPA across all scheduled activities are presented in Table 1. There was a statistically significant reduction in the percent of children sedentary across all scheduled activities with the exception of “other,” where a slight decrease in the percent of children sedentary was observed. During scheduled PA there was a 16.9 and 17.4% reduction in percent of girls and boys observed sedentary from baseline until final post-assessment, respectively. The largest reduction in the percent of children sedentary was observed during organized PA, with an approximate 23.7 and 24.8% reduction for girls and boys, respectively. Conversely, statistically significant increases in the percent of girls and boys engaged in MVPA were observed during free play PA, organized PA, and assembly. A statistically significant increase in the percent of girls in MVPA was also observed during enrichment. The largest statistically significant increase in MVPA for girls and boys were observed during assembly with a 12.6% increase for girls and a 6.9% increase for boys. A change of 13.6 and 9.2 in the percent of girls and boys engaged in MVPA during swimming was observed but those changes did not reach statistical significance. The largest statistically significant increase during scheduled PA for girls and boys occurred during organized PA (8.1 and 6.4 respectively).

Table 1. Percentage of Girls and Boys Engaged in Sedentary and MVPA by Scheduled Activity.

				Percentage of Girls Sedentary and MVPA by Scheduled Activity								Percentage of Boys Sedentary and MVPA by Scheduled Activity
	Percent of Total Scansa			Sedentary				MVPA				Sedentary				MVPA
Scheduled Activity	2011	2012	2013	2011	2012	2013	Δ	2011	2012	2013	Δ	2011	2012	2013	Δ	2011	2012	2013	Δ
Enrichment	20.5	16.0	12.6	89.2	84.7	80.1	-9.0	1.5	2.3	3.1	1.6	84.5	81.1	77.7	-6.8	2.9	3.1	3.3	0.5
Physical Activity	56.6	61.4	70.5	65.6	57.1	48.7	-16.9	13.5	15.1	16.8	3.3	57.9	49.2	40.5	-17.4	16.9	18.6	20.4	3.5
Free playb	48.6	31.2	28.0	70.7	62.1	53.4	-17.3	8.5	15.1	16.2	7.7	61.8	51.8	39.6	-22.2	14.2	16.5	18.8	4.5
Organizedb	40.5	48.9	56.0	72.0	55.2	48.3	-23.7	6.9	10.9	15.2	8.1	64.2	45.3	39.2	-24.8	12.2	15.4	18.5	6.4
Swim/waterb	10.9	19.9	16.0	49.3	41.6	33.9	-15.4	36.8	41.3	50.4	13.6	48.3	40.9	33.4	-15.0	37.7	42.3	46.9	9.2
Bathroom/Changingc	13.2	9.6	8.7	83.8	79.0	74.1	-9.7	2.2	2.0	1.6	-0.6	80.3	76.9	73.4	-6.9	2.9	2.6	2.3	-6.9
Assembly	6.\2	3.6	4.2	80.6	71.3	62.1	-18.4	4.4	8.6	16.9	12.6	74.7	67.1	59.5	-15.1	5.1	8.6	12.0	6.9
Other (i.e. devotion, transition)	3.7	9.5	4.1	85.2	85.1	85.1	-0.1	1.5	1.3	0.9	-0.7	87.6	81.0	87.2	-0.4	2.5	3.1	3.7	1.2

Percentages are adjusted means based on multilevel mixed effects linear and non-linear regression nesting scans within groups of children, within sites Based on 12,803 SOSPAN and SOPLAY scans over 98 program days in the Summer of 2011, 2012, and 2013

Statistically significant changes at p<0.05 are bolded

^aPercent of scans that took place in specified context by year

^bFree play, organized, and swim/water activities are sub settings of Physical Activity time.

^cIncludes times when children were waiting for other children to use the bathroom (i.e., 5 children enter the bathroom while others wait their turn)

Table 2 presents changes in the percent of boys and girls sedentary and engaged in MVPA, by grade level, during scheduled PA. Changes in the percent of girls observed sedentary ranged from a 22.1 to 11.5% decrease, while changes for boys ranged from a 21.3 to 8.5% decrease. All changes in the percent of girls and boys observed sedentary were statistically significant. Changes in the percent of boys engaged in MVPA ranged from a 1.3 to 5.8% increase, while changes for girls ranged from a 5.3% decrease to a 7.9% increase. Changes in the percent of girls and boys engaged in MVPA reached statistical significance except for the 4^th and 5^th grade level.

Table 2. Percentage of Girls and Boys Engaged in Sedentary and MVPA by Grade Level During Schedule Physical Activity.

	Girls								Boys
	Sedentary				MVPA				Sedentary				MVPA
	2011	2012	2013	Δ	2011	2012	2013	Δ	2011	2012	2013	Δ	2011	2012	2013	Δ
K-1st	71.8	60.8	49.7	-22.1	14.7	18.6	22.6	7.9	67.5	57.3	47.2	-20.3	17.3	19.9	22.5	5.2
2nd-3rd	70.8	61.7	52.6	-18.3	13.3	16.6	20.0	6.7	69.0	58.4	47.8	-21.2	16.4	18.7	21.0	4.6
4th-5th	65.2	59.5	53.8	-11.5	24.3	21.7	19	-5.3	58.0	48.0	49.5	-8.5	20.2	27.9	21.3	1.3
Mixed Grade Levels	67.3	57.8	48.2	-19.1	14.6	18.3	21.9	7.2	61.1	50.4	39.8	-21.3	19.7	22.6	25.5	5.8

Percentages are adjusted means based on multilevel mixed effects linear & nonlinear regression Based on 8,348 SOPLAY scans over 98 program days in the Summer of 2011, 2012, and 2013

Statistically significant changes at p<0.05 are bolded

Changes in staff behaviors

Changes in staff behaviors are presented in Table 3. Of the 13 staff behaviors observed, 11 changed in the desired direction. Of these 11 changes, nine reached statistical significance. All staff behaviors that promote children's PA changed in the desired direction and reached statistical significance. Changes in these staff behaviors ranged from an 11.4% increase in staff engaged in PA with children to a 2.1% increase in choice of PA opportunities provided. Staff engaged in PA with children and providing children a choice of PA opportunities showed decreases from baseline to the midpoint (25.4% vs. 20.4%) before increasing during the final year. Staff verbally promoting PA showed accelerated increases between final year (4.8% vs. 10.9%) when compared to increases at the midpoint (2.9% vs. 4.8%).

Table 3. Increases and Decreases of Staff Physical Activity Promotion Behaviors During Scheduled Physical Activity Time from Baseline to Post-assessment.

	Percent of scans observed during scheduled physical activity time
	Summer 2011	Summer 2012	Summer 2013	Δ From baseline to final assessment
Physical Activity Promoting Staff Behavior
Staff leading or instructing physical activity	9.5	14.3	19	9.5
Staff verbally promoting physical activity	2.9	4.8	10.9	8.0
Frontline staff engaged in physical activity with children (i.e. playing the game)	25.4	20.4	36.8	11.4
Choice provided (i.e. more than one activity opportunity provided)	9.4	1.9	11.5	2.1
Small game (i.e. games with less than 10 children participating)	0	1.7	3.6	3.6
Girls only physical activity opportunity	0.6	2.1	3.6	3.0
Physical Activity Discouraging Staff Behavior
Staff verbally discouraging physical activity	2.3	1.3	2	-0.3
Withholding physical activity as a consequence of misbehavior	2.1	1.6	1.0	-1.1
Children standing in line and waiting for turn	18.9	7.8	14.9	-4.0
Playing elimination game (i.e. children eliminated from PA opportunities)	8.5	7	5.5	-3.0
Frontline staff giving instructions	10.9	11.4	11.9	1.0
Frontline staff disciplining children	2.1	2.3	2.5	0.4
Idle-time (i.e. children waiting for direction from staff with no specific task)	53.1	18.2	10.7	-42.4

Percentages are adjusted means based on multilevel mixed effects linear & nonlinear regression

Bolded numbers are statistically significant changes at p < 0.05

Based on 8,348 SOSPAN scans over 98 program days in the Summer of 2011, 2012, and 2013

Only three of the staff behaviors that discourage child PA demonstrated statistically significant changes in the desired direction including child idle time (-42.4%), staff withholding PA from children as a consequence of misbehavior (-1.1%), and children standing in line waiting for their turn (-4.0%). No other changes reached statistical significance. For children standing in line waiting for their turn an initial decrease of 11.1% was observed between baseline and the midpoint of the study. However, from the midpoint to the final summer, an increase of 7.1% was observed. For idle-time an initial decrease of 34.9% was observed from baseline to the midpoint with a smaller but continued decrease of 7.5% from the midpoint to the final summer.

Discussion

While significant work has been done in the afterschool setting to assist YMCA's across the country with the adoption of PA Standards,⁴⁰ this is one of the first studies to evaluate an intervention to increase SDC staff's PA promotion and related changes in children's PA via systematic observation. The findings reported in this paper allowed for the continued evaluation of the changes in staff behaviors and child activity levels following the final summer of evaluation. The majority of the staff behaviors moved in the desired direction over the three year study and improved from the midpoint of the study to the final summer. A corresponding decrease in the percent of children sedentary between intervention year one and two was also observed. Continued increases in the percent of children engaged in MVPA were also observed during free play and organized PA. Taken together, these improvements across the 3 year study indicate the strategies developed and implemented in this study can lead to sizable changes in staff behaviors that are both theoretically and empirically linked to children's activity levels. These strategies, in turn, can be used to assist SDCs in meeting PA Standards.

This intervention is unique from previous interventions to promote PA. The mechanism for change in this study was a competency-based professional development training emphasizing building staff and program leader skills to create a physical activity supportive SDC. The skills covered in this training were readily applicable to the SDC program without changing any programmatic components. The strength of this approach is that the strategies are easily adaptable to each SDCs' unique circumstances/needs. Staff were not asked to implement/deliver new activities, but rather, were trained to integrate the LET US Play principles into the games they were already playing with children. The training continued to demonstrate effectiveness at the final intervention summer as represented by the large increases in staff behavior and child activity level compared to the midpoint summer. These findings illustrate that the adoption of PA Standards, coupled with the 5Ms training and LET US Play principles as described herein can produce continued changes in staff behaviors and may be effective for aligning staff behaviors with those called for in PA Standards.

Changes in staff behaviors were also accompanied by a reduction in the percent of children sedentary and increases in the percent of children engaged in MVPA. At the completion of midpoint evaluation, the strategies were most effective at reducing the percentage of children sedentary.⁹ This trend continued to the final evaluation with a statistically significant reduction in the percent of children sedentary in five of the six scheduled PA opportunities. This is an important finding since reducing children's time sedentary is emerging as a public health goal.⁴¹

Reductions in the percent of children sedentary were accompanied by increases in the percent of children engaged in MVPA during free play and organized PAs, as well as, during assembly. An increase in the percent of girls engaged in MVPA was also observed during enrichment activities. Staff were trained to institute short activity breaks ⁴² during long periods of scheduled inactive time which likely explains the changes in MVPA during these times. Further, the smallest percent of children sedentary and the highest percent of children engaged in MVPA occurred during swimming and water activities. This finding is consistent with previous research that found increases in the percent of children engaged in MVPA during water activities in SDCs. ⁸ Scheduling swim time or outdoor activities that include water in the form of water balloons, hoses, or sprinklers could be one strategy for programs to increase children's MVPA and decrease sedentary time during the summer heat.

At baseline more girls and boys were engaged in MVPA during free play compared to organized PA opportunities. This finding is consistent with other studies ^43,44 and has led to a call for the integration of more free play PA opportunities into afterschool programs.⁴⁴ However, following the intervention a comparable percentage of boys and girls were engaged in MVPA during free play and organized PA opportunities. These finding suggests that there are characteristics of organized activities that minimize PA, and training staff to modify organized PA to comply with the LET US Play principles can illicit comparable if not more MVPA than free play opportunities.

The percent of children observed sedentary decreased across all grade levels, while the percent of children engaged in MVPA increased in all grade levels except fourth and fifth. It is unclear why the fourth and fifth grade groups did not increase the percent of children engaged in MVPA. One explanation may be that as children age they become more intent on fitting into a social group of their peers rather than pleasing adult supervisors, ⁴⁵ and therefore, strategies that focus on increasing staff PA supportive behaviors become less effective as children age. Thus, strategies to increase older children's engagement in MVPA need to be explored because the reduction in children's activity levels as they enter adolescence is well documented.⁴⁶

A diminished gap between the girls and boys engaged in MVPA during free play and organized PA was observed at the conclusion of intervention summer one.⁹ This trend continued into intervention the second summer, with the percent of girls engaged in MVPA increasing by 7.7 and 8.1% during free play and organized PA respectively while the boys increased by 4.5 and 6.4% during free play and organized PA respectively. It is widely accepted that girls are less active than boys,⁴⁶ therefore, strategies that can minimize the gap between girls and boys PA levels are needed. This study provides initial evidence that the LET US Play principles, in concert with the 5Ms training model, providing girls only PA opportunities, engaging in PA with girls, and verbally encouraging PA, has the potential to fill that need.

The limitations of this study include: a small number of SDCs and the lack of a control group. The 4 SDCs in which the intervention was evaluated may not be representative of all SDCs. Further, observed increases and/or decreases in staff behaviors may have occurred in the absence of the intervention due to history, selection bias, regression to the mean, and/or the “Hawthorne effect.” The lack of a control group does not allow us to confirm or refute that limitation. However, it is unlikely the magnitude of the changes observed were caused by anything other than the intervention given the majority of the targeted staff behaviors changed in the desired directions and were accompanied by changes in child activity levels.

This study also has several strengths, including the use of a community based participatory model, the amount of data collected via systematic observation, and multiple post assessments. A collaborative partnership between SDCs and university led to the development of strategies that were both relevant to public health goals and feasible for SDC programs to achieve. Input from program leaders also ensured that the intervention was adaptable to the unique context of each program and, therefore, adoptable, enhancing the likelihood of changes to routine practice.⁴⁷ The nearly 13,000 scans completed in this study confirm that the data herein is representative of the participant sites. Further, multiple post-assessments allowed for the observation of trends in staff behaviors and child activity levels over several time points.

In conclusion, study findings show that a competency-based professional development training may be effective at increasing PA promoting and decreasing PA discouraging staff behaviors and related child activity levels. The majority of changes in staff behaviors and child activity levels observed at the midpoint evaluation were sustained through a final year of intervention and evaluation. This finding suggests that continued support and training is necessary for staff and program leaders to sustain these changes. While this study has a limited sample size the lessons learned may be broadly applicable to large scale SDC program providers. For instance, programs may need to implement ongoing training and evaluation if they are to continue to meet the PA Standards. However, further studies are needed to confirm and build upon these findings.