Factors influencing implementation of a preschool-based physical activity intervention

Abstract

Examining factors that influence implementation of key program components that underlie an intervention’s success provides important information to inform the development of effective dissemination strategies. We examined direct and indirect effects of preschool capacity, quality of prevention support system and teacher characteristics on implementation levels of a component, called Move Outside (i.e., preschool classroom teachers to provide at least 40 min of outdoor recess per day), that was fundamental to the success of a preschool-based physical activity intervention. Level of implementation, defined as the percent of daily goal met for the Move Outside component, was assessed via direct observation. Items assessing preschool capacity, quality of prevention support system and teacher characteristics were selected from surveys and an environmental checklist completed by preschool directors and teachers. Preschool classroom was used as the unit of analysis (Year 1: n = 19; Year 2: n = 17). Results from Bayesian path analyses showed that the three factors were not significantly associated with level of implementation in Year 1, but preschool capacity was directly associated with level of implementation in Year 2 (β= 0.528, 95% CI: 0.134, 0.827). The current findings suggest that factors that influence level of implementation appear to differ as an intervention evolved over time.

Introduction

Physical activity is important to the growth and development of preschool-age children (ages 3–5 years) [1]. National guidelines on physical activity recommend that children in this age group engage in 180 min of light, moderate and vigorous physical activity per day [2]. However, studies that used objective measures of physical activity have shown that at least 50% of preschool-age children do not meet these guidelines [2–4]. These data indicate the need for developing effective interventions to increase physical activity among children in this age group.

Approximately 61% of American children ages 3–5 years are in a regular childcare arrangement, of which 38.4% are enrolled in center-based preschools [5]. The opportunity to reach a large group of young children makes these center-based facilities a potential point of intervention. A limited number of randomized controlled trials (RCTs) have been conducted to test the effectiveness of preschool-based physical activity interventions [6–8], and a few have demonstrated promising results [9–12]. To produce significant public health impact, the next step would be to disseminate those effective programs at a broader level. However, widespread dissemination will require identifying the components that underlie an intervention’s success and the factors that influence implementation of those components, which will allow researchers and practitioners to develop effective implementation strategies [13, 14]. To date, few preschool-based physical activity interventions have identified factors associated with overall program implementation; no study has specifically focused on factors that influenced implementation of components that underlie an intervention’s success.

The Study of Health and Activity in Preschool Environments (SHAPES) intervention was an RCT that was found to be effective in increasing children’s moderate-to-vigorous physical activity (MVPA) in preschools [15]. SHAPES consisted of three main components designed to increase preschoolers’ MVPA by offering physical activity opportunities through indoor playtime (Move Inside), recess (Move Outside) and active learning (Move To Learn) [16]. Process evaluation analyses of SHAPES showed that implementation levels of one of the three intervention components, the Move Outside component, were significantly associated with preschool day MVPA over the intervention period in girls (R.P.S., in review). Girls who attended classrooms classified as high-implementers of this component engaged in significantly more minutes of MVPA than girls who attended the low-implementer classrooms or control classrooms; no difference was found between the low-implementer and control classrooms (R.P.S., in review). These findings indicate that Move Outside was the key program component underlying the intervention’s success, and improving its implementation may further increase the effectiveness of SHAPES in future dissemination efforts. Accordingly, the purpose of the current study was to examine the effects of preschool capacity, quality of prevention support system and teacher characteristics on implementation levels of the SHAPES Move Outside component.

Methods

Conceptual model

The current study was guided by the Durlak and DuPre framework [17]. This framework posits that program implementation of health promotion interventions for youth is influenced by five types of factors: (i) community-level factors (e.g. politics), (ii) organizational capacity (e.g. administrative support), (iii) prevention support systems [e.g. technical assistance (TA)], (iv) provider characteristics (e.g. self-efficacy) and (v) innovation characteristics (e.g. compatibility). Although the Durlak and DuPre framework [17] has delineated 23 elements to describe the 5 factors, not all the elements have been tested in empirical studies. We found limited empirical evidence supporting the positive influence of elements within organizational capacity, quality of the prevention support systems and provider characteristics on program implementation. Previous studies have found that elements within organizational capacity, including administrative support [18] and school climates [19]), were directly associated with implementation levels of school-based prevention programs. Elements within provider characteristics (i.e., teachers’ self-efficacy), and quality of prevention support system (i.e., provision of training and types of training) were also found to be directly associated with increased implementation by teachers of a school-based physical activity [20] and drug prevention interventions [21]. In addition to the direct influence, other studies have suggested a mediation effect of elements within provider characteristics on the relationship between levels of implementation and other factors. A school-based positive youth development program found that relationships between school climate and teachers’ implementation were partially mediated by teachers’ beliefs and attitudes [19]. A school-based drug prevention program showed that change in teachers’ self-efficacy partially mediated the relationship between types of training and implementation fidelity [21].

Based on these findings, the current study focused on examining the direct and indirect effects of elements within preschool (organizational) capacity, quality of prevention support system and teacher (provider) characteristics on implementation of the SHAPES Move Outside component. Specifically, we hypothesized that as follows:

• Level of implementation would be positively and directly influenced by preschool capacity, quality of prevention support system and teacher characteristics.

• Preschool capacity and quality of prevention support system would have indirect effects on level of implementation through teacher characteristics.

We selected 34 relevant items from the SHAPES process evaluation that are designed to measure various elements of the preschool capacity, quality of prevention support systems and teacher characteristics based on the framework by Durlak and DuPre [17], a review by Naylor et al. [22], and several empirical studies [18, 20, 21, 23–25]. Then, exploratory factor analysis (EFA) was used to verify the contribution of each selected item in explaining variances of a given factor and to reduce the selected items into a smaller set of elements. The final EFA results included 16 items grouped under 6 elements within the three factors. There were three elements within preschool capacity: physical activity practices, organizational general capacity and leadership support; one element in quality of prevention support system: efficacy of training and TA; and two elements in teacher characteristics: teachers’ self-efficacy and skill proficiency. As little research has explored the pathways through which these elements interact to influence implementation outcomes, elements within a given factor were summed into a composite index score which was used in the analysis (Fig. 1).

Fig. 1.

Conceptual framework for understanding factors that influence implementation of the SHAPES Move Outside component.

Study design

The present study used a prospective observational study design. Data were taken from the process evaluation of the SHAPES intervention. The rationale, design and method of the SHAPES study have been described previously [16]. Briefly, SHAPES was a 3-year RCT that tested if an ecological physical activity intervention increases children’s MVPA [15]. Sixteen preschools randomly selected from two districts in Columbia, South Carolina were pair-matched and randomly assigned to either the intervention (n = 8) or a waiting list control group (n = 8). Data for outcome evaluation and process evaluation were collected at baseline, during and post-intervention.

The current study analyzed process evaluation data related to implementation of the Move Outside component assessed during each intervention year: preschool capacity, quality of prevention support system and teacher characteristics measured at baseline and annual process evaluation assessments. Process data for Year 3 were excluded from the analyses because of the large amount of missing data in preschool capacity. Due to changes in process evaluation methodology and teacher turnover, we treated the data collected from Years 1 to 2 as two cross-sectional samples and analyzed them separately. The current study was approved by the University of South Carolina’s Institutional Review Board.

Participants

The participants were intervention classrooms for 4-year-old children. The intervention was implemented in the same classrooms for 2 years but with two separate cohorts of students. During Year 1, a particular group of students was assessed at baseline and follow-up. During Year 2, another cohort (i.e., new students in a particular teacher’s classroom) was assessed at baseline and follow-up. Of the 20 intervention classrooms, 19 and 17 of them had complete process evaluation data in Years 1 and 2, respectively. For both Years 1 and 2, the average number of children per classroom ranged from 14 to 20, and all classrooms were led by female teachers. For Year 2, fourteen classrooms were led by teachers that had implemented SHAPES in Year 1, and three classrooms were led by new teachers.

The intervention classrooms were nested in eight intervention preschools. The characteristics of these preschools varied, with the number of enrolled students ranging from 199 to 870. Fifty percent of the preschools were public schools. Thirty-eight percent of the preschools served predominantly Caucasian children, 38% served predominately African American children, and 24% had an equal distribution in race/ethnicity. Sixty-three percent of the preschools offered full time programs, and 75% provided physical education.

Overview of SHAPES-move outside components

SHAPES aimed to increase preschoolers’ MVPA through maximizing all physical activity opportunities throughout the preschool day. The intervention consisted of three elements: 1) Move Inside, 2) Move Outside and 3) Move to Learn. Pfeiffer et al. [16] provide detailed descriptions of the rationale and design of the multi-component intervention, and the present study focused on the Move Outside component. Move Outside asked teachers to provide at least 40 min of structured and unstructured physical activity at outdoor recess whenever possible, which should mainly consist of 1) child-initiated free play activities and 2) at least one 5-min session of teacher-led, structured activities (e.g. ‘track team,’ where children jog around the playground with the teachers and peers).

SHAPES employed a flexible and adaptive framework to accommodate specific preschool environments and preferences (e.g. available space and schedules). With the training and TA provided by university-based interventionists, each classroom teacher acted as an organizational change agent to flexibly integrate the three intervention elements into her specific classroom and organizational constraints. To facilitate implementation, the SHAPES prevention support system entailed developing partnerships and providing training and ongoing TA to increase teachers’ knowledge, skills and confidence to integrate Move Outside into their specific classrooms. These supports included workshops, site visits, newsletters and sample outdoor recess activities.

Study variables and measures

Descriptions of the measurement items, data sources and timing of data collection are presented in Table I. The current study selected items from multiple instruments used in the SHAPES process evaluation. These items were completed by different individuals at different time-points during the intervention period. The dependent variable level of implementation was assessed using direct observation data collected by an observer during the intervention period. Items related to organizational capacity were taken mainly from the baseline preschool director survey and the Early Childhood Environment Rating Scale, Revised Edition (ECERS-R) [26], completed by an observer at baseline. The ECERS-R is a 43-item standardized rating scale that has been widely used to evaluate the resources and quality of early childhood education programs [27–30], and the scale has been demonstrated to be reliable at the individual item and total scale score levels [26, 28]. Items measuring quality of prevention support system were completed by the participating teachers at the end of each intervention year. Items assessing teacher characteristics were completed by the participating teachers and university-based interventionists at the end of each intervention year. Unless otherwise stated, all items measuring organizational capacity, teacher characteristics and quality of prevention support system were on 4-point Likert scales.

Table I.

Psychometric properties of items measuring study variables

Factors/elementsa	Items descriptions		Scale	Data source, instrument	Factor loadinga	Total variance explaineda	Index score (SD)b	Range
Level of implementation (1 item)							Y1: 62.9 (55.4)	Y1: 0–186
							Y2: 56.7 (40.6)	Y2: 0–85
Percent of goal met for the Move Outside component	Observed number of minutes of recess during the observation day		Number of minutes	SHAPES process evaluation checklist	–	–
Organizational capacity (12 items)						Y1: 90.8%	Y1: 65.5 (7.15)	Y1: 50.0– 72.85
						Y2: 90.8%	Y2: 62.95(6.55)	Y2: 51.5–72.0
Physical activity practices (3 items)dα = 0.81	1. Active (free) play time is provided to all children		1 = 15–30 min/d 4 = >60 min/day	Preschool Director; Baseline (director survey)	0.85
	2. Structured physical activity (teacher-led) is provided to all children		1 = <1 time/mo 4 = daily		0.82
	3. Supervision of gross motor activities		1 = inadequate 7 = excellent	Observer; Baseline ECERS-R	0.88
General capacity (8 items)dY1: α = 0.87 Y2: α = 0.93	1. Encouraging children to communicate		1 = inadequate 7 = excellent	Observer; Baseline ECERS-R	0.91
	2. Supervision of gross motor skills				0.91
	3. General supervision of children				0.91
	4. Staff-child interaction				0.91
	5. Opportunities for professional growth				0.91
	6. Space for gross motor play				0.92
	7. Gross motor equipment				0.98
	8. Schedule				0.77
	9. Classroom overall functioningb		1 = Low	Interventionist;	Y1: 0.91
			4 = Very Strong	Annual rating form	Y2: 0.91
Leadership support (1 item)	1. Rate the adequacy of the support you received from your school’s administration to incorporate the SHAPES program?b		1 = very inadequate 5 = very adequate	Teachers; annually (teacher survey)b	Y1: 0.97 Y2: 0.98
Quality of prevention support system (2 items)						Y1: 50.6%	Y1: 3.0–8.9	Y1: 15.3 (1.19)
						Y2: 66.0%	Y2: 4.0–9.0	Y2: 14.5 (0.62)
Efficacy of training and TA (2 items)	1. How would you rate the adequacy of the support you received from the SHAPES intervention team from USC to incorporate the SHAPES program?		1 = very inadequate	Classroom teacher; Annually (teacher survey)c	Y1: 0.71
			5 = very adequate		Y2: −0.83
	2. What problems, if any, did you have in carrying out structured activities at recess?		Total number of problems encountered		Y1: −0.71
					Y2: −0.83
Provider characteristics (2 items)						Y1: 61.7%	Y1: 12–16	Y1: 7.4 (1.53)
						Y2: 55.0%	Y2: 13–15	Y2: 7.4 (1.69)
Skill proficiency (1 item)		1. Rate the class room teacher’s skills in dealing with changes, challenges, and disruptions	1 = poor	Interventionist; Annually (evaluation form)c	Y1: 0.79
			4 = very strong		Y2: 0.74
Self-efficacy (1 item)		1. To what extent did you feel prepared to carry out the SHAPES program	1 = very unprepared	Classroom teacher; annually (teacher survey)c	Y1: 0.79
			5 = very prepared		Y2: 0.74

^aNumber of elements, factor loadings and total variance explained were estimated using the EFA, with principal component procedure and non-orthogonal (oblique) rotation. Scree tests and eigenvalue value >1 were used to determine the factors to retain for rotation. Items with no variability across participants and loading smaller than |0.40| were excluded. Cronbach’s α was calculated for elements with ≥3 items to assess internal consistency, and a value of ≥0.7 was considered to be acceptable.

^bIndex score for level of implementation was calculated as the average minutes of Mover outside across the observation days/40 min) � 100. Index score for the three factors were calculated by summing the raw score of the relevant items. For example, index score for provider characteristics was calculated by summing the raw scores of the two included items.

^cFactor loadings for items measured on an annual basis (e.g. classroom overall functioning) were presented by intervention year.

^dCronbach’s α for the general capacity and functioning subscale were also presented by intervention years because it consisted of items that were measured on an annual basis.

Level of implementation

Based on process evaluation analyses (R.P.S., in review), among various implementation indicators, only the number of minutes of outdoor recess provided was significantly associated with program outcome (i.e., MVPA). Accordingly, this study defined level of implementation as the percent of goal met for the Move Outside component (i.e., dosage/completeness). Level of implementation was calculated as the percent of daily goal met for the Move Outside component, which is 40 min of recess time per day for half-day programs and 80 min for full-day programs. An observer visited each of the intervention classrooms (Year 1: n = 20; Year 2: n = 17) and used the process observation checklist to record the number of minutes of outdoor recesses being provided by the classroom teachers. As preschools were given flexibility in scheduling outdoor recesses, the observations took place over the entire school day. The observations were scheduled based on convenience of the participating classroom teachers. In Year 1, observations were conducted on four fall days and four spring days for each of the intervention classrooms. The average minutes across the four fall days and four spring days was used to calculate percent of daily goal met. In Year 2, the same procedure was used, but the observation was conducted on one day of observation in the fall and one day in the spring due to time and resources limitations. For both years, two data collectors observed 10% of the observations to assess inter-rater reliability, which was >0.80 (Table I).

Preschool capacity

Thirteen items were used to assess three elements of preschool capacity, including physical activity practices, preschools’ general capacity and functioning and leadership support. Three items assessed preschool’s physical activity practices regarding the opportunities for active play and structured physical activity, and scheduling during the preschool day. Nine items assessed preschool’s general capacity and functioning, in which eight 7-point items assessed preschool’s capacity in providing service and gross motor activities and one 4-point item assessed preschool’s overall functioning. One item was used to assess leadership support based on teachers’ perceptions of support received from the preschool leadership (Table I). Raw scores of the 13 items were summed to create a composite score for preschool capacity. A higher score indicated a higher capacity for implementing the SHAPES intervention.

Teacher characteristics

Two items were included to assess two elements of teacher characteristics, including self-efficacy and skills proficiency. One item was used to measure teachers’ self-efficacy of implementing the intervention and another item completed by the interventionists was used to evaluate teachers’ skills in resolving implementation issues (Table I). Raw scores of the two items were summed to create a composite score for teacher characteristics, with higher scores indicating teacher characteristics as being more favorable to a successful implementation.

Quality of prevention support system

Quality of prevention support system was measured, defined as quality of training and TA provided by the SHAPES staff. Two items were used to measure the quality of the training and TA as reflected by perceived ease of implementation specifically related to the Move Outside component and adequacy of overall support received from the SHAPES staff (Table I). A composite score was calculated by adding raw scores of the two items, with higher scores indicating that teachers perceived the prevention support system is more favorable to a successful implementation.

Analysis

Preschool classroom was used as the unit of analysis. Bayesian path analysis was used to test a prior hypothesized path model (Fig. 2). The Bayesian approach was selected because it is more appropriate for modeling data based on a small sample [33, 34], which is often the case in process evaluation data. In the path diagram (Fig. 2), preschool capacity and quality of prevention support system are the predictor variables, teacher characteristics is the mediator and level of implementation is the outcome variable. The average indirect effects were calculated as a*b (preschool capacity → teacher characteristics → level of implementation) and z*b (quality of prevention support system → teacher characteristics → level of implementation). The total indirect effect (c) was calculated as a*b + z*b.

Fig. 2.

Final path models for predicting implementation of the Move Outside component during Years 1 and 2 of the SHAPES intervention. Bolded solid path indicates significant paths. Standardized path coefficients are shown. Asterisk indicates significant association with 95% CI not including zero.

Bayesian path analysis was conducted using M-plus software (version 6.11). As accurate prior information was not available, we assigned independent non-informative uniform priors to all the unknown parameters. The regression coefficients were assigned to follow a normal distribution. The variance parameters followed an inverse-gamma distribution, in which a large variance in the normal prior above implies a non-informative prior. Bayesian estimates of all parameters and variance components were calculated based on 10 000 samples after 1000 burn-in iterations. Posterior mean, posterior standard error and 95% Credibility Interval (CI) at the 0.025 and 0.975 quantiles of the average direct and indirect effects were also obtained. The direct and indirect effects were determined to be statistically significant if the 95% CI did not include zero.

The convergence of the final model was assessed by trace plots, Proportional Scale Reduction (PSR) index, and the autocorrelation plot. A tight and horizontal shape of the trace plots, values for PSR of 1, and values for autocorrelation plots of ≤0.1 indicate good model convergence [31]. We evaluated the model-to-data fit based on the 95% confidence interval for the difference between the observed and the replicated χ² values (95% CI for χ² value) and the posterior predictive P values (PPP). A lower negative value of 95% CI for χ² value and a PPP value close to 0.5 are ideal.

Results

Descriptive statistics of the study variables are presented in Table I. Level of implementation across preschool classrooms ranged from 0% to 186% (M = 62.9%, SD = 55.4%) for Year 1 and from 0% to 185% (M = 56.7%, SD = 40.6%) for Year 2. These indicate a great variation in implementation across classrooms, with some classrooms not implementing the Move Outside component at all and some over-achieving the prescribed goals. The hypothesized models for both years had a good convergence, as the PSR values were close to 1 for all the estimated parameters, the trace plots showed a tight and horizontal shape, and the autocorrelation plots showed all parameters had value ≤0.1. The models also provided a good fit, as the lower bound of the 95% CI for χ² value achieved a negative value and the PPP value was close to 0.5 (Table II).

Table II.

Direct and indirect effects of preschool capacity, teacher characteristics, and quality of prevention support system on level of implementation of the SHAPES Move Outside component

			Year 1		Year 2
			Estimates (SD)	95% CI	Estimates (SD)		95% CI
Direct effects
PRESCH	→ IMLEVEL (path c’)		−0.130 (0.692)	−1.470 to 1.226	0.443 (0.197)		0.031 to 0.795
TEACHER	→ IMLEVEL (path b)		−0.109 (0.569)	−1.196 to 1.012	0.207 (0.319)		−0.401 to 0.787
SUPPORT	→ IMLEVEL (path e’)		0.232 (0.353)	−0.481 to 0.891	0.209 (0.303)		−0.440 to 0.816
PRESCH	→ TEACHER (path a)		0.738 (0.095)	0.540 to 0.909	0.291 (0.152)		0.005 to 0.597
SUPPORT	→ TEACHER (path d)		0.370 (0.111)	0.169 to 0.604	0.740 (0.116)		0.480 to 0.932
Indirect effects
PRESCH	→ TEACHER	→ IMLEVEL (path a*b)	−0.010 (0.054)	−0.117 to 0.098	0.008 (0.015)		−0.019 to 0.042
SUPPORT	→ TEACHER	→ IMLEVEL (path d*b)	−0.004 (0.022)	−0.049 to 0.040	0.012 (0.019)		−0.025 to 0.052
Model fit
95% CI for difference between observed and replicated
χ2 values			−15.435 to 20.059			−15.695 to 19.813
PPP values			0.426			0.426

Notes: Boldfaced indicates a significant effect with 95% CI not including zero. Estimates: average of the posterior means, SD: average of the posterior standard deviations, 95% CI: lower and upper bounds of the 95% CI. IMLEVEL = implementation levels; PRESCH = preschool capacity; SUPPORT = quality of prevention support system, SHAPES = Study of Health and Activity in Preschool Environments; TEACHER = teacher characteristics.

The final path diagram with standardized path coefficients is presented in Fig. 2. The hypothesized models explained 20% and 51% of the variance in level of implementation for Year 1 and Year 2, respectively. For Year 1, none of the selected factors had statistically significant direct or indirect associations with level of implementation. For Year 2, preschool capacity was found to have significant direct association with level of implementation ($\beta \hspace{0.5em}$= 0.443, 95% CI: 0.031, 0.795) (Table II). The result is interpreted as follows: for every one unit increase in the score of preschool capacity, level of implementation increased by an average of 0.53 units.

Discussion

The main finding of the present study was that the effects of preschool capacity on level of implementation differed by intervention years. Preschool capacity was found to be directly associated with level of implementation in Year 2, but not in Year 1. Teacher characteristics and quality of prevention support system were neither directly nor indirectly associated with level of implementation in both years. The different associations between the 2 years may suggest that our conceptual framework was inadequate to explain level of implementation during various phases of the implementation. Although our conceptual model was guided by the Durlak and DuPre framework [17] and previous empirical studies [18, 20–25], these studies only suggested factors that influenced level of implementation averaged across the intervention period, but not by individual intervention year. Therefore, it is possible that our framework left out the factors that are most salient in predicting level of implementation during an early intervention phase like year 1 of the SHAPES intervention.

However, the most salient factors that affect level of implementation in the early intervention phase has only been addressed by a few researchers. Fixsen et al. [32] suggested that, in addition to individuals’ skills, organizational capacity, and support system, whether the implementers can overcome several psychological barriers, such as fear of change and awkwardness associated with trying new things, may also exert a strong influence on level of implementation in the early intervention phase. Beets et al. [19] examined the effects of school climate, teachers’ belief and teachers’ attitude on curriculum delivery of a school-based positive youth development program. Although all three factors were positively and significantly associated with teacher’s curriculum delivery in two intervention years, the associations were stronger in the earlier than later years. However, none of these studies focused on physical activity interventions.

Teacher characteristics and quality of prevention support system had null associations with level of implementation in both years. These may be a result of lack of variability in these two variables, with most of the participating classrooms having high ratings for both variables. Although the lack of variability in these two variables might be due to insufficient sensitivity of the instrument in recognizing variability at the upper end [33], it is more likely that the consistently high ratings accurately reflect the positive impact of the intervention (i.e., training and on-going TA) on teachers’ perceptions of the programs, self-efficacy, and skills. The lack of variability could be because the training and TA were only provided by one investigation team, with standardized protocols for both components. We would expect more variability in teacher characteristics and quality of prevention support system if the support system involved multiple teams or if a train-the-trainer model was used.

Strengths of the current study include objective measures of level of implementation and comprehensive assessment of the preschools’ characteristics. Moreover, the use of the Bayesian estimation method provided the statistical power to test the proposed relationships that are usually lacking in implementation research. However, there are several limitations of the present study that warrant further exploration in future research. Findings from the path analysis cannot prove causality [34]. With a small sample, the stability of the path model should be viewed with caution; this study should be replicated in a larger sample. Although the participating classrooms are nested in eight preschools, we were unable to analyze the data with a multilevel model due to limited statistical power associated with the number of preschools and the number of classrooms per preschool (ranging from 1 to 4). The number of observation days changed from 8 days in Year 1 to 2 days in Year 2. This reduction may have limited the representativeness of the observation data in representing the implementation of the Move Outside components. Due to this methodological difference, data collected in the 2 years were analyzed as two cross-sectional samples. This may have prevented the examination of a potential longitudinal association between the selected factors measured in Year 1 and level of implementation in Year 2 [32]. Although multiple factors were measured, some potential determinants of the level of implementation were likely to have been omitted in this study, such as competing programs within the preschool, teachers’ beliefs and motivation [35].

In conclusion, preschool capacity, quality of prevention support system and teacher characteristics influenced implementation of the SHAPES Move Outside component differently across the 2 years, indicating that the level of implementation across different intervention phases may be influenced by different sets of factors. However, factors influencing program implementation appears to vary depending on the intervention contexts. To better understand what factors are associated with the success of intervention components, in what setting and in which phases of the intervention, researchers and practitioners need to build the evidence base by making the assessment of program implementation and its influences as a routine measure whenever feasible.