Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Dec 1.
Published in final edited form as: Health Educ Behav. 2017 Jan 4;44(6):876–884. doi: 10.1177/1090198116686334

Evaluating and Refining the Conceptual Model Used in the Study of Health and Activity in Preschool Environments (SHAPES) Intervention

Ruth P Saunders 1, Karin Pfeiffer 2, William H Brown 3, Erin K Howie 4, Marsha Dowda 5, Jennifer R O’Neill 6, Kerry McIver 7, Russell R Pate 8
PMCID: PMC5493997  NIHMSID: NIHMS860396  PMID: 28052694

Abstract

This study investigated the utility of the Study of Health and Activity in Preschool Environments (SHAPES) conceptual model, which targeted physical activity behavior (PA) in preschool children, by examining the relationship between implementation monitoring data and child PA during the school day. We monitored implementation completeness and fidelity based on multiple elements identified in the conceptual model. Comparing high-implementing, low-implementing and control groups revealed no association between implementation and outcomes. We performed post hoc analyses, using process data, to refine our conceptual model’s depiction of an effective preschool PA-promoting environment. Results suggest that a single component of the original four-component conceptual model, providing opportunities for moderate-to-vigorous physical activity (MVPA) through recess for 4-year-old children in preschool settings, may be a good starting place for increasing MVPA. Interventions that are implemented with optimal levels of completeness and fidelity are more likely to achieve behavior change if they are based on accurate conceptual models. Examining the mechanisms through which an intervention produces its effects, as articulated in the conceptual model that guides it, is particularly important for environmentally-focused interventions because they are guided by emerging frameworks. The results of this study underscore the utility of using implementation monitoring data to examine the conceptual model upon which the intervention is based.

Keywords: process evaluation, implementation monitoring, preschool children, conceptual models

The Study of Health and Activity in Preschool Environments (SHAPES) intervention focused on facilitating changes in preschool environments and instructional practices to create physical activity (PA)-promoting environments to increase PA in preschool children (Howie et al., 2014; Pfeiffer et al., 2013). SHAPES effectively increased moderate-to-vigorous PA (MVPA) in intervention compared to control schools (Pate et al., 2016). SHAPES was a group randomized control trial conducted in 16 preschools in the Southeastern US, with 8 intervention preschools (Pfeiffer et al., 2013). This process study included the parent study plus a one-year extension (2008–2011) (Howie et al., 2014). The mean age of the 567 children who participated in the three waves of SHAPES over three years was 4.5 years. About half (49%) were male; nearly half (47.8%) were African American, 38.3% were white and 13.9% were classified as “other/mixed” race.

SHAPES aimed to increase MVPA during the school day by creating PA-promoting preschool environments. The PA-promoting environment was defined by the components of the SHAPES conceptual model: providing PA opportunities via Move Inside, Move Outside, and Move to Learn in the context of a Supportive Social and Physical Environment. For full-day programs, complete delivery was defined as 60 minutes of PA opportunity per day. This could be achieved with at least 10 minutes of indoor, non-curricular PA opportunities (Move Inside); at least two 20-minute sessions of recess, including at least two 5-minute sessions of structured activity daily (Move Outside); and at least two 5-minute sessions of active learning (Move to Learn) (Howie et al., 2014; Pfeiffer et al., 2013). High fidelity delivery was defined as children enjoying PA and engaging in high levels of MVPA within a social environment in which adults modeled and encouraged PA.

The SHAPES chain-of-events logic model incorporated the intervention conceptual model. This model outlined how project activities (inputs) were expected to create a PA-promoting environment (outputs), which would result in greater PA among preschool children (outcomes). It also organized the comprehensive evaluation plan (Cooksy, Gill, & Kelly, 2001) (Figure 1). Intervention staff worked with preschool teachers, who in turn operated as organizational change agents (Commers, Gottlieb, & Kok, 2007) and carried out the intervention (Pfeiffer et al., 2013). As recommended (Durlak & DuPre, 2008), interventionists provided training, site visits, ongoing technical assistance, and resource materials (Howie et al., 2014).

Figure 1.

Figure 1

Open in a new tab

SHAPES process evaluation chain-of-events logic model, measures and data sources.

SHAPES implementation was flexible and adaptive striving to maximize PA opportunities throughout the school day (Howie et al., 2014). This approach has been successful in school-based interventions (Bond, Glover, Godfrey, Butler, & Patton, 2001; Patton, Bond, Butler, & Glover, 2003; Ward et al., 2006). Interventionists provided examples and targets for overall PA (300 and 150 minutes/week for full-day and half-day programs, respectively). However, each preschool teacher could achieve the intervention goals in a manner appropriate to her classroom environment. For example, a teacher might employ different configurations of minutes in Move In, Move Outside, and Move to Learn to achieve the common goal.

The importance of systematically-planned, conceptually-based interventions (Bartholomew, 2006; Green & Kreuter, 1999) that incorporate multilevel ecological models (Sallis, Owen, & Fisher, 2008; Stokols, 1992) is widely accepted in health promotion (Golden & Earp, 2012). For maximum impact, interventions should address contextual factors at ecological levels beyond the level of the individual (Stokols, 1996), They should also be informed by level-specific theory- and evidence-based strategies (Bartholomew, 2006; McLeroy, Bibeau, Steckler, & Glanz, 1988). Nevertheless, a conceptual model will be an effective guide to intervention planning only to the extent it accurately reflects influences on behavioral outcomes.

If a conceptual model does not address the determinants of behavior, it follows that the intervention based on that model, even if implemented with high fidelity, is unlikely to produce desired outcomes (Astbury & Leeuw, 2010; Chen, 2015). Conceptual models continually evolve based on new evidence. Therefore, the construct validity of conceptual models should be tested (Baranowski & Stables, 2000; Steckler & Linnan, 2002). Process evaluation can be applied to improve theory-based interventions by examining the effects of theory-based components on program outcomes (Baranowski & Stables, 2000; Steckler & Linnan, 2002), though few methods have been developed for conducting this type of examination (Haynes et al., 2016).

In this process evaluation study we investigate the appropriateness of the conceptual model that guided the SHAPES intervention. The specific purposes of this paper are to describe completeness and fidelity of intervention delivery at the classroom level by preschool teachers (Analysis A); examine the relationship between completeness and fidelity and PA outcomes in preschool children based on the conceptual model (Analysis B); and explore alternative conceptual models of a PA-promoting environment in preschools (Analysis C).

Methods

Process evaluation planning was guided by a systematic approach designed to collect quantitative implementation data based on the SHAPES conceptual model (Saunders, 2015). The process evaluation questions, addressed in Analysis A, were “To what extent did the change agents in preschool settings (teachers) provide PA opportunities via the SHAPES intervention components, Move Inside, Move Outside, and Move to Learn (completeness)?” and “To what extent were the components delivered with fidelity (i.e., fun and active within a socially-supportive environment)?” A variety of methods were used to address these questions, including a classroom observation checklist, child PA behavior observation, teacher survey, and interventionist ratings (described below and in Table 1).

Table 1.

Summary of Process Evaluation Methods

Characteristic of Change Model Addressed Year Data Sources Timing for Implementation Assessment Procedures
Completeness: children have opportunity to obtain MVPA
Move Inside: ≥ 10 min/day
Move Outside: ≥ two 20-min sessions of recess & 5 min of structured PA daily
Move to Learn: ≥ two 5-min activities daily		 1 Observe intervention implementation Fall: 4 days
Spring: 4 days		 Process evaluator used checklist to observe throughout school day; daily mean calculated
Teachers Once per year in Spring Self-completed survey; % weekly goal met calculated
2 & 3 Observe intervention implementation Fall: 1 day
Spring: 1 day		 Process evaluator used checklist to observe throughout school day; daily mean calculated
Teachers Once per year in Spring Self-completed survey; % weekly goal met calculated
Fidelity-PA: children were physically active during opportunity
Characteristics of PA Opportunities: ≥ 50% of opportunity time in MVPA		 2 & 3 Observe classroom level PA at 5-min intervals Fall: 1 day
Spring: 1 day		 Process evaluator used OSRAC-P to observe child PA during intervention components; mean daily % time in PA calculated
Total PA for whole school day: % time spent in Total PA during school day 1 Observe classroom level PA at 5-min intervals Fall: 4 days
Spring: 4 days		 Process evaluator used OSRAC-P to observe a subset of children throughout school day; % time in activity calculated
2 & 3 Fall: 1 day
Spring: 1 day
Fidelity-Social environment: modeling & prompting for PA and enjoyment
Social Environment:
-Teachers and adult staff verbally encourage PA in children during all PA time
-Teachers and adult staff actively participate in PA with children during all PA time		 1 Observe intervention implementation Fall: 4 days
Spring: 4 days		 Process evaluator used checklist to observe throughout school day; daily mean calculated
2 & 3 Fall: 1 day
Spring: 1 day
2 & 3 Teachers Self-completed survey; mean % weekly goal met calculated
2 & 3 Interventionists Interventionists used rating scale; mean calculated
Enjoyment: Children enjoy PA 2 & 3 Observe intervention implementation Fall: 1 day
Spring: 1 day		 Process evaluator used checklist to observe throughout school day; daily mean calculated
2 & 3 Teachers Once per year in Spring Self-completed survey; % weekly goal met calculated
Overall Implementation 1–3 Interventionists Once per year in Spring Interventionists used rating scale; mean calculated
Open in a new tab

Move Inside: adult-led, structured physical activity; Move Outside: recess; Move to Learn: daily lessons; Total PA: light + moderate + vigorous physical activity.

Abbreviations: PA, physical activity; MVPA, moderate to vigorous physical activity.

Process Evaluation Instruments and Procedures

The process evaluation methodology differed between Year 1 and Years 2 and 3 of the intervention. Classroom observations in Year 1 were made during selected times over four days per semester (fall and spring). In Years 2 and 3, classroom observations were done across an entire single day per semester due to resource constraints. Neither the core intervention components nor the process instruments changed. In all three years level of implementation was determined by triangulating among multiple data sources.

Completeness

Completeness (i.e., PA opportunities) was assessed via observation of minutes of PA opportunity and teacher self-report for all three years. Independent data collectors used the process observation checklist to record the number of minutes of PA opportunities provided across the school day, categorized by intervention component (Move Inside, Move Outside and Move to Learn). Components could be provided flexibly in brief periods throughout the day, so observations took place over the entire school day. In Year 1, the average of minutes across the four fall days and four spring days was used to calculate percent of daily goal met (300 and 150 minutes/week for full-day and half-day programs, respectively). The same procedure was used in the second and third intervention years, except the percent of daily goal met was based on the average of one day of observation in the fall and one day in the spring. Two data collectors observed 10% of both the process and OSRAC observation sessions to assess inter-rater reliability, which was > 0.80 for all categories for both methods.

Completeness also was assessed using a teacher survey, completed by the lead teacher in each classroom in the spring of each year. The teacher survey assessed self-reported frequency and duration of Move Inside, Move Outside and Move to Learn. A sample item is “Which of the following describes how much time was spent each day, on average, in Move Outside (recess)?” Response options were ≥30, 20–29, 10–19, 0–9 minutes; each response was converted to an average number (e.g., 20–29=25 minutes). Minutes of opportunity were summed for all components to yield total daily opportunity.

Fidelity

Fidelity was assessed three ways. First, the PA social environment (i.e., encouraging and modeling PA) was assessed for each component as a part of classroom observation in Years 1–3. When an opportunity was observed (Move In, Move Outside, Move to Learn), a 4-point scale (4=all of the time, 3=most of the time, 2=some of the time, 1=none of the time) was used to rate fidelity of the social environment. A sample item to assess social environment was “At least one teacher or adult staff actively participates in PA with children.” A yearly mean that combined components was calculated. In Years 2 and 3, teachers rated adult modeling of PA with a 3-point scale (1=supervise; 2=encourage the children to be physically active; 3=encourage and be active with the children) on the teacher survey and interventionists rated adult support for child PA on a 4-point scale (1=none of the time; 4=all of the time) each spring.

Second, the OSRAC-P (Brown et al., 2006) was modified to estimate group-level PA behaviors in all three years of the study. The OSRAC-P is a momentary time sampling observational system used to assess young children’s PA and associated contextual conditions (Brown et al., 2006). Group-level behaviors were assessed by classroom; a random selection of 6 students was observed for 5 minutes each during each 30-minute observation session. Four to seven 30-minute observation sessions, each with a different subset of 6 children, were conducted for each classroom on a given observation day. Two hundred seventeen hours of direct observation were collected to assess child PA across the school day, including during SHAPES components. In each year, a yearly average of the percent of intervals spent in total physical activity across the school day was calculated. In Years 2 and 3, process forms and the OSRAC-P were completed concurrently such that the percent of intervals spent in MVPA during Move Inside, Move to Learn, and Move Outside was calculated.

Third, in Years 2 and 3, child enjoyment of SHAPES was assessed by a data collector during class observation once in fall and once in spring using a 4-point scale (1=none of the time; 4=all of the time) each time a PA opportunity was observed, and by teacher rating with a 4-point scale for each PA opportunity component (1=hated it; 4=loved it) on the teacher survey. A sample item to assess enjoyment is “Most students appeared to enjoy PA.” A yearly mean that combined components was calculated.

Overall Implementation

Finally, interventionists rated implementation progress each spring for each intervention component for each of the three years using one item with a 4-point scale (4=substantial progress, 3=moderate, 2=minimal, 1=no progress); a single mean for all components was calculated by averaging two ratings (one per interventionist) for each year.

Child PA Measures: Accelerometer Data

The study was approved by the University of South Carolina Institutional Review Board (approval number Pro00004884). Written informed consent was obtained from children’s parents or guardians prior to data collection. The outcome measure for PA was measured by ActiGraph GT1M and GT3X (Pensacola, FL) accelerometers during a 5-day period (Monday–Friday). Measurement procedures have been published previously (Pfeiffer et al., 2013). This analysis used only time during preschool attendance. Days on which a child was present for <50% of the preschool day were excluded, and children with <3 days of monitor wear were excluded. Accelerometer data were reduced using cut-points developed for 3- to 5-year-old children to categorize intervals as MVPA(>420 counts/15-sec) and total PA (≥200 counts/15-sec) (España-Romero, Mitchell, Dowda, O’Neill, & Pate, 2013; Pate, Almeida, McIver, Pfeiffer, & Dowda, 2006). Minutes per hour of MVPA and total PA were calculated, using each child’s wear time during the hours of the school day as the divisor.

Statistical Analysis

Analysis A: Process Data: Completeness and Fidelity

The scores reflecting level of implementation for each data source were organized into a table by teacher/classroom. The criterion for complete implementation was defined as reaching at least 70% of the total PA opportunity goal (for all components combined). For PA fidelity and social environment fidelity, respectively, criteria were defined as children spent ≥20% of time in total PA during one school day as measured by OSRAC-P and an average rating ≥3 on a 1–4 rating scale. Thus, multiple data sources were triangulated to assess overall level of implementation each year (Table 2). Classification as “high” implementation in Year 1 required evidence of implementation from at least 4 of 6 (67%) data sources and in Years 2 and 3 from at least 4 of 7 data sources (57%), based on evidence that 60% or higher implementation is associated with desired program outcomes (Durlak & DuPre, 2008).

Table 2.

SHAPES Process Results Summary, Implementation by Classroom in Years 1, 2, and 3

Dose or Completeness Fidelity-PA Fidelity-Social Environment Overall Sum
Essential Element Category PA opportunity observed: % Criteria PA opportunity teacher: % weekly goal met % MVPA during PA opportunity† % Total PA (LMVPA) for 1 day Social Environment Overall Implementation Met criteria for higher implementation
Data Source Process observation Teacher reported OSRAC and process observation OSRAC observation Proc obs. Int. staff Staff, teach, process Teach, process Intervention staff Triangulated across all data sources
Year Year Year Year Year Year Year Year Year
Classroom/Teacher 1 2 3 1 2 3 2 3 1 2 3 1 1 2 3 2 3 1 2 3 1 2 3
1A * ✓ ✓ * 22 * * * ✓ * ✓ * *
1B(y1)/1D(y3) * ✓ ✓ * ✓ * 10 ✓ * ✓ * * ✓ ✓ * ✓ ✓ * ✓
1C ✓ ✓ ✓ 8 8 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
2A(y1)/2C(y2) ✓ ✓ * ✓ * 18 * ✓ * ✓ ✓ ✓ * ✓ * ✓ ✓ * ✓ ✓ *
2B ✓ ✓ ✓ ✓ ✓ ✓ 9 13 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
3A ✓ ✓ ✓ ✓ 8 9 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
3B ✓ ✓ 5 9 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
4A ✓ * * ✓ * * * * ✓ * * * * * * * * * *
4B ✓ ✓ ✓ ✓ ✓ 8 8 ✓ ✓ ✓ ✓ ✓ ✓ ✓
4C ✓ ✓ ✓ ✓ ✓ 10 13 ✓ ✓ ✓
5A(y1)/5B(y2,3) ✓ ✓ ✓ ✓ 10 7 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
6A ✓ ✓ ✓ 24 12 ✓ ✓ ✓ ✓ ✓ ✓
6B ✓ ✓ ✓ ✓ ✓ 12 24 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
7A(y1)/7D(y2,3) ✓ ✓ 7 5 ✓
7B ✓ ✓ 5 20 ✓ ✓ ✓ ✓
7C ✓ ✓ ✓ ✓ 1 14 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
8A ✓ ✓ ✓ ✓ ✓ 6 24 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
8B ✓ * ✓ * ✓ * 20 ✓ * ✓ * ✓ * ✓ * ✓ * ✓
8C ✓ ✓ ✓ ✓ 18 12 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
8D ✓ ✓ 19 12 ✓ ✓ ✓ ✓ ✓ ✓ ✓
% of teachers meeting criteria 65 53 76 60 53 76 11.2 12.9 50 53 47 35 45 41 65 88 88 45 65 71 35 53 76
Criteria ≥ 70% ≥ 70% ≥ 50% ≥ 20% ≥ 2.5 ≥ 3.0 ≥ 3/5 data sources ≥ 2/3 data sources ≥ 3.0 ≥ 4/6 ≥ 4/7 ≥ 4/7
Open in a new tab
†

Years 2 and 3 only; based on Move Inside and Move Outside

✓= Met criteria for implementation

Blank space=Criteria not met

*

= No classroom that year

Abbreviations: PA, physical activity; MVPA, moderate to vigorous physical activity, LMVA, Light plus MVPA.

Note: for classrooms, the number indicates the school and the letter indicates the teacher.

Analysis B: Associations between Completeness and Fidelity, and PA Outcomes Based on Initial Conceptual Model

Missing MVPA data at follow-up, assessed by accelerometer, were imputed for analysis (n=33 for wave 1, n=19 for wave 2, and n=22 for wave 3) using multiple imputation (data augmentation with Markov Chain Monte Carlo generation of imputed values) in SAS. The intervention and control groups were compared on demographic and PA variables with and without follow-up data. In the control schools, children with missing data at follow-up had higher values for MVPA at baseline than children with complete data.

Classrooms were grouped into implementation category (control, low and high) based on triangulated process data. A mixed analysis of covariance model was used to compare accelerometer-assessed MVPA minutes per hour among control, low- and high-implementing classrooms. All analyses were performed using Proc Mixed in SAS, adjusted for baseline, wave (or year), sex, race, parent education, and length of school day, with classroom treated as a random variable. For calculations of p-values, MVPA was square-root transformed.

Analysis C: Alternate Conceptual Models of the PA-Promoting Environment

In an intermediate step, we explored correlations between process variables and accelerometer-assessed Total PA for each year separately to assess construct validity of specific variables within the conceptual model. High correlations were considered evidence of construct validity and used to develop an alternate conceptual model. Classrooms were then grouped into low- or high-implementing classrooms based on an alternate conceptualization of the PA-promoting environment for all three waves of data. Mixed analysis of covariance models were used to compare the children in control, low- and high-implementing classrooms on MVPA.

Results

Analysis A: Process Data: Completeness and Fidelity

Table 2 presents an overview of the level of implementation for each classroom/teacher based on multiple data sources (see Supplemental Tables 1–3 for yearly results). For completeness, percentage of goal met in providing PA opportunities was similar in Years 1 and 2 and higher in Year 3 (60%, 53%, 76% for teacher report and 65%, 53% and 76% for process observation). No preschool met the criterion of 50% MVPA during PA opportunities in Years 2 or 3. PA Fidelity, based on total percentage of OSRAC-observed total PA during the school day, remained around 50% all three years. Social environment fidelity, based on observation, intervention staff rating and teacher rating, showed a similar pattern to teacher-reported completeness; teacher-reported child enjoyment was high in Years 2 and 3 (88% and 88%). Interventionist rating of overall implementation indicated improved implementation over time (45%, 65% and 71%). Based on triangulating data from multiple data sources, 35%, 53%, and 76% of preschool classrooms in Years 1, 2 and 3, respectively, met the implementation criteria. There was variability within schools and within a given classroom across time.

Analysis B: Associations between Completeness and Fidelity and PA Based on Initial Conceptual Model

Comparisons between control, low- and high-implementing groups based on the initial conceptual model for the PA-promoting environment and accelerometer-derived MVPA among preschool children revealed no significant associations between implementation level and outcomes, although means trended in the expected direction for females, with higher levels of PA for higher compared to lower implementers and lower implementers compared to controls(Table 3).

Table 3.

Comparison of Control, Low Implementing, and High Implementing Groups on School Day MVPA (minutes/hour), Mean (SE)

Comparison using triangulated data to define implementation1 Comparison using provision of MOVE OUT PA opportunities to define implementation2
Control Low Implementers High Implementers p Control Low Implementers High Implementers p
Total Sample (n=567) Total Sample (n=567)
MVPA 6.8 (0.2) 7.3 (0.3) 7.2 (0.3) .41 6.8 (0.2) 7.1 (0.3) 7.4 (0.3) .21
Males (n=278) Males (n=289)
MVPA 7.5 (0.2) 7.8 (0.3) 7.4 (0.3) .74 7.6 (02) 7.8 (0.4) 7.6 (0.3) .85
Females (n=289) Females (n=283)
MVPA 6.1 (0.3) 6.8 (0.4) 6.9 (0.3) .13 6.1 (0.3) 6.4 (0.3) 7.4 (0.3) .02
Open in a new tab
1

Adjusted for wave, sex, race, parent education and length of school day; p-value from square root transformed MVPA.

2

Adjusted for baseline, wave, sex, race, parent education, and length of school day; p-value from log transformed analysis. High implementers different from control and from low implementers; Low and control are not different.

Abbreviations: PA, physical activity; MVPA, moderate to vigorous physical activity

Analysis C: Exploring Alternate Conceptualizations of the PA-Promoting Environment

Correlational Study

Correlations between accelerometer-assessed total PA during the school day and the elements comprising completeness and fidelity varied (range: −.39 to .39), with some items not correlated or correlated in an unexpected direction (see Supplemental Table 4). There was, however, one suggestive pattern: Move Outside (recess) PA opportunity positively and significantly correlated with Total PA during the school day (i.e., teacher-reported in years 1 and 3 was r=0.37 and .27 and process-observed in years 1, 2 and 3 was r=.23, .32 and .39). This suggests that a single-dimension indicator, opportunities for PA through Move Outside (recess), may be a better way to conceptualize a PA-promoting environment. We explored the relationship between this single-dimension indicator of the PA-promoting environment and accelerometer-assessed study outcomes.

Associations between Move Outside PA Opportunities and Study Outcomes

High-implementation of Move Outside, compared to low-implementation and control, was significantly associated with more MVPA in girls but not boys (Table 3). Although not significant, the trend for total sample was also in the expected direction.

Discussion

We monitored implementation completeness and fidelity based on the elements identified in our four-component conceptual model (providing PA opportunities via Move Inside, Move Outside, and Move to Learn in the context of a Supportive Social and Physical Environment), which was informed by descriptive information (Brown, Pfeiffer, et al., 2009; Pate et al., 2006) and empirical investigations designed to increase PA in preschoolers (e.g., (Brown, Googe, McIver, & Rathel, 2009). However, our conceptual model of the PA-promoting preschool environment had not been validated empirically with preschool PA, and comparisons of control, low-implementing, and high-implementing groups revealed no association between implementation and outcomes. Given the positive intervention impact on MVPA (Pate et al., 2016), we performed post hoc analyses with our process measures to refine our conceptual understanding of an effective preschool PA-promoting environment.

The results suggest that a simpler conceptual model with one component, providing increased PA opportunities through Move Outside (recess) in the preschool setting, may be sufficient to increase school day MVPA. Being outdoors has been shown in a review of the literature to be correlated with PA in preschoolers (Hinkley, Crawford, Salmon, Okely, & Hesketh, 2008). However, it is possible that other components of SHAPES contributed in ways not assessed in this study, by influencing teacher norms or motivation to promote PA. Perhaps teachers accepted and practiced providing PA opportunities outdoors versus indoors, since a common convention is to keep children from moving in the classroom to maintain order. Or perhaps the social environment, in which adults model and encourage PA, could have more impact if it were implemented with higher fidelity.

A simpler conceptual model that is effective is important because changing multiple practices within the preschool setting is challenging. Stakeholders are asked to make difficult, time- and labor-intensive, and sometimes disruptive structural changes and would likely appreciate focused efforts based on an accurate conceptual model that addresses the minimal number of core activities needed to produce beneficial outcomes. Thus, a simple message about increasing PA opportunity outside would likely be easier to support.

SHAPES intervention delivery improved over the three years, possibly due to teacher experience, interventionist experience and/or the time needed for organizational change to take place. As is commonly reported in the literature (Alhassan & Whitt-Glover, 2014; Finch et al., 2014; Herbert et al., 2013), variability in implementation occurred over time for a given teacher and within a given school at any point in time. This variability suggests that classroom- and school-level factors influenced preschool teacher implementation, which we are investigating as a reflection of setting complexity (Craig et al., 2008; Foster-Fishman, Nowell, & Yang, 2007; Hawe, Shiell, & Riley, 2004, 2009).

Additional investigations should focus on identifying the most effective strategies for providing outdoor PA opportunities in preschool settings (Institute of Medicine, 2011; Physical Activity Guidelines for Americans Midcourse Report Subcommittee of the President’s Council on Fitness, Sports & Nutrition, 2012) and examining the role of integrated, indoor PA opportunities.

Study Strengths and Weaknesses

Study strengths include the randomized study design, conceptually-based intervention and evaluation approach, structural intervention, and comprehensive process evaluation. However, several limitations should be noted. The OSRAC-P, which has established reliability and validity (Brown et al., 2006; Brown, Pfeiffer, et al., 2009; Brown, Googe, et al., 2009), was modified for this study to observe multiple children’s levels of PA, to obtain a group (classroom) level estimate, versus an estimate for a single child for 30 consecutive minutes; however, inter-rater reliability was good. Process evaluation methodology changed between Years 1 and 2, which affected the ability to directly compare Year 1 with Years 2 and 3. We addressed this by conducting analyses by year and cautiously interpreting the suggestive patterns.

Implications for Theory and Practice

Practitioners and researchers should develop ecological conceptual models a priori, collect process data to quantify implementation of model-based intervention components, and examine the conceptual model underlying the intervention. This is important because conceptual models define the mechanisms through which the intervention produces desired outcomes. This study suggests that providing PA opportunities for 4-year-old children in preschool settings through recess may be a starting place for increasing MVPA, though additional exploration is needed. This work contributes to a conceptual understanding of a PA-promoting environment and may facilitate focused and effective change efforts within preschool settings.

Acknowledgments

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article is based on support from Eunice Kennedy Shriver National Institute of Child Health and Human Development (5R01HD055451). Clinicaltrials.gov Registry Number: NCT01885325

The authors thank Ann Blair Kennedy, DrPH, and Gaye Groover Christmus, MPH, for technical and editorial assistance in the development of the manuscript.

Footnotes

Declaration of Conflicting Interests

The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Contributor Information

Ruth P. Saunders, Department of Health Promotion, Education, and Behavior, University of South Carolina, Columbia SC, USA

Karin Pfeiffer, Department of Kinesiology, Michigan State University, East Lansing MI, USA.

William H. Brown, Department of Educational Studies, University of South Carolina, Columbia SC, USA

Erin K. Howie, School of Physiotherapy and Exercise Science, Curtin University, Perth, Western Australia

Marsha Dowda, Department of Exercise Science, University of South Carolina, Columbia SC, USA.

Jennifer R. O’Neill, Department of Exercise Science, University of South Carolina, Columbia SC, USA

Kerry McIver, Department of Exercise Science, University of South Carolina, Columbia SC, USA.

Russell R. Pate, Department of Exercise Science, University of South Carolina, Columbia SC, USA

References

  1. Alhassan S, Whitt-Glover MC. Intervention fidelity in a teacher-led program to promote physical activity in preschool-age children. Preventive Medicine. 2014;69:S34–S36. doi: 10.1016/j.ypmed.2014.07.024. http://doi.org/10.1016/j.ypmed.2014.07.024. [DOI] [PubMed] [Google Scholar]
  2. Astbury B, Leeuw FL. Unpacking black boxes: mechanisms and theory building in evaluation. American Journal of Evaluation. 2010;31(3):363–381. http://doi.org/10.1177/1098214010371972. [Google Scholar]
  3. Baranowski T, Stables G. Process evaluations of the 5-a-day projects. Health Education & Behavior: The Official Publication of the Society for Public Health Education. 2000;27(2):157–166. doi: 10.1177/109019810002700202. [DOI] [PubMed] [Google Scholar]
  4. Bartholomew LK. Planning health promotion programs: an intervention mapping approach. 1. San Francisco: Jossey-Bass; 2006. [Google Scholar]
  5. Bond L, Glover S, Godfrey C, Butler H, Patton GC. Building capacity for system-level change in schools: lessons from the Gatehouse Project. Health Education & Behavior: The Official Publication of the Society for Public Health Education. 2001;28(3):368–383. doi: 10.1177/109019810102800310. [DOI] [PubMed] [Google Scholar]
  6. Brown WH, Googe HS, McIver KL, Rathel JM. Effects of teacher-encouraged physical activity on preschool playgrounds. Journal of Early Intervention. 2009;31(2):126–145. http://doi.org/10.1177/1053815109331858. [Google Scholar]
  7. Brown WH, Pfeiffer KA, McIver KL, Dowda M, Addy CL, Pate RR. Social and environmental factors associated with preschoolers’ nonsedentary physical activity. Child Development. 2009;80(1):45–58. doi: 10.1111/j.1467-8624.2008.01245.x. http://doi.org/10.1111/j.1467-8624.2008.01245.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Brown WH, Pfeiffer KA, Mclver KL, Dowda M, Almeida MJCA, Pate RR. Assessing preschool children’s physical activity: the Observational System for Recording Physical Activity in children-preschool version. Research Quarterly for Exercise and Sport. 2006;77(2):167–176. doi: 10.1080/02701367.2006.10599351. [DOI] [PubMed] [Google Scholar]
  9. Chen H. Practical program evaluation: theory-driven evaluation and the integrated evaluation perspective. 2. Los Angeles: SAGE Publications; 2015. [Google Scholar]
  10. Commers MJ, Gottlieb N, Kok G. How to change environmental conditions for health. Health Promotion International. 2007;22(1):80–87. doi: 10.1093/heapro/dal038. http://doi.org/10.1093/heapro/dal038. [DOI] [PubMed] [Google Scholar]
  11. Cooksy LJ, Gill P, Kelly PA. The program logic model as an integrative framework for a multimethod evaluation. Evaluation and Program Planning. 2001;24(2):119–128. http://doi.org/10.1016/S0149-7189(01)00003-9. [Google Scholar]
  12. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ (Clinical Research Ed ) 2008;337:a1655. doi: 10.1136/bmj.a1655. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology. 2008;41(3–4):327–350. doi: 10.1007/s10464-008-9165-0. http://doi.org/10.1007/s10464-008-9165-0. [DOI] [PubMed] [Google Scholar]
  14. España-Romero V, Mitchell JA, Dowda M, O’Neill JR, Pate RR. Objectively measured sedentary time, physical activity and markers of body fat in preschool children. Pediatric Exercise Science. 2013;25(1):154–163. doi: 10.1123/pes.25.1.154. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Finch M, Wolfenden L, Morgan PJ, Freund M, Jones J, Wiggers J. A cluster randomized trial of a multi-level intervention, delivered by service staff, to increase physical activity of children attending center-based childcare. Preventive Medicine. 2014;58:9–16. doi: 10.1016/j.ypmed.2013.10.004. http://doi.org/10.1016/j.ypmed.2013.10.004. [DOI] [PubMed] [Google Scholar]
  16. Foster-Fishman PG, Nowell B, Yang H. Putting the system back into systems change: a framework for understanding and changing organizational and community systems. American Journal of Community Psychology. 2007;39(3–4):197–215. doi: 10.1007/s10464-007-9109-0. http://doi.org/10.1007/s10464-007-9109-0. [DOI] [PubMed] [Google Scholar]
  17. Golden SD, Earp JAL. Social ecological approaches to individuals and their contexts: twenty years of “Health Education & Behavior” Health Promotion Interventions. Health Education & Behavior. 2012;39(3):364–372. doi: 10.1177/1090198111418634. http://doi.org/10.1177/1090198111418634. [DOI] [PubMed] [Google Scholar]
  18. Green LW, Kreuter MW. Health promotion planning: an educational and ecological approach. 3. Mountain View, CA: Mayfield Pub. Co; 1999. [Google Scholar]
  19. Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ (Clinical Research Ed ) 2004;328(7455):1561–1563. doi: 10.1136/bmj.328.7455.1561. http://doi.org/10.1136/bmj.328.7455.1561. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. American Journal of Community Psychology. 2009;43(3–4):267–276. doi: 10.1007/s10464-009-9229-9. http://doi.org/10.1007/s10464-009-9229-9. [DOI] [PubMed] [Google Scholar]
  21. Haynes A, Brennan S, Redman S, Williamson A, Gallego G, Butow P CIPHER team. Figuring out fidelity: a worked example of the methods used to identify, critique and revise the essential elements of a contextualised intervention in health policy agencies. Implementation Science: IS. 2016;11(1):23. doi: 10.1186/s13012-016-0378-6. http://doi.org/10.1186/s13012-016-0378-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Herbert B, Strauss A, Mayer A, Duvinage K, Mitschek C, Koletzko B. Implementation process and acceptance of a setting based prevention programme to promote healthy lifestyle in preschool children. Health Education Journal. 2013;72(3):363–372. http://doi.org/10.1177/0017896912446553. [Google Scholar]
  23. Hinkley T, Crawford D, Salmon J, Okely AD, Hesketh K. Preschool children and physical activity: a review of correlates. American Journal of Preventive Medicine. 2008;34(5):435–441. doi: 10.1016/j.amepre.2008.02.001. http://doi.org/10.1016/j.amepre.2008.02.001. [DOI] [PubMed] [Google Scholar]
  24. Howie EK, Brewer A, Brown WH, Pfeiffer KA, Saunders RP, Pate RR. The 3-year evolution of a preschool physical activity intervention through a collaborative partnership between research interventionists and preschool teachers. Health Education Research. 2014 doi: 10.1093/her/cyu014. http://doi.org/10.1093/her/cyu014. [DOI] [PMC free article] [PubMed]
  25. Institute of Medicine. Early Childhood Obesity Prevention Policies. Washington, DC: The National Academies Press; 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. McLeroy KR, Bibeau D, Steckler A, Glanz K. An ecological perspective on health promotion programs. Health Education Quarterly. 1988;15(4):351–377. doi: 10.1177/109019818801500401. [DOI] [PubMed] [Google Scholar]
  27. Pate RR, Almeida MJ, McIver KL, Pfeiffer KA, Dowda M. Validation and calibration of an accelerometer in preschool children. Obesity (Silver Spring, Md ) 2006;14(11):2000–2006. doi: 10.1038/oby.2006.234. http://doi.org/10.1038/oby.2006.234. [DOI] [PubMed] [Google Scholar]
  28. Pate RR, Brown WH, Pfeiffer KA, Howie EK, Saunders RP, Addy CL, Dowda M. An intervention to increase physical activity in children: a randomized controlled trial with 4-year-olds in preschools. American Journal of Preventive Medicine. 2016;51(1):12–22. doi: 10.1016/j.amepre.2015.12.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Patton G, Bond L, Butler H, Glover S. Changing schools, changing health? Design and implementation of the Gatehouse Project. The Journal of Adolescent Health: Official Publication of the Society for Adolescent Medicine. 2003;33(4):231–239. doi: 10.1016/s1054-139x(03)00204-0. [DOI] [PubMed] [Google Scholar]
  30. Pfeiffer KA, Saunders RP, Brown WH, Dowda M, Addy CL, Pate RR. Study of Health and Activity in Preschool Environments (SHAPES): Study protocol for a randomized trial evaluating a multi-component physical activity intervention in preschool children. BMC Public Health. 2013;13(1):728. doi: 10.1186/1471-2458-13-728. http://doi.org/10.1186/1471-2458-13-728. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Physical Activity Guidelines for Americans Midcourse Report Subcommittee of the President’s Council on Fitness, Sports & Nutrition. Physical activity guidelines for Americans midcourse report: strategies to increase physical activity among youth. Washington, DC: U.S. Department of Health and Human Services; 2012. [Google Scholar]
  32. Sallis J, Owen N, Fisher E. Ecological Models of Health Behavior. In: Glanz K, Rimer BK, Viswanath K, editors. Health Behavior and Health Education: Theory, Research, and Practice. 4. San Francisco: Jossey-Bass; 2008. pp. 465–486. [Google Scholar]
  33. Saunders RP. Implementation monitoring and process evaluation. Thousand Oaks, CA: SAGE; 2015. [Google Scholar]
  34. Steckler AB, Linnan L, editors. Process evaluation for public health interventions and research. 1. San Francisco, Calif: Jossey-Bass; 2002. [Google Scholar]
  35. Stokols D. Establishing and maintaining healthy environments. Toward a social ecology of health promotion. The American Psychologist. 1992;47(1):6–22. doi: 10.1037//0003-066x.47.1.6. [DOI] [PubMed] [Google Scholar]
  36. Stokols D. Translating social ecological theory into guidelines for community health promotion. American Journal of Health Promotion: AJHP. 1996;10(4):282–298. doi: 10.4278/0890-1171-10.4.282. [DOI] [PubMed] [Google Scholar]
  37. Ward DS, Saunders R, Felton GM, Williams E, Epping JN, Pate RR. Implementation of a school environment intervention to increase physical activity in high school girls. Health Education Research. 2006;21(6):896–910. doi: 10.1093/her/cyl134. http://doi.org/10.1093/her/cyl134. [DOI] [PubMed] [Google Scholar]

RESOURCES