Sustainability of physical activity promoting environments and influences on sustainability following a structural intervention in residential children’s homes

Abstract

Research examining sustainability of health promotion programs within organizational settings is limited. The Environmental Interventions in Residential Children’s Homes (ENRICH) was a structural intervention that trained Wellness Teams (WTs) within residential children’s homes (RCH) to target environmental changes that promote physical activity (PA) among residential youth. This study examines the sustainability of PA promoting environments and influences on sustainability within RCHs. A sustainability survey was administered to 14 RCHs 2 years after receiving ENRICH. Variables included sustainability of PA promoting environments, Organizational Influences, perceived organizational and individual benefits, and implementation of PA and general (i.e. Global) wellness activities. Activities reported as sustained and barriers were used descriptively to inform sustainability. Path analyses explained the relationship between sustainability influences and sustainability of PA promoting environments. Sustainability was found in 8 of 14 (57%) RCHs. Sustained activities reflected greater Global versus PA implementation. Global implementation mediated the relationship between Organizational Influences and sustainability, which may have been more easily achieved since Global activities were most likely controlled by WTs and did not require extensive organizational support from RCH administrators. Results highlight the importance of defining and assessing different implementation types when measuring sustainability and influences on sustainability within RCHs organizations.

Introduction

The development of sustainable health promotion programs can inform evidence-based practices that lead to improved health outcomes [ 1 , 2 ]. Sustainability is often described as the continuation of program activities or benefits beyond initial funding [ 2–4 ]. However, the continuation of program activities must ultimately occur within ‘real-world’contexts [ 3 , 5 ].

Youth participation in regular physical activity (PA) is associated with positive life-long health behaviors that can mitigate the early onset and burden of chronic disease [ 6 ]. Organizational settings (e.g. schools) are ideally positioned to improve population-level health behaviors including PA by providing supportive structures and opportunities for youth to be active [ 7–10 ]. To date, few studies have examined the sustainability of youth PA programs beyond initial program implementation and evidence remains limited to school-based PA programs [ 11–15 ]. Furthermore, the definition and assessment of sustainability varies across study settings [ 1 , 3 , 4 ]. Although schools remain the most common setting for youth-based PA programs, a large minority of at-risk youth are underrepresented within health promotion research [ 9 , 16–19 ].

Residential children’s homes (RCHs) are group-based organizations that provide temporary and long-term services to foster youth who have been removed from their families due to abuse, neglect, abandonment or emotional problems [ 20 ]. In 2010, ∼400 000 children in the United States were placed in foster care [ 21 ]. Children who enter foster care often require greater services for physical and mental health conditions [ 22 , 23 ], are more likely to have social and behavioral problems compared to non-foster youth [ 24 ], and are more likely to experience worse health outcomes as adults [ 25 ].

The Environmental Interventions in Residential Children’s Homes (ENRICH) was a group randomized structural intervention, designed to increase the amount of time RCH youth spent in moderate-to-vigorous PA within residential RCH organizations in North and South Carolina by (i) offering greater PA opportunities and resources, (ii) increasing support from staff, (iii) increasing media messages and adult modelling (iv) and supporting the development of organizational policies to promote PA among youth [ 16 , 19 ]. To facilitate implementation of the ENRICH intervention, collaborative partnerships were formed between a university-based research team and RCH organizations throughout North and South Carolina [ 19 ]. RCH organizations were randomly assigned to Early (active intervention 2004–2006, n = 17) or Delayed intervention (control with active intervention 2006-2008, n = 12) condition. There were three phases of ENRICH: implementation (2 years), transition (1 year) and sustainability (Early condition only, assessed 2 years later). The development and implementation phases of ENRICH, including descriptions of the intervention, program goals, implementation approach, study design and individual behavioral outcomes have been reported [ 16 , 19 ] and are summarized here.

RCH staff formed Wellness Teams (WTs), that served as organizational change agents, and received trainings and technical support from ENRICH staff throughout the active intervention period. To enhance program implementation and sustainability, ENRICH used a flexible intervention approach that encouraged WTs to develop specific goals and activities that related to their particular needs, resources and RCH population [ 3–5 , 15 , 26 ]. As described previously in reference [ 19 ], extensive process evaluation measures were administered in both Early and Delayed intervention groups throughout the project to monitor fidelity and completeness of implementation. Criteria were established for assessing high or low implementation for ‘ PA implementation’ (development and implementation of PA objectives and plans to make PA opportunities available, and creating a social and media environment that supported PA) and ‘ Global implementation ’ (general implementation of WT plans, effectiveness of the WT in completing ENRICH tasks as planned, affecting policy changes and engaging RCH staff in support of ENRICH and environmental changes) [ 16 , 19 ].

ENRICH had no effect on increasing PA based on intent-to-treat analyses [ 16 ]. However, when controlling for random assignment, organizations with high PA and Global implementation, or more PA-promoting environments, were found to have significantly more active youth than low PA and Global implementing organizations [ 16 ]. Furthermore, ENRICH did have a significant impact on the PA social environment among the Early intervention group (unpublished data). Based on these results, it is important to examine the extent of sustained environmental change and related organizational activities. Moreover, the relationship between factors influencing sustainability of PA promoting environments within organizational settings remains limited to only a few school-based interventions [ 27 , 28 ].

The primary objectives of this study are to: (i) report the sustainability of PA-promoting environments following the ENRICH intervention and, (ii) examine influences on the sustainability of PA promoting environments targeted during the initial intervention phase of ENRICH.

Methods

Defining sustainability

ENRICH sustainability was defined as maintenance of PA-related activities implemented after the active intervention period, perceived ease of implementation of PA activities, perceived permanence of PA-related environmental changes, and prediction of future maintenance of PA-related environmental changes.

ENRICH sustainability was assessed in 2008; after the Early intervention group received the 2-year ENRICH intervention (plus 1 year of transition). The ENRICH sustainability goal was to identify RCHs that sustained PA promoting environments. The sustainability framework was informed by a comprehensive process evaluation plan that defined complete and acceptable delivery of ENRICH and was used to monitor and assess implementation of ENRICH elements during the active intervention [ 16 , 19 ]. The survey included items that fit conceptually with the definition of sustainability and potential influences on sustainability.

ENRICH sustainability design

Of the 17 RCH organizations that participated in the Early intervention from 2004 to 2006, 14 (82%) completed a 60-min telephone-based sustainability survey in 2008, 2 years after transitioning from the active implementation phase of the intervention in order to examine the extent to which ENRICH PA elements were maintained over time without ENRICH support. Sustainability surveys were administered by the ENRICH project coordinator and were completed by a member of the participating RCH WTs. The University of South Carolina Institutional Review Board approved this research study.

ENRICH sustainability survey and operationalizing study variables

The ENRICH Sustainability Survey includes 71 items on nutrition and PA. Data for this study used 43 items that reflected individual and organizational-level factors related to implementation and sustainability of PA promoting environments within RCHs. The survey included open-ended questions and scaled response items.

The first step in creating the sustainability variables involved an iterative process of reviewing and categorizing the survey items according to the sustainability framework to best reflect sustainability and influences on the sustainability process (i.e. sustainability constructs). Each construct was created using multiple items from the sustainability survey. Response format for most survey items used a four-point Likert scale to identify the extent to which PA elements or activities were maintained (e.g. 0 = not at all, 1 = partially, 2 = mostly or 3 = fully). Next, the initial study variables were created from the sustainability constructs.

The primary outcome variable, ‘ Sustainability of ENRICH PA’ reflects the maintenance or continuation of activities that supported PA promoting environments over the previous year, indicated by four constructs: ‘To what extent were PA activities implemented over the past year’ (14 prompted activities), ‘Ease of implementation or maintenance’ (three items), ‘Permanence of PA changes’ (single item) and ‘Prediction for future maintenance of PA changes’ (two items). A mean index score was then calculated and later included in the creation of the Sustainability of ENRICH PA variable.

‘ Organizational Resources and Support ’ reflects the availability of Organizational Resources and Support to maintain existing PA promoting environments, and is represented by five single-item constructs: ‘Sufficient staff and time, Adequate funding history, Collaborative resources, Program champion for implementation and maintenance’ and ‘Management support’. ‘ Organizational Infrastructure’ involves the organizational capacity to maintain PA promoting environments, identified by seven single-item constructs: ‘WT maintenance, Staff permanency, Clear job descriptions, Effective communication, Employee recognition, Daily procedures and processes’ and ‘Diffusion of ENRICH to other parts of the organization’. ‘ Organizational Planning’ refers to the continuation of developing wellness plans and monitoring the implementation of planned activities, and evaluating their impact. Organizational Planning was identified by two single item constructs: ‘Having a PA Plan’ and ‘Monitor implementation of plan and evaluate impact’. ‘ Perceived Organizational Effects’captured the overall impact of ENRICH within the RCH organization, as perceived by the WT respondents and was identified by two constructs, ‘Organizational benefits and capacity’ (mean of three-scaled items) and ‘ENRICH fit RCH needs’ (single-scaled item). ‘ Perceived Individual Effects’reflects the overall perceived impact of ENRICH to increase PA among RCH youth, identified by a single item, Individual benefits.

To account for the small sample size ( n = 14) and maximize statistical power, the number of initial variables was reduced to three by collapsing similar variables to form the final three variables included in subsequent analyses. Organizational Resources and Support, Organizational Infrastructure and Organizational Planning variables were combined to form ‘ Organizational Influences’ ; Perceived Organizational Effects and Perceived Individual Effects were combined to form ‘ Perceived PA Effects’ . No changes were made to ‘ Sustainability of ENRICH PA’ . Table I presents the operationalization of each study variable beginning with items from the sustainability survey and ending with the final study variables and response format.

Table I.

Operationalization of ENRICH sustainability and influences on sustainability

ENRICH Sustainability survey items	Sustainability constructs	Initial variables	Final variables	Response format
14 probing questions	PA activities implemented in past year a	Sustainability of ENRICH PA	Sustainability of ENRICH PA (Cronbach’s α = 0.799)	‘0 = not sustained’ ‘1 = very little sustained’ ‘2 = somewhat sustained’ ‘3 = all or most sustained’
Three items	Ease of PA implementation and maintenance
One item	Permanence of PA changes
Two items	Prediction for future PA maintenance
One item	Sufficient number of staff	Organizational Resources and Support	Organizational Influences (Cronbach’s α = 0.814)	‘0 = no org. support, infrastructure, planning’ ‘1 = little org. support, infrastructure, planning’ ‘2 = some org. support, infrastructure, planning’ ‘3 = much or high org. support, infrastructure, planning’
One item	Time in daily schedule
Two items	Adequate funding
One item	Resources through collaboration with other organizations
One item	Program champion for implementing and maintaining PA
One item	Management support
One item	WT maintenance	Organizational Infrastructure
One item	Permanent personnel
One item	Job descriptions
One item	Communication
One item	Employee development
One item	Daily procedures and processes
One item	Spread to other parts of organization
One item	Create a PA plan	Organizational Planning
One item	Monitor plan and impact
Three items	Organizational capacity benefits	Organizational Effects of ENRICH	Perceived PA effects (Cronbach’s α = 0.757)	‘0 = no effect’ ‘1 = little effect’ ‘2 = some effect’ ‘3 = large effect’
One item	ENRICH fit RCH needs
Three items	Individual benefits	Individual Effects of ENRICH

^a Response format (0 = not sustained, 1.5 = somewhat sustained, 3 = all or most sustained).

RCH characteristics and level of implementation

As previously reported in reference [ 16 ], general RCH characteristics were obtained from the ENRICH Organizational Assessment and included the number of residents, state location (NC/SC), RCH setting (rural/urban), RCH structure (simple = 0, complex = 1), presence of a recreation director (no = 0, yes = 1), on-site school (no = 0, yes = 1) and whether the organization was undergoing a major transition (no = 0, yes = 1).

Information on PA and Global implementation at the end of the active intervention were obtained from five data sources; WT contact, environmental observations, ratings from measurement and ENRICH staff. PA implementation was based on carrying out PA objectives, overcoming scheduling barriers to providing PA and providing PA opportunities whereas Global implementation was based on overall assessment of ENRICH implementation and progress (i.e. not PA-specific) [ 19 ]. Data were then triangulated to assess low or high level of PA and Global implementation based on predetermined criteria [ 19 ].

Data analysis

Assessing sustainability of ENRICH PA promoting environments

Sustainability of ENRICH PA was determined by calculating a mean score from seven items: six-scaled items and combined responses to the 14 probing questions that reflected PA activities maintained over the past year ( Table I ). Scores could range from 0 (not sustained) to 3 (fully sustained). The criteria for evidence of sustainability for ENRICH PA were informed by Durlak and DuPre [ 26 ] who, in a review of 483 studies summarized in five meta-analyses plus 59 additional studies assessing the impact of implementation on study outcomes, documented that implementation of at least 60% of program activities is associated with favorable health promotion program results . For this study, a cut-point score of 2.40 (80%) was used to indicate level of sustainability (low = <2.40, high = ≥ 2.40).

The 14 probing questions included in the sustainability survey served two purposes: (i) responses were used to create the ENRICH PA sustainability variable, as previously described; (ii) responses were used descriptively to inform which reported elements were sustained across RCH organizations; responses were dichotomized to reflect activities that were fully sustained or not.

One open-ended question elicited reported barriers to implementation and/or sustainability of PA activities within each RCH. Hand-written responses were thematically coded and frequency counts were used to quantify the extent to which coded themes were addressed across participating RCHs. Data were used descriptively to inform the results of ENRICH PA sustainability.

Assessing influences on sustainability of ENRICH physical activity PA promoting environments

Data were analyzed using SPSS (version 23). Descriptive statistics were reported for all study variables using frequencies (percentages) and means (SDs) for categorical and continuous variables, respectively. Cronbach’s alpha was used to assess internal reliability of the three variables created for this study, Organizational Influences, Perceived PA Effects, and Sustainability of ENRICH PA. Preliminary analyses were performed to select variables for the path model, bivariate correlations examined the strength of association between Organizational Influences, Perceived PA Effects, and Sustainability of ENRICH PA; level of PA and Global implementation (high versus low) and sustainability scores were compared using ANOVAs. Variables significantly associated with Sustainability of ENRICH PA were subsequently included for path analysis (analysis of a moment structures, AMOS) to examine the relationships between hypothesized influences on sustainability and the dependent variable, Sustainability of ENRICH PA. Because the hypothesized effects were directional, statistical significance was set at P < 0.10 for this investigation.

Results

RCH characteristics

A total of 14 RCHs completed the ENRICH Sustainability Survey in 2008. At the time the survey was administered, nearly 3500 youth were under the care of participating RCHs. Most RCHs were located in NC and were situated within rural settings (71%, respectively). Over half of RCHs were considered simple organizations, based on the number of youth residents and number and type of services provided, including the presence or absence of a recreational director. Presence of an on-site school was evenly distributed within the RCH sample, and 57% reported that their RCH was not undergoing any major organizational transition ( Table II ). Cronbach’s alpha for the three study variables ranged from 0.757 to 0.814, indicating good internal consistency ( Table I ).

Table II.

RCH characteristics and level of PA and Global implementation (2004–2006)

Group home	RCH Size ( n )	Location	Rural /Urban	Simple /Complex	Recreation director	On-site school	Organizational transition
1	96	NC	Rural	Simple	No	No	High
2	26	NC	Rural	Simple	No	No	Low
3	150	NC	Rural	Simple	Yes	Yes	High
4	617	NC	Rural	Simple	Yes	Yes	High
5	322	NC	Urban	Complex	No	Yes	Low
6	2	NC	Urban	Simple	No	No	Low
7	26	SC	Rural	Complex	No	No	High
8	47	SC	Urban	Simple	No	No	High
9	1400	NC	Rural	Simple	Yes	Yes	Low
10	1	NC	Rural	Simple	No	No	High
11	450	NC	Urban	Simple	Yes	No	Low
12	110	SC	Rural	Complex	Yes	Yes	Low
13	12	SC	Rural	Complex	No	Yes	Low
14	152	NC	Rural	Complex	No	Yes	Low
Total	3411	NC = 10	Rural = 10	Complex = 5	Yes = 5	Yes = 7	High = 6
		SC = 4	Urban = 4	Simple = 9	No = 9	No = 7	Low = 8

Implementation and sustainability

The ENRICH PA sustainability scores, sustainability classification and level of PA and Global implementation for participating RCHs are presented in Table III . All 14 organizations reported sustaining activities that supported PA promoting environments. Based on the sustainability scoring criteria of 80%, sustainability scores for eight RCHs (57%) were 2.40 or greater, indicating higher sustainability of ENRICH PA elements. Six RCHs (43%) met the criteria for lower sustainability. Approximately 43% ( n = 6) of RCHs were considered high PA implementers and 50% ( n = 7) were considered high Global implementers at the end of the 2-year intervention, respectively. Among the eight high sustaining RCHs, three were classified as high PA and Global implementers; three RCHs were low and high PA and Global implementers; two RCHs were classified as low PA and Global implementers. Level of PA and Global implementation also varied for RCHs classified as low sustaining organizations; three were low PA and Global implementers, two were high and low PA and Global implementers, respectively; one RCH was classified as a high PA and Global implementer.

Table III.

ENRICH PA sustainability scores, sustainability classification and level of PA and Global implementation by RCH

RCH	ENRICH PA sustainability score	Sustainability classification a	PA implementation	Global implementation
1	2.94	High	High	High
2	2.15	Low	High	Low
3	2.29	Low	High	Low
4	2.38	Low	High	High
5	2.40	High	High	High
6	2.60	High	Low	Low
7	2.67	High	Low	High
8	2.23	Low	Low	Low
9	2.36	Low	Low	Low
10	2.48	High	Low	High
11	2.86	High	Low	High
12	2.87	High	High	High
13	2.55	High	Low	Low
14	1.99	Low	Low	Low
		High = 8	High = 6	High = 7
		Low = 6	Low = 8	Low = 7

^a Level of sustainability (<2.40 = low; ≥2.40 = high).

As shown in Table IV , the overall mean Sustainability of ENRICH PA score suggests that PA promoting environments were reportedly sustained (2.48 ± 0.28). Results for Organizational Influences indicate some Organizational Resources and Support were available, a moderately supportive infrastructure was in place, and some Organizational Planning was done to provide PA promoting environments during the previous year (2.13 ± 0.63). The mean score for Perceived PA Effects were moderately high, suggesting that some positive effects were generally perceived regarding the changes that were made to support youth PA within the RCH organizations and that youth generally benefited from these changes (2.22 ± 0.35).

Table IV.

Mean scores for sustainability of ENRICH PA, Organizational Influences and perceived PA effects among RCHs (n = 14)

Initial variables	Final variables	Mean ± SD
Sustainability of ENRICH PA	Sustainability of ENRICH PA	2.48 ± 0.28
Organizational Resources and Support;	Organizational Influences	2.13 ± 0.63
Organizational Infrastructure;
Organizational Planning
Perceived Organizational Effects;	Perceived PA effects	2.22 ± 0.35
Perceived Individual Effects

The sustainability of PA elements targeted by ENRICH were obtained from 13 of the 14 RCHs ( Table V ). The most frequently reported elements sustained were providing opportunities for youth PA, ensuring that PA was enjoyable, ensuring that youth could access and use different kinds of PA equipment, and ensuring that youth had safe places where they could be active (92%, respectively); scheduling time for youth PA, ensuring free time for PA every day after school, and ensuring that youth could choose what types of PA they wanted to do (85%, respectively). Elements reported as least sustained included ensuring that time allocated for PA did not conflict with scheduled appointments and creating/enforcing PA policies to pay for uniforms and PA for teams and activities outside of the RCH (54%, respectively); ensuring that adults who provided PA programs were skilled or adequately trained, and creating/enforcing PA policies for transportation so youth could participate in sports and other activities outside of the RCH (46%, respectively).

Table V.

Descriptive summary of ENRICH PA elements sustained in RCHs at 1-year follow-up

ENRICH PA elements	RCH response n (%)
Opportunities for youth PA	12 (92%)
Ensure PA was enjoyable	12 (92%)
Ensure youth can get to and use many different kinds of PA equipment	12 (92%)
Ensure youth can get to safe places for PA	12 (92%)
Scheduled time for youth PA	11 (85%)
Ensure free time for PA after school, 5 days/week	11 (85%)
Ensure that youth could choose PAs they want to do	11 (85%)
Adults/staff encourage/support youth PA	10 (83%)
Emphasize adults/staff modeling PA	9 (69%)
PA media messages in RCH	8 (62%)
Ensure time for PA does not conflict with scheduled appointments	7 (54%)
PA policies to pay for uniforms, equipment for PA teams and activities outside RCH	7 (54%)
Ensure adults providing PA programs are skilled or trained	6 (46%)
PA policies for transportation to off-site PA teams/activities	6 (46%)

One RCH missing ( n = 13).

Barriers to implementation and sustainability

Table VI presents the reported barriers to implementing and sustaining ENRICH PA elements from 13 RCHs. The greatest barriers included motivating children to be active, staff resistance and scheduling time for PA (31%, respectively); high child turnover, training staff to promote or model PA, and cost to buy PA equipment (15%, respectively). The least reported barriers were setting PA goals, the need for PA equipment, and health conditions of the children (0.07%, respectively).

Table VI.

Reported barriers to implementing and sustaining ENRICH PA elements

Reported barriers	n (%)
Motivating children to be more active	4 (30.7)
Staff resistance	4 (30.7)
Scheduling time for PA	4 (30.7)
High child turnover rate	2 (15.3)
Training staff to promote/model PA	2 (15.3)
High rate of staff turnover/Need for more staff	2 (15.3)
Cost to buy PA equipment for indoor and outdoor use	2 (15.3)
Setting PA goals	1 (0.07)
Need for PA equipment	1 (0.07)
Health conditions of children	1 (0.07)

One RCH missing ( n = 13).

Factors associated with sustainability of ENRICH PA

Results from the bivariate correlations revealed that Global implementation and Organizational Influences were associated with Sustainability of ENRICH PA; r = 0.973, P = 0.000 and r = 0.582, P = 0.029, respectively. PA implementation was significantly associated with Organizational Influences ( r = 0.570, P = 0.033) and Global implementation ( r = 0.571, P = 0.033). Perceived PA Effects were associated with Global implementation only ( r = 0.597, P = 0.024). RCH size, state, location (urban/rural), setting (simple/complex), presence of a recreational director, on-site school and organizational transition were not significantly associated with sustainability ( P > 0.05). A 2 × 2 ANOVA tested the effects of Global (high versus low) and PA (high versus low) implementation at the end of the active intervention on sustainability scores. The main effect of Global implementation was significant with high implementers ( M = 2.66, SE = 0.09) having greater sustainability scores than low implementers ( M = 2.28, SE = 0.10), F ( 1 , 10 ) = 7.31, P = 0.022, _p η ² = 0.42. There was no effect of PA implementation on sustainability scores, nor was there an interaction effect between Global and PA implementation on sustainability (data not shown), P ’s > 0.05.

Organizational Influences, and Global implementation were related to each other as well as Sustainability, and Organizational Influences were related to PA implementation. It was hypothesized that Organizational Influence might be related to sustainability through Global and PA implementation. This indirect effect is of interest because it would provide direction on where resources could be allocated to increase sustainability. A path analysis with Bias-Corrected Bootstrapping was performed to examine the mediation of the relationship between Organizational Influences and Sustainability by Global and PA implementation ( Fig. 1 ). The Barron and Kenny framework [ 29 ] was used to assess mediation, and all steps were satisfied. All estimates reported were standardized. The final model fit the data well, χ ² ( 1 ) = 0.07, P = 0.792; Goodness of Fit Index (GFI) = 0.997; Root Mean Square Residual (RMR) = 0.004; Root Mean Square Error of Approximation (RMSEA) < 0.001; Normed Fit Index (NFI) = 0.996. The three endogenous variables’ total effects were significant in the model, PA implementation ( R² = 0.16, P = 0.008), Global implementation ( R² = 0.34, P = 0.009) and Sustainability ( R² = 0.52, P = 0.100). There was no significant direct effect of Organizational Influences on Sustainability, b = 0.17, P = 0.45; however, there was a significant indirect effect of Organizational Influences on Sustainability through Global Implementation, b = 0.27, P = 0.065, but not through PA implementation, b = −0.09, P = 0.33.

Fig. 1.

Results of the PATH analysis for associations between Organizational Influences, PA and Global implementation, and ENRICH PA sustainability.

Discussion

Sustainability of ENRICH PA

Over 80% of responding RCHs reported sustaining 8 of the 14 ENRICH PA elements: providing opportunities for youth PA, ensuring PA is enjoyable, ensuring youth could get to and use different types of PA equipment, ensuring that youth could access safe places to be active (92%, respectively), scheduling time for PA, ensuring free time for PA after school, ensuring that youth could choose the type of PA they wanted to do (85%, respectively), and getting adults and staff to encourage and support youth PA (83%). The most commonly reported barriers to implementing and sustaining ENRICH PA were motivating children to be active, staff resistance and scheduling time for PA (30.7%). Based on the seven-item measure for sustainability, evidence for higher sustainability of ENRICH PA was found in 8 of 14 RCHs (57%).

Several school-based PA interventions have demonstrated evidence for sustaining program components over time [ 11 , 12 , 14 , 15 ]; however, most have focused on sustained curricular changes and instructional practices. ENRICH targeted changes within RCH environments that promoted youth PA. Due to differences in organizational setting, function, population, programmatic activities and methods to assess sustainability, it is difficult to compare results across studies. The LEAP study, however, focused on environmental changes within the school organization to promote PA among adolescent girls [ 10 ]. Saunders et al. [ 15 ] used multiple process data sources to document LEAP sustainability 3 years after the intervention. Six schools demonstrated consistent evidence of sustained instructional practices and three schools sustained environmental changes [ 15 ].

The methods to inform the sustainability of ENRICH PA were guided by the LEAP sustainability framework [ 15 ] and previous literature [ 4 , 26 ]. Unlike in LEAP, ENRICH developed an instrument to assess the extent to which PA elements were sustained in each RCH. It is recommended that planning for sustainability coincides with the initial phases of program development in which methods and measures can be specifically developed to monitor implementation and diffusion of the program throughout the organization [ 3 , 30 ]. ENRICH accomplished this by creating a conceptual framework and instrument to assess sustainability of ENRICH PA as part of the comprehensive ENRICH process evaluation plan. The sustainability assessment included potential influences on sustainability that were also examined in this study.

Influences on the sustainability of ENRICH PA

Results from path analyses revealed that Organizational Influences had a significant indirect effect on the sustainability of ENRICH PA that was mediated by Global implementation. These findings suggest that organizational factors including available resources, support, communication, infrastructure and planning are important for building organizational capacity to implement activities that support PA throughout all levels of the RCH organization. Organizational factors including strategic planning, organizational capacity, evaluation and communication are important influences on program implementation and future sustainability [ 30 ]. Developing organizational capacity to implement, monitor, evaluate and maintain health promotion practices may serve as a critical first step to creating physical and social environments that promote health [ 31 ].

Evidence suggests that implementation varies across different organizational settings [ 15 , 16 , 32 ]. As previously reported in reference [ 16 , 19 ], ENRICH implementation was rigorously monitored using multiple data sources; consequently, this enabled the conceptualization and assessment of PA and Global implementation. Results of this study appear to support the potential of Organizational Influences on Global implementation only. Reasons as to why Global but not PA implementation mediated the relationship between Organizational Influences and sustainability of ENRICH PA remain unclear. It is possible that the implementation of PA objectives and activities to create PA promoting environments (PA implementation) may have been challenging without a strong organizational infrastructure to facilitate PA programming [ 30 ]. Conversely, Global implementation reflected the general implementation of WT plans, effectiveness of the WT in completing ENRICH tasks as planned, affecting policy changes, and engaging RCH staff in support of ENRICH and environmental changes. These more general activities may have reflected the development of effective working relationships between ENRICH staff and WTs in conjunction with broader organizational support for environmental change [ 33 ]. Global implementation may also have been easier to achieve as the responsibility for implementing the various ENRICH PA elements rested on the WTs themselves, who may have had greater control in implementing selected activities given infrastructure challenges. Indeed, descriptive results indicate that most ENRICH PA elements reported to be sustained were within the direct control of the WTs (e.g. provide opportunities for youth PA, ensure PA was enjoyable and youth had a choice in the types of PA offered). In contrast, the least sustained elements required greater reliance on RCH direct care staff to support and model PA, in the face of considerable staff turnover, and the creation/enforcement of policies that promoted PA beyond the RCH organization, which may require additional organizational resources. This interpretation is consistent with results of school-based PA interventions whereby teacher-led instructional practices are more frequently implemented and sustained compared with broader organizational/environmental changes [ 11–13 , 15 , 27 ]. Additionally, teacher-led implementation can be further enhanced with sufficient training, establishing greater accountability, providing flexibility and ensuring program activities fit with the broader organizational climate and mission [ 11–13 ]. Other influences include program design, flexibility of program activities and procedures, program setting and broader socio-political factors [ 2–4 , 29 ].

Strengths and limitations

This is the first known study to report on the sustainability and influences on sustainability of a structural intervention that aimed to create PA promoting environments within RCH organizations. Strengths of this study include the use of a conceptual framework of sustainability, the development of a sustainability instrument that also assessed influences on sustainability, and use of a rigorous scoring cut point (80%) to determine sustainability. Moreover, assessing level of PA and Global implementation at the end of the active intervention informed sustainability and influences on sustainability in both descriptive and quantitative analyses. Finally, the inclusion of PATH analysis explained the directional relationship between hypothesized influences on sustainability and sustainability of ENRICH PA.

This study has limitations. First, data were obtained from a single, self-report measure of sustainability, although multiple items were used to conceptualize and assess sustainability. Second, the cross-sectional design limited the ability to make causal inferences [ 34 ] and study findings cannot be generalized beyond RCH organizations in the southeastern region of the United States. Third, due to the small sample of RCH organizations included in this study, initial study variables had to be combined. By reducing the number of variables, the extent to which the initial study variables may have explained the relationship with sustainability of ENRICH PA is unknown; however the final set of study variables demonstrated good internal consistency reliability, though future validation studies are needed. Fourth, although ENRICH sustainability was observed 2 years after RCHs received the active intervention, it is unclear whether ENRICH activities have been maintained since data were collected in 2008. For most health promotion programs, long-term measures of sustainability are often constrained by limited funding timelines [ 3 , 5 ]. ENRICH used a 2-year, group-randomized cross-over design that enabled sustainability to be assessed in 82% of RCHs who first received the intervention (2004–2006). Finally, we measured organizational sustainability of program activities in this study but did not include assessments of youth behavior or health outcomes.

Future recommendations

This study adds to the paucity of existing literature demonstrating the utility of using process evaluation and implementation monitoring data to determine the sustainability of PA promoting environments in organizational settings. Furthermore, we provide a framework for assessing sustainability and influences on sustainability within RCH organizations. Future research should examine the feasibility of applying this framework to other organizational settings. Research is also needed to examine the impact of sustained PA promoting environments on PA behaviors among youth residents. Furthermore, health promotion programs that are implemented within organizational settings should assess an organization’s capacity to implement program components throughout all levels of the organization, prior to program implementation.

Funding

This work was supported by The Duke Endowment. Any opinions, findings, conclusions or recommendations contained herein are those of the authors and do not necessarily reflect the views of The Duke Endowment.

Conflict of interest statement

None declared.