A mixed-methods exploration of implementation of a comprehensive school healthy eating model one year after scale-up

Abstract

Objective

To study the implementation of a school-based healthy eating (HE) model one year after scale-up in British Columbia (BC). Specifically, to examine implementation of Action Schools! BC (AS! BC) and its influence on implementation of classroom HE activities, and to explore factors associated with implementation.

Design

Diffusion of Innovations, Social Cognitive and Organizational Change theories guided our approach. We used a mixed-methods research design including focus group interviews (seven schools, sixty-two implementers) and a cross-sectional multistage survey to principals (n 36, 92 % response rate) and teachers of grades 4 to 7 (n 168, 70 % response rate). Self-reported implementation of classroom HE activities and reported use of specific AS! BC HE activities were primary implementation measures. Thematic analysis of focus group data and multilevel mixed-effect logistic regression analyses of survey data were conducted.

Setting

Elementary schools across BC, Canada.

Subjects

Thirty-nine school districts, thirty-six principals, 168 grade 4 to 7 teachers.

Results

Forty-two per cent of teachers in registered schools were implementing AS! BC HE in their classrooms. Users were 6·25 times more likely to have delivered a HE lesson in the past week. Implementation facilitators were school champions, technical support and access to resources; barriers were lack of time, loss of leadership or momentum. Implementation predictors were teacher training, self-efficacy, experience with the physical activity component of AS! BC, supportive school climate and parental post-secondary education.

Conclusions

Our findings reinforce that continued teacher training and support are important public health investments that contribute to successful implementation of school-based HE models after scale-up.

Environments and lifestyles have undergone visible changes in recent decades. Indeed, these changes have been implicated in the epidemic of physical inactivity and unhealthy eating habits that contribute to obesity and chronic disease risk^{( 1 – 4 )}. Specific dietary culprits are consumption of non-nutritious energy-dense foods and beverages, increased portion sizes, low fruit and vegetable intake, eating away from home and consuming convenience/fast food at home. Overall these factors contribute to increased energy intake^{( 3 )}. The health consequences of these changes are substantive and beginning in childhood have been associated with decreases in quality of life^{( 5 )}, increases in overweight/obesity^{( 6 )} and increased chronic disease risk^{( 7 – 10 )}. Thus, it seems imperative to address these modifiable dietary factors during childhood as one means to counter these disturbing public health trends.

Schools are popular settings for healthy eating (HE) interventions as children from across social strata spend considerable time in school and consume at least one meal or snack there^{( 11 , 12 )}. Further, schools traditionally provide nutrition education to students^{( 13 )}. Finally, schools represent natural ‘settings’ where interventions can be targeted across multiple levels, from the student to the global school environment^{( 12 , 14 )}.

School-based HE interventions, especially those that incorporate environmental strategies, have had modest positive impacts on knowledge and eating behaviours^{( 14 – 18 )}. Thus they are a recommended component of public health action for young people^{( 11 )}. To take meaningful action we need to better understand how effective preventive health interventions can be scaled up, implemented and sustained to achieve desired outcomes^{( 19 – 24 )}. Key aspects of diffusion of innovations (innovation characteristics)^{( 25 – 28 )}, social cognitive (adopter/teacher)^{( 25 , 27 , 28 )}, social marketing^{( 29 )} and organizational theory (organization/school)^{( 25 – 32 )} have been key to successful implementation over the short (efficacy trials) or long term (maintenance/sustainability trials). Planning for the state-wide scale-up of the Child and Adolescent Trial for Cardiovascular Health (CATCH) in Texas^{( 29 )} identified the importance of factors such as networks, cost, training, ongoing support, targeted approaches and the importance of both supporting and examining implementation after adoption. However, we know relatively little about factors that influence implementation of school HE interventions after scale-up in real-world settings; this deficit must be overcome to enhance the effectiveness of target interventions^{( 33 – 37 )}.

Currently being implemented in British Columbia (BC), Canada, Action Schools! BC (AS! BC) is a comprehensive school health-based model that engages elementary-school stakeholders and supports their capacity to create individualized action plans to enhance HE and physical activity (PA) opportunities for children. The model targets six key ‘zones’ for action: (i) the environment (including policies); (ii) the classroom; (iii) physical education; (iv) extra-curricular; (v) school spirit; and (vi) family and community^{( 31 )}. The PA and HE components of the overall model were developed, evaluated and scaled up using a phased approach (starting with PA). The PA model was designed and evaluated during the first phase and after demonstrating efficacy^{( 4 , 31 , 32 , 38 )} was scaled up in the 2004–05 school year. Then, after the feasibility and efficacy of the HE component were demonstrated (2006–07)^{( 39 )}, it was integrated into the AS! BC model and scaled up in the autumn of 2007 (Phase II).

Scale-up was planned and implemented by a central management team and technical support unit that shoulders an array of duties including: working with stakeholders; marketing the initiative; registering schools and teachers; preparing and supporting regional trainers (n 75) and two Master trainers; providing resources (HE ‘bin’ of equipment, materials and curriculum resources) and training for teachers and the school community; and providing direct email, telephone and web-based technical support to teachers and school administrators. An overview of the AS! BC HE intervention package, which has been described previously^{( 39 )}, is summarized in Table 1.

Table 1.

Key components of the Action Schools! BC (AS! BC) Healthy Eating (HE) intervention

Dose/component	Description
Dose	The HE intervention included lessons and experiential activities for the classroom. In Phase I, teachers were asked to provide two HE activities/week and one tasting activity/month in the classroom( 39 ). In Phase II, based on formative evaluation results, three of the six Action Zones (School Environment, Classroom Action, and Family & Community) were emphasized and teachers were asked to implement one multi-lesson Vegetable & Fruit unit per term (i.e. two units/year), as well as a tasting activity every month and a minimum of three other whole school community activities which included either activities that targeted the school environment or activities that targeted the family and community zone (e.g. Fruit and Vegetable Fridays)
Classroom HE Resource (CHEAR)	A set of activities and lesson plans to help teachers plan classroom HE lessons and activities
Action Pages!	A resource directory to link teachers with ‘field tested’ resources to use in their HE lessons and activities
Classroom HE Action Pack	A kit of HE equipment that supports classroom lessons and experiential activities like taste testing (e.g. colanders, ice bags to keep vegetables cold, cutting boards and knives, placemats, food models, etc.)
Support Team & Training	A central technical support unit provided the AS! BC HE resources (training workshops – one per school year, written materials, Action Packs, school newsletter inserts for families, etc.), bi-monthly school visits from a Master trainer and ongoing telephone support to school staff
School Action Team	A committee of school stakeholders (e.g. interested teachers, administrators, parents) helped to create a HE Action Plan for the school and to support implementation
Whole School Community Activities	In addition to the classroom action zone, actions were planned across five other zones. The Family & Community zone involved activities such as healthy food fundraising and healthy school feasts. School-wide events could be something like Fruit and Veg Fridays or a fruit and vegetable challenge. School environment could include applying for the provincial school fruit and vegetable snack programme or planting a vegetable garden

Provincial scale-up of AS! BC HE presented us with the unique opportunity to assess implementation and the factors that influenced it after scale-up. Specifically, our study set out to: (i) identify the contextual factors that influenced implementation; (ii) describe implementation and its relationship with the provision of HE lessons in the classroom; and (iii) assess how the characteristics of the innovation, schools and teachers were hierarchically associated with implementation after scale-up.

Experimental methods

Procedures

The study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human subjects/patients were approved by both the Clinical Research Ethics Board at the University of British Columbia and the Human Research Ethics Board at the University of Victoria. Recruitment was conducted in the 2007–08 school year. We adopted the same recruitment protocols and data collection procedures that were used in the AS! BC PA implementation study^{( 40 )}. Briefly, principals received a letter/email inviting them to participate. Volunteer schools supported recruitment by identifying a person who would act as the contact and study coordinator. These coordinators then distributed teacher invitations and collected the contact information of any volunteers. On data collection day the coordinator received and distributed a sealed package with a unique code and a web address link to an online questionnaire (Class Climate V5·1; Scantron Corporation) to each participant. We also funded a substitute teacher so that participants had time to complete the survey and provided honorariums. Focus groups were conducted during the school lunch period and lunch was provided.

Research design and theoretical framework

We used mixed methods to explore implementation after scale-up. We used a qualitative approach^{( 41 )} to explore contextual factors and as previously described by Mâsse et al., we used concepts from Rogers’ Diffusion of Innovations, Social Cognitive and Organizational Change theories^{( 42 – 45 )} to guide the quantitative examination of implementation and the hierarchy of theoretically derived influences on it^{( 40 )}. We analysed data from the qualitative and quantitative methods independently and integrated data during the interpretation stage^{( 46 )}.

Sample

We obtained school district approval from thirty-five out of fifty-nine BC school districts (59 % response rate; excluding three school districts that piloted the survey). For the qualitative evaluation of contextual factors that influenced implementation, we purposively selected seven schools that were implementing AS! BC HE in the first year of scale-up and represented a broad range of school types and geographic locations. Sixty-two staff across seven schools participated in focus groups. For the quantitative evaluation of implementation that examined the hierarchy of factors influencing implementation, we conducted a cross-sectional multistage survey of school principals and teachers of grades 4 to 7 in BC (2008–09 school year). All schools that received training to deliver AS! BC HE, during the 2007–08 school year, were eligible to participate. Exclusion criteria included pilot, Independent, Francophone and First Nations schools. Of the eighty-eight eligible schools, thirty-nine agreed to participate (44 % response rate). Within participating schools, 36/39 principals (92 % response rate) and 168/238 grade 4 to 7 teachers (70 % response rate) completed the survey.

Instruments

Implementation context

We explored implementation qualitatively using semi-structured focus group interview questions that addressed different aspects of the AS! BC HE model: (i) its influence within the school and community; (ii) the response of teachers and administrators in the school; (iii) implementation challenges; and (iv) facilitators and changes necessary to sustainability.

Implementation

We assessed ‘current use of the AS! BC HE model’ using a seven-item measure modelled after the Steckler et al. current use questions (‘yes’/‘no’)^{( 47 )}. For the quantitative analyses, items were dichotomized to determine whether teachers were current users or not of the model. Non-users represented those who said they never used the model, were contemplating or preparing to use the model, or reported they had used the model in the previous school year but were not current users. In contrast, users represented those who indicated using the model in the current school year no matter what frequency and duration of use. Validity of this dichotomous index was assessed by correlating it with a measure that assessed whether the teachers were using specific AS! BC HE materials and resources (Pearson correlation =0·50).

We assessed implementation as ‘classroom delivery of HE lessons/activities’, by asking (i) how many times in the past week teachers had incorporated HE lessons or activities into their class routine (0–7) and (ii) whether they had used or displayed (in the past or recently) instructional resources and materials, tracked consumption, prepared and/or tasted fruit and vegetables or went on a field trip (yes/no). In each of these categories AS! BC HE users were asked about specific AS! BC promoted resources while non-users answered generic questions.

Implementation factors; characteristics of schools, teachers and innovation

We assessed ‘organizational climate’ with one item that asked principals whether HE was a priority for their school (5-point response ranging from ‘strongly disagree’ to ’strongly agree’).

We assessed ‘organizational capacity’ with four items that asked school principals whether: (i) teachers had enough time to implement AS! BC; (ii) their schools had resources for the implementation of AS! BC; (iii) their schools had funds to support the implementation of AS! BC; and (iv) they themselved were committed to provide the resources to the implementation of AS! BC (5-point response ranging from ‘strongly disagree’ to ‘strongly agree’). Results from the factor analysis demonstrated that this scale measured one main dimension explaining 57·3 % of the total variance with a Cronbach’s α of 0·80.

We assessed ‘level of institutionalization’ with nine items that were modelled after Goodman et al.’s^{( 48 )} measure. The scale assessed whether the school had implemented guidelines and practices to ensure the uptake and implementation of AS! BC (i.e. made AS! BC part of school culture, established guidelines/policies, adapted the concepts to fit the schools and allocated resources to ensure uptake; ‘yes’/‘no’ or ‘yes, established clearly’, ‘yes, somewhat established’ and ‘no’). A weighted average was computed to account for the differences in response format. Results from the factor analysis demonstrated that this scale measured one main dimension explaining 35·7 % of the total variance with a Cronbach’s α of 0·83.

We assessed ‘environmental influences’ by measuring whether the school was in compliance with recent nutritional guidelines mandated by the Province level (equivalent of State level). Specifically, this item assessed the extent to which the schools had eliminated access to and availability of less healthful foods and beverages at school (vending machines, cafeteria and/or school store), for fundraising activities, at special school events or in the classroom (‘yes, fully implemented’, ‘in the process of implementing’ and ‘no’). This index was dichotomized into yes/no for the analysis.

We assessed teacher ‘self-efficacy’ with seven items that were modelled after the Brenowitz and Tuttle scale^{( 49 )}. The scale measured whether teachers: (i) understood the concepts of AS! BC; and (ii) had confidence in their skills, their ability to devote time and their ability to motivate and engage students in HE activities (5-point response ranging from ‘strongly agree’ to ‘strongly disagree’). Results from the factor analysis demonstrated that this scale measured one main dimension explaining 51·2 % of the total variance with a Cronbach’s α of 0·88.

We assessed ‘behavioural capacity’ of the teacher with two items. One item measured if the teacher received AS! BC training for HE (‘yes’/’no’). One item measured whether the teachers were users of the AS! BC PA component.

We asssessed ‘innovation attributes’ using an eleven-item scale modelled after Rogers’ adoption questions^{( 47 )} using a 5-point response format ranging from ‘strongly agree’ to ‘strongly disagree’. We assessed whether teachers’ perceived AS! BC HE to be: (i) advantageous compared with other initiatives; (ii) complex to use; (iii) compatable with their teaching philosophy; (iv) easy to try in their classroom; and (v) if it created positive outcomes. Results from the factor analysis demonstrated that this scale measured one main dimension explaining 47·8 % of the total variance with a Cronbach’s α of 0·91.

Covariates

We assessed the following variables as covariates: gender and teaching experience of teachers, urbanicity, size of school and percentage of the population around the school with post-secondary education. We obtained school-level covariates by linking the school postal code with information from the Ministry of Education and the 2006 Canadian Census.

Statistical analyses

Qualitative data

Upon completion of each focus group, we transcribed each audio recording verbatim and imported them into the software NVivo 2·0 for analysis. This allowed us to create a framework for organizing and theming data. Data were initially grouped into NVivo nodes based on the focus group questions. We conducted basic content analysis to examine the data for meaning. Text was grouped into preliminary categories, re-examined for recurring themes and relationships, and reduced into key themes and sub-themes^{( 50 )}.

Quantitative data

We used the statistical software package Stata version 13·1 to generate descriptive characteristics of participants. We developed a mixed-effects logistic regression analysis to determine if implementation of HE activities was significantly associated with current use of AS! BC HE (described in the ‘Instruments’ section as a dichotomous indicator which assessed whether teachers were current users or not). We developed a multilevel mixed-effect logistic regression model to identify factors that influenced AS! BC HE implementation. To account for the nested structure of the data we developed a two-level effect model with schools and school districts entered as separate levels while attributes of the innovation, characteristics of the teachers (self-efficacy, training received and users of AS! BC) and characteristics of the schools (organizational climate, organizational capacity, level of institutionalization and environmental influences), teachers’ covariates (gender and teaching experience) and school covariates (school size, percentage of the population with post secondary education and urbanicity) were entered as fixed effects. Our results report reqression coefficients, 95 % confidence intervals and odds ratios which are estimated by exponentiating the estimated coefficient. We first examined univariate associations and then proceeded with a multivariable analysis where all variables were entered at once regardless of whether they were univariately significant. We opted to include all variables given the conceptual framework we tested^{( 40 )}. We applied multiple imputation to accommodate for missing values among the independent variables, using twenty replicates and the iterative Markov chain Monte Carlo (MCMC) procedure.

Results

Qualitative exploration of contextual factors that influenced implementation

Focus group participants reported successfully implementing AS! BC HE in the first year after dissemination. At the school level four themes emerged regarding the benefits of implementing AS! BC HE: (i) enhanced community partnerships; (ii) links to more HE resources; (iii) creation of a HE culture in the school; and (iv) links to environmental initiatives within the school. At the student level teachers and administrators highlighted the positive impact on children’s attitudes and behaviours. In the words of one teacher: ‘It is getting children back to eating healthy foods, and if they see their teacher and their classmates doing it, sometimes, they’ll be more open to try new things’.

Participants identified (i) having a school ‘leader/champion’, (ii) support from the AS! BC central team and (iii) access to resources and funding as the top three implementation facilitators. Participants from some schools also indicated that community and school engagement supported their ability to successfully implement activities.

Conversely, implementation was not without challenges. Teachers and administrators cited ‘lack of time’ as the most common barrier. For instance, one teacher stated: ‘To find the time... that was the biggest obstacle’. Other challenges noted were lack of resources, leadership and momentum. Related to resource limitation one teacher’s quote highlighted how the cost of the Action Bin was problematic: ‘I think the cost of the kits, we get one per grade level (free)... so if you have three classrooms at a grade level and we only have one kit and you have to buy one and they are 200 each, I think that is a challenge, somehow the cost has to be more reasonable’.

Lack of leadership caused by high staff turnover was another common theme. If staff champions or leaders of implementation moved away from the school there was often no one to spearhead the programme, ‘...there was no one to fill his position and I didn’t know anything about it’. Finally, while the school staff found the programme exciting at first, it was difficult to sustain positive momentum going throughout the school year; as expressed by one participant: ‘it was great to implement, people were excited, but then as you go in the year, it gets forgotten because there are other things’.

When asked about sustainability participants highlighted the need to educate and connect the parents: ‘I really believe that we need to continue to focus on educating the parents because that is where the lunches are coming from’. Official integration of the AS! BC HE activities within the curriculum rather than implementation as an ‘…extra’ was another theme. Similarly a participant suggested expansion of the AS! BC HE activities and highlighted the importance of keeping the HE model relevant, fresh and responsive to teachers’ needs: ‘I think the program is phenomenal in its infancy, but what I would like to see is more education… I would not like to do the same program year after year. I would like it stepped up to the next level. Like how to read food labels…’.

Quantitative analysis of implementation

Sample characteristics

We describe participating schools, principals and teachers in Table 2. Schools had an enrolment of 294 students on average and were mostly from census metropolitan areas where 32 % of the population had post-secondary education. Most principals and teachers were female (76 % and 71 %, respectively) and Caucasian (91 % and 85 %, respectively); administrative or teaching experience varied widely.

Table 2.

Descriptive information about the British Columbia schools (n 38) and school principals (n 34) and teachers (n 164) who participated in the study (2008–2009)

	Mean, sd or %	25th–75th percentiles
School demographics (n 38)
School size represented by number of students (n 38)
Mean	294·3	168–398
sd	158·2
Range	32–807
Percentage of the population with post-secondary education (n 38)
Mean	32·1	27·1–37·1
sd	9·0
Range	6·5–46·2
Census metropolitan area and census agglomeration influenced zones (n 37)
Census metropolitan area	60·5
Tracted/non-tracted census agglomeration	13·1
Strongly to weakly influenced zones	26·3
School principal characteristics (n 34)
Sex (n 33)
Female	75·8
Male	24·2
Administrative experience (n 33)
1–5 years of experience	33·3
6–10 years of experience	18·2
11–15 years of experience	24·2
16–20 years of experience	18·2
20+ years of experience	6·1
Ethnicity (n 33)
Caucasian	90·9
Others	9·1
Teacher characteristics (n 164)
Sex (n 164)
Female	71·3
Male	28·7
Teaching experience (n 164)
1–5 years of experience	18·3
6–10 years of experience	25·0
11–15 years of experience	20·7
16–20 years of experience	16·5
20+ years of experience	19·5
Ethnicity (n 164)
Caucasian	85·4
Chinese	4·9
South-East Asian	3·0
Other	6·7

Implementation

On average 42 % of teachers were using AS! BC HE in their classroom with 39 % providing classroom HE activities weekly and 28 % providing these activities on a monthly basis. Among those who were using the model, 96 % of teachers used the Classroom Action HE books and manuals, 83 % used promotional resources, 75 % utilized instructional resources and 53 % kept weekly logs. AS! BC provided equipment to schools to facilitate HE activities; >50 % of teachers used measuring cups (54 %) and cutting boards (51 %) while others used colanders (47 %) and peelers (40 %). Interestingly, some teachers who did not formally adopt the AS! BC HE model used AS! BC instructional and promotion resources (43 %), equipment (10 %) and activities (11 %). Teachers who adopted the model were more likely to report having used HE related resources, activities and equipment (Table 3). Teachers’ classroom tracking of HE activities was not significantly associated with use of AS! BC; 39 % of those who adopted the model, compared with 36 % of those who did not, reported tracking their classroom activities.

Table 3.

Comparing the number of Healthy Eating (HE) classroom lessons activities in the past week and use of classroom HE resources, activities and equipment among users and non-users of Action Schools! BC as reported 2008–2009

Variable	Descriptor	Coefficient	95 % CI	OR	P value
No. of HE lessons in the last week	0 v. 1 or 2	1·832	0·887, 2·777	6·246	0·000
Use of HE book resources	e.g. Veggies and Fruit of BC booklet, Canada’s Good Guide, HE Calendar, etc.	3·351	1·929, 4·774	28·542	0·000
Tracked HE activities done with students		0·413	−0·309, 1·136	1·512	0·262
Displayed HE promotional resources	e.g. posters	1·317	0·495, 2·139	3·732	0·002
Displayed HE instructional resources	e.g. Rainbow of Choice or Is It Fruit poster	1·428	0·648, 2·189	4·130	0·000
Used cutting board		2·321	1·417, 3·224	10·185	0·000
Used measuring cups		2·072	1·217, 2·927	7·941	0·000
Used colander		2·810	1·726, 3·895	16·616	0·000
Used scrub brush		2·262	1·164, 3·360	9·604	0·000
Used of peeler		2·286	1·256, 3·316	9·835	0·000
Used food group placemats		0·893	0·112, 1·674	2·442	0·025
Used fruit and vegetable placemats		1·560	0·559, 2·562	4·761	0·002
Used HE learning activities	Edible Plant Parts, Serving Size	2·068	1·207, 2·928	7·907	0·000
Used HE tasting activities	Smoothies, BC Grown & Eat Your Colors	1·328	0·508, 2·15	3·772	0·002
Used snacking activities	Healthy T Days, Crunch and Sip, ReThink Your Drink	2·294	1·159, 3·429	9·924	0·000
HE field trips*		n.d.	n.d.	n.d.	n.d.

^*Due to the small number of responses (n 4) we have not displayed the data.

Use of AS! BC HE was strongly associated with the number of times teachers reported providing HE lessons or activities in the classroom in the last week (P < 0·001). Teachers who implemented the HE model were 6·25 (OR) times more likely to provide a HE lesson or activity compared with those who did not use the model. Forty-five per cent of those who adopted the HE model provided at least one HE activity per week compared with 16 % of those who did not adopt the model.

Implementation factors

Organizational climate was conducive to implementation of the innovation. Among principals surveyed, 81 % strongly agreed or agreed that HE was a top priority at their school; 86 % reported that priority of HE had increased compared with three years previous. Approximately 51 % of principals attributed this to both adopting and implementing AS! BC HE and the Provincial Guidelines for Food and Beverage Sales in the Schools. Almost 75 % of principals reported that activities addressing the school environment (policy and physical environment changes) increased; 58 % reported that family and community and 50 % that school-wide activities also increased.

Among teachers, a significantly higher percentage of those who adopted AS! BC HE (27 %) noted that the overall initiative (AS! BC PA and HE) was a top priority in their school compared with only 12 % of those who did not adopt the model (χ ²(2)=13·34, P=0·001). When asked whether HE was a priority at their schools, 89 % and 85 %, respectively, believed that this priority had increased over the last three years. More than 50 % of teachers at AS! BC HE registered schools received training to deliver the model; 73 % of them had previously adopted AS! BC PA. Teachers demonstrated moderately high self-efficacy (3·5/5) and relatively high positive perceptions of the attributes of the innovation (Table 4).

Table 4.

Descriptive information for the underlying factors hypothesized to be associated with uptake of Action Schools! BC as reported in 2008–2009

	Mean, sd or %	25th–75th percentiles
Characteristics of the schools (n 38)
Organizational climate/support (n 33)
Strongly disagree	3·0
Disagree	6·1
Neither	9·1
Agree	60·6
Strongly agree	21·2
Organizational capacity/resources (n 33)
Percentage of dedicated Physical Education specialists
Mean	3·68	3·25–4·25
sd	0·89
Range	1·75–5·00
Level of institutionalization (n 31)
Mean	6·87	5·50–8·50
sd	1·97
Range	3–10
Environmental influences (n 33)
High	81·8
Low	18·2
Characteristics of the teachers (n 164)
Self-efficacy (n 162)
Mean	3·54	3·14–4·00
sd	0·68
Range	1·00–5·00
Users of AS! BC PA (n 164)
Yes	72·6
No	20·6
Training (n 164)
Yes	50·6
No	49·4
Attributes of the innovation (n 587)
Attributes of AS! BC (n 158)
Mean	3·77	3·50–4·13
sd	0·66
Range	1·00–5·00

AS! BC PA, Action Schools! BC Physical Activity.

Univariable analyses are shown in Table 5. Two covariates were positively associated with use and appear to relate to each other. Larger schools were less likely to be users and schools in the least urbanized areas were more likely to be users than all other census conglomerations combined. Interestingly, use was more likely if school principals felt strongly that the climate of their school was not supportive of HE or conversely less likely if principals strongly agreed or agreed that their climate for HE was positive. Teachers who had high self-efficacy were more likely to be users as were those who used AS! BC PA, received HE training and had more positive perceptions regarding the attributes of the innovation.

Table 5.

Univariable results: characteristics of schools and teachers as well as attributes of the innovation and covariates associated with implementation of Action Schools! BC

	Coefficient	95 % CI	OR	P value
Covariates (school level)
School size	−0·004	−0·007,−0·001	1·00	0·016
Percentage of the population with post-secondary education	−0·009	−0·075, 0·056	0·99†	0·781
Census metropolitan area and census agglomeration influenced zones
Tracted/non-tracted agglomeration v. metropolitan area*	−0·325	−1·870, 1·219	0·72	0·680
Strongly to weakly influenced zones v. metropolitan area*	1·008	−0·248, 2·264	2·74	0·116
Covariates (teacher level)
Sex (female v. male*)	0·282	−0·519, 1·082	1·33	0·491
Teaching experience	0·183	−0·080, 0·446	1·20	0·173
Characteristics of schools
Organizational climate	−0·365	−0·844, 0·113	0·69	0·134
Organizational capacity	−0·025	−0·655, 0·605	0·98	0·939
Level of institutionalization	0·183	−0·093, 0·459	1·20‡	0·194
Environmental influences	−0·580	−1·671, 0·512	0·56	0·298
Characteristics of teachers
Self-efficacy	1·636	0·914, 2·359	5·14‡	0·000
Users of AS! BC PA (users v. non-users*)	1·960	0·960, 2·959	7·10	0·000
Training (yes v. no*)	1·845	1·034, 2·656	6·33	0·000
Attributes of the innovation
Attributes of AS! BC	1·373	0·571, 2·175	3·95‡	0·001

AS! BC PA, Action Schools! BC Physical Activity.

^*Referent group.

^†Incremental change of 10 %.

^‡Comparing 25th and 75th percentile scores.

The multivariable analysis displayed in Table 6 showed that the covariate (larger school size) identified in the univariable analysis did not remain significantly associated with implementation but that schools in neighbourhoods characterized by higher levels of post-secondary education were less likely to be users. Similarly only the multivariate analysis showed use was lower in schools where the principals strongly agreed or agreed that that the climate was supportive of HE. Higher teacher self-efficacy, having AS! BC HE training and using AS! BC PA were significant factors in the univariable analysis and remained so in the multivariate model, while positive perceptions of the innovation were no longer significant.

Table 6.

Multivariable results: characteristics of schools and teachers as well as attributes of the innovation and covariates associated with implementation of Action Schools! BC

	Coefficient	95 % CI	OR	P value
Fixed-effects parameters
Constant	−1·963	−8·929, 5·004	–	0·0581
Covariates (school level)
School size	−0·001	−0·006, 0·003	0·90	0·555
Percentage of the population with post-secondary education	−0·119	−0·230, −0·009	0·89†	0·035
Census metropolitan area and census agglomeration influenced zones
Tracted/non-tracted agglomeration v. metropolitan area*	−0·968	−3·009, 1·072	0·38	0·352
Strongly to weakly influenced zones v. metropolitan area*	−0·285	−2·974, 2·403	0·75	0·835
Covariates (teacher level)
Sex (female v. male*)	0·380	−0·642, 1·403	1·46	0·466
Teaching experience	0·013	−0·324, 0·350	1·01	0·939
Characteristics of schools
Organizational climate	−0·794	−1·408, −0·181	0·45	0·011
Organizational capacity	0·199	−0·478, 0·875	1·22	0·564
Level of institutionalization	0·114	−0·211, 0·439	1·12‡	0·492
Environmental influences	−0·959	−2·504, 0·587	0·38	0·224
Characteristics of teachers
Self-efficacy	1·255	0·412, 2·097	3·51‡	0·004
Users of AS! BC PA (users v. non-users*)	1·786	0·570, 3·002	5·96	0·004
Training (yes v. no*)	1·459	0·519, 2·400	4·30	0·002
Attributes of the innovation
Attributes of AS! BC	0·662	−0·264, 1·587	1·94‡	0·161

AS! BC PA, Action Schools! BC Physical Activity.

^*Referent group.

^†Incremental change of 10 %.

^‡Comparing 25th and 75th percentile scores.

Discussion

We were provided the unique opportunity to assess factors that influenced implementation following a provincial scale-up (equivalent of state level) of an evidence-based, comprehensive, whole-school HE model (AS! BC HE). The public health importance of evaluating the factors that influence successful implementation of effective prevention and health promotion programmes for children and youth has been very elegantly addressed in a systematic review by Durlak and DuPre^{( 21 )}, who examined 542 interventions that focused across a wide range of topics (including physical health and academic performance). In that review, higher levels of implementation led to outcomes with mean effect sizes two to three times higher compared with studies where implementation was poor. Importantly, they found that perfect or near-perfect implementation of any intervention was unrealistic. Positive results were achieved at about 60 % of desired implementation levels, with very few studies achieving more than 80 %^{( 21 )}. In our study implementation was also not perfect. In the first year after scale-up just under half of teachers in registered schools identified themselves as users of the model. However, more than 60 % of those were using the model (materials and activities) weekly or monthly. Importantly, teachers who implemented the AS! BC model were more likely to deliver HE lessons or activities in the classroom and this is an important determinant of student-level outcomes^{( 21 )}.

Barriers to implementation included lack of time (the most common barrier), lack of leadership (loss due to staff turnover) and loss of momentum. Facilitators were support from a technical unit (including training) and access to resources (materials and funding: all resources were provided free for the school through provincial funding). These operational factors have been identified consistently in the school health literature^{( 31 , 34 , 35 , 51 – 56 )}. Durlak and DuPre characterized leadership, access to resources (funding), time and loss of momentum due to competing demands in their broader implementation framework as delivery system factors^{( 21 )}.

Training is recognized as a key issue identified in the efficacy, effectiveness, institutionalization/maintenance and dissemination literature^{( 29 , 34 , 37 , 52 , 53 , 57 )}. This issue was not identified in our qualitative data most likely because a mechanism was in place for schools to access ongoing AS! BC training and we were talking to school staff who had received the training. However, participants suggested further training as a way to maintain momentum.

Our approach was embedded in diffusion of innovations, social cognitive and organizational change theories and assessed factors associated with these theories. Interestingly, training emerged as a significant teacher-level factor in our quantitative analyses. Two other teacher-level characteristics that influenced implementation, experience implementing a similar model (AS! BC PA) and self-efficacy, also theoretically relate to training. Our findings uphold the tenets of social cognitive theory and speak to the importance of investing in teachers as key delivery agents for health promotion in elementary schools. If a teacher was trained to deliver HE activities, had previous experience and training to deliver a similar health promotion activity (AS! BC PA) and/or believed s/he was able to do so, s/he was more likely to implement the innovation. These findings also reflect the importance of the training and support system that has been identified as a critical implementation factor in a broader review of implementation conducted by Durlak and DuPre^{( 21 )}.

At the school level, school climate was the only significant factor but it emerged as a weak negative influence on implementation. This finding challenged our belief that a positive school climate for HE would positively influence adoption and implementation after scale-up. While counterintuitive at first, it may be that in schools where principals noted that HE was not a priority (which is how we operationalized school climate), they were more likely to adopt AS! BC HE as it would help them to address this need at their school. Furthermore, this finding would align with Riis et al. who noted that recent changes in school policies were more likely to occur in states were levels of obesity were higher^{( 58 )}. This seems to align with our finding as HE was noted to be of high importance and this recognition probably encouraged schools where HE had not been a top priority to take action.

Based on the importance of the basic tenets of Diffusion of Innovations theory in the related literature, we anticipated that attributes of the innovation would also emerge as important. In fact, the univariable analysis supported this assumption while the multivariable did not. It may be that perceptions of the innovation were most influential at the early stage of adoption/registration. As teachers were confronted with implementation issues they may have become more influenced by factors such as training, experience and individual efficacy. Our cross-sectional study was not able to discern the specific answer to this question. However, longer-term prospective studies would shed more light on key factors related to implementation over time.

The broader school or community context influenced implementation. Teachers were more likely to implement the model if schools were in communities where residents had lower levels of education. Teachers in these communities may have perceived a greater need to deliver health-related programmes.

Although we did not initially set out to test an overarching implementation framework, our results highlight critical elements that have been identified in many^{( 21 , 59 )}. Specifically, they highlighted the importance of community, delivery system (school level) and support system factors to implementation. Future research should integrate these frameworks in addition to existing theory to enhance our comprehensive understanding of implementation after scale-up.

Our findings must be interpreted in light of some limitations. First, we assessed implementation of HE lessons and activities broadly and of the AS! BC lessons/activities specifically, using self-report measures. To mitigate the limitations of our self-report instruments, we pilot tested and evaluated their psychometric properties in the larger sample. Second, as data are cross-sectional we are unable to differentiate factors that may influence implementation over time. Third, the sample size was limited given that we focused on schools that registered in AS! BC HE during the first year of scale-up and finally the school-level response rate was low, introducing the potential of a positive bias. However, when we compared the characteristics of the schools that participated in the study with the larger population of schools in the province, school size, percentage of the population with post-secondary education and geographic locations of schools were similar.

Finally, we would be remiss not to comment on the sociopolitical context for action as it is recognized as one community-level factor that influences implementation^{( 21 )}. We introduced AS! BC into schools in BC during a time when the sociopolitical context was highly supportive. In 2006, BC won the bid to host the 2010 Winter Olympic Games. Government invested substantively in initiatives such as the scale-up of AS! BC to create the ‘healthiest jurisdication to ever host the Olympic Games’^{( 30 )} and leave a legacy of health in the Province. In addition to supporting AS! BC, the BC Ministry of Education mandated guidelines for the sale of food and beverages in the school in 2008–09. Further, there was public support for action to encourage HE based on substantive media coverage related to the ‘childhood obesity epidemic’^{( 30 )}. Taken together these represent a powerful confluence of factors that would potentially impact school climate and enhance schools’ readiness to make changes in their HE environment.

Conclusion

Our study extends the relatively meagre body of literature that has addressed implementation after scale-up of school-based health promotion programmes. Our findings reinforce that teacher training and support are important investments as they contribute to successful programme implementation. Further, resources must be provided, and school-level leadership nurtured, as they are also critical to sustained implementation after scale-up. Thus, public health and education decision makers should enact communication strategies that enhance the relevance of health promotion efforts to education stakeholders and support training and technical support for teachers and schools to ensure effective scale-up of health-related programmes. Finally, adopting an evidence-based approach to guide scale-up of effective school-based HE and PA interventions enhances the likelihood of a positive return on the investment in the health and well-being of our children.