Closing the Gap: A Mixed Methods Exploration of Barriers and Facilitators to Evidence-Based Practices for Obesity Prevention in Head Start

Abstract

Objective:

Identify positive and negative deviant cases using quantitative fidelity data from a previous implementation of a nutrition intervention (WISE) and determine barriers and facilitators to fidelity through qualitative interviews with deviant cases.

Design:

Explanatory sequential mixed methods

Setting:

Head Start (HS) agencies in two, southern US states

Intervention:

WISE introduces children to fruits and vegetables using evidence-based practices (EBPs) of role modeling, positive feeding, mascot use, and hands-on exposure.

Participants:

Quantitative fidelity data were collected in 42 HS classrooms. Recruitment for qualitative interviews prioritized those who were positive or negative deviants across fidelity components (African American, n = 21; White, n = 19; and Hispanic, n = 3).

Analysis:

A directed content analysis approach informed by the i-PARIHS framework

Phenomenon of Interest:

Barriers and facilitators to WISE EBP implementation

Results:

Qualitative analyses identified themes of culture, leadership support, and mechanisms for embedding change as key contextual factors. Key findings related to recipient characteristics were beliefs about what works, personalized strategies to use WISE, and classroom management. Primary themes for the innovation construct were time and preparation, degree of fit, and WISE advantage. Finally, findings relative to the construct of facilitation included trainer support and desire for additional training.

Conclusions and Implications:

The study of cases at the extreme ends of the fidelity spectrum can provide unique perspectives on barriers and facilitators to implementation of interventions.

Introduction

The early care and education setting (i.e., childcare, daycare) increasingly has incorporated obesity prevention and nutrition promotion practice to address the obesity problem in the U.S. However, there continues to be a research-practice gap between evidence-based and actual classroom practices. In a study of recommendations from the Academy of Nutrition and Dietetics, nearly all early childhood educators (ECEs; 93%) reported using statements that direct children to external cues of hunger (e.g., “This is yummy. You should eat some.”) rather than internal cues (e.g., “Is your belly feeling hungry?”).¹ This finding is consistent with observational research documenting that ECEs were 10 times as likely to enact the detrimental practice of pressuring children to eat as they were to use the evidence-based practice (EBP) of cuing children to internal signals of hunger and satiety.²

The Academy of Nutrition and Dietetics recommendations noted other gaps of implementation including shortfalls in offering healthy foods, role modeling healthy intake, and teaching children about nutrition.¹ Another review documented similar gaps between recommendations and practice for compliance with food service guidelines, mealtime behaviors (e.g., time provided for meals), and provision of support for guided physical activity.³ Closing the gap between recommendations and practice is critical to promoting positive child outcomes in early care settings.

Despite documented gaps, early care and education classrooms are important settings for promoting healthy weight and nutrition outcomes for children. Two recent reviews demonstrate the positive effect of obesity interventions in childcare. In Sisson and colleagues’ review, approximately half of intervention studies targeting obesity demonstrated desired outcomes; studies targeting at-risk children had the highest efficacy.⁴ Results were even more promising in studies targeting intermediary outcomes such as physical activity, dietary intake, and screen time, with most demonstrating desired effects. In a similarly-timed systematic review, Ward and colleagues ⁵ found that multi-component, multi-level interventions were most likely to achieve desired outcomes but acknowledged that these are likely the most difficult type of interventions to implement. Neither review included measures of uptake, fidelity, or sustainability of use of EBPs. Therefore, little is known about the quality of implementation of these interventions. To date, there is a paucity of research focusing on the barriers and facilitators to implementing nutrition interventions in childcare. Thus, studies documenting barriers and facilitators to initial use and long-term sustainment of multi-component interventions in childcare are warranted to improve intervention uptake and to reduce the gap between expectations for evidence-based practice and current classroom practices.

A method that may prove useful to that end is the application of positive deviance methods. Positive deviance approaches study individuals who are doing well with an EBP adoption to determine factors that may translate to supporting others to adopt the EBP. Positive deviance methods⁶ have helped address health care implementation concerns such as increasing patient safety,⁷ sustaining hand hygiene practices,⁸ and improving vaccination rates. ⁹ They have also been used to elucidate health behaviors in community settings, such as HIV risk reduction behaviors¹⁰ and long-term breastfeeding predictors.¹¹ Several studies have expanded positive deviance methods to examine negative deviance among low-performing sites or individuals to understand barriers to uptake.^12,13 Exploration of both positive and negative deviance has illustrated differences among sites (e.g., leadership style, team processes) that contribute to performance and may not have been clear by looking only at the high performers.¹²

One nutrition intervention designed for childcare is Together, We Inspire Smart Eating, (WISE), which includes 4 EBPs to increase children’s acceptance and intake of target fruits and vegetables. WISE is an obesity prevention and nutrition promotion curriculum designed for use in early care and education settings.¹⁴ WISE lessons occur on a weekly basis throughout the school year and introduce children to eight fruits and vegetables. WISE training for early educators focuses on skill development in four EBPs: (1) mascot use to provide a familiar, friendly character associated with fruits and vegetables;^15–20 (2) role modeling by educators to allow children to observe a trusted adult interacting with the food;^21–23 (3) positive feeding practices to support children’s self-regulation and autonomy;^23–25 and (4) provision of multiple, hands-on experiences with target foods (i.e., food experiences) to increase exposure and support food acceptance.^26–32 Research has documented positive impacts of WISE on child intake of target foods at home.³³ However, WISE, like other behavior change interventions,^34,35 has also demonstrated variability in fidelity to the EBPs, which may dampen the desired outcomes.

The purpose of this mixed-method study was (a) to identify positive and negative deviance cases of implementation fidelity (i.e., who does well and who does poorly at embracing EBP in the child care setting) using quantitative fidelity data from a previous implementation of (WISE) and (b) to explore barriers and facilitators to fidelity through qualitative interviews with deviant cases. In so doing, this study illustrates methods, application of theory, and potentially salient constructs that may inform other implementation efforts as well.

Methods

Study Design

The current study used an explanatory sequential mixed-methods design.³⁶ First, the research team used secondary data from a previous WISE implementation to identify early childhood educators (ECEs) who successfully implemented (i.e., meeting desired fidelity) and ECEs who did not implement as intended (i.e., not achieving minimum desired fidelity) for the four EBPs targeted in WISE. Quantitative fidelity data were collected by trained observers using a fidelity instrument developed specific to WISE EBPs.³⁷ These data were collected in the 2014 – 2015 and the 2015- 2016 schools year in two rural, southern states of the US.

For the qualitative sample, all educators from previous WISE implementations across 3 Head Start agencies were included in the potential participant pool (N= 84). Participants (N = 34) completed interviews that lasted between 23 minutes and 1 hour and 19 minutes (mean = 45 minutes) about their experience with WISE and its EBPs. All interviews were performed by 1 of 2 interviewers familiar with the i-PARIHS constructs and their application to this study. The PI has graduate and postgraduate training in qualitative and mixed methods research and trained the research assistant through (a) guided readings and discussions, and (b) pilot interviews. The interviewers recorded the interviews on two devices, an electronic recorder and a cell phone back up. These were transferred to secure computer servers. A professional company specializing in qualitative research transcription, Landmark Associates (https://www.thelai.com/) completed the transcription. Data were shared with the company using their data secure uploading feature. These activities are the first aim of a larger study described elsewhere³⁸ and were approved by the Institutional Review Board at the University of Arkansas for Medical Sciences. Participants provided verbal consent for the study per the approved protocol. Interviews began in the October of 2016 and continued through March of 2017; all interviews were conducted face-to-face.

Measures

Quantitative.

A direct fidelity assessment was developed, consistent with guidelines from Schoenwald et al.³⁹ In this study, data from this measure was used to categorize teachers and to inform sample selection for the qualitative portion of the study. Items indicating fidelity were assessed on a 1 to 4 scale. Data were collected by trained, reliable observers in 49 classrooms during each month of the school year. Observers completed a standardized training consisting of in-person session instruction on (a) the intent of each item with provision of examples, (b) distinguishing between categorizations, and (c) discrete integration into the classroom setting. Before observing classrooms independently, each observer was required to exhibit interrater reliability of 85% with 1 of 2 gold-standard observers on 2 occasions. Gold-standard observers exhibited greater than 90% interrater agreement with one another. Interrater reliability was calculated by determining the percentage of items on which observers rated within a narrow margin of error (± 1 for counts ≤ 4, ± 2 for counts > 4) relative to the gold-standard observer. This level of reliability was obtained within 3 live observations for all observers after video training. Observers (N = 6) included undergraduate students of sociology and child development, graduate-level students in nutrition and psychology, and professionals from education and public health.

Based on field observations, educators were given a fidelity score on each of the four EBPs considered key to WISE intervention. Items were measured on a 1 to 4 scale (1 =Not at All, 2 = Sometimes, 3 = Quite a Bit, 4 = Very Much). Fidelity observers were provided with anchors for the meaning of each rating specific to each item. Cases were identified as positive deviants when school year averages were 3 or better for mascot use, hands-on exposures, and role modeling. Negative deviants on these EBPs averaged a 2 or less across the school year. Positive deviants on feeding averaged above 2.5 on supportive feeding practices and unsupportive feeding practices below 1.5. These cutoffs were determined based on the distribution of scores and minimally acceptable practice. Additional detail on the process for WISE fidelity monitoring can be found elsewhere.³⁷

Qualitative.

The Promoting Action on Research Implementation in Health Service (i-PARIHS) implementation framework provided the foundation for the interview guide to identify aspects of the context and innovation that influence adoption of EBPs.⁴⁰ In i-PARIHS, successful implementation is predicted to be a function of the combination of qualities of the innovation being implemented, characteristics of the recipients of the innovation, and features of the context in which the innovation takes place. The i-PARIHS framework further recognizes that weaknesses in any of (or across) these areas will require greater facilitation (i.e., implementation support) to achieve success. Each construct from the i-PARIHS model provided a topic area for questioning in the interviews as well as a basis for probe questions as the interview progressed (See appendix). For application of i-PARIHS to this study, Recipients refers to the ECE’s who received the WISE training. Context includes factors at the center or in the community separate from WISE. Innovation refers to characteristics of the WISE intervention. Facilitation refers to aspects of the implementation plan that supported (or were thought to) support WISE uptake. The original interview guide for the pilot was developed by the PI to elicit potential barriers and facilitators to EBP implementation within each i-PARIHS construct or for each WISE EBP. The guide was then refined through pilot interviews and team feedback.

Our team conducted 5 pilot interviews with a draft of the interview guide and with participants who were similar to study targets over a two-month period. Both interviewers attended each pilot interview. To improve the skill and consistency of the interviewers, interviewers [TS and KD] observed one another and conducted a reflection and discussion after each interview. This process allowed the PI and RA to identify probes that were effective or that missed opportunities to elicit information and facilitated the RA learning skills through observation and getting direct feedback for improvement. The research team reviewed transcripts of the pilots to identify questions that worked well or needed improvement. The final interview guide reflected input from all co-authors as well as necessary refinements identified by the interviewers’ experiences with the pilots. The pilots resulted in a reduction of length to the guide, revision of questions to improve clarity, and shifts in wording to reduce eliciting educator defensiveness (i.e., “what would convince other teachers” vs. “what would convince you”).

For each interview, the interviewer tailored questions to knowledge of individual participants’ implementation successes or lack of uptake for the targeted practices. For example, an ECE with low fidelity scores for use of mascot was asked, “What makes it hard to use the Windy Wise puppet in your classroom?” An ECE with high fidelity scores was asked. “What makes it easy for you to use the Windy Wise puppet in your classroom?” The team reviewed transcripts and interviewer notes throughout data collection.

Analysis

Interviews were transcribed verbatim, and then imported into QSR NVivo 10 for analysis.⁴¹ Consistent with Qualitative Research Methods In Implementation Science approaches which tend to be more pragmatic and deductive,⁴² the analysis process reflected a directed content analysis approach ⁴³ that focused on identifying salient themes in participant interviews within i-PARIHS constructs and for each EBP of WISE. The i-PARIHS framework for implementation was applied to develop a template of codes to sort relevant text into 4 major constructs (inner/outer context, recipients, facilitation, and characteristics of the innovation).⁴⁰ Transcripts were further coded into EBPs emphasized in the WISE program (positive feeding practices, mascot use, role modeling, and hands-on exposure). Each segment was also coded as a barrier or facilitator. See Figure 1. Statements that indicated more than 1 category were coded in both categories. Additionally, analysis identified overarching themes within the context construct that affected educators’ and centers’ abilities to implement WISE successfully.

Figure 1.

Analysis coding scheme directed by the integrated Promoting Action on Research in Implementation in Health Services framework.

The 2 analysts were the Principal Investigator (PI, TS) who had graduate course work and experience in qualitative methods and a Research Assistant (RA, TT) trained by the PI. After establishing a codebook based on i-PARIHS, the PI [TS] first coded 1 interview in tandem with the RA [TT]. Next, both the PI and RA independently coded each interview. Then, the PI and RA came together after coding each interview to engage in a consensus building process by talking through ambiguous instances and comparing them with the existing conceptual definitions in the codebook. Interviews were compared and discussed to evaluate inter-coder reliability, resolve coding discrepancies, and strengthen thematic understanding between the coders. During this process, the codebook was continuously refined and expanded to include emerging subthemes within each main construct. The PI wrote initial definitions for each code, and refinements and additions were made to the code definitions through consensus building process between the two coders. Reliability was measured using the kappa statistic generated by QSR Nvivo 10.^41,44On the 5^th interview in this process, the coders demonstrated reliability (kappa > .70⁴⁵) and ceased further revision to the codebook. The team held bi-weekly meetings to facilitate shared meaning between coders and to ensure coding validity and reliability.

Results

Quantitative.

Participants.

Quantitative observations were conducted in 8 centers from 3 Head Start agencies (N = 70 teachers). The race/ethnicity of participants was largely African American (64%), followed by White (28%), and Hispanic (5%). Educators in this sample were mostly over age 41 (56%); 22% were between 35 and 40, and 22% were between 19 and 34. Most educators had an Associate’s (35%) or Bachelor’s Degree (35%); fewer had a high school diploma or General Education Certificate (7%), some college 16%, or a Master’s Degree (7%).

Fidelity.

Quantitative data identified 3 cases from different centers who were positive deviants on 3 of the 4 EBPs; 2 were recruited and participated in qualitative interviews. There were 42 cases that were negatively deviant on 1 or more practices and exhibited no positive deviance; 13 were recruited and participated in qualitative interviews. The remaining cases were positively deviant for some EBPs and negatively deviant for others (n = 13) or directors without fidelity data (n = 6).

Qualitative.

Participants.

The qualitative sampling strategy prioritized educators who were positive deviants and negative deviants on EBPs. Those with more variation in fidelity by EBP were recruited secondarily. Of those targeted based on deviance, 2 declined; 2 were deceased; and 12 were no longer employed with the agencies. Table 1 provides a summary of the number of positive and negative deviant cases for each WISE EBP for those observed in the quantitative study and those who agreed to participate in the qualitative interviews. Interviewers also interviewed directors from Head Start centers in the educator participant pool to provide information on the context in which teachers were attempting to adopt WISE. Participants were from 7 Head Start locations with centers serving rural geographic areas (n = 6), an urban area (n =3), and a suburban area (n = 1). The race/ethnicity of participants was largely African American (53%), followed by White (38%), and Hispanic (9%). The role of participants was split across lead teachers (42%), assistant teachers (42%), and directors (18%).

Table 1.

Number of Positive and Negative Deviants for Each WISE Practice

	ROLE MODEL		MASCOT		HANDS-ON		FEEDING
	Positive	Negative	Positive	Negative	Positive	Negative	Positive	Negative
Observed	18	6	12	9	7	14	5	61
Recruited	10	1	8	2	4	5	2	26

^*Participants could be a deviant case on some practices but not others. Cases that were not deviant were in the mid-range of fidelity, not meeting criteria for either positive or negative deviance. Observations for feeding were collected for both assistant and lead teachers resulting in greater total numbers whereas the other practices were observed at the classroom level where teachers often delegate lesson responsibilities.

Overview.

Based on the i-PARIHS framework, the qualitative analyses grouped results into 4 conceptual categories of 1) context, 2) recipients, 3) innovation, and 4) facilitation. Table 2 provides key quotes as examples. The influence of context was seen as a significant contributor to the uptake of WISE EBPs with influences from center culture factors, leadership support, mechanisms for integration of changes, and the consistency of the setting with the new evidence. Across WISE EBPs, key characteristics of the educators (recipients) acted as either barriers or facilitators to uptake of the EBPs. These were beliefs about “what works,” personalized strategies to use WISE, and classroom management. Primary themes for the innovation construct were time and preparation, degree of fit, and perception of WISE advantage. There was some overlap in the first 2 of these depending on the perception that WISE could be integrated into the learning day or required separate instructional time. Similarly, the degree that ECEs embraced the use of WISE materials influenced their perception. Findings on facilitation primarily reflected the initial daylong training, which was appreciated by most participants. However, educators reported the lack of additional follow-up training as a barrier.

Table 2.

Examples of Barriers and Facilitators by i-PARIHS Framework Construct

	Direction	ECE Quote (Participant ID)
1. Context
1.1 Center culture	+	“We’re always interested in having new things. What we wanna’ do is make the children aware of the world around them. Any experience to help them, I welcome it. (2813000)”
1.2 Feadership support	+	“She [the director] was always willing to help me get the food experience in…If I needed something, she was always there to help me with it. She was just always supporting” (413022).
	−	“Sometimes management have to realize that they were once in my shoes, so you don’t wanna be too pushy. Relate to us a little bit more and know that, “Okay, I was once doin’ tliis.” Cuz all of ‘em that’s holdin’ a position here, you was once at the bottom, so let’s not forget that. I mean it’s gonna get done, but just kinda watch how you—we’re all adults and sometimes it’s not like that. (28700)
1.3 Mechanisms for embedding change	+/−	It was important for educators to “have somebody on-call whenever we needed anything” (80).
1.4 Context’s stability	−	“…we’ve been short-staffed…We’ve had a sub, and she’s not familiar with the WISE. It’s been hard” (80).
1.5 Educator concerns about children’s context	−	“You always live in fear of this could possibly be the only balanced or full meal this child might get that day” (28600).
1.6 Meal location/policy	+	“We do a conversation. I want them to talk to me about the foods. Even at mealtime, we say, “This is family style. This is the way we sit. This is a family. We are family.” We sit and we talk about the foods that we’re eating.” (281800)
2. Recipients
2.1 Beliefs about what works	+	“You need to use that puppet because that really gets the kids excited.” (413222).
a. Mascot
	−	“Well, I’m a quiet person. I am very quiet. I know I have to be outgoing, but overall, I am quiet, kind of standoffish, but I learned—in the beginning, I did not want to do—I’m just that way.” (413231)
b. Hands-on	+	“The kids love it, because they get to help with the recipes, and I think that’s what they like most. I have seen that with the children that are picky eaters, that if they help out with the recipes and get to make it themselves…they’re more willing to try it than if they don’t.” (413202)
	−	“I just didn’t think small group is good for three-year-olds. I just don’t think it cuz their attention spans are just—I just prefer large group.” (28700)
c. Role modeling	+	“‘Cause I think that they trust me and know that I wouldn’t do anything that would harm them. And if they see me doing it, I think they think that it’s OK for them too” (413031).
d. Feeding	−	“I don’t think it’s really appropriate to ask me if they’re hungry, cuz they have breakfast. I don’t think that’s appropriate to ask them if they’re hungry.” (281100)
2.2 Personalized strategies to use WISE	+	“We’ve got a basket where Windy delivers our stuff, and Windy sits on the basket and she guards it. We accessorized ours. We made ours a girl, and I made her little bows, and we change out her bows. Windy’ll come when we’re reading the letter and deliver the letter. Then, when we’re talking about—when we get the pictures out and we’re talking about the growth cycle and the different kinds and everything, Windy’ll come out and talk to the kids.” (70)
2.3 Classroom management	−	[It was hard] “trying to fit it in, when to do it, how to do it, what else to do with the students while we were doing this with this group …. If you do three what are the others gonna be doing?” (ID 281200)
3. Innovation
3.1 Time and preparation theme	−	“took a lot of planning and us working together to figure out what was the best way.” (414)
3.2 Degree of fit
a. Integration into teaching time	+	“You can always tie in counting, your literacy, your vocabulary. I’m like, so you still doing your teaching. Your teaching is not stopping. Teaching is still going all throughout the WISE activity” (281000).
b. Connecting with parents	+	“The children will go back home and they will tell their mother or father, whoever they live with, that they actually tried tomatoes. I was really amazed.” (282500)
c. Personal health	+	“I’ve never used a blender, so I’ve learned how to use a blender. Different things that I’ve learned. Being a single mom, you don’t have the perks of buying a blender or having this or having that, so I’ve learned a lot. (413032)
3.3 WISE advantage
a. Resources	+	“And the manual was very helpful, because you could just read, and it would tell you step-by-step what to say. Once you got Windy and got your props and everything, the manual was really helpful in those words that you use to capture the kids’ attention, so I used the manual a whole lot, and the talking pictures, and everything.” (413011)
b. Appliances	−	“Because then it’s too time consuming. You’re blending stuff and then we discuss it, then they taste, then you blend again, we discuss, they taste” (28600)
4. Facilitation
4.1 Interactive Training	+	“When we go to trainings with other centers, some of them come up with really out-there ideas. I’m like, ‘Oh, I never thought about that for the class!’ It’s good when we all get together, cuz some people just think so outside the box. You’re like, “Oh, I didn’t even think about that. That’s great” (413202)
4.2 Desire Additional Training	−	“I guess just more training, more…knowledge is the key to everything so, just train, train, train. When a new educational idea comes up, that’s all you hear about for years and years and years.” (57)

Context.

As shown in Table 2, three major context themes of center culture, leadership support, and mechanisms for embedding change were observed and consistent with the i-PARIHS conception of the inner context, largely at the local level. Elements such as teamwork, enthusiasm for change, and a passion for children facilitated a positive WISE intervention experience. Alternatively, educators noted competition with other ‘new’ activities dampened original excitement for implementation. Further, failure to include kitchen staff in the implementation planning led to conflicts and decreased implementation. Thus, culture acted as a barrier for some and a facilitator for others.

Leadership support influenced WISE adoption. Educators described the importance of the directors’ initial enthusiasm and approach for introducing WISE to staff as well as her role in supporting the ongoing use of the curriculum throughout the school year. In some cases, directors facilitated the organizational structures needed to support the implementation of WISE (e.g., organize a meeting to determine roles). In other cases, the director took on additional responsibilities to ensure effective implementation (e.g., organizing supplies). The common facilitating element was the director’s attitude toward the program. Educators in centers without leadership support struggled to identify roles and processes for implementation.

Mechanisms for embedding change, both broad and specific to WISE, were critical contextual factors for understanding successful WISE implementation. WISE-specific mechanisms included identifying time in the schedule, planning within available resources (e.g., budget, appliances at center), and designating roles (e.g., recipe selection). WISE-specific mechanisms facilitated success when they were in place before the implementation effort began and were maintained and refined during the school year. At several sites, one “main person” served in this role; at other sites, the “designated person” rotated throughout the school year.

One key mechanism for change was the ability of ECEs to plan with their colleagues and classroom co-teachers. This “planning time” supported not only WISE, but also encouraged an environment for learning from one another and addressing areas that required change. Group teacher planning sessions varied in frequency, ranging from an hour a day to once a week. The absence of mechanisms for embedding change, similar to lack of leadership support, led to confusion, frustration, and failed efforts to integrate WISE. Examples included supplies coming at the wrong time or not at all and a lack of coordination over sharing and storing supplies.

The context’s stability and system constraints influenced adoption of WISE EBPs. This theme mirrored the structure and systems characteristic of the inner context at the organizational level in i-PARIHS. Educators described staff shortages and turnover as barriers for implementing WISE lessons; substitute educators were not familiar with WISE, and experienced teachers were shifted frequently from their primary classroom.

Characteristics of the mealtime context such as scheduling acted as either a barrier or facilitator for positive feeding practices, especially at meals. An assistant teacher explained,

“We have a time limit on how long we can eat…. Talk a little bit, but we need to eat. Because we don’t have a lot of time, remember. We can’t have the dishes all day. She [kitchen staff] needs our plates… I’m like okay, ‘Remember last time [I] had to dump your [child’s] plate because it was time to dump, and we were sad because we didn’t have time to eat it.’” - ID 413202

Context facilitators of positive feeding and role modeling included meals served in a classroom (versus a cafeteria) and providing ECEs with the same meals that the students receive examples of structures and systems of the local organization.

Beyond the limited time in the preschool schedule, ECE’s were influenced by their concerns about children’s contexts. Commonly, educators worried that children would soon be going to public schools where children “got 15 minutes, 20 at the max, and out you go” (ID 413042). This concern lead them to pressure children and encourage hurried meals. In addition, educators described a lack of systematic approach in their context to addressing food insecurity. This gap led to educators feeling a personal responsibility to address food insecurity, worrying about children “getting enough at home” (ID 413031), and altering feeding from best practice. In this circumstance, educators placed extra emphasis on food intake to the exclusion of manipulating or exploring foods (e.g.,” I want them to know that you need food to survive” –ID 413032). Educators also shared their own food and encouraged children to eat extra in this situation. In summary, the context of implementing centers provided noteworthy factors to consider for understanding implementation success or failure.

Recipients.

The first recipient-level theme, beliefs about “what works”, was a primary driving factor, consistent with the i-PARIHS recognition of the impact of values and beliefs of implementers. That is, the WISE EBPs may or may not have aligned with what educators considered to be effective. For example, several educators felt that the Windy Wise mascot was a vital element to the success of the WISE curriculum, helping kids to “taste the food and see” (ID 281300). Many educators credited Windy Wise when children were successful in trying new foods and when they enjoyed WISE activities.

Educators also believed that providing hands-on exposure to fruits and vegetables worked well. Educators who succeeded in this practice described the learning opportunities created for children and the impact of these experiences on children. Specifically, when WISE lessons were conducted in small groups to achieve the goal of hands-on exposure, educators said that children had greater opportunity to participate; peer-to-peer influences were more likely; and educator-child communications were optimized (e.g., “one-on-one”). Ultimately, these ECEs agreed that hands-on exposure and active child participation led to students trying the target food.

These beliefs about “what works” were also important for role modeling. Educators described being aware that children need to “see someone else” (ID 58) to be ready to try something new. Further, they believed that failing to try the food as the adult means the children are “not even gonna make the effort to try it” (ID 413011). ECEs described being more likely to engage in the practice of positive feeding not only because they believed it worked, but because they believed alternative practices did not work. Several educators described the risks of rushing children because it “could result in them choking themselves” or could “make them sick” (ID 281800). One educator summarized how negative practices can backfire:

“I think once you have asked them a couple of times to taste it, and they’ve had that opportunity, then it’s up to that child… I think if you keep on and on, they’re gonna just push their plate away.” -ID 413042

One barrier to use of EBPs was demonstrated when educators’ beliefs about “what works” did not align with WISE EBPs. For example, some educators did not believe that using a mascot was necessary; others felt that WISE lessons didn’t need to be in small groups to achieve hands-on exposure; and still others felt that children couldn’t be trusted to know how much to eat at meals, which undermined positive feeding practices. The ECEs conducted WISE lessons in large groups with their whole classrooms or combined with other classrooms (i.e., limiting hands-on exposure counter to best practice and standards^46,47). Others felt that small groups were for classrooms that could not manage large groups well, and some ECEs were proud to do it in large groups, “because my children are obedient” (ID 28600). Similarly, educators enacting negative feeding practices described doubting that children could be trusted to decide how much to eat for reasons such as “they might wanna go play” (ID 70) and “they are so young” (ID 281900). These beliefs about “what works” remained even after the training and a year of using the curriculum.

A second recipient-level theme was the degree to which educators personalized (but did not compromise) use of the WISE EBPs. This theme reflects 2 characteristics of recipients that are highlighted by I-PAIRHS: 1) power and authority and 2) skills and knowledge of the recipients. That is, some teachers felt empowered to make WISE their own and had ideas and skills to do so. For use of the mascot, educator personal strategies included using Windy Wise to signal to children that it was time for WISE, deliver letters from the farmer, encourage children at mealtimes, and suggest that children talk with parents about target foods. Many of these examples were included in training (e.g., letter delivery, “Whooo tried it?” chant); educators improvised to add others as well (e.g., daily wrap-up, lunchtime visits). Educators also described making role modeling concrete with specific behaviors they intentionally repeated, particularly with less-popular foods (e.g., positive statements about food facts). While training emphasized eating the food with children and making positive comments, educators described other tactics, including connecting it to foods the children had eaten before, noting differences in opinion, and sharing experiences about how their own preferences changed over time.

The final recipient-level theme was classroom management, also consistent with i-PARIHS’ recognition of recipient skills and knowledge. Educators with classroom management skills were more successful in implementing WISE EBPs. These ECEs assimilated WISE activities into their schedules and lesson plans with ease. For example, teachers who successfully implemented hands-on exposure in small groups included a plan for children waiting to rotate through the small group (e.g., “drawing a sweet potato” (ID 281800)). Conversely, educators who struggled to adopt WISE often described barriers to managing their classroom schedules and children’s behaviors. Although small groups are a considered best practice in early childhood⁴⁷ and are consistent with programmatic guidelines,⁴⁶ several educators were perplexed about using this strategy in their classrooms. ECE comments highlighted that WISE needed a foundation of fundamental classroom management skills.

Innovation.

The time and preparation theme constituted a barrier for approximately half of the participants, which is relevant to the usability characteristics of innovation recognized by i-PARIHS. Educators described the overall business of their day and the “scary” nature of adding “something else into the schedule” (ID 413222), which was “already hectic” (ID 413031). This view was most common among educators who felt that WISE was separate from teaching time, describing it as “taking away” from “those teaching moments” (ID 413011).

Other educators did not view WISE as “something else” but as a facilitator to meet existing learning standards with greater ease. This finding was consistent with the major theme of degree of fit as well. Degree of fit is conceived of as compatibility of the innovation with existing values and practices in the i-PARIHS framework. At least 17 educators noted that the lessons saved them time and planning by providing all that was needed to meet their established objectives (e.g., Teaching Strategies Gold). These educators reported using WISE to build math skills (e.g., measurements, counting, ratings), literacy skills (e.g., vocabulary words, describing words), and science concepts (e.g., “sequence and the cycle of the seed” (ID 13011)).

There were three other primary sub-themes within degree of fit that acted as facilitators for the WISE program: connecting with parents, supporting children’s health, and improving their own health habits. Direct contact with parents included sharing recipes, showing pictures of children trying new foods, and telling parents when a child liked a WISE food experience. Reports from parents about the impact of WISE on making or trying new foods in the home motivated educators as well. Commonly, educators described a general sense that “you know it’s helping” (ID 413042) because children are “introduced to things that they don’t get at home” (ID 13082). Sometimes “over half the class” (ID 281300) demonstrated significant shifts in “being excited to try new foods” (ID 57) as illustrated by one lead ECE who said,

“One little girl that sat here, she was a very picky eater. By the end of the year, she would take a bite… [at first] she would play with it. She would touch it, but she never would put it in her mouth, and then as we progressed, she would smell it, and then she would lick it. And then eventually she would bite it. ” - ID 88

WISE also provided a high degree of fit for educators by supporting their personal health. Educators said that WISE provided exposure to new foods and new skills for food preparation. This exposure often led to instances where educators described “doing them at home…introducing them to my own kids” (ID 281000).

The final theme for innovation was WISE advantage, consistent with the characteristics of relative advantage in i-PARIHS and other implementation theories. Educators described specific aspects of the WISE innovation that they perceived as providing an advantage to current practice (or not). The WISE EBP facilitators included the Windy Wise mascot and the resources provided (e.g., letters, manual). Despite low fidelity in puppet use across many classrooms, educators commonly described that the mascot supported them in transitioning to WISE activities in the classroom and getting children excited about trying new foods, “just by her presence” (ID 282500). In terms of resources, the letters from farmers provided a connection to parents at home and provided a common ground between ECEs and with kitchen staff.

However, one primary WISE characteristic acted as a barrier. Although WISE can be conducted without appliances, several educators described frustration in their attempts to use microwaves, blenders, and crockpots for WISE recipes. Most commonly, this related to concerns of time but also included the concerns for child safety.

Facilitation.

The educators in this sample participated in an implementation training that included a 6-hour interactive training session and received newsletters that reviewed key concepts from WISE training throughout the school year. Interviews queried educators about their experience with this level of facilitation. Overall, participants found the training valuable, especially its interactive nature, which provided “time to practice” (ID 413912), getting “more ideas” (ID 401) for using WISE, and an opportunity to “see how it felt like when the children was doing it” (ID 13011). Participants also appreciated receiving and interacting with the curriculum materials and connecting with other teachers at the training.

The primary training barrier was the desire for additional training opportunities. Educators offered a variety of ideas about the modalities and frequencies of additional training. Suggestions included refresher trainings up to “three times per year” (ID 13000) and demonstrations in their classrooms to “see how it’s done” (ID 401). Brief, but more frequent (e.g.,” an hour long,” “short review”), interactions were suggested.

Discussion

This mixed methods study used existing quantitative data to inform purposive sampling of both positively and negatively deviant individuals for qualitative interviews to identify barriers and facilitators to successful implementation of the WISE intervention. Inclusion of both extremes of the fidelity spectrum provided unique insight into barriers and facilitators of the use of EBPs of the WISE intervention. Further, the application of the i-PARIHS framework illustrated how factors influencing the use of evidence are both similar and unique to settings where the framework previously has been applied (e.g., clinics).

Findings within the construct of Context highlighted the importance of culture, leadership, and existing mechanisms to embed change. Previous studies have shown that positive leadership is associated with greater organizational commitment,⁴⁸ improved reception of new EBPs among staff implementing change,⁴⁸ and improved sustainability.⁴⁹ As in studies in health care settings,⁵⁰ the culture for learning and leadership support were influential for WISE adoption. Educator comments in this study reflected how leadership spilled over (positively or negatively) to affect staff implementation.

In this study, staff turnover was another notable contextual barrier to the ability to implement and sustain WISE. Turnover is a barrier to evidence implementation in many settings.^51–53 However, school and early care and education settings may be particularly influenced by turnover.^54–56 Annual turnover rates in Head Start programs, such as the one for this study, are estimated at 21%.⁵⁷ Experience of any turnover in a given year is documented above 50% for many program types (e.g., public school sponsored, private childcare).⁵⁷ A study by Lee and colleagues ⁵⁸ evaluating an after-school intervention in YMCAs found that treatment sites with any turnover (60%) were no different than control sites in achieving physical activity goals; that is, it was as if the intervention sites with turnover had not received the intervention. Together, these studies suggest that workforce development⁵⁹ and workplace-level supports ⁴⁹ may be important in developing an implementation strategy, regardless of the EBPs that are to be implemented. Indeed, reviews of predictors of sustainability of interventions in schools and disadvantaged communities have identified workforce stability as key.^49,60

Consistent with previous work on general determinants of mealtime practices,^61,62 the context for feeding (e.g., scheduling, mealtime setting) held influence over educators’ ability to role model and use positive feeding practices with WISE. To date, most studies designed to understand mealtime practices have focused on individual-level characteristics (e.g., knowledge, personal health habits).^63–68 Current findings on the influence of Context highlight the need to measure and attend to contextual factors ⁶⁹ in interventions in the early care and education setting.

Individual recipient factors were also influential. Specifically, beliefs and skills were drivers of EBP uptake, similar to the importance of attitudes and skills identified for sustainability of interventions.⁶⁰ Recipients’ beliefs about “what works” were a major driver of their adoption (or failed adoption) of WISE EBPs. This reflects a review of prior studies, which found that key factors for the implementer were perceiving the need for the practice and seeing its benefits.⁷⁰ Provision of counter evidence and peer testimonials could be tested as novel implementation strategies to shift beliefs that were understood to be in the service of children but were actually inconsistent with EBPs.

In addition to beliefs, recipients’, classroom skills influenced the integration of WISE EBPs in their classroom. Skilled teachers had the “requisite skills”⁷⁰ to use WISE as intended (e.g., offer lessons in small groups) and felt comfortable making appropriate individual-level adaptations to personalize WISE implementation in their classroom. Most changes educators used for personalizing WISE were extensions and additions as described by Wiltsey-Stirman and colleagues.⁷¹ Such adaptations have been suggested to improve sustainability and effectiveness in previous work.^60,71 At least one other study has documented the value of a skilled, experienced workforce for supporting integration of evidence into new settings.⁷² Given the rates of turnover in early care and education settings and that turnover was cited as a barrier to implementation, this further supports the importance of overall workforce development for promoting evidence uptake in early care and education setting. Workforce development may reduce turnover and lead to improvements in fidelity and implementation of a range of evidence-based programs.

Educators also noted barriers and facilitators specific to WISE itself (i.e., the innovation). Time to implement was an important barrier, but only for educators who felt WISE was an add-on to their instructional time. This finding is consistent with research related to the sustainability of mental health programs in schools. ⁷³ Framing the EBPs to assist educators with meeting their needs (i.e., guidelines, curricula concepts) may be important for promoting uptake of nutrition interventions in educational settings.

Educators in this study valued the initial level of facilitation received (i.e., 6 hour training) but also desired additional opportunities for training and support. Research documenting both quality and quantity of training reveals that training is an important predictor of program success.^49,73 The addition of trainings throughout the year and onsite trainings with demonstrations were described as desirable. However, educators did not mention further supports beyond additional education, which may reflect lack of familiarity with other approaches to supporting implementation.

This study had both limitations and strengths. First, this study took place in two southern states among Head Start educators only. Implementation of WISE into other geographic areas, different cultural contexts,⁷⁴ and additional settings (e.g., family childcare homes) will likely bring forward additional distinct barriers and facilitators for these settings. Second, there were few positively deviant cases across all EBPs; i.e., there were more participants that did not adopt the EBPs. The lack of positive deviant cases in the sample could have biased findings towards negative impacts on adoption and fidelity. This was particularly true for positive feeding practices, even after a lower cut point was adopted. Therefore, participants were considered for their deviance on each EBP. Some teachers were positively deviant in one or two areas while being average or negatively deviant in other areas. This finding is realistic for real-world implementation – that teachers will be quicker to excel (i.e., demonstrate fidelity) in some areas than others. Inclusion of these teachers allowed us to capture wider variation in implementation perspectives across the EBPs. Acknowledging areas of strengths and areas for improvement when supporting implementation with teachers (rather than just focusing on weaknesses) may provide motivation to reach ideal standards across EBPs. A strength of this study was the strong quantitative foundation that provided a robust assessment of fidelity (across entire school year) to inform the sampling approach for qualitative interviews. An additional strength was the focus on understanding stakeholder perspectives on WISE implementation. Although WISE development included stakeholder input from the beginning, improving real-life implementation requires ongoing community input, consistent with a Community-Engaged Dissemination/Implementation approach.⁷⁵

Implications for Research and Practice

A recent systematic review of application of positive deviance methods critiqued studies for having poorly defined methods and being resource intensive.⁶ In contrast, this study leveraged existing observation-based fidelity data to create an efficient plan defining deviance and sampling cases, areas identified as a needing concrete examples in deviance methods.⁷ As in some previous applications of positive deviance methods,^9,13 this study considered both ends of the fidelity spectrum; both those performing well and those performing poorly. Further, deviance in this study was specific to the EBP. These aspects of this method increased the information about barriers and facilitators. The current study demonstrates that looking at positive deviance alone may limit the ability to gather useful information to improve future implementation efforts.

The current study suggests new ideas for implementation strategies. For example, documenting and circulating intervention benefits for the local context could increase implemented’ perceptions that the implementation is fitting a need, an important facilitator in this study. Consistent with research regarding the positive influence of providing program effectiveness data for school-based mental health programs,⁷³ this type of strategy may increase and maintain motivation for use of EBPs. Experiencing success can contribute to increased sustainability, in turn.⁷³ Further, structured opportunities for peer-to-peer engagement and education may be promising implementation approaches that leverage the facilitators identified in this study.

In conclusion, the study of deviant cases provided a robust approach to understand barriers and facilitators to the WISE intervention in childcare. The research team is currently working with educator stakeholders to design multifaceted strategies to support WISE implementation, which addresses the range of barriers and facilitators identified in this work. Similar examination of implementation of other interventions could promote identification of common barriers and facilitators to nutrition promotion and obesity prevention in the childcare setting. Results of such studies have the potential to inform future intervention development, development and modification of implementation strategies to support existing interventions, and appropriate adaptations to interventions in the childcare setting. In turn, improved implementation of this and other similar interventions may contribute to the achievement of larger effect sizes (i.e., greater desired impact) and higher sustainability.