Pebbles, Rocks, and Boulders: The Implementation of a School-Based Social Engagement Intervention for Children with Autism

Abstract

Few evidence-based practices (EBPs), defined as the use of empirically supported research and clinical expertise for children with autism, have been successfully implemented and sustained in schools. This study examined the perspectives of school personnel (n = 39) on implementing a social engagement intervention for children with autism. Semi-structured interviews, informed by the Domitrovich and colleagues’ (2008) framework, were conducted. Participants were asked about: 1) school factors that affect the general implementation of EBPs; 2) their specific experiences implementing the social engagement intervention; and 3) barriers to and facilitators of implementing the social engagement intervention. Data were analyzed using an integrated approach. General (e.g., implementation process, leadership, support, staff) and intervention-specific (e.g., staff, barriers, facilitators) implementation themes were identified. These findings suggest that a variety of factors should be considered when implementing EBPs in schools and that implementing social engagement interventions for children with autism may require additional specific support for implementation.

“I believe people fall into three categories: pebbles, rocks, and boulders. The pebbles [are] the “yes” people. They will walk over broken glass or fire to do whatever it is they have to do to get the job or keep the job because they ultimately love the job. The rocks are the people who will sit back and take pause, ask intelligent questions, poke holes in a theory, and question the new curriculum. And the boulders are the people who are not [going to] move.”

--- Principal

The number of children with autism served in public schools has increased dramatically within the last decade (Blumberg et al., 2013), making the improvement of school-based autism services a national priority in the United States (US; Lester & Kelman, 1997; Locke, Kratz, Reisinger, & Mandell, 2014). In the US, the Individuals with Disabilities Education Act (IDEA, 2004) and the No Child Left Behind Act of 2001 (NCLB, 2002) require that school systems implement “scientifically based” (i.e., evidence-based) educational and behavioral practices. Evidence-based practices (EBPs) are increasingly required in schools by policy-makers (Fixsen, Blasé, Metz, & Van Dyke, 2013). Though the definition of EBPs varies widely across fields, Cook and Odom (2013) define EBPs as “practices and programs shown by high-quality research to have meaningful effects on student outcomes” (p.136). Few autism EBPs have been successfully adopted, implemented, and sustained in school settings (Stark, Arora, & Funk, 2011; Dingfelder & Mandell, 2011; Locke et al., 2014), likely because few EBPs for children with autism have actually been developed for and tested within school settings (Stahmer, Suhrheinrich, Reed, Bolduc, & Schreibman, 2011). Conducting research in partnership with the settings in which EBPs are intended to be used (e.g., schools) increases the potential that the results will be more relevant and context specific, improve the likelihood of sustainment, and result in positive outcomes for children (Weisz, Chu, & Polo, 2004; Green, 2008; Green, Glasgow, Atkins, & Stange, 2009; Drahota et al., 2016).

Barriers to and facilitators of EBP implementation in educational, medical, and community mental health settings have been widely documented. Findings from these studies suggest that a variety of individual- (e.g., skill, attitudes, beliefs) as well as organizational-level (e.g., implementation context, organizational policies and procedures) factors may affect successful EBP implementation (Forman et al., 2013; Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005; Majid et al., 2011; Stahmer & Aarons, 2009). Numerous frameworks have emerged in the field of implementation science to conceptualize and understand the factors related to implementation outcomes within these settings (e.g., Damschroder et al., 2009; Domitrovich et al., 2008; Fixsen et al., 2005; Fixsen, et al., 2013; Proctor et al., 2011; Odom, Cox, & Brock, 2013).

Understanding the perceived barriers to and facilitators of EBP implementation specific to schools is an important first step in developing unique implementation strategies for school-based practitioners (Kasari & Smith, 2013). In their framework for understanding EBP implementation in schools, Domitrovich and colleagues (2008) posit that the implementation of EBPs in schools is influenced by a broad array of interdependent factors at the macro, school, and individual levels. The macro-level includes community factors such as policies and practices at the government and school board levels that may influence the quality of implementation within schools. The school-level represents the school as an organizational entity that influences intervention implementation and includes factors such as schools’ organizational functioning, policies within the building, resources available to support implementation, and climate. The individual-level includes factors associated with the implementer (in our case, administrators and other school personnel) that affect the quality of intervention implementation such as professional and psychological characteristics and attitudes towards the intervention (Domitrovich et al., 2008). Little is known about the factors that influence the use of autism EBPs in schools. Understanding the factors that influence schools’ adoption and continued use of new interventions, specifically EBPs for autism, can lead to more effective strategies to support the sustained use of these interventions and, more generally, advance the field of implementation science (Odom, Duda, Kucharczyk, Cox, & Stabel, 2014). Implementation factors may ultimately inform autism intervention fidelity within school settings, which historically has varied in studies where non-research personnel (e.g., teachers, paraprofessionals) served as interventionists (Pellecchia et al., 2015; Locke et al., 2015; Mandell et al., 2013; Stahmer et al., 2015; Suhrheinrich et al., 2013).

Research indicates that limited resources, small staffing ratios, and a lack of time and training may present barriers to successful implementation of autism EBPs in school settings (Kucharczyk et al., 2015; Mandell et al., 2013). Perhaps in part due to these obstacles, teachers report varying perceptions about their ability to effectively implement autism EBPs. In a recent survey, early childhood, primary, and secondary education teachers reported that they did not feel confident implementing EBPs for students with autism in their classroom (Brock et al., 2014). Stahmer and colleagues (2005) found that early intervention providers often report modifying elements of EBPs for children with autism in their classroom, which may ultimately lead to titrated intervention outcomes (Weisz, Weiss, Han, Granger, & Al, 1995).

As social impairments greatly affect the academic functioning of children with autism, social engagement interventions may have the most meaningful results when implemented within school settings (Locke et al., 2015). However, social engagement interventions may present a unique set of barriers in schools, in that their implementation often involve shifting the ways in which policies are implemented and programs are delivered in an organization. In particular, some social engagement EBPs for children with autism are delivered during recess in schools. However, schools often have unique policies that interfere with intervention implementation (e.g., cancellation, detention, lack of recess period), staff allocation (e.g., prioritization of competing staff demands), as well as the availability and accessibility of resources (e.g., playground materials) during the recess period (Locke et al., 2015).

While much of the research on autism interventions delivered within schools has been conducted in the US, the increasing number of children with autism in mainstream settings is a global trend (Majoko, 2016; Dillon & Underwood, 2012). Despite the growing number of children with autism served in schools, more research on community-partnered approaches to implementation of autism EBPs in school settings is needed, particularly surrounding individual- and organizational-level factors that predict successful implementation in schools. Elucidating the perspectives of both frontline implementers and administrators allows a deeper understanding of stakeholders’ perspectives on implementation and creates a more sustainable model of EBP use in schools that may begin to narrow the well-documented research-to-practice gap that exists in autism research (Stahmer, Suhrheinrich, Reed, & Schreibman, 2012). Their perspectives are relevant because of the school-based delivery model, in which staff members (e.g., teachers, support staff) are responsible for implementing the intervention, while administrators carry out the logistics to support the sustainment of these interventions over time. Furthermore, this may allow for the identification of contextual factors that drive the variability in implementation success observed in school settings (Mandell et al., 2013). We gathered qualitative data to explore the implementation of a social engagement intervention for children with autism within the school context. We interviewed school staff members and administrators about: 1) school factors related to general EBP implementation; 2) their experiences implementing or overseeing the social engagement intervention; and 3) barriers and facilitators specific to the implementation of the social engagement intervention for children with autism.

Methods

Semi-structured interviews were conducted with administrators (i.e., principals, vice principals), general education teachers, and school personnel (i.e., classroom assistants, aides, counselors) in 14 schools from six urban school districts in the Northeastern US. Interviewees were selected because they participated in a randomized controlled trial of a school-based social engagement intervention, Remaking Recess (Kretzmaan, Locke, & Kasari, 2012), which is designed for children aged 5 to 11 with autism. Implementation varied, such that some schools consistently used the intervention while other schools used it infrequently. Between one and three participants were interviewed in each school, with the exception of one school where six participants were interviewed. All children with autism attended a general education classroom for 80% or more of the school day. Approval was obtained from the Institutional Review Board and participating school districts.

Participants

Thirty-nine participants, including 15 elementary school administrators, 10 teachers, and 14 other school personnel completed semi-structured interviews. Participants were predominantly female (n = 35), averaged 45.2 years of age (SD = 13.5), and had a wide range of experience working with children with autism (0–38 years; M = 7.8, SD = 8.4). Of the participants, 30 were white, eight were African American, and one was Latino; 20 had a graduate degree, 13 had a bachelor’s degree, and six had a high school degree.

Procedure

We used the individual- and school-level factors from the Domitrovich and colleagues’ (2008) framework to develop an interview guide to elicit information about: 1) school factors related to general intervention implementation in schools; 2) administrators, teachers’ and other school personnel’s experiences implementing Remaking Recess and 3) barriers and facilitators related to implementing Remaking Recess. The interview guide included open-ended questions and follow-up probes for use when appropriate. Two separate but parallel interview guides were developed – one for school administrators who did not directly implement the intervention and one for school staff members who did (see Appendix). All participants consented to participate prior to the interview. Each interview was audio recorded and lasted 35–60 minutes. Participants were compensated $50 for their participation.

Remaking Recess

Remaking Recess is a school-based social engagement intervention for children with autism that incorporates evidence-based intervention strategies such as role-plays, behavioral rehearsal, and the use of peer models (Kretzmaan, Locke, & Kasari, 2012). Studies of Remaking Recess have found improvements in peer engagement on the playground via direct training and consultation with school personnel during recess (Kretzmann, Shih, & Kasari, 2015; Locke et al., 2015). The objective of the intervention is to train school staff members in facilitation strategies to promote peer engagement and relationships. The intervention was designed to transfer skills from the research team to school personnel, so that schools will be equipped to continue to support children with autism after external support is withdrawn. The training modules include the following: 1) scan and circulate the cafeteria/playground for children who may need additional support; 2) identify children’s engagement states with peers; 3) follow children’s lead, strengths, and interests; 4) provide developmentally and age appropriate activities and games to scaffold children’s engagement with peers; 5) support children’s social communicative behaviors (i.e., initiations and responses) and conversations with peers; 6) create opportunities to facilitate reciprocal social interaction; 7) sustain children’s engagement within an activity or game; 8) coach children through difficult situations with peers should they arise; 9) provide direct instruction on specific social engagement skills; 10) individualize the intervention to specific children in order to generalize the intervention to other students in their care; 11) work with typically developing peers to engage children with autism; and 12) fade out of an activity/game so children learn independence (Kretzmann et al., 2015; Locke et al., 2015). The majority of study activities took place with participating children and school personnel outside of the classroom environment. During inclement weather, intervention activities took place in the cafeteria, library, gymnasium, classroom, hallway, or auditorium for indoor recess. All intervention activities were embedded into children’s daily activities, the setting in which school personnel were already working, and each session targeted one didactic skill. Each skill was explained to school personnel and contextualized to include its applicability to children with autism and importance in relation to the development of children’s social functioning. Subsequently, interventionists modeled how to use the targeted skill with children with autism and their peers. Then, school personnel were asked to try the skill, so interventionists could provide immediate feedback. At the end of the session, school personnel were given “homework” to practice the skills during the days when the interventionists were not present. Homework was reviewed at the next session. For more information on Remaking Recess, please visit http://www.remakingrecess.org.

Data Analysis

All interviews were conducted by the first author (JL). Interviews were transcribed and uploaded to NVivo QSR 10 for data management. The coding scheme was developed using a rigorous, systematic, transparent, and iterative approach. JL, CBW, AO, and RSB independently coded four initial transcripts line-by-line to identify recurring codes. They met as a group to discuss recurring codes and developed a codebook using an integrated approach to coding as certain codes were conceptualized during the interview guide development (i.e., deductive approach) and other codes were developed through a close reading of the four transcripts (i.e., inductive approach; Bradley, Curry, & Devers, 2007). The development of the codebook used an integrated approach and included operational definitions of each code, examples of the code from the data, and when to use and not use the code. The coding scheme was refined and then applied to the data line-by-line to produce a descriptive analysis of each code, which was then refined throughout the data analytic process (Bradley, Curry, & Devers, 2007). A list of codes and definitions is provided in the Appendix. CBW and AO coded all data and overlapped on 20% of randomly selected transcripts to determine inter-rater reliability. They met together on a regular basis to discuss, clarify, and compare emerging codes to ensure consensus. Agreement was calculated based on the number of words agreed upon; agreement between raters was excellent (κ ≥ .85; Landis & Koch, 1977).

Results

We organize the primary six codes into those that related to the general implementation of EBPs and those that were more specific to the use of Remaking Recess. One code (staffing) appeared in both categories, as participants described staff members’ role in the general implementation of EBPs as well as in Remaking Recess.

General Implementation

Participants discussed the varied ways in which new programs are implemented in US public schools. In general, participants described the implementation process, the personnel involved, and how they were (or were not) supported in their implementation of new programs. Table 1 provides examples drawn from each code.

Table 1.

Examples from codes

Codes	Examples
Implementation Process	Teacher: “Most things that stick are really top-down. And, that’s probably been the rule rather than the exception in the last I’d say ten years. It’s very rare these days that something can start grass roots, but it can still happen but it really takes an individual to kind of go against the tide and make a case.” Principal: “ If it’s a district adopted program, then it comes from the district in terms of the in-service dates that teachers would be trained, how principals are supposed to schedule in-service time - and that’s really the primary way we go about training teachers with that new program.”
Staff	Teacher: “I have team meetings with different classrooms - because I’m not just with one grade. I’m with kindergarten, first, second and then I have third as well. And, other than that it’s paperwork.” Classroom assistant: “It’s a team effort. You know, we’ll have some of the experts from [the district], we’ll have the teacher, special education teacher, the principal, the parent, and just coming up with different things that we think would help the child.” Principal: “To get new programs implemented, you have to know the power structure of your school and go to some of those pebbles and rocks, who are the power players. The rocks have to be zealots that you position around the building. They will bring along the other rocks, they’ll bring along the pebbles, and sometimes they can even move the boulders. They are the people who are smart enough to either go around the boulder or stick dynamite in the boulder and blow it up so that we can move forward…”
Leadership	Principal: “…we sat and talked probably for an hour, and I listened, and I asked questions. At the end of that conversation, I asked [the teachers] to get me some information, let me understand this before we say yes and ask teachers and students and me to do another job. They came back well-prepared and they laid everything out, and I said, “Okay, how can I help?” From that point, we made it a part of the fabric of the school…I supported it, and it shows up in my weekly letters to the teachers. We talk about the progress at faculty meetings. We talk about the successes and areas for improvement. But, in the end, people did [the recess program] because we were able to show them, and we were able to get buy-in because they could see the difference of what kids were able to do.” Teacher: “[The principal] is very understanding of students who have autism. She asks that they are included as much as possible. If we need to have meetings to come up with more strategies, we can.” Principal: “It’s well-received if people feel like it’s coming from a colleague to generate that interest instead of always coming from me. So, I’m always very mindful and I orchestrate these things, although it doesn’t appear that way. Behind the scenes, I’m doing a lot of this and then when it comes time to present it, I may be sitting to the side and letting them see a colleague doing it.”
Support	Teacher: “I think [the principal] has been very supportive of it. I think she wants to see it work, and I think just the fact that she has allowed you guys to come in and work in the school shows her support.” Classroom assistant: “[The principal] even asking for me to take part in this is very supportive…because she has come up to me a few times throughout the program and asked me, ‘How it is going?’ or some different things about it.” Teacher: “I know the principal wrote a little note saying, ‘Thank you for looking into this program.’”
Barriers	Teacher: “I think it’s because their plates are so full, or they feel that their plates are so full. They don’t want to figure out how to do something else that they’re not required to” Classroom assistant: “I think [the teachers] have in their mind, they have to get this much done in the day, and I need all of the time to do what I already planned and not something additional that somebody else is telling me I need to do. Even if it’s going to be beneficial…” Principal: “We are running low on equipment this year because much of it just gets lost…Sometimes, they disappear by the end of the year. I think they break, they are left out on the playground, a teacher forgets to pick up a ball and then a neighborhood child or something like that walks away with it. But, they become depleted.” Principal: “You know, I cannot promise some extra staff because everyone is staffed slimly these days, so it is something that they would have to take on themselves to do.” Teacher: “I don’t have enough knowledge of autism to know if I’m saying the right thing, or doing the right thing, or responding the right way. They’re very foreign to me so although I try to treat them like the other kids, I don’t always get the reaction that I get from the other kids and then I don’t know what to do.”
Facilitators	Classroom assistant: “I’ve gotten good support from [teacher], in terms of brainstorming things to do. She lets us borrow games. [Student] gets to practice games in her room during break time. She is okay with that. So, she’s been supportive and likes to hear what’s going on and how everything is going with that.” One-to-one assistant: “I have had teachers ask me how it is going. When I come back in, they say, “What is going well? Anything that is bad about it?” Just concern and acknowledging my work.” One-to-one assistant: “When they see the kids having fun, it makes life a little bit easier. Because then they see like why I’m doing it. And, they understand. Most of them know that [the target student] has autism, so I’m pretty sure that they know that he needs the extra push for socialization - more so than any other kid.”

Implementation Process

Participants most often described the implementation process as top-down, coming from either the school district or the principal. District-initiated programs, which reportedly take precedence over school-initiated programs, most often involved implementing a new academic curriculum. Principal-initiated programs were more likely to address a specific non-academic need (e.g., an anti-bullying program). Though less frequent, instances of bottom-up program initiation typically involved staff members: identifying a problem in their classrooms or school, conducting research about programs that might be helpful in addressing that problem, presenting their ideas to the principal, and ultimately launching and taking ownership of a new program.

Training and cost restrictions also emerged as important codes in the implementation process. Staff members noted the importance of receiving training in new programs. They felt that when new interventions were adopted, they received limited training, opportunities for hands-on practice, or preparatory time. Cost also emerged as an important component of the implementation process, due to budgetary limitations in many US school districts. Staff members often reported that they purchased materials and supplies for programs out-of-pocket. They also described creative strategies for funding new programs such as identifying and applying for grants and partnering with local researchers and graduate students.

Staff

Staff members were described as the foundation for general implementation. Administrators, teachers, and school personnel noted the importance of understanding the power structure and hierarchy of schools in order to bring about change. One principal stated that the staff comprises different types of people – those that are eager and willing to do or try anything that is asked of them (pebbles), those that are willing to try new things after some thought and consideration (rocks), and those that will challenge or refuse to engage in anything (boulders). Often the principals’ role was to understand the power dynamics and identify the pebbles, rocks, and boulders of the school and find willing staff members, the pebbles and rocks, and position them around the building among the boulders so the boulders are not all together in order to pilot test new programs prior to widespread implementation.

Leadership

Principals were frequently described as the implementation gatekeepers in their schools and the ones who set the tone for openness to new interventions. Staff members frequently reported that principals most effectively supported implementation by creating opportunities for ideas to be shared in a welcoming environment. Successful principals empowered staff to seek out additional information and pilot programs prior to whole-school implementation. There was ongoing communication and thoughtful support for staff in their use of new programs (e.g., the provision of workshops/training, books, meeting times).

Support

Participants varied in their definition of what constituted effective implementation support. Some participants described it as a passive process (e.g., administrators granting permission to participate, asking them to participate on behalf of a student), whereas others described more active forms of support such as providing training, professional development opportunities, materials (e.g., intervention supplies, books), time (e.g., job coverage while they engaged in intervention activities), and recognition (e.g., job titles such as “expert playground specialist” or “lead paraprofessional”); thank you notes, pats on the back, recognition during school-wide announcements).

Implementation Factors Specific to Remaking Recess

Staff members and administrators reported factors specific to the implementation of Remaking Recess, including logistics, requirements, materials, staffing, space, and the target population.

Staff

Participants noted that an interdisciplinary and collaborative team approach could improve implementation of Remaking Recess, if specific roles were defined for each team member based on their job responsibilities. Staff members reported that working with students with autism requires a lot of time. Teacher, classroom assistants, and aides had responsibilities that included meeting with or calling parents, developing individualized education plans, completing communication logs/journals, and providing support in other classrooms. As a result, there are multiple staff members (teacher, special education teacher/coordinator, speech and occupational therapists, aides, etc.) involved in educating and caring for students with autism, each with a delineated set of responsibilities. Participants indicated that having many individuals involved often led to fragmented care and a diffusion of responsibility, particularly around identifying a designated Remaking Recess implementer.

Barriers

A number of barriers specific to Remaking Recess were noted. Time for planning and implementing activities was the most frequently described barrier to implementation. Participants described competing demands, such as managing district-mandated programs, that made it difficult to find time to implement Remaking Recess. Many staff members reported they typically use recess to prepare for the afternoon’s events, grade papers, or provide academic enrichment for children in need. Most participants, especially teachers, described being too busy to take on another responsibility, especially during a non-instructional period where their students were not in their care.

The second theme that emerged was that the schools did not have the resources to properly implement Remaking Recess. Many of the participating schools experienced significant budget cuts that limited materials available to facilitate intervention-specific activities on the playground or during indoor recess. Remaking Recess also requires an adult facilitator on the playground, and extra staff members are scarce in many US public schools. Participants noted challenges in consistently dedicating a staff member to be the facilitator during recess. Even when staff members were available, participants noted that many teachers and staff were not knowledgeable about or trained to work with children with autism. Participants noted that oftentimes staff members without any autism experience were assigned to work with students with autism.

Facilitators

The implementation facilitator most frequently identified by staff members was principal and teacher support. Specifically, staff members described both emotional and instrumental support. Examples of emotional support included principals and teachers inquiring about the intervention, noting children’s behavioral and social improvements, and having frank discussions about the best ways in which to support children with autism during recess. Others described teachers allowing extra time for recess or excusing children who were late returning from recess. In addition to emotional support, teachers expressed needing resources and space for implementation, particularly during indoor recess. Participants also mentioned that open communication and meeting with colleagues facilitated implementation. Some staff members were able to arrange meetings before or after school to discuss and plan intervention strategies with one another for the recess period. Lastly, staff members reported that directly observing student progress and receiving positive feedback from colleagues about Remaking Recess increased their own motivation to implement the intervention.

Discussion

This study describes the general process of implementing EBPs in US public schools as well as factors specific to the Remaking Recess intervention. While implementation research in schools has increased, the relative contribution of individual- and organizational-level factors that affect successful implementation of autism EBPs is understudied (Bonham, Willging, Sommerfeld, & Aarons, 2014; Beidas et al., 2015), particularly in school settings. Consistent with the Domitrovich and colleagues’ (2008) framework, several individual- and organizational-level constructs emerged (i.e., implementation process, staff, leadership, support, barriers, facilitators). These findings were consistent with previous qualitative studies examining mental health intervention implementation in schools where time, staffing, and support were identified as critical to implementation (Forman, Olin, Hoagwood, Crowe, & Saka, 2009; Massey, Armstrong, Boroughs, Henson, & McCash, 2005). What we learned from these principals, teachers, and other school personnel may be transferred to other contexts and may help researchers and community stakeholders identify strategies for successful implementation of EBPs in schools at different stages of implementation (e.g., planning/design, implementation, sustainment). Based on our experiences, we offer some recommendations to researchers and community stakeholders who may be interested in implementing EBPs in schools.

Systematically studying the implementation of EBPs in schools is challenging in part because the process of initiating a new intervention dramatically varies across schools and districts. The results from this study indicate that the process of implementing a new intervention in a school can be either top-down or bottom-up depending on the structure of the setting and the type of intervention (e.g., curricular or problem-specific). A top-down approach was more commonly noted for curriculum or school-wide programs while a bottom-up approach was used to address non-academic needs (e.g., anti-bullying, conflict resolution). It is important to understand which of these approaches is likely to be successful for the type of intervention being implemented. Training and cost also are important factors to consider when selecting EBPs (Saldana, 2014). Understanding the general implementation process and how to introduce a new program to a school may point to specific contextual factors within the particular setting that are necessary for successful implementation (Durlak & DuPre, 2008). The factors that are important and malleable are likely to vary depending on which direction the decision comes from and the amount of support and resources available (McIsaac, Mumtaz, Veugelers, & Kirk, 2015). These data suggest that there are multiple factors affecting successful implementation, including leadership, staff, support, and specific barriers to and facilitators of implementation within the school context. The results are consistent with Fixsen and colleagues (2013) conceptualization of the use of EBPs in schools in that implementation simultaneously proceeded bottom-up and top-down.

The process by which EBPs are implemented may differ depending on the context. It is important to establish relationships with key stakeholders, such as central office or district officials and head of school administrators, to facilitate their ability to support the implementation and sustainment of an EBP (Chambers & Azrin, 2013). Framing EBPs in a way that aligns with a school’s missions and goals, identifying and positioning the staff or the pebbles, rocks, and boulders within and around the school, and understanding the power dynamics within the school are important planning steps that may address logistical barriers to implementation. Subsequently, didactic and in vivo training with key stakeholders and teams, particularly the pebbles and rocks, may increase buy-in and lead to changes in practice (potentially for the boulders). Odom and colleagues (2013) suggest that professional development training that combines efficacious intervention practices with specific implementation strategies that support the use and sustainment of EBPs may improve services for students with autism. Given the emphasis of communication and collaboration within organizations, it also is important to communicate with school staff members about what is being adopted and implemented even though they may not directly work with the intended population. Piloting the intervention very visibly but on a small scale also may be critical. This step ensures that the program becomes part of the fabric of the school and builds a sense of community and support for the intervention.

There may be unique factors specific to working with children with special education needs that are important to consider when implementing autism EBPs in public schools. Participants noted that they often worked on a team of staff members, each serving unique roles, in order to meet the diverse needs of students with autism. Engaging the entire school team may be an important first step, in order to identify which staff members would be the most appropriate candidates to serve as implementers (Eiraldi, Wolk, Locke, & Beidas, 2015). In addition, given time and staffing constraints in schools, EBPs that align with children’s individual social and academic goals are likely to be more successfully implemented.

More specifically, Remaking Recess may differ in nuanced ways from other EBPs (e.g., staff involved, types of materials, manner of delivery) because implementation occurs during recess. Although recess is a natural opportunity for children to learn and practice social skills with peers, (Yuill, Strieth, Roake, Aspden, & Todd, 2007) many schools in the US have significantly reduced recess time (Ramstetter, Murray, & Garner, 2010). In other countries (e.g., Australia, Finland, United Kingdom, Japan), opportunities for recess are more frequent (Ridgers, 2010; Ridgers, Timperio, Crawford, & Salmon, 2012; He et al., 2013; Haapala et al., 2014), yet implementation challenges may also arise. In the US, social engagement interventions often are not prioritized in the same way that academic or behavioral interventions are in schools (Adelman & Taylor, 1998). Therefore, these interventions may not be naturally woven into the fabric of the school and may require additional thought and effort to implement successfully. While several of the barriers and facilitators identified in this study also may transfer to other EBPs, Remaking Recess likely presented some unique challenges such as the use of recess and the need for playground and activity materials, which may not be an essential expense when resources are limited. These challenges may apply to other interventions (e.g., transitions, functional routines) and settings (e.g., cafeteria, classroom) as well. For interventions that are not seen as an essential part of the schools’ mission, positive feedback from supervisors and/or direct observation of student progress may be especially important. Therefore, regularly measuring student outcomes and securing administrative support may be critical for implementation success.

It is important to ensure that all necessary logistical components are in place prior to widespread implementation of EBPs in schools. It also may be helpful to think of barriers to and facilitators of implementation as potential drivers for change that require consideration during the planning/design, implementation or sustainment of an EBP (Gibson et al., 2015). Implementation context should be considered at the outset of the study and stakeholder feedback should be solicited and incorporated at the intervention development stage rather than during the data analysis stage (Dingfelder & Mandell, 2011; Weisz et al., 2004). Interventions are frequently developed without consideration of context including school resources, policies, and constraints, which may limit the use and sustainment of these programs (Iadarola et al., 2015; Locke et al., 2015). Rather, community-based participatory models, where stakeholders are actively engaged in the development and implementation process, may be an excellent way to improve the relevance, use, and effectiveness of these programs for school personnel and children with autism (Drahota et al., 2016; Stahmer & Pellecchia, 2015).

Limitations

Several study limitations should be noted. First, these data were collected from two urban areas in the Northeastern US. Urban settings are among the most difficult contexts for successful implementation and may benefit the most from training and low-cost EBPs (Eiraldi et al., 2015); however, future studies that include participants from a broader sample of school contexts may lead to greater transferability of these findings. Second, this study employed a social engagement intervention specific to children with autism. This is a resource-intensive intervention targeting a subset of students with autism. EBPs that are broader and encompass all students may be more appealing to schools in order to maximize their investment. Third, while participants discussed top-down district-level decision-making, no district officials were interviewed for this study. Future studies should examine the perspectives of other key stakeholders involved in the implementation of school-based interventions. Lastly, because of the in-person interview format and audio recordings, participants may have been motivated to respond in a socially desirable manner.

Conclusions

This study highlights factors situated within an implementation science framework that are important for researchers to consider to successfully implement EBPs for children with autism in public schools. The results captured the varied perspectives of school administrators, teachers, and other school personnel involved in the implementation of a social engagement intervention for children with autism. The findings suggest that there are multiple factors that researchers should consider prior to the implementation of EBPs in public school settings. Specifically, both individual- and school-level constructs are important to consider for successful implementation of EBPs in schools, and further research that empirically investigates these factors is important. Future studies should examine the interdependent individual-, school-, and district-level factors associated with implementation as identified in the Domitrovich and colleagues’ (2008) framework, that may help schools overcome the challenges that impede successful implementation. Because schools continue to play an important role in the provision of services for children with autism, it is imperative to determine the best ways in which to support schools in their use and sustainment of EBPs. Cook and Odom (2013) state that, “EBPs are not guaranteed to work for everyone” (p. 137). However, EBPs that fit within the school context and align with the priorities and diverse philosophies of schools and districts may have the greatest likelihood of success (Atkins, Rusch, Mehta, & Lakind, 2015). Understanding who comprises the pebbles, rocks, and boulders in school settings and optimizing the ways in which they work together may ultimately lead to successful implementation of autism EBPs in schools.