Implementation of school-based services for students with autism: Barriers and facilitators across urban and rural districts and phases of implementation

Evidence-Based Practices in Schools

The implementation of effective practices for children with autism spectrum disorder (ASD) is mandated for school services (IDEA, 2004; NCLB, 2002). Evidence-based practices (EBPs) are those with substantial support in research and clinical expertise (APA, 2006). Several EBPs for ASD have been identified by the National Professional Development Center for ASD (Wong et al., 2015). Policy requirements for EBP use within schools have steadily increased since the Individuals with Disabilities Act was first established (See Table 1 for definitions; Fixsen et al., 2013), and when community providers use EBPs appropriately, there are positive outcomes for children with ASD (Humphrey & Parkinson, 2006). However, data on EBP implementation within community based programs suggests limited use and moderate to low fidelity (Locke et al. 2015; Pellecchia et al. 2015; Stahmer et al. 2014; Suhrheinrich et al. 2013). Additionally, even when teachers demonstrate accurate use of some EBPs, they may vary in the fidelity with which they implement other EBPs (Pellecchia et al., 2015) or they may not continue to use them over time (e.g., Suhrheinrich et al., 2018). Characterizing current implementation practices and identifying factors that act as “barriers to” or “facilitators of” EBP implementation in schools is an important first step to understanding the current process of EBP implementation within the school context. Understanding this process may help identify needs of school stakeholders prior to implementation to promote alignment between research and the school setting (Kasari & Smith, 2013), as the fit of the intervention to school contexts could impact the fidelity of implementation (Harn et al., 2013). Implementation science can provide frameworks and structure to understanding current implementation processes in schools, which may lead to more effective implementation methods to support the increased use of EBPs (Odom et al., 2014).

Table 1.

California Education Department Terms
Term	Definition
California Autism Professional Training and Information Network (CAPTAIN)	CAPTAIN is a multiagency network developed to support the understanding and use of Evidence Based Practices for individuals affected by Autism Spectrum Disorder across the state.
District Location	School District Locations based on The National Center for Education Statistics. Urban areas with populations over 2,500, and Rural areas are areas that do not lie within Urban areas.
Due Process	The right of parent participation, and challenge, in all aspects of assessment, identification and placement is assured; involves mediation or administrative hearing procedures and complaint procedure in case of disputes.
Individuals with Disabilities Education Act (IDEA).	Legislation that makes available free appropriate public education to eligible children with disabilities throughout the nation and ensures special education and related services to those children.
Individualized Educational Planning (IEP)	The right of a child to an educational program designed to meet his/her individual needs and based on adequate assessment is assured. By age 16, this includes the development of an Individual Transition Plan (ITP) to provide for transition into the world of work.
General Education	Education provided to students that follow the general content standards in California (not specialized education).
Special Education	(a) Specially designed instruction, at no cost to the parent or guardian, to meet the unique needs of individuals with exceptional needs whose educational needs cannot be met with modification of the regular instructional program; and (b) Related services, at no cost to the parent or guardian, that may be needed to assist those individuals to benefit from specially designated instruction.
Special Day Class	A Special Education Class which provides special education services to students whose needs cannot be met by the general education program or other support services.
Special Education Local Plan Area (SELPA)	Consortiums in geographical regions of sufficient size and scope to provide for all special education service needs of children residing within the region boundaries. Each region develops a local plan describing how it would provide special education services.
Participant Roles and Definitions
Term	Definition
Administration	Refers to leaders within the school district (Assistant Principals, Principals, Director of Special Education, etc.).
Autism Coordinators	Manages programs for students with ASD and coordinates with teachers and administration in the implementation of programs for students with ASD.
Autism Specialist	Manages or gives advisement in implementation of programs for students with ASD.
Behavior/Learning Specialist	Creates plans and/or gives advisement for all students regarding behavior intervention and individualized learning.
Paraprofessional	Provides direct support to students and implements programs under the supervision and guidance of the teacher.
Program Specialist	Evaluates and gives advisement in the implementation of programs for students in special education.
School Psychologist	Conducts psychological evaluations, provides psychological services, and gives input in the implementation of programs for students with ASD.
Program Supervisors	Supervises the implementation of programs in special education and provides training to professionals implementing the programs.
Special Education Coordinators	Manages programs in special education and coordinates with teachers and administration in the implementation of special education programs.
Speech-Language Pathologist	Provides speech-language services and gives input in the implementation of programs for students with ASD.
Teacher on Special Assignment	Teachers that train other teachers in the implementation of programs in special education.

^*Note. professionals with these job titles can have a variety of roles or could be equivalent in their responsibilities with other job titles, depending on the needs of the school district. For example, in a rural district, the Special Education Coordinator may only have the “Special Education Coordinator” job title, but also have the responsibilities of a Behavior/Learning Specialist, due to few enough children in the district requiring these services.

Implementation Science

Implementation science is the study of methods to promote the adoption and integration of evidence-based practices, interventions, and policies into routine care (Eccles & Mittman, 2006). The Exploration, Preparation, Implementation and Sustainment (EPIS; Aarons et al., 2011) framework provides additional structure for understanding factors that support the implementation process. The EPIS framework has been used across service sectors, including child welfare, community mental health, and education (Aarons, et al., 2011; Brookman-Frazee, et al., 2019; Moullin et al., 2019; Brookman & Stahmer, 2018; Stahmer et al., 2018), and is being used to guide the exploration of factors that support EBP in schools (Willging et al., 2016). EPIS characterizes the implementation process using four phases: Exploration, in which implementers consider how specific EBPs and factors may influence the implementation process; Preparation, in which the EBP is selected and support systems are developed; Implementation, in which active training and implementation of the EBPs occurs; and Sustainment, in which the intervention is stabilized and supported through funding systems and ongoing fidelity monitoring (Aarons et al., 2011). In addition to its emphasis on multiple phases of implementation, the EPIS model also accounts for the separate and interactive influences of organizational and provider characteristics (“inner context”) and broad system-level factors such as inter-organizational environment and student support/advocacy (“outer context”), bridging factors that link outer and inner contexts, and EBP characteristics at each phase of the implementation process (Moullin et al., 2019). The limited study of contextual variables that impact EBP implementation for students with ASD indicate inner context factors as district and classroom characteristics (Locke et al., 2019; Stahmer et al., 2018; Suhrheinrich, 2020) and outer context factors as state- and SELPA-level climate and culture, leadership, and structure (Stahmer et al., 2018). In the present study, we consider the district structure and personnel-related factors as inner context, and district location (urban and rural) as an outer context factor. To define urban and rural areas, we used the definition used by the National Center for Education statistics and the US Census Bureau: an urbanized area is an area with populations of 2,500 or more, while rural areas are defined as populations of less than 2,500.

Factors Affecting Implementation of EBP for ASD

The limited study of contextual variables that impact EBP implementation for students with ASD indicate inner context factors (specifically, organizational and personnel characteristics, Locke et al., 2019; Stahmer et al., 2018; Suhrheinrich, 2020) and outer context factors (specifically, state- and SELPA-level climate and culture, leadership, and structure, Stahmer et al., 2018). Thus far, most studies have evaluated inner context factors such as classroom characteristics. However, outer context factors are also important to consider when evaluating effective implementation practices. One outer context factor, location (whether the school is located in an urban versus rural district), has been indicated as impacting special education more broadly (Jung & Bradley, 2006), but has not been explored in implementation of school-based ASD services. Location has been evaluated as a factor in special education programs (but not ASD specific programs): schools in rural areas have shortages in teachers and staff (Knapczyk et al., 2001) and have less access to personnel who are highly trained in special education (Pennington et al., 2009). Another study evaluated urban-rural differences in parent satisfaction (but not EBP implementation) with ASD services and found that while parents in both rural and urban areas experienced challenges accessing services and trained educators, parents from rural areas reported significantly more difficulty accessing professionals who were trained in treating ASD (Murphy & Ruble, 2012). Another study evaluated EBP implementation for students with ASD in urban districts (but not rural districts) and reported barriers of staffing, prioritization of competing demands, level of respect and support, and availability of resources as impacting implementation (Locke, 2015). Using qualitative methods, the current study applies the EPIS framework to characterize factors impacting the implementation of practices for students with ASD across urban and rural school districts. The guiding questions of the study include: 1) Are contextual factors perceived as barriers or facilitators, and do these perceptions vary by district location and 2) What are the key factors impacting implementation across the EPIS Exploration, Preparation, Implementation and Sustainment phases?

Method

A focus group approach was used to obtain a comprehensive understanding of participant’s perceptions of the implementation process for new practices for students with ASD in urban and rural school districts. The focus group approach targets identification of participant’s experiences with, or opinions about, the topic under investigation; the use of a structured interview guide; and the exploration of subjective experiences of participants in relation to predetermined research questions (Gibbs, 1997; Merton & Kendall, 1946). Focus group methodology is commonly used in implementation research to gather information about settings as well as perspectives on contextual factors impacting implementation (Hamilton & Finley, 2019).

Participants

Participants (n = 33) were school-based service providers who supported children with ASD in preschool through high school/transition public school programs (See Table 1 for definitions of participant categories). Six separate focus groups were conducted: 4 groups included participants (n = 3; n = 4; n = 7; n = 9) from urban-located school districts and 2 groups included participants (n = 2; n = 8) from rurally-located school districts. Participants were mostly female (n = 32), and worked as program specialists (n = 11), autism specialists (n = 5), speech language pathologists (n = 2), behavior/learning specialists (n = 3), program supervisors (n = 2), special education coordinators (n = 2), autism coordinators (n = 1), school psychologists (n = 2), teachers on special assignment (n = 4), and Special Education Local Plan Area (SELPA) coordinators (n = 1). A SELPA is an intermediary agency who oversees the implementation of special education services throughout California. All participants were employed through a school district of SELPA and provide or oversee special education services throughout these entities and schools. For example, specialists are specialized staff that provide training and coaching to teachers, while SELPA coordinators are responsible for the provision educational and associated programming for districts in their SELPA, including resource allocation and implementation of professional development and intervention programs.

Procedure

Participants were recruited in partnership with the California Autism Professional Training and Information Network (CAPTAIN), an agency that author JS has partnered with to explore the implementation of EBP. CAPTAIN is a multi-agency network developed to support the understanding and use of EBP for individuals affected by ASD across the state of California (captain.ca.gov). Participants were recruited to include representation from urban and rural areas, based on The National Center for Education Statistics (NCES, 2006). All eligible CAPTAIN members from districts representing the highest and lowest 20% of districts based on student enrollment were sent a recruitment letter via email inviting them to participate in a 1-hour focus group. The potential participants who accepted the invitation completed consent forms outlining the risks and benefits of the study. These procedures were approved by the Institutional Review Board at San Diego State University. Focus groups were conducted in-person in County Offices of Education or public library conference rooms across the state of California: two groups each in Stockton, Camarillo, and San Diego and facilitators varied. Only participants and researchers were present during the focus groups.

Consistent with a well-established focus group methodology (Merton, 1987; Schensul, 1999), a structured guide was developed by the authors to facilitate discussion among participants through exposure to uniform stimuli and to provide a basis for the quantification and characterization of responses within and across focus groups. The focus group facilitators were female, had experience with qualitative data collection, held either M.A. or Ph.D. degrees, and worked as research faculty or staff. JS was one of the facilitators. All facilitators followed a guide with detailed instructions, questions, and additional probes to ensure consistency across groups. All participants were given a participant guide which included the focus group agenda and questions. Participants were asked to recall an implementation effort to introduce or scale up services for students with ASD within their district. Guiding questions targeted: 1) key personnel involved in decision making and change, 2) perceived barriers to implementation, 3) resources needed and how they would be accessed, and 4) perceived implementation process and likelihood of success across training components. Participants received a $25 incentive.

Data Analysis

Focus groups were audio recorded, transcribed, and then coded using NVivo QSR 11 (QSR International; 2012). Field notes were also collected by a second researcher (not the facilitator) during each focus group. Coders (MM, BV) used an iterative coding and review process informed by a framework-driven analytic approach that is often employed in qualitative implementation research (Hamilton & Finley, 2019). In this approach, framework constructs may be used to develop categories or codes for qualitative data, while remaining open to findings that may emerge outside of set codes. The first stage of the process involved a deductive approach that used the EPIS framework to identify constructs of interest a priori. These constructs included the stages of implementation (Exploration, Preparation, Implementation and Sustainment) as well as factors such as leadership, funding, personnel characteristics, and staffing processes. Specifiers, including barrier and facilitator, were created to assign to individual codes. Barrier was coded when a factor was described as a challenge to successful implementation of a practice. Facilitator was coded when a factor was described as helping in successful implementation of a practice. The second stage included the development of a preliminary codebook by the research team which was informed by the identified constructs of interest and contained operational definitions of codes, guidelines for use, and examples for inclusion for each code. An iterative approach guided development of the codebook, in which transcripts were reviewed by the research team and through discussion modifications were made to the codebook according to emerging ideas and themes.

The coding scheme was then applied to all focus group transcripts by authors MM and BV and interrater reliability was calculated for 50% of the transcripts. The coders met weekly to compare and discuss coding, and the discrepancies were discussed and resolved to ensure consensus, thus interrater reliability was 100%. NVivo coding queries were then used to search the transcript for relationships between codes in different categories (i.e. the “attitudes and buy-in” factor is associated with the “Exploration” phase of implementation). The qualitative coding was then utilized to identify most salient factors and themes. To facilitate characterization of relevant themes between urban and rural districts, the team identified those that were shared and unique to each type of district location. Themes are presented by focus group type (urban and rural), consistent with guidelines for focus group methodology that state the unit of analysis is the focus group, not the individual participant (Hamilton & Finley, 2019).

To answer our first guiding question, inner-context factors (personnel and organization-related themes) are discussed as barriers to or facilitators of implementation across urban and rural school districts. For the second guiding question, we present key themes across the EPIS stages of implementation.

Community Involvement

This study was conducted with the California Autism Professional Training and Information Network (CAPTAIN) as a community partner. All participants were community-based services providers within school programs. Additionally, co-author BR is a certified behavior analyst and community service provider.

Results

Several key themes (inner context factor) were identified including personnel-related themes, such as attitudes and buy-in, knowledge and skills, staffing and burnout as well as organizational themes, such as leadership approval, support and expectations, district structure, competing priorities, time for effective professional development, litigation and due process, and materials and resources. A majority of the key themes were identified as both barriers to and facilitators of implementation of programs for students with ASD, however a few themes were indicated as only a barrier or facilitator (see Figure 1). Exploration of themes across district location (urban or rural; outer context factor) revealed variability (see Figure 1). Further exploration across phases of implementation also indicated differences in key themes (see Figure 2). Themes are numbered based on saliency, and themes that have the same number are tied for saliency. Themes are centered based on fit, for example, if a theme is aligned left, it fits more in the urban than in rural settings.

Figure 1.

Barriers and facilitators across district type, numbered by saliency (1 being the most salient). Barriers and facilitators numbered with ”a” and “b” are equivalent in saliency.

Figure 2.

Themes across district type and EPIS stages of implementation.

Personnel-Related Themes

Several personnel-related themes emerged. Overall, these were shared by participants representing both urban and rural districts, however, some facilitators and barriers were unique to rural or urban districts. Examples of these shared and unique factors for each personnel-related theme are provided below.

Attitudes and Buy-In.

This theme identified attitudes and buy-in of both direct providers (Teacher, Speech Pathologists) and leadership (Coordinators, Principals) as influencing the implementation of practices. Participants from both urban and rural districts described the influence of attitudes and buy-in as a barrier, as providers’ attitudes towards the implementation of training, EBPs, and new programming were often described as negative; however, participants from urban districts also reported that attitudes and buy-in sometimes helped implementation. Participants from both urban and rural districts noted that lack of buy-in from other direct providers, particularly general education teachers, made it difficult for them to promote the implementation of new practices within their school or district:

I think one of the biggest challenges, honestly, is working with the general ed teachers, up in my county anyways. And the teachers in our special day classrooms, they want the training, they want the information. So actually trying to disseminate the information to the general ed teachers and then actually getting them to implement some of the interventions are, for me, a little bit more difficult.

Knowledge and Skills.

This theme captured descriptions of instances where the presence or lack of provider or leader (Teacher, Principal, Assistant Principal, etc.) knowledge or skills influenced implementation. Participants in urban districts described staff knowledge and skills as both facilitators and barriers, however, they were more often described as barriers.

Participants in rural districts described staff knowledge and skills as barriers to implementation. In urban districts, participants also described how a principal’s knowledge of special education influences the principal’s ability to evaluate teacher use of practices:

I think principals are more in tune to with what’s happening in general [ed]. So they know when they walk into a classroom what they’re supposed to see, and so it’s easy for them to evaluate teachers. And when we’re talking about evidence-based practices for children with autism, most principals don’t have any of that background knowledge.

Participants in rural districts also described instances where principals did not have any experience regarding special education, which created implementation challenges:

We had a brand new principal that had no idea about special ed, she’d never worked on a campus with special ed before. I’m not sure how that even happens, but, completely clueless. So she was wonderful in that her attitude was, “Whatever you need me to do, I’m happy to do. Just tell me what it is, because I don’t know automatically.” But […] it meant that there was one more person to try to train and figure out how to schedule time with.

Additionally, participants described that, due to the lack of direct provider knowledge and skills in special education, participants experienced challenges in training those providers to implement practices:

It would be literally problem solving one kid at a time. And going in and training the teacher in one sort of thing, “Do this when he does that.” how to use a token economy, how to prompt. Because they weren’t people with a big history in special ed.

Staffing.

This theme captured descriptions of how staff loss (direct providers or leadership) influenced the implementation of practices. While participants in both urban and rural districts described staff turnover as a barrier, participants in rural districts described it as a more impactful barrier. Participants in rural districts described that particularly in rural counties, direct service providers such as teachers or paraprofessionals do not work for their districts for long periods of time:

In the smaller counties, especially if you’re hiring the younger teachers or interns, they don’t usually stick around for very long. So, you try to implement something, and you do a lot of training with them for your one or two years, three if you’re lucky, and then they’re gone.

Participants in urban districts reported instances where they planned to implement a new program, however, then found that teachers were not qualified to implement the program with fidelity:

The problem that we are experiencing is there is not enough qualified teachers to carry on the special day program. So every year there is an increase in openings, but not enough qualified staff to deliver this intensive support in EBP.

Additionally, staffing at the trainer level was discussed. Urban district participants mentioned that generally there was one person spearheading training, and their availability to train all providers was limited:

But because of staffing, because of time, and not being able to—they’re the only staff member that does that job on the campus so it’s really hard to get them.

Burnout.

This theme captured descriptions of workload influencing the ability to implement new practices (descriptions of burnout, stress, wearing many hats, etc.). Participants in both urban and rural districts described burnout as a barrier to implementation.

Participants in rural and urban districts reported that teachers’ workloads impeded their implementation of EBPs as they described teachers to be too burnt out to attend trainings or receive coaching:

It’s more of ‘I’m so busy during the day, I don’t want to do that. I’m so burnt out, I don’t really trust in the system, I’m not going to these things.’ And it used to be like, teachers were very happy to get to their jobs.

Participants in urban districts also described teachers as being overwhelmed with their workloads, which further impedes their ability to implement new practices.

I mean the teachers are just like already overwhelmed. A lot of times they ’re already in survival mode and it’s hard to move them beyond the survival mode. It’s like “ok well what are we going to do to keep this going?” So they can’t see sustainability when they’re focused on survival.

Organizational Themes

Several key themes related to organizational factors were identified based on participant’s comments including leadership approval, support and expectations, district structure, competing priorities, time for effective professional development, litigation and due process, and materials and resources. Although barriers and facilitators within these key themes were sometimes shared, there were more unique factors between urban and rural districts at the organizational level. Examples of these shared and unique factors for each organizational-related theme are provided below.

Leadership Approval, Support, and Expectations.

This theme captured descriptions of how the presence or lack of approval, support, or expectations by leaders within schools or districts (e.g. Director of Special Education, Principal, etc.) influenced the implementation of practices (change in programming, training practice or outcomes, or use of EBP). Overall, in both urban and rural districts, leadership approval, support, and expectations were described as facilitators that supported or helped the successful implementation of a practice. However, in urban districts, participants also described these as barriers. For example, participants described the lack of administrative support as a barrier to gaining staff participation.

Participants from urban districts expressed the importance of administrative support on teacher provider use as well as to promote training:

It’s hard because as an autism specialist, I am not a supervisor. And I think that’s where a disconnect is because we can go in as specialists, we can make our recommendations, we can train, we can do all of those things, we can support, but really when push comes to shove in terms of implementation, if you don’t have that administrative backup, there’s no muscle behind what you want to do. And you can have all the compassion and passion that you have, and you bring, but if they’re not doing it, who’s going to make them?

Coaching a new teacher, one of the things that was said to me was, “I don’t care if you’re in CAPTAIN or whatever you are, you are not an administrator”. Luckily, administrators supported me, on that particular aspect. So it’s very important to have that administrator support, coming from that level.

Participants consistently reported that although the approval for release time for training is necessary for implementation, it is generally given based on policy adherence rather than on training needs. Furthermore, at times leadership will not provide the resources or staff required due to a lack of funding:

I need this aide, and this will save you $200,000 in 3 years because we won’t have to hire this aide later. Can we do that? And it’s like, no because it costs more money right now. But then if we don’t do this now, I promise you we’re going to have 5 years of this.

District Structure.

This theme captured participant’s descriptions of instances where the structure of their district (geographical area, population, etc.) influenced the implementation of practices. Participants from both rural and urban districts described the influence of district structure as a barrier; however, it was reported as a more impactful barrier by rural districts. In rural districts, participants described geographical issues (long distances between school sites) and staff shortages as challenges to conducting trainings and managing cases:

We can get a half day [training]. Being able to do a webinar would be even better. Because again, when you’re in a rural district, a half day is a lot. And you’re the only person in your whole district that can manage a crisis.

Meanwhile, participants from urban districts reported population size and communication with leaders as challenges:

For our district we have so many kids, and a lot of kids with autism, and a lot of people in the administrative level […] that there’s a lack of communication from our special ed department down to the administrators of the school sites.

Competing Priorities.

This theme captured participant’s descriptions of instances where other priorities influenced implementation (i.e. funding, policy). While participants from both rural and urban districts described funding as both a facilitator and barrier to implementation, it emerged as a main barrier for urban districts.

Participants from urban districts described that other competing priorities make it difficult for them to gain administrator approval for trainings to support the implementation of EBPs:

ELL [English Language Learning] training and those types of things that are mandates that are there, you know front and center, that they’re faced with every day and reports and all of that. So, I think it’s really difficult to get their ear in terms of providing specific trainings for evidence-based practices. And to find the time to let them release their teachers for training. I think that’s a huge hurdle, that release time.

In urban districts, participants also reported challenges in convincing decision-makers of the cost-effectiveness of proactively investing in programs:

You’d have to forefront a lot in order to save on the backend, but nobody wants to put that money out there right now to save in the future because we’re always just right now.

Time for Effective Professional Development.

This theme captured participant’s descriptions where time, lack of time, availability, or scheduling influenced implementation. While participants in both urban and rural districts described that time was a barrier, participants in rural districts discussed time as a more impactful barrier.

Participants in rural districts described instances where the trainees and trainers do not have time to participate in training or coaching due to drive time or the lack of dedicated training time:

Getting time designated to do that, and finding ways to do that in a meaningful, organized structured way – the time is not there. The time is not there, we can’t get people released from their classrooms because we don’t have enough subs.

Participants in urban districts with few staff qualified to provide those trainings, no time to follow up for fidelity checks and coaching, and heavy workloads and lack of release time for trainees:

I do feel like that need for that follow-up and that coaching piece, not just like we give the training and give the materials and then we’re gone. And I think that that is a real time constraint for those of us in this middle level that we all feel like we see the importance of it, we see research on that, but being able to carve out the time from everything else that we have to do to actually be able to be side-by-side and problem-solve. It’s just that we’re not full-time coaches or professional development providers, we just have so many other things on our plate.

Litigation and Due Process.

This theme captured participant’s descriptions of how litigation and due process influenced the implementation. Due process is a formal means for caregivers to submit a complaint about their child’s special education services provided in accordance with the Individuals with Disabilities Education Act (IDEA), and if not resolved, the caregiver may take file a civil action against the school or district (IDEA, 2004). Participants in urban districts often described litigation and due process as facilitators to increasing the implementation whereas participants in rural districts did not discuss litigation.

In urban districts, participants described how new practices and programs were implemented to prevent due process from occurring:

My director is constantly telling us, ‘this team has really kept us out of due process.’ He tells us that all the time, ‘you guys are doing a great job of just keeping this from going.’ He recognizes that the programs we implement, like those social communication classes, some of them only have six, seven kids in and we have a teacher assigned for that period. And so that’s a pretty heavy investment.

Materials and Resources.

This theme captured participant’s descriptions of how access to materials and resources (e.g., curriculum, manipulatives, and classroom space) influenced implementation. While participants in both urban and rural districts described materials and resources as both barriers and facilitators to implementation, participants in rural districts described them as more impactful barriers.

Specifically, participants in rural districts described lack of curricula:

Our [Special Day Class] SDC program hasn’t really had a curriculum for ten years, ever since I’ve been there. So, my speech intern, […] she has really taken a passion to that, so she took her own time to research curriculum, she’s presented it to the curriculum director, presented it to their Special Ed Director. So she’s getting things approved and like jumping through loopholes and doing all these different things she needs to do to try to get an actual curriculum for those classes. But if it wasn’t for her then we still wouldn’t have anywhere near to a curriculum for them.

EPIS Phases

To explore factors affecting school-based ASD services throughout the implementation process, we identified themes across phases of the EPIS implementation framework. This section also identifies the influences of implementation that are most salient in each phase across district location (urban or rural; see Figure 2). Figure 2 indicated the most salient themes in each stage of implementation within urban, rural, or both districts. For example, funding is a theme in the exploration stage in urban districts.

Exploration.

This theme captures instances that are related to the Exploration phase of implementation, where participants describe the presence or absence of gathering information and resources and identifying needs prior to selecting a new program. In discussions related to the Exploration phase, participants in urban districts mentioned competing priorities as a salient theme, participants both urban and rural districts discussed student needs as a salient theme, and participants in rural districts also mentioned approval, support, expectations as a salient theme. Participants in urban districts discussed how competing priorities affect funding decisions for autism programs as well as litigation. Participants in rural districts discussed how student needs influence the programs developed each school year. For example, if a cluster of students need specific autism supports, a new special day class will be developed to meet those needs.

Preparation.

This theme captures instances that are related to the Preparation phase of implementation, where participants describe the presence or absence of identifying resources, developing the training plan or program, and preparing or hiring staff for the new practice. In discussions related to the Preparation phase, participants in urban districts mentioned attitudes & buy and competing priorities as salient themes, participants in both urban and rural districts mentioned approval, support, and expectations as a salient theme, and in rural districts, participants mentioned district structure, student needs, and staff knowledge & skills as salient themes. In urban districts, participants discussed how competing priorities affect funding, which in turn affects the development of programs including resources and trainings. In rural districts, participants discussed that needs of the program changed depending on the needs of the students.

Implementation.

This theme captures instances that are related to the Implementation phase, where participants describe the presence or absence of actively implementing a program; training staff, managing the program, and evaluating the effects of the program. In discussions related to the Implementation phase, participants in urban districts mentioned approval, support, & expectations, district structure, and competing priorities as salient themes, participants in both urban and rural districts mentioned attitudes and buy-in as salient themes, and participants in rural districts mentioned staff knowledge and skills, student needs, and time as salient themes. Participants in urban districts expressed that the teachers lacked buy-in due to their workload. Participants in rural districts reported that the lack of staff knowledge and skills impacted procedures. Some participants from rural districts also reported that veteran teachers are less likely to buy-in to new interventions and practices, and that the lack of time for training was described to be a major hindrance.

Sustainment.

This theme captures instances that are related to the Sustainment phase, where participants describe the presence or absence of maintaining a program over time; retraining staff, identifying outcomes, and following through with intervention procedures. In discussions related to the Sustainment phase, participants in both urban and rural districts mentioned time as a salient theme. Participants in urban districts reported that teachers were attending trainings, but there was no follow-up or sustainment plan, so the teachers did not continue to implement strategies that they learned during the initial training. Participants in both urban and rural districts stressed that the lack of enforcement also contributes to lack of sustainment.

Discussion

This study explored the process of implementation of practices for students with ASD in public school programs in urban and rural school districts. Participants described instances where they implemented and supported others to implement these practices, and key themes in influences on implementation were then identified using the EPIS framework: our team coded organizational, personnel-related factors (inner context) and policy, funding factors (outer context) in the implementation of school-based services for students with ASD.

Inner and Outer Context Factors

Although EPIS has been used as a framework to guide implementation in school programs (Locke et al., 2019; Stahmer et al., 2018; Suhrheinrich, 2020), there has been limited evaluation of outer context factors. While our results section categorizes themes into inner context factors (personnel-related themes and organizational themes) as influences on implementation, outer context factors were also coded as influences on implementation (state policies and funding), and differences in implementation were examined in one outer context factor (location).

Although not specifically labeled as an “outer context” factor in the EPIS framework, there is evidence to suggest that the influence of location as a factor exists in special education services and thus is examined as an outer context factor in this study. Based on the location, districts can differ in organizational structure, services available, and personnel factors (Jung & Bradley, 2006; Murphy & Ruble, 2012; Pennington et al., 2009). Our study focuses specifically on school-based services for students with ASD but supports this previous research on location as an influence on implementation in school-based special education services as participants from different locations (urban versus rural) described different barriers and facilitators to implementation. Other outer context factors to consider are funding given to and policies mandated in school districts. For example, states and SELPAs can vary in their methods to calculate the allocation of funding to Special Education services (Willis et al., 2020), which could impact inner context factors such as the number of resources and time for training that the school personnel receive.

Our study reflects how both inner and outer context factors influence the implementation of practices. For example, the district structure theme was categorized as an organizational factor (inner context), however, it is also influenced by location, policies, and funding (outer context). District participants from urban areas emphasized district policy as a concern whereas district participants from rural areas discussed regional challenges (long distance travel for trainings) as a concern. It is important for future studies to consider both inner and outer context factors, as both contextual information could be used to adapt clinical interventions and implementation processes to fit such differences to maximize fidelity and improve student outcomes (Harn et al., 2013).

Common Organizational and Personnel Factors

Our study identified inner context factors, both at the organizational and personnel level, as influences the implementation of school-based services for students with ASD across district location. At the organizational level, key themes included leadership, district structure and growth, competing priorities, litigation and due process, and time. This is consistent with literature indicating that implementation leadership and implementation climate influence the use of EBPs (Aarons et al., 2014; Novins et al., 2013) as well as overall organizational climate (Glisson et al.2010). In this study, participants reported competing priorities, such as funding for other trainings and programs, as a barrier to receiving leadership support for implementation. Leadership emerged as a barrier and facilitator for teacher buy-in and implementation and for approval of release time. The leadership landscape in schools is complex and warrants more investigation to effectively support implementation efforts. District structure (i.e. size, distance) and growth is another organizational theme that emerged.

Personnel-related factors were also indicated as important themes in implementation of EBPs in schools. Personnel, or provider, factors have been highlighted in implementation frameworks (Aarons et al., 2011; Moullin et al., 2015) and are often studied in implementation research. Provider attitudes and buy-in have been found to impact the adoption, use, and implementation of EBPs (Aarons, 2004; Aarons et al., 2011; Henggeler et al., 2008; Reding et al., 2014). Negative provider attitudes towards the implementation of EBPs have been identified as a significant barrier to EBP use (Harn et al., 2013; Stahmer et al., 2012). In the present study, participants from both urban and rural districts identified attitudes and buy-in as main barriers to implementation. Specifically, participants reported a disconnect between special education staff and general education staff or leadership. Participants also reported factors such as burnout, staffing and knowledge and skills as factors that ultimately impede personnel in undertaking implementation efforts. For example, participants report that although the principal is responsible for evaluating special education teachers and programs, they often do not have experience in special education and therefore may not be able to identify and promote effective implementation strategies. These findings highlight the need to explore how to effectively support staff in implementing EBPs in classroom settings. Across our findings, themes were frequently discussed as both barriers and facilitators indicating that when a factor, such as leadership support, is present it is influential in successful implementation and when not present, can hinder efforts. This again illustrates the complexity of the special education school system. While buy-in and support might be present from some administrators and teachers, other key players essential to the process are not involved or supportive.

Differences across District Location

While there were many common factors indicated by participants from both urban and rural districts, there were also differences. Differences between access to services by families of children with ASD living in rural versus non-rural areas has been well documented by researchers. For example, studies report that families living in rural areas have less access to support groups, receive later diagnoses, and have fewer available providers than in non-rural areas (Drahota et al., in revision; Mandell et al., 2005; Mello et al., 2016). With regard to school services specifically, evidence suggests that district location may account for differences in organizational structure, services available, and personnel factors (Jung & Bradley, 2006; Murphy & Ruble, 2012; Pennington et al., 2009). Additionally, stakeholders (parents, administrators, and educators) in urban and rural areas also identify different factors impacting intervention implementation. For example, parents, educators and school administrators in urban areas report (1) tensions between participant groups (with professional educators, paraprofessionals, and parents), (2) a need for ASD-specific training, and (3) a desire for school culture accepting differences as facilitators and barriers to the implementation of new interventions (Iadarola et al., 2015), whereas parents of children with ASD in rural areas report (1) financial burdens, (2) insufficient number of professionals, programs, and resources, (3) lack of family-centered services, and (4) lack of education of pediatricians/providers as key barriers (Elder et al., 2016). Outcomes from the current study align closely with those identified by Elder and colleagues (2016), suggesting confirmation of findings, but also a need for additional evaluation with varied participants from the public school system. Adapting interventions to fit the differences in these contextual variables could help maximize fidelity of EBP implementation (Harn et al., 2013).

EPIS Phases

In a recent systematic review of the EPIS framework, researchers found that projects measured factors primarily across two of the EPIS phases, implementation and preparation (Moullin et al., 2019). This study adds to this existing literature by evaluating inner and outer context factors across all four phases of implementation in school implementation efforts. Findings indicate personnel factors are particularly salient in the implementation phase with factors such as staff knowledge and skills and attitudes and buy-in playing a key role. Whereas in the exploration, preparation, and sustainment phases, student needs, and time for training and follow-up were reported as primary factors influencing implementation. These outcomes make an additional contribution to the growing body of work on EPIS as a guiding framework for implementation.

Leadership support emerged as a theme throughout each phase of implementation in different capacities. For example, in the exploration and preparation phases, administrators have competing priorities that can affect funding decisions for new programs and practices, while in the implementation phase leadership may affect training efforts and in the sustainment phase, leadership support affects teachers’ follow-through with implementing practices. This is consistent with research in other fields that points to leadership as a critical component of implementation across phases (Brookman-Frazee et al., 2019).

Limitations and Considerations

The results of this study may inform tailoring of EBP implementation strategies based on school district characteristics (Owens et al., 2014). However, as participants consisted of a small sample size drawn from one state, the results of this study may not reflect a full representation of variability within other states or regions. Our sample reflects the complex make-up of school-based service providers in California, which may not align with the organizational structure and job roles in other states and internationally. Future studies should recruit participants more broadly and explore additional relevant contextual factors. Additionally, although the sample of participants representing rural and urban areas reflected the overall membership in CAPTAIN, there were more participants from urban school districts. Despite the limitations and considerations, the methodological approach and outcomes may be useful in future research and implementation efforts across more varied school-based programs supporting students with ASD.

Implications and Future Directions

The outcomes of this study suggest the benefit of additional research on contextual factors supporting school-based implementation. By better understanding how perceived barriers and facilitators of implementation vary by inner and outer context factors and across phases of implementation, implementation interventions can be tailored for maximum impact. A systematic review of implementation efforts that have used the EPIS framework found that a majority of studies study inner-context factors (90%), while only 57% study outer-context factors. This study serves as a first step in analyzing inner and outer-context factors in special education implementation efforts. Furthermore, to our knowledge this study is one of the first to characterize influences on implementation in special education programs across rural and urban locations.

This study is part of a larger project focused on tailoring a leadership strategy, the Leadership and Organizational Change for Implementation (LOCI), to support the use of EBPs in schools. The LOCI intervention has been used to train leadership in healthcare organizations to create a climate that supports the use of EBPs in healthcare agencies and to improve implementation outcomes (Aarons et al., 2015; Aarons et al., 2017).This project serves as the first step in identifying implementation factors that will be used to guide adaptation of LOCI. Given that participants from both urban and rural school districts reported leadership support as a major influence on the implementation of EBPs, this is an important direction of research to pursue. The themes identified through this qualitative work may be integrated targets of leadership development or improved implementation climate as part of the adapted LOCI.