Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jan 1.
Published in final edited form as: Eur J Behav Anal. 2019 Jul 17;21(1):55–73. doi: 10.1080/15021149.2019.1641952

Implementation Science, Behavior Analysis, and Supporting Evidence-based Practices for Individuals with Autism

Samuel L Odom 1,2,3, Laura J Hall 2, Jessica Suhrheinrich 2,4
PMCID: PMC7518547  NIHMSID: NIHMS1534225  PMID: 32982588

Abstract

Although applied behavior analysis researchers have created efficacious treatment and intervention practices for children and youth with autism spectrum disorder (ASD), there is a gap between research and practice. Implementation Science (IS) and Organizational Behavior Management (OBM), based with Applied Behavior Analysis, are two parallel fields that could close this gap. This paper provides descriptions of both IS and OBM, highlighting their commonalities and unique featuers. The paper concludes with examples of how researchers have used IS and OBM to promote practitioners’ use of evidence-based practices and services for children and youth with ASD.

Applied behavior analysis (ABA) has had a profound impact on socially significant behavior change, and individuals with autism have been primary beneficiaries of ABA practices (Peters-Scheffer, Didden, Korzilius & Sturmey, 2011). Much has been learned from ABA about practices that are efficacious when implemented with fidelity (Wong et al., 2015). Yet, a gap exists between the knowledge generated by the science of behavior analysis and the routine use of practices with individual clients or students (Dingfelder & Mandell, 2011). Research indicates that it takes up to 17 years for new practices to be developed, evaluated, and integrated into routine care (Morris, Wooding, & Grant, 2011). Improving the efficiency and effectiveness of this process will increase the impact of ABA-based interventions and the science broadly.

In ABA there has been a strong tradition of focusing on the fidelity or treatment integrity of which an intervention is implemented (Greham, Gansle, & Noelle, 1993). Intervention fidelity is recognized as a critical area of focus across human service disciplines (Durlak & DuPre, 2008; O’Donnell, 2008). However, closing the research to practice gap for children with ASD as well as other recipient groups, requires a broader systems perspective that leads to “scaled up” use of effective practices with fidelity. The purposes of this paper are a) to articulate the need for increasing use of evidence-based intervention practices and services for individuals with ASD, b) to examine two disciplinary fields that have emerged to support the adoption and implementation of new practices and/or services in existing programs (i.e., implementation science and organizational behavior management), and c) to provide examples of how each has been, or could be used to support practitioners use of evidence-based practice and services for children and youth with ASD.

The Case for the Importance of Effective Programs for Children and Youth with Autism Spectrum Disorder

The prevalence of autism has accelerated rapidly in the last 20 years. The Center for Disease Control and Prevention in the United States has reported a prevalence rate 1 in 59 school-aged children (Baio et al., 2018). Autism is characterisized by challenges in social communication and restrictive and repetitive behavior (American Psychiatric Association, 2013), functioning that covers a wide spectrum of abilities, and associated co-morbidities such as intellectual disability and mental health disorders (e.g., social anxiety, depression, sleep disorders, seizures). Building on the early work by Ivar Lovaas and colleagues (Lovaas, Schreibman, & Koegel, 1974), researchers have actively developed and demonstrated the efficacy of ABA-based practices and programs for children and youth with autism (Wong et al., 2015). Despite the identification of evidence-based practices, the outcomes for young adults with ASD as they leave schools and transition into adulthood are among the worst of any disability group (Roux, Rust, Anderson, & Shattuck, 2017). That is, although practices with demonstrated effifacy exist (Wong et al., 2015), it appears that practitioners are not implementing them in programs and services for children and youth with autism (Hess, Morrier, Heflin, & Ivey, 2008).

Approaches to Supporting Adoption and Implementation of Effective Practices

In response to the need for scaling up the use of effective practices in human services and also private industry, two relatively independent fields, Implementation Science and Organizational Behavior Management, have emerged. In the subsequent sections, each will be described and similarities between the two will be highlighted.

Implementation Science

Two pioneers in Implementation Science (IS), Eccles and Mittman (2006), defined IS as the “study of methods to promote the adoption and integration of evidence-based practices, interventions, and policies into routine care” (p.1). As a discipline, IS emerged in response to the recognition that research was generating evidence of effective practices but practitioners were not implementing them in their own programs. The discipline has become formalized over the last two decades with the publication of the journal Implementation Science, in 2006, the formation of the Global Implementation Society (https://globalimplementation.org/society/ ), an international Global Implemenation Initative (https://globalimplementation.org ), and currently a biennial schedule of international conferences.

Conceptual Frameworks in Implementation Science

Conceptual frameworks in IS are a linked set of procedural components that lead to implementation to practices or services (Eccles & Mittman, 2006), and researchers have proposed a number of different conceptual frameworks (Albers, Mildon, Lyon, & Shlonsky, 2017). To serve as illustrative examples, the authors have chosen two specific implementation frameworks: the Exploration, Preparation, Implementation, and Sustainment (EPIS) model (Aarons, Hurlburt, & Horwitz, 2011) and the Active Implementation Framework model developed by the National Implementation Research Network (NIRN, Fixsen, Blase, Naoom, & Wallace, 2013; Metz et al., 2015). These frameworks reflect feaures or concepts often seen in other IS frameworks and have been the most frequently employed for promoting practices and services for individuals with ASD.

The EPIS model (see Figure 1) proposes that multiple factors affect the degree to which a program or practices (i.e., identified as innovation in this figure) is implemented. One set of factors operates outside of the context in which the practice, such as discrete trial training, would be implemented. These outside factors include service system policies, funding, and client advocacy, to cite just a few. Factors internal to the organization also that affect implementation include leadership, provider characteristics (e.g., openness to change), and quality of fidelity monitoring and support. Factors that can bridge these two type of influences are partnerships with community academic leaders and purveryors. In addition, factors related directly to the innovation itself, such as characteristics of the innovation and fit with the organizational context may affect implementation. The influences of these factors often operate in a reciprocal manner through interconnections and linkages.

Figure 1.

Figure 1.

Open in a new tab

EPIS Conceptual Model. (Figure originally appeared in Moullin, Dickson, Stadnick, Rabin, & Aarons, 2019). Open access publication.

The NIRN model, like EPIS, specifies factors operating on different dimensions, which are called implementation drivers (See Figure 2). Like the EPIS model, leadership plays a primary role, with the forms of leadership identified as technical (e.g., arranging time, providing funding) and adaptive (e.g., motivating). A second set of drivers are related to the organization and include variables at the systems level (e.g., specified in EPIS as outer context), facilitative administration, and a data system that informs decisions. The third set of drivers focus on actual selection of the practice or innovation, training to implement the intervention, and coaching to support implementation with fidelity.

Figure 2.

Figure 2.

Open in a new tab

NIRN Drivers (National Implementation Research Network, 2015, reproduced with permission)

Progressive phases of implementation frameworks.

Many IS model developers propose that implementation of an evidence-based program or set of practices within an organization occurs in progressive phases. In their review of 25 implementation conceptual frameworks, Myers, Durlak, and Wandersman (2012) found that most models progress from early phases focusing on introduction of the program/practice to be implemented to later phases that focus on ensuring future use of the program (i.e., sustainment). As can be seen in Figure 3, the EPIS model (Aarons, et al., 2012) focuses on four such phases: Exploration, Preparation, Implementation, and Sustainment, with steps and activities specified within each phase. Exploration involves needs and resource assessment, identification of potential EBP and consideration of how they would fit with the context, and organizational readiness. The Preparation phase includes planning how and when to integrate the EBP into the existing system. During the Implementation phase the EBP is put in place with training, evaluating outcomes, and on-going monitoring being key activities. Sustainment involves ongoing leadership, funding, embedded EBP culture, and social network support. In the NIRN model (Figure 3), phases are similar in function. They included Exploration, Installation, Initial Implementation, and Full Implementation (National Implementation Research Network, n.d.), with the later phase including features that address sustainment across time. These temporal, progressive phases may be helpful in understanding the implementation process and factors as they are applied to unique service settings.

Figure 3.

Figure 3.

Open in a new tab

Progressive Phases of EPIS (Aarons et al., 2011; open source) and NIRN (National Implementation Research Network, no date; reproduced with permission).

Key Concepts of Implementation Science

In their review and analysis of the IS literature, Williams and Beidas (2018) proposed a shift from examination of specific IS models to examinations of specific “core determinates” that influence implementation. The convergence of these determinants (i.e., across reviews of IS conceptual frameworks) provide insight to the concepts that characterize the IS field. The description of IS for this paper relies most closely on the organization determinents that Williams and Beidas (2018) identified and the organization features articulated by Li, Jeffs, Barwik, and Stevens (2018). We also have drawn from multiple concepts described in systematic reviews of the implementation science literature (Albers et al., 2017; Crable et al., 2018; Damschroder et al., 2009; Fixsen et al., 2009; Lyon et al., 2018). In their earlier review of 61 implementation and dissemination models, Tabak, Khoon, Chambers, and Brownson (2012) found that IS models often operate across levels of socio-ecological systems, similar to Bronfenbrenner’s ecological systems theory (Bronfenbrenner & Morris, 2006). In such a conceptual scheme, factors that influence implementation operate at a distal level, characterized by cultural and/or political variables, at an organizational level in which a new program/practice may be implemented, and at a proximal level close to the actual implementation itself. Key IS concepts are found in Table 1, and are ordered from the distal to proximal features.

Table 1.

Corollary Features of Implementation Science and Organizational Behavior Management (From Glenn & Malott, 2004; Glenn et al., 2016; Lee, Jeffs, Barwick, & Stevens, 2018; Williams & Beidas, 2018)

Implementation Science Organizational Behavior Management
Macrosystem Level (Distal)
Sociopolitical Context Meta-contingencies
Organizational Level
Organization culture and climate Interlocking Behavioral Contingencies (IBC) aligned in a Cultural Lineage
Leadership-champion Leadership – integrates key processes or skills
Resources Resources
Readiness to Implement Degree of Consistent and Aligned IBC
Microsystem Level (Proximal)
Teamwork-Collaboration-Communication Teamwork – Collaborative behaviors-Communication
Data-based decision making Data-based decision-making
Training Training
Coaching Behavioral Coaching/Performance Feedback
Open in a new tab

Socio-cultural context refers to influences exerted through cultural or social norm, values, and priorities, perhaps historical events, and also broader social policies or regulations. For example, the rapid acceleration of ASD has increased the emphasis at the organizational (i.e., schools) and services level (i.e., classes and clinics) to implement EBP. Organizational culture refers to “shared assumptions, values, norms, and behavioral expectations” (Williams & Beidas, 2018, p. 10). Associated concepts include organizational climate (i.e., impact of work on implementers sense of personal well-being) and implementation climate (i.e., shared agreement about extent to which EBP use is expected/rewarded and organization policies support implementation). Another similar concept is the perceived need for the intervention by stakeholders and stakeholders’ involvement in designing key features of the implementation process (Albers et al., 2017). Leadership is provision of guidance or direction for the organization and/or the implementation efforts. A similar concept of having a champion refers to influential members of the organization who support and advocate for implementation. Resources refer to the things needed for the implementation to be employed. They may include funding for implementation efforts, staffing and workload, and allocated time for staff (Li et al., 2018). The readiness of an organization refers to the degree that organizational members are psychologically and behaviorally prepared to make the necessary organization changes (Weiner, Amick, & Lee, 2008). Such readiness is tied to organizational resource capacity, compatibility of the organizational/practitioner culture and the program or practices to be implemented.

At the more proximal level, teamwork is characterized by the emotional valences of social networks of implementers and nature of informal and formal communication among implementers. Evaluation, monitoring, and feedback consists of leaders judging quality of their implementation but also leaders seeking feedback about the success of the program/practice implementation (Li et al., 2018). Training by direct implementers and even for leaders, is a necessary but alone not a sufficient feature of the process that ensures a high level of implementation.

Applied Behavior Analysis and Organizational Behavior Management

Similar to implementation scientists, behavior analysts have created a conceptual framework guiding their understanding of the inter-relationships of systems, organizations, and communities that effect implementation outcomes. This work exists in a sub-discipline of behavior analysis called organizational behavior management (OBM). The objects of study in OBM are both the behavior of individuals in organizations and the behavior of organizations as functioning entities (Glenn & Malott, 2004). OBM “focuses on what people do, analyzes why they do it, and then applies an evidence-based intervention strategy to improve what people do” (Cunningham & Geller, 2011, pp. 70–71). Its aim is to establish a technology of broad-scale performance improvement and organizational change so that employees are more productive and organizations and institutions are more effective in achieving their goals (https://behavior.org/help-centers/behavior-in-organizations/#). In 1982, the OBM Network (www.OBMNetwork.com) was founded to develop, support, and enhance the growth and vitality of organizational behavior management. A primary source of dissemination for OBM is through the Journal of Organizational Behavior Management (JOBM), which has been publishing research since 1977, although OBM work has also appeared in other scientific journals (e.g. Behavior and Social Issues).

Key Concepts of Behavior Analysis and Organizational Behavior Management

OBM behavior analysts start the process of intervention or organizational change by collecting baseline data on performance within the identified system., such as by conducting a Behavior Systems Analysis. The central premise of a Behavioral Systems Analysis is that organizations are complex systems and that changes in any one aspect of performance in an organization may affect the performance in other components of the organization (Ludwig, 2017). Therefore, a systems analysis would consider data on the dynamic interactions among its internal components, the relationship of the components to the critical systems, and the performance of the organization as a whole (Rodriguez, Bell, Brown, & Carter, 2017). The goal of conducting a systems analysis is to design an intervention that will result in improvement in areas of poor performance, maintain components of high performance, and align individual performance with organizational goals (Brethower, 2018).

At the individual level, a Performance Improvement Analysis could be used to determine the desired performance described in observable and measurable terms, and identify performance goals required to meet this performance. The baseline data obtained on the identified performance goals would lead to training, coaching and performance feedback for individuals who would be implementing the practice or program.

Identifying and manipulating contingent relationships is a hallmark of behavior analysis. On the systems level, metacontingencies are the focus, which Glenn et al. (2016) have defined as “reoccurring interlocking behavioral contingencies that have an effect on an aggregate product” (p. 13). These interlocking behavioral contingencies (IBC) occur when the behavior of one individual acts as the antecedent for another individual’s behavior and the consequences are shared by both individuals (Dagen & Alavosius, 2008). Most IBCs involve verbal behavior (e.g., rules) of participants or have a social component as described in Skinner’s book, Verbal Behavior (1957). For example, the rule-governed behavior (vocalized or covert verbal statements) of a worker on an assembly line is reinforced by an individual paycheck and also by contributing to a high quality finished product that is sold and maintains the organization that employs all the workers (Glenn et al., 2016).

The aspect of the organization that is replicated, or stays the same over extended time, can be considered the cultural lineage (Glenn & Malott, 2004). Typically, the cultural lineage is maintained even when there are changes to the participants in the organization. For example, in the autism intervention model initially developed by McClannahan and Krantz (2006), the procedural features of the model has maintained when staff or even leadership leave the program. Similarly, it would be important to the success of an organization that individuals have the competencies needed to access the IBCs of the system. In this example, when a new staff member takes a job at one of the dissemination sites using this model, her competencies would need to be aligned with the staff performance expectations (e.g., appropriate delivery of behavior-specific praise, providing sufficient opportunities to respond), which would result in a positive evaluation from her supervisor.

Behavior analysts agree that leadership plays an essential role in initiating, shaping and sustaining socially significant changes at all system levels (Houmanfar & Mattaini, 2016). Effective leaders are highly skilled observers of the context, have knowledge about the competence of individuals that comprise the team and allocate tasks accordingly, ensure effective communication within the team and throughout the component and system, and use data to monitor fidelity and progress and to determine needed adaptations (Alavosius, Hourmanfar, Anbro, Burleigh, & Hebein, 2017).

The functional relationships between meta-contingencies, their product and the cultural or social consequences is clear in simple organizations, but increase in complexity in larger organizations. For example, Tourinho and Vichi (2012) illustrate this complexity by comparing individuals in a fishing village who only fish for sustenance (i.e., to gather and eat the product of their efforts--fish) with a similar context when fisherman gather the fish to sell. The latter example requiring organization of specialized components (e.g., fleet maintenance, packaging fish). The management of each component would build and sustain interlocking behavioral contingencies (i.e., individual’s performance of their specific jobs) that leads to the common goal (e.g., wage payment used for sustenance).

Frequently emitted individual behaviors having a cumulative, socially significant effect have been called macrobehaviors. Glenn and colleagues (2016) referred to these as “socially-learned operant behavior observed in the repertoires of several/many behaviors of members of a cultural system” (p. 18). There are occasions when these macrobehaviors result in a cultural cusp – or the point where there is a significant sociocultural change in the system, organization, or community. An example of a system of interlocking contingencies that resulted in a cultural cusp is reflected in efforts to promote seat belt use in the U.S. in the early 1980s. In 1984, as a result of data linking seat belt wearing to the prevention of serious injury and death during automobile accidents, the U.S. Congress enacted legislation that required drivers and passengers to wear seat belts. Tickets for not wearing seat belts were contingently administered by police (Geller, Bruff, & Nimmer, 1985). Information about the data for the consequences of not wearing a seat belt in an accident became more publicized, and short pointed messages (e.g., “Click it or Ticket”) began to appear in the public media. Also, community-based interventions were developed (Geller et al., 1985) to positively reinforce seat belt use. The convergence of initiatives created interlocking contingencies that increased seatbelt use by 70% over two decades. Behavior analysts would identify this as a cultural cusp, resulting in seatbelt use becoming a cultural norm in U.S. society.

OBM Research Focus

Similar to implementation scientists, OMB researchers conceptualize factors that influence practitioners’ use of interventions or practices as occurring at the systems, organization, and community levels (Glenn & Malott, 2004), However, in a review of studies published in JOBM from 1998 to 2009 (VanStelle et al., 2012) and a subsequent update focusing on human services research (Gravina et al., 2018), authors found that the majority of the research focused on the performer level, with relatively few studies targeting organizational and community level variables. The most common interventions were antecedent and training interventions with most of the studies also including feedback and praise. This trend appears to be shifting with more recent publications in OBM focusing on systems change and manipulation of interlocking behavioral contingencies. For example, in 2016 two issues of JOBM had a special section focused on applications of behavioral science devoted to sociocultural challenges (Houmanfar, 2016). There was also a recent special section of JOBM focused on leadership and cultural change (Houmanfar & Mattaini, 2016).

Exploring Linkages Between IS and OBM

Although IS and OBM developed somewhat independently, both have a common goal of promoting individuals’ use of an innovation, program, or practices (e.g., use of EBPs for children with ASD by teachers, seat-belt use by drivers and passengers). In this section, commonalites between the two fields and well as one unique difference are examined.

The Necessity of An Operationalized Program/Practices

To employ IS or OBM models or concepts, the program or practices to be implemented must be operationally defined (Crable et al., 2018). Whether the goal is promoting teacher’s use of EBP for students with autism or employees’ “safe lifting” in an industry setting, each feature of the program must be specified, ideally with measures of fidelity or treatment integrity. This has sometimes been called the “it” of implementation, and without such clear specification the implementation process is either inefficient or impossible.

Key Concepts

Common key concepts in IS and OBM appear in Table 1. These concepts are organized from distal to proximal influences. IS acknowledges the influence of factors happening at the sociocultural and sociopolitical level (Atkins, Rusch, Mehta, & Lakind, 2016), while Glenn and Malott (2004), from an OBM perspective, have discussed environmental complexity (e.g., government regulation, economic fluctuations). At the more proximal organization level, primary concepts in IS are organizational culture and climate, whereas OBM has similar concepts embedded in the cultural lineage of the organization. Leadership is highlighted in both IS and OBM. The concept of readiness for implementation is a common IS concept and is paralleled in OBM by the degree to which the IBCs necessary for successful implementation are aligned with goals within and across system components and levels of hierarchy. A term related to readiness that is consistent for both IS and OBM is the access to or allocation of the necessary resources (e.g., funding, staff time) to support implementation.

At the most proximal level, teamwork/collaboration/communication among staff who will be implementing the practice/program and perhaps their supervisors are key elements for both IS and OBM. Similarly, both ideally depend on data collection to inform decisions, and some form of direct training for individuals implementing the program/practices. Both IS and OBM researchers have long been well aware that workshops or training without performance feedback in context are ineffective (Stein, 1975). This has been demonstrated in research across varied settings and participants (Yoon et al., 2007).

Supporting Implementation of Interventions and Services for Children and Youth with ASD

A growing body of work directly applies IS and OBM concepts and practices to support the use of EBPs in programs for children and youth with ASD. This area of study aims to accelerate the integration of EBP and improve sustainment of interventions across service settings. Some reports are based on identifiable conceptual frameworks while others examine individual features of IS and OBM as identified in Table 1.

IS Conceptual Frameworks Applied to ASD

Several recent implementation research efforts have employed IS frameworks to select strategies and target outcomes and guide study procedures. For example, following the EPIS conceptual framework, Stahmer and colleagues adapted an identified EBP for ASD (i.e., pivotal response training) to fit the school/classroom context (Stahmer, Suhrheinrich, Reed, & Schreibman, 2012), which addresses an Innovation Factor (see Figure 1). As part of a randomized efficacy trial of the adapted practice, significant associations between inner context variables (i.e., leaders’ early involvement in recruitment and provision of space; teachers’ attitudes toward intervention ) and implementation outcome variables (i.e., fidelity, use, sustainment) were identified (Suhrheinrich, Reith, Dickson & Stahmer, in press). In a similar line of research, Brookman-Frazee and colleagues have also used EPIS to guide work focused on improving use of evidence-based strategies with youth with ASD presenting with challenging behavior in community mental health programs. This included extensive effort to address Innovation/Context Fit by developing an intervention based on parent need (Brookman-Frazee, Baker-Ericzén, Stadnick, & Taylor, 2012), provider need (Brookman-Frazee, Drahota, Stadnick et al., 2012), and service delivery context (e.g., weekly psychotherapy/counseling session lastly approximately 50 minutes).

Additional ASD services reseearch guided by the EPIS framework is currently in progress. In a protocol publication (i.e., description of a study that is planned but for which results are not yet reported), Brookman-Frazee and Stahmer (2018) describe barriers to implementation across community mental health and education service settings supporting youth with ASD and are currently conducting a randomized trial evaluating implementation interventions aimed at Inner Context factors: improving organizational leadership of implementation and improving provider attitudes toward adopting new EBPs (Brookman-Frazee and Stahmer, 2018). Also in process is an exploratory study examining the association of EPIS outer and inner context variables on use of EPBs in educational services for students with ASD (Stahmer, Suhreinrich, Shetter, & Hassrick, 2018). The relationship between classroom-level outcomes and multiple inner and outer context factors is being explored, including: inter-organization networks, implementation climate and leadership, provider attitudes, and leadership and coaching practices (Stahmer, et al., 2018).

Researchers have also employed the NIRN conceptual framework to ASD services research. To promote teachers’ use of EBPs for children with ASD in 12 states, Odom, Cox, and Brock (2013) used the NIRN conceptual framework to develop statewide systems of professional development and reported changes in classroom quality and teachers’ use of EBPs. In addition to student academic outcomes, the authors also evaluated competency drivers such as teacher fidelity as a training and coaching outcome. Also, with high school programs for students with ASD, Odom et al. (2014) proposed using the NIRN framework to promote the use of a comprehensive treatment model, which resulted in implementation in the treatment group of a cross-site RCT study (Steinbrenner, Odom, Hall, & Hume, in press). Prior to training teachers, the research team completed other strategic Installation procedures, including establishing and implementation team and securing leadership support. In an inservice training program in Hong Kong, Ho, Lam, Sam, & Arthur-Kelly (2018) followed a NIRN-based implementation model that prepared teachers of student with ASD to teach recognition of and reaction to emotion signals from others. The training program moved from exploration to installation to implementation phases, resulting in teachers implementing the model well and reporting positive outcomes for their studients. In summary, there is a substantial, and growing, body ASD services research utilizing IS frameworks.

IS and OMB Concepts

Although most of the identified conceptual frameworks for supporting implementation of EBP for children and youth with ASD have emerged from the IS discipline, there are several shared concepts in OBM (see Table 1). For example, at the macrosystem level, international initiatives exist that influence sociopolitical context and meta-contingencies inherent in IS and OBM implementation conceptualizations. The Autism Speaks Global Public Health Initiative (https://www.autismspeaks.org/global-autism-public-health-initiative-gaph) and the World Health Organization (https://www.who.int/mental_health/action_plan_2013/eb_resolution_childhood/en/) proactively work with government leaders to create supports for use of EBPs and services for children and youth with ASD. In the U. S., initiatives that resulted in changing state insurance laws have changed the contingencies for access to services for many families of children with ASD (Mandell et al., 2016). Employing an ethnographic approach to study factors affecting implementation of EIBI programs in Sweden, Roll-Pettersson, Olsson, and Ala’l-Rosales (2016) noted as barriers the tensions among national service agencies, which reflects the influence of sociopolitical contextual fators and meta-contingencies.

At the organizational level, Williams et al. (2019) examined the association between organizational culture and climate, as predictor variables, with teacher fidelity and use of three EBPs in elementary school programs for children with ASD. Teachers in schools with “comprehensive” profiles (i.e., high proficiency culture, positive climate) had significantly higher fidelity for two of the three EBPs examined and more positive work attitudes as compared with schools having school profiles reflecting less supportive organizational cultures and climates. Similarly, Kratz et al. (2019) defined organizational climate as the degree that teachers perceive a practice as being feasible as well as being expected and rewarded by supervisors. They found a significant association between public school teachers’ perceived climate and the fidelity with which teachers implemented the STAR program (i.e., an ABA comprehensive treatment program) with students having ASD.

Although implicit in many of the IS conceptual frameworks, researchers from both the OBM and IS literature propose leadership as a distinct construct associated with implementation of evidence-based practices (Lyon et al., 2018). In addition to leadership style, leaders have control of resources (e.g., staff time, space, funding for training) that they may chose to allocate or not allocate to implementation efforts. Readiness and degree of aligned IBC are similar IS and OBM constructs that may influence implementation of EBPs. Hustus and Owens (2018) defined readiness at the practitioner level as support for or resistence to adoption and implementation of practices. For example, Stahmer and Aarons (2009) used the EBPAS to assess pracitioners’ attitudes toward use of EBPs in their programs, with the hypothesis that such attitudes may affect implementation, and Kratz et al. (2019) reported that teacher attitudes did predict EBP fidelity . Readiness and alignment of IBC also extends up the organizational level to include program administrator’s intention to support implementation of an intervention (e.g., provision of training, time for planning, aligning recognition for teachers’ efforts to implement). Although similar in concept to organizational climate, readiness or the degree of aligned IBC may be used to determine if necessary prerequisites for success are in place in an organization and may predict the success with which EBPs or other evidence-based programs are likely to be adopted and used with fidelity.

At the microsystem or context most proximal to children/youth with ASD, a number of strategies have been employed to promote teachers use of EBPs. Most of the implementation research has occurred at this level for both IS (Mullin, Dickson, Stadnick, Rabin, & Aarons, 2019) and OBM (Gravina et al., 2018). A comprehensive review of this literature is beyond the scope of this paper, but examples can be provided. As noted, common point of agreement for both IS and OBM is that didactic training through workshops, while important for introducing concepts, will usually not lead to implementation of EBPs. Hall, Grundon, Pope, and Romero, 2010) and Smith, Parker, Taubman, and Lovaas (1992) documented this “noneffect” phenomena, respectively, for paraprofessionals working with preschool children having ASD who participated in a one-day workshop and group home staff who attended a week-long training.

One key feature for promoting implementation from both the IS and OBM perspective is behavioral coaching. In their Collaborative Model for Promoting Competence and Success, Ruble et al. (2018) employed initial assessment and consultation to establish transition goals for adolescents with ASD, and then provided behavioral coaching for the teachers to promote implementation of the transition plans, finding positive effects on goal attainment. Other researchers have similarly demonstrated the importance of coaching and particularly the role of performance feedback in supporting teachers adoption and use of EBPs (Stahmer et al., 2015).

Leaders in OBM and IS also identify teamwork, collaboration, and communication as key influence on implementation. For example, in the inservice training program previously noted, Ho et al. (2018) employed a collaborative teaching model that involved initial introduction of skills to be learned, immediately practicing newly learned concept, communicating their experiences with other teachers learning the instructional strategies, and observing other teachers implementing the intervention lessons.

Conclusion

For children and youth with ASD, ABA has generated a great deal of knowledge about effective interventions and human services for this population, yet the gap between research and practice remains wide. To bridge this gap, both IS and OBM have developed unique and common strategies to support practitioners in learning, implementing, and sustaining effective approaches for children and youth with ASD. IS researchers have developed a number of identifiable conceptual frameworks for implementation, with several being used directly with children and youth having ASD. In addition, IS and OBM have in common a number of individual practices that researchers have employed in supporting teachers’ and practitioners’ implementation of EBPs. Taken together, these fields provide helpful guidance to support the adoption, use, and organizational scale-up of EBPs for children and youth with ASD. IS and OBM practices are especially important when considering international deployment of EBPs for children and youth with ASD, as inter-organisational networks, cultural political factors, and meta-contingencies vary among nations. Also, the relevance IS and OBM extends beyond practices and services for children and youth with ASD. For behavior analysis to move forward as a discipline that addresses socially and globally important problems, professional development programs in institutions of high education that train ABA practitioners will be required, especially in countries where these programs do not currently exist. Initiating and sustaining such training programs will require institutional and organizational changes. The IS and OMB strategies highlighted in this paper could be instrumental in achieving such changes.

Acknowledgements

Samuel L. Odom is Senior Research Scientist at the Frank Porter Graham Child Development Institute at the University of North Carolina at Chapel Hill, Adjunct Professor at San Diego State University, and Visiting Professor at Stockholm University. Laura J. Hall is Professor at San Diego State University. Jessica Suhrheinrich is Associate Professor at San Diego State University.

The work in this article was supported by Grant No. H325E170001, Office of Special Education Programs, and Grant No. R324A150047 from the Institute on Education Sciences, both in the U. S. Department of Education, and Grant No. K01MH109574 from the National Institute on Mental Health, U. S. Department of Health and Human Services. The contents of this article does not necessarily reflect or represent the policy of the departments. Also, the authors appreciate the helpful comments and assistance by Mark Alavosius, Allison Metz, and Greg Aarons.

Footnotes

Ethical Issues and Conflict of Interest Statement

Because the manuscript does not report an empirical study involving human or animal participants, approval from our respective Institutional Review Boards at the University of North Carolina at Chapel Hill and San Diego State University was not required. The authors afirm that they have no conflict of interests associated with this manuscript.

Contents of this paper were originally presented at the 2nd International Summit and Conference on Behavior Analysis and Autism in Higher Education, Stockholm, Sweden.

References

  1. Aarons GA, Hurlburt M, & Horwitz SM (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23. doi: org/10.1007/s10488-010-0327-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aarons GA, Green AE, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, Dilovsky JF, Hecht DB, & Chaffin MJ (2012). Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science, 7 (32). doi: 10.1186/1748-5908-7-32 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Albers B, Milton R, Lyons AR, & Shlonsky A (2017). Implementation frameworks in child, youth, and family services: Results from a scoping review. Child and Youth Services, 81, 101–116. doi: 10.1016/j.childyouth.2017.07.003 [DOI] [Google Scholar]
  4. Alavosius MP, Houmanfar RA, Anbro SJ, Burleigh K, & Hebein C (2017). Leadership and crew resource management in high-reliability organizations: A competency framework for measuring behaviors. Journal of Organizational Behavior Management, DOI: 10.1080/01608061.2017.1325825 [DOI] [Google Scholar]
  5. American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders: Dsm-5 Arlington, VA: Author. [Google Scholar]
  6. Atkins MS, Rusch D, & Mehta TG (2016). Future directions for dissemination and implementation science: Aligning ecological theory and public health to close the research to practice gap. Journal of Clinical Child and Adolescent Psychology, 45, 215–216. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Baio J , Wiggins L, Christensen DL, Maenner MJ, Daniels J, Warren Z, … & Dowling NF (2018). Prevalence of autism spectrum disorder among dhildren aged 8 years — Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2014. MMWR Surveill Summ, 67(No. SS-6):1–23. DOI: 10.15585/mmwr.ss6706a1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Brethhower D (1997). Related applications to OBM: Behavioral systems analysis Cambridge, MA: Cambridge Center for Behavioral Studies, Inc; https://behavior.org/wp-content/uploads/2018/05/Behavioral-Systems-Analysis-Intro.pdf [Google Scholar]
  9. Bronfenbrenner U & Morris PA (2006). The bioecological model of human development. Lerner R & Damon W (Eds.), Handbook of Child Psychology, Vol. 1 (pp. 793–828). New York: John Wiley & Sons. doi: 10.1002/9780470147658.chpsy0114 [DOI] [Google Scholar]
  10. Brookman-Frazee L, Chlebowski C, Suhrheinrich J, Finn N, Dickson KF, Aarons GA, & Stahmer AC (2019). Characterizing shared and unique implementation influences in two community services systems for autism: Applying the EPIS framework to two large-scale autism intervention community effectiveness trials. Adminisration and Policy in Mental Health and Mental Health Services Research, online publication 10.1007/s10488-019-00931-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Brookman-Frazee L & Stahmer AC (2018). Effectiveness of a multi-level implementation strategy for ASD interventions: Study protocol for two linked cluster randomized trials. Implementation Science, 13(1), 66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Crabel L, Biancarelli D, Walkey AJ, Allen CG, Proctor EK, & Drainoni M (2018). Standardizing an approach to the evaluation of implementation science proposals. Implementation Science, 13:17. doi: 10.1186/s13012-018-0770-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Cunningham TR, & Geller S (2011). What do healthcare managers do after a mistake? Improving responses to medical errors with organizational behavior management. Journal of Communication in Healthcare, 4, 70–87. doi: 10.1179/175380611X13022552566290 [DOI] [Google Scholar]
  14. Dagen JC, & Alavosius MP (2008). Bicyclist and motorist environments: Exploring interlocking behavioral contingencies. Behavior and Social Issues, 17, 139–160. doi: 10.5210/bsi.v17i2.2062 [DOI] [Google Scholar]
  15. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, & Lowery JC (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Scoping Studies: Advancing the Methodology, 4, 50. doi: 10.1186/1748-5908-4-50 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Diener LH, McGee HM, & Miguel CF (2009). An integrated approach for conducting a behavioral systems analysis. Journal of Organizational Behavior Management, 29, 108–135. doi: 10.1080/01608060902874534 [DOI] [Google Scholar]
  17. Dingfelder HE & Mandell DS (2011). Bridging the research-to-practice gap in autism intervention: An application of diffusion of innovation theory. Journal of Autism and Developmental Disorders, 41, 597–609. 10.1007/s10803-010-1081-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Domitrovich CE, Bradshaw CP, Poduska JM, Hoagwood K, Buckley JA, Olin S, … & Ialongo NS (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: a conceptual framework. Advances in School Mental Health Promotion, 1, 6–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Drahota A, Aarons GA, & Stahmer AC (2012). Developing the Autism Model of Implementation for autism spectrum disorder community providers: Study protocol. Implementation Science, 7:85 (Online publication). [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Durlak JA, & DuPre EP (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350 [DOI] [PubMed] [Google Scholar]
  21. Eccles MP, & Mittman BS (2006). Welcome to implementation science. Implementation Science, 1, 1–3. doi: 10.1186/1748-5908-1-1 [DOI] [Google Scholar]
  22. Farrell LV, Cox MG, & Geller S (2007). Prompting safety-belt use in the context of a belt-use law: The flash-for-life revisited. Journal of Safety Research, 38, 407–411. doi: 10.1901/jaba.2010.43-321 [DOI] [PubMed] [Google Scholar]
  23. Fixsen DL, Blase KA, Naoom SF, & Wallace F (2009). Core implementation components. Research on Social Work Practices, 19, 531–540. doi: 10.1177/1049731509335549 [DOI] [Google Scholar]
  24. Fixsen DL, Naoom SF, Blase KA, Friedman RM, & Wallace F (2005). Implementation research: A synthesis of the literature Tampa, FL: University of South Florida, Louis de la Parte. [Google Scholar]
  25. Geller ES, Bruff CD, & Nimmer JG (1985). The “Flash-for-Life”: A community- based prompting for safety-belt promotion. Journal of Applied Behavior Analysis, 18, 309–314. doi: 10.1901/jaba.1985.18-309 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Glenn S, & Malott M (2004). Complexity and selection: Implications for organizational change. Behavior and Social Issues, 13, 89–106. DOI: 10.5210/bsi.v13i2.378 [DOI] [Google Scholar]
  27. Glenn S, Malott M, Andery MA, Benvenuti M, Houmanfar R, Sandaker I, … Vasconcelos LA (2016). Toward consistent terminology in a behaviorist approach to cultural analysis. Behavior and Social Issues, 25, 11–27. DOI: 10.5210/bsi.v25i0.6634 [DOI] [Google Scholar]
  28. Gravina N, Villacorta J, Albert K, Clark R, Curry S, & Wilder D (2018). A literature review of organizational behavior management interventions in human service settings from 1990 to 2016. Journal of Organizational Behavior Management, DOI: 10.1080/0116008061.2018.1454872 [DOI] [Google Scholar]
  29. Gresham FM, Gansle KA, & Noelle GH (1993). Treatment integrity in applied behavior analysis with children. Journal of Applied Behavior Analysis, 26, 257–263. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Hall LJ, Grundon GS, Pope C, & Romero AB (2010). Training paraprofessionals to use behavioral strategies when educating learners with autism spectrum disorders across environments. Behavioral Interventions, 25, 37–51. DOI: 10.1002/bin.294 [DOI] [Google Scholar]
  31. Hess KL, Morrier MJ, Heflin LJ, Ivey ML (2008). Autism treatment survey: Services received by children with autism spectrum disorders in public school classrooms. Journal of Autism and Developmental Disorders, 38, 961–971 [DOI] [PubMed] [Google Scholar]
  32. Ho FC, Lam CS, Sam K, & Arthur-Kelly M (2018). An exploratory study on collaborative modes of professional development and learning for teachers and students with autism spectrum disorder. Support for Learning, 33, 142–150. [Google Scholar]
  33. Houmanfar RA (2016). Discussions and research: Leadership, cultural change, and beyond. Journal of Organizational Behavior Management, 36, 2–3. DOI: 10.1080/0608061.2016.1203711 [DOI] [Google Scholar]
  34. Houmanfar RA, & Mattaini MA (2016). Leadership and cultural change: Implications for behavior analysis. Behavior Analyst, 39, 41–46. DOI: 10.1007/s40614-016-0064-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Hustus CL, & Owens JS (2018). Assessing readiness for change among school professionals and its relationship with adoption and reported implementation of mental health initiatives. Child and Youth Care Forum, 47, 829–844. [Google Scholar]
  36. Kratz HE, Stahmer A, Sie M, Marcus SC, Pellecchia M, & Locke J (2019). The effect of implementation climate on program fidelity and student outcomes in autism support classrooms. Journal of Consulting and Clinical Psychology, 87, 270–281. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Li S, Feffs L, Barwick M, & Stevens B (2018). Organizational contextual features that influence the implementation of evidence-based practices across healthcare settings: A systematic integrative review. BMC Systematic Review, 7:72. doi: 10.1186/s13643-081-0734-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Locke J, Beidas RS, Marcus S, Stahmer A, Aarons GA, Lyon AR, … & Mandell DS (2016). A mixed method sudy of individual and organizational facors that affect implementation of interventions for children with autism in public schools. Implementation Science 11:135 (Online publication). [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Lovaas IO, Schreibman L& Koegel RL (1974). A behavior modification approach to the treatment of autistic children. Journal of Autism and Childhood Schizophrenia, 4, 111–129. [DOI] [PubMed] [Google Scholar]
  40. Ludwig TD (2017). Process safety behavioral systems: Behaviorsv interlock in complex metacontingencies. Journal of Organizational Behavior Management, 37, 224–239. doi: 10.1080/01608061.2017.1340921 [DOI] [Google Scholar]
  41. Lyon AR, Cook CR, Brown EC, Locke J, Davis C, Ehrhart M, & Aarons GA (2018). Assessing organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science, 135. doi: 10.1186/s13012-017-0705-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Mandell DS, Barry CL, Marcus SC, Xie M, Shea K, Mullan K, & Epstein AJ (2016). Effects of autism spectrum disorder insurance mandates on the treated prevalence of autism spectrum disorder. JAMA Pediatrics 170, 887–893. doi: 10.1001/jamapediatrics.2016.1049 [DOI] [PubMed] [Google Scholar]
  43. McClannahan LE, & Krantz PJ (2006). Behavior analysis intervention for school-age children at the Princeton Child Development Institute. In Handleman J & Harris S (Eds.), School-age education for children with autism (pp. 143–162). Austin, TX: Pro-Ed. [Google Scholar]
  44. Metz A, Bartley L, Ball H, Wilson D, Naoom SF, & Redmond P (2015). Active implementation frameworks for successful service delivery: Catawba County child wellbeing project. Research on Social Work Practice, 25, 415–422. doi: 10.1177/1049731514543667. [DOI] [Google Scholar]
  45. Morris ZS, Wooding S, & Grant J (2011). The answer is 17 years, what is the question: understanding time lags in translational research. Journal of the Royal Society of Medicine, 104, 510–20. doi: 10.1258/jrsm.2011.110180 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Moulins JC, Dickson KS, Stadnick NA, Rabin B, & Aarons GA (2019). Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implementation Science, 14:1 (Online publication). [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Myers DC, Durlak JA, & Wandersman A (2012). The quality implementation framework: A synthesis of critical steps in the implementation process. American Journal of Community Psychology, 50, 462–480. doi: 10.1007/s10464-012-9522-x [DOI] [PubMed] [Google Scholar]
  48. National Implementation Science Network. (n.d.). Active implementation framework: Stages of implementation, Lesson 7 Chapel Hill, NC: Frank Porter Graham Child Development Institute, University of North Carolina; Retrieved from https://implementation.fpg.unc.edu/modules-and-lessons [Google Scholar]
  49. National Implementation Science Network. (2015). Implementation drivers: Assessing best practice Chapel Hill, NC: Frank Porter Graham Child Development Institute, University of North Carolina; Retrieved from https://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/NIRN-ImplementationDriversAssessingBestPractices.pdf [Google Scholar]
  50. Odom SL, Cox A, & Brock M (2013). Implementation science, professional development, and Autism Spectrum Disorders: National Professional Development Center on ASD. Exceptional Children, 79, 233–251. [Google Scholar]
  51. Odom SL, Duda M, Kucharczyk S, Cox A, & Stabel A (2014). Applying an implementation science framework for adoption of a comprehensive program for high school students with autism spectrum disorder, Remedial and Special Education, 35, 123–132. [Google Scholar]
  52. O’Donnell CL (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78, 33–84. [Google Scholar]
  53. Peters-Scheffer N, Didden R , Korzilius H & Sturmey P (2011). A meta-analytic study on the effectiveness of comprehensive ABA-based early intervention programs for children with Autism Spectrum Disorders, Research in Autism Spectrum Disorder, 5, 60–69. 10.1016/j.rasd.2010.03.011. [DOI] [Google Scholar]
  54. Proctor EK, Landsverk J, Aarons G, Chambers DA, & Mittman BS (2008). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36, 24–34. doi: 10.1007/s10488-010-0319-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Rodriguez MA, Bell J, Brown M, & Carter D (2017). Integrating behavioral science with human factors to address process safety. Journal of Organizational Behavior Management, 37, 301–315. DOI: 10.1080/0608061.2017.1340924 [DOI] [Google Scholar]
  56. Roll-Pettersson L, Olsson I, & Ala’I-Rosales S (2016). Bridging the research to practice gap: A case study approach to understanding EIBI supports and barriers in Swedish preschools. International Electronic Journal of Elementary Education, 9, 317–336. [Google Scholar]
  57. Roux Anne M., Rast Jessica E., Anderson Kristy A., and Shattuck Paul T. National Autism Indicators Report: Developmental Disability Services and Outcomes in Adulthood Philadelphia, PA: Life Course Outcomes Program, A.J. Drexel Autism Institute, Drexel University, 2017. [Google Scholar]
  58. Ruble LA, McGrew JH, Toland M, Dalrymple N, Adams M, & Snell-Rood C (2019). Randomized control trial of COMPASS for improving transition outcomes of students with autism spectrum disorder. Journal of Autism and Developmental Disorders, 48, 3586–3595. [DOI] [PubMed] [Google Scholar]
  59. Skinner BF (1957). Verbal behavior New York: Appleton-Century Crofts. [Google Scholar]
  60. Smith T, Parker T, Taubman M, & Lovaas OI (1992). Transfer of staff training from workshops to group homes: A failure to generalize across settings. Research in Developmental Disabilities, 13, 57–71. doi: 10.1016/0891-4222(92)90040-D [DOI] [PubMed] [Google Scholar]
  61. Stahmer AC, & Aarons GA (2009). Attitudes toward adoption of evidence-based practices: A comparison of autism early intervention providers and children’s mental health providers. Psychological Services, 6, 223–234. 10.1037/a0010738. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Stahmer AC, Rieth S, Lee E, Reisinger EM, Mandell DS, & Connell JE (2015). Training teachers to use evidence-based practices for autism: Examining procedural implementation fidelity. Psychology in the Schools, 52, 181–189. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Stahmer AC, Suhrheinrich J, Reed S, & Schreibman L (2012). What works for you? Using teacher feedback to inform adaptations of an evidence-based practice for classroom use. Autism Research and Treatment doi: 10.1155/2012/709861 [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Stahmer AC, Suhrheinrich J, Schetter PL, & Hassrick EM (2018). Exploring multi-level system factors facilitating educator training and implementation of evidence-based practices (EBP): A study protocol. Implementation Science, 13:3 (Online publication) [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Stein JT (1975). Some ethical considerations of short-term workshops in the principles and methods of behavior modification. Journal of Applied Behavior Analysis, 8, 113–115. doi: 10.1901/jaba.1975.8-113 [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Steinbrenner J, Odom SL, Hall LJ, & Hume KA (in press). Assessing implementation of a comprehensive treatment program for high school students with autism spectrum disorder. Exceptional Children [Google Scholar]
  67. Suhrheinrich J, Rieth SR, Dickson KS, Stahmer AC (in press). Exploring inner-context factors associated with implementation outcomes in a randomized trial of classroom pivotal response teaching. Exceptional Children [Google Scholar]
  68. Tabak RG, Khoong EC, Chambers DA, & Brownson RC (2012). Bridging research and practice. American Journal of Preventive Medicine, 43, 337–350. doi: 10.1016/j.amepre.2012.05.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Tourinho E, Z., & Vichi C (2012). Behavioral-analytic research of cultural selection and the complexity of cultural phenonmena. Revista Latinoamericana de Psicologia, 44, 169–179. [Google Scholar]
  70. VanStelle SE, Vicars SM, Harr V, Miguel C, Koerber JL, Kazbour R, & Austin J (2012). The publication history of the Journal of Organizational Behavior Management: An objective review and analysis: 1998–2009. The Journal of Oranizational Behavior Management, 32, 93–123. doi: 10.1080/01608061.2012.675864 [DOI] [Google Scholar]
  71. Weiner BJ, Amick H, & Lee SY (2008). Conceptualization and measurement of organizational readiness for change: a review of the literature in health services research and other fields. Medical Care Research Review, 65, 379–436. doi: 10.1177/1077558708317802 [DOI] [PubMed] [Google Scholar]
  72. Williams NJ, & Beidas R (2018). The state of implementation science in child psychology and psychiatry: A review and suggestions to advance the field. Journal of Child Psychology and Psychiatry Advance online publication. doi: 10.1111/jcpp.12960 [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Williams NJ, Frank HE, Frederick L, Beidas RS, Mandell DS, Aarons GA, Green P, & Locke J (2019). Organizational culture and climate profiles: Relationships with fidelity to three evidence-based practices for autism in elementary schools. Implementation Science, 14–15. (Online publication). [DOI] [PMC free article] [PubMed] [Google Scholar]
  74. Wong C, Odom SL, Hume KA, Cox AW, Fettig A, Kucharczyk S…. & Schultz TR (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorders: A comprehensive review. Journal of Autism and Developmental Disorders, 49, 1951–1966. doi: 10.1007/s10803-014-2351-z [DOI] [PubMed] [Google Scholar]
  75. Yoon KS, Duncan T, Lee SW-Y, Scarloss B, & Shapley K (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, REL 2007–No. 033) Washington, DC: U.S. Department of Education; Retrieved from http://ies.ed.gov/ncee/edlabs [Google Scholar]

RESOURCES