Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jul 1.
Published in final edited form as: Autism. 2019 Dec 23;24(5):1152–1163. doi: 10.1177/1362361319895923

Individual and Organizational Characteristics Predicting Intervention Use for Children with Autism in Schools

Jill Locke 1, Christina Kang-Yi 2, Lindsay Frederick 1, David S Mandell 2
PMCID: PMC7308214  NIHMSID: NIHMS1544941  PMID: 31867987

Abstract

Several interventions have demonstrated efficacy in improving social outcomes for children with autism, but they often are not used in schools. This study examined individual and organizational factors associated with the use of a research-informed social engagement intervention, Remaking Recess, for children with autism in elementary schools. Twenty-eight school personnel from 12 schools in five districts in the northeastern USA participated. Schools were randomized to: 1) training in Remaking Recess only; or 2) training in Remaking Recess with implementation support. School personnel rated their attitudes about evidence-based practices, organizational readiness, and fidelity. Independent observers rated school personnel’s fidelity at baseline (pre-intervention training) and exit (post-intervention training). The results suggest that self-rated fidelity was lower when staff perceived the use of Remaking Recess was required; however, observer-rated fidelity was lower when staff rated Remaking Recess as appealing. In addition, self-rated fidelity was higher when there was a sufficient number of staff, positive individual growth and organizational adaptability. The results also indicated higher observer-rated fidelity when staff perceived positive influence over their coworkers. The results suggest that both individual (attitudes) and organizational (influence, staffing, growth, adaptability) characteristics may affect implementation success in schools. A collaborative decision-making approach for EBP use is recommended.

Keywords: attitudes, organizational readiness, autism spectrum disorder, social engagement intervention, implementation

Evidence-based practices (EBPs) for children with autism often have not been successfully adopted, implemented, or sustained in public schools (Brookman-Frazee et al., 2010; Lord et al., 2005; Stark et al., 2011). In some cases, when EBPs are used, only 25–50% are implemented with fidelity (as the intervention developers intended; Cook & Odom, 2013; Gottfredson & Gottfredson, 2002). These findings suggest that EBPs may be unlikely to yield improvements in student outcomes (Durlak & Dupre, 2008). Low EBP fidelity may be due to barriers to implementation that exist at the individual-(e.g. skill, attitudes, and beliefs) and organizational-levels (e.g. implementation context, organizational policies, and procedures) that may impede EBP use (Fixsen et al., 2005; Forman et al., 2013; Majid et al., 2011; Stahmer & Aarons, 2009).

EBP implementation in schools is fraught with challenges including the constraints of educational timelines, limited resources, lack of training and organizational constraints such as physical space (Forman et al., 2013; Locke et al., 2015; Owens et al., 2014; Kucharczyk et al., 2015; Mandell et al., 2013). Because of these challenges, there is a growing interest in identifying malleable individual and organizational factors associated with successful EBP implementation in schools (Locke et al., 2016). Individual provider and organizational characteristics have been shown to be critical in the successful implementation of EBPs in non-education service sectors, such as mental health settings (Aarons et al., 2012; Beidas et al., 2013; Beidas et al., 2014; Beidas et al., 2016; Bonham et al., 2014). Of these characteristics, both individual attitudes and organizational readiness have been identified as significant predictors of EBP use (Alanen et al., 2009; Attieh et al., 2014; Stahmer & Aarons, 2009; van Sonsbeek et al., 2015).

Individual attitudes are one of the key determinants of behavior (Lavin & Groarke, 2005). Attitudes comprise multiple dimensions including intuitive appeal of a provider toward an EBP, likelihood of adopting an EBP given organizational requirements to do so, openness to new practices, and divergence between usual and new practices. The more positive an individual’s attitude about a behavior, the greater likelihood that individual will act on the behavior (Lavin & Groarke, 2005). Attitudes predict implementation of EBPs in both physical and mental healthcare settings (Arnadottir & Gudjonsdottir, 2016; Lavin & Groarke, 2005; Stahmer & Aarons, 2009; van Sonsbeek et al., 2015), and may alter how practitioners decide to implement EBPs (Stahmer & Aarons, 2009; van Sonsbeek et al., 2015). There is some evidence that several dimensions of attitudes (intuitive appeal of EBPs, effects of organizational policy and requirements on the likelihood of using EBPs, openness to adopting new EBPs) are positively associated with EBP use for students with autism (Stahmer & Aarons, 2009). Attitudes about EBPs may be malleable and shaped to improve implementation of EBPs in schools (Cook et al., 2015). Previous research has found that providing intervention training opportunities to teachers can enhance their attitudes and effectiveness in working with students with autism (Leblanc, Richardson, & Burns, 2009). Cook and colleagues’ (2015) recently developed a pre-implementation intervention to address beliefs and attitudes about EBPs and found more favorable attitudes towards EBPs after the intervention and greater fidelity.

Organizational readiness, or motivation for change at the organizational level, also may affect implementation success (Lehman, Greener, & Simpson, 2002). Simpson (2002) posits a process model for program change whereby a new innovation, in our case, Remaking Recess (discussed below), is introduced into routine use. Simpson (2002) argues that the transfer process, or adoption of an EBP, may be influenced by organizational attributes such as organizational and personal readiness (e.g., motivation and resources). Organizational readiness may be important for EBP use in different ways. For example, if staff within the organization lack motivation to use EBPs, implementation is unlikely to be initiated. Or, given inadequate institutional resources, it may not be possible to use EBPs. Likewise, Lehman et al., (2002) suggest that if staff in an organization do not have the attributes necessary for change to occur, such as adaptability and growth-orientation, then change is unlikely to emerge. Organizational readiness can appear at an individual, group, unit, department, or organizational level (i.e. teachers, school administration, and district officials) (Weiner, 2009).

Individual staff attributes have been used in Lehman, Greener, and Simpson’s (2002) organizational readiness scale (ORC) to assess organizational function and readiness for change. Researchers identified four key parts of individual staff attributes: (1) growth, where the individual feels that they have, or do not have, the opportunity for professional growth and training (i.e. trainings or professional development); (2) efficacy, where staff’s confidence in their own skills may impact likelihood of adopting change; (3) influence, the feeling that one has influence over their coworkers and that their thoughts and ideas are respected; and (4) adaptability, or the staff’s ability to be open to new ideas and change (Lehman, Greener, & Simpson, 2002). The literature suggests that when organizational readiness for change is higher, individual members within that organization are more likely to initiate change, put forth more effort supporting change, and overcome or resist obstacles to change (Weiner, 2009).

Other key aspects of organizational readiness include: (1) program needs – specific materials that the program requires for implementation; (2) training needs – training in the EBP or intervention; (3) pressures for change – organizational support and pressure to change current practice; (4) space – physical space to carry out the EBP or intervention; (5) staffing – sufficient staffing to deliver the intervention; and (6) resources – financial resources to support the use of the intervention (Lehman et al., 2002). Previous research on organizational readiness in schools by Goh, Cousins, and Elliott (2006) found that school staff collaborated well but felt that bureaucracy impeded school personnel’s ability for change. Another study found that administrative acceptability and support served as a primary indicator of successful intervention implementation in schools (McDougal, Moody Clonan, & Martens, 2000). These studies highlight that school culture may not only impact readiness for change (Goh et al., 2006), but it also may affect implementation of school-based interventions (McDougal, Moody Clonan, & Martens, 2000).

Although these constructs have been explored in other settings (e.g., mental health settings), they have not been explicitly studied within the context of EBPs for children with autism. These factors are important to examine because autism EBPs are so critical, yet often poorly implemented in schools (Locke et al., 2015; Kretzmann, Shih, & Kasari, 2015; Mandell et al., 2013; Pellecchia et al., 2015; Stahmer et al., 2015; Suhrheinrich et al., 2013). If individual attitudes and organizational readiness are associated with EBP use for children with autism, we may begin to understand the active mechanisms (e.g., attitudes, organizational readiness) that catalyze successful implementation in schools. We can then use targeted implementation strategies to address individual attitudes and organizational readiness to ultimately improve the quality of services provided to children with autism in schools.

Autism EBPs that address social functioning are critically important as they target one of the core symptoms of the disorder (Kasari et al., 2012; Kretzmann et al., 2015), yet few of these interventions have been tested in the school environment. The two most common approaches that have the strongest evidence to improve socialization for children with autism in schools are: 1) adult-facilitated, where an adult creates opportunities for children with autism to engage with their peers (i.e. structured play groups); and 2) peer-mediated models, where a group of typically developing children are trained to model social skills (i.e. modeling) (Harper, Symon, & Frea, 2008; Kasari et al., 2012; Laushey & Heflin, 2000; Rogers, 2000). One school-based intervention, Remaking Recess, combines both adult-facilitated and peer-mediated intervention strategies and has shown significant results in improving children’s social outcomes in well-resourced public schools (Kretzmann et al., 2015; Locke et al., 2018a; Locke et al., 2018b).

Remaking Recess is a school-based social engagement intervention for children with autism that uses research-informed intervention strategies such as role-play, behavioral rehearsal, and peer models (Kretzmann et al., 2012). The objective of Remaking Recess is to train school personnel in how to facilitate and promote peer engagement for children with autism on the playground. The intervention was designed to transfer skills from the research team to school personnel, so that school personnel will be equipped to continue to support children with autism after external support is withdrawn. Because Remaking Recess occurs on the school playground, its implementation requires at least one adult to be present and available to engage in intervention activities.

Remaking Recess includes the following components: 1) scan and circulate the cafeteria/playground for children who may need additional support to engage with their peers; 2) identify children’s engagement states with peers; 3) follow children’s lead, strengths, and interests (engaging in activities that are motivating and of interest to the student with autism); 4) provide developmentally and age appropriate activities and games to scaffold children’s engagement with peers; 5) support children’s social communicative behaviors (i.e., initiations and responses) and conversations with peers; 6) create opportunities to facilitate reciprocal social interaction; 7) sustain children’s engagement within an activity or game; 8) coach children through difficult situations with peers should they arise; 9) provide direct instruction on specific social engagement skills; 10) individualize the intervention to specific children in order to generalize the intervention to other students in their care; 11) work with typically developing peers to engage children with autism; and 12) fade out of an activity/game so children learn independence (remakingrecess.org).

Remaking Recess has been tested in several pilot development studies and randomized controlled trials that indicate effectiveness (e.g., improvements in playground peer engagement and social network inclusion when implemented by school personnel) in public elementary schools (Kretzmann et al., 2015; Locke et al., 2018a; Locke et al., 2018b; Shih et al., in press). Recently, Locke and colleagues (2018b) used a randomized controlled trial that compared the effects of training school personnel in Remaking Recess with and without school-level implementation support for elementary-aged children with autism in public schools. Although school personnel had low fidelity to Remaking Recess in both conditions, children with autism showed significant reductions in solitary engagement and increased joint engagement with peers, further replicating the results from the original trial (Kretzmann et al., 2015). However, implementation support provision was associated with greater improvements in social network inclusion and friendship nominations than training in Remaking Recess alone (Locke et al., 2018b). These results suggest that school-level implementation supports may have a positive effect on outcomes above and beyond training in Remaking Recess alone, and that further research is needed into the active mechanisms (individual- and organizational- factors) that facilitate successful implementation. The purpose of this paper was to address the following research questions:

  1. Are individual attitudes about EBPs associated with the use of Remaking Recess?

  2. Is organizational readiness associated with the use of Remaking Recess?

We predict that both individual attitudes and organizational readiness will affect implementation of a social engagement intervention, Remaking Recess, for children with autism spectrum disorder in schools.

Methods

Participants

Participants included school personnel of elementary-aged children with autism recruited from 12 public elementary schools in the Northeastern United States. The university institutional review board and each school district approved the study. School personnel included 11 teachers and 17 other staff (classroom assistants, one-to-one assistants and noontime aides) that participated in a randomized controlled trial of Remaking Recess (see Locke et al., 2018b). School personnel were predominantly female (85.7%, n = 24), averaged 39.5 years of age (SD = 13.0), and had a wide range of experience working with children with autism (0–21 years; M = 6.2, SD = 5.7). Participants were 60.7% (n = 17) white, 35.7% (n = 10) were African American, and 3.6% (n = 1) were Latino. Their highest educational attainment was as follows: 28.6% (n = 8) had a graduate degree, 42.9% (n = 12) had a bachelor’s degree, 7.1% (n = 2) had an associate degree, and 21.4% (n = 6) had a high school degree.

Measures

Fidelity.

Fidelity, or the use and quality of intervention delivery or intensity, was measured in two ways: 1) observer-rated; and 2) self-rated. Independent observers (blinded to study randomization) rated fidelity on seven components of Remaking Recess (i.e., attended to child engagement on the playground, transitioned child to an activity, facilitated activity, participated in activity, fostered communication, employed peer models, and provided direct instruction of social skills) during a randomly selected lunch recess period. Use of Remaking Recess was scored “0” for “no” and “1” for “yes” to determine whether school personnel used each component of Remaking Recess. Seven is the total possible score for use of Remaking Recess components on the observer-rated fidelity checklist. The percentage of completed steps (completed steps/total number of Remaking Recess components) was averaged and used for analyses. Quality of intervention delivery or how well school personnel used each component of the intervention also was coded on a Likert scale from “1” (not well) to “5” (very well) for each component of Remaking Recess that was used. An average use by quality rating for each component was used for analysis. All components were included in the quality rating; however, if the Remaking Recess component was not used, the score was 0. Reliability was collected on 20% of sessions during the study (mean percent agreement = 85; range 80–90%). Participants were informed that a member of the research team would conduct observations on the playground; however, they were not explicitly informed when data collection would occur or that data were being collected on their use of Remaking Recess at baseline (pre-intervention) or exit (post-intervention). This was a conscious decision to capture whether implementation naturally occurs during a random recess period without an unintentional prompt by the research team.

School personnel used a separate, but parallel fidelity measure to self-rate their use and quality of intervention delivery at each time point. Frequency of intervention delivery or how often school personnel reported using each component of the intervention was measured and coded on a Likert scale from “1” (once in the past week) to “4” (everyday) for each component of Remaking Recess that was used. The frequency of use for each Remaking Recess component was averaged and used for analyses. Both observer- and self-rated fidelity were collected at baseline (before intervention activities commenced) – to determine if any Remaking Recess component was implemented prior to receipt of training- and exit (post-intervention training). School personnel completed the fidelity checklist on their own.

Attitudes about EBPs.

Attitudes about the use of Remaking Recess were measured using the Evidence-Based Practice Attitude Scale (EBPAS; Aarons, 2004). The referent was changed on the EBPAS from “evidence-based practices” to the intervention of interest in this study, Remaking Recess. The EBPAS is a 15-item measure that assesses four general attitudes toward adoption of EBPs: appeal, requirements, openness, and divergence on a scale from “0” (not at all) to “4” (to a very great extent). These subscales have been shown to be critical in provider attitudes towards adoption of EBPs (Aarons, 2004). The appeal subscale (4 items; possible maximum score = 16) is the intuitive appeal of a provider toward an EBP and comprises items such as, “it ‘made sense’ to you” and “you felt you had enough training to use it correctly” (Aarons, 2004). Secondly, the requirements subscale (3 items; possible maximum score = 12) measures the likelihood of adopting the EBP given organizational requirements to do so and contains items such as, “it was required by your supervisor” and “it was required by your state” (Aarons, 2004). Thirdly, the openness subscale (4 items; possible maximum score = 16) measures provider openness to new practices comprising items such as, “I like to use new types of therapy/interventions to help my students” (Aarons, 2004; Cook et al., 2019). Lastly, the divergence subscale (4 items; possible maximum score = 16) represents perceived divergence between usual and new practices and consists of items such as, “I know better than academic researchers how to care for my students” (Aarons, 2004; Cook et al., 2019). Studies suggest moderate to good internal consistency for the EBPAS total score (Cronbach’s α = 0.76, 77, 0.79) and subscale reliabilities ranged from 0.67 to 0.91 (Aarons, 2004; Aarons et al., 2010; Aarons, McDonald, Sheehan, & Walrath-Greene, 2007). The EBPAS has been previously used with early intervention autism providers (Stahmer & Aarons, 2009) and autism support teachers (Locke et al., 2016). The EBPAS was collected at exit (post-intervention) once all participants were trained in Remaking Recess.

Organizational Readiness.

Organizational readiness was assessed using the Texas Christian University Organizational Readiness for Change (TCU-ORC) scale adapted for the school context (Lehman et al., 2002). The TCU-ORC comprises 115 Likert-type items (scored on 5-point agree-disagree response scales, where “1” = disagree strongly and “5” = agree strongly) to represent 18 content domains grouped into four areas: motivation for change (program needs [10 items; possible maximum score = 50], training needs [10 items; possible maximum score = 50], pressures for change [9 items; possible maximum score = 45]), institutional resources of the program (space [9 items; possible maximum score = 45], staffing [9 items; possible maximum score = 45], training [9 items; possible maximum score = 45], resources, [9 items; possible maximum score = 45]), personality attributes of the staff (individual professional growth [10 items; possible maximum score = 50], efficacy [10 items; possible maximum score = 50], influence [9 items; possible maximum score = 45], adaptability, [10 items; possible maximum score = 50]), and organizational climate of the program (clarity of mission, cohesion, autonomy, communication, stress, and openness to change; Lehman et al., 2002). Only the domains under the following three areas: motivation for change, institutional resources of the program, and personality attributes of the staff were used in this study; the organizational climate of the program domain was not used. The organizational climate variables were not used because we did not have the minimum number of raters necessary for data aggregation (n=3) in each school setting. Individual ratings may not represent the entire school’s organizational climate; therefore, they were excluded from analyses. The use of individual scores for the former domains is appropriate when the individual perception is the referent of interest (Marsh et al., 2012). All items were maintained with minor changes made to item wording, such as replacing the word “patient” with “child,” and “clinician” with “school personnel” to preserve the integrity of the original items and constructs. The TCU-ORC was collected at baseline (pre-intervention).

Procedure

The research team met with school district officials to obtain a list of eligible schools with children with autism who were included with their typically developing peers for 80% or more of the school day. All children with autism needed a designated staff member available during the recess period for intervention implementation. The research team then met with the principal at each prospective school to discuss the research activities and obtain a letter of agreement to conduct research on their campus. All recruitment materials (e.g., informational handouts, flyers) were distributed to the school, and the research team met with all interested participants to inform them about the study and their role as a study participant, so they were able to make an informed decision regarding their participation. Once informed consent from school personnel and families of children with autism was obtained, school personnel completed all study measures (EBPAS and TCU-ORC) at baseline (before intervention activities commenced) and exit (post-intervention training) as well as all research activities described below. School personnel received $50 for their participation.

Schools were randomly assigned to: 1) training in Remaking Recess, a school-based social engagement intervention for children with autism (n = 6) (Kretzmann et al., 2012); or 2) training in Remaking Recess with implementation support (n = 6). Participating school personnel from all enrolled schools were trained in Remaking Recess to support children with autism during recess (Kretzmann et al., 2015; Locke et al., 2015). A MA- or PhD-level coach trained school personnel in Remaking Recess across 12 sessions over six weeks (two sessions per week) with children with autism and in the presence of typically developing peers. Each session took place during the child’s lunch recess period (approximately 30–45 minutes) and targeted one didactic skill (listed above). Coaches explained the didactic skill and its application to children with autism and their peers. Coaches then modeled how to use the didactic skill with children with autism and their peers on the playground. Subsequently, school personnel were asked to try the skill in the presence of the coach for immediate feedback. At the end of each session, school personnel were assigned “homework” to practice the skill during the days when coaches were not present. Homework was reviewed at the next session.

Once trained, school personnel were expected to implement Remaking Recess each time they were on the playground with children with autism for the remainder of the school year and beyond. For some school personnel (e.g., one-to-one aides), Remaking Recess implementation occurred daily; for other school personnel (e.g., teachers), Remaking Recess implementation only occurred during assigned recess duty days, on average two days per week. The number of school personnel responsible for Remaking Recess implementation ranged from 1–3 each day of planned intervention.

Schools randomized to training in Remaking Recess with implementation support received three additional consultation and implementation support sessions with school administrators over the six weeks. Implementation support consisted of discussion of how to address common barriers to implementation including how to: 1) schedule staffing during recess; 2) build internal capacity (e.g., a team of trained Remaking Recess implementers); 3) amend school-wide recess policies (i.e., detention, the removal of recess, and alternatives for indoor recess during inclement weather); 4) provide tangible support and resources (i.e., materials and space); 5) improve implementation climate (e.g., recognizing, appreciating, and rewarding implementer efforts, being visible on the playground/in the cafeteria, etc.); 6) adapt and modify the intervention to fit the needs of the school (e.g., identifying the core versus peripheral components); and 7) embed Remaking Recess within the school culture. See Locke et al., (2018b) for more details. School champions (e.g., school administrators, counselors, psychologists, teachers, and support staff who were designated by the principals) were presented with this list of potential implementation supports and selected one to three topics that would aid in supporting Remaking Recess implementation. See Locke et al., (2018b) for the list of frequently selected implementation support topics. Once topics were selected, a MA- or PhD-level coach supported schools teams to develop implementation plans to support successful Remaking Recess implementation. For example, schools that were interested in embedding Remaking Recess within the school culture focused on raising awareness among the faculty, staff and students that Remaking Recess is an important new program that addresses socialization in children with autism and their peers.

Data Analysis

Study data were managed using Research Electronic Data Capture (REDCap), a secure, web-based application designed to support data capture for research studies (Harris et al., 2009). Descriptive analyses for school personnel demographic characteristics were conducted for baseline comparisons between Remaking Recess only and Remaking Recess with implementation support conditions to assess the success of randomization. The two groups were not statistically significantly different in age, race/ethnicity, roles in school, or years of work experience with children with autism. Both the Remaking Recess and Remaking Recess with implementation support conditions were combined as there were no significant differences between the groups (see Locke et al., 2018b). First, generalized linear regression analyses were conducted to examine the associations between: (1) EBPAS subscales (appeal, requirements, openness, and divergence scores) and self-rated number of Remaking Recess steps completed and frequency of intervention delivery; and (2) staff’s perception of organizational readiness for the implementation of Remaking Recess and self-rated Remaking Recess fidelity.

Results

The mean score of observer-rated quality of Remaking Recess fidelity (Mean = 0.6, SD = 1.17) was lower than the mean score of self-rated fidelity, 0.8 (SD = 1.32). However, the mean number of completed Remaking Recess steps rated by observer (Mean = 1.1, SD = 1.80) was higher than the self-rated mean number of steps completed (Mean = 0.1, SD = 0.18). Both observer- and self-rated quality of Remaking Recess use was poor and the number of Remaking Recess components used was low.

Individual Attitudes

Higher scores on the EBPAS Appeal subscale were associated with lower observer- rated quality of Remaking Recess fidelity (F(1,26) = 9.08, p = 0.01). All other EBPAS subscales were not significantly associated with observer-rated quality of Remaking Recess fidelity. Higher scores on the EBPAS Requirements subscale were associated with lower self-rated use of Remaking Recess (F(1,26) = 5.08, p = 0.03). All other EBPAS subscales were not significantly associated with self-reported Remaking Recess fidelity.

Organizational Climate and Readiness for Change

Staff-rated influence, the feeling that school personnel have influence over their coworkers or that their thoughts and ideas are respected was positively associated with observer-rated number of Remaking Recess steps completed (F(1,26) = 4.97, p = 0.03). None of the other organizational readiness for change scales were significantly associated with the observer-rated quality of Remaking Recess fidelity. See Table 1 for mean scores of Remaking Recess fidelity, individual attitudes towards evidence-based practices, and organizational readiness for change.

Table 1.

Mean scale scores of Remaking Recess fidelity, individual attitudes toward evidence-based practices, and organizational readiness for change

Mean SD Minimum Maximum
Fidelity
 Observer-rated quality of fidelity 0.6 1.17 0.0 5.0
 Observer-rated completion of steps 1.1 1.80 0.0 7.0
 Self-rated frequency of use 0.8 1.32 0.0 4.0
 Self-rated completion of steps 0.1 0.18 0.0 0.6
Evidence-based Practices Attitudes Scale (EBPAS)
 EBPAS Requirements Subscalea 8.5 4.70 0.0 12.0
 EBPAS Appeal Subscale 13.3 2.85 7.0 16.0
 EBPAS Openness Subscale 14.4 1.71 10.0 16.0
 EBPAS Divergence Subscale 4.5 2.57 0.0 9.0
Staff-rated organizational readiness for change (TCU-ORC)
Motivation for Change
 Program needs 30.5 9.56 10.0 45.6
 Training needs 34.2 10.54 10.0 50.0
 Pressures for change 29.4 7.88 10.0 42.9
Institutional Resources of the Program
 Space 31.2 7.76 13.3 43.3
 Staffinga 31.0 7.21 15.0 43.3
 Training 34.3 5.81 20.0 42.5
 Resources 32.2 5.88 18.6 42.5
Personality Attributes of the Staff
 Individual professional growtha 35.4 5.04 26.0 46.0
 Efficacy 38.5 4.43 28.0 48.0
 Influence 32.6 6.11 20.0 43.3
 Adaptabilityb 38.2 4.85 30.0 50.0
Open in a new tab

Note. EBPAS scaling 0 = not at all, 1 = to a slight extent, 2 = to a moderate extent, 3 = to a great extent, 4 = to a very great extent. TCU-ORC scaling 1 = disagree strongly, 2 = disagree, 3 = uncertain, 4 = agree, and 5 = agree strongly.

a.

Negatively associated with self-reported fidelity at p < .05.

b.

Positively associated with staff’s perception of implementation intensity at p < .01 and completeness of Remaking Recess steps at p < .03.

The presence and quality of staff (F(1,26) = 5.01, p < 0.03) and the potential for individual professional growth (F(1,10) = 5.31, p < 0.03) were positively associated with self-rated fidelity. Staff-rated organizational adaptability for change also was positively associated with the staff’s perception of the implementation intensity (F(1,26) = 7.72, p < 0.01) and number of Remaking Recess steps completed (F(1,26) = 5.09, p < 0.03). None of the other organizational readiness for change scales were significantly associated with Remaking Recess fidelity.

Discussion

Our results suggest that school personnel’s perceptions and individual attitudes about EBPs are associated with observer- and self-ratings of fidelity. Namely, when school personnel perceived the use of Remaking Recess as appealing, observers rated fidelity lower, and when school personnel perceived their use of Remaking Recess as required, self-rated fidelity also was lower. In addition, the results indicated that some aspects of organizational readiness may be important for implementation use, in particular, staff influence, staffing, the potential for professional growth within the organization, and organizational adaptability.

It is important to first acknowledge that the use of Remaking Recess was low and the quality of Remaking Recess fidelity was poor across both groups. We note that both observer- and self-rated fidelity were measured in this study. This was intentional to determine whether blinded observers could corroborate self-reported fidelity. Although the literature suggests that observer-fidelity is generally lower than self-report, we found similar ratings in our study. We speculate that these findings may be due to the perceived value of implementing a social engagement intervention in school in that socialization often is not as big of a priority in comparison to other initiatives and programs that focus on academics or behavior (Locke et al., 2015; Locke et al., 2018), so teachers and staff are more comfortable reporting what they actually do. In addition, although the low fidelity ratings observed in this study are consistent with the literature on real-world randomized effectiveness trials with elementary-aged children with autism in under resourced school settings (Locke et al., 2015; Mandell et al., 2013; Pellecchia et al., 2015; Stahmer et al., 2015; Suhrheinrich et al., 2013), we recognize that the infrequent use of Remaking Recess observed in this study is concerning considering that half of the participants also received implementation support. It is likely that low fidelity is due to implementation challenges in real-world conditions, which often have less-than-optimal delivery of EBPs (Southam-Gerow et al., 2010). School personnel may have faced implementation challenges such as bandwidth, time, training, or resources to use such a resource-intensive intervention without more targeted implementation supports. In this study, we tailored the implementation strategy to the needs of the school; however, we did not use blended implementation strategies that target potential mechanisms to facilitate successful implementation that may have been more appropriate. For example, choosing an implementation strategy that focuses on altering beliefs and attitudes prior to full-scale EBP training (pre-implementation) and implementation may have been instrumental in ensuring school personnel were ready to learn Remaking Recess (Cook et al., 2015). We also could have strategically targeted school leadership and used Aarons and colleagues’ (2015) Leadership and Organizational Change for Implementation intervention to improve implementation leadership to support the use of Remaking Recess. Implementation strategies that target both individual and organizational processes may be necessary to improve fidelity of interventions that are complex and resource-intensive, such as Remaking Recess. In light of the low fidelity ratings observed in this study, there were some important associations that warrant discussion and further research.

The results of our study are consistent with previous research in community mental health settings suggesting an association between attitudes about EBPs and fidelity (Beidas et al., 2013). Contrary to our hypotheses, we found that observer-rated quality of Remaking Recess delivery was lower when school personnel perceived the use of Remaking Recess as appealing. This finding is counterintuitive; however, one explanation for this result is the nature of how the observations were conducted. Because fidelity was low and school personnel did not complete many Remaking Recess steps, quality of intervention delivery often could not be scored. In other words, it is not possible to rate the quality of something that did not occur. Further research is needed to better understand these findings.

When school personnel perceived that their use of Remaking Recess was required, self-rated fidelity was lower. This finding is important given the top-down approach that many central offices/school districts and principals use in decision making and school programming for children with autism (Locke et al., 2015). Because EBPs are now increasingly required in public schools by policymakers (Fixsen et al., 2013; Odom et al., 2013), many school districts have mandated the use of EBPs for their students with autism (Stahmer et al., 2015). However, our results suggest that mandates alone may not yield greater use of EBPs. School and district administrators may need to use different approaches to ensure adoption and implementation of Remaking Recess or other EBPs to avoid the perception of mandatory use.

Implementation researchers such as Fixsen et al., (2013) and Aarons et al., (2011) suggest that implementation begin with an exploration phase that allows stakeholders to weigh possible intervention solutions before a team decides which EBP to implement. While this might prove to be a slow process for achieving successful EBP implementation, it may be worth the time, resources, and energy investment if it ultimately results in greater adoption and use. As such, we recommend a collaborative decision-making approach involving multiple stakeholders (central office, school administrators, intervention agents) - including those with autism -to facilitate implementation of autism-related EBPs (Pellicano, Dinsmore, & Charman, 2014; Powell et al., 2015).

It also is possible that these study results are context specific. Remaking Recess is intended to be used on the playground during recess, where many school personnel (e.g., teachers) may not be required to monitor or supervise play-based interventions (Newman, Brody, & Beauchamp, 1996). In the present study, the 11 teachers who served as intervention agents had to relinquish a preparatory period in order to support children with autism during recess. Additional recess duties may not be a feasible model for teachers and may have resulted in less favorable attitudes towards the intervention in this study. Nonetheless, these results highlight the importance of individual attitudes about EBPs. Future research on implementation strategies that address individual attitudes that align use of a specific EBP with usual care in schools is warranted.

In addition to individual attitudes about EBPs, the organizational readiness of school personnel may be an important factor associated with school personnel’s use of Remaking Recess. In this study, only one aspect of organizational readiness (influence) was associated with greater observer-rated fidelity, but three aspects of organizational readiness (staffing, individual growth, and organizational adaptability) were associated with greater self-rated fidelity. These findings may be specific to the Remaking Recess intervention and the accompanying training provided to school personnel. First, we found that greater influence was positively associated with observer-rated number of intervention steps completed. It may be possible that in schools where school personnel have influence over their colleagues or are widely respected, school personnel showcased their Remaking Recess “expertise” by using more steps with children with autism on the playground. This may be a way for school personnel to model the use of Remaking Recess to their colleagues.

Second, we found that sufficient staffing for implementation was positively associated with Remaking Recess fidelity. Remaking Recess requires a dedicated staff person to facilitate opportunities for children with autism to engage with their peers during recess – a period that often is understaffed (Locke et al., 2015). Typically, recess staff monitor for safety (Murray et al., 2013) and may not have the capacity to also deliver an intervention if there are no other staff on the playground.

Third, ratings of individual growth were related to Remaking Recess fidelity. All school personnel received one-on-one consultation and training from an MA- or PhD-level coach – a unique opportunity to learn a new intervention and expand their knowledge base about autism and best practices. Many school personnel often are not trained in what autism is or strategies to support children with autism during the school day (Locke et al., 2015); thus, this training may have been an opportunity for individual growth for participants given the increasing number of students with autism in public schools.

Lastly, we found that organizational adaptability also was associated with Remaking Recess fidelity. School personnel tend to prioritize academics over and above teaching social skills (Locke et al., 2014; Murray et al., 2013). Our training in Remaking Recess challenged school personnel to view recess as an instructional period of the day for children with autism to practice social skills and engage with their peers. School personnel that had the flexibility to see the importance of recess may be more likely to use the intervention. There may be unique elements of Remaking Recess (occurs on the playground where teachers and other school personnel often are not present, requires a dedicated staff member to implement when schools often are understaffed, takes place daily when time is a precious commodity) that inherently make it difficult for implementation in schools (Forman et al., 2009; Locke et al., 2015; Owens et al., 2014).

Providers and researchers ought to carefully consider the time, context, and frequency of use of autism EBPs in schools as many autism EBPs are time and resource intensive. Although these interventions are important and should be delivered in public schools to ensure all children with autism have access to quality evidence-based care, school personnel ought to consider the contextual appropriateness and feasibility of EBP implementation and determine ways to work around barriers to implementation. Thus, it is critical to ensure the school context is adaptable to using supports for children with autism. Furthermore, because organizational readiness may play a pivotal role in promoting EBP use, it is important to understand how school leadership may ensure school personnel are ready for implementation prior to the investment of resources, time, and energy. Additional research in this area is needed.

Limitations

Several limitations are noted. First, the small sample size may affect the generalizability of the findings and preclude analyses that detect potential mediation or moderation effects. Mediation or moderation models may point to the relationships between individual- and organizational-level constructs and unpack the complexities of implementation of a recess-based intervention in schools. Second, this study examined associations between school personnel rated measures of attitudes about EBPs and organizational readiness and fidelity of a social engagement intervention for children with autism. Although there was a time lag in measurement, the use of the same rater (school personnel) for both independent and dependent variables may be biased (Podsakoff, MacKenzie, & Podsakoff, 2003). Third, due to the resource-intense nature of conducting research in schools, fidelity ratings were only completed once at each time point (baseline and exit). Continuous ratings of fidelity may allow for more nuanced and subtle intervention implementation changes. Also, while observers were blind to intervention randomization, they were not blind to the timing of data collection (i.e., pre- or post-intervention). In addition, observer-rated fidelity was based on live coding and while we believe that participants could not see the independent raters taking data on their use of Remaking Recess due to playground size, observers’ distance from facilitation, etc., we cannot be certain that none of the participations saw the raters. Fourth, the TCU-ORC was not developed for use in schools; therefore, it may not fully capture what organizational readiness truly looks like in schools. We understand that organizational measures are typically designed for aggregating a number of raters in one setting. However, this was not possible to attain in this study as there was generally one teacher or staff person per student with autism who participated. We were unable to gather data from multiple raters within the entire school (i.e., teachers and other classroom staff) to allow for data aggregation on a larger scale; therefore, organizational climate from the TCU-ORC was not examined in this study. These constructs may be critical to understand as implementation often is a multi-level and tiered process that involves multiple stakeholders in a school. Additional research is needed to explore how organizational constructs like readiness can be reliably measured in the education sector (Lyon et al., 2018). Since the completion of this study, additional organizational measures have been adapted for use in schools that may more accurately characterize organizational culture and climate in schools (e.g., Organizational Social Context, Organizational Health Inventory, etc.). Lastly, due to scheduling limitations, in a few cases, teachers had to relinquish their preparatory periods to participate in Remaking Recess training, which occurred during recess. This scheduling was organized in order to avoid obstructing class time.

Implications for Future Research

The results of this study have important implications about the focus of future implementation efforts to improve EBP use in schools. Implementation efforts ought to consider the autonomy of schools and school personnel in EBP decision making and selection. In addition, the contextual appropriateness of the EBP within the school context should be examined to ensure the EBP fits within existing practices, programs, and curricula. Future research also necessitates exploration of implementation strategies that target individual provider (teachers and classroom staff) attitudes as well as organizational structures (e.g., implementation leadership and implementation climate) that may improve EBP use for children with autism in public schools.

Conclusion

The results of this study suggest that individual attitudes about EBPs in addition to characteristics within the organization may be relevant to use of a social engagement intervention, Remaking Recess, for children with autism in public schools. These findings underscore the importance of addressing school personnel’s attitudes about EBPs and school personnel’s’ organizational readiness prior to active implementation. Changing staff perceptions of the EBP and the organization’s readiness for implementation may result in higher fidelity. Future research is needed to understand the nuances of each phase of the implementation process including adoption (decision making), preparation (ensuring the setting is ready for implementation), and active implementation for autism EBP use in schools (Aarons, Hurlburt, & Horwitz, 2011).

Acknowledgments

This study was funded by the Autism Science Foundation (Grants #13-ECA-01L) and FARFund Early Career Award, as well as NIMH K01MH100199 (Locke). We thank the children, staff, and schools who participated in this study.

Footnotes

Disclosure of Potential Conflicts of Interest: The authors declare that they have no conflict of interest.

Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent: Informed consent was obtained from all individual participants included in the study.

References

  1. Aarons GA (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 2, 61–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aarons GA, Ehrhart MG, Farahnak LR, & Hurlburt MS (2015). Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10, 11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aarons GA, Glisson C, Hoagwood K, Kelleher K, Landsverk J, & Cafri G (2010). Psychometric properties and United States norms of the Evidence-Based Practice Attitude Scale (EBPAS). Psychological Assessment, 3, 701–717. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Aarons GA, Green AE, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, Silovsky JF, & Chaffin MJ (2012). Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science, 7, 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Aarons GA, Hurlburt M, & Horwitz SM (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Aarons GA, McDonald EJ, Sheehan AK, & Walrath-Greene CM (2007). Confirmatory factor analysis of the Evidence-Based Practice Attitude Scale (EBPAS) in a geographically diverse sample of community mental health providers. Administration and Policy in Mental Health, 34, 465–469. [DOI] [PubMed] [Google Scholar]
  7. Alanen S, Kaila M, & Välimäki M (2009). Attitudes toward guidelines in Finnish primary care nursing: A questionnaire survey. Worldviews on Evidence-Based Nursing, 6, 229–236. [DOI] [PubMed] [Google Scholar]
  8. Arnadottir SA, & Gudjonsdottir B (2016). Icelandic physical therapists’ attitudes toward adoption of new knowledge and evidence-based practice: Cross-sectional web-based survey. Physical Therapy, 96, 1724–1733. [DOI] [PubMed] [Google Scholar]
  9. Attieh R, Gagnon M, Estabrooks CA, Légaré F, Ouimet M, Vazquez P, & Nuño R (2014). Organizational readiness for knowledge translation in chronic care: A Delphi study. BMC Health Services Research, 14, 534–545. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Beidas RS, Aarons G, Barg F, Evans A, Hadley T, Hoagwood K, & Mandell DS (2013). Policy to implementation: Evidence-based practice in community mental health – Study protocol. Implementation Science, 8, 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Beidas RS, Edmunds J, Ditty M, Watkins J, Walsh L, Marcus S, & Kendall P (2014). Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Administration and Policy in Mental Health, 41, 788–799. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Beidas RS, Marcus S, Aarons GA, Hoagwood KE, Schoenwald S, Evans AC, & Mandell DS (2016). Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics, 169, 374–382. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Bonham C, Willging C, Sommerfeld D, & Aarons GA (2014). Organizational factors influencing implementation of evidence-based practices for integrated treatment in behavioral health agencies. Psychiatry Journal, 2014, 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Brookman-Frazee LI, Taylor R, & Garland AF (2010). Characterizing community-based mental health services for children with autism spectrum disorders and disruptive behavior problems. Journal of Autism and Developmental Disorders, 40, 1188–1201. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Cook BG, & Odom SL (2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79, 135–144. [Google Scholar]
  16. Cook CR, Lyon AR, Kubergovic D, Wright DB, & Zhang Y (2015). A supportive beliefs intervention to facilitate the implementation of evidence-based practices within a multi-tiered system of supports. School Mental Health, 7, 49–60. [Google Scholar]
  17. Cook CR, Davis C, Brown EC, Locke J, Ehrhart MG, Aarons GA, Larson M, & Lyon AR (2019). Confirmatory factor analysis of the evidence-based practice attitudes scale with school-based behavioral health consultants. Implementation Science, 13, 116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Durlak JA, & DuPre EP (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350. [DOI] [PubMed] [Google Scholar]
  19. Fixsen D, Blase K, Metz A, & Van Dyke M (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213–230. [Google Scholar]
  20. Fixsen DL, Naoom SF, Blase KA, & Friedman RM (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida. [Google Scholar]
  21. Forman SG, Olin SS, Hoagwood KE, Crowe M, & Saka N (2009). Evidence-based interventions in schools: Developers’ views of implementation barriers and facilitators. School Mental Health, 1, 26–36. [Google Scholar]
  22. Forman SG, Shapiro ES, Codding RS, et al. (2013). Implementation science and school psychology. School Psychology Quarterly, 28, 77–100. [DOI] [PubMed] [Google Scholar]
  23. Goh SC, Cousins JB, & Elliott C (2006). Organizational learning capacity, evaluative inquiry and readiness for change in schools: Views and perceptions of educators. Journal of Educational Change, 7, 289. [Google Scholar]
  24. Gottfredson DC, & Gottfredson GD (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime and Delinquency, 39, 3–35. [Google Scholar]
  25. Harper CB, Symon JBG, & Frea WD (2008). Recess is time-in: Using peers to improve social skills of children with autism. Journal of Autism and Developmental Disorders, 38, 815–826. [DOI] [PubMed] [Google Scholar]
  26. Harris PA, Taylor R, Thielke R, et al. (2009). Research electronic data capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support, Journal of Biomedical Informatics, 42, 377–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kasari C, Rotheram-Fuller E, Locke J, & Gulsrud A (2012). Making the connection: Randomized controlled trial of social skills at school for children with autism spectrum disorders. Journal of Child Psychology and Psychiatry, 53, 431–439. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Kretzmann M, Locke J, & Kasari C (2012). Remaking recess: The manual. Unpublished manuscript funded by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) under grant number UA3 MC 11055 (AIR-B). http://www.remakingrecess.org [Google Scholar]
  29. Kretzmann M, Shih W, & Kasari C (2015). Improving peer engagement of children with autism on the school playground: A randomized controlled trial. Behavior Therapy, 46, 20–28. [DOI] [PubMed] [Google Scholar]
  30. Kucharczyk S, Reutebuch CK, Carter EW, Hedges S, El Zein F, Fan H, & Gustafson JR (2015). Addressing the needs of adolescents with autism spectrum disorder: Considerations and complexities for high school interventions. Exceptional Children, 81, 329–349. [Google Scholar]
  31. Laushey KM, & Heflin LJ (2000). Enhancing social skills of kindergarten children with autism through the training of multiple peers as tutors. Journal of Autism and Developmental Disorders, 30, 183–193. [DOI] [PubMed] [Google Scholar]
  32. Lavin D, & Groarke A (2005). Dental floss behaviour: A test of the predictive utility of the theory of planned behaviour and the effects of making implementation intentions. Psychology, Health & Medicine, 10, 243–252. [Google Scholar]
  33. Leblanc L, Richardson W, & Burns KA (2009). Autism spectrum disorder and the inclusive classroom: Effective training to enhance knowledge of ASD and evidence-based practices. Teacher Education and Special Education: The Journal of the Teacher Education Division of the Council for Exceptional Children, 32, 166–179. [Google Scholar]
  34. Lehman WEK, Greener JM, & Simpson DD (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22, 197–209. [DOI] [PubMed] [Google Scholar]
  35. Locke J, Beidas R, Marcus S, Stahmer AS, Aarons GA, Lyon AR…& Mandell (2016). A mixed methods study of individual and organizational factors that affect implementation of interventions for children with autism in public schools. Implementation Science, 11, 135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Locke J, Kang-Yi C, Pellecchia M, & Mandell DS (2018a). It’s messy but real: Implementing a social engagement intervention for children with autism in two urban public schools: A one-year pilot study. Journal of Research in Special Educational Needs. doi: 10.1111/1471-3802.12436. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Locke J, Lee K, Cook CR, Frederick L, Vázquez-Colón C, Ehrhart MG, … & Lyon AR (2018). Understanding the organizational implementation context of schools: a qualitative study of school district administrators, principals, and teachers. School Mental Health, 1–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Locke J, Olsen A, Wideman R, Downey MM, Kretzmann M, Kasari C, & Mandell DS (2015). A tangled web: The challenges of implementing an evidence-based social engagement intervention for children with autism in urban public school settings. Behavior Therapy, 46, 54–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Locke J, Rotheram-Fuller E, Xie M, Harker C, & Mandell DS (2014). Correlation of cognitive and social outcomes among children with autism spectrum disorder in a randomized trial of behavioral intervention. Autism: International Journal of Research and Practice, 18, 370–375. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Locke J, Shih W, Kang-Yi C, Caramanico J, Shingledecker T, Gibson J, Frederick L, & Mandell DS (2018b). The impact of implementation support on the use of a social engagement intervention for children with autism in public schools. Autism 10.1177/1362361318787802 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Lord C, Wagner A, Rogers S, Szatmari P, Aman M, Charman T, & Yoder P (2005). Challenges in evaluating psychosocial interventions for autistic spectrum disorders. Journal of Autism & Developmental Disorders, 35, 695–708. [DOI] [PubMed] [Google Scholar]
  42. Lyon A, Cook C, Brown E, Locke J, Davis C, Ehrhart M, & Aarons GA (2018). Assessing organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science, 13, 5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Majid S, Foo S, Luyt B, Zhang X, Theng YL, Chang YK, & Mokhtar IA (2011). Adopting evidence-based practice in clinical decision making: Nurses’ perceptions, knowledge, and barriers. Journal of the Medical Library Association: JMLA, 99, 229. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Mandell DS, Stahmer AC, Shin S, Xie M, Reisinger E, & Marcus SC (2013). The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention. Autism, 17, 281–295. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Marsh HW, Ludtke O, Nagengast B, Trautwein U, Morin AJS, Abduljabbar AS, Koller O (2012). Classroom climate and contextual effects: Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47, 106–124. [Google Scholar]
  46. McDougal JL., Moody Clonan S, & Martens BK (2000). Using organizational change procedures to promote the acceptability of prereferral intervention services: The school-based intervention team project. School Psychology Quarterly, 15, 149. [Google Scholar]
  47. Murray R, Ramstetter C, Devore C, Allison M, Ancona R, Barnett S, … & Okamoto J (2013). The crucial role of recess in school. Pediatrics, 131, 183–188. [DOI] [PubMed] [Google Scholar]
  48. Newman J, Brody PJ, & Beauchamp HM (1996). Teachers’ attitudes and policies regarding play in elementary schools. Psychology in the Schools, 33, 61–69. [Google Scholar]
  49. Odom SL, Cox AW, Brock ME, & National Professional Development Center on ASD. (2013). Implementation science, professional development, and autism spectrum disorders. Exceptional Children, 79, 233–251. [Google Scholar]
  50. Owens JS, Lyon AR, Brandt NE, Warner CM, Nadeem E, Spiel C, & Wagner M (2014). Implementation science in school mental health: Key constructs in a developing research agenda. School Mental Health, 6, 99–111. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Pellecchia M, Connell JE, Beidas RS, Xie M, Marcus SC, & Mandell DS (2015). Dismantling the active ingredients of an intervention for children with autism. Journal of Autism and Developmental Disorders, 45, 2917–2927. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Pellicano E, Dinsmore A, & Charman T (2014). What should autism research focus upon? Community views and priorities from the United Kingdom. Autism, 18, 756–770. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Podsakoff PM, MacKenzie SB, & Podsakoff NP (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879–903. [DOI] [PubMed] [Google Scholar]
  54. Powell BP, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM…& Kirchner JE (2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10, 1–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Rogers SJ (2000). Interventions that facilitate socialization in children with autism. Journal of Autism and Developmental Disorders, 30, 399–409. [DOI] [PubMed] [Google Scholar]
  56. Shih W, Dean M, Kretzmann M, Locke J, Senturk D, Mandell DS, Smith T, Kasari C, & AIR-B Network. (in press). Improving social connections at school for elementary aged children with autism spectrum disorder (ASD): A multisite randomized trial. School Psychology Review. [Google Scholar]
  57. Simpson DD (2002). A conceptual framework for transferring research to practice. Journal of Substance Abuse Treatment, 22, 171–182. [DOI] [PubMed] [Google Scholar]
  58. Southam-Gerow MA, Weisz JR, Chu BC, McLeod BD, Gordis EB, & Connor-Smith JK (2010). Does cognitive behavioral therapy for youth anxiety outperform usual care in community clinics? An initial effectiveness test. Journal of the American Academy of Child & Adolescent Psychiatry, 49, 1043–1052. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Stahmer AC, & Aarons GA (2009). Attitudes toward adoption of evidence-based practices: A comparison of autism early intervention providers and children’s mental health providers. Psychiatric Services, 6, 223–234. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Stahmer AC, Reed S, Lee E, Reisinger EM, Mandell DS, & Connell JE (2015). Training teachers to use evidence-based practices for autism: Examining procedural implementation fidelity. Psychology in the Schools, 52, 181–195. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Stark KD, Arora P, & Funk CL (2011). Training in school psychologists to conduct evidence-based treatments for depression. Psychology in the Schools, 48, 272–282. [Google Scholar]
  62. Suhrheinrich J, Stahmer AC, Reed S, Schriebman L, Reisinger E, & Mandell D (2013). Implementation challenges in translating pivotal response training into community settings. Journal of Autism and Developmental Disorders, 43, 2970–2976. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. van Sonsbeek MS, Hutschemaekers GM, Veerman JW, Kleinjan M, Aarons GA, & Tiemens BG (2015). Psychometric properties of the Dutch version of the Evidence-Based Practice Attitude Scale (EBPAS). Health Research Policy & Systems, 13, 69. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Weiner BJ (2009). A theory of organizational readiness for change. Implementation Science, 4, 67. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES