Implementation Science in School Mental Health: A 10-Year Progress Update and Development of a New Research Agenda

Abstract

Implementation science has grown considerably in school mental health over the past decade, driven by the persistent gaps between knowledge about “what works” to promote student well-being and what is typically delivered in schools. In 2014, Owens and colleagues outlined an implementation research agenda for the field of school mental health, emphasizing professional development and coaching, evidence-based practice fidelity, and sustainment. This paper provides a 10-year progress review across these domains and articulates an updated research agenda focused on: (1) pragmatic implementation strategies, (2) implementation mechanisms explaining the connection between implementation strategies and outcomes, (3) strategic intervention redesign to promote implementation, and (4) de-implementation of low-value practices. We propose critical research questions in these areas and conclude by highlighting contemporary study designs and methods (i.e., hybrid trial designs; sequential, multiple assignment, randomized trials; rapid qualitative approaches; economic analyses) that can be leveraged to investigate implementation research questions in school mental health over the next decade. In doing so, we hope to promote shared understanding of evolving implementation research priorities and opportunities to promote equitable, pragmatic, and sustainable implementation efforts in school mental health and drive synergy across projects.

Introduction

For decades, researchers have focused on “what works” to address student mental health concerns, resulting in an array of evidence-based practices (EBP) to be delivered by school-based professionals (Evans et al., 2023; Fabiano & Evans, 2019). However, methods to ensure use of evidence in authentic educational contexts have been slower to develop, leading to underutilization of these practices (Lyon & Bruns, 2019a). Limitations in the reach, effectiveness, and equity of our school mental health service systems have been made especially apparent by the growing children’s mental health crisis, exacerbated by the onset of the COVID-19 pandemic (Bussières et al., 2021). Efforts to close the research-to-practice gap in school mental health can be conceptualized through the lens of implementation science—the study of methods to promote the uptake of research findings and EBP into routine service delivery (Eccles & Mittman, 2006). In 2014, driven by a growing wave of implementation-explicit work across healthcare and other sectors, Owens et al. (2014) articulated a research agenda intended to launch school mental health into the choppy “high seas” of implementation. Ten years later, we hope to highlight progress that has been made relevant to the research questions posed at that time and offer an updated perspective with more precise “charting” of a research agenda to help steer the field of school mental health on its continuing implementation voyage.

Implementation science is founded on repeated observations that EBP are inconsistently adopted, delivered with fidelity, and/or sustained over time in “real-world” delivery settings. In school mental health, substantial research has documented this phenomenon across the prevention and intervention spectrum, as well as the barriers and facilitators (i.e., implementation determinants) that drive it (e.g., Connors et al., 2021; Langley et al., 2010; Locke et al., 2019; Stahmer et al., 2015). For conceptual clarity, it is important to distinguish implementation from dissemination, or the targeted distribution of information to a specific audience (NIH, 2019). Although interest in dissemination science and the identification of dissemination strategies (i.e., intentional methods to communicate strategically about an EBP to specific partners to alter antecedents of behavior change) is growing, it has received far less attention than implementation across sectors and will not be an explicit focus of the current paper. Readers are referred to Baker et al. (2021) for a comprehensive review of dissemination science as it relates to school mental health. Further, as the field of implementation has continued to evolve and expand, distinct and complementary research and practice arms have emerged (Lyon et al., 2020a). A critically important counterpart to implementation research is implementation practice which involves conducting applied work to improve research evidence use and service quality rather than the production of generalizable knowledge about implementation (Franks & Bory, 2015). While this paper is focused explicitly on implementation research, the research agenda presented prioritizes questions with relevance for implementation practitioners.

School-based research has become increasingly oriented toward addressing implementation gaps, especially surrounding the provision of social, emotional, and behavioral supports and services. Core “building blocks” of implementation research include the identification and prioritization of implementation determinants, selection and testing of implementation strategies (i.e., active methods and techniques that address determinants to improve implementation), and measurement of implementation outcomes (e.g., adoption, fidelity, sustainment; Lyon & Bruns, 2019a). Further, because school mental health service access and utilization equals or exceeds mental health services provided in any other child-serving sector (Duong et al., 2020)—especially for marginalized and minoritized populations (e.g., Kataoka et al., 2011; Lyon et al., 2013; Wilk et al., 2022)—ensuring high-quality services also becomes an ethical and social justice imperative. Although schools improve access to services, there remain numerous avenues through which school mental health services still experience (and can perpetuate) inequities (Cipriano et al., 2022; Liu et al., 2022b). By considering both accessibility and quality, implementation science provides one prominent pathway to achieve the promise of school mental health. The need to explicitly center equity also is inherent in both school mental health (due to its population focus and orientation toward an accessible public sector service delivery system; Weist et al., 2023) and implementation science (due to its emphasis on contextually appropriate improvements in service quality; Baumann et al., 2023). Mirroring the broader implementation field (Adsul et al., 2024; Shelton & Brownson, 2023), studies at the intersection of school mental health and implementation science have begun to explicitly incorporate equity through pathways, such as the Adapting Strategies to Promote Implementation Reach and Equity (ASPIRE) framework (Gaias et al., 2022) and the emergence of equity-explicit implementation strategies that target barriers like implicit bias (Liu et al., 2022a; Owens et al., 2024).

Ten years out from Owens and colleagues’ (2014) research agenda, school mental health research and practice is situated within a social and policy context shaped by the COVID-19 pandemic and a contentious sociopolitical climate. Unmet youth mental health needs are rising, particularly for young people who experience marginalization related to racism, poverty, and/or discrimination based on LGBTQ+ status (Murthy, 2022; Ormiston & Williams, 2022). There is a growing recognition that schools are a key setting to help address this unmet need (IES, 2023); at the same time, school systems and personnel are experiencing tremendous stress, disruption, and staff turnover (Steiner & Woo, 2021), which pose heightened implementation challenges. Recent investments in schools, including federal COVID-19 relief funding, have offered opportunities to expand and improve the reach of comprehensive school mental health services (e.g., via enhanced EBP and school–community partnerships), but concerns remain about continued resources, implementation, and sustainment (Hoover, 2024).

Situated within the context of the advancing field and the current sociopolitical climate, the purpose of the current paper is to (1) review progress on the research questions originally outlined by Owens et al. (2014) and (2) present an updated agenda for implementation research in school mental health that reflects the advancing field. We will not provide broad definitions or overviews of implementation concepts, as those were addressed in Owens et al. (2014) and can also be found elsewhere (c.f., Forman et al., 2013; Lyon & Bruns, 2019a; Sanetti & Collier-Meek, 2019). The updated agenda focuses on four domains: (1) pragmatic implementation strategies, (2) implementation mechanisms, (3) strategic intervention redesign for implementation, and (4) de-implementation of low-value practices. We conclude by featuring example research designs and methods that can be applied to the four domains and illustrate how each design or method could link to a proposed implementation research question.

Review of Progress

The primary goal of Owens et al. (2014) was to present a blueprint with relevance to funders, research–practice partnerships, and researchers. In doing so, the authors first described an array of school-specific contextual issues relevant to implementation (e.g., school calendars, multiple types of relevant professionals). Then, they outlined a research agenda—and specific research questions—surrounding three core implementation components: (1) professional development (PD) and coaching, (2) EBP integrity/fidelity, and (3) EBP sustainment. We provide an update on progress for each research question posed within each domain, with particular attention to PD and coaching strategies given their prominence in school mental health research and notable strides since 2014. In identifying updated literature, we (1) reviewed papers cited in the original Owens et al. (2014) paper and searched for updated works from those research teams, (2) conducted forward searches on the key articles cited in the relevant sections of the 2014 paper, and (3) gathered a group of established experts in school mental health and implementation science to identify rigorously conducted and well-cited works. We scoped our search to focus on PreK-12 school mental health promotion, prevention and intervention, including studies involving teachers, administrators, school mental health providers, and other professionals involved in supporting student mental health and social–emotional well-being.

Professional Development (PD) and Coaching

Among the three areas outlined in the 2014 research agenda, we identified notable strides over the past decade particularly in the domain of PD (Admiraal et al., 2019; Stormont et al., 2015). Here we highlight selected work to illustrate what has been learned about the questions posed ten years ago. As noted in the prior paper, effective PD extends beyond onetime training to include ongoing consultation and coaching and other related implementation strategies (Cook et al., 2019) targeting practice change at the level of the school-based implementer (e.g., teacher, school clinician). The initial training component of PD, or “conducting educational meetings,” includes meetings targeted toward different groups of professionals (e.g., teachers, principals, central administrators, other organizational leaders, and community and family members) to teach them about the new practices (Cook et al., 2019; Lyon et al., 2019b). Onetime training continues to be associated with increased knowledge and attitudes but, not necessarily practice change in schools (Anderson et al., 2019) or other settings (Valenstein-Mah et al., 2020). Despite this evidence, we notice educational meetings and onetime training to be the prevailing PD modality used in schools (Langreo, 2022; Wilkinson et al., 2020).

Beyond initial training, a robust school mental health implementation research base supports the value of ongoing, post-training consultation or coaching (Ennis et al., 2019; Erchul, 2023; Pas et al., 2023). Consultation or coaching can be provided by someone internal or external to the school system, focused on the implementers who actually provide the EBP (Cappella et al., 2015). Some coaching or consultation is more expert-driven and hierarchical, whereas other models are more collaborative and guided by the teacher’s expertise and goals (Pas et al., 2023). The term “consultation” has been used more historically in the education literature including consultation models dating back to the 1970s (e.g., Behavioral Consultation; Bergan, 1977). “Consultation and coaching” represent similar processes and we use the more recent term, “coaching,” hereafter (Erchul, 2023; Pas et al., 2023). Research since 2014 has evaluated coaching as an implementation strategy to promote educator practices including classroom management (e.g., Bradshaw et al., 2018; Fabiano et al., 2018; Owens et al., 2020; Reinke et al., 2008; Shernoff et al., 2018; Sutherland et al., 2015) and classroom climate (e.g., Allen et al., 2015; Reinke et al., 2008), as well as to promote school mental health clinician’s use of EBP (Lyon et al., 2022; Meyer et al., 2022). Progress made on the PD and coaching-related research questions from Owens et al. (2014) are reviewed below.

Question 1. How Much Coaching (Dose) is Needed Following Initial PD to Enable Providers to Deliver an EBP or Its Components with High Integrity and Produce Desired Student Outcomes?

Inherent in this question is the hypothesis that “more is better” with respect to coaching dosage. Evidence from the past decade provides mixed support for this hypothesis. An observational study of coaching dose within the coaching arm of a randomized controlled trial found that higher coaching dosage across two school years for 192 preschool teachers implementing My Teaching Partner was associated with higher teacher implementation quality of emotional support practices. This implementation outcome mediated the relation between coaching dosage and increases in student outcomes of classroom engagement (Pianta et al., 2022). A second observational study of in-person classroom coaching with 210 teachers that focused on enhancing implementation of a Tier 1 mental health promotion program (PAX Good Behavior Game) found that more frequent coaching was associated with more frequent implementation, but no better implementation quality (Pas et al., 2015). However, these observational studies did not account for variables that may have contributed to higher doses of coaching, such as teacher preferences or needs, necessitating experimental design of coaching dose. In this vein, Lyon et al. (2022) randomly assigned 75 school-based mental health clinicians to 2, 4 or 8 weeks of coaching following initial training in implementation of measurement-based care with Tier 3 interventions. Interestingly, although some consultation was critical to implementation, they found no evidence for additional benefits from higher consultation dosages. These mixed results illustrate the importance of empirical studies to identify the impact of coaching dose, and factors related to the intervention, implementer, or context that may impact the relation between coaching dose and outcomes.

Some factors that may impact that relation include the quality of the coaching relationship (e.g., coach–implementer fit; (Johnson et al., 2016; Partee et al., 2022)), implementer motivation, interest in, and openness to receiving coaching, intervention complexity and usability, teacher experience and burnout (Holdaway & Owens, 2015) and teacher stress, workload demands, and other priorities aside from the intervention (Owens et al., 2018). Measurement of these factors would likely necessitate collection and comparison of multi-reporter data from coaches and implementers on the coaching experience and multi-level contextual factors (such as from a determinants checklist adapted for schools) potentially related to implementation outcomes. For instance, receptivity to coaching and teacher motivation has been related to growth and change in teacher practices, respectively (Owens et al., 2017b, 2021). Some of this work is reflected in Question 2, as various implementer-level determinants may operate as mechanisms of implementation or intervention fidelity. Of note, findings to date on the relation between coaching dose and outcomes have been conducted in the context of research studies with voluntary implementer participation, so future work is warranted to understand this relation in naturalistic implementation efforts in schools outside the confines of a research study with voluntary participation.

Question 2. What Strategies (or Combinations of Strategies) are Most Likely to Enhance Provider Knowledge, Skills and Beliefs, Intervention Integrity, and Student Outcomes?

A large body of evidence from the past decade supports modeling, practice, observation and performance feedback as core components of coaching that predict implementation of academic, social, emotional, and behavioral interventions (e.g., Glover et al., 2023; Kraft & Blazar, 2018; Noell & Gansle, 2014). However, intervention fidelity varies considerably within study samples based on implementer and school context factors, motivating the more recent question of identifying what coaching strategies work best for whom. Some research suggests that tailored consultation based on implementer needs and preferences is desirable to school mental health partners (Larson, 2022). In addition, there is emerging evidence that supports the effectiveness of tailored consultation. Namely, a randomized trial with 58 elementary school teachers found that, among teachers with lower baseline knowledge, skills and intervention supportive beliefs, those receiving an individually tailored, multi-component consultation package made greater gains in their use of effective classroom management strategies compared to those who received a standard problem-solving consultation package (Owens et al., 2017a). Yet, teachers without these implementer-level barriers showed equal benefit across individualized and standardized consultation conditions and were satisfied with the consultation received. This finding suggests that individualization may be more important for those who experience implementation challenges. Further, a small multiple baseline study with four teachers who received consultation within a Response to Intervention (RtI) model used predetermined implementation decision rules for transitioning teachers between tiers of consultation support and found that teachers needed varying levels and types of consultation support to improve their implementation outcomes (LaBrot et al., 2020). Together, these findings point toward future opportunities to study the effectiveness and scalability of tailored or multi-tiered levels of coaching strategies to suit individual-level determinants.

In summary, research in the past decade has advanced our understanding how implementer knowledge, skills, beliefs, and preferences influence the effectiveness of coaching dosage and strategies. Although tailored coaching based on individual implementer needs and preferences has been recommended among school mental health stakeholders (Larson, 2022), questions still remain to be explored about how coaching should be tailored, when, and for whom to achieve optimal implementation outcomes. While used frequently in school mental health implementation research (Baffsky et al., 2023), coaching is often tested as a packaged implementation strategy. Thus, future research to summarize, distill and evaluate coaching components (e.g., goal setting, action planning, modeling, observation, performance feedback, role play) in isolation and/or combination is needed.

Question 3. What Models of Coaching (External vs. Internal Coach) are Most Likely to be Adopted and Best Promote the Skills Taught Within PD?

Differences in the effectiveness of internal (e.g., district-employed) vs. external (e.g., EBP purveyors) coaching is still a nascent area in school mental health implementation research. Trade-offs between costs and funding sources needed, coach capacity or bandwidth, and diversity of training and expertise have been repeatedly pointed out as important considerations for school systems (Flannery et al., 2018; Giordano et al., 2020). Evidence about effective educator PD and coaching models for academic instruction more broadly is a useful guidepost as we develop evidence in school mental health implementation coaching. Specifically, a multi-site qualitative study of secondary schools found that external coaches were most effective at providing training and influencing early adoption, whereas internal coaches provided more in vivo classroom modeling, problem-solving and feedback for continued implementation (Thompson, 2022). A mixed methods study of internal and external coaches supporting 63 teachers on professional learning teams in Belgium found more positive teacher satisfaction with internal coaches yet better team effectiveness with external coaches (Compen & Schelfhout, 2020). More specific to school mental health, Giordano and colleagues (2020) found that among 15 Head Start teachers receiving coaching for social–emotional classroom practices, those with internal peer coaches demonstrated better intervention fidelity and student social skills as compared to those with external coaches. External district- and internal building-level coaches are commonly used in positive behavioral interventions and supports (PBIS) (Amaya & Amundson, 2024) and in some comprehensive school behavioral health models in the field (e.g., Battal et al., 2020; Marx et al., 2020). Overall, this is still an emerging area in school mental health implementation research and evidence from recent studies of school-based coaching more broadly points to strengths and limitations of both external and internal coaching models. Research to date is only beginning to explore the differential effectiveness of internal versus external coaches, with generally mixed findings. It may also be the case that internal and external coaches can work together in tiered coaching models to build internal coaches’ capacity for longer term workforce sustainability within the school system (Frazier et al., 2019; Ouellette et al., 2024). Although little research has explicitly attended to equity in coaching, it will be important to explore practical, systems issues such as how to ensure that under-resourced school systems can access effective, tailored, and multi-level (as needed) internal and external coaching models. Finally, beyond examining models of coaching, there is still little research to date empirically examining which components of coaching (internal or external) are essential to effectiveness.

Question 4. To What Extent are Implementer Motivation and Perceptions Malleable, and Does Change in These Factors Facilitate Implementer Skill Development?

Implementer motivation, perceptions, and engagement in PD and coaching for new initiatives has continued to be an important area of focus (Holmes et al., 2021) in addition to interest in the effects of teacher stress and burnout on classroom practice and teacher and student outcomes (Herman et al., 2020). Emerging research on the Beliefs and Attitudes for Successful Implementation in Schools (BASIS; Lyon et al., 2019a) implementation strategy, designed to target implementer motivation and enhance the effects of training and coaching, points to the potential malleability of implementer perceptions and beliefs. BASIS is a pragmatic and theoretically driven strategy that applies strategic education, social influence techniques, motivational interviewing, action planning, and problem-solving planning to improve EBP uptake. Clinician-focused and teacher-focused versions have been developed and tested. BASIS has demonstrated promise in enhancing participants’ attitudes, subjective norms, self-efficacy, and EBP adoption (Cook et al., 2015; Larson et al., 2021; Lyon et al., 2019a; Merle et al., 2023), and additional large-scale RCT work is ongoing (Lyon et al., 2021, 2024). Additional evidence is accumulating on the use of motivational interviewing techniques within coaching models to improve teacher self-efficacy in classroom behavior management interventions, reduce work-related stress (Bradshaw et al., 2018; Lee et al., 2019), and improve implementation (Owens et al., 2021). Overall, these advancements suggest that implementer motivation and perceptions may be malleable and point to the promise of targeting these characteristics to enhance implementation. However, enhancing implementer motivation is still an early and open area of inquiry, with motivational interviewing approaches as the predominant method to date.

EBP Fidelity

The 2014 research agenda also outlined three critical research questions related to the multiple dimensions of intervention integrity (restated using Owens et al.’s original terminology but discussed in this paper using the term “fidelity”), including adherence and competence. Addressing these questions requires measuring multiple dimensions of intervention fidelity, particularly within naturalistic trials in which school personnel implement interventions.

Question 1. What Level of Integrity (Adherence and Competence) is Good Enough to Produce the Intended Student Outcomes?

Several studies in the past decade have examined the relation between intervention fidelity and student outcomes. Although a number of studies have found positive relations between aspects of fidelity (e.g., fidelity to school mental health models; fidelity to aspects of schoolwide positive behavioral interventions and supports) and student outcomes (e.g., Barclay et al., 2022; Reinke et al., 2021), other findings have been more mixed (see Rojas-Andrade & Bahamodes, 2018 for a review). Regarding the specific research question posed by Owens et al. (2014), there has been less work specifically examining benchmarks for “good enough” levels of fidelity that might be needed to produce intended student outcomes. The few studies that have examined fidelity benchmarks have focused on teacher fidelity to behavioral classroom practices. Using weekly observation data, Owens et al. (2018) found that disruptive student behavior was significantly lower in classrooms where teachers responded effectively to a minimum threshold of disruptive behavior (51% of rule violations). Similarly, teachers who reached a minimum benchmark of responding effectively to least 51% of rule violations witnessed progressively lower rates of challenging behavior classwide and among students with or at-risk for ADHD over time (Owens et al., 2020). In contrast, disruptive behavior worsened over time in classrooms where teachers did not meet this threshold. Although additional benefits were observed when teachers responded effectively to even higher proportions (e.g., 60%, 70%, 80% of rule violations), these data provide preliminary evidence for a “good enough” level of fidelity to support desired student outcomes within the context of teacher-delivered behavior management approaches. Additional research with teacher and school mental health professionals is needed to identify minimum thresholds for fidelity that produce desired outcomes, as such minimum thresholds may be more achievable and feasible to sustain than thresholds requiring near perfection (e.g., 80–100%). Additionally, given the mixed results relating fidelity to student outcomes (i.e., only about 40% of the studies reviewed by Rojas-Andrade & Bahamodes (2018) observed an association between components of fidelity and outcomes), it is also important to identify factors that may moderate the relationship between fidelity and outcomes.

Question 2. Are the Multiple Dimensions of Integrity Differentially Predictive of Student Outcomes?

The past decade has seen progress as well as a number of challenges related to addressing this question. As emphasized by the special series in this journal, Advancing the Science of Integrity Measurement in School Mental Health Research, many researchers have led innovative and important work regarding the measurement of the multiple dimensions of fidelity, but this literature is still limited (Sutherland & McLeod, 2022). There has been an increase in the number of school-based intervention studies assessing at least one dimension of fidelity in the past decade; however, only a minority of studies from this time period reported on dimensions other than adherence (Sanetti et al., 2020). Developing and administering measures of the multiple dimensions of fidelity that are feasible, reliable, valid, and appropriate to the stage of the intervention development pipeline is a necessary step to advance the literature about the relations between fidelity dimensions and student outcomes (Sutherland & McLeod, 2022).

Studies that have tested the relations between multiple dimensions of fidelity and student outcomes have found mixed results. In a trial with school-based providers, Husabo et al. (2021) examined therapist adherence and competence in two school-based CBT programs and did not find relationships between either dimension of fidelity and student outcomes. Sutherland et al. (2018) found that teacher competence, but not teacher adherence, partially mediated the treatment effect of a teacher coaching model on student outcomes (i.e., externalizing problems). Other recent studies examining the relationship between single dimensions of fidelity and outcomes have found positive relationships between student outcomes and treatment dosage (DuPaul et al., 2024; Girio-Herrera et al., 2021) as well as session attendance (Margherio et al., 2023). Additionally, Pas et al. (2022) found mixed relationships between latent profiles of fidelity (using indicators of adherence, dosage, quality, and participant responsiveness) to a teacher coaching model and student outcomes. Taken together, the past decade of school mental health research has led to some evidence that multiple dimensions of fidelity may differentially predict student outcomes, but the primary conclusion from the current body of work is that additional measurement development work may be needed before researchers can fully answer this question.

Question 3. What Accountability Structures (e.g., Administration or Peer Networks) and/or Incentive Programs (Used in Combination with PD) Reduce Variability in Integrity?

There has also been increasing interest in the role of accountability structures and incentive programs in supporting treatment fidelity, although we are not aware of published studies that directly address this question. McLeod et al. (2022) wrote about the promise of “learning school systems,” which use an efficient, data-driven process to collect data and provide feedback about treatment fidelity and/or student outcomes to support the continuous improvement of EBP implementation. Learning school systems—like learning healthcare systems—can be realized through plan–do–study–act (PDSA) cycles, a rapid learning approach to (a) plan a small test of change based on a hypothesis about why a problem exists, (b) enact the change, (c) observe the result, and (d) evaluate and monitor whether the plan worked (Taylor et al., 2014). Several recent papers have described the use of PDSA cycles to support quality improvement via targeted implementation efforts in school mental health systems, finding promising results in terms of quality improvement outcomes (Bohnenkamp et al., 2023; Connors et al., 2020, c; Heatly et al., 2023). These processes can be used by school teams to collect, review, and make quality improvement and implementation decisions based on fidelity data (McLeod et al., 2022). Nevertheless, although PDSAs have been associated with observed systemwide quality improvement and implementation outcomes in school mental health (Bohnenkamp et al., 2023; Connors et al., 2022c), we are not aware of studies directly testing the impact of PDSA cycles on reducing variability in fidelity in schools.

Furthermore, although incentive-based implementation strategies have emerged in community mental health (e.g., Beidas et al., 2017), we are unaware of published studies in the past decade that test incentive programs to support intervention fidelity in schools. The school context may have unique considerations related to the acceptability and feasibility of incentives (Lyon et al., 2018; Thayer et al., 2022). It is important for research–practice partnerships to consider these contextual factors in order to advance work on accountability structures and incentive programs in schools.

EBP Sustainment

Sustainment has been broadly conceptualized as the continuation of effective practices and programs after the initial implementation stage or after program funding or external support has ended (Stirman et al., 2012). This includes studies of (1) the extent to which core program elements continue to be implemented with sufficient fidelity or intensity, (2) the extent to which program outcomes and benefits are maintained, (3) the impact, extent, or characteristics of modifications made to the program, and (4) the capacity to continue to use the program (Stirman et al., 2012). The 2014 research agenda outlined two core research questions related to sustainment.

Question 1. What Does it Mean for an EBP to be Sustained in SMH? What are the Critical Dimensions of Sustainment and How are They Best Measured?

There has been notable growth in research on different definitions of sustainment in school mental health, based on looking at whether the intervention was sustained in whole or in part, at various follow-up intervals, and within or outside the context of grant-funded research. For instance, several recent studies have examined the extent to which EBPs were sustained following the end of research trials or implementation efforts. Measuring partial or full program continuation, these studies have typically found moderate to low levels of sustainment at the district, school, and individual implementer level. A survey study of 77 evidence-based prevention programs funded through a statewide seed grant initiative across schools and community organizations found that about 69% of programs were continuing in some capacity two years or more after initial funding. However, 60% of these programs reported significantly scaling back their efforts (e.g., fewer schools, fewer people served; Cooper et al., 2015). In another study, conducted two years after an effectiveness trial of an intervention for behaviorally at-risk students in 48 elementary schools across five districts, only one of the districts continued to implement the program (Woodbridge et al., 2014). Another post-trial study of a peer victimization and bullying prevention program in eight schools found that five schools reported full implementation, and three reported that they sustained the program only in part (Leadbeater et al., 2015). In contrast, a study of trauma-informed universal mental health intervention found that no schools sustained the program at one-to two-year follow-up (Arnold et al., 2021).

Other sustainment studies have focused on extensiveness and quality of practice implementation by individual teachers involved in research trials. For instance, in a study of MOSAIC—a classroom intervention with universal and targeted supports to facilitate positive peer relationships and academic success—Kassab et al. (2023) observed the relative usage of MOSAIC practices during the follow-up year compared to the trial year, and found teachers showed good sustainment of 75% of strategies (operationalized as observed usage 80% or higher), and greater use of some strategies in the follow-up year as compared to the randomized trial year. In another study of the sustainment of the Research-based Developmentally Informed (REDI) project among 37 preschool teachers, Bierman et al. (2013) observed no significant changes in the observed implementation quality of the Promoting Alternative thinking Strategies (PATHS) social–emotional learning curriculum at a 1-year follow-up.

There are fewer studies of sustainment as part of implementation efforts that occurred outside of research trials. Findings from a survival analysis study of a statewide effort to implement evidence-based mental health interventions with community clinicians suggest that school-specific EBP may be particularly vulnerable to discontinuation compared to EBP in other sectors (Brookman-Frazee et al., 2018). Another study of a national dissemination effort of the LifeSkills Training (LST) middle school program in 158 districts found that 51% of districts sustained the program in at least one participating school 2 years after grant-supported implementation; of the 419 schools across these districts, 30% reported implementing LST at the same level or higher, 11% at a reduced level, and 59% had discontinued use (Combs et al., 2023). One study of a non-grant-funded, school district-led effort to implement a small group trauma intervention found that once a district-level mandate to implement the program was lifted, about half the schools stopped implementing the program (Nadeem & Ringle, 2016).

Relatively less progress has been made in the area of operationally defining and measuring sustainment in school mental health. Across the sustainment studies reported above, most used semi-structured interview data or surveys to identify the number of schools, classrooms, or teachers using a specific practice (e.g., Combs et al., 2023; Nadeem & Ringle, 2016). One study used health claims data to assess implementation by clinicians employed by mental health agencies (Brookman-Frazee et al., 2018). To identify practice-level sustainment, some studies have used observational measures focused on the number of strategies or quality of implementation (e.g., Bierman et al., 2013; Kassab et al., 2023) or self-report checklists (e.g., Kassab et al., 2023). Given the limited advancement surrounding the clear measurement of sustainment in school mental health, some direction may be taken from work within the broader implementation science where sustainment measurement is more robust (see Moulin et al., 2020, for a review of sustainment measures).

Question 2. What are the Essential Ingredients (e.g., Coaching, Integrity, Adaptation, Student Outcomes, Contextual Characteristics) for Successful Sustainment (Predictors/Processes)?

Recent research points to factors at multiple levels (e.g., intervention characteristics, implementer perceptions, initial level of implementation support provided, school characteristics) that influence successful sustainment. One emergent theme is that school personnel appear to be sustaining practices that have higher ease of use and which they perceive to be effective or useful. For example, a qualitative study of 13 teachers using BEST in Class, a coaching program designed to promote the use of effective teaching practices that foster positive teacher–child interactions for students at risk for an emotional disability classification, revealed that half of teachers used practices like making rules, praise, and relationship building. Practices that required more intentional implementation and planning (e.g., opportunities to respond), and were therefore presumably more difficult to use, were applied less often (Washington-Nortey et al., 2023). Similarly, in their study of sustainment of MOSAIC practices, Kassab and colleagues (2023) found that teachers who received a year of intensive coaching were observed to have strong sustainment (less than a 20% decline), and tended to implement strategies that were feasible and easy to use (i.e., greetings, reinforcing expectations for behavior and inclusiveness) more frequently in the follow-up year than teachers who did not receive such coaching. This study also offers preliminary evidence that strength of initial implementation strategies may impact strength of sustainment. Finally, Bierman et al. (2013) found higher sustained use of the PATHS social–emotional learning (SEL) curriculum relative to the literacy curriculum, which the authors attributed to feedback from teachers that PATHS filled a particular void for them in teaching SEL. Together, these studies offer evidence that when teachers perceive strategies as easier, needed, and/or more effective than others, it may reinforce teachers’ usage of such strategies over time.

There is also emerging literature on the impact of broader contextual characteristics on sustainment such as system-level implementation support structures and community and school characteristics. In a large national sample of schools implementing school wide PBIS, McIntosh et al. (2016) found that state-level support systems appeared to facilitate sustained high-quality implementation (e.g., statewide PBIS leadership team, state-level trainers, standardized trainings and curricula, recognition of schools with strong implementation, and use of national assessment and guidance tools). This study also found that elementary schools were more likely to sustain PBIS compared to middle and high schools, and that schools with higher numbers of students receiving free and reduced lunch (FRL) were less likely to sustain at high quality (McIntosh et al., 2016). Another study of a behavioral health curriculum found that sustainment was higher in school districts with strong administrator support and lower in districts serving higher proportions of ethnic minority students (Combs et al., 2023). Similar contextual factors have emerged across qualitative studies pointing to multi-level influences similar to those found in studies of initial implementation. These include policies, mandates, funding, district-level leadership support and guidance, resource constraints, staffing stability, shared leadership between leaders and implementers, school-level training and PD, data and feedback, and efforts to establish buy-in (e.g., Leadbetter et al., 2015; Nadeem & Ringle, 2016; Scaletta & Tejero Hughes, 2021; Woodbridge et al., 2016). The emerging evidence that sustainment is more challenging for schools and districts with higher proportions of minoritized students and students receiving free and FRL warrants attention (e.g., Combs et al., 2023; McIntosh et al., 2016). These findings raise the need to attend to equity issues in sustainment research and practice, especially on sustainment strategies conducted with school communities serving high numbers of minoritized students. For example, Nadeem et al. (2023) piloted a sustainment planning process for a trauma-focused EBP that included community organizations, school leaders, providers, funders, advocates, and others serving the same city school district. Such work highlights the potential need for increased and longer periods of implementation funding, innovative funding models, additional resources for dedicated program and leadership staffing, and ongoing expert consultation for school and district leaders.

Lastly, there appear to be pre-implementation factors and processes that have bearing not only on implementation but also sustainment. Specifically, the MOSAIC study found that teachers with higher pre-intervention teaching quality had better sustainment compared to those with lower pre-intervention quality (Kassab et al., 2023). With respect to processes to support sustainment, a study of schoolwide PBIS found that buildings who were able to move more quickly through early implementation steps were also more likely to sustain than those that did not (McIntosh et al., 2016), suggesting that early-stage team collaboration processes have later implications for sustaining new practices. Across studies, baseline implementation quality has been found to predict sustainment, and sustained implementation in one program domain predicts sustained implementation in other program domains (e.g., Bierman et al., 2013; Kassab et al., 2023). Finally, intentional and ongoing sustainment planning processes involving key personnel and leadership from the beginning of implementation appear important for program continuation (e.g., Leadbetter et al., 2015). Future research can continue to build on the sustainment progress described above, ultimately working toward the development and testing of effective sustainment strategies.

Updated Research Agenda

Much progress has been made related to Owens et al.’s original research agenda, and now is an appropriate time to continue to push the field forward and chart a new course for the next decade. Although contemporary implementation research in schools tends to emphasize PD and coaching approaches (Baffsky et al., 2023) above the multitude of other implementation strategies (e.g., incentives, leadership supports; see Cook et al., 2019) and elevates fidelity above other implementation outcomes (Sanetti & Collier-Meek, 2019), we envision a future that builds on these discoveries and extends them into the broader domains of implementation science.

Owens et al. (2014) emphasized the importance of contextual factors unique to implementation of EBP in schools, including organizational and leadership factors, the diverse workforce of school-based professionals as implementers, and the complexities of academic calendars. Additional factors that can vary widely across implementation projects include, but are not limited to, implementation initiative funding (e.g., federal service or grant; research grant; foundation funding; local, state, or district funds), implementer characteristics, incentives or costs for individual implementers or school systems to take part, and school system expectations or mandates. These remain highly relevant to the updated research agenda described below and align with a growing emphasis in the broader field of implementation on clearly specifying and studying context in research studies and trial designs (Kemp et al., 2019; Pinnock et al., 2017). Further, we have observed in the literature that unique contextual factors in schools (such as those detailed by Owens et al., 2014, see above)—which, at their core, are not health service delivery organizations—tend to yield lower adoption rates for mental health EBP than might be observed in implementation research conducted in healthcare settings such as specialty mental health clinics or hospital settings (e.g., Connors et al., 2021; Nadeem et al., 2018). This may be especially true at more intensive levels of mental health service delivery (e.g., Tier 3) which are often seen as deviating from, or in addition to, the core educational mission of schools (Atkins et al., 2017). As a result of the unique contextual factors influencing implementation of mental health EBP in schools, research–practice partnerships that include school administrators, implementers and recipients working together to carefully consider implementation expectations and operationalizations of success are especially critical. A recent special issue of School Mental Health focused on these partnerships underscores their importance (Lawson & Owens, 2024).

Below, we outline an updated agenda for implementation research in schools focused on the four priority domains mentioned earlier. First, we describe the value of developing and testing pragmatic (i.e., rigorous, relevant, efficient) implementation strategies that are usable for implementation researchers and practitioners alike. Second, we emphasize the importance of identifying and measuring the proximal variables that explain the effects of implementation strategies on outcomes (i.e., implementation mechanisms). Third, we present frameworks and methods that can drive research on usability and contextual appropriateness of our best EBP through systematic, proactive redesign. Fourth, we explore the knowledge and studies we need to understand the best ways to prune or deliberately de-implement low-value or ineffective school mental health practices. The paper concludes with a discussion of methods and study designs that can be leveraged to address different aspects of the agenda.

Pragmatic Implementation Strategies

Definition and Rationale

In recent years, significant progress has been made in the identification and categorization of implementation strategies specific to advancing implementation science and practice in schools (Cook et al., 2019; Lyon et al., 2019b). This lays the foundation for more advanced implementation research that tailors and tests implementation strategies and their impacts on implementation and student outcomes. However, testing the impact of implementation strategies on implementation outcomes has been relatively slow to advance across sectors (Proctor et al., 2023) and the impact of this work will be limited if the strategies developed and studied are not practical in real-world school settings. Therefore, similar to Owens et al.’s (2014) original recommendation that researchers should work to identify efficient, “good enough” approaches to implementation, future research should prioritize developing and testing pragmatic implementation strategies. Pragmatic approaches are those that are relevant and meaningful to practitioners and feasible and efficient to use in the local context (Glasgow, 2013). Unfortunately, there has been a tendency for implementation researchers across settings to develop complex and high-intensity implementation strategies in an effort to maximize theoretical effects (Beidas et al., 2022), which can render the strategies difficult to use in the real world outside of the context of a research study. Prioritizing and testing truly pragmatic implementation strategies will help school mental health researchers avoid the potential pitfall of “recreating the research-to-practice gap” (Beidas et al., 2022; Westerlund et al., 2019) by developing strategies that cannot be used in real-world school settings.

State of the Science

Although research on pragmatic implementation strategies in school settings is currently limited, there have been recent advances in methods to adapt and optimize implementation strategies to be more pragmatic. For example, the Cognitive Walkthrough for Implementation Strategies (CWIS; Lyon et al., 2021) is a method to evaluate complex implementation strategies for usability issues that might inhibit their use. The CWIS has been applied to adapt Leadership and Organizational Change for Implementation (LOCI)—a complex implementation strategy to support implementation leadership and climate, developed in mental health and substance use treatment settings (Aarons et al., 2015, 2017)—for elementary school principals (Collins et al., 2024). Applying processes like cognitive walkthroughs to design implementation strategies or adapt them for the school context is one example of a promising method to move the field toward implementation strategies that are usable, feasible, and pragmatic within the school context. Additional methods relevant for developing pragmatic implementation strategies can be found in the review of methods and designs later in this paper.

Deploying pragmatic implementation strategies in schools also requires identifying the minimal dose needed to effect change: a central question in intervention research (Glasgow et al., 2014). Aligned with this goal, methods to adapt implementation strategies to the local context (e.g., individual schools, classrooms, or implementers) can support strategy pragmatism by optimizing them to the necessary intensity (Kilbourne et al., 2024). For example, a recent study evaluating implementation strategies to support school professionals’ delivery of CBT found that the most effective strategy was an adaptive approach that began with a relatively low-intensity implementation strategy and augmented it with a higher-intensity strategy for slow responder schools (Smith et al., 2022). These results signal the promise of adaptive implementation strategies as an approach to provide the most appropriate level of implementation support for a given context, thus improving their efficiency.

Critical Research Questions

Question 1: What partner-engaged methods can be used most efficiently and effectively to develop or select implementation strategies that are pragmatic within the school context?

Question 2: Which pragmatic implementation strategies are effective to support the delivery of mental health EBP in schools? Under what circumstances?

Question 3: Can adaptive approaches improve the efficiency of school mental health implementation strategies? Can strategies be effectively adapted to the individual implementer, in addition to the individual school?

Implementation Mechanisms

Definition and Rationale

Identification and evaluation of the pathways through which implementation strategies operate is a promising avenue for pursuing strategy pragmatism, as well as investigating other important implementation research questions in school mental health. Implementation mechanisms are the processes or events through which implementation strategies operate to bring about positive implementation outcomes (Lewis et al., 2018). Specification of mechanisms can allow for investigation of null effects, replication of positive findings, identification of which strategy components are inert or unnecessary, and adaptation of strategies to fit new contexts (Lewis et al., 2021b).

Ideally, implementation mechanisms are grounded in theory, such as adult learning or behavior change theories. With a strong theoretical foundation, study of implementation mechanisms can facilitate theory building and revision. For instance, the theory of planned behavior (TPB)—which states that behavioral intentions are driven primarily by attitudes, perceived social norms, and self-efficacy (Ajzen, 1991)—is the most commonly applied socio-cognitive theory for developing and testing implementation strategies (Godin et al., 2008; Lewis et al., 2020). The TPB closely relates to the original Owens et al. (2014) recommendation that implementation research in schools evaluate the extent to which implementer motivation and perceptions are malleable; and there are multiple instance of the TPB being applied in the identification of implementation determinants and design of implementation strategies in school mental health (e.g., Hugh et al., 2022; Lyon et al., 2021; Zhang et al., 2023). When applying implementation strategies based on the TPB, measurement of attitudes, social norms, and self-efficacy can shed light on the validity of the theory or the functioning of the strategy (e.g., if mechanisms change but are not associated with behavior change then the theoretical link might not be supported). Understanding implementation mechanisms also can allow for tailoring and streamlining of strategies to maximize the impact of components on these proximal mediators. Strategy components found to not operate on their putative mechanisms could be redesigned to increase their impact or considered for removal from a multifaceted implementation package.

Importantly, even effective implementation strategies have the potential to create implementation disparities when impacts vary across schools, and it is often necessary to adapt strategies to ensure consistent effects on implementation outcomes. For instance, a coaching strategy might effectively increase the adoption and high-fidelity delivery of an evidence-based SEL program in a well-resourced school, but is less effective improving the implementation in a school with fewer resources in the same district. The ASPIRE framework (Gaias et al., 2022), mentioned earlier, provides a series of concrete steps for strategy adaptation to reduce the likelihood that an implementation strategy will exacerbate health disparities by promoting equitable implementation. However, as detailed in ASPIRE, an understanding of implementation mechanisms is critical to facilitate this process because it allows for rapid confirmation that a strategy (e.g., coaching with performance feedback) continues to influence the proximal outcomes identified in its theory of change (e.g., fluency building; Massar et al., 2023).

State of the Science

Although the study of mechanisms is closely aligned with the interests of some sponsors of school mental health research—such as the National Institute of Mental Health’s Experimental Therapeutics approach (Frank et al., 2022)—systematic reviews of implementation mechanisms in mental and behavioral health indicate that this type of inquiry is rare (Lewis et al., 2020; Williams, 2016). Mechanistic implementation research in school mental health is particularly rare but increasing, with most examples coming from protocol papers for active projects. For instance, Kwak et al. (2022) are conducting a study to evaluate the mechanisms through which a variety of implementation strategies (e.g., teaming, training, and PDSA cycles) will operate to support the implementation of a mental health prevention program for school personnel. Turner et al. (2022) are conducting an evaluation of strategies for implementing PBIS with educators in rural schools (e.g., teaming, systems support, training, ongoing technical assistance, feedback) that attends explicitly to the mechanisms through which strategies operate (organizational readiness, school team functioning, psychological safety).

As school mental health researchers increasingly develop and test implementation strategies, they should explicitly specify and report their strategies (Moore et al., 2021) and conceptualize and evaluate the mechanisms through which those strategies promote implementation outcomes. To achieve this, the field would benefit from more consistent operationalization of mechanisms. We argue that mechanisms are best defined as the intrapersonal or interpersonal variables that change when a strategy is applied and not the application of strategy components themselves. For instance, for a coaching strategy that involves feedback to school clinicians, the provision of feedback might be a component of strategy fidelity (i.e., delivery as intended) but would not be considered a mechanism through which the strategy is effective. Instead, processes such as awareness of discrepancies between current states of practice and goal states might be considered. Building from this perspective, we propose a series of high-priority research questions.

Critical Research Questions

Question 1: Through which putative mechanisms do the most common implementation strategies in schools (e.g., training, coaching) operate?

Question 2: Where are there gaps between identified implementation mechanisms and available measurement instruments to evaluate those mechanisms?

Question 3: When effective, do implementation strategies operate through the same mechanisms across diverse school contexts?

Strategic Intervention Redesign

Definition and Rationale

Across service sectors, evidence suggests that not all EBP are equal with regard to their implementation potential (i.e., implementability). For instance, a study by Ahuna et al. (2023) found that 75% of the variability in youth mental health therapists’ intentions to implement specific EBP was accounted for by characteristics of the intervention rather than individual and organizational characteristics. Similarly, another study found that 57% of the variability in program start-up for 27 EBP focused on child, family, and adult well-being implemented across 1287 sites could be attributed to the programs’ characteristics (Alley et al., 2023). In schools, Eisman et al. (2020c) reported that teacher ratings (n = 171) of the design and packaging quality of a mental health curriculum were related to the fidelity with which the curriculum was delivered, especially for teachers with less experience. These findings underscore how intervention-level determinants are critical to implementation success. Despite this, innovation characteristics are an underemphasized “forgotten domain” in contemporary implementation science, receiving considerably less attention than individual and organizational determinants (Lewis et al., 2021a).

Although relatively little implementation research focuses on the innovation level, strategic intervention redesign, which addresses intervention-level determinants by making adaptations (i.e., planned, proactive modifications; Stirman et al., 2019) to create a more implementable intervention, is an important, yet underutilized, implementation strategy (Lyon & Bruns, 2019b). In school mental health, the innovations being implemented may range from specific components of evidence-based interventions (e.g., measurement-based care; Connors et al., 2022a) to complex, multi-level service models (Moore et al., 2024) such as positive behavioral interventions and supports (Sugai & Horner, 2006). No matter what their complexity, many of these interventions could benefit from prospective approaches to evaluate the extent to which they meet user needs and make changes to improve contextual fit, usability, and implementation potential (Lyon et al., 2020b). While many school mental health researchers do not attend explicitly to intervention design (or redesign) (Lyon & Bruns, 2019a), we have observed that purveyors of school-focused prevention and intervention programs that may not have a strong evidence base often attend carefully to the surface look and feel of their interventions, creating engaging and visually appealing materials. This can provide a competitive advantage for non-EBP that often drives building- and district-level adoption decisions.

State of the Science

While “there is no implementation without adaptation” (Lyon & Bruns, 2019b), adaptations of EBP often occur reactively (vs. prospectively) which may impact the extent to which they are fidelity consistent (Pettersson et al., 2024; Stirman et al., 2019). Many intervention adaptation or redesign models and approaches have emerged in implementation science, such as the Model for Adaptation Design and Impact (MADI; Kirk et al., 2020), the Adaptive Designs Accelerating Promising Trials Into Treatments (ADAPT-IT; Meurer et al., 2012), Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI; Brewer et al., 2024), and the Discover, Design/Build, Test (DDBT) framework (Lyon et al., 2019c). Other models and approaches have been developed with the goal of supporting professionals to make decisions about whether or not to adapt an intervention as well as how to evaluate an adaptation process (Miller et al., 2020). The Framework for Reporting Adaptations and Modifications to Evidence (FRAME)-based interventions is the most prominent example of a descriptive approach (Stirman et al., 2019). With the recent emergence of most of these models, we look forward to research evaluating their utility. One framework that has been applied repeatedly in school mental health to increase implementability and fit (e.g., Bearss et al., 2022; Liu et al., 2022a; Pfiffner et al., 2022) is the DDBT framework (Lyon et al., 2019c). Evaluation of the impact of this framework is underway in the multiple school and community-based projects supported by the NIMH-funded University of Washington ALACRITY Center (Munson et al., 2022).

Some of the most compelling approaches to redesign interventions draw heavily from the field of human-centered design (HCD), which uses information about the settings and individuals who will ultimately use a product to inform its (re)development (Norman & Draper, 1986). HCD and implementation science are increasingly being integrated in health services research (Chen et al., 2021; Dopp et al., 2019a; Lyon et al., 2023). Although it is common to associate HCD methods with digital innovations, application of those methods need not be limited to that medium. In the context of youth mental health services, Lyon et al. (2020b) detailed how HCD processes also can be brought to bear on other health services research products, including psychosocial EBP and implementation strategies in service delivery settings such as schools. Specifically, leveraging HCD can allow for implementation-focused intervention redesign to ensure EBP fit with schools’ and districts’ existing priorities, initiatives, and constraints. Due to the methods through which it elevates user voices (Rose, 2016), HCD also may create opportunities to help ensure that adaptations are culturally appropriate and equitable, although it has not historically been used in this way and cannot be assumed to achieve equity outcomes without careful reflection and evaluation (Stiles-Shields et al., 2022). Nonetheless, HCD models for psychosocial intervention redesign have great potential to improve contextual appropriateness through deep engagement with end users. In light of the growing focus on strategic intervention redesign in implementation science in general—and in school-based implementation research, in particular—we argue that future research in the school mental health space should increasingly explore the impact of prospective adaptations to improve fit. Below, we articulate specific research questions for considering how intervention design and redesign can advance implementation in school mental health.

Critical Research Questions

Question 1: To what degree do EBP in school mental health vary with regard to their usability and implementability? Are there common design problems that consistently inhibit large-scale adoption?

Question 2: What redesigns of EBP are needed to improve usability, adoption, and impact?

Question 3: Can HCD-driven approaches to redesign improve equitable implementation and outcomes?

De-Implementation and Pruning

Definition and Rationale

De-implementation is typically thought of as a systematic process focused on stopping the use of low-value, ineffective, or harmful practices (e.g., Johnson et al., 2018; Norton & Chambers, 2020). McKay et al.’ (2018) have outlined three primary cases in which interventions should be deimplemented: (a) intervention lacks evidence or is harmful, (b) alternative interventions are more effective or efficient, or (c) the issue of concern is resolved. In school mental health, similar factors may apply. In addition, the reality is that implementation is often a zero-sum game. Schools need to strategically make room for new initiatives as opposed to continually adding new initiatives, otherwise risking diluting efforts, and leading staff to be overwhelmed. This process of de-implementation (a.k.a., pruning; Cook et al., 2019) differs from a failure to sustain in that it is purposeful, with leaders considering the value of existing practices, current priorities, staffing and financial capacity, and the expertise of existing personnel and teams.

State of the Science

In health care more broadly, there has been growing interest in de-implementation, but this work is still in its infancy, focused on identifying existing frameworks (Nilsen et al., 2020), characterizing and developing de-implementation strategies (Ingvarsson et al., 2022; McKay et al., 2018), and defining outcomes (Prusaczyk et al., 2020). Within the context of school mental health, research to date focuses primarily on describing the processes through which schools make de-implementation decisions. The theoretical concept of “escalation of commitment,” is a decision-making bias where leaders continue to use low-value or ineffective programs. In such cases, administrators may continue implementation due to a range of external and internal pressures, and may attribute poor program performance not to the program itself but to implementation issues, leadership, or limitations of the performance indicators (Barrett et al., 2023). In other cases, districts may make the choice to cut programs that had been previously well supported due to limited resources. A case study of the de-implementation of a school-based trauma program over a two-year period highlighted the role of severe budget and staffing cuts and a district-level decision to refocus staff on providing mandated services (Nadeem & Ringle, 2016). The role of racial equity and culture in de-implementation decisions is understudied (Clinkscales et al., 2024). However, there is some evidence that schools do consider equity issues as they make de-implementation decisions. For instance, the increased uptake of restorative practices and culturally responsive, equity-driven social–emotional learning and classroom practices is often coupled with the de-implementation of exclusionary discipline practices that have disproportionate negative impact on minoritized students (e.g., Farr et al., 2020; Gregory et al., 2021).

As the science of de-implementation in school mental health evolves, it will be critical to examine how well emergent theories, strategies, and outcomes applied in health care apply to schools; identify unique factors that are inherent to the educational context; and design and test strategies that can support targeted de-implementation processes. Such work will help the field to develop theory and practice about what should drive decisions about what to de-implement, and how to apply decision-making frameworks (e.g., how to assess evidence, fit with the context, implementation and resource burden). To further understand de-implementation, mixed methods and rapid qualitative approaches (see Designs and Methods, below) may be useful to observe and characterize what decision-making looks like in routine practice (e.g., level at which decisions are made, who decides, perceptions about the intervention or program, what outcomes matter to decision makers, role of costs), and what de-implementation strategies are already being used.

Critical Research Questions

Question 1: How are schools and districts making decisions to de-implement existing programs and initiatives (e.g., factors considered, characteristics of programs de-implemented, decision-making processes, key decision makers)?

Question 2: What role do resources and costs play in de-implementation decisions? Specifically, what is the cost–benefit of keeping a current practice versus de-implementing it (i.e., costs of existing practices, opportunity costs, ongoing maintenance costs, costs of de-implementation)?

Question 3: What models or strategies might enhance decision-making and processes related to strategic deimplementation (e.g., measurement of existing practices before implementation and de-implementation)?

Designs and Methods to Support the Research Agenda

Over the past decade, a range of study designs and methods have been furthered in the broader implementation science field that can be leveraged to improve school mental health intervention adoption, implementation, and sustainment and to address the priority questions in each domain. We review here several of these methods—hybrid trial designs; sequential, multiple assignment, randomized trials (SMARTs); rapid qualitative approaches; and economic analyses—to provide additional direction for investigating some of the research questions articulated above (see Table 1). All of these designs can, and should, be applied in the context of strong research–practice partnerships to ensure relevance and representation of community voice.

Table 1.

Examples of Study Design or Method to Advance Implementation Science in SMH Research

Study design or method	Example research question	Example application
Hybrid	Implementation Mechanisms	Hybrid Type 3 multi-school site trial to primarily assess implementation strategy mechanisms across buildings and contexts and secondarily assess effectiveness of EBP on student outcomes
Effectiveness–Implementation Trial (Design)	Question 3: When effective, do implementation strategies operate through the same mechanisms across diverse school contexts?
SMART Trial (Design)	Pragmatic Implementation Strategies Question 3: Can adaptive approaches improve the efficiency of school mental health implementation strategies? Can strategies be effectively adapted to the individual implementer, in addition to the individual school?	Study designed to develop a multi-level adaptive implementation strategy (MAISY) and empirically evaluate the relative implementation outcomes associated with different combinations of strategies based on timing or implementer response to initial strategies provided
Rapid Qualitative Approaches	Strategic Redesign Question 2: What redesigns of EBP are needed to improve usability, adoption, and impact?	Exploratory study using a rapid ethnographic toolkit from www.impscimethods.org (a website fueled by the work of research centers funded by the National Institutes of Health) to understand the reasons why an intervention was adapted when implemented and inform potential redesign efforts
Cost/Economic Analysis	De-Implementation Question 2: What role do resources and costs play in deimplementation? Specifically, what is the cost–benefit of keeping a current practice versus de-implementing it?	Formative study applying the Cost of Implementing New Strategies (COINS) method for documenting costs associated with implementation strategies (Saldana et al., 2014) and recent resources provided by IES (https://ies.ed.gov/blogs/research/post/have-a-cost-analysis-to-plan-or-execute-we-have-a-mod-ule-for-that) prior to initiating a de-implementation effort

Hybrid Trials

Hybrid trial designs, briefly mentioned and recommended by Owens et al. (2014), simultaneously evaluate intervention effectiveness and implementation within the same study to shorten the translational research pipeline (Curran et al., 2022). A continuum of hybrid Type 1, 2, and 3 designs are available to suit the appropriate balance of emphasis on effectiveness (e.g., student outcomes) versus implementation outcomes (e.g., adoption, fidelity, feasibility). On one end of the continuum, Type 1 trials focus primarily on testing the effectiveness of an intervention, while also gathering information about implementation outcomes. On the other end of the continuum, Type 3 trials focus mainly on testing implementation outcomes associated with an implementation strategy or strategies, with a secondary goal of collecting effectiveness outcomes associated with the intervention. Finally, Type 2 trials have a dual focus: testing interventions and implementation strategies simultaneously (Curran et al., 2022). Increasingly, hybrid Type 1 trials are not considered “true” implementation research studies since implementation strategies and outcomes are observed, not empirically evaluated. Consistent with the National Institutes of Health’s Science of Implementation in Health and Healthcare (SIHH) study section and the journal Implementation Science, which do not review Type 1 trials (NIH, 2024; Wensing et al., 2021), we focus our discussion here on Type 2 and 3 hybrid trial designs as they have the most explicit implementation focus.

Hybrid trials have largely become the industry standard in implementation science, but are still underutilized in school-based research. Aside from a few notable examples of recently completed school-based Type 2 trials in the domains of computer-assisted interventions for autism (Pellecchia et al., 2020) and cognitive behavioral therapy (Eiraldi et al., 2024), nearly all hybrid trial examples we identified are study protocols for ongoing work. These include Type 2 trials testing a positive school climate intervention with an implementation facilitation strategy (Eisman et al., 2020a); testing a scalable implementation model for an evidence-based early childhood mental health intervention in Ugandan schools (Huang et al., 2022); testing a pragmatic, motivational enhancement implementation strategy (i.e., BASIS) for Trauma-Focused Cognitive Behavioral Therapy delivered in schools (Lyon et al., 2021); and testing a team science intervention alongside tailored implementation strategies (Kuriyan et al., 2021). Relevant Type 3 study protocols in school mental health include effectiveness–implementation trials of a co-designed, multi-component implementation strategy to support implementation of the PAX Good Behavior Game (Baffsky et al., 2022); implementation of a proactive classroom management strategy (Positive Greetings at the Door) via a teacher-focused version of BASIS (Lyon et al., 2024); and a comparison of implementation strategies for PBIS in rural settings (Turner et al., 2022).

SMART Designs

Another study design emerging in the field of implementation science in school mental health is the sequential, multiple assignment, randomized trial (SMART). These experimental designs can be applied—among other uses—to empirically develop an adaptive intervention using multiple stages of randomization to learn how the intervention can be optimized or tailored for service recipients (Advancing the science of adaptive interventions, 2024; Lei et al., 2012). SMARTs have been identified as an ideal fit for the future of school mental health intervention research, especially within a multi-tiered system of support (August et al., 2018). Application of SMART designs to school mental health implementation research is still in its infancy, but a notable example is the Adaptive School-based Implementation of Cognitive Behavioral Therapy (ASIC) study, mentioned above in the section on pragmatic strategies (Kilbourne et al., 2018). In that SMART, augmentation decisions were based on response status of schools after eight weeks and results indicated that an adaptive strategy that included training/TA with facilitation resulted in the highest average CBT delivery (Smith et al., 2022).

A variation of the SMART design that school mental health implementation scientists may want to follow is the multi-level implementation SMART, which can be used to construct an optimized multi-level adaptive implementation strategy (MAISY). MAISYs involve randomizations at multiple levels to learn how to sequence and adapt implementation strategies at the system (e.g., school district), unit (e.g., school) and implementer (e.g., teacher, school practitioner) level (Solving implementation challenges, 2024). Although MAISYs are just emerging in outlets such as conference proceedings (e.g., Connors et al., 2022b), a wide net of applied implementation research questions about which implementation strategies are most effective, when, for whom (based on student, school implementer, and/or school setting characteristics) can be pursued via application of these innovative SMART designs in school mental health.

Rapid Qualitative Approaches

Qualitative research and mixed methods are central to implementation science (Hamilton & Finley, 2020; Palinkas, 2014). Yet, the time lag and resources to conduct traditional qualitative research can run counter to the goals of implementation research to speed the evidence-to-practice gap. Rapid qualitative analyses vary in technique, but typically involve a team of researchers conducting iterative data collection and analysis and/or rapid synthesis of results to increase efficiency (Vindrola-Padros & Johnson, 2020). Importantly, rapid qualitative approaches reduce time and cost but do not compromise rigor or value. Comparisons of rapid to traditional qualitative approaches have found a range of modest to substantial time saved in the interpretation and analysis phases (due to variation by research teams), large savings in transcription costs, and very similar amounts of data collected, results, and recommendations (Gale et al., 2019; Nevedal et al., 2021; Taylor et al., 2018). We were challenged to find examples of rapid qualitative approaches applied to school mental health implementation research; however, within the broader implementation science literature there are numerous examples of success using these methods in resource-constrained settings (c.f., Moran et al., 2023; Sperber et al., 2019).

Economic/Cost Evaluation

Economic evaluations include analyses to minimize cost, improve cost effectiveness, study cost utility, and report cost–benefit (Kim et al., 2022). Economic evaluations have been increasingly emphasized in healthcare implementation research (Eisman et al., 2020b) and are likely to have great value in school implementation research given the resource constraints and intervention overload in most school systems. To our knowledge, cost analyses have been conducted in academic intervention research (Barrett et al., 2020) as well as school mental health research (Abou Jaoude et al., 2024; Bradshaw et al., 2020; Malik et al., 2021), but this is still an emerging method in education research, including in school mental health implementation science. The Institute of Education Sciences (IES), a primary funder of school mental health research, recently began requiring cost analyses in nearly all of their grant mechanisms as a component of its Standards for Excellence in Education Research (SEER) principles (IES, 2022). Researchers interested in learning more about these methods may be interested in published guidance for implementation scientists’ use of mixed methods findings to inform economic evaluations (Dopp et al., 2019b).

Conclusion

In this paper, we sought to reflect on the decade that has passed since Owens and colleagues (2014) articulated an agenda for implementation science in school mental health. Although some significant progress has been made, much work remains as the field continues its shift from the identification of implementation determinants to developing, testing, and understanding the mechanisms of implementation strategies, including those that are pragmatic and adaptive. As with the original research agenda, full consensus surrounding a blueprint for future implementation research in school mental health is neither the goal nor the expectation of this paper. Nevertheless, it is our hope that some shared understanding of evolving research priorities and novel designs/methods can continue to drive complementarity and synergy across projects.

Throughout this paper, we have attempted to underscore opportunities to advance equitable implementation in school mental health. While implementation of EBP in schools has the potential to support equity, all implementation processes carry risks of maintaining or inadvertently exacerbating inequities (Liu et al., 2019), which necessitates attention. As with many other fields, equity-explicit and anti-racist implementation science has rapidly accelerated over the past five years (Baumann et al., 2023; Shelton et al., 2021). This includes the development of multiple equity-oriented implementation frameworks (e.g., Eslava-Schmalbach et al., 2019; Woodward et al., 2019), processes to surface and address equity concerns in implementation strategies (Gaias et al., 2022), and guidance about how to incorporate equity into the evaluation of context and measurement of implementation outcomes (Brownson et al., 2021). Relatively few but outstanding examples of this work can be found within the school mental health implementation literature. For instance, Renshaw and Phan (2023) recently proposed an expansion of implementation reporting recommendations for school-based mindfulness interventions to center equity (e.g., through explicit attention to principles that research teams might employ to center diversity, equity, and inclusion during intervention adaptation). The previously mentioned ASPIRE framework (Gaias et al., 2022) for school-based implementation strategy adaptation serves as another great example. As school mental health implementation research continues to evolve over the next decade, we hope to see expanded emphasis on ways to promote equitable implementation.

The field of school mental health sits squarely at the intersection of two worlds: health and education. The pace with which implementation science in health has advanced continues to far exceed that in education, a fact evidenced by the number of healthcare-related citations in the current paper, especially in more recently emerging areas of inquiry (e.g., strategic intervention redesign, implementation mechanisms). Nevertheless, it is important to acknowledge that there is strong interest in implementation science within the larger education research community (Gaias et al., 2023) and that examples in education outside of the mental health domain also are rapidly expanding (e.g., Komesidu & Hogan, 2023; Moir, 2018; Olswang & Prelock, 2015). Training programs such as the IES-funded Research Institute for Implementation Science in Education (RIISE) (https://ies.ed.gov/funding/grantsearch/details.asp?ID=4646) have emerged to match this interest and support educational scholars in developing rigorous implementation research portfolios. As these efforts continue to advance, it will be important that implementation research in school mental health—and education more generally—does not simply adopt the health-oriented conceptualization of implementation wholesale and uncritically. Instead, we envision continued development of an education-specific science of implementation that recognizes the unique constraints and opportunities in schools and leverages the tremendous wealth of information that has been generated in health care, while avoiding some of the potential pitfalls (Beidas et al., 2022). Overall, we are highly optimistic about the future of implementation science in school mental health and wish both budding and established researchers in this space, “fair winds and smooth sailing.”