Abstract
Just as there is a robust science that supports development and rigorous testing of clinical innovations, the emerging field of implementation science is developing new theory-based knowledge regarding a growing portfolio of meticulously tested implementation strategies that seek to improve uptake of evidence-based practices by targeting barriers at multiple levels within health care settings.
Studying and documenting implementation strategies associated with uptake during the development and trial of a clinical innovation could subsequently position the researcher for a more seamless transition and handoff of the innovation to clinical and operational leaders.
The objective of this manuscript is to introduce the concept of implementation strategies: what they are; the rigor with which they are defined and applied to address barriers to clinical innovation adoption; how strategy selection may vary based on contextual, innovation, and recipient factors; how to document the application of strategies over the course of an implementation study; and how testing their effectiveness is the focus of implementation research trials.
1. Introduction
Basic science and clinical trial researchers can spend years, even decades, developing interventions or innovations that have limited, if any, implementation in routine clinical care; often despite multiple studies documenting clinical efficacy or effectiveness. In fact, researchers have estimated a 17-year gap from the time that a clinical innovation has proven effective to when it is provided routinely to patients, with only half of evidence-based practices being implemented into care at all (Institute of Medicine, 2001; Balas & Boren, 2000). This lack of uptake is associated with a substantial cost not only to health care systems and patients through the lack of advancement in clinical quality of care, but also to the researchers and funding agencies that have dedicated significant time and resources to advancing scientific knowledge.
This “quality chasm” has led to the rigorous study of how to facilitate and improve the implementation of evidence-based innovations into routine clinical care (Institute of Medicine, 2001). The National Institutes of Health define implementation as the “use of strategies to adopt and integrate evidence-based health interventions and change practice patterns within specific systems” (Institute of Medicine, 2009; National Institutes of Health, 2016). Just as there is a robust science that supports development and rigorous testing of clinical innovations, the emerging field of implementation science is developing new theory-based knowledge regarding a growing portfolio of rigorously tested implementation strategies that seek to improve uptake of evidence-based practices by targeting barriers at multiple levels within health care organizations, typically through partnerships between clinical operations and researchers (Aarons et al., 2014). Specifically, implementation science aims to:
develop effective strategies for implementing evidence-based practices, thereby improving health-related processes and outcomes;
produce generalizable knowledge regarding these strategies by understanding the processes, barriers, and facilitators that influence implementation success or failure; and
develop, test, and refine relevant theories, conceptual frameworks and measures to advance the science of implementation (Grimshaw et al., 2012).
The objective of this manuscript is to introduce the concept of implementation strategies: what they are; the rigor within which they are defined and applied to address barriers to clinical innovation adoption; how strategy selection may vary based on contextual, innovation, and recipient factors; how to document them over the course of an implementation study; and how testing their effectiveness is the focus of implementation trials.
2. Definitions for implementation science terms used in this manuscript
Throughout, this manuscript uses the term “innovation” broadly. In a clinical setting, this may be a new clinical intervention such as a brief psychotherapy or the application of an existing medication in a new setting, such as the use of buprenorphine for opioid/alcohol addiction in primary care clinics. Within implementation science, it is the implementation strategies that are considered the “interventions.” While clinical outcomes are studied in implementation efforts, it is the implementation strategy or strategies applied that are the primary focus of study. Implementation strategies are defined as approaches or techniques used to enhance the adoption, implementation, sustainment, and scale-up (or spread) of an innovation (Proctor et al., 2013; Powell et al., 2019a). Discrete implementation strategies are a single approach or technique, such as distributing educational materials, informing local opinion leaders, revising professional roles, or using clinical reminders. Typically, however, the challenges involved with effectively implementing a clinical innovation require the use of multifaceted implementation strategies that combine two or more discrete strategies. For example, a multifaceted strategy to implement a clinical innovation such as buprenorphine for opioid/alcohol addiction in primary care may include (a) educating primary care providers about the benefits and evidence that supports the use of buprenorphine in primary care and (b) establishing a registry of patients with alcohol and opioid disorder diagnosis that can link to a (c) clinical reminder about considering the use of buprenorphine when the patient’s record is accessed. In this example, education of primary care providers (discrete strategy), establishment of a patient registry (discrete strategy), and use of a clinical reminder (discrete strategy) are a collection of discrete strategies that, when applied together, represent a multifaceted implementation strategy to support use of buprenorphine for opioid/alcohol addiction.
3. A taxonomy of implementation strategies- the ERIC study
Every innovation has unique characteristics that reflect and interact with the context in which the clinical innovation is being implemented, as well as the individuals that will be administering or receiving the innovation (Rogers, 2003). As described above, discrete implementation strategies may serve as elements of a broader, multifaceted implementation strategy that is hypothesized to be responsible for improving implementation outcomes. When a discrete strategy is tailored to the content of a specific innovation, the action or process represented by the strategy can appear to be so content specific (e.g., clinical reminders to wash hands or use other infection control procedures that are posted in the care environment) that the relevance of the discrete strategy to other innovations may be lost (e.g., clinical reminders embedded in the electronic medical record). Thus, as with other sciences, implementation science strives to characterize its variables with sufficient levels of abstraction to support aggregating knowledge obtained through multiple studies. Such a common lexicon is essential to support replication efforts, critical reviews, and syntheses of the existing literature.
The Expert Recommendations for Implementing Change (ERIC) project applied a rigorous consensus development process, in part, to address this need for a common nomenclature for discrete implementation strategies. The ERIC consensus development process resulted in a compilation of 73 discrete implementation strategies and their definitions (Waltz et al., 2014; Powell et al., 2015). This process also identified nine thematic clusters of the 73 discrete implementation strategies that may be useful for organizing related strategies across studies (Waltz et al., 2015). Table 1 presents these clusters and select examples of the discrete strategies within them.
Table 1.
USE EVALUATIVE AND ITERATIVE STRATEGIES | |
---|---|
Strategy | Example |
Assess for readiness and identify barriers and facilitators | While this strategy may be applied throughout an implementation process, it is particularly useful before starting formal implementation. This assessment is typically guided by a conceptual framework or model and is used to identify factors that may influence implementation. For example, if there is poor leadership support, those implementing the innovation may want to focus on leadership engagement and education about the innovation. Alternatively, the assessment may determine that some staff at the site have prior experience in using the clinical innovation and may be willing to serve as an implementation champion. |
Audit and provide feedback | Monitoring the degree to which an innovation is applied and providing that information to key stakeholders at an implementation site can be a powerful tool. For example, if a site is implementing an evidence-based psychotherapy, noting the number and percentage of patients who receive the new therapy, and which clinicians are providing the therapy, can be used to track progress, identify new providers to engage in using the evidence-based psychotherapy, and reward those that are providing the psychotherapy. |
PROVIDE INTERACTIVE ASSISTANCE | |
Strategy | Example |
Provide clinical supervision | Ensuring that a new clinical innovation is implemented in a way it can achieve outcomes consistent with efficacy and effectiveness trials frequently requires clinical supervision of the innovation. In the example provided above, implementing a new evidence-based psychotherapy, clinical supervision may be provided through formal training followed by recording cases and ongoing supervision by an expert in the practice. |
Facilitation | Facilitation is a multifaceted strategy that applies a variety of discrete strategies (e.g., audit and feedback, conducting educational meetings, identifying champions) depending on what is needed given the context and characteristics of those that are providing (clinicians) and receiving (patients) the innovation. For example, at one site where providers are skeptical of the new clinical innovation, a facilitator may want to conduct educational meetings early in the implementation process to present not only the research evidence that supports the clinical innovation but also testimonials from providers at the site that are early adopters of the innovation. At another site, it may be most important to help the site reorganize their clinical teams so clinical roles that support uptake of the clinical innovation are clearly defined. We provide more information on facilitation in the case example in Section 6. |
ADAPT AND TAILOR TO CONTEXT | |
Strategy | Example |
Promote adaptability | Identifying ways that a clinical innovation can be adapted to meet local needs can be an essential component of successful implementation. When adapting clinical innovations, it is critical to identify and ensure that core components of the innovation associated with clinical outcomes are maintained for fidelity, while those that may be mutable are assessed and changes are considered that could increase integration of the innovation into the clinical setting. |
Tailor strategies | It may be necessary to also adapt and tailor discrete implementation strategies to address barriers and leverage facilitators identified during the implementation process. For example, when providing educational meetings about the clinical innovation (see example below) an in-person presentation may not be feasible. In this case, a virtual presentation (e.g., via tele-video) may be needed. |
DEVELOP STAKEHOLDER INTERRELATIONSHIPS | |
Strategy | Example |
Identify and prepare champions | Common to most implementation efforts is identifying and preparing clinicians and/or other staff who dedicate themselves to leading, supporting and marketing an implementation effort to overcome indifference or resistance that the innovation may provoke in an organization. |
Identify early adopters | By identifying early adopters at the local site or other settings, both those charged to begin implementing the clinical innovation in their care and those that are already applying it can learn and even be inspired by their experiences. |
Recruit, designate, and train for leadership | Change efforts require certain types of leaders, and organizations may need to recruit accordingly rather than assuming their current personnel can implement the change. For example, designated change leaders might include an executive sponsor and a day-to-day manager of the effort. |
TRAIN AND EDUCATE STAKEHOLDERS | |
Strategy | Example |
Conduct educational meetings | Educational meetings typically occur early in the implementation process and may be provided through formal didactic sessions or through small group or even individual educational efforts. The value of these meetings includes ensuring that all key administrative and clinical staff are aware of the implementation effort, there is a common understanding of the components of the clinical innovation (e.g., what it encompasses, the target clinical population, the evidence that supports its application) and there is an opportunity about implementation innovation itself. |
Create a learning collaborative | Learning collaboratives can be a valuable strategy through which clinical and operation staff from a clinic, team, or site that may be further along in the implementation process or who have experienced similar implementation barriers can share their ‘lessons learned’. |
SUPPORT CLINICIANS | |
Strategy | Example |
Create new clinical teams | Embedding a new innovation into an existing clinical setting may necessitate the creation of new clinical teams. For example, depending on the clinical innovation, it may be advisable or necessary to change who serves on the clinical team, adding different disciplines and different skills to make it more likely that the clinical innovation is delivered (or is more appropriately delivered). |
Revise professional roles | Revising professional roles includes the expansion of roles to cover provision of the clinical innovation and the elimination of service barriers to care, including personnel policies. |
ENGAGE CONSUMERS | |
Strategy | Example |
Prepare patients/consumers to be active participants | Patients are also a key person involved in the implementation of a clinical innovation. Preparing consumers to inquire about specific practices can involve asking questions and educating patients/consumers about the existence of treatments supported by evidence, as well as explicitly inviting them into the process of treatment decision-making. |
Intervene with patients/consumers to enhance uptake and adherence | This strategy may include patient/consumer reminders and financial incentives to attend appointments. Feedback regarding patient/consumers’ understanding and use of the treatment may also provide important information concerning the degree to which the innovation is being delivered with fidelity. |
UTILIZE FINANCIAL STRATEGIES | |
Strategy | Example |
Fund and contract for the clinical innovation | Governments and other payers of services may issue requests for proposals to deliver the innovation, use contracting processes to motivate providers to deliver the clinical innovation, and subsequently develop new funding formulas that make it more likely that providers will deliver the innovation. For example, when implementing collaborative care management for mental health services in primary care, a highly evidence-based practice, implementation slowed beyond research studies until the provision of care management by nurses was reimbursed by Medicare, the Department of Veterans Affairs and insurance companies. |
Develop disincentives | Some implementation efforts provide financial disincentives for failure to implement or use the clinical innovations. For example, this strategy could include tying promotion decisions to the use of certain innovations. |
CHANGE INFRASTRUCTURE | |
Strategy | Example |
Change record systems | An implementation effort may change records systems to allow better assessment of implementation or clinical outcomes. These changes may include modifying the format of progress notes and treatment plans to enhance documentation that the clinical innovation is being implemented. |
Mandate change | Leadership and/or governing organizations may declare the priority of the innovation and their determination to have it implemented in terms of a mandate. It is important to ensure that the individuals mandating the change have the power to do so, as implementers often lack such authority. For example, when the Joint Commission on Accreditation of Healthcare Organizations mandated that outcome measures be regularly collected in all settings under the Behavioral Health Standards (i.e. SUD specialty care, Residential, CWT), efforts by the VA to implement measurement-based care in mental health settings incorporated this mandate as a strategy to encourage rapid change in clinical practice. |
The ERIC compilation of implementation strategies is being used both prospectively (Huynh et al., 2018; Lyon et al., 2017) and retrospectively (Rogal et al., 2017; Perry et al., 2019) to provide a more comprehensive accounting of the discrete strategies employed in implementation trials. Historically, more highly controlled implementation trials have focused on the use of a small number of discrete implementation strategies (Grimshaw et al., 2005; Mazza et al., 2013). In contrast, implementation trials that have focused more heavily on use of formative evaluation techniques (Stetler et al., 2006), work logs, and/or the ERIC compilation to capture strategy use have identified the utilization of large numbers of strategies (BootsMiller et al., 2004; Hoagwood et al., 2014; Hysong et al., 2007; Magnabosco, 2006; Powell et al., 2016; Boyd et al., 2018; Bunger et al., 2017; Proctor et al., 2019; Rogal et al., 2017). Thus, some of the early benefits of the ERIC compilation to implementation science is that it prospectively supports consideration of a broad array of strategies organized across 9 thematic clusters, and retrospectively it supports a broad accounting of the strategies utilized.
4. Selecting an implementation strategy
As described in “Clarity out of Chaos: Use of Theory in Implementation Research” (Damschroder, 2019), included in this special journal issue, multiple domains and constructs must be considered when identifying which strategy or collection of discrete strategies is likely to be needed to support implementation of a clinical innovation within a particular clinical setting. With 73 distinct implementation strategies available for consideration (Powell et al., 2015), researchers and implementers may find it very challenging to contemplate which strategy, or collection of strategies, to use in a given effort to put a clinical innovation into practice. Using an implementation science framework or theory to help inform these decisions can potentially make this task less daunting by guiding the: (a) understanding of factors or determinants that may influence implementation, and (b) selection of implementation strategy (or strategies if multifaceted) (Waltz et al., 2015; Sales et al., 2006). More specifically, an implementation science framework/theory can help the researcher to: identify promising implementation strategies; identify or develop complementary improvement tools to support implementation; increase the probability for success in implementing the clinical innovation; and confirm or propose refinements to the framework/theory based on results, thereby contributing to the evidence base for the value and applicability of the framework/theory (Sales et al., 2006).
Some implementation science frameworks propose a specific strategy as an integrated component. For example, the integrated “Promoting Action on Research Implementation in Health Services” (i-PARIHS) framework specifically proposes use of “facilitation” to guide and support clinical staff through change processes or contextual challenges to implementation (Harvey and Kitson, 2016). Similarly, planned action models such as “Replicating Effective Programs” (REP) may be useful, as they specify a stepwise approach (strategy) to be taken within stages in the process of implementing a clinical innovation into practice (Kilbourne et al., 2007; Institute for Healthcare Improvement, 2017). Other frameworks such as the “Consolidated Framework for Implementation Research” (CFIR) are less prescriptive in terms of proposing specific implementation strategies, but include a domain (i.e., the CFIR “Process” domain) that addresses broad processes of implementation effort (Damschroder et al., 2009). Selection of a particular framework, theory, or model to guide implementation efforts should be done only after careful consideration of the specific goals and scope of an initiative (Nilsen, 2015).
Selection of an implementation strategy should also be informed by an assessment of the determinants of current practice within the targeted setting, including identification of implementation barriers and facilitators that may influence uptake of the clinical innovation. Formative evaluation (FE) is a rigorous assessment process typically involving collection of qualitative and quantitative data to identify the determinants of current practice, barriers, and facilitators for a practice change or implementation of a clinical innovation (Stetler et al., 2006). This type of assessment is sometimes referred to as a needs assessment of factors to be considered and addressed in developing, tailoring, and operationalizing an implementation strategy. FE can be useful for informing initial selection of an implementation strategy for a given study and/or for refining that strategy during the course of the study based on what is being learned to maximize potential for success (Stetler et al., 2006).
5. Documenting and Reporting Implementation Strategies
Building a stronger evidence base for implementation strategies requires that their use be contemporaneously tracked and that they be reported in the literature with sufficient detail (Michie et al., 2009; Powell et al., 2019b; Proctor et al., 2013). Much like clinical protocols, it is recommended that implementation strategies be carefully documented and any changes occurring during the course of implementation noted. However, this can be difficult given the iterative nature of implementation. Even if implementation strategies are detailed in a study protocol or trial registry, it is often unrealistic to expect that they will not need to be adapted or altered as unanticipated challenges or barriers emerge during the implementation process (Aarons et al., 2011; Dunbar et al., 2012; Hoagwood et al., 2011). These changes are likely to occur within and between implementing sites in research studies and applied efforts (Boyd et al., 2018; Bunger et al., 2017; Rogal et al., 2017), and without rigorous methods for tracking implementation strategy use, efforts to understand what strategies were used and whether or not they were effective can be stymied. Poor reporting can cloud the interpretation of results, precluding replication in research and practice, and limiting the ability to synthesize findings across studies (Michie et al., 2009; Proctor et al., 2013).
A number of guidelines focus specifically on reporting implementation strategies in enough detail so that they can be replicated in research and/or practice (Albrecht et al., 2013; Bragge, et al., 2017; Colquhoun et al., 2014; Hoffman et al., 2014; Patient Centered Outcomes Research Institute, 2019; Proctor et al., 2013; Workgroup for Intervention Development and Evaluation Research, 2008). Proctor et al. (2013) recommend naming and defining strategies in ways that are consistent with the published literature, and carefully operationalizing each discrete or component strategy by specifying: 1) actor(s), 2) action(s), 3) action target(s), 4) temporality (i.e., timing and sequencing), 5) dose, 6) implementation outcomes affected, and 7) theoretical, empirical, or pragmatic justification. Bunger et al. (2016) provide an applied example of reporting a multifaceted implementation strategy, detailing the 11 components of a learning collaborative according to the Proctor et al (2013) guidance. This guidance is consistent with the Patient Centered Outcomes Research Institute’s (2019) recently released Standards for Studies of Complex Interventions, which can be applied to clinical innovations being evaluated as well as the implementation strategies used to integrate them into routine care. Use of the Proctor et al (2013) guidelines in reporting implementation strategies helps ensure that strategies can be optimized over time and that effective strategies can be replicated in research and practice. As an applied example, we report the previously discussed facilitation strategy, according to the Proctor et al (2013) guidelines.
6. Case Example
In this example, the implementation strategy under study was facilitation, which is defined within the ERIC taxonomy as, “A process of interactive problem solving and support that occurs in a context of a recognized need for improvement and a supportive interpersonal relationship” under the “provide interactive assistance” cluster (Powell et al., 2015). Implementation facilitation is a broad strategy comprised of multiple discrete strategies. Which discrete strategy is applied at a given time is based on the needs of the setting context, the clinical innovation being implemented, and the individuals that will be using and receiving the innovation. A Department of Veteran Affairs (VA) funded study (Kirchner, 2014) tested the effectiveness of the implementation facilitation strategy on the uptake of primary care mental health integration (PC-MHI). PC-MHI, within the VA, is a blend of care management and colocated, collaborative care in which mental health providers are co-located within primary care clinics to provide increased access to mental health services and consultation, early identification and intervention for mental health concerns, and elimination of barriers to mental health care (Possis, 2016). The study was guided by the ‘Promoting Action on Research Implementation in Health Services’ (PARIHS) framework (an earlier version of the i-PARIHS framework described above) (Harvey & Kitson, 2016). Within this framework, facilitation is conceptualized as the “active ingredient” (strategy) used to address barriers and leverage enabling factors to enhance chances for successful implementation. The mixed methods study used a multisite, quasi-experimental design with nonequivalent comparison groups. Eight primary care (PC) clinics from two VA networks received implementation facilitation. These sites were compared with eight matched clinics in two matched networks that received national programmatic support only, comprised primarily of education and national calls that provided technical support.
To ensure that the implementation facilitation strategy was fully documented, the team applying facilitation used a structured tracking log to document the type of activities they conducted, date of the activity, individuals participating in the activities (e.g., clinical leadership, providers), and time that the activities took (within 15-minute intervals). This was provided to the project’s evaluation team on a weekly basis. In addition, the evaluation team conducted monthly qualitative debriefings with the facilitators to document how the strategy was executed through facilitators’ activities and use of other discrete implementation strategies, the rationale for each activity or strategy applied, results of their application, as well as the local context in which the activities and strategies were applied.
The principal outcome of the study was the number of PC-MHI encounters at the implementation facilitation (IF) sites compared to those receiving national support alone. In the first 6-month period after completing implementation of PC-MHI, PC patients at IF clinics had nine times the odds (OR=8.93, p<0.001) of also being seen in PC-MHI compared to patients at non-IF clinics (Kirchner, 2014). Thus, this multifaceted implementation facilitation strategy was shown to be an evidence-based approach to support implementation of a complex clinical innovation in sites with barriers within the setting (context) and/or among those using the innovation (recipients).
7. Handoff of strategies as a package with the innovation
As described earlier in this manuscript, the ability to integrate findings about barriers to and facilitators of an innovation’s uptake, and application of implementation strategies to address or leverage these factors, can increase chances for implementation success and decrease the time to systematic innovation uptake. One can imagine that taking steps to identify likely barriers to an innovation’s future uptake and addressing them early on during innovation development and efficacy testing could result in a clinical trialist being poised to move forward more expeditiously and successfully in subsequent effectiveness and implementation research. Key actions like documenting patient and provider experiences (recipients) during the efficacy trial and identifying potential strategies to address barriers and leverage facilitators to implementation can directly inform and improve the design of subsequent effectiveness studies. Likewise, studying and documenting the implementation strategies applied in effectiveness trials and how they varied based on differing contexts and settings could subsequently position the researcher for a more seamless transition and ultimate handoff of the innovation to clinical and operational leaders in diverse healthcare or community settings.
Highlights.
The application of theory-based implementation strategies can improve the uptake of clinical innovations.
Testing the effectiveness of implementation strategies is a focus of implementation research studies.
Early identification and documentation of effective implementation strategies during the development and study of a clinical innovation can improve uptake into healthcare settings
Acknowledgement
The authors would like to thank Nyssa Curtis for her contribution in editing and preparing this manuscript for publication.
Funding
This work was supported by Department of Veterans Affairs grant QUE 15–289.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- Aarons GA, Hurlburt M, & Horwitz SM (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23. 10.1007/s10488-010-0327-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Aarons GA, Ehrhart MG, Farahnak LR (2014). The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership. Implement Science , 9:45. doi: 10.1186/1748-5908-9-45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Albrecht L, Archibald M, Arseneau D, & Scott SD (2013). Development of a checklist to assess the quality of reporting of knowledge translation interventions using the Workgroup for Intervention Development and Evaluation Research (WIDER) recommendations. Implementation Science, 8(52), 1–5. 10.1186/1748-5908-8-52 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Balas EA, Boren SA Yearbook of Medical Informatics: Managing Clinical Knowledge for Health Care Improvement. Stuttgart, Germany: Schattauer Verlagsgesellschaft mbH; 2000 [PubMed] [Google Scholar]
- BootsMiller BJ, Yankey JW, Flach SD, Ward MM, Vaughn TE, Welke KF, Doebbeling BN: Classifying the Effectiveness of Veterans Affairs Guideline Implementation Approaches. American Journal of Medical Quality 2004, 19(6):248–254 [DOI] [PubMed] [Google Scholar]
- Boyd MR, Powell BJ, Endicott D, & Lewis CC (2018). A method for tracking implementation strategies: An exemplar implementing measurement-based care in community behavioral health clinics. Behavior Therapy, 49, 525–537. 10.1016/j.beth.2017.11.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bragge P, Grimshaw JM, Lokker C, Colquhoun H, & The AIMD Writing/Working Group. (2017). AIMD - a validated, simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies. BMC Medical Research Methodology, 17(38), 1–11. 10.1186/s12874-017-0314-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brownson RC, Colditz GA, & Proctor EK (Eds.). (2017). Dissemination and Implementation Research in Health: Translating Science to Practice (2 edition). New York, NY: Oxford University Press. [Google Scholar]
- Bunger AC, Hanson RF, Doogan NJ, Powell BJ, Cao Y, & Dunn J. (2016). Can learning collaboratives support implementation by rewiring professional networks? Administration and Policy in Mental Health and Mental Health Services Research, 43(1), 79–92. 10.1007/s10488-014-0621-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bunger AC, Powell BJ, Robertson HA, MacDowell H, Birken SA, & Shea C. (2017). Tracking implementation strategies: A description of a practical approach and early findings. Health Research Policy and Systems, 15(15), 1–12. 10.1186/s12961-017-0175-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- Colquhoun H, Leeman J, Michie S, Lokker C, Bragge P, Hempel S, McKibbon K, Peters G, Stevens K, Wilson M. and Grimshaw J. (2014). Towards a common terminology: a simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies. Implementation Science, 9(1). 10.1186/1748-5908-4-50 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Curran GM, Bauer M, Mittman B, Pyne JM, & Stetler C. (2012). Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50(3), 217–226. 10.1097/MLR.0b013e3182408812 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, & Lowery JC (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. doi: 10.1186/1748-5908-4-50 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dunbar J, Hernan A, Janus E, Davis-Lameloise N,, Asproloupos D, O’Reilly S, Timoshanko A, Stewart E, Bennett C, Johnson G. and Carter R. (2012). Implementation salvage experiences from the Melbourne diabetes prevention study. BMC Public Health, 12(1). 10.1186/1471-2458-12-806 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Going Lean in Health Care. IHI Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement; 2005. www.IHI.org. Accessed March 6, 2017. [Google Scholar]
- Grimshaw JM, Eccles M, Lavis J, Hill S, Squires J. (2012). Knowledge translation of research findings. Implement Science, 7:50. doi: 10.1186/1748-5908-7-50 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, & Donaldson C. (2005). Effectiveness and efficiency of guideline dissemination and implementation strategies. International Journal of Technology Assessment in Health Care, 21(1). DOI: 10.1017/S0266462305290190 [DOI] [PubMed] [Google Scholar]
- Harvey G, & Kitson A. (2016). PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implementation Science, 11(1), 33 10.1186/s13012-016-0398-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hoagwood KE, Olin SS, Horwitz S, McKay M, Cleek A, Gleacher A, Lewandowski E, Nadeem E, Acri M, Chor KHB, et al. (2014). Scaling up evidence-based practices for children and families in New York State: toward evidence-based policies on implementation for state mental health systems. Journal of Clinical Child and Adolescent Psychology, 43(2):145–157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hoagwood KE, Chaffin M, Chamberlain P, Bickman L, & Mittman B. (2011). Implementation salvage strategies: Maximizing methodological flexibility in children’s mental health research Panel presentation presented at the 4th Annual NIH Conference on the Science of Dissemination and Implementation, Washington, D. C. [Google Scholar]
- Hoffman TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, Michie S. (2014). Better reporting of interventions: Template for intervention description and replication (TIDieR) checklist and guide. BMJ, 348(g1687), 1–12. 10.1136/bmj.g1687 [DOI] [PubMed] [Google Scholar]
- Huynh AK, Hamilton AB, Farmer MM, Bean-Mayberry B, Stirman SW, Moin T, & Finley EP. (2018) A Pragmatic Approach to Guide Implementation Evaluation Research: Strategy Mapping for Complex Interventions. Front. Public Health 6:134. doi: 10.3389/fpubh.2018.00134 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hysong SJ, Best RG, & Pugh JA (2007). Clinical practice guideline implementation strategy patterns in veterans affairs primary care clinics. Health Services Research, 42(1 Pt 1), 84–103. 10.1111/j.1475-6773.2006.00610.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Institute of Medicine 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: The National Academies Press; 10.17226/10027. [DOI] [PubMed] [Google Scholar]
- Institute of Medicine 2009. Initial national priorities for comparative effectiveness research. Washington, DC: The National Academies Press; 10.17226/12648. [DOI] [Google Scholar]
- Kane M, & Trochim WMK (2007). Applied social research methods series Concept mapping for planning and evaluation. Thousand Oaks, CA, US: Sage Publications, Inc. [Google Scholar]
- Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, Stall R. (2007). Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implement Science.;2:42. doi: 10.1186/1748-5908-2-42 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kirchner JE, Ritchie MJ, Pitcock JA, Parker LE, Curran GM, Fortney JC (2014). Outcomes of a Partnered Facilitation Strategy to Implement Primary Care–Mental Health. Journal of General Internal Medicine, 30 Oct 2014. doi: 10.1007/s11606-014-3027-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, & Damschroder L. (2016). A systematic review of the use of the Consolidated Framework for Implementation Research. Implementation Science: IS, 11, 72 10.1186/s13012-016-0437-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lyon AR, Cook CR, Locke J, Davis C, Powell BJ, & Waltz TJ (2017, September). Importance and feasibility of a revised compilation of implementation strategies to support education sector behavioral health. Implementation Science, 13(Supplement 3): 39, A24. doi: 10.1186/s13012-018-0714-0 [DOI] [Google Scholar]
- Magnabosco JL (2006). Innovations in mental health services implementation: A report on state-level data from the U.S. evidence-based practices project. Implementation Science; 1:13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mazza D, Bairstow P, Buchan H, Chakraborty SP, Van Hecke O, Grech C, & Kunnamo I. (2013). Refining a taxonomy for guideline implementation: results of an exercise in abstract classification. Implementation Science: IS, 8, 32 10.1186/1748-5908-8-32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Michie S, Fixsen DL, Grimshaw JM, & Eccles MP (2009). Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implementation Science, 4(40), 1–6. 10.1186/1748-5908-4-40 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mittman BS Implementation science in health care In: Brownson RC, Colditz GA, Proctor EK, eds. Dissemination and implementation research in health: translating science to practice. New York: Oxford University; Press; 2012;400–418. [Google Scholar]
- Nadeem E., Olin S., Hoagwood KE., & Horwitz SM. (2013). Understanding the components of quality improvement collaboratives: A systematic literature review. Milbank Quarterly, 91(2), 354–394. 10.1111/milq.12016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- National Institutes of Health. Dissemination and implementation research in health (R01). 2016; http://grants.nih.gov/grants/guide/pa-files/PAR-16-238.html. Accessed March 2, 2017.
- Nilsen P. (2015). Making sense of implementation theories, models and frameworks. Implementation Science, 10(1), 53 10.1186/s13012-015-0242-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Patient Centered Outcomes Research Institute. (2019). PCORI standards for studies of complex interventions. Retrieved March 16, 2019, from https://www.pcori.org/research-results/about-our-research/research-methodology/pcori-methodology-standards#Complex [Google Scholar]
- Perry CK, Damschroder LJ, Hemler JR, Woodson TT, Ono SS, & Cohen DJ (2019). Specifying and comparing implementation strategies across seven large implementation interventions: a practical application of theory. Implementation Science, 14(1), 32 10.1186/s13012-019-0876-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Possis E, Skroch B, Mallen M, Henry J, Hintz S, Billig J, & Olson D. (2016). Brief immersion training in Primary Care–Mental Health Integration: Program description and initial findings. Training and Education in Professional Psychology, 10(1), 24–28. 10.1037/tep0000103 [DOI] [Google Scholar]
- Powell BJ, Garcia K, & Fernandez ME (2019a). Implementation strategies In Chambers D, Vinson C, and Norton W. (Eds.), Advancing the Science of Implementation Across the Cancer Continuum (pp. 98–120). New York: Oxford University Press. [Google Scholar]
- Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, & Weiner BJ (2019b). Enhancing the impact of implementation strategies in healthcare: A research agenda. Frontiers in Public Health, 7(3), 1–9. 10.3389/fpubh.2019.00003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Powell BJ, Beidas RS, Rubin RM, Stewart RE, Wolk CB, Matlin SL, Weaver S, Hurford MO, Evans AC, Hadley TR, et al. (2016). Applying the policy ecology framework to Philadelphia’s behavioral health transformation efforts. Administration and Policy in Mental Health and Mental Health Services Research, 43(6):909–926 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. (2015). A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Science, 10(1), 21 10.1186/s13012-015-0209-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Proctor EK, Powell BJ, & McMillen JC (2013). Implementation strategies: Recommendations for specifying and reporting. Implementation Science, 8(139), 1–11. 10.1186/1748-5908-8-139 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Proctor E, Hooley C, Morse A, McCrary S, Kim H. & Kohl PL (2019). Intermediary/purveyor organizations for evidence-based interventions in the US child mental health: characteristics and implementation strategies. Implementation Science 14:3 10.1186/s13012-018-0845-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Resick PA, Monson CM, & Chard KM (2017). Cognitive Processing Therapy for PTSD : A Comprehensive Manual. New York, NY: The Guilford Press; Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=1432985&site=ehost-live [Google Scholar]
- Ritchie M. Tailoring implementation strategies to context—extreme facilitation: helping challenged healthcare settings implement complex programs (S11). In: Proceedings of the 8th Annual Conference on the Science of Dissemination and Implementation. 14–15 December 2015. Implement Sci. 2016;11(Suppl 2):100. doi: 10.1186/s13012-016-0452-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rogal SS, Yakovchenko V, Waltz TJ, Powell BJ, Kirchner JE, Proctor EK, Gonzalez R, Park A, Ross D, Morgan TR, Chartier M, & Chinman MJ (2017). The association between implementation strategy use and the uptake of hepatitis C treatment in a national sample. Implementation Science: 12:60. doi 10.1186/s13012-017-0588-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rogers EM (2003). Diffusion of innovations (5th ed). New York: Free Press. [Google Scholar]
- Sales A, Smith J, Curran G, & Kochevar L. (2006). Models, strategies, and tools. Theory in implementing evidence-based findings into health care practice. Journal of General Internal Medicine, 21 Suppl 2, S43–49. 10.1111/j.1525-1497.2006.00362.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H, … Smith JL (2006). The Role of Formative Evaluation in Implementation Research and the QUERI Experience. Journal of General Internal Medicine, 21(Suppl 2), S1–S8. 10.1111/j.1525-1497.2006.00355.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Waltz TJ, Powell BJ, Matthieu MM, et al. (2015). Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Science: IS;10,109. https://doi: 10.1186/s13012-015-0295-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Waltz TJ, Powell BJ, Chinman MJ, Smith JL, Matthieu MM, Proctor EK, et al. (2014). Expert recommendations for implementing change (ERIC): protocol for a mixed methods study. Implementation Science: IS, 9, 39 10.1186/1748-5908-9-39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Workgroup for Intervention Development and Evaluation Research. (2008). WIDER recommendations to improve reporting of the content of behaviour change interventions. Retrieved from http://interventiondesign.co.uk/wp-content/uploads/2009/02/wider-recommendations.pdf