Methods to Improve the Selection and Tailoring of Implementation Strategies

Abstract

Implementing behavioral health interventions is a complicated process. It has been suggested that implementation strategies should be selected and tailored to address the contextual needs of a given change effort; however, there is limited guidance as to how to do this. This article proposes four methods (concept mapping, group model building, conjoint analysis, and intervention mapping) that could be used to match implementation strategies to identified barriers and facilitators for a particular evidence-based practice or process change being implemented in a given setting. Each method is reviewed, examples of their use are provided, and their strengths and weaknesses are discussed. The discussion includes suggestions for future research pertaining to implementation strategies and highlights these methods' relevance to behavioral health services and research.

Introduction

Evidence-based practices (EBPs) are underutilized in community settings. Implementation science, the study of methods to promote the integration of research into routine practice,¹ has advanced knowledge of ways to increase the use of EBPs. In the United States, national agencies have prioritized the development and testing of implementation strategies,² which are “methods or techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice.”^3(p2) Implementation strategies can be single component or “discrete” strategies (e.g., disseminating educational materials, reminders, and audit and feedback); however, most are multifaceted and multilevel, involving more than one discrete component.^4,5 Multifaceted strategies can be built by combining the over 70 discrete implementation strategies that have been described in published taxonomies.^5–7 However, selecting the most appropriate implementation strategies is a complex task – one for which the literature has provided limited guidance.⁸ This article outlines challenges associated with selecting implementation strategies, presents an argument for selecting and tailoring implementation strategies to address the unique needs of implementation efforts, and fills a gap in the literature by suggesting four potential methods that may be used to guide that process.

Challenges associated with selecting implementation strategies

It is challenging to select from the many implementation strategies that may be relevant to a particular change effort given: limitations of the empirical literature; the underutilization of conceptual models and theories in the literature; and variations related to the EBPs and the contexts in which they are implemented.⁸ The evidence-base for implementation strategies has advanced considerably in recent years and numerous randomized trials have been conducted.⁹ However, reviews and syntheses of that literature provide limited guidance regarding the types of strategies that may be effective in particular circumstances. This is particularly true in behavioral health settings, as there are far fewer randomized controlled trials and head-to-head comparisons of implementation strategies than in other medical and health service settings.^10–12 Thus, while it is well established that simply training clinicians to deliver EBPs is insufficient,¹³ it is less clear which strategies are needed at the client, clinician, team, organizational, system, or policy levels to facilitate implementation and sustainment.

Conceptual frameworks¹⁴ can guide research and practice by suggesting factors that influence implementation outcomes^15,16 and providing some direction for the selection and tailoring of strategies.^17,18 But researchers often fail to explicitly refer to guiding conceptual frameworks,^12,19 and when they do, they often simply borrow a subset of constructs or outcomes without framing their study within the broader context of the framework.²⁰ It is not always clear how to translate frameworks into implementation strategy design,²¹ as many are primarily heuristic and do not indicate the direction or nature of causal mechanisms. Thus, while conceptual frameworks and theories can inform all aspects of implementation research,²² they provide a necessary but not sufficient guide for selecting and tailoring implementation strategies.

The considerable variation in EBPs and other process changes has implications for selecting strategies. Scheirer,²³ for example, has proposed a framework of six different intervention types that vary in complexity and scope, from interventions implemented by individual providers (e.g., cognitive behavioral therapy) to those requiring coordination across staff and community agencies (e.g., multisystemic therapy, assertive community treatment) to those embracing broad-scale system change (e.g., Philadelphia's recovery transformation).²⁴ These intervention types may require the use of unique constellations of implementation strategies to ensure that they are integrated and sustained.²⁵ Contextual variation also has implications for selecting strategies,⁸ as settings are likely to vary with regard to patient-level (e.g., fit between patient's cultural values and EBP,²⁶ patient “buy in” to EBP);²⁷ provider-level (e.g., attitudes toward EBPs²⁸ and specific behavior change mechanisms⁷); organizational-level (e.g. culture and climate,²⁹ implementation climate³⁰); and system-level characteristics (e.g., policies and funding structures^31,32).

Selecting and tailoring discrete strategies to address contextual needs

The complexities associated with selecting implementation strategies and the considerable variation inherent to different EBPs and contexts has led scholars to suggest that implementation strategies should be selected and tailored to address the unique needs of a given change effort.^4,33–35 The first step in selecting and tailoring implementation strategies is to conduct an assessment of factors that influence implementation processes and outcomes,³⁴ such as the characteristics of the innovation,^23,36 characteristics of the setting in which it will be implemented, the characteristics and preferences of involved stakeholders, and other potential barriers and facilitators.³⁷ Numerous resources can guide the assessment of these factors.^37,38 The second step of the process involves selecting and, when necessary, tailoring strategies that can potentially address the context-specific factors identified in the pre-implementation assessment.³⁴ An example of selecting strategies to address an identified barrier would be preventing “therapist drift” by selecting fidelity monitoring, audit and feedback, or ongoing consultation. Discrete strategies may also need to be tailored to address a particular barrier. For example, in-person trainings may be difficult to scale-up in community settings because they require substantial expenditures of time and money; thus, the training may need to be to be delivered as a web-based module.

The need for systematic methods for selecting and tailoring implementation strategies

Selecting and tailoring strategies to address contextual needs “has considerable face validity and is a feature of key frameworks and published guidance for implementation in health care.”⁴ While the empirical evidence supporting the approach is not yet robust, a Cochrane Review found that strategies tailored to address identified barriers to change were more likely to improve professional practice than no intervention or the dissemination of guidelines.³³ Yet many implementation studies have failed to use strategies that are appropriately matched to contextual factors.³⁹ Implementers have too often become attached to single strategies (e.g., educational workshops) and uncritically applied them to all situations.³⁹ Furthermore, in a review of 20 studies, Bosch and colleagues⁴⁰ found that when researchers have attempted to match implementation strategies to identified barriers, there has often been a theoretical mismatch (e.g., clinician-focused strategies are used to address barriers that at the organizational level). Baker et al. note, “although tailored interventions [implementation strategies] appear to be effective, we do not yet know the most effective ways to identify barriers, to pick out from amongst all the barriers those that are most important to address, or how to select interventions [implementation srategies] likely to overcome them.”^33(p20) Linking strategies to barriers, facilitators, and contextual features remains a creative, emergent, and non-systematic process that occurs during implementation efforts.³⁵ It is clear that “systematic and rigorous methods are needed to enhance the linkage between identified barriers and change strategies.”^39(p169) These methods should take relevant theory and evidence into account, elicit stakeholder feedback and participation, and be specified clearly enough to be replicated in science and practice. To date, few studies have advanced candidate methods capable of addressing that need.

Methods for Selecting and Tailoring Implementation Strategies

This article fills a gap in the implementation science and behavioral health literatures by suggesting four methods that can be used to improve the process in which strategies are linked to the unique needs of implementation efforts: concept mapping,^41,42 group model building,^43–45 conjoint analysis,⁴⁶ and intervention mapping.⁴⁷ These methods were selected based upon the authors' expertise and a narrative search of the literature in multiple disciplines (e.g., implementation science, public health, engineering, marketing, etc.). They were also chosen because they have been used to develop interventions in other contexts, have substantial bodies of literature that could guide their use, and are not proprietary. The evidence for their effectiveness in improving implementation and clinical outcomes is yet to be determined; however, their use in other contexts suggests that they may be an important step in a research agenda aiming to integrate the perspectives of implementation stakeholders, make implementation planning more systematic, and improve the linkage between implementation barriers and strategies. Below each method is reviewed and examples of their use are provided. Table 1 contains a brief description of each method and lists some advantages and disadvantages to them to select and tailor implementation strategies.

Table 1. Overview of methods for selecting and tailoring implementation strategies.

Method:	Brief Description:	Advantages and Disadvantages:
Concept Mapping	A mixed methods approach that involves generating, structuring, and analyzing ideas to create a visual map of concepts that are rated on specified dimensions (e.g., importance and feasibility)	Advantages: -Participatory process builds buy-in -Communicates complex information visually -Data can be collected online as well as in person -Ratings allow for identification of barriers that may be most important and feasibly addressed Disadvantages: -May require further training or methodological consultation -Requires use of proprietary software
Group Model Building	A system dynamics-based method that involves engaging stakeholders in the collaborative development of causal loop diagrams that model complex problems to identify opportunities and strategies for improvement	Advantages: -Inherently integrated approach that involves the identification of barriers and facilitators and strategies to overcome them -Addresses problems that are “dynamically complex” such as implementing EBTs -Highly participatory and has potential to galvanize stakeholder groups -Ability to mathematically model consequences of proposed solutions prior to implementing them may reduce use of strategies that would be ineffective or too costly Disadvantages: -May require further training or methodological consultation -Complexity of some models may be overwhelming to stakeholders
Conjoint Analysis	A quantitative method that requires participants to select different “product” profiles, which allows for the determination of how they value different attributes of products, services, interventions, implementation strategies, etc.	Advantages: -Provides a clear step-by-step method for selecting and tailoring strategies to unique settings -Forces stakeholders to consider attributes of strategies at a granular level, enhancing the precision with which strategies are tailored to context Disadvantages: -May require further training or methodological consultation -May yield what stakeholders want, but not what theyneed
Intervention Mapping	A systematic, multi-step method for developing interventions (or implementation strategies) that is inherently ecological and incorporates theory, evidence, and stakeholder perspectives	Advantages: -Provides a systematic way of operationalizing the strategy development process -Has been used across a diverse array of stakeholders and conditions -Explicitly incorporates theory, evidence, and stakeholder perspectives Disadvantages: -May require further training or methodological consultation

Concept mapping

Overview

Concept mapping is a mixed methods approach that organizes the ideas of a group to form a common framework.⁴² Data collection involves engaging stakeholders in brainstorming, sorting, and rating tasks. The brainstorming task is structured through a “focus prompt” (e.g., “a potential barrier to implementing this EBP is____” or “one strategy for implementing this EBP in our organization [service system, state, etc.] is ____”). Stakeholders are encouraged to generate as many items as possible. These items are then compiled and culled to a list of no more than 100 statements. Stakeholders then sort the statements into conceptually consistent piles. Stakeholders also are asked to rate each statement on a number of dimensions (e.g., importance, feasibility, changeability) using a Likert scale. These data are then analyzed using multidimensional scaling and hierarchical cluster analysis. The end product is a map that contains different shapes representing distinct concepts that can also be depicted in a cluster rating map to reflect varying ratings on specified dimensions such as importance and feasibility.⁴² Kane and Trochim's text⁴² provides an accessible introduction to concept mapping. Concept mapping software, training, and consultation is available through Concept Systems Global, Inc.⁴⁸

How it has been used

Concept mapping has been used in strategic planning, community building, curriculum development, the development of conceptual models, and evaluation.⁴⁹ Increasingly, it has been successfully used to address implementation-related objectives such as identifying barriers and facilitators,^15,50 assessing program fidelity,⁵¹ and creating a conceptual model to translate cancer research into practice.⁵²

Examples from behavioral health

Concept mapping can improve the selection and tailoring of strategies in a number of ways. First, it can be used to identify factors that may affect the implementation of a specific EBP in a specific setting, and to determine which factors are most important and actionable from the perspectives of a wide range of stakeholders.^15,50,53 For example, Green and Aarons⁵³ compared the perspectives of policy and direct practice stakeholders regarding factors that influence the implementation of EBPs. Figure 1 shows an example of a cluster rating map taken from their study. Each point on the map represents a specific barrier or facilitator of EBP implementation that policy stakeholders identified. The 14 conceptually distinct clusters were derived from the sorting process and subsequent analysis (i.e., multidimensional scaling and hierarchical cluster analysis). The layers on the map depict the policy stakeholders' “importance” ratings. Thus, the cluster of barriers and facilitators labeled “funding” was seen as more important than the cluster labeled “agency compatibility.” Green and colleagues provide additional guidance and examples of the use of concept mapping in relation to implementation.⁴¹

Figure 1. Policy stakeholder cluster rating map⁵³.

Second, concept mapping can be used to ensure that a full-range of implementation strategies are considered by supplementing strategies generated through stakeholder brainstorming sessions with those identified through relevant literature,^5,6 and generating conceptually sound categories of strategies from which to select. Waltz et al.⁸ have recently used concept mapping to create conceptually distinct categories of strategies and determine their importance and feasibility according to experts in implementation science and clinical practice.

Finally, concept mapping could be used to generate consensus about the strategies (and categories of strategies) that can most effectively, efficiently, and feasibly address a specific barrier or set of barriers (e.g., a focus prompt could be, “In order to more effectively engage patients in treatment, we need to____”).

Group model building

Overview

Group Model Building comes from the field of system dynamics and involves the “client” in identifying and implementing solutions to “messy” problems.⁴³ System dynamics is characterized by the use of social systems to obtain stakeholder feedback that leads to models for studying causal loops (including variables, relationships, and feedback) to better understand potential consequences of the system's structure prior to the actual implementation. Group model building has evolved to include numerous structures and formats (see Rouwette and Vennix⁵⁴ for a review), with guidelines for choosing the optimal approach.⁴⁴ The reference group approach is perhaps the most commonly used. In this approach, a reference group of between 8-10 “clients,” who could be directors, managers, supervisors, providers, and consumer advocates involved in implementation, serve as critics to the modeling process in a series of iterative discussions and meetings. The group meetings require at least two guides, though others suggest a minimum of five, including: a “gatekeeper,” “process coach,” “modeling coach,” “facilitator” (who has the most important role), and a “recorder.”⁵⁵ The facilitator must be a neutral party (ideally a group model building expert) who can elicit client knowledge, translate the knowledge into system dynamics terms, and create a group atmosphere in which open communication can occur with the goal of achieving consensus and commitment. The recorder keeps track of all model elements throughout the discussion series. Once the structure and roles are established, group model building engages the following three steps. First, a problem statement is articulated by the reference group.⁴³ For example, the problem statement might be, “Clinicians are not integrating the EBP into their practice.” Second, the model building is initiated. This requires careful delineation of the relevant variables by the referent group and is termed the “conceptualization stage.” For instance, the referent group might identify training needs, resources, clinician confidence, time constraints, and supervision as relevant variables to be included in the model. The facilitator then generates a causal loop or stock and flow diagram based on this information. Figure 2 provides an example of a causal loop diagram related to implementation.⁵⁶ These models can be drawn freehand or created with the aid of model building and simulation software. Third, a mathematical formula is generated to quantify specified parameters. This model building stage requires the expertise of the group model building team; the referent group is only used for consultation given its time consuming and complicated nature. The model is then ready for simulation of the proposed solution. More information about group model building can be found in Hovmand's text,⁵⁷ and training and consultation opportunities can be sought through the System Dynamics Society.⁵⁸

Figure 2. Main feedback loops in an implementation model⁵⁶.

How it has been used

Group model building has been used to collaboratively solve problems among stakeholders with the aid of system dynamics experts and implement solutions across for-profit (e.g., oil, electronics, software, insurance, and finance industries), non-profit (e.g., universities, defense research), and governmental (from the city to national level) sectors, as well as between organizations.⁴⁵ The Office of Behavioral and Social Sciences Research and other leaders in the field have prioritized the development and utilization of systems science methodologies to capture the complexity of threats to public health and to address the science to practice gap.^59,60

Example from behavioral health

Huz and colleagues⁶¹ used a group model building approach to address gaps in the continuity of vocational rehabilitation services for individuals with severe mental illness. The process consisted of four meetings that occurred over a six-month period. After an initial introductory meeting, the system dynamics simulation model was developed during a two-day conference involving 12 to 18 managers of mental health and rehabilitation services from the involved county, state, and non-profit agencies, as well as one or more client advocates. This model was used to guide discussion and develop an action plan for moving toward a more integrated vocational rehabilitation system. The third and fourth sessions involved the continued exploration of insights from the simulation model, but moved toward the group's strategies for change and their progress toward achieving their action plan. Huz and colleagues⁶¹ found that the group model building process was well received by both the modeling team and the target group. More importantly, the process resulted in significant shifts in their approach to integrating care and reduced the group's reliance on “silver bullet” solutions that are not effective in improving care. Participants also became more aligned in their perceptions of systems goals and the strategies that they will use for integrating care.⁶¹

Conjoint analysis

Overview

Conjoint analysis is a method used to measure stakeholders' preferences for product (or service, program, implementation strategy, etc.) attributes, to learn how changes in those attributes influence demand, and to forecast the likely acceptance of products that are brought to market.^62,63 It is particularly useful in addressing situations in which there are inherent tradeoffs,⁶² for instance, when implementation strategy X has more support than strategy Y, but strategy Y is less costly than strategy X. Though there are many different types of conjoint analysis,^62–65 all of them ask respondents to consider multiple conjoined product features and to rank or select the products that they would be likely to purchase or use.⁶³ In “full-profile” techniques, different products with varying attributes are shown one at a time.^62,63 For example, respondents might be presented with a single “card” that has five discrete strategies (e.g., educational materials, training, supervision, fidelity monitoring, and reminders) comprising a single multifaceted strategy, and be asked to rate the strategy on a scale from 0 to 100, where 100 would mean “definitely would use to implement” a given EBP in a given context. In choice-based conjoint or discrete choice experiments,^62,66 products are displayed in pairs or sets and respondents are asked to chose the one they are most likely to use or purchase (see Figure 3). Respondents generally complete 12-30 of these questions, which are designed using experimental design principles of independence and balance of the attributes. Independently varying the attributes shown to respondents and recording their responses to the product profiles affords the opportunity to statistically deduce the product attributes that are most desired and have the most influence on choice.⁶³ Orme⁶⁷ provides a useful introduction of conjoint analysis for beginners, and Sawtooth Software provides a variety of software packages and resources that users will find helpful and accessible.⁶⁸

Figure 3. Example item from a study using conjoint analysis⁶⁹.

How it has been used

Conjoint analysis has primarily been used in marketing research to determine the products and features that consumers value most;⁶² however, as Bridges et al.⁶⁶ note, it has also been applied to a range of health topics (e.g., cancer, HIV prevention, diabetes). While it is just beginning to be applied in implementation research,^8,46,69 conjoint analysis can enhance the rigor of implementation strategy selection and tailoring processes. The examples provided below demonstrate the utility of two types of conjoint analysis: menu-based choice⁷⁰ and discrete choice experiments.^62,66

Examples from behavioral health

Menu-based choice methods are particularly relevant for obtaining stakeholders' insights regarding the types of strategies that should be considered among the many that could potentially be applied.^5,8 Menu-based choice methods are useful for generating data that can inform the mass customization of products and services such as automobiles, employee benefit packages, restaurant menus, and banking options.⁷⁰ Waltz and colleagues⁸ are using a menu-based choice approach to develop expert recommendations for the strategies that are most appropriate for implementing clinical practice changes within the Department of Veterans Affairs (VA). Expert panelists were given a set of 73 discrete strategies and were tasked with building multifaceted implementation strategies for three different clinical practice changes (measurement-based care for depression, prolonged exposure for posttraumatic stress disorder, and metabolic monitoring for individuals taking antipsychotics). For each practice change, panelists were provided with three different narratives that described relatively weak, moderate, and strong contexts. Each panelist indicated how essential (e.g., absolutely essential, likely essential, likely inessential, absolutely inessential) each discrete strategy would be to successfully implementing the clinical practice at three different phases of implementation (pre-implementation, implementation, and sustainment). While the results of this study are forthcoming, the analysis of these data will involve calculating Relative Essentialness Estimates^70,71 for each discrete strategy (where 1 represents the highest degree of endorsement and 0 represents the lowest) and generating count-based analyses that will be used to characterize the most commonly selected combinations of essential strategies for each scenario.⁸ These results will then be provided to the expert panelists to develop final consensus statements about the strategies most appropriate for implementing each practice change in each of the three implementation phases and contexts.

Menu-based choice methods may be most appropriate for selecting strategies, but discrete choice experiments⁶⁶ may be more helpful in tailoring strategies to specific populations and contexts. Each strategy has a number of modifiable elements; for example, the ICEBeRG Group⁷² pointed out that what might be seen as a relatively straightforward implementation strategy, audit and feedback, actually has at least five modifiable elements (content, intensity, method of delivery, duration, and context) that may influence its effectiveness. It may be helpful if modifications to these elements are informed by clinicians' preferences. Cunningham and colleagues⁶⁹ used a discrete choice experiment to determine the relative influence of 16 different attributes of practice changes and implementation strategies on the preferences of 1,010 educators (see Figure 3 for a sample survey question). The attributes included contextual and social attributes (e.g., presenter's qualities, colleague support, compatibility with practice), content attributes (e.g., supporting evidence, focus on knowledge vs. skills, universal vs. targeted), and practice change process attributes (coaching to improve skills, workshop size, training time demands, internet options). Their findings suggested ways in which stakeholders' preferences converged, namely their desire for small-group workshops conducted by engaging experts who would teach skills applicable to all students. Through the use of latent class analysis, the investigators found two different segments of educators (the change ready segment [77%] and the demand sensitive segment [23%]) who expressed different preferences for the design of implementation strategies. The change ready segment was willing to adopt new practice changes and preferred that schools have autonomy in making practice change decisions, that coaching be provided to all participants, and that participants receive post-training follow-up sessions. The demand sensitive segment was less intent on practice change, thought that practice changes should be at the discretion of individual teachers, recommended discretionary coaching, and preferred no post-training follow-up support.

Intervention mapping

Overview

Intervention mapping draws upon mixed-methods research, theory, community stakeholder input, and a systematic process for intervention development.⁷³ There are five steps in intervention mapping.^47,73,74 First, a needs assessment is conducted to identify determinants (barriers and facilitators) and guide the selection of intervention components. Second, proximal program objectives are specified to produce matrices with multiple cells that contain behavioral or environmental performance objectives, and the determinants of these objectives as identified in the needs assessment. Third, a list of intervention methods that are matched to the proximal program objectives identified in step two is generated. To achieve this, one must brainstorm and delineate methods, and then translate those methods into strategies.⁷⁴ Fourth, the implementation strategy is designed. To design the implementation strategy, the strategies listed in step three are operationalized in a way that clearly delineates what they entail, as well as how they will be delivered. Program materials such as an implementation intervention manual are designed. Fifth progress is monitored and evaluated.

The content produced throughout the stages of intervention mapping are both the implementation strategy and the tools needed to evaluate the effectiveness of the implementation strategy. The evaluation incorporates mixed methods measurement of both the processes and outcomes of the implementation strategy. There is an additional step of intervention mapping dedicated to adoption and implementation, but this step is not directly applicable here given that this is already a central consideration of implementation research. Bartholemew et al.⁴⁷ provide a useful text on intervention mapping, and further resources are available through the developers' website.⁴⁷

How it has been used

Intervention mapping has been used to develop a number of health-related programs, including those addressing sex education,⁷⁵ obesity prevention,⁷⁶ breast and cervical cancer prevention,⁷⁷ and cardiovascular health.⁷⁸ While intervention mapping has been proposed as a method for tailoring implementation strategies to identified barriers,^22,39 it has not been widely used, with a few notable exceptions.^79,80

Example from behavioral health

Intervention mapping was recently used in an attempt to increase physicians' adherence to guidelines for depression in the Netherlands.⁸⁰ The purpose of the guidelines was to ensure that assessments of depression symptoms by physicians were transparent and uniform. Zwerver and colleagues⁸⁰ provide a detailed account of the intervention mapping process, which resulted in the development of a multifaceted implementation strategy. Figure 4 provides a succinct summary of the associated determinants (barriers and facilitators), learning objectives, theory-based methods, and discrete implementation strategies that were generated. The multifaceted implementation strategy was subsequently tested in a randomized controlled trial, and was found to increase adherence to the guidelines in comparison to implementation as usual.⁸¹

Figure 4. Example matrix of determinants, learning objectives, theory-based methods, and implementation strategies⁸⁰.

Discussion

This article provides an overview of four methods that can be used to select and tailor implementation strategies to local contextual needs, an underdeveloped but critical area in implementation science.⁴ The discussion focuses on these methods' common strengths and weaknesses, recommendations for comparative effectiveness research on the selection and tailoring of strategies, and suggestions for reporting this type of research. This represents a first step in a research agenda aiming to improve behavioral health services by more systematically and rigorously selecting and tailoring strategies for implementing effective practices.

The methods share a number of strengths. First, all are inherently participatory, engaging diverse stakeholders and importantly, providing concrete steps or processes for facilitating communication. All four methods, particularly concept mapping and group model building, have the potential to galvanize stakeholders around common goals and create consensus regarding the implementation approach. This emphasis on stakeholder participation is critical.⁸² Second, these methods all comprise systematic, transparent, replicable processes, which is a step forward for selecting and tailoring implementation strategies, an area previously described as a black box.⁴⁰ Third, all can represent and respond to the complexity of implementation efforts,⁸³ in that they provide the structure for considering factors at different levels and facilitate the selection and tailoring of strategies to address them.

The methods share a few primary weaknesses. If not informed by theory and evidence, they may be helpful in eliciting preferences, but not necessarily provide the supports needed to implement EBPs effectively. For example, in the discrete choice experiment cited above, there was a segment of teachers that believed practice changes should be at the discretion of individual teachers, that coaching should be discretionary, and that no post-training follow-up support was necessary. This conflicts with evidence that coaching and post-training follow-up are important.^84,85 Thus, it would likely be less effective to give teachers the choice to opt out of coaching or follow-up training. Clearly there is a balance to be struck between a paternalistic overemphasis on the research literature and a disregard of evidence. One way of accomplishing this is to “seed” the methods with theoretically or empirically justified strategies and attributes, and allow stakeholders to supplement them based upon their expertise and experience. This would bolster the likelihood that selection and tailoring methods are guided by the best available theory and evidence, while preserving the benefits of stakeholder engagement and preference. Another weakness of all of the methods is that they all require consultation or specific training. Finally, with some notable exceptions,⁴⁵ there is a dearth of empirical evidence supporting their effectiveness and cost-effectivness in obtaining improved implementation and clinical outcomes. The latter weakness is not suprising given that implementation science is a relatively new field that is gradually moving toward more precise applications of implementation strategies.^4,86

Each method holds promise for increasing the rigor of implementation research; however, which method should be used is ultimately an empirical question. It is also possible that some of these methods could be usefully combined. For example, conjoint analysis might be particularly useful for tailoring implementation strategies after strategy selection is completed via one of the other methods (concept mapping, group model building, or intervention mapping). Three types of comparative effectiveness research⁸⁷ are warranted. First, there is a need for comparisons between implementation strategies that have been selected and tailored to address contextual factors and more generic multifaceted implementation strategies. Although preliminary studies suggest the effectiveness of systematic selection of implementation strategies,⁸⁸ the evidence is not robust.^4,33 Second, there is a need to comparatively test different methods for selecting and tailoring strategies. It will be important to determine which methods are most acceptable and feasible for stakeholders, whether or not they result in similar constellations of implementation strategies, and whether some are more efficient and effective in identifying key contextual factors and matching discrete strategies to address them. Finally, it will be essential to assess the cost-effectiveness of these approaches.^89,90 Several ongoing studies will contribute to the evidence-base for selecting and tailoring implementation strategies to context-specific factors, including the Tailored Implementation for Chronic Disease research program³⁴ and a study recently funded by the National Institute of Mental Health that will test tailored versus standard approaches to implementing measurement-based care⁹¹ for depression (R01MH103310; Lewis, PI).

Publications reporting efforts to select and tailor strategies to address contextual factors should be clear about the methods used for selection and tailoring, the specific discrete strategies selected, and the links between strategies and barriers and other contextual factors. First, researchers should carefully report how they selected and tailored the strategies, and provide descriptions of any systematic methodology (e.g., concept mapping, group model building, etc.) that they may have used. Second, to move the field forward, there is a need for researchers to clearly describe the implementation strategies used in their studies, as the description of implementation strategies in published research has been noted to be very poor.^3,92 Proctor and colleagues³ have suggested reporting guidelines for implementation strategies that require researchers to carefully name, conceptually define, and operationalize strategies, and specify them according to features such as: (a) the actor; (b) the action; (c) action target; (d) temporality; (e) dose; (f) implementation outcome affected;⁸⁹ and (g) the empirical, theoretical, or pragmatic justification for use of the strategy.³ Authors may benefit from consulting those guidelines, or others that have recently been set forth^93,94 when reporting the results of implementation studies. Finally, in order to make the links between strategies and the contextual factors that they are intended to address explicit, it may be helpful if researchers use logic models, which could clarify how and why their carefully selected and tailored implementation strategy is proposed to work.^95,96 These suggestions should lead to more consistent and transparent reporting, and improve the ability to understand how strategies exert their effects.

Implications for Behavioral Health

Improving the quality of behavioral health services has proven to be a challenging task, with barriers and facilitators to change emerging at all levels of the service system.^97–110 Both conceptual¹¹¹ and empirical literature^25,112 suggests that these barriers and facilitators must be taken into account and that different strategies may be required to overcome challenges associated with various interventions, implementation contexts, and stages of implementation. Moving toward more contextually sensitive implementation strategies is an exciting step forward for implementation science that holds potential for improving the quality of behavioral health care. The methods presented in this article have the potential to increase the rigor of the selection and tailoring process. While they may not make implementation easier at the outset, these methods provide a step-by-step process for selecting and tailoring implementation strategies and for engaging stakeholders that may be very attractive to researchers and other implementation leaders. These initial investments are also likely to payoff in the long run as implementation failures due to overlooked and unaddressed barriers to improvement are avoided.

These methods are likely not the only ones that could be used to guide selection and tailoring of strategies. There is a need for ongoing dialogue that might lead to the identification of additional avenues for systematically selecting and tailoring strategies. The move toward more the thoughtful and systematic application of implementation strategies will increase the legitimacy of the field by enhancing the scientific rigor of proposed studies, strengthening the evidence-base for implementation strategies, and providing support for causal mechanisms and theory. Ultimately, there is reason to hope that the use and evaluation of systematic methods for selecting and tailoring implementation strategies to contextual needs will propel the field toward a greater understanding of how, when, where, and why implementation strategies are effective in integrating evidence-based care and improving clinical outcomes in behavioral health service settings.⁴