Dissemination and Implementation of Evidence-Based Practices: Training and Consultation as Implementation Strategies

Abstract

To provide effective treatment for individuals with mental health needs, there is a movement to deploy evidence-based practices (EBPs) developed in research settings into community settings. Training clinicians in EBPs is often used as the primary implementation strategy in these efforts, despite evidence suggesting that training alone does not change therapist behavior. A promising implementation strategy that can be combined with training is consultation, or ongoing support. This paper reviews the literature on consultation following initial training. A model of consultation is presented as well as preliminary findings regarding effective consultation techniques. Future directions are offered.

Mental health problems constitute a major public health concern given that they impair functioning and are the second leading cause of disability, accounting for more than 15% of the overall burden of disease in the United States (Satcher, 2000). Fortunately, evidence-based practices (EBPs), defined as “the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences” (American Psychological Association, 2005, p. 1), have been developed to treat a variety of mental health problems in youth and adults (U.S. Surgeon General, 1999). However, EBPs are only useful to the extent that they are implemented properly in clinical practice, and there is a chasm between optimal care and the care typically received by individuals (President’s New Freedom Commission on Mental Health, 2003). The challenge of dissemination and implementation is not unique to healthcare, and efforts to understand implementation of innovation have been ongoing since the work of Rogers (Green, Ottoson, Garcia, & Hiatt, 2009). A number of encouraging developments around system-wide training efforts, such as the Evidence-Based Psychotherapy initiative in the United States Veteran Health Administration (see McHugh & Barlow, 2010) and the Improving Access to Psychological Therapies (IAPT, 2008) in Great Britain, are currently occurring. However, too often, training community clinicians using a brief workshop is the primary implementation strategy used to deploy EBPs in community settings (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005; Williams & Martinez, 2008) despite research suggesting that workshops incur only small effects on clinician behavior (Forsetlund et al., 2009). Recent research suggests the effectiveness of adding ongoing consultation following a workshop to impact clinician behavior (Beidas, Edmunds, Marcus, & Kendall, 2012). This paper discusses a potentially key component in efforts in closing the research-training-practice gap—the provision of consultation, or ongoing support. Specifically, we argue that brief training as an implementation strategy is necessary but not sufficient, and that the incorporation of ongoing support (e.g., consultation) into training is potentially critical for effective implementation of EBPs. After reviewing literature that points toward the effectiveness of training followed by ongoing consultation, we present a model of consultation and preliminary findings regarding useful consultation techniques.

Education in the Implementation of EBPs

A recent compilation of implementation strategies used to transport EBPs into community health and mental health settings identifies six broad types of implementation strategies: planning, educating, financing, restructuring, quality management, and attending to policy contexts (Powell et al., 2012). Although each component is worthy of empirical investigation and further discussion, a discussion of each is beyond the scope of the current review. We focus on the broad theme of “education.” Within this category, there are a number of educative strategies, including but not limited to: the distribution of educational materials (e.g., Cochrane Effective Practice and Organization of Care Group; Farmer et al., 2008), workshop trainings (e.g., Davis et al., 1999), the creation of practice guidelines (e.g., Corrigan, Steiner, McCracken, Blaser, & Barr, 2001), computer-assisted decision-making (Chorpita, Bernstein, Daleiden, & Research Network on Youth Mental Health 2007), and ongoing support strategies, such as audit and feedback (Jamtvedt, Young, Kristoffersen, O’Brien, & Oxman, 2006), consultation (e.g., Herschell, Kolko, Baumman, & Davis, 2010), facilitation (Kauth et al., 2010), and peer support networks (Institute for Healthcare Improvement, 2003). Classifying these various strategies as “education implementation strategies” marks progress. Nevertheless, the comparative potency of these strategies or whether certain combinations of strategies are optimal for improving implementation under varied conditions remains largely unknown. An examination of the empirical work to date points to a potentially promising combination— training (i.e., attending a workshop on an EBP prior to attempting to implement it) and ongoing support (e.g., consultation; Beidas et al., 2012; Herschell et al., 2010).

Definition of Terms

A common criticism of the dissemination and implementation (DI) field is its lack of consistent terminology (Michie, Fixsen, Grimshaw, & Eccles, 2009). Lack of consistency is evident when attempting to label ongoing support strategies following training. Various terms have been used, such as supervision (e.g., Mannix et al., 2006), consultation (e.g., Beidas et al., in press), coaching (e.g., Miller, Yahne, Moyers, Martinez, & Pirritano, 2004), monitoring with feedback (e.g, Parsons, Reid, & Green, 1993) and audit and feedback (Jamtvedt et al., 2006). Although terms vary, it is difficult to determine whether these strategies are qualitatively distinct from each other given the often limited descriptions of what these strategies entail.

The ongoing support strategy of primary focus in this review is consultation, however, it is noted that elements of consultation may be present in alternatively labeled ongoing support strategies (e.g., coaching). We follow Caplan and Caplan’s (1993) definition of consultation as “a process of interaction between two professionals–the consultant, who is a specialist, and the consultee, who invokes the consultant’s help in a current work problem that he believes is within the consultant’s area of specialized competence” (p. 11). According to Caplan and Caplan (1993), the primary goal of mental health consultation is to improve mental health care provided to the patient. They distinguish consultation from supervision based on the non-hierarchical relationship between consultant and consultee; a consultant, in contrast to a supervisor, accepts no direct responsibility for the implementation of care, and the consultee is free to accept or reject the consultant’s suggestions. The only difference in status stems from the “authority of ideas” (Caplan & Caplan, 1993, p. 14), such that a consultant may possess expert knowledge regarding a particular practice or population. Moving forward, “consultation” will be the term of choice when referring to ongoing support efforts that fall under the umbrella of the Caplan and Caplan (1993) definition. However, when different terms were used in original articles, these terms will be noted in footnotes.

Training and Ongoing Consultation as Complementary Implementation Strategies

A review of implementation strategies indicates that the addition of consultation to training appears to improve provider and patient outcomes. The Cochrane Effective Practice and Organization of Care (EPOC) Group concluded based on their examination of 81 trials of continuing medical education that educational meetings alone provide generally small effects on physician performance and health care outcomes (Forsetlund et al., 2009). In a review of mental health services, Herschell et al. (2010) found that multi-component trainings which included multiple training methods and multi-component trainings that specifically involved workshop follow-ups such as ongoing consultation and supervisor feedback enhanced clinician skill, adherence, knowledge, and rates of implementation, as well as client outcomes, more consistently than workshops alone. Another review of training in mental health care demonstrated that cognitive-behavioral therapy (CBT) training studies with the strongest effects on clinician and client outcomes were characterized by extensive training (i.e., more than 137 hours of training) and often incorporated graded training in which clinicians who were not progressing received additional training and support (Rakovshik & McManus, 2010). Beidas and Kendall (2010) employed a systems-contextual perspective in their review of training in EBPs, which recognizes the various contextual levels within which training is embedded (e.g., organizational, therapist, and client). Their review identified training as most successful when it addressed multiple levels of the systems-contextual model, included consultation and/or supervision, and incorporated active learning techniques.

In addition to demonstrating improved outcomes as compared to workshops by themselves (e.g., Miller et al., 2004), some studies have found the incorporation of ongoing consultation to training to be more effective than self-training (Sholomskas et al., 2005), workshops plus self-training materials (Kelly et al., 2000), and diploma courses that do not provide consultation (Bradshaw, Butterworth, & Mairs, 2007). Consultation appears to maintain clinician skill (Mannix et al., 2006). A recent examination found a positive linear relationship between participation in consultation after workshop training and therapist adherence and skill in delivering CBT for youth anxiety as assessed by independently-rated role-plays (Beidas et al., 2012). In this study, 115 clinicians (social workers, psychology graduate students, master’s and doctoral level clinicians, school psychologists, guidance counselors, psychiatrists, and nurse practitioners) were randomly assigned to one of three conditions: (1) a one-day workshop that covered a specific manual (i.e., Coping Cat; Kendall & Hedtke, 2006) and procedures of CBT for child anxiety, (2) computer training on CBT for child anxiety accomplished through a commercially-developed interactive DVD (CBT4CBT; Kendall & Khanna, 2008), and (3) a one-day workshop that included a focus on principles of CBT and active learning (including behavioral role play exercises). Following one day of training, participants completed three months of weekly consultation via telephone or Internet. Consultation consisted of continued didactics, role-plays, and case discussion. Outcomes of interest included adherence (i.e., the presence of core CBT components) and skill (i.e., competence in delivering treatment according to the CBT model) during independently-rated role-plays. CBT knowledge was also assessed. Training condition was not differentially predictive; clinicians in all conditions demonstrated improvements in skill, adherence, and knowledge from pre- to post-training and further improvements in skill and adherence from post-training to post-consultation. Clinicians who attended more consultation calls posttraining evidenced greater improvements in skill and adherence compared to clinicians who attended less calls, indicating the importance of consultation, regardless of workshop training methods. Specifically, each hour of consultation following training improved therapist adherence by .4 points and skill by .3 points.

Although the data suggest the improved efficacy of training that incorporates ongoing support, one might argue that the greater amount of time of training rather than consultation, specifically, accounts for improved outcomes. In contrast to this argument, a less intensive training program yielded similar results as those of a more intensive training program (Westbrook, Sedgwick-Taylor, Bennett-Levy, and McManus, 2008). The effectiveness of a 10-week program, which included weekly didactics and group consultation¹ was compared to a 1-year diploma course on CBT for anxiety and depression. From baseline to post-training in the shortened training program, significant improvements were found in assessor-rated and self-rated CBT skills and, most importantly, client functioning. These results were then compared to those obtained in the longer diploma course. Although trainees in the diploma course evidenced higher total scores on the independent evaluator-rated measure as compared to participants in the shortened course, no significant differences were found across subscale scores, suggesting that clinicians obtain comparable individual skills in both courses. No significant differences were found across courses according to clinicians’ self-reported skill and client outcome data. Thus, the shortened course yielded similar benefits as those of the diploma course. Of note were the high satisfaction ratings for the shortened training. Further, participants rated consultation as a highly important training component. These findings suggest that greater intensity of training does not always yield vastly incremental benefits. It may be the specific components of training, such as consultation, and not the length of training that contributes to greater effectiveness.

Research on the implementation of Multisystemic Therapy (MST; Henggeler, Schoenwald, Borduin, Rowland, & Cunningham, 1998), an intensive family- and home-based treatment for antisocial youth, into community settings provides some support for the specific role of consultation in successful implementation. Typical training in MST includes a 5-day workshop for therapists and supervisors followed by one hour per week of on-site supervision, one hour per week of telephone consultation with an MST expert, and quarterly training booster sessions (Schoenwald, Heiblum, Saldana, & Henggeler, 2008). Analyses have demonstrated associations between MST consultation and therapist and client outcomes. Specifically, perceived consultant competence, as rated by MST therapists, predicted therapist adherence and improved youth outcomes. After controlling for consultant competence, “alliance” (i.e., therapists’ perception of the attentiveness and supportiveness of the consultant) was negatively associated with therapist adherence and youth outcomes (Schoenwald, Sheidow, & Letourneau, 2004). In other words, when consultant competence is low, a supportive alliance between consultant and therapist negatively predicts therapist adherence. Also, a supportive alliance appears to have a negative effect on child outcomes when the consultant spends too little time on applying MST procedures in specific cases. The MST literature corroborates the links between ongoing consultation and therapist adherence and youth outcome and illustrates the importance of examining specific aspects of consultation, given that it does not contribute to universally positive outcomes.

In summary, although methodological limitations, such as small samples, lack of random assignment and comparison groups (see Herschell et al, 2010 for a discussion) in the empirical literature necessitate cautious conclusions, a promising finding across studies is the potential of providing ongoing support following initial training to impact therapist behavior (Beidas et al., 2012). The “train and hope” (Stokes & Baer, 1977) notion of a brief workshop appears insufficient to improve and sustain clinician and client outcomes. From the perspective of treatment researchers, this assertion is not surprising: Clinicians in randomized controlled trials (RCTs) undergo intensive training that involves close supervision in order to ensure fidelity to the treatment protocol (Roth, Pilling, & Turner, 2010). It appears that the investment in rigorous training and ongoing support strategies may be similarly warranted when striving to disseminate EBPs into community settings. Furthermore, it is possible that ongoing support may be more necessary in dissemination efforts in order to ensure treatment fidelity in the face of various barriers that community clinicians may face (e.g., time constraints for treatment).

Dismantling Consultation

Given that consultation is a promising implementation strategy, it would be useful to understand the mechanism through which consultation works. Delineating a conceptual model of consultation marks a critical step toward understanding and dismantling consultation and can serve to guide empirical research. Thus, this section aims to provide a model of consultation and preliminary identification of effective consultation techniques based upon a selective review of the literature.

The “What” of Consultation

Caplan and Caplan (1993) offered a 4-type model of mental health consultation, which may serve as a useful rubric for developing and categorizing consultation strategies in DI efforts. Client-centered case consultation (Type 1) primarily aims to help the client and involves the consultant’s expert assessment of the client’s problem and the offering of treatment recommendations. Consultee-centered case consultation (Type 2) primarily aims to help the consultee improve his/her professional functioning and involves identifying and remedying consultee difficulties (e.g., lack of skill). Program-centered administrative consultation (Type 3) aims to prescribe an effective course of action for program implementation and involves assisting in the development and/or improvement of a program. Lastly, consultee-centered administrative consultation (Type 4) aims to understand and remedy difficulties or barriers encountered by consultees when implementing a new program. Of note, although consultation is divided into four types, these types are not mutually exclusive and a consultant-consultee relationship may encompass all four types. In order to demonstrate the various types of consultation, the following examples are provided.

Client-centered case consultation

In Caplan and Caplan’s (1993) depictions of Type 1 consultation, they describe consultants entering consultees’ workplaces and meeting with the clients themselves in order to assess the clients and develop treatment recommendations. Within the context of implementation efforts, we expand Type 1 consultation to also include case discussion between a consultant and consultee that assists the consultee in forming a conceptualization of his/her client’s difficulties as well as a treatment plan. For example, Type 1 consultation was provided during the consultation described in Beidas et al. (2012). During each call, time was allotted for the clinicians and consultants to discuss cases in order to identify if they might be appropriate candidates for CBT for youth anxiety. The consultant prompted discussion with questions aimed to assess the clients’ appropriateness for CBT, such as, “What is the child’s primary presenting problem? How does it interfere in his/her daily life?” The consultant then assisted the clinicians in developing CBT treatment plans or recommended alternative treatments if clients did not seem appropriate for CBT. The difference between Type 1 and Type 2 consultation is the focus on understanding the client and targeting treatment appropriately.

Consultee-centered case consultation

In contrast to Type 1 consultation, Type 2 consultation focuses on improving the clinician’s professional functioning. For example, one study provided Type 2 consultation as a complement to training in three of five training conditions when training substance abuse counselors to implement motivational interviewing (Miller et al., 2004). These conditions included: (a) workshop training followed by written feedback from the consultant based on a review of practice tapes, (b) workshop training followed by 6 individual consultation² sessions in which the consultant provided positive reinforcement, helped the clinician problem-solve difficulties, and provided opportunities for the clinician to practice motivational interviewing skills, and (c) a combination of these methods. Of note, clinicians in these three conditions evidenced better skill than those in a self-training condition posttraining and better skills than those in the workshop only condition at 4-month follow-up.

Detailed descriptions of Type 2 consultation are provided in two studies examining the training of direct care staff in a community group home for adults with severe intellectual disabilities (Harchik, Sherman, Sheldon, & Strouse, 1992; Parsons et al., 1993). The training and consultation in both studies were as follows: First, the staff member participated in a mini-workshop with a consultant, the focus of which was decided upon by the consultant based on the staff member’s current performance and the needs of the adults. Workshops consisted of consultant modeling, role-plays, and the provision of feedback. Following the workshop, the consultant observed the staff member run teaching groups with the adults in the home and then provided feedback based on the consultant’s written observations. The consultant provided specific praise when the staff member correctly implemented a skill and also provided suggestions to the staff member on ways to improve his/her skills. These training methods resulted in improved staff behavior (i.e., use of token reinforcement system and teaching behavior) as well as client behaviors (i.e., engagement). It is worth noting that even though the focus of Type 2 consultation is on the clinician, improvement can be found in both clinician and client outcomes.

Program-centered administrative consultation

In contrast to other types of consultation, which occur during implementation, Type 3 consultation often occurs before implementation; the consultant meets with consultees to lay the groundwork for implementation of a new program. However, Type 3 consultation may also involve improvement of implementation on a programmatic level after implementation has begun. Given that the focus of the current review is on consultation as a strategy following clinician training, Type 3 consultation is beyond our scope. For an example of Type 3 consultation pre-implementation, see Schoenwald et al. (2008).

Consultee-centered administrative consultation

In contrast to Type 3 consultation with its programmatic focus, Type 4 consultation focuses on consultees as they attempt to implement a new program. The goal of Type 4 consultation is to assist consultees in overcoming any difficulties or barriers they may be experiencing. For example, Beidas et al. (2012) included Type 4 consultation in their consultation calls (along with time allotted for both Type 1 and Type 2 consultation). Clinicians were provided opportunities to discuss difficulties they encountered delivering CBT in their individual settings and the consultant assisted clinicians in problem-solving these barriers. For example, a critical component of CBT for child anxiety is exposure tasks, where youth face their fears. In a research setting, exposures often involve leaving the room (e.g., doing a survey for a socially phobic youth). Some clinicians reported that this would not be allowed in their agency, and the consultant worked with the clinicians to problem-solve around how to remain true to the principles of CBT and exposure while working within the confines of their organizational structure. In another study, half of the clinicians were randomly assigned to participate in Type 4 consultation following workshop training (Luoma et al., 2007). Consultation sessions focused on relapse prevention, which encouraged counselors to discuss barriers to adoption and ways to overcome such barriers. Consultation also incorporated the principles of Acceptance and Commitment Therapy (Hayes, Strosahl, & Wilson, 1999), which encouraged counselors to mindfully accept any uncomfortable thoughts or feelings that might arise from attempting the new intervention and to create behavioral goals. A higher sense of personal accomplishment and significantly higher levels of adoption were reported by participants in the consultation condition.

The “How” of Consultation

The Caplan and Caplan (1993) model provides a framework for categorizing types of consultation and can guide empirical investigations in order to determine which type or combination of types is most effective for particular outcomes. Despite the descriptive value of Caplan and Caplan’s (1993) consultation model and its usefulness for guiding research, the model lacks a causal explanation for how consultation works. Thus, we offer a model describing the causal mechanism through which consultation is hypothesized to operate in order to further guide the development and provision of optimal consultation in DI efforts.

We propose that Milne’s (2009) model of evidence-based clinical supervision is applicable to consultation within the scope of dissemination and implementation of EBPs. Supervision is defined as “the formal provision, by approved supervisors, of a relationship-based education and training that is work-focused and which manages, supports, develops and evaluates the work of colleague/s” (Milne, 2007, p. 439). Although supervision differs from consultation in that a hierarchical and evaluative relationship is often specified in supervision, both supervision and consultation share similar goals—to improve clinician functioning and the mental health care provided to the patient. Given the similar aims as well as the similar techniques found across supervision and consultation (e.g., feedback), the mediators and mechanisms of supervision may apply to consultation. However, we acknowledge that moderating factors may differ between supervisory and consultant-consultee relationships (e.g., supervisor having responsibility for ensuring supervisee fidelity), which may have an effect on the potency of certain techniques (e.g., instructions from a supervisor who signs off on one’s therapy notes may impact clinician behavior more powerfully than instructions from a consultant who has no regulatory power).

To construct an empirically-based model of supervision, Milne, Aylott, Fitzpatrick, and Ellis (2008) reviewed the supervision literature using a best-evidence synthesis approach (Petticrew & Roberts, 2006). In contrast to reviewing all studies within an area regardless of rigor or effectiveness, a best-evidence synthesis approach aims to glean good practices based on reviewing only effective studies. Using this approach, 26 supervisory techniques were identified across 24 effective supervision studies. Techniques included enactive (i.e., behavior-based), iconic (i.e., image-based), and symbolic (i.e., word-based) techniques. Overall, the techniques fell into the categories of teaching (75% of the studies), corrective feedback (63% of the studies), and observing (42% of the studies). In order of frequency, from highest to lowest, the 26 techniques were as follows: training (i.e., teaching and instructing), feedback, observing, goal-setting, question-and-answer, modeling, planning, praise and reinforcement, discussion, prompts, role-play, explanation, monitoring, review and reflection, summarizing, challenging, self-collaborating, confidence building, disagreeing, formulating, and understanding. In addition to identifying supervision techniques (i.e., mediators), the review also identified mechanisms of change that accounted for improved competence. A majority of the identified mechanisms (e.g., improved skill, increased emotional awareness) mapped onto the four components of the experiential learning cycle delineated in Kolb’s (1984) theory of experiential learning: experiencing, reflecting, conceptualizing, and experimenting. Milne et al. (2008) posited that within the context of moderating variables, the supervisory techniques identified yielded positive outcomes (i.e., therapist changes) primarily through assisting the supervisee in moving through Kolb’s (1984) learning cycle. Change in therapist behaviors may then predict client changes. This same model of experiential learning may be appropriate for consultation in DI efforts.

Empirical investigations can examine whether this model applies to consultation within the context of DI efforts and whether differences in contextual variables between supervision and consultation differentially moderate the effectiveness of these techniques on outcomes. In the following sections, we offer a review of preliminary findings regarding effective consultation techniques, each of which appears to be consistent with Kolb’s (1984) experiential learning theory as well as those identified by Milne et al. (2008). We note that this review is not comprehensive but serves to highlight prominent techniques found in consultation, including instruction, case review, self-evaluation, and performance-based feedback.

Instruction

A majority of the techniques identified by Milne et al. (2008) fell under the category of teaching/instructing. Instruction is similarly present during consultation and may prove critical for furthering clinician knowledge and skill following workshops. For example, teaching/instruction appeared beneficial for general practitioners who had no prior training in CBT as they learned CBT for panic disorder (Heatley, Rickettes, & Forest, 2005). Training consisted of two 3.5-hour didactic workshops followed by six 1.5-hour group consultation³ sessions. Consultation sessions focused on applying skills taught during the workshops and consisted of further elaboration of the clinical approaches introduced during the workshops. Following training, participants evidenced improved ability to assess and plan for the management of panic disorder based on their performance in role-plays. Additionally, according to patient records, participants evidenced greater use of CBT techniques with their patients.

Case review

Another common feature of consultation includes case review in either discussion form (e.g., Beidas et al., 2012) or tape review (e.g., Lau, Dubord, & Parikh, 2004). Case review offers opportunities for reflection and conceptualization, two of Kolb’s (1984) active learning stages. Studies provide preliminary support for the value of case review in consultation. For example, one study offered a 10-session training program that included group consultation³ sessions for therapists learning CBT for adults with mood and anxiety disorders (Lau et al., 2004). During group consultation, time was spent watching videotapes or listening to audiotapes of actual sessions and providing instructor and peer feedback. Improved skill (as assessed by independent raters watching therapy tapes) was found following participation in the training and consultation, and clients demonstrated reduced symptomatology. Similar gains in competence were found in counselors trained in CBT for substance abuse via 35 hours of didactic and practice-based training, followed by ongoing group and individual consultation⁴ consisting of tape review and session-by-session feedback (Morgenstern, Morgan, McCrady, Kelly, & Carroll, 2001).

Self-evaluation

Opportunities for self-evaluation, consistent with Kolb’s (1984) reflecting stage, also appear beneficial. For example, community mental health therapists who attended a 2-day workshop on motivational interviewing, followed by eight group consultation⁴ sessions allowing opportunities of self-evaluation, evidenced improved motivational interviewing skills (Schoener, Madeja, Henderson, Ondesma, & Janisse, 2006). Additionally, their substance-abusing clients evidenced increased change talk. Similarly, nurses who participated in didactic and interactive workshops followed by consultation.⁴ characterized by guided discovery demonstrated improved CBT skills (Mannix et al., 2006). Guided discovery during audiotape review encouraged reflection on the part of the nurse. Consultation also involved the sharing of feedback from others. Following initial training, half of the nurses were randomly selected to participate in ongoing consultation. Six months later, all nurses were reassessed. Those who had discontinued consultation evidenced a slight decline in CBT skills whereas those who had participated in ongoing consultation evidenced further improvements in skills and reported greater use of CBT skills with their patients.

Feedback

In addition to highlighting the potential importance of case review and self-evaluation, the studies referenced above demonstrate the potential effectiveness of feedback during consultation. In accord with Kolb’s (1984) theory of learning, feedback can contribute to reflection and conceptualization as well as assist in improving clinician skill. Some studies suggest the effectiveness of training that involves live monitoring paired with performance-based feedback, such as the previously described training of direct care staff in community group homes for adults with severe intellectual disabilities (Harchik et al., 1992; Parsons et al., 1993). The previously described RCT comparing feedback and coaching following training in motivational interviewing (Miller et al., 2004) provides further support for the effectiveness of feedback as a consultation strategy.

Additional support for the effectiveness of feedback as a consultation technique comes from an RCT examining the training of elementary school teachers in behavioral interventions (Noell et al., 2005). Teachers were randomly assigned to one of three follow-up conditions after receiving training in how to conduct behavioral interventions: (a) weekly meetings to discuss implementation and student progress and to address teacher questions, (b) weekly meetings that incorporated the same content as condition one and discussion geared toward strengthening teachers’ commitment to implementation, or (c) performance feedback in the form of reviewing graphs of student behavior and the percentage of steps completed by the teacher each day, praising the teacher for the steps they completed well, and identifying areas in need of improvement. Of note, in contrast to the first two conditions, which consisted of weekly meetings, the performance feedback condition involved daily meetings that were gradually faded following improved implementation rates. Implementation rates, based on how completely the behavior monitoring records were filled out, and student outcomes, based on direct observations of behavior at baseline and at 3-week follow-up, were statistically higher in the performance feedback condition as opposed to the other two conditions, which did not statistically differ from each other. Performance feedback appears to be a promising consultation technique, however, given that the frequency of contact differed across conditions, future studies that involve equal allocation of time are needed before firm conclusions can be made regarding the critical role of performance feedback.

Summary

A hybrid of the Caplan and Caplan (1993) model and the Milne (2009) model provides an organizing framework to categorize consultation and guide DI facilitators in the provision of particular techniques during consultation (see Figure 1). In addition to examining the relative effectiveness of type of consultation, empirical investigations can examine which techniques or combination of techniques are most effective for certain outcomes and under certain conditions, in accord with a systems-contextual approach (Beidas & Kendall, 2010; Glisson et al., 2008; Sanders, Turner, & Markie-Dadds, 2002).

Figure 1.

Hybrid consultation model.

Future Directions

This review of the literature provides support for the use of consultation as a complementary implementation strategy alongside workshop training. Two models delineate what constitutes consultation and how consultation affects change in clinicians. Although this paper provides a preliminary review of what may constitute effective consultation techniques, many questions remain. Spence, Wilson, Kavanagh, Strong, and Worrall (2001) stated the following regarding supervision: “the evaluation of any supervision system needs to examine who needs what type of supervision, from whom, how often, at what cost, and to what benefit” (p. 150). A similar statement is warranted regarding consultation. We encourage the complex questions embedded within this statement to guide future research.

An important preliminary step in identifying critical components of consultation is to identify what consultation typically entails. The difficulty in this is quickly realized when reviewing the literature on education as an implementation strategy. Rakovshik and McManus (2010) criticized the training field for lacking detailed descriptions of training techniques used. With some exceptions (e.g., Harchik et al., 1992; Parsons et al., 1993), the studies reviewed herein also lacked detailed descriptions of the procedures involved in consultation, which makes it difficult to ascertain which type of consultation was provided and what specific techniques were used. Limited detail prevents researchers from replicating others’ work and from conducting needed dismantling studies. Thus, we encourage more detailed record-keeping regarding the specific consultation formats and techniques implemented during empirical investigations and DI efforts. We also recommend the allocation of additional journal space to the provision of such details. The creation of consultation manuals, such as the one created for MST (Schoenwald, 1998), and monitoring fidelity to manuals via standardized fidelity checks would contribute to more detailed knowledge of what consultation consists of and foster more methodologically sound examinations of the effectiveness of consultation components.

Despite lacking details, a review of the consultation literature yields numerous techniques worthy of further study, including but not limited to instruction/teaching (Heatley et al., 2005), case review (Lau et al., 2004), self-evaluation (Schoener et al., 2006), instructor and/or peer feedback (Lau et al., 2004; Harchik et al., 1992), role-plays and further didactics on topics requested by clinicians (e.g., Beidas et al., 2012), on-site observation (e.g., Parsons et al., 1993), consultation with experts in addition to case supervision with supervisors (e.g., Schoenwald et al., 2008), and relapse prevention consultation (e.g., Luoma et al., 2007). Although Miller et al. (2004) and Noell et al. (2005) conducted comparative analyses, it remains largely unclear whether, in their entirety, each technique is effective, whether each technique is equivalent to each other in terms of potency, and/or whether potency is enhanced by combining techniques or delivering them in certain ways and/or under certain conditions. Additionally, it is unknown whether certain approaches are more effective for certain types of consultation (e.g., whether self-evaluation is more effective during Type 4 versus Type 1 consultation). We recommend further comparative research that examines the effectiveness of consultation types and techniques as guided by the hybrid model presented above, which combines Caplan and Caplan (1993) and Milne’s (2009) work. In accord with a systems-contextual perspective (Beidas & Kendall, 2010) and the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009), we encourage examination of the contextual levels within which consultation is embedded and how contextual factors moderate consultation.

In addition to comparing the effectiveness of consultation techniques under various conditions, future studies would benefit from the improvement of outcome measures. Few studies reviewed examined client outcomes or clinician skill using psychometrically sound measures, which calls into question the ultimate impact of the findings. We recognize that client outcomes are a few steps removed from training and consultation and that effect sizes of the impact of training and consultation on client outcomes will be smaller than effect sizes of the impact of clinician behavior on client outcomes. However, we believe that the improvement of outcome measures would strengthen our understanding of consultation and improve future DI efforts. Further, using outcome measures gathered on patients as a tool for consultation is a potential avenue for future research, given initial outcomes suggesting the importance of outcomes tracking (Bickman et al., 2012).

Additional outcomes worthy of examination include cost-benefit analyses (Addis, Wade, & Hatgis, 1999). An investment of time and money is made in training and consultation efforts. Cost-benefit analyses would reveal whether this investment is warranted and whether certain types of consultation are more efficient than others. For example, a recent cost-effectiveness study indicated that either self-training may or expert-led training and consultation⁵ could be considered the most cost-effective strategy for training clinicians in motivational interviewing, depending on the threshold value of decision makers (Olmstead, Carroll, Canning-Ball, & Martino, 2011). Determining threshold values, thus, is important for concluding what training and consultation approaches are cost-effective. As we become better informed of what effective consultation consists of, we can focus efforts on delivering it in a cost-effective way.

Given that the ultimate goal is to reach all individuals in need of services, it is important to examine how to bring training and consultation to scale while maintaining effectiveness and feasibility (Beidas et al., 2011). Scalability refers to quickly increasing the number of trainees without significantly increasing the resources necessary to train people (Weingardt, Cucciare, Bellotti, & Lai, 2009). A promising area for scalability is the use of technology. What role can technology play in both the acquisition and maintenance of skill? Cucciare, Weingardt, and Villafranca (2008) suggest that technology-based platforms are cost-effective and offer the same opportunities for blended learning (integration of multiple learning methods) as traditional in-person trainings. Future research on the use of technology in training and consultation is encouraged given positive findings from initial studies. For example, Dimeff et al. (2009) found that online training was comparable to an instructor-led training program in dialectical behavior therapy. Similarly, Beidas et al. (2012) found roughly equivalent outcomes in trainees who completed computer-based training and trainees who completed in-person training and found that a web conferencing platform for consultation was both feasible and beneficial. Weingardt et al. (2009) found improved CBT knowledge acquisition and self-efficacy in substance abuse counselors who participated in online training and four web conferencing consultation⁶ sessions.

Conclusion

The President’s New Freedom Commission on Mental Health (2003) set forth an ambitious goal for transforming mental health care in America by making EBPs the standard of care. This review addressed a critical step in achieving that goal—training and supporting community clinicians in how to deliver EBPs. Research suggests that the most effective education implementation strategies to date incorporate ongoing support (i.e., consultation) following initial training. However, questions remain regarding which specific techniques contribute to the improved outcome. Additional research is necessary to answer the many remaining questions regarding which training and consultation methods, by whom, for whom, under which circumstances, and to what costs, are most beneficial. Pursuit of these answers will assist us in maximizing DI efforts and improving the well-being of individuals with mental disorders and society-at-large.