Consultation as an implementation strategy for evidence-based practices across multiple contexts: Unpacking the black box

Abstract

There is great interest in the dissemination and implementation of evidence-based treatments and practices for children across schools and community mental health settings. A growing body of literature suggests that the use of one-time workshops as a training tool is ineffective in influencing therapist behavior and patient outcomes and that ongoing expert consultation and coaching is critical to actual uptake and quality implementation. Yet, we have very limited understanding of how expert consultation fits into the larger implementation support system, or the most effective consultation strategies. This commentary reviews the literature on consultation in child mental health, and proposes a set of core consultation functions,processes, and outcomes that should be further studied in the implementation of evidence-based practices for children.

Implementation science, an emerging discipline over the past decade, focuses on understanding the dissemination and implementation of new innovations into various settings (e.g., Lomas, 1993). In mental health, as evidenced by the National Institute of Mental Health Strategic Plan (National Institute of Mental Health, 2008), there has been tremendous interest in identifying the most effective strategies to disseminate and implement evidence-based practices (EBPs) into various settings, including community mental health clinics and schools. Early work focused on understanding the implementation process from the perspective of barriers and facilitators(e.g., Harned, Dimeff, Woodcock, & Skutch, 2011; Langley, Nadeem, Kataoka, Stein, & Jaycox, 2010) whereas more recent work focuses on the use of implementation strategies(e.g., Glisson et al., 2012; Glisson et al., 2010).

Implementation strategies are active processes that increase the adoption, uptake, and sustainability of EBPs (Powell et al., 2011).The current issue is specifically focused on the role of consultation as an implementation support strategy once a decision to adopt an intervention has been made. Most of the research on this topic has been conducted around implementation strategies which fall under the umbrella of education activities, most commonly, training of providers in specific interventions (Powell et al., 2011). Typical training efforts tend to include printed educational materials (e.g., treatment manuals) and/or continuing education workshops (Herschell, McNeil, & McNeil, 2004). Reviews of such strategies in medicineand related fields by the Cochrane Collaboration, a vast international network of researchers that conducts systematic reviews of empirical research, suggest that printed education materials such as treatment manuals have minimal effect on therapist or patient outcomes (Farmer et al., 2008; Grimshaw et al., 2001; Giguère et al., 2012).Similarly, agrowing body of literature suggests that the use of one-time workshops as a training tool is ineffective in influencing therapist behavior, although they do influence therapist knowledge and attitudinal change towards EBPs(Beidas & Kendall, 2010; Grimshaw et al., 2001; Rakovshik & McManus, 2010).

A potential complement to one-time workshops appears to be ongoing support following training (Beidas & Kendall, 2010; Herschell, Kolko, Baumann, & Davis, 2010; Rakovshik & McManus, 2010). Outside of behavioral health, there is a burgeoning industry of consultation, coaching, and technical assistance offered in a variety of contexts, both private and public. Given that this field has developed largely outside of the confines of academia, systematic research evaluating consultation strategies has generally not been conducted.As EBPs are increasingly being taken upin the community, there is a need to translate approaches to training, supervision,and quality assurance that have been used in clinical trials into real world practice settings (Schoenwald, 2011). Bringing implementation science frameworks to study the various consultation approaches is essential to the effective transport of science into practice. Existing research that has examined provider-level behavior change for physicians, nurses, teachers, and others implementing innovative screening, health care practices, or teaching strategies highlights the vital role of focused consultation or coaching over time (e.g., Grimshaw et al., 2001; Grol & Grimshaw, 1999; Grol & Grimshaw, 2003; Joyce & Showers, 2002). Specifically, such research suggests the importance of ongoing interaction, specific feedback provided in a timely manner, and the use of multifaceted approaches that address providers’ training needs as well as barriers to implementation (e.g., Grimshaw et al., 2001; Grol & Grimshaw, 1999; Grol & Grimshaw, 2003; Joyce & Showers, 2002; Scheeler, Ruhl, & McAfee, 2004; Schouten, Hulscher, Everdingen, Huijsman, & Grol., 2008).

The existing evidencein behavioral health is very limited, however,research has demonstrated that external support after training predicted therapist fidelity following training, above and beyond type of training method (Beidas, Edmunds, Marcus, & Kendall, 2012). In this introduction, we will provide background on consultation, suggest a conceptual model, the Interactive Systems Framework, in which consultation isone componentof a broader implementation support system (Wandersman et al., 2008), and delineate the main functions, processes, and outcomes of consultation.

Background

We begin by clarifying terms. Many have been used to describe ongoing support as an implementation strategy (e.g., consultation, supervision, coaching, ongoing support)(Edmunds, Beidas, & Kendall, in press). We use the umbrella term consultation rather than supervision in the papers that follow. The reason is because supervision, as it is applied in mental health contexts, generally refers to a relationship where cliniciansare supervised as part of their work in an agency, clinic, or school district (Schoenwald et al., 2008). In our perspective, consultation, or coaching (as it is often referred to education), is the more appropriate term for external support provided within dissemination and implementation efforts. Consultation connotes an imparting of specific expertise in intervention techniques as well as experience and knowledge of the application of these techniques in different settings. Despite the differences, there are facets of consultation that are similar to supervision. Like supervision, consultation can be described as “relationship-based education and training that is work-focused and which manages, supports, develops and evaluates the work of colleague/s” (Milne, 2007).

In practical terms, the field of consultation in mental health is growing rapidly beyond the confines of research grants and encompassesnational,state-, county-, and local-policy (e.g., Gleacher et al., 2011; California Department of Mental Health, 2010; Lee et al., 2012;Oregon Department of Human Services, 2009; SAMHSA Bulletin, 2012).Expert consultation is used as a core component within many of the EBP training and technical support programs that have emerged from these policies. Thus, it is important to both define and understand the functions of and the most effective strategies for delivering consultation.A growing body of literature provides support for the use of ongoing, focused consultation across interventions and setting. A number of review papers have highlighted that ongoing consultation is critical to promote uptake and adherence to EBPs (Beidas & Kendall, 2010; Herschell et al., 2010). Yet, there is tremendous need for research to identify effective consultation strategies and the mechanisms through which consultation impacts implementation outcomes (Beidas & Kendall, 2010; Herschell et al., 2010; Rakovshik & McManus, 2010; Weisz, Ugueto, Herren, Afienko, & Rutt, 2011). In one empirical study, consultation hours predicted therapist adherence and skill following training in cognitive-behavioral therapy for child anxiety. Each hour of consultation improved adherence by .4 point and skill by .3 point on a 7 point Likert scale, suggesting a good return on investment (Beidas et al., 2012). Another study examining consultantadherence to a consultation protocolfor multi-systemic therapy found a link betweenconsultant adherence and therapist fidelity, as well as post-treatment youth behavior problems and functioning(Schoenwald, Sheidow, & Letourneau, 2004). Specifically, therapist-perceived consultant expertise was positively associated with therapist adherence and improved youth outcomes(Schoenwald et al., 2004).

Interactive Systems Framework

The Interactive Systems Framework (ISF) is a potential lens through which to place consultation within the larger systemic occurrences of any dissemination and implementation effort (Wandersman et al., 2008). Placing consultation within a larger framework is important because consultationfocused on a singular innovation is likely not a sustainable implementation strategy on its own. The ISF was developed in reaction to other implementation models which were largely focused on unidirectional transmission of information, from researchers to the community, and has been increasingly identified as a useful way to think about dissemination and implementation efforts (e.g., Chinman et al., 2012; Halgunseth et al., 2012; Lesesne et al., 2008; Smythe-Leistico et al., 2012; Taylor, Weist, & Deloach, 2012; Wandersman et al., 2008). Despite the relatively young discipline of implementation science, over sixty-one models have already been identified in dissemination and implementation science (Tabak, Khoong, Chambers, & Brownson, 2012).We considered other frameworks that provide guidance in how to conceptualize consultation (e.g., Berwick, 1996; Ferlie& Shortell, 2001;Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005). The selection of the ISF was guided by its ecological perspective, community-centered nature, emphasis on the dissemination and implementation process,and particularly its focus on the need for a support system through which to provide ongoing support to providers.

This heuristic framework is community-centered, in that it starts with what the community needs and desires and includes them as an active component of the dissemination and implementation process. The ISF posits three main systems which interact with one another and the larger system within which dissemination and implementation takes place. These systems include: (1) synthesis and translation system, (2) delivery system, and (3) support system. The purpose of the synthesis and translation system is to take research and to synthesize and translate it into a more usable product for consumers (i.e., therapists, clients). The delivery system refers to the delivery of services at the organizational, community, state or national level. The third system, the support system, is the most germane to consultation.

This support system refers to the underlying infrastructure to support providers in the implementation of EBPs through training and consultation and more generally supporting the organization in being as effective as possible (Wandersman et al., 2008). Clear delineation of what this underlying infrastructure must include is not available to date from the empirical literature. Evidence from systematic reviews in medicine (e.g., Grol & Grimshaw, 1999; Grimshaw et al, 2001; Nadeem, Olin, Hill, Hoagwood, & Horwitz, in press), empirical research on innovation implementation (e.g., Klein, Conn, & Sorra, 2001), and emerging research on organizationally-focused strategies for improving clinical care (e.g., Glisson et al., 2010; Saldana & Chamberlain, 2012)suggest that a successfulsupport system is likely to include administrative or leadership support, a facilitative organizational climate and culture, engagement of stakeholders and staff at multiple levels, ongoing feedback, and direct clinical and logistical support to implementing providers. Clinical consultationis an important component of this broader support system.

To help us conceptualize the interactions between the support and delivery systems in the ISF, a recent logic model posits the necessary components of an evidence-based system for innovation support (EBSIS). This includes tools, training, technical assistance, and quality assurance (Wandersman, Chien, & Katz, 2012). EBSIS suggests that all four of these components are necessary to ensure best outcomes in implementation of EBPs. Technical assistance is defined as “an individualized, hands-on approach to building an entity’s capacity for quality implementation of innovations, usually following training” (Wandersman et al., 2012), and maps on to our primary focus of this special issue, consultation. Expert consultation is an important component of this support system, however, there has been little work to date to help us understand how this operates (Wandersman et al., 2012). A review of the literature suggests that several important empirical findings have been identified with regard to dosage, delivery-method, collaboration, and proactive nature of consultation (Wandersman et al., 2012). From this review, Wandersman and colleagues (2012) make the following recommendations. First, it appears that the more consultation that is provided, the better the outcomes are, although there may be important moderators of this relationship. Second, on-site consultation has been recommended as the gold-standard although off-site/virtual consultation may be more cost-effective and perhaps easier for community therapists to attend. Research is needed to understand when on and off-site consultation is empirically indicated particularly relevant to the intervention the consultation is designed to support (Whitaker, Lutzker, Self-Brown, & Edwards, 2008). Collaboration and customization of consultation to the needs of stakeholders is also important(Wandersman et al., 2012). Although this model provides important first steps in delineating the functions and processes of consultation, more empirical work is needed to move the fieldforward, particularly across multiple contexts.

Primary Functions,Processes, and Outcomes of Consultation

As noted above, there is a dearth of research delineating the core components of consultation (Edmunds, Beidas et al., in press), including processes,functions, and outcomes(Weisz, Chorpita et al., 2011). The primary purpose of this special issue is further develop the empirical literature related to consultation across different child-serving settings (e.g., schools, community mental health clinics).To help frame this empirical work,we offer definitions of consultation processes and functions based on a selective review of the literature and our collective experiences with consultation. The papers in this issue present new findings related to some of these different functions, how they relate to each other, and impact outcomes.

Consultation Functions

Based on our review of the literature and our experiences as consultants, the core functions of consultation likely include: (1) continued training, (2) problem-solving implementation barriers, (3) provider engagement, (4) case support, (5) accountability, (6) mastery skill-building, (7) appropriate treatment adaptation and (8) planning for sustainability (See Table 1).

Table 1.

Functions of Consultation

Functions	Subfunctions
Continuing training closer in time to implementation of new skills	Case conceptualization and assessment (initial and ongoing)
	More extensive training on intervention components and additional relevant topics
	Provision of further resources (e.g., patient-specific resources like CBT for comorbid conditions; prior experiences from other settings, or research literature, additional treatment or assessment resources)
Problem-solving implementation barriers	Client and community (e.g., perceived relevance)
	Provider (e.g., prior experience, attitudes)
	Organizational (e.g., support from leadership, resources, buy-in from providers, supervisor, and leadership)
	Treatment, intervention, or program (e.g., innovation-setting fit; ease of use)
Engagement	Direct service providers
	Leadership, supervisors, and other relevant staff
Case application	Intervention-client fit
	Skill application and refinement
Accountability	Holding providers and consultants accountable to what has been agreed upon (this can include discussion of fidelity measures and adherence monitoring tools)
Treatment adaptation	Client presentation, culture, age
	Clinicians practice and values
	Organizational context
Mastery skill-building	Modeling
	Behavioral rehearsal
	Spaced practice
	Direct feedback (e.g., audit and feedback, fidelity monitoring, behavior rehearsal)
Sustainability planning	Helping providers and organizations continue to use a practice
	Helping organizations build infrastructure for internal capacity (e.g., internal supervision, peer supervision)
	Helping organizationsplan for their own funding

Continued training

One of the primary functions of consultation is to continue to provide didactic training (Beidas et al., 2012; Edmunds et al., this issue). This is particularly important for complex skills that cannot be taught fully in a one or two-day workshop, such as case conceptualization,assessment, and the provision of multi-session complex treatment protocols. Continued training through consultation also allows opportunity for more in-depth didactics related to the EBP and provision of further resources.

Continued instruction on case conceptualization and assessment

Assessment and caseconceptualization is a key facet to the provision of quality treatment, yet typical workshop trainings often do not allow for sufficient time to address these issues. Recent research has shown a great deal of imprecision and inconsistency in diagnostic and assessment practices, which is a significant issue given that the selection of appropriate treatments is a foundational aspect to the implementation of EBPs (Jensen-Doss, 2005; 2011). Moreover, the use of assessment tools to continually track progress, tailor interventions, and provide feedback is an important focus for consultation, especially given the minimal use of data to guide practice (Kelley & Bickman, 2009).

Continued instruction on core intervention components

Given that most training workshops are brief, it can be difficult to cover all of the topics necessary to administer a complex multi-session EBP. In our experience as consultants, we have found three primary purposes for continued didactics on intervention content: review, keeping information fresh, and tailoring consultation to the needs of the consultees. Continuing to review learned materials allows for deeper understanding and spaced practice with topics. Second, it allows for exploration of topics closer in time to the actual implementation of the EBP, which is an important predictor of eventual success (Palinkas et al., 2008). Third, ongoing didactics in consultation can be developed in collaboration with provider requests on topics that they feel they need further help with, which can generate more engagement and interest in the consultation process (Wandersman et al., 2012). For example, in one recent consultation study, participants were queried during the training to identify the potential topics on which they would like further didactic instruction on during consultation in an effort to collaboratively develop the consultation curriculum (Beidas et al., 2012) and this responsiveness was reported as a useful component of consultation in qualitative interviews with therapists (Beidas et al., this issue).

Further resources

Having the opportunity to continue to interact with providers after training allows for consultants to provide a wealth of resources in the time that they are needed. For example, providers may have a specific client population for which they need further resources (e.g., comorbid conditions, health conditions) which they can raise in consultation and be directed to the latest literature rather than having to find the information on their own. Further, consultants can share information, literature, and practical tools that are germane to the specific questions and emerging issues. These include successful adaptations or refinements that others have made, sample forms and documentation, and information about other practices that may be related to treatment provision (e.g., crisis response and management).

Problem-solving implementation barriers

Successful implementation hinges not only on providers’ learning the intervention itself but also on their ability to surmount barriers to implementation. Recent conceptual models for dissemination and implementation of interventions highlight corefactors that could be barriers or facilitators to use of a particular innovation(e.g., Aarons, Hurlburt, & Horwitz, 2011;Damschroder et al., 2009; Feldstein & Glasgow, 2008; Fixsen et al., 2005; Greenhalgh, Robert, MacFarland, Bate, & Kyriakidou, 2004). These models all identify factors at multiple levels of the ecological model, such as: (1) characteristics of the intervention (e.g., ease of use, relevance,compatibility with the setting), (2) characteristics of the service system (e.g., policy, regulations, financing),(3)organizational factors within the broader support system (e.g., time, money, tangible supports,leadership and staff buy-in and engagement), (4) provider characteristics (e.g., attitudes, prior experience, prior training), (5) community characteristics (e.g., perceived relevance by community members, community demographics), and (6) client factors (family engagement in the intervention, relevance to family needs).

Through ongoing consultation, consultants can help providers, clinic and/or school administrators, and other staff in an organization to address issues at each level. This can be accomplished by consultants’ sharing of experience from the field, sharing specific strategies that have worked in similar settings, and facilitating discussion amongst providers at different levels in the organization (i.e., clinicians or teachers, supervisors, managers). Learning collaboratives, which have become increasingly popular as an implementation strategy for supporting the uptake of EBPs in mental health, use expert consultants to facilitate regular dialogue about implementation barriers among quality improvement team members from multiple sites(e.g., Ebert, Amaya-Jackson, Markiewicz, Kisiel, & Fairbank, 2012; Institute for Healthcare Improvement, 2003). Through ongoing phone calls and in-person meetings, consultants share their own expertise on a given practice and provide a platform for peer support where providers can discuss specific implementation barriers they are encountering, share strategies they are using to try and address these barriers, and learn from the successes and failures of others. Although the evidence base for this particular approach still needs to be built, it may be a promising strategy(Nadeem, Olin, Hill, Hoagwood, & Horwitz, in press; Schouten et al., 2008).

Provider engagement

To successfully implement an EBP, engagement is needed from stakeholders, including providers, administrative leadership, supervisors, and related staff. This type of engagement is a complex process and requires supportive conditions created at multiple levels of the implementation support system, particularly from leadership. In particular, organizational leadership helps create the culture, climate, and infrastructure that allows providers the time, space, and resources they need to implement a new practice, and play a key role in obtaining buy-in from their supervisors and colleagues (e.g., Aarons, Glisson et al., 2012).The Availability,Responsiveness, and Continuity (ARC)organizational intervention provides one example of an theoretically-driven, empirically-tested approach to providing expert consultation focused on organizational social context variables such as culture, climate, and structure(e.g., Glisson et al., 2012; Glisson et al., 2010).

While it may not be feasible or necessary in all instances to use the ARC model to improve organizational climate and culture, it would behoove clinical consultants, who have traditionally focused on training and supporting clinicians to implement a specific treatment, to expand their skill sets to include approaches that target organizational social context factors and allow them to more effectively interface at different levels of the organization. Expert consultants can consult directly to organizational leadership to help support EBP implementation in ways that are appropriate to their role, a component of quality improvement collaboratives and ARC (e.g., Glisson et al., 2012; Nadeem et al., in press).As a further note, consultants must recognize the importance of quality interpersonal relationships and collaboration as they take their role within the larger implementation support system (e.g., Mitchell, Stone-Wiggins, Stevenson, & Florin, 2004; Salyers, McKasson, Bond, & McGrew, 2007; Wandersman et al., 2012).

Expert consultation can also play an important role in supporting a favorable implementation climate. In addition to direct consultation to providers and other staff within an organization, the very presence of an expert consultant indicates to providers that their organization is committed to supporting their learning. A study of provider perspectives on consultation indicated that consultation engages providers by providing connectedness in learning through the groups’ shared experiences, authenticity of experience (discussing actual cases), and by being responsive to provider needs (Beidas, Edmunds et al., this issue).

Direct case application

As providers begin to field a new practice, there is a need for the consultation to focus on direct translation of the new techniques to real-world cases (e.g., Edmunds et al., this issue; Gleacher et al., 2011; Palinkas et al., 2008). In consultation, providers can share their experiences trying the techniques with the children, families, teachers, or classrooms that they are working with. Consultants can provide specific feedback regarding fit of the intervention with the presenting issues and how to most effectively engage families in the treatment process. They may also help providers refine their skill set, identify specific strategies they may use with their clients, and practice the actual language they will use in session.

Appropriate treatment adaptation

As providers begin to apply the intervention in real world settings, questions often arise about possible adaptations to the practice. Providers may seek to adapt an intervention in order to fit with clients’ cultural values and beliefs (Bernal, Jimenez-Chafey, &Domenech Rodríguez, 2009), clients’ developmental levels (Piacentini & Bergman, 2001), or to address additional clinical issues (Chorpita, Daleiden, & Weisz, 2005).Providers may also wish to incorporate their own ways of conveying a skill by replacing an activity from a manual with one that they have used in the past. Researchers have also begun to explorehowlongstanding theories of behavior change (e.g., theory of planned behavior) may fuel providers desire to adapt and modify treatments. These theorieshighlight the role of behavioral, intentions, which are influenced by factors such asperceived social norms, expectancies, self-efficacy, and attitudes, all of whichoperate on both the provider and client-level(Pappadopulous, Siennick, Schur, & Gentry, 2002). Additionally, at the organizational level, sites may be seeking to make adaptations to enhance innovation-setting fit. For example, there may be some organizational, logistical, or financial constraints that would only allow for billing of individual sessions and not group sessions, and sites may use consultation to help them determine how such changes could impact their adherence to the core components of anEBP (e.g., Gleacher et al., 2011).

Although there has been very little research on the efficacy of interventions after they have been adapted, there is general consensus that any adaptations must retain corecomponents of the intervention (e.g., Amaya-Jackson & DeRose, 2007;Ngo et al., 2008). For instance,some studies of have found that tailoring interventions can increase their effectiveness(e.g., Bernal, 2006; Hirachi, Catalano, & Hawkins, 1997; Rossello & Bernal, 1999). However, there is evidence that some cultural adaptations may actually dilute the effectiveness of the original treatment(Kumpfer, Alvarado, Tait, & Turner, 2002).In order to help organizations make thoughtful adaptations, Aarons and colleagues (2012) have developed the Dynamic Adaptation Process (DAP), a collaborative, multiple stakeholder process that involves identifying core elements and adaptable characteristics of an EBP, providing training and support that incorporates planned adaptations, and fidelity monitoring. As organizations consider and implement adaptations, consultation around these issues is important.

Accountability

Accountability refers to the process of holding providers and consultants accountable to what has been agreed upon prior to engaging in the consultation-consultee relationship. Importantly, it is a bidirectional relationship; both parties must be accountable. On the provider end, ongoing consultation meetings promote accountability because providers must report their progress and experiences using the intervention to an outside party (Falender & Shafranske, 2004). More specifically, consultation can provide a forum for discussing content from the fidelity and adherence monitoring systems (e.g., adherence checklists completed by providers, clients, and supervisors, review of recorded sessions, direct observation) or other performance indicators that sites may put in place (e.g., Schoenwald et al., 2011; Schoenwald, Henggeler, Brondino, & Rowland, 2000) which can be conceptualized as part of the quality assurance process (Wandersman et al., 2012). This may be a more cost-effective and efficient way to capture fidelity, rather than the typical and time-consuming way fidelity is measured in randomized controlled trials (i.e., independent evaluators rating videotaped sessions).

Accountability should also be modeled within the consultation itself, where the consultant is also accountable to mutual goals set before embarking on the consultation relationship. Theoretically, consultation from a CBT perspective should consist of collaborative goals and specific agendas for consultation sessions that can monitored over the course of the consultation (e.g., Milne, 2007). The relationship between the consultee and consultant may also serve as motivation for greater accountability by engaging providers and encouraging them to come prepared to consultation sessions (Beidas et al., this issue).In practical terms, if an organization is paying for consultation, there may be more mutual accountability of the organization, including its leadership, and consultant for the outcome of the consultation.

Mastery skill-building

Often, didactic workshops are delivered passively, which is not consistent with the adult learning literature (Beidas & Kendall, 2010). Active learning has been suggested as the most effective way to impart clinical skills because it enhances learning, particularly when engaging in new or complex skills (Cross et al., 2011).Active learning means that the individual experiences and reflects while learning (Kolb, 1984) which is typically accomplished using modeling and practice opportunities. One method that can be used is the behavioral rehearsal methodology (Beidas, Cross, & Dorsey, in press) which can be defined as a simulated interaction between a therapist and a trained actor where the therapist takes on the role that they are expected to take on in the future (Cross, Matthieu, Cerel, & Knox, 2007). Behavioral rehearsal has been found to be an effective way to increase provider fidelity (Cross et al., 2011). This methodology can be used within the context of consultation to increase the likelihood of mastery skill-building. In a recent study, behavioral rehearsal was used during virtually delivered consultation sessions (Beidas et al., 2012; Edmunds, Kendall et al., this issue). Further, behavioral rehearsal can be paired with performance feedback to further improve provider skill and adherence (Ivers et al., 2012; Scheeler et al., 2004). The extant literature on audit and feeback may provide guidance in developing methods for performance feedback within the mental health context and points to the importance of sufficient intensity of feedback (Jamtvedt, Young, Kristoffersen, O’Brien, & Oxman, 2006).

Planning for sustainability

Consultation can also be used to help providers and organizations plan for when they will no longer be receiving expert support, given that consultation is likely to be time-limited (although we recommend that the line between expert consultants and providers always stay open to facilitate the bidirectional relationship necessary for dissemination and implementation work;Caplan & Caplan, 1993). Consultation can serve this purpose through three primary ways: (1) to assist in the continued use of an EBP, (2) to provide consultation on how to build infrastructure and internal capacity for provision of the EBP, and (3) to help organizations consider how to plan for their own funding to sustain a practice. The recent establishment of technical assistance and training centers through states and other governmental entities may help to facilitate this type of consultationand support (e.g.,Gleacher et al., 2011; Lee et al., 2012;Oregon Department of Human Services, 2009).Further, at the policy level, changes in insurance, Medicaid, and Medicare reimbursment policies to allow providers to bill for consultation could also faciliate the sustainability of consultation. Without this policy change, it may be difficult to sustain consultation in a fee for service setting where an hour spent in consultation is synonymous with lost revenue.Although directly engaging in this kind of reform is beyond the scope of what EBP-focused consultants can do and are trained to do, they may have a role to play in advising decision-makers when appropriate.

Consultation Processes

Cutting across all of the above categories are some of the processes that have been identified in reviews of CBT-based supervision and consensus statements on quality supervision from the broader literature on training and supervision (e.g., Falender & Shafranske, 2004; Milne, Aylott, Fitzpatrick, & Ellis, 2008). Specifically, this literature has identified agenda-setting, collaborative goal setting, formulation, planning, discussion (including disagreements, communication of understanding, challenging, explanation), didactic instruction, modeling, role-play, monitoring or observation, review and reflection, summarizing, confidence-building, praise and reinforcement, and feedback as core processes that can be used in consultation(Milne, 2009; Milne et al., 2008). One challenge is that these processes are not implemented consistently in the field, as both consultants and trainees may shy away from the processes that are most linked to mastery skill building. Further, with the exception of a study linking therapist-perceived consultant adherence to therapist fidelity (Schoenwald et al., 2004), there has been minimal research examining how the use of these supervision processes in consultation relates to implementation of the EBPs.

Consultation Outcomes

Clearly defining the outcomes of consultation is as critical as identifying the functions and processes. As part of the ISF, the ten-step Getting to Outcomes (GTO) framework has been proposed to allow stakeholders to identify what the desired outcomes are when implementing a new intervention (Wandersman et al., 2010). In the second step of GTO, stakeholders can define what the goals of consultation ought to be collaboratively based on the needs assessment that is conducted in step one. Later in the GTO framework, these outcomes are measured as part of the evaluation process. This is one potential way to consider which outcomes of consultation to be concerned with, and its strength includes the collaborative nature of deciding outcomes. However, this likely would result in different outcomes based on contexts and settings.

We also suggest that research on consultation should consider the relevant implementation and client outcomes delineated by Proctor and colleagues (2009; 2011). This allows for a standard and uniform set of outcomes that may be measured and which would allow the field to build upon emerging findings. Because consultation is a type of implementation strategy, it stands to reason that implementation outcomes are the most proximal outcomes. Potentially salient variables include:acceptability, adoption, appropriateness, costs, feasibility, fidelity, innovation penetration, and innovation sustainability (Proctor et al., 2011). Further, improved service outcomes, although more distal to consultation, may be ultimate indicator of success. Service outcomes include client-level satisfaction, functioning, and symptomatology.

As researchers develop studies of consultation, study outcomesshould align with the specific research questions being addressed. These may relate to the specific functions of consultation as they are outlined in this paper, the particular consultation model being used, the sequencing and role of consultation within a broader implementation strategy, the phase of implementation being addressed,and the agreed upon goals by consultants and consultees.In the case of consultation to support implementation of a specific treatment after it has been adopted, fidelity to the intervention model, uptake, and penetration may be primary proximal outcomes, with client-level symptoms and functional outcomes the secondary distal outcomes. In addition, examination of important, but understudied outcomessuch ascost-effectiveness require thoughtful alignment with the research questions. For instance, there may be questions about the incremental cost-benefit of one consultation approach over another in achieving the desired client outcomes, provider fidelity, or penetration of a particular innovation.

The Current Issue

The studies in this special issue help us to identify the potential effects of consultation on implementation outcomes in the context of intervention and implementation research across a range of practices and contexts (e.g., schools and community agencies; teachers and clinicians; prevention and treatment programs), begin to “unpack” consultation process itself, and help us place consultation within the larger implementation support system. They also represent different consultation models, ranging from expert consultation (or coaching) in the context of clinical trials to observational studies of consultation in the context of real-world, large-scale rollouts of EBPs.

Five of the papers in this issue focus on the relationship between features of the consultation or coaching (e.g., frequency, content, coaching strategies) and implementation outcomes (e.g., implementation quality, adherence, therapist satisfaction). Papers from Becker, Domitrovich, & Ialongo (this issue) and Reinke, Herman, Stormont, Newcomer, & Davis (this issue) accomplish this within the context of clinical trials on school-based interventions delivered by teacher. Edmunds and colleagues (this issue) and Bearman and colleagues (this issue) focus their investigations on whether active learning strategies (e.g., behavioral rehearsal, skill modeling) predict the implementation of evidence-based therapeutic elements within therapy sessions for youth receiving treatment in community mental health clinics. Masia and colleagues (this issue) provide information on therapist’s fidelity in delivering a treatment for social anxiety in the school setting from the perspective of expert supervisors.

The next set of papers help us to understand the fit of consultation for providers and for the larger implementation support system as it is conceptualized in the Interactive Systems Framework (ISF) (Wandersman et al., 2008).Lyon, McCauley, Ludwig, Vander Stope, & Cosgrove (this issue) use mixed methods to better understand school-based mental health provider perspectives on the factors that influenced decisions about initial participation and continued participation as well as the features of consultation that appeal most to them. Similarly, Beidas and colleagues (this issue) use qualitative methods to identify therapists’ perspectives on the effective aspects of consultation. Finally, within the context of a statewide evidence-based treatment dissemination program, Nadeem, Gleacher, Pimentel, Hill, & Hoagwood (this issue) explore the content of consultation calls provided to clinic-based supervisors who supervised the clinicians who took part in this statewide program in order to identify some of potential consultation structures for large-scale implementation efforts conducted outside of research studies. Our overarching purpose is to assemble the most recent thinking and research on consultation processes in order to advance both its empirical base and its practical application.

Conclusion

It is now widely accepted that ongoing consultation is critical to successful implementation of a new program or practice in real world settings. As such, consultation in some form or another is included in most successful implementation efforts be it a grant-funded effectiveness study or a single organization rolling out a new program. Yet, despite the widespread use of expert consultation, questions remain with regard to the adequate dosage, ingredients, and processes through which consultation operate. A next step in the field is for us to develop a better understanding of the core functions,processes, and outcomes of consultation and place our emerging consultation models within the context of the broader implementation support systems that organizations, states, and other localities have in place to support EBPs. The articles in this special issue each provide the field with an important perspective and data to bear on these issues. As we unpack consultation processes and identify the core consultation needs from providers and organization, the next steps will be for us to explicitly and empirically test the impact of consultation strategies and models on implementation outcomes and child and family outcomes.