“Old Dogs” and New Skills: How Clinician Characteristics Relate to Motivational Interviewing Skills Before, During, and After Training

Abstract

Objective

The relationships between the occupational, educational, and verbal-cognitive characteristics of health care professionals and their Motivational Interviewing (MI) skills before, during, and after training were investigated.

Method

Fifty-eight community-based addiction clinicians (M = 42.1 yrs., SD =10.0; 66% Female) were assessed prior to enrolling in a two-day MI training workshop and being randomized to one of three post-workshop supervision programs: live supervision via tele-conferencing (TCS), standard tape-based supervision (Tape), or workshop training alone. Audiotaped sessions with clients were rated for MI skillfulness with the Motivational Interviewing Treatment Integrity (MITI) coding system v 2.0 at pre-workshop and 1, 8, and 20 weeks post-workshop. Correlation coefficients and generalized linear models were used to test the relationships between clinician characteristics and MI skill at each assessment point.

Results

Baseline MI skill levels were the most robust predictors of pre- and post-supervision performances. Clinician characteristics were associated with MI Spirit and reflective listening skill throughout training and moderated the effect of post-workshop supervision method on MI skill. TCS, which provided immediate feedback during practice sessions, was most effective for increasing MI Spirit and reflective listening among clinicians with no graduate degree and stronger vocabulary performances. Tape supervision was more effective for increasing these skills among clinicians with a graduate degree. Further, TCS and Tape were most likely to enhance MI Spirit among clinicians with low average to average verbal and abstract reasoning performances.

Conclusions

Clinician attributes influence the effectiveness of methods used to promote the acquisition of evidence-based practices among community-based practitioners.

Motivational Interviewing (MI) is a client-centered, collaborative, counseling style that guides individuals towards resolving their ambivalence about making important changes in their behavior (Miller & Rollnick, 2009). MI has demonstrated efficacy for treating alcohol and drug use disorders, as well as other health related behaviors, across a range of treatment settings and patient populations (e.g., Hettema, Steele, & Miller, 2005). MI has been successfully implemented as a stand-alone counseling style (Rollnick, Miller, & Butler, 2008), has been a central component of structured therapies that provide individuals with personalized feedback about their substance use (Ball et al., 2007; Project MATCH Research Group, 1997), and has been used as a prelude to other skills-based interventions (Steinberg et al., 2005). The broad applicability of MI suggests it is a highly transferable and flexible intervention that can directly promote important lifestyle changes as well as enhance the utility of other treatments.

Identifying the necessary and sufficient components of an effective counseling style is guided by the theoretical model upon which a treatment is based and evidence that supports a link between specific elements of that counseling style and clinical outcome. Theoretical formulations of both the process of learning MI (Miller & Moyers, 2007) as well as its therapeutic efficacy (Miller & Rose, 2009) have highlighted the importance of the clinician’s overarching therapeutic stance (i.e., MI Spirit) as well as the directional use of counseling skills, in particular reflective listening, in the service of influencing specific forms of client speech (Miller & Moyers, 2007). MI Spirit is characterized by a collaborative counselor-client relationship, in which the client’s autonomy is respected and his or her own reasons for change are elicited. Evidence supports the importance of relationship variables, such as a positive therapeutic alliance and a collaborative partnership, as these factors have accounted for a significant amount of variance in treatment outcome across a range therapeutic modalities and populations (Martin, Garske, & Davis, 2000; Najavits & Strupp, 1994), including substance abusers (Crits-Christoph et al., 2009, 2011; Meier, Barrowclough, & Donmall, 2005; Moyers, Miller, & Hendrickson, 2005; Najavits & Weiss, 1994). In contrast, a confrontational, less collaborative, and more directive counseling style, a style that is inconsistent with MI, can negatively impact the efficacy of psychosocial interventions across a range of settings (Najavits & Strupp, 1994) and has yielded poor treatment outcome among individuals seeking help for their substance use (Gaume et al., 2009; Miller, Benefield, & Tonigan, 1993). While reflective listening is central to many therapeutic modalities, including client-centered counseling styles (e.g., Traux & Carkuff, 1967), it has been identified as a very important, although challenging, skill to learn when building an MI repertoire (Miller & Rollnick, 2002). Clinicians’ MI skillfulness is based on their effective use of reflective listening, including the use of complex reflections (Miller & Rollnick, 2002). Thus, when learning MI, clinicians are encouraged to utilize more reflections and fewer questions and are oriented to the directional use of complex reflections, statements that can add substantial meaning or emphasis to what a client says, to help guide a client towards change.

While MI incorporates therapeutic skills (e.g., a collaborative stance and reflective listening) found in other treatment modalities, it has been described as a “simple but not easy” counseling style to learn (Miller & Rollnick, 2009). Interactive training workshops have been the primary avenue for introducing clinicians to MI as well as to other evidence-based practices (Baer et al., 2004; Ford, 1979; Madson, Loignon, & Lane, 2009; Rounsaville et al., 1986). However, while workshop training alone can promote the acquisition of new counseling skills, it does not yield sustained changes in clinical practice (Miller & Mount, 2001; Walters et al., 2005). Training programs that can approximate a clinician’s treatment encounters, as well as provide post-instruction supervision that includes feedback and coaching, offer the most effective framework for influencing clinical practice (Ford, 1979; Grol, 2001; Henry, Schaacht, Strupp, Bulter, & Binder, 1993), including the acquisition of MI (Miller et al., 2004; Smith et al., in press). This training structure has demonstrated consistently stronger efficacy compared to less intense training formats for increasing both MI Spirit and reflective listening relative to the use of questions (Miller et al., 2004; Moyers et al., 2008), although the evidence is less consistent for increasing the use of complex reflections (Miller et al., 2004; Moyers et al., 2008). Importantly, despite these effective training strategies, there remains significant variability in the acquisition and implementation of new counseling skills among trainees (Miller et al., 2004; Schoener et al., 2006). Thus, while training strategies may help reduce the gaps in skill level among clinicians, current training practices do not level the field in terms of counseling skill. These findings suggest that factors other than the structure of a training program may play an important role in the acquisition and implementation of new counseling practices.

Few studies have systematically investigated how clinician characteristics influence the effectiveness of training workshops and different supervision programs designed to train community-based clinicians in new evidence-based practices. However, recent conceptual frameworks highlight clinician attributes as being one important factor in promoting the successful dissemination and implementation of evidence-based practices into the community (Beidas & Kendall, 2010). Moreover, clinician training is the primary mechanism by which individuals come into contact with empirically supported treatments. While a “one-size-fits-all” training structure may not necessarily be the most efficient or cost-effective framework for influencing clinical practice (Carroll, Martino & Rounsaville, 2010; Martino et al., 2011), the time and resources needed to train clinical staff are directly related to how a training protocol is designed (Olmstead, Carroll, Canning-Ball, & Martino, 2011). Thus, understanding which training procedures are most effective for different pre-training skill-sets can be an important step in promoting a more efficient dissemination of evidence-based practices within the health services fields (Beidas & Kendall, 2010).

Certain clinician characteristics, such as therapeutic orientation and counseling style, correlate with the self-reported use of MI counseling techniques (Ball et al., 2002) and can influence the efficacy of training strategies for other empirically validated treatments (Sholomskas et al., 2005). Several lines of investigation have also studied the relationship between a clinician’s baseline skill set, professional and educational experiences, and the acquisition of a new clinical repertoire. In general, evidence suggests having stronger counseling skills prior to training is associated with greater skill acquisition following training (Ford, 1979; Henry, Schacht, et al., 1993; Rounsaville et al., 1986; Siqueland et al., 2000). This finding has also been demonstrated among individuals learning MI (Martino et al., 2011; Smith et al., in press). Clinical experience, operationalized as the number of years in the counseling profession, has been positively associated with the acquisition of counseling skills in some studies (Henry, Schacht, et al., 1993, Siqueland et al., 2000), unrelated to changes in MI Spirit and reflective listening in another investigation (Baer et al., 2009), and there is some evidence to suggest that greater professional experience may indicate the need for a more intensive training program in order to facilitate the acquisition of the fundamental skills of MI (Martino et al., 2011). Evidence for educational degree has been mixed; while a clinician’s degree status has not been consistently related to treatment outcome (Durlak, 1979; McLellan, Woody, Luborsky, & Goehl, 1988), having a graduate degree has predicted better MI skill acquisition in some investigations (Baer et al., 2009) but not others (Hartzler et al., 2007).

In addition to clinicians’ professional and educational experiences, it has been argued that knowledge of the cognitive characteristics of trainees may help guide a more refined attribute-by-training match, which in turn can improve the effectiveness of supervision techniques (Holloway & Neufeldt, 1995). Theoretical models of therapist skill development have highlighted the importance of higher order cognitive processes in the development of counseling skills and argue for specific methods to influence these mechanisms during training (Bennett-Levy, 2006; Binder, 1999). Cognitive ability significantly predicts academic and job performance across a range of educational and occupational settings (Ones, Viswesvaran, & Dilchert, 2005; Kuncel & Hezlett, 2010) as well as skill acquisition following training (Colquitt, LePine, & Noe, 2000). Further, verbal and quantitative abilities, as assessed by the Graduate Record Examination, have been utilized for selecting candidates into graduate level training programs and significantly predict the academic performance and faculty ratings of trainees during their training (Kuncel, Wee, Serafin, & Hezlett, 2010). However, to date, there has been little investigation of how the verbal-cognitive skills of clinicians relate to the specific methods used to facilitate the development of a new counseling repertoire. MI is a verbally-based process that relies on the clinician’s skill to educe information during a clinical interaction and to convey this information to the client verbally in the form of reflections. To the extent that a clinician’s verbal and abstract reasoning skills differentially impact the utility of different training methods for teaching MI skills, specific training methods may be matched to the strengths and needs of potential trainees.

This report presents the results of analyses that tested the relationships between the occupational, educational, and cognitive characteristics of health care professionals, and their acquisition of reflective listening skills and a collaborative therapeutic stance that promotes a client’s autonomy and emphasizes their ideas about change, two important components of an MI counseling style. The analyses are based on a subset of clinicians (i.e. those who completed all baseline assessments) enrolled in a larger MI training study (Smith et al., in press). In the original report, Smith and colleagues (in press) randomized community substance abuse treatment clinicians to one of three methods of supervision after completing a training workshop: Workshop only (WO), Tape-based Supervision (Tape), and Tele-Conference Supervision (TCS). Results indicated that clinicians in the TCS group demonstrated greater MI skill than clinicians in the WO group on measures of MI Spirit and Reflection to Question ratio, and exhibited a significant decrease in MI non-adherent behaviors (i.e., MINA). Those in the Tape group outperformed TCS on the percentage of Complex Reflections. Building on these findings, this study seeks to advance our understanding of health care professional training by addressing three important questions in the context of training a diverse sample of community-based clinicians in an MI counseling style: 1) What is the relationship between clinician characteristics and MI counseling skill level prior to formal training? 2) What is the relationship between clinician characteristics and the magnitude of change in skill level following a standard training workshop? and 3) Do clinician characteristics moderate the efficacy of different post-workshop supervision methods? Based on previous findings it is hypothesized that: 1) pre-training skill level will be significantly related to post-training skill across all outcomes and supervisory conditions; 2) tape supervision, a more delayed supervision structure that is more heavily weighted on verbal discourse and the analysis of past sessions, will be more effective for clinicians with a graduate level of training, more clinical experience, and stronger verbal and abstract reasoning performances for all MI skill outcomes; and 3) TCS, a more direct and immediate training structure based on supervisor prompts and modeling, will be more effective for clinicians with less professional experience, a non-graduate level of training, and lower verbal and abstract reasoning performances.

Method

The Institutional Review Board of the New York State Psychiatric Institute and the Institutional Review Boards governing each of the participating treatment clinics approved all study procedures. A more detailed account of the screening, training, and evaluation procedures is presented in Smith et al., (in press). A description of the procedures that pertain to this study is presented below.

Clinician Participants (Trainees)

Clinician participants from 26 community substance abuse treatment programs affiliated with the Long Island and New York Nodes of the NIDA Clinical Trials Network (CTN) were invited to participate. Clinicians had to be between 18 and 75 years of age, to provide counseling services directly to clients, and to be employed at least half time at their treatment program in order to participate in the study. Potential clinician participants could not enroll if they had attended an MI workshop in the three months prior to beginning the study, had received training to be an MI trainer, or had participated in a previous MI research trial.

Supervisors

Five doctoral level clinical psychologists served as MI supervisors. All supervisors participated in a standard two-day MI training workshop and then submitted audiotaped clinical interactions that were rated for proficiency by an independent rater. All supervisors had to score at the competency level or higher on all Motivational Interviewing Treatment Integrity (MITI) v 2.0 (Moyers et al., 2005) indices (see below) before supervising the clinician participants. All supervisors received training in the use of the MITI for supervision purposes. Two supervisors (KMC and AB) received training for using the MITI for supervisory purposes from the developers of the instrument.

Training Procedures

Recruitment and Assessment

Study personnel presented the training project to the clinic treatment staff and a member of the research team interviewed potential clinician participants. Clinicians who met the inclusion criteria and who chose to participate were consented into the study. The recruitment and training of clinicians occurred in eight successive waves, with each wave separated by eight weeks. Within each study wave, clinicians completed self-report instruments that recorded demographic information (i.e., gender, ethnicity, educational level, age) and employment factors (i.e., number of years in the drug counseling field, number of years at their current position, personal recovery status, self-identified theoretical orientation, employment setting) prior to the start of training. Verbal and abstract reasoning performance was also assessed prior to the beginning of training with a brief self-administered instrument.

Workshop Training

All clinicians submitted an audiotape of a counseling session they had with a client in their clinic during the week prior to the training workshop. This served as a pre-training baseline assessment. Clinicians then completed a 2-day MI training workshop that was conducted over two eight-hour days scheduled approximately a week apart. The workshop focused on MI Spirit, specific counseling skills (Open Questions, Affirmations, Reflections, and Summary Statements), and eliciting and recognizing change talk. Didactic instruction, experiential exercises, and role-playing were used to convey the concepts. During the week following the workshop training, clinicians submitted an audiotape of a counseling session with a client in their clinic. This served as a post-workshop assessment that captured any changes in MI skill over the time of the workshop training. A member of the research staff obtained informed consent from the clients prior to each of the counseling sessions being audiotaped. A different client participated in each of the two (pre- and post-workshop) audiotaped sessions.

Post-Workshop Supervision

At the end of the second workshop day, participants were randomized to one of the three post-workshop supervisory conditions: WO, Tape, or TCS. Clinicians in the WO training group attended the initial two-day workshop but did not receive post-workshop supervision. Clinicians receiving post-workshop supervision (TCS or Tape) participated in five MI practice sessions with standard patients portrayed by trained actors. The actor would travel to each clinician’s place of employment and simulate a clinical interaction based on a pre-determined role. Clinicians in the Tape group sent audiotapes of their simulated clinical encounters with the standard patients (i.e., actors) to their supervisors. Supervisors listened to the complete audiotaped session, took notes on the clinician’s MI counseling style, and then rated a randomly selected 20-minute segment of the session. All clinicians received a graphical representation of their MI performance. The supervisor then called the clinician to review the graph, provide feedback, discuss the clinician’s experience, and conduct role-plays to practice the use of different MI skills. All supervision sessions were conducted prior to the next practice interview.

Clinicians in the TCS group received real time feedback via telephone (i.e., ear phone) during the simulated clinical encounter (Smith et al., in press). The TCS model was developed to provide immediate performance feedback that differentially reinforced MI-consistent and MI-inconsistent counseling styles and allowed the supervisor to model MI Spirit and style. The clinician and supervisor discussed the session following its completion. Supervisors also rated a randomly selected 20-minute segment of the session and sent the clinician a graphical representation of their MI performance. These forms were reviewed during a phone call prior to the next TCS session.

Post-Training Assessments

Clinicians in all three conditions submitted two audiotaped counseling sessions with clients enrolled in their treatment clinics after the post-workshop supervision period ended. The first was submitted eight weeks following the workshop training (this was one week after the supervision period ended) and the second tape was submitted 20 weeks after the workshop training (13 weeks after the supervision period ended). These sessions provided post-supervision assessments of MI skill with clients seeking help for their substance use. Clients, while enrolled in the treatment clinic, were consented research participants not familiar to the clinicians. No client participated in more than one follow-up counseling session.

Measures

Clinician Characteristics

Clinician characteristics (i.e., gender, ethnicity, age, counseling style) were assessed as part of a self-report questionnaire administered following informed consent and prior to the beginning of the two-day workshop training. A clinician’s educational degree and professional experience were obtained as part of this assessment. Clinicians were asked to report the number of years they had been working as a counselor in the field of substance abuse treatment and to select the highest educational degree they attained from a list of degrees (e.g., HS/GED, College, Masters, Doctorate).

Motivational Interviewing Treatment Integrity Coding System (MITI; Moyers et al., 2005, 2010)

The MITI was used to quantify a clinician’s MI skillfulness during interactions with clients at each of the four assessment points. The MITI is a coding system developed to rate the quality and quantity of clinician MI counseling behaviors during a randomly selected 20-minute audiotaped segment of a counseling session. Clinicians are rated along two global dimensions (Spirit and Empathy) and the frequencies of six counseling behaviors are recorded. The global Spirit ratings were used in the present study. They capture the degree to which the clinician supports the client’s autonomy, evokes the client’s reasons for change, and supports a collaborative counseling environment. Ratings were made along a continuum ranging from 1 (low) to 7 (high).

The MITI behavioral counts are based on specific types of counselor language that are important in an MI counseling style. They are Simple Reflections, Complex Reflections, Closed Questions, Open Questions, MI Adherent (MIA) and MI Non-Adherent (MINA) behaviors. Two measures of reflective listening skill, based on the MITI indices, were used for the present investigation: percentage of Complex Reflections (the number of Complex Reflections made divided by the total number of reflections made, multiplied by 100), and Reflection to Question Ratio (the total number of reflections made by a clinician divided by the total number of questions asked (open questions plus closed questions). In addition, the number of MINA statements made by the clinician, which are statements not consistent with an MI style (e.g., labeling, arguing, giving advice without permission) was calculated. In general, MI training aims to guide the development of a counseling repertoire that is characterized by stronger MI Spirit (a collaborative alliance), a low frequency of MINA statements, a greater use of Complex Reflections (relative to Simple Reflections), and the use of more reflections and less questions (a greater Reflection to Question Ratio).

All audiotaped assessment sessions were rated by the trained supervisors. All raters were blind to the supervision group assignment and timing of the sessions they were coding (i.e., pre-workshop assessment or post-workshop assessments). Also, no supervisor rated a clinician that had been his or her supervisee during the study. Ten percent of the audiotaped sessions were rated by two supervisors to assess the reliability of the MITI ratings. Intraclass correlation coefficients for the MITI indices ranged between good (MI Spirit = 0.60) to excellent (Reflection to Question Ratio = 0.88) (Smith et al., in press).

Shipley Institute of Living Scale (SILS; Shipley, 1940)

The SILS was administered to clinicians following their informed consent and prior to beginning the training workshop to estimate the breadth of their verbal repertoire and abstract reasoning skills. The SILS is a brief 60-item measure that consists of two self-administered subtests, a 40-item vocabulary test and a 20-item abstract reasoning test. The total administration time is 20 minutes, 10 minutes for each subtest (Zachary, 2000). Vocabulary scores were converted to age normed T-scores (M=50 and SD=10). Abstract reasoning scores (i.e., Abstraction Quotient (AQ)) were calculated by subtracting a clinician’s observed performance from their predicted performance (Zachary, 2000). Predicted performance is based on their vocabulary score, age, and education. Raw scores are converted to standard scores with a mean of 100 and standard deviation of 15, with higher scores indicating stronger abstract reasoning performance (Zachary, 2000).

Data Analysis

Three sets of analyses were conducted to test the relationships between clinician characteristics and each of the four MI skills assessed at the four measurement points (pre-workshop and one, eight, and twenty weeks post-workshop training). MI Spirit and reflective listening are important components of an MI counseling style and we chose to investigate each MI skill independently. Evidence suggests they are modestly related, although conceptually distinct, entail the use of different training exercises, and have been differentially responsive to training techniques (Baer et al., 2004; Miller et al., 2004). Distributions for some of the clinician characteristics and outcome variables were non-normal; they required transformations and/or the use of non-parametric models, which are outlined below. No significant differences in skill level were demonstrated between the two post-supervision assessments (i.e., there was no time effect), thus they were combined for the final set of analyses that tested the interaction between clinician characteristics and post-workshop supervision method.

Our analytical approach followed several steps. Clinician characteristics and MI counseling skills prior to receiving formal training (pre-workshop assessment) were tested with correlation coefficients. Second, linear regression models were used to test the relationship between clinician characteristics and MI skill level following workshop training. The predictor variables for each model were entered in a hierarchical fashion using three block entries. The first block entry included pre-workshop MI skill level; the second block included the number of years a clinician reported being in the drug counseling field and educational degree (graduate degree versus other); the third block included verbal and abstract reasoning performance (vocabulary and abstract reasoning scores). This a priori ordering was based on the hypothesis that present skill levels and recent professional experience may exert stronger effects on skill acquisition than verbal or cognitive factors. Thus, the effects of the verbal and abstract reasoning performances on MI skill acquisition were tested after accounting for baseline skill level, and professional and educational experiences. Each variable block was tested for statistical significance when entered into the regression model. Significant block entries were then probed and statistically significant variables within a block are presented and discussed.

The third set of analyses tested if clinician characteristics moderated the effect of post-workshop supervision on MI skill. Generalized Linear Mixed effect models within a GEE analytical framework were chosen since they are the optimal approach to test for the effects of clinician characteristics on training outcomes while simultaneously accounting for the random effects of the clinicians. Furthermore, they do not require complete outcome data across time points, so that cases missing data at one but not both follow-up assessment points could be included in the analyses under the assumption of missing at random. A separate regression model was used for each of the four MI indices. The predictor variables for each model were entered in three blocks. The first block included MI skill level at post-workshop, the effect of time, and the contrast variables that tested the main effects of post-workshop supervision method (Tape versus WO; TCS versus WO). This block duplicated the findings in Smith and colleagues (in press) and was not the focus of this investigation. Block two included the main effects of clinician characteristics (years in the field (log transformed), education (graduate degree vs. other), vocabulary and AQ scores). Block three included clinician characteristic by post-workshop supervision condition interaction terms. Block three (the one containing the interaction terms) was the primary block of interest; it tested if certain supervision techniques were more or less effective depending on the educational, professional, and verbal-cognitive characteristics of clinician trainees. The statistical significance of block three (the interaction between clinician characteristics and supervision condition) was evaluated by testing the change in −2 log-likelihood values. Significant block entries were probed and statistically significant parameter estimates for the individual variables within a block are discussed and graphically presented in Figure 1.

Figure 1.

The significant clinician characteristic by post-workshop supervision condition interaction terms for the outcome measures of MI Spirit and Reflection to Question Ratio demonstrated in the generalized linear mixed effect models. MI skill outcomes are observed scores and standard errors for the average of both follow-up assessments and categorized by clinician characteristics and post-workshop supervision method (WO= workshop only; Tape= taped based supervision; TCS= Teleconferencing supervision). Dashed lines denote the proficiency thresholds for each of the MI skill outcomes.

Normal distributions were utilized for a majority of the linear models. However, for the third set of analyses, the MINA and Reflection to Question Ratio outcomes followed a negative binomial distribution. A limitation of the negative binomial distribution is that only pseudo log likelihoods can be computed in the analysis. Thus, maximum log likelihood statistics (as well as the change in −2 log-likelihood statistic) could not be computed and the block entries for these models could not be directly tested with this statistic. Thus, the individual parameter estimates for the interaction terms were directly investigated for significance following their entry into the model.

Results

Participants

Of the 100 clinician-participants enrolled in the parent study, 94 completed the MI workshop, were randomized to one of the three supervisory conditions, and provided the eight-week and twenty-week post-workshop follow-up audiotapes (Smith et al., in press). Fifty-eight of those clinicians completed the full set of baseline measures prior to the training workshop and are the focus of the present study. There were no significant differences between the 58 clinicians who completed the full-set of baseline measures and the 36 clinicians who did not complete the assessment on demographic characteristics and MI skill assessed at baseline, post-workshop, and the 20-week post-workshop follow-up assessment. The two groups were also equivalent on all but one of the MI skill indices at the 8-week follow-up assessment. The group of clinicians who did not complete the baseline assessments demonstrated a greater Reflection to Question Ratio at the week-eight assessment compared to those who did complete the assessments.

Of the 58 clinicians who completed the baseline assessments, 34 (59%) were employed in methadone maintenance clinics (MMTP), 15 (26%) were employed in drug-free outpatient clinics, and 9 (15%) were employed in residential treatment programs; 40% were African-American, 33% were Latino, and 19% were Caucasian. Approximately 43% of the clinicians reported they had a history of substance abuse. In terms of the therapeutic orientations that clinicians reported adapting a considerable amount of the time, 79% reported using CBT, 45% reported employing a harm reduction approach, 32% reported using AA/NA principles, and 10% of the clinicians reported employing an MI approach (the choices were not mutually exclusive). It is interesting to note, that although a large proportion of the clinicians were employed in methadone maintenance clinics, a minority of clinicians endorsed employing a harm reduction approach. During training, many of the MMTP counselors reported viewing their programs as abstinence-promoting programs that utilized methadone as a medicinal intervention. Thus, they did not view their treatment modality as a harm reduction intervention, a label more strongly associated with needle exchange programs. A majority of the clinicians reported being satisfied with their counseling approach. Educationally, 69% reported earning a bachelor’s degree or higher, 12% reported attending some college after high school, and 19% reported ending their formal education after earning a high school diploma or GED. As demonstrated in Table 1, there were no significant differences among the clinicians assigned to the three MI Training groups in pre-workshop MI skills or across most of the demographic categories. There was a significant difference between the Tape and TCS groups on baseline vocabulary performance, with the Tape group demonstrating better performance on this assessment (95% CI for mean difference: 1.9; 17.3, p < .02).

Table 1.

Clinician Demographic, Counseling, Cognitive, and MI Skill Characteristics Presented by Post-workshop Supervision Condition.

Supervision Group
Clinician Characteristics	Workshop Only (n=20)	Tape–based (n=21)	TCS (n=17)
Female (n)	70% (14)	67% (14)	59% (10)	χ2(2) =0.53; p <0.77
Age	42.2 (11.2)	40.4 (9.9)	44.4 (9.1)	F(2,52) = .83; p <0.52
Racea
Hispanic	25% (5)	24% (5)	53% (9)	χ2(2) =0.91; p <0.64
African-American	50% (10)	33% (7)	35% (6)
Caucasian	24% (4)	24% (5)	12% (2)
Other	1% (2)	19% (4)	0% (0)
Educationa
High School	30% (6)	9.5% (2)	18% (3)	χ2(2) =0.50; p <0.47
Part College	5% (1)	19% (4)	12% (2)
College	45% (9)	29% (6)	40% (7)
Graduate Degree	20% (4)	43% (9)	30% (5)
Counseling Model
AA/NA	25% (5)	40% (8)	29% (5)	χ2(2) =1.09; p <0.58
CBT	80% (16)	81% (17)	77% (13)	χ2(2) =0.12; p <0.94
MI	32% (6)	26% (5)	29% (5)	χ2(2) =0.13; p <0.94
Harm Reduction	35% (7)	45% (9)	59% (10)	χ2(2) =2.11; p <0.35
Satisfaction with Counseling Approach	61% (11)	55% (11)	71% (12)	χ2(2) =0.95; p <0.62
In Recovery	35% (7)	38% (8)	59% (10)	χ2(2) =2.46; p <0.29
Years in Counseling Fieldb	8.2 (6.4)	7.3 (7.1)	6.0 (3.8)	F(2,55) = .59; p <0.56
Cognitive Measures
Vocabulary	49.9 (8.4)	52.8 (10.0)	43.5 (10.3)	F(2,55) = 4.80; p <0.02
Abstraction Quotient	101.2 (10.1)	99.8 (11.8)	101.2 (14.6)	F(2,55) = 0.10; p <0.91
Pre-Workshop MI Skills
Spirit	4.1 (0.8)	4.4 (1.1)	3.9 (1.1)	F(2,55) = 0.99; p <0.38
Percent Complex Reflections	35.9 (23.5)	42.1 (24.0)	33.8 (34.4)	F(2,55) = 0.50; p <0.62
Reflections/Question	0.47 (0.31)	0.49 (0.42)	0.43 (0.38)	F(2,55) = 0.17; p <0.85
MINA	1.2 (1.4)	1.8 (2.1)	4.8 (5.4)	F(2,55) = 0.23; p <0.80

Note:

^acategories for comparison for race were Caucasian versus other; for education it was graduate degree (MA/PhD) vs no graduate degree.

^bKrusal-Wallis test was used given significant skewness of the data.

TCS= Tele-Conferencing supervision.

Missing data

Of the 58 participants included in this study, all participants provided baseline data and post-workshop assessment tapes. Furthermore, 57 of the 58 participants had MITI data at one (53/58 provided week 8 MITI data, 54/58 provided week 20 MITI data) or both (48/58) follow-up outcome points and were included in the analyses testing the interactions between clinician characteristics and supervision conditions. One participant did not provide any follow-up data and was excluded from the analyses testing the interactions. Missing data was predominantly due to poor recording quality, such that audiotapes were inaudible and could not be scored.

What is the relationship between counselor characteristics and an MI counseling style prior to training?

Zero-order correlations between clinician characteristics and pre-workshop MI skill levels are presented in Table 2. Having more years in the counseling profession was associated with greater MI Spirit and a greater Reflection to Question Ratio. Similarly, stronger vocabulary performance was associated with greater MI Spirit and a less confrontational or directive counseling style (i.e., fewer MINA behaviors) prior to formal training in MI.

Table 2.

Correlation Coefficients that Assessed the Relationship Between Clinician Characteristics and MI Skill Ratings Prior to Workshop Training.

Clinician Characteristics	Spirit	Pct CR	Ref/Quest	MINA
Education	0.17	0.08	0.14	0.12
Yrs in Field	0.33**	0.20	0.33**	0.19
Vocabulary	0.27*	0.02	0.04	−0.27*
AQ	0.18	0.02	0.04	0.03

^**p ≤ .02,

^*p ≤ .05.

AQ= Abstraction Quotient. Education (graduate degree versus non-graduate degree) was tested with point-biserial correlations. Years in the counseling field was Log transformed for the analysis.

Pct CR = Percent Complex Reflections. Ref/Quest = Reflection to Question Ratio. MINA = Motivational Interviewing Non-Adherent.

What is the relationship between counselor characteristics and the magnitude of change in MI counseling skills following workshop training?

Table 3 presents the results of the hierarchical multiple regression models that tested the relationship between counselor characteristics and MI skill level following workshop training. Pre-workshop skill level accounted for a significant amount of variance in the post-workshop MI Spirit (17%), F_(1,56) = 11.63, p=.001, and MINA outcomes (14%), F_(1,56) = 8.7, p = .005 (See Table 3, Step 1). The positive association between pre-workshop skill level and post-workshop skill level indicates that greater MI skillfulness prior to training predicted relatively greater gains in MI skill between pre- and post-workshop assessments. The relationship between pre-workshop skill level and magnitude of change remained after controlling for all other clinician characteristics (see Table 3, Step 3).

Table 3.

Change in R-squared, Individual Regression Estimates (b-weights) and Standard Errors (SE) in Each of the Three Variable Blocks that Tested the Relationship between Clinician Characteristics and the Magnitude of change in MI Skill Ratings after Workshop Training.

Clinician Characteristics	MI Skill Ratings
	Spirit	Pct CR	Ref/Question	MINAa
Step 1: ΔR2	0.17***	<.01	<.01	0.14**
Pre-Workshop Skill	0.47(0.14)***	0.05(0.11)	0.09(0.18)	0.26(0.09)**
Step 2: ΔR2	0.07	0.07	0.21**	0.01
Pre-Workshop Skill	0.38(0.14)***	0.06(0.11)	−0.04(0.17)	0.26(0.09)**
Years in Fielda	1.05 (0.47)	2.71 (9.4)	0.34(0.12)**	−0.24(0.24)
Graduate Degree	−0.20(0.30)	−12.1(6.2)	0.17(0.08)*	0.15(0.16)
Step 3: ΔR2	0.09*	0.04	0.03	0.00
Pre-Workshop Skill	0.28(0.14)***	0.05(0.11)	−0.04(0.17)	0.26(0.10)**
Years in Fielda	1.17(0.45)*	4.4 (9.50)	0.36(0.12)	−0.24(0.24)
Graduate Degree	−0.63 (0.33)	−9.8 (7.30)	0.13(0.10)	0.11 (0.19)
Vocabulary	0.04 (0.02)*	−0.16(0.33)	0.00(0.0004)	0.00 (0.01)
AQ	0.01(0.01)	0.35(0.24)	0.00(0.003)	0.00 (0.01)
Multiple R2	0.34	0.11	0.24	0.16

Note:

^alog transformed for analyses;

^***p < .001;

^**p < .01;

^*p < .05.

AQ= Abstraction Quotient.

Step 1 entered pre-workshop skill ratings, Step 2 entered years in the field and degree status (MA/Ph.D. vs. other), Step 3 entered vocabulary and AQ performances.

Multiple R² is the variance accounted for when all predictor variables have been entered (Step 3).

Professional experience accounted for a significant amount of variance in the magnitude of change in MI skillfulness during workshop training for the Reflection to Question ratio (9%, F_(1,49) = 3.59; p < .04; Step 2, Table 3). Having more years of counseling experience predicted a greater use of reflections relative to questions following workshop training. Professional and educational experience did not account for post-worskhop differences in MI Spirit, Percent Complex Reflections, or MINA (Table 3, Step 2 entries were not significant). Cognitive performance accounted for a significant amount of variance in MI spirit following workshop training (9%, F_(1,50) = 3.62, p <.04 (see Table 3, Step 3). Stronger vocabulary performance predicted greater MI Spirit ratings. Cognitive performance did not predict Percent Complex reflections, MINA, or the Reflection to Question Ratio following workshop training.

Do clinician characteristics moderate the effect of post-workshop supervision methods on MI skill acquisition?

The results of the mixed linear effect models testing the interaction between clinician characteristics and post-workshop supervision indicated significant entries for all three variable blocks (i.e., post-workshop skill level, clinician characteristics, and clinician characteristic by supervision condition interactions). Post-workshop skill levels (Block 1) were significantly positive predictors of MI skill at follow-up for the measures of MI Spirit (F_(1,53) = 22.7; p < .001), Percent Complex Reflections (F_(1,53) = 10.84; p < .002), and the Reflection to Question Ratio (F_(1,48) = 6.13; p < .02). These results indicate that stronger skill level following workshop training predicted relatively stronger MI skills at follow-up across all three post-workshop supervision conditions. In contrast, post-workshop frequency of MINA did not predict MINA frequency at post-supervision follow-up (F_(1,53) = 2.70; p < .11).

Clinician characteristics did not moderate the effect of supervision method on the frequency of MINA. However, significant independent effects were demonstrated for vocabulary performance (F_(1,49) = 16.17; p < .001) and education (F_(1,49) = 8.46; p =.01) on the frequency of MINA at follow-up. Stronger vocabulary skills and having a graduate degree predicted a lower frequency of MINA following the post-workshop supervision period across all three supervision conditions (WO, Tape, and TCS). Clinician characteristics (Block 2) significantly predicted post-supervision MI Spirit and Reflection to Question Ratio. However, the interpretation of these effects must be qualified because of the significant interactions discussed below. There was no significant relationship between clinician characteristics and Percent Complex Reflections at follow-up.

Significant clinician characteristic by post-workshop supervision interactions (Block 3) were demonstrated for MI Spirit (Δ −2 log-likelihood = 23.52, Δ df =8, p < .005) and the Reflection to Question Ratio (recall pseudo-log likelihood statistics prohibited a Δ −2 log-likelihood test for this variable). The individual parameter estimates indicated that the effect of supervision method on MI Spirit was moderated by educational level (F_(2,41) = 4.27; p < .03), vocabulary performance (F_(2,41) = 8.187; p < .001), and abstract reasoning performance (F_(2,41) = 5.98; p < .01). The effect of supervision method on the Reflection to Question ratio was moderated by educational level (F_(2,36) = 5.85; p < .01) and vocabulary performance (F_(2,36) = 5.18; p < .02). No significant interactions were demonstrated for the Percent Complex Reflections (Δ −2 log-likelihood = 11.36, Δ df =8, p < .25).

Figure 1 presents the significant interactions between supervision condition and clinician characteristics for MI Spirit and Reflection to Question Ratio at follow-up. The effect of time in the mixed effect linear models was not significant (Smith et al., in press; also see Data Analysis section) so the outcomes graphed represent the average skill level across the two follow-up assessments (week 8 and week 20). Further, the dotted line in each graph denotes the suggested thresholds for a proficient level of MI skill. The threshold of MITI Spirit scores of at least 5 on the 7-point scale represents the first level of proficiency, which has been used to certify clinicians as therapists for clinical trials (COMBINE Study Research Group, 2003). The Reflection to Question Ratio threshold of one also represents the first level of proficiency for this skill (Moyers et al., 2010).

As demonstrated in Figure 1, when characterizing clinicians by graduate degree status, TCS yielded stronger effects for improving both MI Spirit and the Reflection to Question Ratio among those clinicians without a graduate degree. In contrast, Tape supervision was more effective for increasing performance on these outcomes among clinicians with a graduate degree. Using the proficiency thresholds as a marker of supervision efficacy, clinicians without a graduate degree were more likely to meet the MI Spirit proficiency threshold if they received TCS. Clinicians holding a graduate degree were more likely to meet the proficiency thresholds for MI Spirit and Reflection to Question Ratio if they received Tape-based supervision.

Characterizing clinicians based on their vocabulary performance (independent of degree status) indicated that receiving post-workshop supervision (either TCS or Tape) was most effective for increasing MI Spirit among clinicians with a relatively lower vocabulary performance level (middle panels Figure 1). In contrast, clinicians with relatively stronger vocabulary performance met the MI Spirit proficiency threshold whether or not they received continued supervision after completing workshop training. In terms of increasing the Reflection to Question Ratio, TCS exerted a stronger effect across the full range of vocabulary performance, however the beneficial effect of TCS was most pronounced among the clinicians in the average to high average range.

Characterizing clinicians according to their abstract reasoning skills (lower panel Figure 1) indicated that Tape-based supervision was most effective for increasing MI Spirit ratings among clinicians in the average range of abstract reasoning skills (i.e., meeting MI proficiency threshold). TCS exerted a relatively stronger effect among clinicians in the lower range of performance. Among clinicians demonstrating relatively greater abstract reasoning abilities, all three training conditions facilitated the meeting of proficiency thresholds.

Discussion

This study investigated the relationship between the professional, educational, and cognitive characteristics of community-based substance abuse treatment clinicians and their skill in implementing an MI counseling style before, during, and after completing a training workshop and a 5-week post-workshop supervision program. There were three major sets of findings. First, prior to training, having more counseling experience was associated with stronger MI Spirit and a greater reliance on reflective listening skills, and stronger vocabulary skills were associated with stronger MI Spirit and a less confrontational counseling style (fewer MINAs). Second, among clinicians with similar baseline levels of skillfulness, those with more counseling experience and a graduate degree demonstrated a greater Reflection to Question Ratio, and those with a relatively stronger vocabulary performance demonstrated stronger MI spirit after workshop training. However, among clinicians with similar baseline levels of skillfulness, those with more counseling experience demonstrated greater improvements in MI Spirit and Reflection to Question Ratio. Third, clinician characteristics moderated the effect of post-workshop supervision on measures of MI Spirit and the Reflection to Question Ratio. In general, clinicians without a graduate degree benefited most from a supervision method that combined feedback and coaching with more immediate in vivo supervision (TCS). Clinicians with a graduate degree benefited most from a supervision method that involved feedback, coaching, and the discussion of previous counseling sessions (Tape-based supervision). When clinicians where characterized by their vocabulary or abstract reasoning performances, strengthening MI Spirit was best facilitated by providing continued supervision after workshop training (either Tape or TCS) for individuals in the lower range of performance; the benefit of continued supervision was diminished among clinicians demonstrating stronger performance in these domains. Further, receiving immediate in vivo supervision facilitated the use of reflective listening skills among clinicians with stronger vocabulary skills.

The demonstrated relationships among clinician characteristics and an MI counseling style prior to training are in line with other investigations (Baer et al., 2009; Miller et al., 2004). The present findings highlight the important roles of counseling experience and vocabulary skills before and after workshop training. MI is a verbally based intervention that relies on a clinician’s verbal repertoire for its implementation. Further, the evaluation of a clinician’s MI skill is primarily based on their verbal interactions with clients. Lower vocabulary performance was associated with lower Spirit ratings and a more confrontational or directive counseling style (i.e., increased MINAs) before and after workshop training. This counseling stance can negatively impact the efficacy of interventions for substance use (Karno & Longabaugh, 2005; Miller, Benefield, & Tonigan, 1993). Employing a therapeutic stance consistent with stronger MI Spirit has been viewed as an important starting point and indicator for the successful acquisition of an MI counseling style (Miller & Moyers, 2007), has been associated with being a more effective treatment provider (Najavits & Weiss, 1994), and is a significant predictor of client responsiveness during MI sessions (Moyers & Martin, 2006). Thus, the present findings suggest verbal flexibility may impact the overall style with which clinicians interact with their clients and contribute to the successful implementation of this non-confrontational counseling stance.

Having more counseling experience was associated with a higher Reflection to Question Ratio and stronger MI Spirit prior to workshop training and it continued to account for differences in clinician skill after workshop training. These results are consistent with studies demonstrating a positive relationship between counseling experience and the self-reported use of MI skills (Ball et al., 2002) as well as post-training competence in other treatment modalities (Henry, Schacht et al., 1993; Siqueland et al., 2000). However, they stand in contrast to previous training studies that have directly assessed MI proficiency (Baer et al., 2004; Martino et al., 2011; Sholomskas et al., 2005). Differences in training methods (Sholomskas et al., 2005), statistical power (Baer et al, 2004), and levels of professional experience (Baer et al 2004, 2009; Miller et al., 2004) may account for some of the differences across investigations. It is also interesting to note that the clinicians in the present investigation had similar levels of experience as the counselors that were designated as being “less experienced” (i.e., M=10.8 years) in the training study by Martino et al., (2011). It was the more experienced counselors (M=19.8 years) that required more intensive training to acquire an MI counseling style in their study. These findings raise the possibility that training programs may need to attend to cohort differences among prospective trainees. At the same time, the present study suggests that relatively more experience in the counseling profession, independent of pre-training proficiency, may be a particularly important attribute that maximizes the benefits of a structured workshop training that includes both didactic and experiential instruction methods.

The positive association between clinician experience and MI skill also suggests that a counseling style that is more consistent with the principles of MI may promote greater longevity in the drug-counseling field. The drug counseling profession is noted for a relatively high attrition rate (McLellan, Carise, & Kleber, 2003). However, Carroll and colleagues (2006) observed a relatively lower turnover rate among clinicians randomized and trained to provide MI in a community-based treatment trial. The lower attrition rate among counselors who received MI training supports the contention that certain counseling styles may be protective against clinician attrition or burnout. However, further investigation is needed to determine if clinicians stronger in these skills are more likely to remain in the counseling profession or if extended exposure to clinical interactions shape a particular counseling style, or both. Identifying potential risk factors for professional turnover and how certain counseling styles interact with professional satisfaction and career longevity has important implications for improving the provision of clinical services in the drug and alcohol treatment field.

Having stronger MI skills prior to training was associated with relatively greater skills after the workshop training period for the MI Spirit and MINA outcomes. The direction of this relationship was consistent with our hypothesis and other studies (Baer et al., 2009; Martino et al., 2011). These findings suggest that more skilled clinicians ram in relatively more skilled following workshop participation. Thus, the training does not eliminate pre-training differences in skill level, and relatively less skilled clinicians remain so after workshop participation. This finding suggests that enhancing the skills of less experienced clinicians may require a different type of training workshop or structure so that the workshop content is appropriate for the skill levels of participating clinicians. While the workshop training in the present study focused on the presentation of MI microskills, it may not necessarily have been optimal for clinicians entering the training with a relatively weaker skill set. Bear et al. (2004) found that clinicians identified as having stronger MI skills after workshop training were also more skilled prior to training. The clinicians in their study had more counseling experience than clinicians in the present investigations. Further, the positive relationship between pre-training skill and post-training skill was demonstrated when clinicians were categorized as being either non-proficient or proficient (exceeding a certain level of performance on 3 of 4 MI outcomes) after training. The categorization of clinicians based on their simultaneous performance across several MI dimensions may have contributed to the positive relationship between pre-training and post-training skill by employing a more stringent operationalization of MI skillfulness. In the present study, each MI skill outcome was assessed along a separate continuum. The different approaches used to define MI skillfulness highlight the need for a more empirically based procedure by which specific proficiency thresholds are identified. Ideally, proficiency thresholds should denote levels of clinician skill that differentially impact client outcomes. However, it remains unclear which system, categorizing clinicians based on their performance along separate skill dimensions or their simultaneous performance across numerous skills, offers the best framework for identifying those clinicians that may have the greatest positive impact on treatment outcome. Delineating the proficiency thresholds that predict better clinical outcome would have direct implications for evaluating the overall utility of workshop training and may help identify the clinicians most likely to benefit from this training format.

Clinicians’ MI skillfulness prior to receiving extended supervision was positively associated with skill level at follow-up across all MI outcomes (MI Spirit, MINA, Percent Complex Reflections, and Reflection to Question Ratio). This relationship was demonstrated among all three post-workshop supervision conditions. These results suggest that post-workshop supervision strategies may not eliminate the differences in skill level that existed prior to more intensive supervision procedures. Thus, while accumulating evidence suggests clinicians can benefit from formal training procedures, relative differences in MI skill level continue to remain after exposure to such gold standard training procedures. Further research is needed to better delineate the parameters of training that are most effective for reducing the continued gaps in clinical practice among clinicians receiving formal training in evidence-based practices. Beidas and Kendall (2010) offer one conceptual framework that may hold promise in this regard. Operating from a systems-contextual perspective of training and implementation, the authors propose that clinician and client outcomes can best be facilitated when training interventions address organizational, therapist, and client level factors. This multi-level training framework may provide an important blueprint for developing and systematically evaluating the utility of different training mechanisms and it includes components identified as being potentially useful in the dissemination of substance abuse treatments (Miller, Sorensen, Selzer, & Brigham, 2006).

This was one of the first investigations to demonstrate that the efficacy of different supervision strategies may be dependent on a clinician’s level of formal education and verbal-cognitive skills. Consistent with our hypotheses, a more direct and immediate training structure based on supervisor prompts and modeling (TCS) was more effective for strengthening MI skills (i.e., MI Spirit and Reflection to Question Ratio) among clinicians without a graduate degree and demonstrating lower vocabulary and abstract reasoning performances (MI Spirit). Tape supervision, a more delayed supervision structure that is more heavily weighted on verbal discourse and the analysis of past clinical encounters, was more effective for clinicians with a graduate degree (Reflection to Question Ratio) and stronger abstract reasoning performance (MI Spirit). The effect of a graduate education on MI skillfulness following training was consistent with some previous studies (Baer et al., 2009) but not with others (Baer et al., 2004; Miller 2004). However, studies demonstrating an effect of education on training outcomes tend to have a lower proportion of clinicians with a graduate education (e.g., Baer et al., 2009). In contrast, studies finding no association between educational attainment and outcome (Baer et al., 2004; Miller et al., 2004) have reported graduate degrees among 79% and 85% of the participating clinicians, respectively. Thus, having a graduate degree has more prognostic significance among more educationally diverse clinician populations.

Cognitive abilities have been demonstrated to predict the acquisition and implementation of skills in other occupational domains (Colquitt, LePine, & Noe, 2000). However, there is little research on how the verbal-cognitive skills of clinicians effect the acquisition and implementation of new counseling practices. Higher order cognitive processes have been highlighted as an important mechanism in facilitating psychotherapy training (Fauth et al., 2007; Haarhoff, Gibson, & Flett, 2011) and cognitive functioning predicts a wide range of outcomes in both general (Gottfredson, 1997) and patient (Aharonovich, Nunes, & Hasin, 2003) populations. The present findings suggest verbal flexibility and abstract reasoning performance may be potentially useful clinician attributes to consider when developing and/or evaluating the effectiveness of specific training protocols.

The proficient implementation of evidence-based practices requires learning a range of general, as well as more nuanced and refined, technical skills associated with a particular intervention strategy. MI highlights the importance of a collaborative therapeutic stance, a non-confrontational counseling style, and emphasizes the importance of reflective listening, with a special emphasis on the use of Complex Reflections for guiding the direction of a counseling session (Miller & Rollnick, 2002). The results of the present study offer preliminary data, which may help guide the selection of specific training procedures for developing this counseling approach. First, prior to implementing the standard workshop training, obtaining a pre-workshop assessment of skills is particularly important since stronger MI Spirit and lower MINA before workshop training predicts relatively stronger performance on these measures after workshop training. Second, if clinicians are fairly homogenous with respect to pre-training skill levels, their experience maybe a useful selection factor, since more experienced clinicians may be more likely to benefit from a workshop training structure. Following workshop training, MI skill level is an important indicator of future MI skill proficiency independent of the type of post-workshop training offered. The presence or absence of a graduate degree is a fairly easy and inexpensive (i.e., time- and cost-effective) designation to make. If post-workshop supervision is offered as part of the training protocol, the use of feedback and coaching with a more immediate in vivo training structure may better promote a collaborative counseling style and strengthen the use of reflective listening skills among those with no graduate degree. Tape-based supervision appears to work best for enhancing an MI therapeutic stance among clinicians with a graduate degree. It is interesting to note that prior studies have indicated that formal training structures can facilitate therapist technical skill development, although they may not be as effective for improving the interpersonal dimensions of a therapeutic interaction (Henry, Strupp, Butler, Schacht, & Binder, 1993). It is possible that the Tape-based supervision framework may provide a less invasive training method that allows for the provision of feedback and coaching, while minimizing the intrusiveness of the supervisory process during clinical interactions. Thus, in effect, it is the clinician and client interactions that guide the use of the newly acquired counseling skills in this context. In contrast, the use of a more directive training method that introduces the physical presence (via ear piece) of the supervisor during the clinical interaction may have been more disruptive among clinicians with a more solid counseling foundation prior to training, thus reducing its efficacy for improving the collaborative nature (i.e., MI Spirit) of the interaction.

The present results also highlight that estimates of verbal and abstract reasoning skills may provide useful information for selecting supervision methods among clinicians who are fairly homogeneous in terms of pre-training skill levels and graduate degree status. While assessing these skills can be more invasive and costly then determining the graduate degree status of a potential trainee, the use of a brief assessment procedure in this study yielded characterizations of clinician verbal and abstract reasoning performances that accounted for a significant amount of variation in skill level and may be informative for improving specific elements of an MI repertoire. Specifically, providing continued supervision that includes feedback and coaching after workshop training may help promote a stronger MI counseling stance among clinicians with lower verbal and abstract reasoning performances. The more direct and immediate training structure of TCS may help refine the use of reflective listening skills among clinicians with strong verbal skills by increasing the relative frequency of this clinician response during clinical interactions. Tape-based supervision may be more useful for facilitating a stronger MI counseling stance among clinicians demonstrating average to high average abstract reasoning performance. Since training clinicians remains the primary mechanism by which evidence-based treatments are disseminated to community practice, it is important to identify the skills needed to acquire and implement new counseling strategies. The present findings support a growing body of literature placing therapist training within a broader cognitive science model, outlining the importance of the cognitive and meta-cognitive processes that influence the development of a new counseling repertoire (Bennett-Levy, 2006; Fauth et al., 2007; Haarhoof, Gibson, & Flett, 2011). Future studies are needed to advance our understanding of the cognitive and behavioral processes underlying the acquisition of key therapeutic skills. Identifying these key mechanisms may help guide the inclusion of specific training procedures, which can further promote the acquisition of empirically supported principles of change.

Providing Complex Reflections require the clinician to provide a deeper or more meaningful response to what was initially expressed by the client and has been emphasized as being an important component of an MI counseling repertoire (Miller & Rollnick, 2002). Evidence suggests Complex Reflecting is also one of the hardest skills to improve during structured training programs (Baer et al., 2004; Miller et al, 2004). Contrary to our hypotheses, the present results did not identify any professional or cognitive factors that could account for differences in this skill among clinicians. Pre-workshop level of Complex Reflections accounted for a substantial amount of variance among clinicians prior to both the workshop and the supervised post-workshop trainings. Furthermore, post-workshop levels of Complex Reflections were equally predictive of outcome across the three post-workshop conditions. We hypothesized that verbal and abstract reasoning abilities would moderate the effect of the different supervision techniques on the acquisition of this skill. The results did not support this contention. It is possible that the strong relationship between pre- and post-training skill levels overshadowed any potentially small effects these factors may have contributed. The present results do suggest that Complex Reflection skills can be enhanced during the period of workshop training. However, both the Tape-based and TCS methods employed in this investigation were more influential in increasing the Reflection to Question Ratio than increasing the absolute level of Complex Reflections. Further investigation is needed to better understand both the clinician and training factors associated with the acquisition and use of the more complex components of this evidence-based practice.

The primary limitation of this study was that it focused on a subset of the clinicians participating in a larger training protocol (Smith et al., in press); not all of the clinicians completed the baseline cognitive assessment battery. Although there were no statistically significant differences between those who completed the cognitive battery from those who did not on most outcome measures, the loss of participants may influence the generalizability of the findings. The smaller sample size may have also reduced the power to detect other moderating effects and prohibited a full exploration of possible mediational links. Further, although a priori research questions were proposed, the findings and subsequent recommendations are preliminary and need to be replicated in larger independent samples. The strengths of this study were the use of a diverse community sample of clinicians supervised in the context of their own work environments, a randomized design in which clinicians were assigned to supervision conditions, a comprehensive baseline assessment that provided the opportunity to investigate the effects of professional, educational, and cognitive performance on skill acquisition, an excellent follow-up rate, and the use of a standard assessment instrument to quantify clinician skill levels throughout the training timeline.

In sum, this study provided a significant opportunity to systematically investigate how clinician characteristics influence the acquisition and implementation of an evidence-based practice for treating substance dependence. Baseline skill levels and clinician characteristics were demonstrated to be significant factors at all points of the training process. As such, the results support the notion that understanding clinician characteristics prior to implementing training may facilitate the continued professional development of community-based practitioners and influence the processes by which evidence-based practices in general, and more specifically MI, can be most effectively disseminated and implemented in community treatment settings.