Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2009 Jul 1.
Published in final edited form as: Drug Alcohol Depend. 2008 Mar 6;96(1-2):37–48. doi: 10.1016/j.drugalcdep.2008.01.020

Community Program Therapist Adherence and Competence in Motivational Enhancement Therapy

Steve Martino 1, Samuel A Ball 1, Charla Nich 1, Tami L Frankforter 1, Kathleen M Carroll 1
PMCID: PMC2692429  NIHMSID: NIHMS53513  PMID: 18328638

Abstract

The extent to which clinicians in addiction treatment programs can implement empirically validated therapies with adequate fidelity that can be discriminated from standard counseling has rarely been evaluated. We evaluated the treatment adherence and competence of 35 therapists from five outpatient community programs who delivered either a three-session adaptation of motivational enhancement therapy (MET) or an equivalent number of drug counseling-as-usual sessions to 461 clients within a National Institute on Drug Abuse Clinical Trial Network multi-site effectiveness protocol. MET therapists were carefully prepared to implement MET using a combination of expert-led intensive workshop training followed by program-based clinical supervision. Independent rating of sessions demonstrated that the adherence and competence items were very reliable (mean interclass correlation coefficients for adherence = .89 and competence = .81) and converged to form two a priori defined skill factors conceptually related to motivational interviewing. Moreover, the factors discriminated between MET therapists and those who delivered drug counseling-as-usual sessions in predicted ways, and were significantly related to in-session change in client motivation and some client treatment outcomes (percent negative drug urine screens). These findings demonstrate the reliability and validity of evaluating motivational interviewing fidelity and suggest that the combination of expert-led workshops followed by program-based clinical supervision may be an effective method for disseminating motivational interviewing in community treatment programs.

Keywords: motivational enhancement therapy, motivational interviewing, therapist adherence and competence, therapist training and supervision, substance abuse treatment

1. Introduction

Motivational interviewing (MI; Miller and Rollnick, 2002), and a manual-based adaptation of MI called motivational enhancement therapy (MET; Miller, Zweben, DiClemente, & Rychtarik, 1992), have accumulated substantial empirical support across single and multi-site clinical trials for enhancing treatment engagement and reducing substance use among clients abusing or dependent on alcohol (Bien et al., 1993; Brown and Miller, 1993; Miller et al., 1993; Miller et al., 1988; Project MATCH Research Group, 1997), heroin (Saunders et al., 1995), cocaine (Stotts et al., 2001), marijuana (MTP Research Group, 2004; Steinberg et al., 2002) and mixed substances (Carroll et al., 2001; Martino et al., 2006; 2000). Meta-analytic reviews (Burke et al., 2003; Hettema et al., 2005) have indicated moderate between-group effect sizes (d = .4 - .5) across substance use outcomes over three-month follow-up periods.

Conceptually MI works through therapists' efforts to interact with clients in a style that is collaborative, supportive of client autonomy and self-efficacy, and eliciting of client “change talk;” namely, statements that indicate client's movement toward behavior change. These components embody the “spirit” of MI delivery (Miller & Rollnick, 2002; Rollnick & Miller, 1995). Therapists use both fundamental, or what Miller and Rollnick (2002) have referred to as “microskills” (open-ended questions, affirmations and reflections delivered with MI spirit), and advanced skills (multiple methods for evoking change talk and handling client resistance) and avoid strategies inconsistent with the approach (such as unsolicited advice or direct confrontation) to increase client motivation for change during the session (Miller & Rollnick, 2002; Rollnick & Miller, 1995). Therapists attend to the balance of client statements supporting behavior change or inaction to gauge client motivation and to adjust their use of MI strategies accordingly. Therapists' capacity to elicit change-oriented client statements and reduce resistant ones, with the aim of strengthening clients' commitment to change, may be a central mechanism of action in MI (Amhrein et al., 2003).

Establishment of the efficacy of MI has led to a proliferation of therapist training resources, including textbooks (Arkowitz et al., 2007; Miller and Rollnick, 1991, 2002; Rollnick et al., 1999; Rollnick, Miller, and Butler, 2007), treatment manuals (Miller et al., 1992; Miller, 1999), training videotapes (Miller et al., 1998; Yahne and Miller, 2002), a supervision manual (Martino et al., 2006), websites (www.motivationinterview.org), and an international training group called the Motivational Interviewing Network of Trainers (Miller & Rollnick, 2002). Given these advancements in MI's empirical testing, theoretical base, and training materials, research on MI has moved toward the most effective means of disseminating MI in community treatment programs (Miller et al., 2006). For example, Miller and colleagues (2004) showed that therapist participation in intensive MI workshops plus post-workshop expert individual supervision with performance feedback and individualized coaching, in comparison to wait-list control or workshop-only groups, resulted in substantial increases in therapists' MI skills and in the frequency of clients' statements associated with greater change.

There has been, however, very little research to date on the extent to which therapists delivering treatment in ‘real world’ community programs can implement MI or MET in an adherent (i.e., with sufficiently frequent and extensive use of interventions consistent with MI and avoidance of strategies antithetical to it) and competent manner (i.e., with adequate skill). The Miller et al. (2004) training study included therapists who were highly motivated to learn MI in that they independently sought training in New Mexico at their own travel expense, and their baseline percentage of MI consistent responses was very high (89%). They also selected their own clients with whom they used MI, an optimal circumstance for demonstrating MI practice (Miller et al., 2004). Community program therapists most likely are more variable in their motivation to learn MI, as well as their pre-trial MI abilities, than these training study participants, and they typically have to implement MI with the wider range of clients who are assigned to them rather than those who may be selected in clinical trials (Humphreys and Weisner, 2000; Humphreys et al., 2005). Moreover, while certification of competence and fidelity monitoring are requirements of efficacy research, the provision of supervision in the form of performance feedback and coaching is highly variable in routine clinical practice. (Fixsen et al., 2005). The lack of effective program-based supervision in MI or MET and other empirically supported treatments is one of the largest barriers to the implementation of these approaches in clinical practice (Carroll and Rounsaville, 2007).

The absence of practical tools for evaluating treatment fidelity may also impede implementation of MI in the community. Existing fidelity monitoring tools developed for clinical efficacy trials such as the Motivational Interviewing Skills Code (MISC; Miller et al., 2003), the Motivational Interviewing Treatment Integrity Code (MITI; Moyers et al., 2004), or the Motivational Interviewing Supervision and Training Scale (MISTS; Madson et al., 2005) may be too complex and impractical for use by clinical supervisors in non-research settings (MISC) and have less than ideal reliability on many rating items (MISC, MISTS). In addition, with the exception of demonstrating that higher levels of MI fidelity (as rated by the MISC) are associated with increases in client motivation within sessions (Miller et al., 2004), none of the existing MI fidelity measures have shown significant positive relationships between their indices of therapist MI performance and important substance abuse treatment outcomes (e.g., client program retention and reduction in substance use). In the absence of these relationships, the validity and clinical utility of using MI fidelity scales for the purposes of training therapists and guiding their clinical practice in MI remain questionable.

In this report we present adherence and competence data drawn from a National Institute on Drug Abuse Clinical Trial Network protocol that examined the effectiveness of a three-session adaptation of the Motivational Enhancement Therapy manual used in Project MATCH (MET; Miller et al., 1992) in comparison to an identical number of the individual sessions provided as part of counseling-as-usual (CAU), both delivered in the first month of outpatient treatment at five different community substance abuse treatment programs in the United States (Ball et al., 2007). Although both conditions were associated with reductions in substance use through the 1-month treatment phase, participants assigned to MET maintained this level of reduced use over a 3-month follow-up period, whereas substance use increased among those assigned to CAU (Ball et al., 2007).

An important feature of this protocol was the careful independent evaluation of the therapists' use of MI strategies, techniques inconsistent with MI, and general substance abuse counseling practices using a MI fidelity monitoring scale developed for use in community programs, the Independent Tape Rating Scale (ITRS). We previously have reported in preliminary analyses that MET and CAU were discriminable in the therapists' delivery of the two treatments (Ball et al., 2007). However, these analyses were conducted without first comprehensively assessing the reliability and validity of the ITRS. The current report addresses these issues in greater detail, with the aim of determining the extent to which MI was successfully implemented in this trial. First, we evaluated the reliability and factor structure of the ITRS. We predict the MI consistent adherence items will converge to form two independent MI skill factors that reflect the fundamental and advanced skill sets that comprise MI. Second, we hypothesize therapists trained to deliver MET will achieve higher levels of MI fidelity (operationalized as greater MI-consistent and lower MI-inconsistent adherence; higher MI-consistent competence) compared with the levels used by clinicians who delivered CAU. We predicted that given the level of initial training and clinical supervision provided during the trial, these differences will be consistent across the five program sites, 35 therapists within conditions, and three sessions. Third, we predict that MI adherence and competence scale scores will be positively associated with increased client motivation within sessions and better client retention (days in treatment) and substance use outcomes (operationalized as days of primary drug abstinence and percent negative urine screens).

2. Methods

2.1. Participants

Participants in this study included therapists, clients, and tape raters. Client participants provided written informed consent, therapists provided either written permission or informed consent depending on local Institutional Review Board requirements, and tape raters signed a confidentiality agreement. Characteristics of each participant group are described below.

Therapists

The protocol treatments were delivered by 35 therapists employed in one of five outpatient non-methadone substance abuse treatment programs that served diverse samples of substance users. Participating therapists had to be willing to (1) be randomly assigned to treatment condition, (2) have their treatment sessions audiotaped and reviewed for initial proficiency certification, regular supervision sessions, and pre- and post-protocol treatment integrity assessment, (3) complete self-report ratings before training and during the protocol, and (4) receive program administrative approval to participate. Most therapists had no prior MI training exposure, and none had been trainers or therapists in research studies involving MI or MET (Ball et al., 2002a). Therapists were predominantly female (60%) and Caucasian (77%). On average, they were 38.9 years old (sd = 11.8), employed at their agencies for a mean of 3.2 years (sd =3.9), had been working as substance abuse counselors for 8.1 years (sd = 6.4), and completed 14.5 years of education (sd = 5.1). Forty-three percent had a master's degree, 46% were state certified substance abuse counselors, and 45% indicated they were in recovery from prior substance abuse problems.

Clients

The client participants were 461 outpatients who were (1) English-speaking, (2) 18 years of age or older, (3) seeking outpatient treatment for any substance use problem, with use of alcohol or any illicit drug at least once in the 28 days prior to randomization, and (4) willing to participate in all study procedures. Minimal exclusions (severe medical or psychiatric instability prohibiting outpatient treatment enrollment, residential instability or imminent incarceration, seeking detoxification) were used to obtain representative community samples (Carroll et al., 2002). On average, participants were 35.3 years old (sd = 9.7), predominantly male (71%) and single (82%), with similar proportions of African Americans and Caucasians (both 42%) and fewer Hispanic Americans (11%) or individuals of other ethnic backgrounds (5%). Participants had completed an average of 12.6 years of education (sd = 2.1). Forty-four percent were employed full or part-time, and 34% were on probation or parole. Clients had mixed primary substance use problems (29% alcohol, 23% cocaine, 16% marijuana, 9% opiates, 4% methamphetamines, 11% alcohol and drug, 8% other).

Raters

Fifteen independent tape raters were trained in the use of the ITRS to evaluate the session audiotapes. On average, the raters were 37.7 year old (sd = 9.7) men (47%) and women (53%) who had completed 18.4 years (sd = 1.6) of education. Most raters had master's degrees in a clinical profession (67%) and had an average of 6.9 years (sd = 9.7) of substance abuse treatment experience, 8.3 years (sd = 7.9) of general psychotherapy experience, and 5.6 years (sd = 5.3) of clinical research experience. Sixty percent of the raters had served as independent raters in prior clinical trial studies testing the efficacy of behavioral treatments. Fifty-three percent reported prior workshop training in MI, on average 9.0 hours (sd = 5.9).

2.2. Assessment of Therapist Adherence and Competence

Therapist adherence and competence were evaluated using the Independent Tape Rater Scale (ITRS; Ball, Martino, Covino, Morgenstern, & Carroll, 2002) adapted from the Yale Adherence Competence Scale (YACS; Carroll et al., 2000). The scale consisted of 39 items, all of which were rated on a 7-point Likert scale with lower ratings indicative of a general absence of behavioral examples and higher scores indicative of more extensive occurrence. This report focuses on the 30 items that address specific therapeutic strategies involving MI-consistent interventions, MI-inconsistent interventions, and general substance abuse counseling interventions and two items addressing the clients' motivational level at the beginning and end of the session.1 As described in Table 1, the 10 MI consistent items included MI skills considered fundamental or more stylistic in nature (i.e., open-ended questions, reflections, affirmations, fostering a collaborative relationship, and MI style or spirit) as well as several items intended to capture more advanced or technical strategies (i.e., client-centered problem discussion/feedback, assessing pros and cons of use/non-use, heightening discrepancies, motivation for change strategies, and change planning).

Table 1.

MI consistent, MI inconsistent and general substance abuse counseling item ratings: adherence and competence means, standard deviations, and intraclass correlation coefficient reliabilities

Items Adherencea Competenceb
MET CAU MET CAU
M SD M SD ICC M SD M SD ICC
MI Consistent (Fundamental Skills)
 1. Open-ended Questions 5.3 1.6 4.4 1.8 .89 5.1 0.9 4.4 0.8 .81
 2. Reflective Statements 6.3 1.0 3.7 1.9 .94 5.2 0.9 4.3 0.9 .88
 3. Affirmations 2.8 1.6 2.0 1.3 .96 4.7 0.8 4.3 0.7 .84
 4. Fostering Collaboration 2.4 1.7 1.9 1.3 .66 4.9 0.7 4.3 0.8 .69
 5. Motivational Interviewing Style 5.0 1.7 2.6 1.7 .86 5.0 0.9 4.1 0.9 .82
MI Consistent (Advanced Skills)
 1. Client-Centered Problem Discussion & Feedback 3.5 1.9 3.1 1.7 .93 4.7 0.8 4.3 0.8 .90
 2. Pros, Cons, and Ambivalence 3.0 2.2 1.2 0.6 .99 4.9 0.8 3.9 1.2 .97
 3. Heightening Discrepancies 1.4 0.9 1.2 0.6 .91 4.9 0.8 4.3 1.2 .91
 4. Motivation for Change 2.7 1.7 1.5 0.9 .87 4.7 0.8 4.3 0.8 .79
 5. Change Planning 2.7 1.9 1.4 1.1 .93 4.8 0.9 4.4 1.0 .90
MI Inconsistent
 1. Emphasis on Abstinence 1.0 0.2 1.2 0.6 .89 4.2 0.8 3.9 1.0 .93
 2. Powerlessness and Loss of Control 1.1 0.6 1.2 0.6 .74 4.5 0.7 3.9 0.9 .78
 3. Unsolicited Advice or Direction Giving 1.8 1.4 3.1 2.0 .96 4.4 0.7 4.1 0.9 .94
 4. Confrontation of Denial and Defensiveness 1.2 0.7 1.4 1.1 .78 4.4 1.4 4.2 1.2 .71
 5. Therapeutic Authority 1.3 0.8 2.5 1.8 .97 4.5 0.9 4.2 0.9 .96
 6. Self-Help Group Involvement 2.0 1.5 2.4 1.7 .98 4.4 0.6 4.2 0.7 .96
 7. Reality Therapy Principles 1.0 0.2 1.3 0.8 .85 4.0 0.8 4.1 1.2 .81
 8. Skills Training 1.0 0.3 1.1 0.3 .55 4.3 1.5 3.6 1.5 .57
 9. Cognitions 1.2 0.7 1.2 0.8 .86 4.4 0.8 3.9 1.2 .88
 10. Psychodynamic Interventions 1.1 0.5 1.2 0.7 .57 4.6 0.7 4.1 1.2 .67
General Substance Abuse Counseling
 1. Assessing Substance Use 2.6 1.7 2.8 1.7 .98 4.4 0.8 4.2 0.7 .96
 2. Assessing Social Functioning and Related Factors 4.1 1.7 4.7 1.8 .92 4.7 0.7 4.3 0.7 .72
 3. Psychopathology 1.3 0.8 1.8 1.3 .98 4.4 0.9 3.9 1.0 .97
 4. Medical and Medication Issues 1.4 0.9 1.5 1.1 .98 4.3 0.8 4.0 1.1 .98
 5. Program Orientation 1.3 0.9 1.7 1.2 .96 4.3 0.6 4.1 1.0 .92
 6. Case Management 1.2 0.6 1.4 0.9 .95 4.4 0.9 4.1 0.8 .94
 7. Psychoeducation re: Substances 1.5 1.3 1.1 1.3 .96 4.6 0.8 4.3 1.0 .96
 8. Formal Treatment Planning 1.3 0.9 1.4 1.0 .92 4.4 0.7 4.1 1.1 .90
 9. Risk Behavior Reduction 1.0 0.1 .98 0.4 .98 4.0 0.5 3.3 1.7 .98
 10. Spirituality/Higher Power 1.3 0.7 .96 1.0 .96 4.4 0.5 4.2 1.1 .97
Open in a new tab

Note. MET = motivational enhancement therapy delivered across three sessions within a 4-week period; CAU = counseling-as-usual delivered in an equivalent manner to MET; ICC = intraclass correlation coefficients based on the Shrout and Fleiss (1979) two-way mixed model.

a

Adherence ratings are on a 7-point Likert scale: 1=not at all, 2=a little (once), 3=infrequently (twice), 4=somewhat (3-4 times), 5=quite a bit (5-6 times), 6=considerably (> 6 times/more depth in interventions), 7=extensively (high frequency/characterizes entire session). Mean adherence ratings are based on the 206 MET and 219 CAU sessions.

b

Competence rating are on a 7-point Likert scale: 1= very poor, 2=poor, 3=acceptable, 4=adequate, 5=good, 6=very good, 7 = excellent). Mean competence ratings are made only when an item occurs within a session. Thus, the sample sizes for the item competence ratings vary considerably from n = 3 for skills training to n = 202 for open-ended questions, both in the MET condition.

Ten MI inconsistent items assessed the use of interventions antithetical to MI (e.g., confrontation of denial, therapeutic authority, unsolicited advice, emphasis on abstinence) or typical of treatment approaches different from MI (e.g., skills training, reality therapy, psychodynamic). Finally, 10 items captured interventions that would characterize ‘typical’ substance abuse counseling including assessment (e.g., monitoring substance use, social functioning), psychoeducation about substance use, and treatment planning (e.g., program orientation, case management).These items were expected to be common in CAU, but not necessarily antithetical to MI. For each item, raters evaluated the therapist on two dimensions using a 7-point Likert scale. First, they rated the extent to which the therapist delivered the intervention (adherence; 1 = not at all, to 7 = extensively). Second, they rated the skill with which the therapist delivered the intervention (competence; 1 = very poor, to 7 = excellent). Item definitions, rating decision rules, and recording procedures were specified in a detailed rating manual that was used by supervisors, trainers, and independent raters (Ball et al., 2002b). Table 1 lists the scale items for each of these three categories.

Because of the large number of tapes generated in the protocol, we elected to randomly select tapes only from those participants who had attended all three sessions (59% of total sample or 70% of participants who had attended at least one study session). Hence, we focused only on participants in each condition who had completed their assigned treatment condition so that outcome analyses could examine therapist adherence and competence controlling for the possible confounding effect of the amount of treatment clients received. We selected a minimum of 10 sets of three-session tapes from within each of the two conditions from within each of the 5 sites. Following this balanced selection procedure, we randomly selected additional tapes from therapists who were not included through this process (n = 7/35 or 20%) to ensure we had ratings for all therapists in the study.

2.3. Assessment of Client Motivation, Retention, and Substance Use Outcomes

Change in client motivation was measured using independent 7-point global ratings of the first and last 5 minutes of the sessions. Each scale point reflected a relative balance of client change talk and resistant statements, such that 1 represented no motivation for changing primary substance use (very little change talk and very strong resistance), and 7 represented extremely strong motivation for change (almost all change talk and very little resistance). We subtracted motivation at the beginning of the session from motivation at the end session to obtain a change in motivation score (range = -6 to 6). Research assistants collected client retention data (days of program enrollment) based on self-reports which were confirmed with client records.

Detailed self-reports of drinking and drug use were collected via a Timeline Followback method (Fals-Stewart et al., 2000; Sobell and Sobell, 1992). We checked self-report accuracy by comparing reports with contiguously collected urine and breath screens; these comparisons showed high correspondence (Ball et al., 2007). Since urine samples were collected on a weekly basis during the 4-week treatment phase, we also calculated the percent of negative drug urine screens during this treatment window for each client.

2.4. Therapist Training, Certification, and Supervision Procedures

To control for therapist levels of experience, education, and commitment to empirically supported therapies, therapists (at least 6 at each site) were randomized to continue CAU as it was typically delivered at the site or to be trained and supervised in MET. Therapists in both conditions delivered three individual 50-minute sessions within a 4-week treatment window. The MET sessions were detailed in a treatment manual (Farentinos & Obert, 2000) adapted from the Miller et al's (1992) Project MATCH manual.

MET therapist training occurred in several steps. First, therapist and program-based supervisors received a 16-hr, expert-led intensive workshop training. Next, they completed audiotaped practice cases supervised by the expert trainers until minimal protocol certification standards had been achieved in three sessions using an initial version of the ITRS (i.e., at least half of the MI-consistent items rated average or above in terms of adherence and competence). The experts also taught the supervisors how to rate the audiotaped sessions using the ITRS and to use the ratings to give MET therapists feedback and coach them maintain adequate performance. Supervisors had monthly consultative calls, including review of co-rated tapes, with their local experts to support their supervisory proficiency. Following the certification process and preparation of supervisors, MET and CAU therapists began to see randomized clients. MET therapists received biweekly supervision. CAU therapists did not receive treatment protocol specific training and received only standard non-protocol supervisory support available at their programs. Therapists in both conditions audiotaped all protocol sessions which were sent to the coordinating center for review. Taped CAU sessions were rated as describe below but not used for supervisory purposes.

2.5. Independent Tape Rater Training

All raters attended an initial 8-hour didactic seminar in which they reviewed the ITRS manual, rating system and forms, and practiced rating the items in both limited therapist-client transactions and in a full protocol session. Following this training, each rater completed ratings for an identical set of 10 calibration tapes randomly selected from the larger pool of protocol tapes to include the two treatment conditions as well as different sites and sessions. Their ratings were compared to the ‘expert’ (SM and SB) consensus ratings. Raters were given individual feedback about their performance and initial item reliabilities were estimated. Next, a second 6-hour rater training was held to address items with lower initial reliability; the raters then completed a set of five additional tapes for final interrater reliability calculation. Combining the didactic and calibration tape components, each rater completed about 44 hours of training. To ensure ongoing rater reliability, a randomly selected common tape was rated on five separate occasions approximately four months apart. Raters were aware of this procedure, but not its timing. Individual ratings were compared to expert ones and feedback was provided to all raters. Two raters drifted (i.e., rated more than half the items more than two scale points above or below the expert ratings) and did not rate tapes again until they had received additional retraining.

2.6. Statistical Analyses

We first calculated scale item reliability using Shrout and Fleiss (1979) intraclass correlation coefficients (ICC's) two-way mixed model (3.1) with item ratings as the random effect and raters as the fixed effect. We report the ICC reliabilities for the mean of the ratings since our scale adherence and competence analyses relied upon combinations of ratings across items. To test for our two hypothesized MI consistent factors (fundamental and advanced skills), we used confirmatory factor analysis using structural models with AMOS (6.0) software (Arbuckle, 2005) using the maximum likelihood estimation and used several indices to determine the acceptability of model fit (Kline, 1998; Marsh et al., 1988; Yadama and Pandey, 1995): nonsignificant (p > .05) chi-square goodness of fit index, a χ2/degrees of freedom ratio < 2, normed fit index (NFI), incremental fit index (IFI), and comparative fit index (CFI) > 0.9, and root mean square error of approximation (RMSEA) < 0.1. Because in larger models (n > 200), the chi-square test usually is significant and often detects trivial differences between sample covariance and fitted covariance matrices (Hu and Bentler, 1995), we relied on the preponderance of evidence from the other indices in determining the best fitting model. Since items proscribed in the MET condition were not prescribed in the contrast condition (CAU), we did not expect the MI-inconsistent items to form a single construct. Similarly, since “general counseling” items could occur together or separately in both conditions and were expected to vary widely across sites, we did not anticipate that these items would necessarily form a construct. As anticipated, exploratory factor analyses of the items used in training and supervision to define MI inconsistent and general counseling items' adherence data failed to show consistent or meaningful factor structures and are not described below. We calculated the respective mean score of the MI inconsistent items that occurred on average at least once per rated session in either condition to permit hypothesis testing and examination of the association of these items with MI consistent ones.

To test for the predicted MET adherence and competence rating differences in fundamental MI, advanced MI, and MI inconsistent strategies (i.e., six comparisons), we conducted ANOVAs using a Bonferroni-corrected α of .0084 (.05/6) with the two mean CFA-derived MI consistent factor and the mean MI inconsistent adherence and competence scores as the separate dependent variables (i.e., six contrasts), treatment condition and program site as the fixed factors, and therapists (nested within condition) as a random factor. We repeated ANOVAs with session number as an additional fixed factor. Including therapist as a random factor is important since wide variation in therapist characteristics within effectiveness trials may impact how well therapist training models (e.g., workshop plus supervision) work in the real world (Crits-Christoph et al., 2003). Multivariate ANOVAs were used to compute estimates (Roy's theta) of the proportion of variance accounted for by treatment condition, program site, session number, and therapist (within condition) effects, with the respective mean adherence and competence scores entered simultaneously in separate analyses (Harris, 1985). These models included two MET and two CAU therapists from each site who had five or more unique client sessions that had been independently rated (n = 387/425 or 91% of rated sessions). This approach was used to provide an adequate representation of therapists' treatment adherence and competence and sufficient balance and variance at the therapist level to evaluate therapist as a random factor. Finally, we conducted Pearson correlations to test for the predicted positive associations between therapist MI consistent adherence and competence and client outcomes (in-session change in client motivation, program retention, primary drug abstinence, and percent drug negative urine screens).

3. Results

3.1. ITRS reliability and validity

ICC reliability estimates for each of the ITRS items are reported in Table 1. As a general rule, ICCs below .40 are poor, .40-.59 are fair, .60-.74 are good, and .75 or above are excellent (Cicchetti, 1994). Using the reliability sample of 15 randomly selected protocol tapes that were each rated by the 15 raters (n = 225), we found that for both adherence and competence scales, 28 of 30 items showed good to excellent reliability; only two items (skills training and psychodynamic interventions) showed fair reliability for adherence and/or competence dimensions and this most likely reflects their marked infrequency across sessions (each of these items occurred in less than 5% of sessions). The 7-point level of motivation scale showed excellent ICC reliability at both the beginning (.96) and end (.96) of the session.

Table 1 also provides means and standard deviations of the adherence and competence items across conditions (see Table 1 footnote for rating score definitions). Overall, MI consistent items occurred on average about 2-4 times within sessions. The most frequently occurring MI consistent items were those characterizing fundamental MI skills: reflections, open-ended questions, and MI style. MI inconsistent items seldom occurred (either not at all or once). When present, unsolicited advice or direction giving (including promotion of self-help group involvement) and use of therapeutic authority (e.g., invoking program policies, noting clinical experience or recovery status as the basis for treatment recommendations) occurred more often, on average about 1-2 times per session. We used the mean rating per session of these items as an indicator of therapists' use of MI inconsistent strategies. Chronbach's alpha among the three items was .84. Assessment of substance use and related social factors were the most frequently occurring general counseling techniques used within sessions. Because Chronbach's alpha for these two items was .39, indicating poor scale reliability between the items, and our hypotheses did not include predictions about the therapists' use of general counseling techniques, we excluded these items from further analyses. Overall, mean competence ratings across items and treatment conditions suggested an ‘average’ therapist skill level.

Table 2 reports the fit indices from the confirmatory factor analysis of our two predicted models for the combined set of three sessions and within each of the three sessions. The results suggest that the hypothesized fundamental MI skills factor had good fit overall and within the three sessions, surpassing almost all thresholds for the fit indices. The hypothesized advanced MI skills factor fit well in the three-session protocol overall, with its best fit in sessions 2 and 3. We used these factors to derive separate factor scores of therapist MI adherence and competence in subsequent analyses (means and standard deviations per condition are in Table 4). Reliability analyses of the two MI skill factors suggested they retained excellent inter-rater reliability, consistent with their individual components (adherence ICC: fundamental skills = .91; advanced skills = .95; competence ICC: fundamental skills = .89; advanced skills = .89). Pearson product-moment correlations between the fundamental and advanced MI skills and MI inconsistent adherence and competence mean scores across conditions (see Table 3) showed that fundamental and advanced MI adherence and competence scores were positively associated with each other in expected ways, but were not redundant. Specifically, for both the fundamental and advanced MI scales, higher adherence scores were associated with higher skill scores (r ranged from .39 to .78, p < .001). Furthermore, therapists' fundamental and advanced MI adherence and competence scores were negatively associated with their use of MI inconsistent strategies (r ranged from -.13 to -.31, p < .01). Competence ratings across each of the strategy categories also were positively associated (r ranged from .23 to .65, p < .001).

Table 2.

Fit indices for alternative models of therapist MI consistent adherence

Model Model statistics
X2 df p X2/df NFI IFI CFI RMSEA
Overall (all sessions)
 Fundamental MI Skills 24.40 5 .00 5.08 .94 .95 .95 .10
 Advanced MI Skills 16.63 5 .00 3.93 .87 .90 .90 .08
Session 1
 Fundamental MI Skills 8.32 5 .14 1.66 .94 .97 .97 .06
 Advanced MI Skills 14.17 5 .02 2.83 .83 .89 .87 .11
Session 2
 Fundamental MI Skills 7.14 5 .21 1.43 .96 .99 .99 .06
 Advanced MI Skills 2.03 5 .85 0.41 .96 1.07 1.00 .05
Session 3
 Fundamental MI Skills 11.89 5 .04 2.38 .91 .95 .94 .10
 Advanced MI Skills 19.63 5 .00 3.93 .87 .90 .90 .00
Open in a new tab

Note. In confirmatory factor analysis, the goodness-of-fit of any predicted latent structure is determined by the preponderance of several indices suggesting a well-fitted model. These fit indices include a nonsignificant chi-square value, chi-square degrees of freedom ratios < 2, a normed fit index (NFI), incremental fit index (IFI), and comparative fit index (CFI) > .90, and the root mean square error of approximation (RMSEA) ≤ .10 degrees of freedom (Marsh et al., 1988; Yadama and Pandey, 1995). Statistics meeting these thresholds are bolded.

Table 4.

Fundamental and advanced MI skills and MI inconsistent strategies adherence and competence ratings by treatment condition, program site, and therapist (nested in condition): univariate and multivariate ANOVAs

ANOVA MET CAU Treatment Condition Program Site Treatment by Site Therapist (in Condition)
Univariate
 Fundamental MI Skills
 Adherence M = 4.38 2.92 F = 222.92 5.99 1.76 2.79
SD = 0.97 0.99 df = 1,374 4,374 4,374 2,374
N = 189 203 p <. 001 p<.001 p = .14 p = .06
 Competence M = 4.85 3.65 F = 111.61 1.41 3.60 4.70
SD = 0.77 1.08 df = 1,238 4,238 4,238 2,238
N = 191 203 p < .001 p = .22 p=.01 p=.01
 Advanced MI Skills
 Adherence M = 2.65 1.67 F = 192.02 4.36 2.64 0.99
SD = 0.84 0.53 df = 1,374 4,374 4,374 2,374
N = 191 203 p < .001 p < .001 p = .03 p = .37
 Competence M = 4.79 4.25 F = 40.19 1.84 2.26 2.37
SD = 0.65 0.79 df = 1,238 4,238 4,238 2,238
N = 186 173 p < .001 p < .12 p = .06 p = .10
 MI Inconsistent Strategies
 Adherence M = 1.69 2.68 F = 93.17 6.85 3.66 2.77
SD = 0.89 1.33 df = 1,374 4,374 4,374 2,374
N = 191 202 p < .001 p < .001 p = .01 p = .06
 Competence M = 4.39 4.26 F = 2.23 2.07 3.54 0.27
SD = 0.62 0.76 df = 1,238 4,238 4,238 2,238
N = 110 176 p = .14 p = .09 p = .01 p = .77
Multivariate
 Adherence θ = .52 .11 .06 .02
F = 133.24 11.92 5.52 2.16
df = 3,372 4,374 4,374 3,373
p < .001 p < .001 p < .001 p = .09
 Competence θ = .50 .06 .08 .04
F = 39.66 3.47 4.84 3.47
df = 3,236 4,238 4,238 3,237
p < .001 P = .01 p < .001 p = .02
Open in a new tab

Note: For univariate ANOVAs, p-values ≤ .0084 (Bonferroni-correction of .05/6) are significant. MET = motivational enhancement therapy; CAU = counseling-as-usual. Univariate ANOVAs (for tests of adherence and competence differences) and multivariate ANOVAs (for deriving Roy's theta) included two MET and two CAU therapists from each site who had five or more unique client sessions that had been independently rated (MET = 387/425). Roy's theta (θ) provides an estimate of the amount of variance accounted for by each effect involving adherence and competence ratings, with each set of ratings entered simultaneously in separate multivariate models (Harris, 1985).

Table 3.

Pearson product-moment correlations between mean fundamental and advanced MI skills and MI inconsistent adherence and competence ratings

Fundamental MI Skills Advanced MI Skills MI Inconsistent
Adh Comp Adh Comp Adh Comp
Fundamental MI Skills
 Adherence --
 Competence .78*** --
Advanced MI Skills
 Adherence .69*** .58*** --
 Competence .48*** .65*** .39*** --
MI Inconsistent
 Adherence -.19** -.31*** -.24*** -.13** --
 Competence .04 .23*** -.04 .55*** -.03 --
Open in a new tab

Note. The sample sizes varied in the calculation of correlation coefficients involving competence ratings because competence is not rated if items comprising competence scores did not occur. The sample sizes for the correlation calculations ranged from a minimum of 276 for radvanced MI competence: MI inconsistent competence to a maximum of 421 for rfundamental MI competence: MI inconsistent adherence.

**

p < .01.

***

p < .001.

3.2. Adherence and Competence Analyses between Treatment Conditions, Program Sites, and Therapists

Multivariate ANOVAs examining adherence to fundamental MI, advanced MI, and MI inconsistent strategies revealed significant differences by treatment condition, site, and the interaction of treatment and site (p < .01), but no significant differences were seen by therapist. The univariate breakdown showed higher fundamental and advanced MI and lower MI inconsistent scores for the MET condition. Similarly, the multivariate ANOVAs examining competence in administering fundamental MI, advanced MI, and MI inconsistent strategies revealed significant differences by treatment condition, site, and the interaction of treatment and site, but also showed a significant therapist effect (p < .05). The factor level breakdown of these findings revealed that the primary differences were seen in the MI fundamental and MI advanced scores rather than MI inconsistent scores (see Table 4). The amount of variance accounted for by treatment condition in both adherence and competence (thetas = 52% and 50%, respectively) was considerably higher than the amount of variance accounted for by therapist (thetas = 2% and 4%, respectively). When the ANOVAs were repeated with session number included as a fixed factor, the pattern of results remained the same. Session number had no significant bearing on treatment adherence or competence by itself or in interaction with treatment condition or program site.

3.3. Adherence and Competence, In-session Change in Motivation, and Client Outcomes

Pearson product-moment correlations between mean fundamental and advanced MI skills and MI inconsistent adherence and competence ratings and in-session change in motivation, client retention, and substance use outcomes (total sample and by condition) are presented in Table 5. As predicted, fundamental and advanced MI adherence and competence was positively related to in-session change in client motivation for the total sample (r ranged from .13 to .22, p < .01) and held most consistently when examining them among the MET condition therapists only. In addition, within the MET condition fundamental MI adherence and competence (r = .18 and .15 respectively, p < .05) and advanced MI adherence (r = .21, p < .001) were positively associated with the percent of negative drug screens obtained during the 4-week treatment phase, whereas the associations between the percent of negative drug screens and MI inconsistent adherence and competence were negative (r = -.17 and -.19, p < .05). Patterns of associations between MI fundamental, advanced, inconsistent strategies and treatment retention and drug abstinence outcomes were more variable.

Table 5.

Pearson product-moment correlations between mean fundamental and advanced MI skills and MI inconsistent adherence and competence ratings and in-session change in motivation, client retention, and substance use outcomes

In-session Change in Motivation Retention (days in treatment) Days Primary Drug Abstinence Percent Negative Urine Screens
4-week
Post-Treatment		 12-week
Follow-up		 4-week
Post-Treatment		 12-week
Follow-up		 4-week
Post-Treatment
Overall
 Fundamental MI Skills
 Adherence .18*** .02 .00 .00 -.06 .05
 Competence .19*** -.05 .02 .00 .00 .04
 Advanced MI Skills
 Adherence .22*** .00 .00 .00 -.04 .08
 Competence .13** -.12* -.01 .01 -.01 -.03
 MI Inconsistent Strategies
 Adherence -.04 .06 -.04 .09 .04 -.08
 Competence -.01 -.14* -.03 -.01 .03 -.08
MET Condition Only
 Fundamental MI Skills
 Adherence .20*** .01 -.03 .13 -.02 .18**
 Competence .22*** -.10 .04 .14* .08 .15*
 Advanced MI Skills
 Adherence .21*** -.06 .03 .08 .02 .21***
 Competence .20*** -.13 .00 .18** .05 .02
 MI Inconsistent Strategies
 Adherence -.11 -.06 -.20*** .06 -.04 -.17*
 Competence -.10 -.28** -.11 .03 .13 -.19*
CAU Condition Only
 Fundamental MI Skills
 Adherence .10 -.11 -.02 -.09 -.06 -.02
 Competence .12 -.16* -.03 -.07 .00 -.01
 Advanced MI Skills
 Adherence .16* -.15* -.09 -.10 -.06 -.03
 Competence .01 -.21** -.05 -.15* -.03 -.09
 MI Inconsistent Strategies
 Adherence .09 .20** .08 .12 .07 -.04
 Competence .03 -.08 .01 -.03 -.01 -.01
Open in a new tab

Note. The sample sizes varied in the calculation of correlation coefficients involving competence ratings because competence is not rated if items comprising competence scores did not occur. The sample sizes for the correlation calculations ranged from a minimum of 162 for rdays in treatment (4-week post-treatment): advanced MI competence in the TAU condition only to a maximum of 425 for rdays in treatment (12-week follow-up)∷fundamental MI competence in the overall sample.

*

p < .05.

**

p < .01.

***

p < .001

4. Discussion

This report addressed adherence and competence of interventions associated with MI and general counseling based on session audiotapes drawn from a large multisite randomized clinical trial conducted in community based drug treatment and delivered by clinicians drawn from the staff of these programs. Principal findings were as follows: First, psychometric analysis of the ITRS indicated that it was psychometrically sound in terms of inter-rater reliability and factor structure. Second, clinicians drawn from the staff of community programs can be trained to administer MET to a discriminable level of adherence and competence. Third, higher levels of MI adherence and competence are associated with increases in client motivation and some positive client treatment outcomes in community programs.

Regarding the psychometric properties of the ITRS, the majority of items had excellent inter-rater reliability. Item-level ICC reliabilities compared favorably with those typically found in treatment fidelity scales designed to measure therapist adherence and competence in previous multi-site randomized clinical trials testing the efficacy of substance abuse and mental health treatments (Barber et al., 2004; Carroll et al., 1998; Hill et al., 1988; Shaw et al., 1999). Moreover, items in this study that were most relevant to the delivery of MI had reliability estimates which exceeded those commonly reported for other MI fidelity measures: 80% of the MI consistent and inconsistent adherence and competence items were in the excellent range versus 70% for the MITI (Moyers et al., 2005), 22% for the MISC therapist items (Moyers et al., 2003), and 12% for the MISTS (Madson et al., 2005). Similarly, reliability estimates for items measuring therapeutic strategies consistent with advanced MI practice (e.g., drawing out pros, cons, and ambivalence; heightening discrepancies; using strategies for evoking motivation for change) were in the excellent range. Past efforts to consistently rate these types of MI items have been uneven, and only the MITI has achieved a similarly degree of agreement for its MI-adherent item (Moyers et al., 2005).

In addition to its high level of interrater reliability in this study, the MI consistent adherence items converged in predicted ways to form two independent factors that captured core conceptual components of MI: 1) fundamental MI skills that underpin the highly empathic and collaborative style or spirit of MI; and 2) advanced MI skills that embody strategies used for evoking client statements which support motivation for change and commitment to behavior change plans. As expected, the adherence and competence ratings for these two factors were positively associated with each other and negatively associated with MI inconsistent adherence ratings. Competence ratings were significantly and highly correlated across therapeutic strategies, suggesting overall skill in implementing substance abuse treatment strategies may relate to general factors such as making interventions relevant and individualized to client problems and maintaining a clear theoretical framework for structuring sessions (Shaw and Dobson, 1988; Shaw et al., 1999), in addition to strategy-specific competency elements.

Moreover, support for the two distinct (fundamental and advanced) MI factors is consistent with recommendations for a two-step process for training therapists in MI. First, therapists commonly learn fundamental MI skills by attending several day workshops (Miller and Mount, 2001; Miller and Rollnick, 2002). Following workshop training, therapists are encouraged to practice MI while receiving clinical supervision based on a supervisor's adherence and competence review of recorded client sessions and feedback and coaching to improve therapists' performance (Miller et al., 2004). During this post-workshop training phase, therapists are more likely to develop advanced MI skills involved in strategically eliciting client self-motivational statements, garnering client commitment to change, and knowing when to transition to more action-oriented or skill-building substance abuse treatment approaches (Miller & Moyers, 2007). The MI consistent items used to assess therapist fidelity in the protocols coalesced around these two MI skill components and, therefore, have much potential for use in monitoring and supervising community program therapists implementing MI. A recently released MI supervision manual called Motivational Interviewing Assessment: Supervisory Tools for Enhancing Proficiency called MIA: STEP (Martino et al., 2006) based on the MI training approach and ITRS has been produced for these purposes.

This study presented a comparatively challenging evaluation of the discriminability of MI from standard practice in that the study therapists were drawn from the staff of the community-based programs and randomized to deliver either MET or CAU in order to ensure comparable levels of overall clinical skill, experience and allegiance to different substance abuse treatment approaches (Carroll et al., 2002). Univariate ANOVA tests of MI adherence and competence discrimination between conditions showed that MET therapists delivered fundamental and advanced MI skill strategies significantly more often and with greater competence than CAU therapists. MET therapists also used significantly lower levels of MI inconsistent strategies in comparison to CAU therapists. Furthermore, while several significant program site differences existed in MI fundamental, advanced, and inconsistent ratings, multivariate ANOVAs indicated that treatment condition accounted for substantially more variance in overall adherence and competence scores than program site, condition by site interactions, the nonspecific effects of therapists within conditions, and the specific treatment session delivered. Overall, MET therapists used fundamental MI skills quite a bit and advanced MI skills less often, though both with average to good levels of competence. The findings suggest that community program therapists can learn to deliver MET with adequate fidelity when provided with initial intensive training workshops and follow-up supervision delivered by on-site, program-based supervisors who have been trained to rate sessions for MI adherence and competence and to provide therapists with feedback and coaching to improve their performance.

This study also showed some significant, though very modest (.13 to .34), associations between therapist MI fidelity and client process and outcome variables. The most consistent findings were that therapists who used fundamental and advanced MI skills more often and with higher levels of competence were significantly more likely to have clients who verbally expressed in-session increases in their motivation to reduce or stop substance use. These findings were more pronounced in the MET in contrast to the CAU condition and are consistent with the Miller et al. (2004) study in which therapists who received repeated feedback and coaching interventions after MI workshop training had significantly better client responses (more statements indicative of change and less suggesting resistance) in their sessions. MI inconsistent adherence and competence was unrelated to in-session change in motivation. In addition, more frequent use of fundamental and advanced MI skills in the MET condition was significantly related to clients having more negative drug urine screens during the 4-week treatment phase. Overall, these findings suggest that training community program therapists to implement MET with fidelity may result in improvement in their ability to enhance their clients' motivation for change. Moreover, the presence of some significant associations between the MI fidelity measure and client process and outcomes is encouraging and a step forward empirically in that none of the existing alternative MI fidelity measures have heretofore reported these types of relationships.

Strengths of this study include the careful examination of therapist MI adherence and competence in a wide range of program sites, therapists, and clients within an effectiveness trial, discrimination of two core MI fidelity areas across treatment conditions, and examination of their relationship to client outcomes. The study also has several limitations. First, programs and therapists participating in the Clinical Trials Network may be more open to training in and evaluation of empirically-supported treatments than community programs not involved in this research-practice partnership (Roman et al., 2006), thus limiting the generalizability of this study's findings. Second, given the low frequency in which therapists in both conditions used most MI inconsistent and general counseling strategies, our analyses involving these therapeutic techniques were limited to those items that occurred enough in the early phase of treatment to warrant further evaluation. Thus, this study did not assess the relationship of other MI inconsistent techniques that are clearly detrimental to MI performance such as direct confrontation (Miller et al., 1993). Third, while our analyses captured 70% of the client participants who received protocol treatment, the decision to limit our process analyses only to participants who completed three sessions limits the variability in outcomes by excluding some clients who dropped out of treatment and may have not done as well. Finally, this study was not a randomized controlled trial of the methods used to train community program therapists in MET. Conclusions made about the effectiveness of such methods for improving MI adherence and competence and the mediational role adherence and competence plays in producing client outcomes requires further study.

Footnotes

1

Items 31-34 involve 7-point Likert scale general ratings of the therapist (overall skillfulness, ability to maintain the session's structure, demonstration of frustration during the session, general client discussions and self-disclosures). Items 35-37 involve 7-point Likert scale ratings of the client (unrelated session discussions, difficulty understanding, and working alliance). Future reports will examine treatment condition differences for these items.

2

As suggested by Kelloway (1998), we compared our two hypothesized models with two alternative ones to determine if models other than the ones we predicted might provide a better description of the adherence data instead of relying solely on a model's absolute fit: (1) a 1-factor model in which all 10 MI consistent items form a single MI fidelity factor structure; and (2) a 2-factor model in which MI fidelity is supported by both fundamental and advanced MI skills factors. The 1-factor and 2-factor MI fidelity alternative models had poor fit (all NFI, IFI, and CFIs < .9). These analyses are available upon request from the first author.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Amhreim P, Miller WR, Yahne CE, Palmer M, Fulcher L. Client commitment language during motivational interviewing. J Consult Clin Psych. 2003;71:862–878. doi: 10.1037/0022-006X.71.5.862. [DOI] [PubMed] [Google Scholar]
  2. Arbuckle JL. AMOS (Version 60) [Computer Program] Chicago, IL: SPSS; 2005. [Google Scholar]
  3. Arkowitz H, Westra HA, Miller WR, Rollnick S. Guilford Press; New York: 2007. Motivational Interviewing in the Treatment of Psychological Problems. [Google Scholar]
  4. Ball SA, Bachrach K, DeCarlo J, Farentinos C, Keen M, McSherry T, Polcin D, Snead N, Sockriter R, Wrigley P, Zammarelli MA, Carroll KM. Characteristics, beliefs, and practices of community clinicians trained to provide manual-guided therapy for substance abusers. J Subst Abuse Treat. 2002a;23:309–318. doi: 10.1016/s0740-5472(02)00281-7. [DOI] [PubMed] [Google Scholar]
  5. Ball SA, Martino S, Corvino J, Morganstern J, Carroll KM. Independent tape rater guide. Unpublished psychotherapy tape rating manual 2002b [Google Scholar]
  6. Ball SA, Martino S, Nich C, Frankforter TL, Van Horn D, Crits-Christoph P, Woody GE, Obert JL, Farentinos C, Carroll KM. Site matters: Motivational enhancement therapy in community drug abuse clinics. J Consult Clin Psych. 2007;75:556–567. doi: 10.1037/0022-006X.75.4.556. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Barber JP, Foltz C, Crits-Christoph P, Chittams J. Therapists' adherence and competence and treatment discrimination in the NIDA Collaborative Cocaine Treatment Study. J Clin Psych. 2004;60:29–41. doi: 10.1002/jclp.10186. [DOI] [PubMed] [Google Scholar]
  8. Bien TH, Miller WR, Boroughs JM. Motivational interviewing with alcohol outpatients. Behav Cogn Psychother. 1993;21:347–356. [Google Scholar]
  9. Brown JM, Miller WR. Impact of motivational interviewing on participation and outcome in residential alcoholism treatment. Psychol Addict Behav. 1993;7:211–218. [Google Scholar]
  10. Burke BL, Arkowitz H, Menchola M. The efficacy motivational interviewing: A meta-analysis of controlled trials. J Consult Clin Psych. 2003;71:843–861. doi: 10.1037/0022-006X.71.5.843. [DOI] [PubMed] [Google Scholar]
  11. Carroll KM, Ball SA, Nich C, Martino S, Frankforter TL, Farentinos C, Kunkel L, Mikulich-Gilbertson S, Morgenstern J, Obert JL, Polcin D, Snead N, Woody GE. Motivational interviewing to improve treatment engagement and outcome in individuals seeking treatment for substance abuse: A multisite effectiveness study. Drug Alcohol Depend. 2006;81:301–312. doi: 10.1016/j.drugalcdep.2005.08.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Carroll KM, Farentinos C, Ball SA, Crits-Christoph P, Libby B, Morgenstern J, Olbert JL, Polcin D, Woody GE, Clinical Trials Network MET meets the real world: design issues and clinical strategies in the Clinical Trials Network. J Subst Abuse Treat. 2002;23:73–80. doi: 10.1016/s0740-5472(02)00255-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Carroll KM, Libby B, Sheehan J, Hyland N. Motivational interviewing to enhance treatment initiation in substance abusers: An effectiveness study. Am J Addict. 2001;10:335–339. doi: 10.1080/aja.10.4.335.339. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Carroll KM, Nich C, Sifry R, Frankforter T, Nuro KF, Ball SA, Fenton LR, Rounsaville BJ. A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug Alcohol Depend. 2000;57:225–238. doi: 10.1016/s0376-8716(99)00049-6. [DOI] [PubMed] [Google Scholar]
  15. Carroll KM, Rounsaville BJ. A vision of the next generation of behavioral therapies research in the addictions. Addiction. 2007;102:850–862. doi: 10.1111/j.1360-0443.2007.01798.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Carroll KM, Connors GJ, Cooney NL, DiClemente CC, Donovan DM, Longabaugh RL, Kadden RM, Rounsaville BJ, Wirtz PW, Zweben A. Internal validity of Project MATCH treatments: Discriminability and integrity. J Consult Clin Psych. 1998;66:290–303. doi: 10.1037//0022-006x.66.2.290. [DOI] [PubMed] [Google Scholar]
  17. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment. 1994;6:284–290. [Google Scholar]
  18. Crits-Christoph P, Tu X, Gallop R. Therapists as fixed versus random effects - some statistical and conceptual issues: a comment on Siemer and Joormann. Psychol Methods. 2003;8:518–523. doi: 10.1037/1082-989X.8.4.518. [DOI] [PubMed] [Google Scholar]
  19. Fals-Stewart W, O'Farrll TJ, Freitas TT, McFarlin SK, Rutigliano P. The Timeline Followback reports of psychoactive substance use by drug-abusing patients: Psychometric properties. J Consult Clin Psych. 2000:134–144. doi: 10.1037//0022-006x.68.1.134. [DOI] [PubMed] [Google Scholar]
  20. Farentinos C, Obert JL. CTN Motivational Enhancement Treatment Manual. 2000 Unpublished manuscript. [Google Scholar]
  21. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231); 2005. [Google Scholar]
  22. Harris RJ. A primer of multivariate statistics. 2nd. New York: Academic Press; 1985. [Google Scholar]
  23. Hettema J, Steele J, Miller WR. Motivational interviewing. Annual Rev Clin Psychol. 2005;1:91–111. doi: 10.1146/annurev.clinpsy.1.102803.143833. [DOI] [PubMed] [Google Scholar]
  24. Hill CE, O'Grady KE, Elkin I. Applying the Collaborative Study Psychotherapy Rating Scale to rate therapist adherence in cognitive-behavior therapy, interpersonal therapy, and clinical management. J Consult Clin Psych. 1992;60:73–79. doi: 10.1037//0022-006x.60.1.73. [DOI] [PubMed] [Google Scholar]
  25. Hu LT, Bentler PM. Evaluating model fit. In: Hoyle RH, editor. Structural Equation Modelling: Concepts, Issues and Applications. Sage; Thousand Oaks, CA: 1995. pp. 77–99. [Google Scholar]
  26. Humphreys K, Weingardt KR, Horst D, Joshi AA, Finney JW. Prevalence and predictors of research participant eligibility criteria in alcohol treatment outcome studies. Addiction. 2005;100:1249–1257. doi: 10.1111/j.1360-0443.2005.01175.x. [DOI] [PubMed] [Google Scholar]
  27. Humphreys K, Weisner C. Use of exclusion criteria in selecting research subjects and its effect on the generalizability of alcohol treatment outcome studies. Am J Psych. 2000;157:588–594. doi: 10.1176/appi.ajp.157.4.588. [DOI] [PubMed] [Google Scholar]
  28. Kline RB. Principles and practice of structural equation modeling. Guilford Press; New York: 1998. [Google Scholar]
  29. Kelloway EK. Using LISREL for structural equation modeling: A researcher's guide. Sage; Thousand Oaks, CA: 1998. [Google Scholar]
  30. Madson MB, Campbell TC, Barrett DE, Brondino MJ, Melchert TP. Development of the motivational interviewing supervision and training scale. Psychol Addict Behav. 2005;19:303–310. doi: 10.1037/0893-164X.19.3.303. [DOI] [PubMed] [Google Scholar]
  31. MarijuanaTreatment Project Research Group. Brief treatments for cannabis dependence: Findings from a randomized multisite trial. J Consult Clin Psych. 2004;73:1015–1025. doi: 10.1037/0022-006X.72.3.455. [DOI] [PubMed] [Google Scholar]
  32. Marsh HW, Balla JR, McDonald RP. Goodness-of-fit indexes in confirmatory factor analysis: the effect of sample size. Psychol Bull. 1988;103:391–410. [Google Scholar]
  33. Martino S, Ball SA, Gallon SL, Hall D, Garcia M, Ceperich S, Farentinos C, Hamilton J, Hausotter W. Motivational Interviewing Assessment: Supervisory Tools for Enhancing Proficiency. Salem, OR: Northwest Frontier Addiction Technology Transfer Center, Oregon Health and Science University; 2006. [November 20, 2007]. http://www.nfattc.org. [Google Scholar]
  34. Martino S, Carroll KM, O'Malley SO, Rounsaville BJ. Motivational interviewing with psychiatrically ill substance abusing patients. Am J Addict. 2000;9:88–91. doi: 10.1080/10550490050172263. [DOI] [PubMed] [Google Scholar]
  35. Martino S, Carroll KM, Nich C, Rounsaville BJ. A randomized controlled pilot study of motivational interviewing for patients with psychotic and drug use disorders. Addiction. 2006;101:1479–92. doi: 10.1111/j.1360-0443.2006.01554.x. [DOI] [PubMed] [Google Scholar]
  36. Miller WR. Enhancing Motivation for Change in Substance Abuse Treatment TIP Series 35. Rockville, MD: US Department of Health and Human Services Publication No (SMA) 02-3693; 1999. [Google Scholar]
  37. Miller WR, Benefield B, Tonigan JS. Enhancing motivation for change in problem drinking: A controlled comparison of two therapist styles. J Consult Clin Psych. 1993:455–461. doi: 10.1037//0022-006x.61.3.455. [DOI] [PubMed] [Google Scholar]
  38. Miller WR, Mount KA. A small study of training in motivational interviewing: Does one workshop change clinician and client behavior? Behav Cogn Psychother. 2001;29:457–471. [Google Scholar]
  39. Miller WR, Moyers T. Eight stages in learning motivational interviewing. J Teach Addict. 2007;5:3–17. [Google Scholar]
  40. Miller WR, Moyers T, Ernst D, Amrhein P. Manual for the Motivational Interviewing Skills Code (MISC) v. 2.0. [November 20, 2007];2003 http://casaa.unm.edu/coding.inst.html.
  41. Miller WR, Rollnick S. Motivational Interviewing: Preparing people for change. 2nd. Guilford Press; New York: 2002. [Google Scholar]
  42. Miller WR, Rollnick S, Moyers TB. Motivational interviewing (6-tape series) University of New Mexico; 1998. [Google Scholar]
  43. Miller WR, Sorensen JL, Selzer JA, Bringham GS. Disseminating evidence-based practices in substance abuse treatment: A review with suggestions. J Subst Abuse Treat. 2006;31:25–39. doi: 10.1016/j.jsat.2006.03.005. [DOI] [PubMed] [Google Scholar]
  44. Miller WR, Sovereign RG, Krege B. Motivational interviewing with problem drinkers: II. The drinker's check-up as a preventive intervention. Behav Psychother. 1988;16:251–268. [Google Scholar]
  45. Miller WR, Yahne CE, Moyers TE, Martinez J, Pirritano M. A randomized trial of methods to help clinicians learn motivational interviewing. J Consult Clin Psych. 2004;72:1050–1062. doi: 10.1037/0022-006X.72.6.1050. [DOI] [PubMed] [Google Scholar]
  46. Miller WR, Zweben A, DiClemente CC, Rychtarik RG. Motivational Enhancement Therapy manual: A clinical research guide for therapists treating individuals with alcohol abuse and dependence (Volume 2, Project MATCH Monograph Series) Rockville, MD: National Institute on Alcohol Abuse and Alcoholism; 1992. [Google Scholar]
  47. Moyers T, Martin T, Catley D, Harris KJ, Ahluwalia JS. Assessing the integrity of motivational interviewing interventions: Reliability of the motivational interviewing skills code. Behav Cogn Psychother. 2003;31:177–184. [Google Scholar]
  48. Moyers TB, Martin T, Manuel JK, Hendrickson SML, Miller WR. Assessing competence in the use of motivational interviewing. J Subst Abuse Treat. 2005;28:19–26. doi: 10.1016/j.jsat.2004.11.001. [DOI] [PubMed] [Google Scholar]
  49. Moyers TB, Martin T, Manuel JK, Miller WR. The Motivational Interviewing Treatment Integrity (MITI) code (coding manual) Albuquerque, NM: University of New Mexico, Center on Alcoholism, Substance Abuse and Addictions (CASAA); 2004. [Google Scholar]
  50. Project MATCH Research Group. Matching alcoholism treatment to client heterogeneity: Project MATCH posttreatment drinking outcomes. J Stud Alcohol. 1997;58:7–29. [PubMed] [Google Scholar]
  51. Rollnick S, Mason P, Butler C. Health Behavior Change: A Guide for Practitioners. Churchill Livingstone; London: 1999. [Google Scholar]
  52. Rollnick S, Miller WR. What is motivational interviewing? Behav Cogn Psychother. 1995;23:325–334. doi: 10.1017/S1352465809005128. [DOI] [PubMed] [Google Scholar]
  53. Rollnick S, Miller RW, Butler CC. Motivational Interviewing in Health Care: Helping Patients Change Behavior. Guilford Press; New York: 2007. [Google Scholar]
  54. Roman PM, Ducharme LJ, Knudsen HK. Patterns of organization and management in private and public substance abuse treatment programs. J Subst Abuse Treat. 2006;31:235–243. doi: 10.1016/j.jsat.2006.06.017. [DOI] [PubMed] [Google Scholar]
  55. Saunders B, Wilkinson C, Phillips M. The impact of a brief motivational intervention with opiate users attending a methadone programme. Addiction. 1995;90:415–424. doi: 10.1046/j.1360-0443.1995.90341510.x. [DOI] [PubMed] [Google Scholar]
  56. Shaw BF, Dobson KS. Competency judgments in the training and evaluation of psychotherapists. J Consult Clin Psych. 1988;56:666–672. [PubMed] [Google Scholar]
  57. Shaw BF, Elkin I, Yamaguchi J, Olmstead M, Vallis TM. Therapist competence ratings in relation to clinical outcome in cognitive therapy of depression. J Consult Clin Psych. 1999;67:837–846. doi: 10.1037//0022-006x.67.6.837. [DOI] [PubMed] [Google Scholar]
  58. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull. 1979;86:420–429. doi: 10.1037//0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]
  59. Sobell LC, Sobell MB. Timeline followback: a technique for assessing self-reported alcohol consumption. In: Litten RZ, Allen J, editors. Measuring alcohol consumption: Psychosocial and biological methods. Humana Press; New Jersey: 1992. [Google Scholar]
  60. Steinberg KL, Roffman RA, Carroll KM, Kabela E, Kadden R, Miller M, Duresky D. Marijuana Treatment Project Research Group: Tailoring cannabis dependence treatment for a diverse population. Addiction. 2002;97:135–142. doi: 10.1046/j.1360-0443.97.s01.5.x. [DOI] [PubMed] [Google Scholar]
  61. Stotts AL, Schmitz JM, Rhoades HM, Grabowski J. Motivational interviewing with cocaine-dependent patients: A pilot study. J Consult Clin Psych. 2001;69:858–862. doi: 10.1037//0022-006x.69.5.858. [DOI] [PubMed] [Google Scholar]
  62. Yadama GN, Pandey S. Effect of sample size on goodness-of-fit indices in structural equation models. J Soc Serv Res. 1995;20:49–70. [Google Scholar]
  63. Yahne CE, Miller WR. La Entrevista motivacional: Preparacion para el cambio. University of New Mexico; 2002. [Google Scholar]

RESOURCES