Site Matters: Multisite Randomized Trial of Motivational Enhancement Therapy in Community Drug Abuse Clinics

Abstract

The effectiveness of motivational enhancement therapy (MET) in comparison with counseling as usual (CAU) for increasing retention and reducing substance use was evaluated in a multisite randomized clinical trial. Participants were 461 outpatients treated by 31 therapists within 1 of 5 outpatient substance abuse programs. There were no retention differences between the 2 brief intervention conditions. Although both 3-session interventions resulted in reductions in substance use during the 4-week therapy phase, MET resulted in sustained reductions during the subsequent 12 weeks whereas CAU was associated with significant increases in substance use over this follow-up period. This finding was complicated by program site main effects and higher level interactions. MET resulted in more sustained substance use reductions than CAU among primary alcohol users, but no difference was found for primary drug users. An independent evaluation of session audiotapes indicated that MET and CAU were highly and comparably discriminable across sites.

Motivational interviewing (Miller & Rollnick, 1991, 2002) and a manual-based adaptation developed for clinical trials research, motivational enhancement therapy (MET; Miller, Zweben, Di-Clemente, & Rychtarik, 1992), are brief interventions to enhance motivation to change addictive behaviors. Together they constitute a carefully defined, tested, and disseminated evidence-based therapy for substance use disorders. Several comprehensive reviews or meta-analyses (Burke, Arkowitz, & Menchola, 2003; Dunn, Deroo, & Rivara, 2001; Hettema, Steele, & Miller, 2005; Noonan & Moyers, 1997) indicate strong empirical support for the efficacy of brief motivational interventions for increasing client retention and decreasing substance use. A meta-analysis of clinical trials involving MET or MI-oriented approaches for substance use disorders indicated a moderate (.4 to .5) effect size over 3-month follow-up periods (Hettema et al., 2005). However, there is significant variability in outcomes across programs, providers, and problem areas, and there are numerous studies in which MI or MET had no significant effect (Hettema et al., 2005; Noonan & Moyers, 1997). This variability raises important questions about the parameters for effective delivery of the model. The current multisite clinical trial evaluated the effectiveness of MET versus usual counseling when delivered to a heterogeneous group of substance abuse outpatients by addiction treatment providers working in community settings.

Most MI or MET research has involved individuals who engage in heavy drinking or alcohol abuse, although several clinical trials have focused on opioid (Saunders, Wilkinson, & Phillips, 1995), cocaine (Stotts, Schmitz, Rhoades, & Grabowski, 2001), marijuana (Marijuana Treatment Project, 2004; Stephens, Roffman, & Curtin, 2000), and dually diagnosed (Martino, Carroll, Nich, & Rounsaville, 2006) patient groups. MI or MET may be less effective as a brief intervention for patients dependent on illicit drugs whose psychosocial problems may require greater treatment structure, direction, and duration than is provided by a time-limited, open exploration of internal motivation. Several studies evaluating comparatively large samples of individuals in community settings who abuse drugs have found no differences between a motivational enhancement intervention compared with standard care (Booth, Kwiatkowski, Iguchi, Pinto, & John, 1998; Donovan, Rosengren, Downey, Cox, & Sloan, 2001; Miller, Yahne, & Tonigan, 2003). In a previous study, we compared a single MI-oriented intake session with a standard intake assessment in 431 outpatients (Carroll et al., 2006). There were no differences between conditions on 1- or 3-month drug use outcomes, but MI resulted in significantly better 1-month retention. The data also suggested somewhat greater improvement in participants whose primary substance was alcohol rather than drugs. It is notable that most studies with null findings in drug abuse samples have involved single sessions of MI (Donovan et al., 2001; Miller et al., 2003) in comparison with a standard care intervention. Multisite studies on alcohol (Project MATCH Research Group, 1997; UKATT Research Team, 2005) and cannabis (Marijuana Treatment Project, 2004) use disorders have delivered several sessions of MET and found it efficacious when compared with other empirically supported therapies, more intensive approaches, or waiting list controls. Although single or spaced sessions may be sufficient to promote positive outcomes in heavy drinkers, repeated exposure to the intervention may be necessary for patients who are severely drug dependent to accomplish sustained reduction in symptoms.

Multisite studies offer the opportunity to explore important questions related to the effectiveness of MET or MI with different types of substance abuse patients, providers, and program settings. Such questions can be difficult to answer by examining the aggregated results of single-site studies of MET or MI because of potentially important variations or confounds across studies with regard to intervention delivery (e.g., number and duration of sessions, extent of reliance on manuals, session content, level of supervision and quality monitoring), treatment setting (e.g., medical care, college counseling, substance use treatment of differing modalities), and therapist selection (e.g., prior experience with approach; Burke et al., 2003; Hettema et al., 2005). An important effectiveness question is whether MET and MI increase retention and decrease substance use when provided by practitioners without prior allegiance to the approach and when provided to a heterogeneous group of substance abuse patients treated in different community programs. Along with the UKATT project (UKATT Research Team, 2005), the current study is one of the few to have attempted to control some of these influences by randomly assigning MET-inexperienced therapists within several program sites to different therapy conditions.

This study addressed several important questions related to the methodological parameters (rather than theoretical mechanisms) of effective MET delivery: (a) Is MET more effective than counseling as usual (CAU) when delivered in community practice settings with diverse groups of substance abuse outpatients and therapists? (b) Does a participant’s primary substance use predict differential response to MET versus CAU? (c) What treatment program site characteristics are associated with the effective application of MET? (d) Is MET discriminable from CAU when delivered by trained and supervised therapists previously inexperienced with the approach? We hypothesized that three individual MET sessions would be more effective than three individual CAU sessions in retaining participants through the initial months of outpatient treatment and in reducing their substance use. We predicted that primary alcohol use participants would have better retention and substance use outcomes with MET than CAU but that MET would provide no added benefit to standard treatment of primary drug use participants. Similarly, program sites treating proportionately greater numbers of primary alcohol users would show better MET than CAU outcomes, whereas sites treating proportionately more primary drug users would show no differences in MET versus CAU outcomes. Finally, we expected that randomly assigned therapists without prior MET training would be able to implement MET at an acceptable level of fidelity that was comparably discriminable from CAU across program sites.

Method

This multisite randomized clinical trial was implemented in five outpatient substance abuse treatment programs within three research–practice partnerships in the National Institute on Drug Abuse (NIDA) Clinical Trials Network (CTN): Connecticut Renaissance in Norwalk, Connecticut, and Liberation Programs in Stamford, Connecticut, within the New England Node (at Yale University); Rehab After Work and Northeast Treatment Centers in Philadelphia within the Delaware Valley Node (at University of Pennsylvania); Tarzana Treatment Center in Tarzana, California, within the Pacific Region Node (at University of California at Los Angeles). Based on a power analysis, targeted enrollment of 100 was planned for each site, however, one (not listed earlier in this paragraph) experienced enrollment problems and was withdrawn, and the minimal data collected were not included in the analyses. Consequently, one of the sites listed earlier was added to replace this loss and was able to randomize 61 before the study closeout date, yielding the final randomized sample of 461.

A common study protocol and informed-consent procedures were approved by the respective Institutional Review Boards affiliated with each university. A Data Safety and Monitoring Board convened by NIDA also approved the protocol and reviewed serious adverse events (n = 83, including two deaths, that were neither study related nor different between the conditions) for the duration of the protocol. Recruitment of participants occurred over a 37-month enrollment period from June 2001 to July 2004.

Participants

Patients

A total of 683 individuals were screened for interest and eligibility to participate in this study, and 222 were ineligible for randomization (see Figure 1, CONSORT diagram) with the most frequent reason being the absence of any self-reported substance use in the month prior to admission. All participants were current substance users, and most met current criteria for either substance abuse (6%) or dependence (88%) based on a structured interview, the Substance Dependence Severity Scale (Miele et al., 2000). Alcohol (62% abuse or dependence) and cocaine (60%) were the most common substance use disorder diagnoses followed by marijuana (27%), opiates (19%), and other drugs (11%).

Figure 1.

CONSORT diagram of eligibility, enrollment, randomization, treatment, and follow-up rates. CAU = counseling as usual; MET = motivational enhancement therapy.

The 461 eligible participants were randomly assigned to one of the two therapy conditions (MET: n = 216; CAU: n = 245) during their 1st month of treatment at each outpatient program site. Of these, 75 (16%) never attended their first session. Most participants who began the therapy completed all three protocol sessions (69%) while 15% completed two, and 16% completed only one session. There were no differences between conditions in the number of sessions completed, indicating a comparable level of exposure to the assigned therapy. Analysis of the total randomized sample (N = 461) and the subsamples who initiated (n = 386 received at least one session) and completed therapy (n = 267 received all three sessions) yielded comparable results, so only data for the full intent-to-treat analysis are reported here. Of these 461 randomized, 68% (81% of therapy initiators) completed the termination (4-week) assessment, 65% (77% of initiators) completed the 8-week follow-up, and 68% (82% of initiators) completed the 16-week follow-up. Most (81% of randomized; 88% of initiators) participants were assessed at least once posttreatment. Rates of follow-up varied significantly across the five program sites (e.g., range = 59%–88%), χ²(4, N = 386) = 23.9, p < .001. There were no significant differences between therapy conditions or Therapy Condition × Program Site interactions in the rates of follow-up or in the presence or frequency of missing data points.

Therapists

Volunteers were drawn from the staff of the participating treatment programs based on their willingness to be randomized to either a MET or CAU therapy condition and to have counseling sessions audiotaped. Randomization was used to ensure comparable levels of interest and commitment to the protocol and prior knowledge of motivational interviewing. When required by local Institutional Review Boards, written informed consent was obtained for therapists. As described in greater detail in an earlier report (Ball et al., 2002), most of the therapists had no prior training exposure to MI or MET, and almost none reported use of therapy manuals in practice. There were no therapy condition or program site main effects or interaction for clinician gender (63% female), race (75% Caucasian; 12% African American; 10% Hispanic; 3% other), age (M = 40.3, SD = 12.3, range = 22–63), years of counseling experience (M = 8.5, SD = 6.7, range = 0–20), years working at the program site (M = 3.8, SD = 4.4, range = 0–16), years of formal education (M = 15.0, SD = 4.9, range = 3–20), percentage with masters’ degrees (46%), percentage with professional credential (35% substance abuse certification, 19% licensed social workers or marriage and family therapists), or percentage in personal recovery (38%)

Treatment Procedures

Randomization

Following baseline assessment, participants were randomly assigned to one of two individual therapy conditions involving three sessions of either CAU or MET. The randomization used a computerized program that was managed by off-site personnel, but accessed locally by a research staff who communicated the assigned therapy condition. This program involved a process of urn allocation (Stout, Wirtz, Carbonari, & DelBoca, 1994) adapted from several previous multisite clinical trials (Marijuana Treatment Project, 2004; Project MATCH Research Group, 1997) to ensure balance within sites on gender, ethnicity, primary substance used, employment, and criminal justice status. Balance was achieved, but apparently at a cost of slightly disparate overall cell sizes (CAU = 245; MET = 216). After randomization, participants began individual sessions in their assigned condition and also began participating in the group counseling offered routinely at the outpatient program.

Provision of therapies

The three (45–55 min each) study therapy sessions in both conditions were delivered within a 28-day time window from the point of randomization. Missed appointments could be rescheduled within this time frame. Participants assigned to CAU received three sessions conducted as the regular individual counseling practiced within the outpatient program. CAU therapists collected information on substance use and psychosocial functioning, explained treatment program requirements, discussed the participant’s goals for treatment, provided early case management and substance abuse counseling, encouraged attendance at 12-step meetings, promoted abstinence, and emphasized follow through with treatment at the clinic. All therapists were expected to meet monthly with a supervisor to review treatment plans and progress. CAU sessions were audiotaped for later independent adherence and competence rating but were not reviewed for ongoing supervisory purposes. CAU therapists did not receive formal clinical supervision focused on the specific technical delivery of their individual counseling.

Participants assigned to MET received three individual sessions following a therapy manual (Farentinos & Obert, 2000) adapted for this study from several sources, principally Miller et al.’s (1992) Project MATCH manual (although our sessions were delivered in the 1st month of outpatient treatment rather than spaced over several months). A motivational interviewing style involved an empathic, client-centered, but directive approach designed to strengthen and consolidate commitment to change and promote a sense of self-efficacy (Miller & Rollnick, 1991, 2002). It attempted to elicit intrinsic motivation to change substance use by resolving client ambivalence, evoking self-motivational statements and commitment to change, and rolling with resistance. In addition to a therapeutic style incorporating “microskills” (open questions, affirmations, reflections, summary statements), three structured session exercises with client handouts were used as tools to facilitate the provision of personalized feedback, explore ambivalence or heighten discrepancies, and (when appropriate to the participant’s readiness) develop a change plan.

MET therapist training and supervision

MET therapist training followed a decentralized model in which each of the participating sites identified a local MET expert trainer with extensive training and supervision experience who attended a centralized training planning seminar conducted by Drs. William Miller and Theresa Moyers to standardize local procedures. Subsequent phone calls among expert trainers reviewed tape rating procedures across sites and conducted calibration exercises. The local MET expert trainers then provided a minimum of 16 hr of didactic training to the MET-assigned therapists and supervisors at their respective sites.

After training, each MET therapist conducted practice sessions with outpatients similar to those in the main phase of the study under close audiotape review and supervision by the local expert MET trainer. The goals of the sessions were for therapists to adapt their usual techniques to conform to manual guidelines, practice MET techniques, and reduce MET-Inconsistent strategies. Local expert trainers and supervisors reviewed and rated all practice sessions conducted by the MET therapists for initial certification using a structured tape rating system and manual. Initial certification was defined as at least adequate or average adherence and competence ratings on MET-Consistent techniques on three consecutive practice sessions. After certification, supervisors rated one tape each week on a random, rotating basis for the MET therapists and discussed feedback in biweekly supervision. If this ongoing tape review determined that a therapist drifted below the initial certification level, then additional supervision and training were provided with assignment of additional practice cases.

Assessment Procedures

Following the initial contact with the outpatient program, prospective participants met with a research assistant who explained the study, obtained written informed consent, and completed baseline assessments. During a 28-day therapy window, participants met briefly on three occasions with a research assistant to obtain self-report and biological (urine and breath) substance use measures. At the end of this 4-week study intervention period and again at an 8- and 16-week follow-up, all participants met with the research assistant to complete an assessment battery similar to what was completed at baseline. Primary participant outcome and secondary therapy process measures were collected using the following instruments.

Substance Use Calendar (SUC)

The SUC is an interview assessment of self-reported substance use (marijuana, cocaine, alcohol, methamphetamine, benzodiazepines, opioids, other drugs) completed at each contact by a research assistant. Adapted from the Time Line Follow-Back interview (Sobell & Sobell, 1992), which has been shown to be a reliable and valid instrument for monitoring substance use (Fals-Stewart, O’Farrell, Freitas, McFarlin, & Rutigliano, 2000), the SUC uses a similar calendar method to allow for a continuous and reliable evaluation of daily substance use (Carroll et al., 2004).

Urine drug analysis

OnTrak Testcups (5.01; Roche Diagnostics, Basel, Switzerland) were used at all sites for urine testing with the following cutoff scores: amphetamines (100 ng/ml), cocaine (300 ng/ml), methamphetamine (500 ng/ml), morphine (300 ng/ml), and THC (50 ng/ml). A separate kit (Teststik BNZ, Roche Diagnostics) was used for benzodiazepines (20 ng/ml cutoff). A total of 1,005 urine samples were collected during the 28-day therapy window. Of these, 274 samples tested positive for at least one drug (27%) with marijuana (126; 13%), cocaine (76; 8%), benzodiazepines (52; 5%), and opioids (48; 5%) being the most frequent. Of the intent-to-treat sample, 389 (84%) had at least one urine specimen taken, and 162 (42%) had at least one of these samples test positive for some form of drug use. The urine samples indicated adequate correspondence with participants’ self-reports of recent substance use with 14% indicating substance use when the participant denied recent use.

Treatment Utilization Form (TUF)

The TUF is an 11-item interview that was adapted and shortened from the Treatment Services Review (McLellan, Alterman, Cacciola, Metzger, & O’Brien, 1992) to measure the extent of additional program services (e.g., non-study-related individual, group, and family counseling; 12-step meetings; medical or vocational appointments) received either at the outpatient program site or by referral to another community service.

Therapy process assessment

All therapy sessions in both conditions were audiotaped, and MET sessions were reviewed and rated by local expert MET trainers and clinical supervisors to evaluate and promote therapy fidelity. In addition, a sample of both MET (n = 206) and CAU (n = 219) sessions was selected for independent review of adherence and competence so that most of the complete cases (i.e., all three sessions) were rated and each therapist and participant had at least one session rated (Session 1s = 161, Session 2s = 136, and Session 3s = 128). After the closeout of each site, 15 independent process raters were trained in the use of an adherence and competence rating system adapted from several reliable and valid instruments, especially the Yale Adherence and Competence Rating System (Carroll et al., 2000). This system guided the rating of 10 items for therapist behaviors within each of three hypothetical (and unidentified to the raters) technique categories: (a) MET-Consistent (e.g., reflections), (b) MET-Inconsistent (e.g., confrontation), and (c) Standard Counseling (e.g., case management). Each item was rated on a 7-point Likert-type scale on two dimensions: adherence (i.e., frequency and extensiveness; 1 = not present, 7 = extensively) and competence (i.e., skillfulness; 1 = very poor, 7 = excellent). An initial sample of 15 tapes rated by all raters indicated a high level of interrater reliability (mean intraclass correlation coefficients) across adherence and competence dimensions for all three scales: MET-Consistent (adherence = .89/competence = .81), MET-Inconsistent (.81/.82), and Standard Counseling (.96/.93).

Data Analysis

Primary outcome measures for Hypotheses 1 (MET vs. CAU effectiveness) and 2 (primary alcohol and primary drug use subgroup analyses) included (a) treatment retention (e.g., number of days in outpatient treatment program, percentage still enrolled in program 16 weeks after randomization) and (b) substance use (e.g., self-reported days per week of primary substance use for each of 16 study weeks, percentage of urines positive for drug use during the 4-week study therapy phase). The primary method of analyzing the two retention measures and the urine drug outcome measure were two-factor (two therapy conditions and five program sites) fixed-effect analysis of variance (ANOVA) or analysis of covariance with the intent-to-treat (n = 461) sample.

The self-report measure of substance use derived from the SUC measure permitted a longitudinal analysis of a days per week use of each participant’s primary substance from baseline through 16 continuous weekly data points. Because this analysis consisted of two discrete time phases (4-week period for active study therapy and 12-week period for follow-up assessment), forcing a single linear estimate through 17 data points was not appropriate because changes in slope should accompany transition between the two time phases. A piecewise hierarchical linear regression model best accommodated missing data while also testing differences in linear estimates between the two time phases (Singer & Willet, 2003). The effects of therapy condition (MET vs. CAU), program site (1–5), Weeks (0–16 weeks), and phase (0–4 therapy week vs. 5–16 follow-up week periods) and all possible interactions were tested for this outcome. All outcome analyses were conducted for the entire sample and then separately for those identified as primary alcohol and primary drug use subgroups to evaluate Hypothesis 2.

To understand the nature of site differences for Hypothesis 3 (e.g., better MET than CAU outcomes for sites with proportionately more alcohol users), we used chi-square and ANOVA analyses to evaluate baseline differences between therapy conditions and sites. On the basis of these analyses and site effects for Hypotheses 1 and 2, we conducted post hoc analyses with one atypical site removed and the number of non-study-related group counseling sessions (received at the program sites during the 28-day study therapy window) entered as a covariate at the program site level (i.e., for all main and interaction effects involving site). Secondary therapy process (adherence and competence) measures for Hypothesis 4 (i.e., MET > CAU therapist ratings for MET-Consistent; MET < CAU therapist ratings for MET-Inconsistent and Standard Care) were evaluated using Therapy Condition × Program Site ANOVAs.

To provide some control for multiple comparisons, we used a familywise alpha of p < .0125 (.05/4) for the four major participant outcomes and p < .008 (.05/6) for the six therapist process ratings. Cohen’s d or phi provided an estimate of effect size for therapy condition across and within each program site for the outcome variables, except the hierarchical linear model (HLM) analyses. Partial eta squared or theta was computed to permit comparison of the proportion of variance accounted for by therapy condition, program site, and therapist effects (within site).

Results

MET Versus CAU Effectiveness for Retention and Substance Use Outcomes

Program retention

Table 1 lists outcome variables by therapy condition and program site. There were no main effects for therapy condition or Therapy Condition × Program Site interactions for the two retention outcomes (days in treatment during study; percentage still enrolled at follow-up). Overall, participants in both therapy conditions were retained in their outpatient program for an average of about 2 months. Most participants (MET = 85%; CAU = 87%) were still enrolled at their program site at the 4-week therapy termination assessment, and these rates were approximately halved by the end of the 16-week follow-up period.

Table 1.

Primary Retention and Substance Use Outcomes by Therapy Condition and Program Site

	Site 1		Site 2		Site 3		Site 4		Site 5		Total
Outcome measure	MET	CAU	MET	CAU	MET	CAU	MET	CAU	MET	CAU	MET	CAU	Therapy condition	Program site	Therapy × Site
% positive urine drug tests (28 days)
M	0.14	0.18	0.14	0.27	0.54	0.44	0.17	0.44	0.35	0.32	0.21	0.28	F(1, 379) = 2.15, p = .144, ${\eta }_{\text{partial}}^2=.01$	F(4, 379) = 8.79, p < .001, ${\eta }_{\text{partial}}^2=.09$	F(4, 379) = 2.32, p < .057, ${\eta }_{\text{partial}}^2=.02$
SD	0.31	0.36	0.31	0.37	0.44	0.43	0.33	0.46	0.41	0.42	0.35	0.40
d		0.08		0.21		−0.23		0.59		−0.07
Days enrolled at treatment program
M	71.30	74.60	65.80	60.60	87.10	96.60	74.60	63.30	58.90	52.90	72.20	69.20	F(1, 376) = 0.22, p = .642, ${\eta }_{\text{partial}}^2=.001$	F(4, 376) = 9.56, p < .001, ${\eta }_{\text{partial}}^2<.09$	F(4, 376) = 0.78, p = .538, ${\eta }_{\text{partial}}^2=.01$
SD	42.30	40.90	38.90	30.40	45.00	38.40	38.10	44.10	44.70	42.90	43.10	42.10
d	−0.08			0.17		−0.25		0.26		0.14
% enrolled at program at 4-month follow-up
M	0.33	0.42	0.29	0.20	0.74	0.86	0.43	0.41	0.44	0.43	0.41	0.46	Wald(1) = 0.002, p = .970	Wald(4) = 19.69, p = .001	Wald(4) = 2.92, p = .570
ϕ		−.033		0.11		−0.14		0.02		0.01

Note. MET = motivational enhancement therapy; CAU = counseling as usual.

Substance use

There was no main effect for therapy condition for either outcome (days per week of self-reported primary substance use over 16 weeks; percentage of urines positive for drugs during 4-week therapy phase). Main effects for weeks (0–16), F(1, 4808) = 210.0, p < .001; phase (study therapy vs. follow-up period), F(1, 4818) = 175.7, p < .001; and the interaction of these two time effects, F(4, 4807) = 253.5, p < .001, indicated that participants demonstrated reductions in self-reported days per week of primary substance use from baseline over the 16 study weeks and between the two study phases.

However, there were significant interactions for Therapy Condition × Phase, F(1, 4819) = 11.3, p < .001, and Therapy × Weeks × Phase, F(1, 4807) = 8.22, p < .004, and a nonsignificant effect ( p = .06) for Therapy × Weeks. Figure 2 presents the significant three-way interaction to facilitate a description of all of these effects. Although participants in both conditions demonstrated significant reductions in self-reported days per week of primary substance use during the 4-week therapy phase, MET participants sustained these improvements for the subsequent 12 weeks of follow-up whereas CAU participants increased their use to baseline levels.

Figure 2.

Therapy Condition × Weeks × Phase interaction (see arrow) for days per week of primary substance use across therapy (Weeks 0–4) and follow-up (Weeks 5–16) phases. CAU = counseling as usual; MET = motivational enhancement therapy.

Site main effects and interactions

A program site main effect was significant for the percentage of positive urines outcome (see Table 1) during the 4-week therapy phase. Two sites (Sites 1 and 2) had lower rates of positive urines than the other sites, and one site (Site 3) had a higher percentage of positive urines. A program site main effect indicated that participants at this same program (Site 3) had a higher number of days enrolled in outpatient treatment than participants at the other sites, and one site (Site 5) had lower retention. Over three fourths of participants at Site 3 were still enrolled in their outpatient program at the end of the study in comparison with less than one half of participants at the other four sites.

With an adjusted alpha, a program site main effect was not significant ( p = .02) for the self-reported days per week of substance use measure. However, a Program × Weeks interaction, F(4, 4814) = 6.21, p < .001, indicated that all sites showed decreased substance use during the 4-week study therapy phase, although one site (Site 4) showed less change. In addition, one site (Site 3) showed continued decreases during the 12-week follow-up phase, whereas two sites (Sites 2 and 4) increased and two sites (Sites 1 and 5) showed no change during follow-up. A Program × Therapy × Phase effect, F(4, 4817) = 5.03, p < .001, was significant, and post hoc analyses revealed that Site 2 participants had greater reductions in substance use in MET than in CAU in which they exhibited significant increases in use from the therapy phase to the follow-up phase, t(4820) = 3.8, p < .01. In the other four sites, participants decreased substance use from the therapy to the follow-up phase in both therapy conditions, but greater decreases were evident in MET than in CAU.

Primary Alcohol and Drug Use Subgroup Effects

As shown in Table 2, alcohol was the most frequently reported primary substance problem (30% to 69% of the samples at sites when considered in combination with other drugs). The second most prevalent type of drug use varied widely and included marijuana, cocaine, and opiates. Separate retention analyses for primary alcohol and primary drug participants yielded no significant therapy condition effects. Separate piecewise HLMs for these two subgroups revealed a significant interaction for primary alcohol users for Therapy × Weeks, F(4, 1632) = 7.26, p < .01; Therapy × Phase, F(1, 1636) = 15.88, p < .001; and Therapy × Weeks × Phase, F(4, 1632) = 13.92, p < .001. These findings parallel those shown in Figure 2 for the whole sample and indicated superior outcomes emerging over the follow-up phase for MET over CAU for primary alcohol users only. There were no significant interactions between therapy and the two time indicators (study weeks or phases) for primary drug users. There were no program site main effects for either substance use group, but significant three-way and four-way interactions involving program site with therapy, weeks, and phase complicated the aforementioned findings.

Table 2.

Patient and Program Differences Across Sites

Site Correlates of Effective MET Versus CAU

Table 2 lists some of the significant baseline and within-treatment differences that were considered tentative explanations for previously described program site outcome differences (see Table 1) and the interaction of therapy with time indicators. Site 3 consisted of a higher percentage of primary marijuana users and had the highest percentage of positive urines during treatment but sustained reductions in substance use during the follow-up phase. This site also had unusually high retention rates, and participants were more commonly (75%) referred from the criminal justice system and, in many cases, formally mandated (57%) as an alternative to incarceration. Site 2 reported more pretreatment days of alcohol use with very low rates of legal coercion, lifetime arrests, and months incarcerated. Participants at this site demonstrated a more pronounced positive response to MET in comparison with CAU across the two study phases.

Site 3 also emerged as atypical compared with the other four sites on several indicators including the lower level of group counseling provided as regular (nonstudy) services at the program during the 4-week therapy phase. Site 3 therapists also delivered significantly shorter MET and CAU sessions for each of the three study sessions than the other four program sites (see Table 2). Despite the lower intensity of services, both MET and CAU at this site had approximately double the retention or enrollment rates at the 16-week follow-up relative to the other sites (see Table 1).

In addition to this program site finding for the number of additional group counseling sessions received, an unexpected difference was found between the therapy conditions. CAU participants received more groups (M = 8.6) than did MET participants (M = 7.3) within the 4-week study therapy phase, F(1,372) = 4.6, p < .05. These therapy and program differences in ancillary group sessions were potentially important confounds. This, combined with Site 3’s atypical characteristics (criminal justice, marijuana, very high retention), influenced our decision to conduct post hoc analyses of the four primary outcomes with Site 3 excluded and the number of group sessions treated as a covariate. Even with these site and group counseling controls, no therapy condition main effects emerged. Program site remained a significant predictor of percentage of urines positive during treatment and days in treatment outcomes. With regard to the days per week of substance use outcome, the site removal and group covariate controls did not alter the significance of the Therapy × Weeks or Therapy × Weeks × Phase interactions or the four-way interaction with program site. The effect of the group session covariate was a significant predictor of outcome for all four retention and substance use indicators.

MET Versus CAU Fidelity

Discriminability of MET versus CAU

Analysis of the independent rating of adherence and competence indicated that the two therapy conditions were discriminable in predicted directions (see Table 3). Therapists assigned to MET demonstrated significantly higher mean item scores on the adherence scale measuring MET-Consistent techniques than did those assigned to CAU. CAU therapists scored higher on adherence ratings for Standard Counseling as well as MET-Inconsistent techniques in comparison with MET therapists. Notably, MET-Inconsistent interventions occurred at very low frequencies in both therapy conditions. In addition to higher MET-Consistent adherence ratings, MET therapists were judged as delivering these interventions at a higher level of competence. Although MET therapists exhibited less frequent Standard Counseling behaviors, the delivery of these interventions was rated at a higher level of competence than when delivered by CAU therapists (see Table 3).

Table 3.

Motivational Enhancement Therapy (MET) and Counseling as Usual (CAU) Adherence and Competence for Three Groups of Therapist Techniques by Therapy Condition and Program Site

	Site 1		Site 2		Site 3		Site 4		Site 5
Scale	CAU	MET	CAU	MET	CAU	MET	CAU	MET	CAU	MET	Therapy condition	Program site	Therapy × Site
MET-Consistent
Adherence
M	2.17	3.54	2.19	3.65	2.31	3.29	2.58	3.89	2.24	3.23	F(1, 408) = 269.99, p < .001	F(4, 408) = 2.89, p < .001	F(4, 408) = 2.02, p = .092
SD	0.59	0.69	0.58	0.83	0.61	0.78	0.80	0.79	0.81	0.76
n	51	39	49	39	44	37	35	40	37	47
Competence
M	3.74	4.81	3.90	4.75	3.88	4.81	4.33	5.05	4.24	4.76	F(1, 375) = 128.54, p < .001	F(4, 375) = 4.32, p < .002	F(4, 375) = 1.61, p = .1715
SD	0.72	0.56	0.87	0.72	0.71	0.59	0.73	0.80	0.75	0.53
n	44	40	39	40	42	37	34	39	26	44
MET-Inconsistent
Adherence
M	1.60	1.17	1.62	1.27	1.57	1.17	1.51	1.05	1.42	1.27	F(1, 409) = 80.27, p < .001	F(4, 409) = 1.87, p = .114	F(4, 409) = 1.99, p = .095
SD	0.52	0.20	0.56	0.34	0.45	0.20	0.47	0.11	0.48	0.35
n	50	38	49	40	44	38	36	40	37	47
Competence
M	3.98	4.24	4.38	4.21	3.90	4.67	4.43	4.45	4.66	4.51	F(1, 281) = 2.41, p = .122	F(4, 281) = 3.32, p < .011	F(4, 281) = 3.90, p < .004
SD	0.81	0.56	0.87	0.77	0.82	0.73	0.60	0.86	0.57	0.60
n	43	22	40	26	40	21	30	10	25	34
Standard Counseling
Adherence
M	1.81	1.61	1.90	1.92	2.15	1.77	1.91	1.56	2.28	1.96	F(1, 408) = 23.22, p < .001	F(4, 408) = 8.68, p < .001	F(4, 408) = 2.04, p = .087
SD	0.60	0.41	0.37	0.58	0.50	0.35	0.42	0.36	0.77	0.68
n	48	39	49	40	44	38	36	40	37	47
Competence
M	3.99	4.41	4.18	4.41	4.08	4.67	4.46	4.67	4.44	4.52	F(1, 405) = 25.25, p < .001	F(4, 405) = 4.20, p < .002	F(4, 405) = 2.142, p = .075
SD	0.63	0.59	0.67	0.59	0.61	0.70	0.55	0.77	0.62	0.50
n	49	38	49	40	44	38	35	37	38	47

Comparability of MET and CAU delivery

There were no significant Program Site × Therapy Condition interactions for the three groups of techniques, indicating that MET and CAU therapist behaviors were consistently discriminable across the five sites. There were site main effects for MET-Consistent and Standard Counseling adherence and competence, but not for MET-Inconsistent scales. Site 5 therapists (regardless of assignment to CAU or MET) exhibited more Standard Counseling techniques than did therapists in these conditions at the other four sites. Site 1 therapists exhibited lower levels of both adherence and competence for Standard Counseling regardless of therapy condition assignment. Both MET and CAU therapists at Site 4 exhibited more adherent and competent MET-Consistent techniques than did the therapists in these two conditions at the other four sites (see Table 3).

Comparing Effects: Therapist Versus Therapy Versus Site

Multivariate analyses of variance (MANOVAs) and estimates (theta) of proportion of outcome variance accounted for by therapist effects (nested within therapy), therapy condition, and program site provided a final evaluation of the possible contribution of therapist effects to outcomes. The MANOVA included the two outcomes for which a Cohen’s d effect size estimate could be computed (percentage of positive urine drug tests; days enrolled in treatment program) and did not include the one dichotomous outcome measure (enrollment at 4 months). A separate analysis for the self-reported days per week of primary substance use outcome was conducted given that the piecewise HLM was not appropriate for inclusion in the MANOVA. Both analyses included participants with nonmissing values for the continuous outcomes tested and (to ensure balance for a test of therapist effects nested within therapy condition) only included the two MET and two CAU therapists from each site who treated the most participants. The MANOVA analysis for urines and days in treatment indicated that program site, F(4, 316) = 13.12, p < .001, and Program Site × Therapy Condition, F(4, 316) = 2.76, p < .03, were significant predictors and together accounted for 20% of the variance in outcome. Neither therapy condition nor therapist (nested within therapy) were significant predictors and accounted for negligible variance in outcome. With regard to the self-report days per week outcome, neither therapy condition nor therapist were significant predictors of outcome.

Discussion

This multisite randomized clinical trial evaluated the effectiveness of MET in comparison with CAU in five community treatment programs participating in the NIDA CTN. Our hypothesis that MET would be more effective than CAU was not supported for either retention measure or the urine drug outcome. However, a Therapy Condition × Weeks × Phase interaction indicated that the two brief interventions resulted in significant and comparable reductions in self-reported primary substance use during the 4-week active therapy phase, but that MET participants sustained these improvements over the 12 weeks after the brief intervention ended. In contrast, CAU participants experienced significantly increased substance use during this follow-up period, returning to their baseline (pretreatment) levels of self-reported use. For this one outcome, our primary hypothesis of superior MET effectiveness was supported. This effect could not be attributed to different patterns of treatment attendance or retention between MET relative to CAU or to the receipt of more group counseling in the outpatient program in the 1st month after admission when participants were in the active study therapy phase.

Our hypothesis that MET would be more effective than CAU for primary alcohol users also received support for the weekly substance use outcome. Primary drug users derived no benefit from MET relative to CAU for any outcome measure. These findings were consistent with several single-site studies involving drug abusers in community treatment settings. Donovan et al. (2001) randomly assigned 654 cocaine abusers on a waiting list for treatment either to receive a resource booklet and instructions to regularly call the waiting list number or to receive an attrition prevention intervention that included a single motivational enhancement session with follow-up phone calls. Both conditions did well with no differential effect on treatment entry, retention, or outcome. Miller et al. (2003) evaluated 208 opiate and cocaine abusers entering outpatient, methadone, or inpatient programs. In addition to the standard services provided, participants were randomly assigned to receive or not receive one additional motivational interviewing session. Both conditions demonstrated significant improvements without any added benefit from motivational interviewing. Rohsenow et al. (2004) found no differences between MET and a meditation–relaxation condition during the first 3 months of intensive outpatient treatment for cocaine dependence but at later follow-up assessments found better MET outcomes only for those with initially lower motivation to change. Together with these studies, our current study and previous single-session MI findings (Carroll et al., 2006) contribute to the developing literature indicating that MET may provide inconsistent added benefit to standard treatment improvement in substance use or retention among patients with drug dependence. Although prior meta-analytic reviews (Hettema et al., 2005; Noonan & Moyers, 1997) have reported comparable effect sizes for alcohol and drugs, the majority of supporting evidence for motivational interventions continues to be in alcohol rather than drug use samples.

The results based on our two substance use (self-report; urine drug) outcome measures did not correspond, and several methodological factors provide likely explanations. Whereas the urine drug indicator summarized dichotomous results over the 4-week therapy phase, the self-reported indicator provided a weekly, continuous measure over a 16-week period. An HLM on the four weekly dichotomous urine results was attempted but was not comparable to the self-report outcome, likely because of the variable detection windows for the different drugs tested. For example, marijuana was the most commonly used substance at one site, and several samples tested in the 1st month could have been positive given the drug’s longer detection window even though participants had stopped or reduced use. The disparity in effects between the self-report versus urine measures also may relate to the primary alcohol versus primary drug subgroup effects. Specifically, MET was associated with better reductions in use among primary alcohol users, which would not be detectable on urine tests, and there were no effects for primary drug users for either self-report or urine outcomes. Thus, the days per week of substance use measure was a more sensitive outcome indicator for the sample as a whole because it detected changes in use patterns of all substances over a more extended period of time.

Meta-analyses of motivational interventions (Burke et al., 2003; Dunn et al., 2001; Hettema et al., 2005; Noonan & Moyers, 1997) indicate that effect sizes across studies are highly variable, but generally positive, and that further research is needed to identify patient or program predictors of outcome. As Table 1 shows, the therapy condition effect sizes at each of the five sites were not only small but often inconsistent in direction. Program site accounted for more of the variance in these outcomes than therapy condition and suggested significant variability in the effectiveness of MET relative to CAU. A closer evaluation of participant and program differences suggested that one site was atypical and the amount of additional group counseling provided differed significantly at this program site and also between therapy conditions. Post hoc analyses controlled for these potential confounds but did not markedly change the findings or eliminate the significant site effects.

Our consistent program site main effects are a common occurrence in most multisite trials and are likely related to patient sample or treatment modality differences. However, Therapy × Site interactions can raise important questions for evidence-based treatments because they undercut claims of robustness by suggesting that efficacy is dependent on the context within which the therapy is implemented, including variations in its delivery or the condition(s) to which it is compared. Design recommendations for addressing Treatment × Site interactions have included either sampling a large enough number of sites (and therapists) so that data analyses can be conducted with adequate power to test for treatment differences in the context of a Treatment × Site interaction (as a random effect) or engaging in extensive efforts to standardize all treatment modalities and procedures so as to potentially minimize the likelihood of such interactions (Crits-Christoph, Tu, & Gallop, 2003). Both approaches have significant cost implications. Moreover, if the central comparison involves an experimental treatment versus treatment as usual, standardization of the latter condition may not be appropriate or feasible. Even with a high level of standardization, Project MATCH (Project MATCH Research Group, 1998) found that the relative efficacy of MET still varied significantly across therapists and sites despite extensive therapy training and fidelity monitoring procedures.

Of interest in this regard, meta-analytic reviews indicate that effect sizes tend to be smaller when motivational interventions are delivered in a manual-guided fashion with carefully defined supervision and adherence monitoring procedures (Hettema et al., 2005). Only one third of studies reviewed have used the kind of detailed manuals, posttraining certification, careful fidelity assessment, and supervisory monitoring involving recording and rating of sessions used by our study. It may be that overreliance on a therapy manual or overemphasis on the technical delivery of MET sessions contributes to an artificial structure that constrains the impact of a motivational interviewing style of interacting with individuals in a collaborative, spontaneous, and genuine manner. It also may be the case that studies that have not standardized treatment delivery through manuals, training, and supervision and instead used one or two highly skilled therapists achieved stronger outcomes that were more related to the allegiance and experience of particular therapists than the actual effectiveness of the intervention model.

The independent evaluation of therapy process based on audiotapes of MET and CAU sessions was a study strength and suggested that a decentralized training model resulted in comparably discriminable therapies across sites. Although we did not directly assess the efficacy of our training model, the results of this study and its companion single-session protocol (Carroll et al., 2006) indicated that community-based therapists can learn to deliver MET effectively even in the absence of a priori allegiance, interest, or training in the model. Future analyses will evaluate the relationship between therapist experience, skill, adherence, and treatment outcomes. Previous multisite clinical trials typically have selected and trained therapists for comparison conditions based on their experience with, allegiance to, or apparent “trainability” in one of the models (Baer et al., 2007; Carroll, Kadden, Donovan, Zweben, & Rounsaville, 1994; Crits-Christoph et al., 1997). Our sample of therapists appeared representative of the substance abuse treatment workforce that surveys (Gallon, Gabriel, & Knudsen, 2003; Lewin Group, 1999; Mulvey, Hubbard, & Hayashi, 2003) have characterized as predominantly White, middle-aged, somewhat more female, college educated, and with over 5 years working in the field or for their agency. However, we note that the intensity of the training provided to these clinicians far exceeded what is typical in community settings and involved 2 full days of intensive didactics and role-playing; criteria-related proficiency standards based on review of session audiotapes; and a consistent, structured local monitoring and supervision process consistent with recent studies evaluating strategies of training therapists in motivational interviewing (Baer et al., 2004; Miller, Yahne, Moyers, Martinez, & Pirritano, 2004). Future research should evaluate whether such intensive training and supervisory models improve patient outcomes and/or yield other benefits that can justify the cost and effort involved for community providers and funding entities.

The strengths of this multisite effectiveness study include its attention to site differences and effects, its recruitment of a diverse sample of outpatients, its delivery of therapies by a diverse group of therapists randomized to training condition, and its significant attention to therapy fidelity. Nonetheless, this study has several limitations. First, almost one third of the participants interested in the study were found ineligible because of a lack of recent self-reported substance use or pronounced psychosocial instability. Second, the results should not be generalized to other substance abuse patient groups (e.g., those who are severely mentally ill), practice settings (e.g., methadone, residential), or practitioners (e.g., therapists with a priori allegiance to MET). Third, although MET was adapted from the Project MATCH manual (Miller et al., 1992), we delivered three sessions in the 1st month of outpatient treatment rather than four sessions delivered on a monthly basis over several months. Fourth, the time spent in training was not balanced across conditions, and therapists assigned to MET received regular, observationally based supervision throughout the trial. Fifth, some contamination of therapy conditions likely occurred. Specifically, most staff counselors at each program site participated in this trial, and four of the five sites provided several groups per week throughout the 16-week study period. Thus, it was not feasible to prevent participants from receiving some group services from therapists and with participants from the contrasted individual therapy condition.

Finally, it must be emphasized that in every analysis, including those adjusting for the effect of potential site and service confounds, the effect of program site and the group session covariate were robust predictors of outcome. Because the group session covariate was measured after randomization, caution is warranted in interpreting the results of the statistical models including this covariate. The endogenous nature of this control (i.e., predictor collected in the context of the therapy conditions being compared) may bias the outcomes of these post hoc analyses. Nonetheless, variability in how standard outpatient treatment was provided in the five programs chosen for participation in this NIDA CTN study appeared to be a more robust predictor across outcomes than the therapy conditions compared or the therapists providing these interventions. Ongoing dissemination efforts should be informed by research that evaluates the critically important patient, provider, and practice setting parameters associated with the successful adoption of evidence-based therapies like MET, as well as the cost-effectiveness of intensive training and supervision procedures for these models relative to standard addiction counseling practice.