Motivational interviewing to improve treatment engagement and outcome in individuals seeking treatment for substance abuse: A multisite effectiveness study

Abstract

Despite recent emphasis on integrating empirically validated treatment into clinical practice, there are little data on whether manual-guided behavioral therapies can be implemented in standard clinical practice and whether incorporation of such techniques is associated with improved outcomes. The effectiveness of integrating motivational interviewing (MI) techniques into the initial contact and evaluation session was evaluated in a multisite randomized clinical trial. Participants were 423 substance users entering outpatient treatment in five community-based treatment settings, who were randomized to receive either the standard intake/evaluation session at each site or the same session in which MI techniques and strategies were integrated. Clinicians were drawn from the staff of the participating programs and were randomized either to learn and implement MI or to deliver the standard intake/evaluation session. Independent analyses of 315 session audiotapes suggested the two forms of treatment were highly discriminable and that clinicians trained to implement MI tended to have higher skill ratings. Regarding outcomes, for the sample as a whole, participants assigned to MI had significantly better retention through the 28-day follow-up than those assigned to the standard intervention. There were no significant effects of MI on substance use outcomes at either the 28-day or 84-day follow-up. Results suggest that community-based clinicians can effectively implement MI when provided training and supervision, and that integrating MI techniques in the earliest phases of treatment may have positive effects on retention early in the course of treatment.

1. Introduction

Motivational interviewing (MI), a treatment strategy developed to enhance motivation for change (Miller and Rollnick, 2002, 1991), has strong empirical support in trials with a number of substance-using populations, particularly problem drinkers (Miller and Wilbourne, 2002; Dunn et al., 2001; Burke et al., 2003; McCambridge and Strang, 2004). MI's burgeoning empirical base and its short-term nature, coupled with the pressures exerted by the treatment system and third party payors to reduce costs and improve client retention and treatment outcomes, have led to MI's being broadly applied in a range of substance abuse treatment settings.

However, there remain a number of important and largely unaddressed issues regarding MI's efficacy in non-research community settings and among diverse populations of substance users. First, although the bulk of studies evaluating MI with drug-using populations have suggested that MI is more effective than no treatment or comparison approaches (Burke et al., 2003; Dunn et al., 2001), several well-conducted studies evaluating MI with comparatively large samples of drug-using individuals have yielded few significant differences between MI and standard care comparison conditions (Miller et al., 2003; Donovan et al., 2001). Second, because the bulk of studies evaluating MI with drug users have evaluated the efficacy of adding an additional MI session to standard treatment, there are relatively few data on the effectiveness of MI under the conditions in which it is most likely to be applied in clinical practice, that is, integrating MI techniques into standard treatment approaches (Dunn et al., 2001).

Third, there are also comparatively little data on the effectiveness of MI in clinical practice and how best to disseminate MI to the clinical community. Only a handful of studies have evaluated the ability of ‘real world’ clinicians in community-based settings to learn and implement MI effectively. Rubel et al. (2000) reported on an uncontrolled evaluation of the impact of a 2-day clinical training workshop on 44 participants' knowledge and practice of MI. Participants' knowledge of MI (assessed through a 15-item multiple choice test) increased after attending the workshop, as did their articulation of statements reflecting techniques of MI in response to written vignettes. Subsequent training trials have suggested a single workshop may be associated with some change in clinician behavior, but these changes may not be substantial enough to strongly affect patient response (Miller and Mount, 2001) and that coaching and feedback appear to be essential for effective implementation of MI (Miller et al., 2004). Finally, comparatively few studies have addressed critical internal validity issues in the effectiveness of training, such as providing data from adherence monitoring regarding whether MI was implemented with adequate fidelity and skill by clinicians and whether MI is discriminable from standard treatment (Miller et al., 2003; Burke et al., 2003).

Institute of Medicine (1998) produced a report highlighting the gap between empirical knowledge and clinical practice in substance abuse treatment in the U.S. The IOM report called for the development of research–practice partnerships in community settings to improve the quality of drug abuse treatment and to broaden the base of knowledge on the effectiveness of empirically supported treatments when implemented in community settings. The National Institute on Drug Abuse's Clinical Trial Network (CTN), a network of 17 academic centers and over 100 community treatment programs in the U.S., was instituted in 1999 to address the research–practice gap. As one of the CTN's first protocols, a randomized clinical trial evaluating the effectiveness of MI in enhancing retention and substance use outcomes in community-based settings was developed.

This trial addressed the following research questions. First, to evaluate the effectiveness of integrating MI techniques and strategies into a single intake/evaluation session at participating community-based treatment programs, relative to standard intake/evaluation counseling, in: (1) enhancing treatment engagement and retention and (2) in reducing substance use. It was hypothesized that MI would be more effective than standard clinical practice in retaining patients through the initial month of treatment (operationalized as the proportion of participants still enrolled in the treatment program 28 days after randomization) and in reducing their substance use (operationalized as days of use of the participant's primary substance of abuse during the 28 days after randomization). Second, we hypothesized that MI would be more effective than treatment as usual in retaining patients in treatment and in reducing substance use through a 84-day follow-up. Important secondary aims of the trial included evaluation of: (1) the ability of clinicians at the participating clinics to learn and effectively implement MI, as assessed by independent adherence/competence ratings based on session audiotapes (e.g., could clinicians drawn from the staff of the participating clinics learn MI adequately and deliver it at an acceptable level of fidelity for the duration of the trial?); (2) whether MI would be discriminable from standard practice in the participating sites. That is, given the wide availability of MI manuals and other training materials, the extent to which MI techniques and styles were present in standard treatment at the participating sites was an important question.

2. Methods

2.1. Overview

A multisite, randomized clinical trial was conducted to compare the effect on retention and substance abuse outcomes of a standard/intake evaluation session for individuals seeking treatment at five community-based treatment settings versus the same standard/intake evaluation session in which MI techniques were integrated. The five participating sites (ADAPT, Inc. in Roseburg, OR; Changepoint, Inc., in Portland, OR; Chesterfield Substance Abuse Services in Chesterfield, VA; Lower Eastside Service Center in New York City; Willamette Family Treatment Services in Eugene, OR) were associated with the CTN and offered predominantly group-based treatment. Because MI has predominantly been evaluated as an individualized counseling approach and may not lend itself to a group format (Walters Ogle and Martin, 2002), the aim of the trial was to evaluate whether integrating MI as early as possible into the individual intake/assessment session that typically precedes patients' assignment to group treatment would enhance retention and substance abuse outcomes relative to standard intake/evaluation approaches. Planned length of treatment was variable in these sites and was determined by participant characteristics (e.g., severity of substance use and comorbid problems). Hence, because homogenizing the context/duration of treatment in which the study treatments took place would have restricted the variability in the patient sample, length of treatment varied across sites. The sites tended to offer weekly group sessions exclusively; treatment at one site (site 4) was somewhat more intense and included family sessions. Few or no individual sessions were offered as part of standard treatment at these sites.

An independent, but largely parallel study evaluating the effectiveness of a longer course (three sessions) of individual MI relative to standard treatment was developed for CTN programs that predominantly offered individual treatment. Because the design and rationale for these studies have been described in detail elsewhere (Carroll et al., 2002; Ball et al., 2002), they are reviewed only briefly below.

2.2. Participants

Participants were individuals seeking treatment for a substance use problem at the five participating programs. Because the intention of the trial was to evaluate the effectiveness of MI in as diverse and representative a sample as possible, minimal exclusions were placed on potential participants; thus, English-speaking individuals were eligible who were: (1) seeking outpatient treatment for any substance use problem and had used alcohol or any illicit drug at least once in the prior 28 days, (2) were 18 years of age or older, and (3) were willing to participate in the protocol (e.g., to be randomized to treatment, be contacted for follow-up assessment, and to have their session audiotaped). Individuals were excluded who: (1) were not sufficiently medically or psychiatrically stable to participate in outpatient treatment or who were highly unlikely to be reached for follow-up due to residential instability or imminent incarceration, or (2) were seeking detoxification only, methadone maintenance treatment, or residential inpatient treatment. Each site sought to recruit and randomize 100 participants.

2.3. Procedures and assessments

Following initial contact with the clinic, prospective participants met with a research assistant who explained the study and obtained written informed consent. The common study protocol, informed consent procedures, and the consent form were all approved by the corresponding Institutional Review Board of the academic center with which each community program was affiliated (the Oregon Health & Sciences University, Virginia Commonwealth University, Johns Hopkins School of Medicine, New York University, and the Yale University School of Medicine). A Data Safety and Monitoring Board (DSMB) convened by the National Institute on Drug Abuse also approved the protocol and reviewed study data and serious adverse events for the duration of the protocol.

The baseline research assessment battery took, on average, less than 1 h to complete, although the sites often required additional paperwork at the intake session to comply with state regulations and clinic policy. Following baseline assessment, participants were randomized to condition (MI or standard evaluation) using an urn randomization (Wei, 1978; Stout et al., 1994) program used in several previous multicenter clinical trials (MTP Research Group, 2004; Project MATCH Research Group, 1997). The urn program was used to balance participants within sites on gender, ethnicity, primary substance used, employment status, and whether the participant was mandated to treatment. After completing their single protocol session, participants were assigned, using normal clinic procedures, to standard treatment at the clinic (typically weekly group treatment). Follow-up interviews were conducted 28-day and 84-day post-randomization to evaluate the impact of the single-session intervention (MI or standard) on retention and frequency of substance use.

Because this protocol was conceived as a randomized effectiveness trial emphasizing generalizability, every effort was made to minimize impact of the research protocol on clinical practice at the sites, to not change the standard intake/evaluation procedures at the participating clinics, and to provide each participant with protocol session as rapidly as possible after the initial application for treatment. Thus, the assessment battery was designed to be as brief, to have high overlap with assessments already in place at the clinics, and to be completed in a single session. The battery included: (1) analyses of urine and breath samples, which were collected at all research assessment sessions (baseline, 28-day and 84-day follow-up). (2) Self-reports of substance use (marijuana, cocaine, alcohol, methamphetamines, opioids, benzodiazepenes, and other illicit drugs) were collected via the Substance Use Calendar, which uses the Timeline Followback method which has been shown to be reliable and valid for monitoring substance use and other outcomes in longitudinal studies (Miller and Delboca, 1994; Sobell and Sobell, 1992; Fals-Stewart et al., 2000); this instrument assessed all type of substance use (alcohol, marijuana, cocaine, opioids, benzodiazepenes, methamphetamines, and other drugs) on a daily basis and allowed a flexible, continuous evaluation of substance use. (3) A brief version of the Addiction Severity Index (ASI) (McLellan et al., 1992), a widely used clinical interview evaluating frequency and severity of substance use and related psychosocial problems, was administered at each assessment session (baseline, 28-day, and 84-day follow-up). The ASI has a high level of psychometric support (Alterman et al., 1994, 2001); the brief version used in the CTN eliminated some questions which were collected elsewhere in the assessment battery and/or not used in the calculation of composite scores. (4) The University of Rhode Island Change Assessment (URICA) (DiClemente and Hughes, 1990), a widely used self-report evaluating the individual's current position regarding readiness for change (e.g., precontemplation, contemplation, and maintenance) (Carey et al., 1999; Sobell et al., 1994), was collected at baseline and both follow-ups. (5) An abbreviated version of the Short Inventory of Problems (SIP-R) was used to assess the participants' perception of the adverse consequences of their substance use. The SIP-R was modified from the Drinker Inventory of Consequences (DrINC) (Miller et al., 1995) for use with drug users and its psychometric properties have been found to be acceptable in previous trials (Miller et al., 1995). (6) Baseline level of HIV risk behaviors and change in those behaviors through follow-up were assessed using the HIV Risk Behavior Scale (HRBS), a 12-item questionnaire developed by Darke and colleagues (Darke et al., 1991; Darke, 1998). Treatment retention data were collected by the research assistants based on self-reports and confirmed with client records; research assistants received extensive protocol-specific training, and research data were not shared with the clinics.

Analyses of urine and breath samples indicated high correspondence with participants' self-reports of their recent substance use. For example, of 1059 breathalyzer samples collected, only 3 had readings above .08. Of the urine specimens collected at the 28-day and 84-day follow-up, only 7.5% indicated recent drug use when the participant denied use of their primary drug within the past 3 days. This rate compares favorably with previous studies of substance-dependent samples which have supported the accuracy of self-report data using the methods described here (Zanis et al., 1994; Hersh et al., 1999; Ehrman and Robbins, 1994).

2.4. Interventions

To minimize the time between application for treatment and the protocol session, the assessment and intervention session was designed to take place on the same day. When this was not possible, the intervention session was required to occur within 1 week of randomization. Across sites, the mean number of days elapsed from randomization to the session was 2.0 (S.D. = 3.4); the median number of days was 0. All protocol sessions (standard and MI) were audiotaped for process assessment.

2.4.1. Standard intake/evaluation session

Participants assigned to this condition received an approximately 2-h assessment/evaluation session during which the clinician collected standard information according to their agency guidelines. This typically included collecting information on the participant's history and current level of substance use, treatment history, and psychosocial functioning; the clinician then provided an orientation to the clinic. Following this single protocol session, the participant was referred to standard group treatment at each site. In some cases, groups were led by the clinician who provided the protocol session but in most cases were led by other staff at the clinic.

2.4.2. Motivational interviewing intake session

Individuals assigned to this condition participated in an approximately 2-h assessment/evaluation session within which the therapist conducted the same intake/orientation session as described above, but did so in a manner that incorporated MI strategies (e.g., practicing empathy, providing choice, removing barriers, providing feedback, and clarifying goals) and that used an MI interviewing style (e.g., asking open-ended questions, listening reflectively, affirming change-related participant statements and efforts, eliciting self-motivational statements with directive methods, and handling resistance without direct confrontation). A detailed manual was developed for this protocol that drew from existing MI manuals and guides (Miller, 1999; Miller and Rollnick, 1991, 2002; Miller et al., 1992) and adapted them to be used in the single-session format and which anticipated a participant sample with a wide range of substance use problems.

2.5. Clinicians and training

All clinicians were volunteers drawn from the staff of the participating treatment programs; in several sites this involved the entire full-time clinic staff. To assure that both MI and standard treatment were delivered by clinicians of comparable levels of interest and commitment to MI and the protocol, clinicians were randomized to deliver either MI or standard evaluation (the clinician/volunteers also provided written informed consent for participation if required by the local Institutional Review Board). Prior to randomization, the clinicians completed a brief pretraining battery that included information on clinician demographics and experience, counseling orientation, and an inventory of clinical techniques they used most frequently. The 37 participating clinicians were predominantly female (68%), Caucasian (81%), and had a mean age of 42 years (S.D. = 9.8). Twenty (54%) had masters degrees, five had bachelors degrees (13.5%), and the remainder had associates or high school degrees. Twenty-two (60%) had received state certification as a substance abuse counselor. The clinicians had been employed at their agency for an average of 4.8 years (S.D. = 4.7), and averaged 7.2 (S.D. = 5.2) years of counseling experience. As described in an earlier report (Ball et al., 2002), most of the clinicians had no prior exposure to MI and almost none reported that they used an MI manual in practice.

Clinician training followed a decentralized model that was intended to provide a high and consistent level of training and ongoing supervision as well as to provide resources that would enable the sites to continue to deliver MI after the trial ended. Thus, an MI expert trainer was identified for each site, who was required to have completed a previous MI trainer's workshop and to have had extensive experience in training and supervising clinicians in MI. The MI expert trainers all attended a centralized initial training/planning seminar (“training of trainers”) conducted by Drs. William Miller and Theresa Moyers, which was intended to standardize training, supervision, and tape rating procedures across sites. The MI expert trainers provided a minimum of 16 h of didactic training to the participating MI clinicians and supervisors at their respective sites. Didactic training followed a standardized format that included review of MI principles and practices, use of training videotapes and role-playing to develop skills, and discussion of implementation issues specific to the MI protocol. In addition, each site identified a clinical supervisor, who was an employee of the site in a clinical leadership role. The local supervisors received additional training in MI and assessment of clinician adherence and skill in delivering MI.

Following didactic training, site supervisors and clinicians were required to successfully complete a minimum of three training cases of MI. All training cases were audiotaped and supervised by both the MI expert trainer (by phone) and the site supervisor (in person), who reviewed the audiotapes and rated the tapes using the adherence/competence rating system described below. The training cases provided an opportunity for each clinician to practice MI under highly supportive conditions with close supervision. Clinicians who were judged as adhering adequately to the MI manual were certified and permitted to begin to treat trial participants. Clinicians who did not meet this minimal threshold (see below) were assigned additional training cases, with written guidelines suggesting specific areas for working more closely within manual guidelines, until they met certification standards.

All session tapes (155 MI, 160 standard treatment) were reviewed and rated by 15 independent process raters, using a validated adherence/competence rating system (Carroll et al., 2000, 1998) which evaluated three types of interventions: 9 items assessed the therapists use of MI techniques and strategies (e.g., use of an MI interviewing style, asking open ended questions, and listening reflectively), 5 items assessed standard drug counseling strategies (e.g., providing program orientation, assessing substance use, and treatment planning), and 9 items assessed interventions seen as antithetical to MI (e.g., providing direct advice, and emphasize therapist authority in decision making). Each item was rated on a 7-point Likert-type scale along two dimensions: frequency (adherence) (1: not present in the session to 7: extensively) and skill (1: very poor to 7: excellent). A reliability sample of 16 tapes that were evaluated by all 15 raters indicated a high level of interrater reliability across adherence and competence dimensions for all three scales; Shrout and Fleiss' (1979) model for random effects indicated a mean ICC estimate for the adherence dimension of .89, .85, and .96 for the MI, non-MI, and general counseling scales, respectively. Estimates for the skill dimension were .81, .82, and .94 for the MI, non-MI, and general counseling scales, respectively.

2.6. Data analysis

Chi-square and ANOVA analyses were used to evaluate baseline differences in participant characteristics between intervention conditions and sites. Outcome measures were evaluated by treatment condition using two models. The first model, mixed effects ANOVA, was used to evaluate those primary outcome measures that were measured only once (e.g., length of time in treatment and total days of drug use in the first 28 days). Given that this study was designed as effectiveness research and hence with an emphasis on generalizability of outcomes to clinical practice, the data were modeled with intervention condition as a fixed effect and site as a random effect. This mixed effect ANOVA allowed for differences in drug use patterns and types at the five sites, as well as expected variations in the ‘standard treatment’ across the different sites by including the variance at the site level within in the model. Dichotomous outcome measures (e.g., whether the participant was still enrolled at the clinic 28 days after randomization) were modeled utilizing a similar hierarchical structure accounting for participants nested within sites as a Bernoulli model with LaPlace iterations to provide a normalizing transformation. A natural log transformation was used to obtain more normal distributions for the continuous variables. The second model, repeated measures ANOVA, was used to evaluate those secondary outcome measures that were assessed at baseline, 28-day, and 84-day follow-up (e.g., ASI composite score and HRBS scores); these analyses were done for the full sample as well as separately for each site. One site (LESC) stopped study enrollment early in the recruitment process due to the events of September 11, 2001, having randomized 23 individuals. Data from this site were included in the intention to treat analyses, but excluded for process analyses and for those analyses in which site effects were modeled. Data analyses were also conducted on the sample of 377 participants who were exposed to a protocol session; these findings are consistent with the intent-to-treat analyses.

3. Results

3.1. Participant characteristics

Across the five sites, a total of 640 individuals were screened; of these, 423 were determined to be eligible for the protocol and provided informed consent. The primary reasons for ineligibility were no substance use in the last 28 days (n = 95, 51.9%), seeking detoxification, inpatient treatment or methadone maintenance (n = 34, 18.6%), lack of sufficient housing to participate in outpatient treatment (n = 15, 8.2%), moving or going to jail within 60 days (n = 12, 6.6%), insufficient psychiatric stability for outpatient treatment (n = 11, 6%), not willing to be randomized for treatment or be reached for follow-up (n = 5, 2.7%), not interested in participating (n = 5, 2.7%), less than 18 years of age (n = 3, 1.6%), did not speak English (n = 2, 1.1%), or previously participated in the study (n = 1, 0.5%). Thirty-four individuals were screened but dropped out during the evaluation process.

A total of 423 participants were randomized to treatment condition (198 to MI, 202 to standard intake/evaluation for the four sites who reached 100). Baseline characteristics by site are presented in Table 1. Although randomization was successful in that there were few significant differences between conditions within sites, there were several statistically significant differences in participant characteristics across sites, including gender (the proportion of female participants ranged from 10% to 67% across sites), education (mean years of education ranged from 11.8 to 12.8), legal system involvement in treatment seeking (the proportion of participants with legal problems that prompted or mandated treatment seeking ranged from .31 to .91 across the sites), and primary reported substance use problem. Regarding the latter, although alcohol was the most frequent primary substance abuse problem reported across the sites (ranging from 30% to 60% of participants), for each site the second more prevalent type of drug use varied widely; these included marijuana, cocaine, and methamphetamines. Across sites, 38% of the participants had had previous alcohol and 47% had had previous drug abuse treatment.

Table 1.

Baseline demographic characteristics and substance use variables by site

Variable (percent or mean (S.D.))	Site 1	Site 2	Site 3	Site 4	Site 5	Total	X2 or F
Female	42.9	28.4	42.6	66.7	10.3	43.2	48.03**
Ethnicity, European American	80	81	72	72	0	71.6	259.4**
African American	0	6.4	20.4	0	62.1	9.8
Latino	.8	2.3	0	0	31	2.7
Multiethnic	17.6	7.3	5.6	26.3	3.4	13.8
Other	1.7	2.8	1.9	1.8	3.4	2.1
Employed full or part time	34	54	43	20	26.1	36.9	28.54**
Married or cohabitating	74.8	81.7	77.8	85.1	93.1	80.6	7.55
Admission prompted by legal system	57	70	46	31	91.3	53.2	47.2**
On probation or parole	35	39	30	36	82.6	37.6	22.8**
Any previous drug/alcohol treatment	60	50	62	69	87	61.7	14.38**
Principal drug used
Alcohol	59.5	60.6	57.9	29.8	27.6	50.3	141.7**
Cocaine	.9	3.7	13.1	4.4	13.8	5.9
Marijuana	14.7	22	22.4	14	51.7	20.2
Opiates	4.3	4.6	4.7	5.3	6.9	4.8
Methamphetamines	20.7	8.3	0	46.5	0	18.1
Benzodiazepenes	0	.9	1.9	0	0	.6
Age	32.8 (10.4)	34.0 (11.2)	32.5 (9.1)	31.2 (9.0)	37.2 (8.8)	32.8 (9.9)	2.6*
Years of education	11.8 (1.8)	12.8 (2.1)	12.4 (1.9)	11.8 (1.6)	11.6 (1.6)	12.2 (1.9)	7.1**
Days of substance use, past 30	8.3 (10.3)	7.9 (8.5)	13.1 (10.4)	10.3 (9.0)	8.8 (10.0)	9.8 (9.8)	5.1**
ASI composite scores, medical	.30 (.37)	.25 (.34)	.17 (.31)	.36 (.34)	.38 (.38)	.27 (.34)	4.286**
Employment	.76 (.25)	.66 (.30)	.60 (.27)	.84 (.23)	.11 (.11)	.72 (.28)	16.73**
Alcohol	.20 (.22)	.17 (.18)	.31 (.28)	.20 (.25)	.09 (.09)	.21 (.23)	7.62**
Drug	.10 (.10)	.08 (.11)	.14 (.11)	.16 (.12)	.07 (.07).	.11 (.12)	8.07**
Legal	.24 (.21)	.12 (.17)	.18 (.20)	.22 (.24)	.16 (.16)	.19 (.21)	6.764**
Family	.14 (.21)	.15 (.21)	.18 (.23)	.26 (.25)	.20 (.20)	.18 (.22)	4.29**
Psychological	.21 (.23)	.23 (.22)	.30 (.25)	.40 (.21)	.15 (.15)	.27 (.22)	14.18**

Note: n's for Sites 1–4 = 100, n for Site 5 = 23. Individual sites are not identified to protect anonymity of participants and clinicians.

^*p < .05.

^**p < .01.

Overall, of the 423 randomized participants, 377 (89%) completed their protocol session, 323 (76%) completed the 1-month (28 day) follow-up and provided a urine or breath specimen (81% of those who completed their protocol session), and 307 (73%) completed the 3-month (84 day) follow-up (77% of those who completed their protocol session). Three hundred and forty-seven participants (82%) were interviewed at least once. Rates of follow-up did not differ by condition within sites, but did vary across sites (completion rates for the 84-day follow-up across the four sites that randomized 100 participants were 65%, 81%, 81%, and 69%).

3.2. Treatment implementation, fidelity, and skill

Session audiotapes were available from 315 of the 377 sessions delivered (59 sessions were either not taped, inaudible, or taped incorrectly). All 315 audiotapes were rated by the independent evaluators to evaluate: (1) the degree to which MI was implemented as intended and could be discriminated from the standard intervention and (2) the level of variation in intervention delivery across sites and therapies, for both the MI and standard intervention conditions. As shown in Table 2, there were consistent, sharp differences across the two conditions, in the expected directions, in ratings of the frequency with which interventions and strategies associated with MI were present in the sessions (MI mean = 3.8, standard mean = 2.2), with statistically significant differences in all sites (NB: site identities are masked). For those sessions in which at least one MI strategy or technique was rated as present (100% of all MI sessions, 44% of all standard sessions), clinicians delivering MI were rated as significantly more skillful in delivering MI interventions (MI mean = 4.6, standard mean = 3.4), with statistically significant site effects as well.

Table 2.

Treatment adherence and skill levels by condition and site

Note: Scores range from 1 to 7. For frequency ratings (adherence), 1 indicates ‘not done at all’, 4 indicates ‘moderately’, and 7 indicates ‘extensively’. For skill ratings, 1 indicates ‘poor’, 4 indicates ‘average’, and 7 indicates ‘excellent’. For adherence ratings, n = 315. Because skill ratings are not done if an intervention did not occur, n's for MET, non-MET, and general skill scores are 309, 226, and 314, respectively.

^*p < .05.

^**p < .01.

As expected, the items tapping interventions associated with general counseling activities were not significantly different by condition (MI mean = 4.2, standard mean = 4.5). Again, clinicians delivering MI were rated as delivering these significantly more skillfully (MI mean = 4.6, standard mean = 4.3). As shown in Table 2, interventions which were antithetical to MI were rarely seen in either condition, as very low mean scores were seen on this scale. MI therapists were, however, rated as using these interventions significantly less frequently than standard treatment therapists (MI mean = 1.4, standard treatment mean = 1.5), but significant differences in the skill level with which these were implemented did not differ by condition.

Although there were statistically significant differences in MI frequency and skill ratings across conditions, there were also significant site effects for most of these dimensions that were likely to reflect variability in the nature of the interventions typically delivered at these sites. To put these differences into context, a multivariate ANOVA analysis (Harris, 1985) of the adherence/frequency ratings from all three scales (MI, non-MI, and general) simultaneously suggested significant effects for condition (F(3,305) = 112.30, p = .00) and site within condition (F(18,921) = 6.11, p = .01). However, the theta values, which provide an estimate in the amount of variance accounted for by each of these effects, suggested condition (θ = .52) accounted for substantially more variance in adherence scores than did site within condition (θ = .23). Similarly, although there were significant effects of both condition and site within condition for the skill scores, the theta values suggested most of the variance in skill scores was associated with condition (θ = .30), rather than condition within group (θ = .10). A similar analysis evaluated the magnitude of therapist effects, and suggested that 47% of variance in the tape rating adherence scores were associated with intervention condition, and only 8% attributable to therapists overall.

3.3. One-month outcomes: retention and substance abuse

Primary outcome variables (retention in treatment and frequency of substance use), by treatment condition and site, are presented in Table 3. As noted above, two approaches were used to evaluate effects of the study treatment on the continuous measure of retention (number of treatment sessions completed). The mixed effect ANOVA model, with the effect of site nested within treatment conditions, evaluated condition effects in the context of variability across the participating sites. This model indicated that across the five sites, participants assigned to MI completed significantly more sessions in the 28 days after randomization than those assigned to standard treatment (mean 5.0 versus 4.0, F(1,334) = 3.8, p = .05). The effect size, expressed as Cohens d, was .24. When each of the sites was evaluated separately, retention was higher in MI than the standard intervention in three of the four sites. Using the dichotomous measure, participants assigned to MI were significantly more likely to be enrolled in treatment at the clinic 28 days after randomization than those assigned to the standard evaluation (84% for MI versus 75% for standard, X²(1) = 3.5, p = .05). In the cases where there was some delay in providing the protocol session, results were similar.

Table 3.

Retention and substance use by site and condition

Variable	Site 1				Site 2				Site 3				Site 4				Site 5
	MI, n = 44		Standard, n = 45		MI, n = 41		Standard, n = 51		MI, n = 37		Standard, n = 42		MI, n = 43		Standard, n = 33		MI, n = 8		Standard, n = 7
	M	S.D.	M	S.D.	M	S.D.	M	S.D.	M	S.D.	M	S.D.	M	S.D.	M	S.D.	M	S.D.	M	S.D.
Twenty-eight day outcomes
Percent retained at site	95.5		84.4		90.2		86.3		62.2		57.1		83.7		69.7		100		100
Number of sessions completed	3.13	3	3.77	3.22	3.46	4	2.43	2.75	3.46	3.43	3.02	3	9.8	6.14	8.12	5.78	5	4.31	4.57	2.37
Days of use of primary substance	3	7.03	2.71	7.07	3.2	7.47	5	8.18	6.24	8.8	5	8.18	1.3	2.33	2.87	4.81
Eighty-four day outcomes
Percent retained at site	67.4		61.9		75.7		66.7		48.4		44.4		58.5		61.3		100		100
Number of sessions completed	8.25	7	8.14	6.68	8.02	6.42	6.33	4.62	3.87	4.53	5.02	6	20.12	15.6	18.38	16.5	2	4	15.1	12.5
Days of use of primary substance	4	10.9	7.84	17.4	5.62	13.4	4.12	7.31	13.4	17.2	12.4	17.3	5.3	9.8	3.16	6.37

The primary outcome measure for evaluating the effects of the study conditions on substance use was the total number of days on which the participant reported using his or her identified primary substance of use in the 28 days following randomization. Both the ANOVA model and the mixed effects model indicated no significant effect of condition on days of substance use for the sample as a whole (ANOVA: F(1,334) = .10, p = .75; mixed effects: F(1,328) = 0.15, p = .70). When sites were evaluated separately, MI was associated with fewer days of substance use in three of the four sites, but these effects were not statistically significant.

3.4. Three-month outcomes: retention and substance abuse

At the 84-day follow-up, retention in treatment remained high overall. Participants assigned to the standard evaluation had completed a mean of 13.2 (S.D. = 13.0) sessions with a mean of 56.5 days of treatment (S.D. = 31.2) and those assigned to MI had completed a mean of 15.2 sessions (S.D. = 14.6) sessions and a mean of 60.7 days of treatment (S.D. = 32.7). However, these differences were not statistically significant, using either model. Overall, 96 (61.5%) of those assigned to MI and 91 (56%) of those assigned to the standard evaluation were still enrolled in the clinic at the 84-day follow-up (X²(1) = 1.1, p = .3). There were no significant differences between groups on substance use outcomes at the 84-day follow-up (ANOVA: F(1,291) = .97, p = .33; mixed effects: F(1,288) = .05, p = .83).

3.5. Subgroup analyses: alcohol users

One advantage of large multisite trials is that they allow some analyses of outcome within specific populations of interest. Given that MI was initially developed and validated as an intervention for alcohol use disorders, and that recent studies suggest that MI may be more effective among alcohol, rather than drug-using, populations (Miller et al., 2003), additional exploratory analyses were conducted to evaluate outcomes for the large subpopulation whose principal substance used was alcohol (n = 177). For this subgroup, those assigned to MI completed significantly more sessions in the 28 days following randomization compared with those assigned to the standard evaluation session (MI mean = 5.1 sessions (S.D. = 5.1), standard mean = 3.3 (S.D. = 3.2)), for both models (ANOVA: F(1,175) = 8.1, p = .01, d = .56; mixed effects: F(1,164) = 10.33, p = .002). The positive effect of MI on treatment retention was also significant at the 84-day follow-up (F(1,154) = 3.79, p = .05, d = .32). Regarding the substance use outcome (i.e., frequency of alcohol use in the 28 days following randomization), the standard ANOVA model including participants from all sites did not suggest significant intervention effects overall (F(1,107) = .6, p = .44). However, the mixed effects model suggested that participants assigned to MI used alcohol less frequently than those assigned to standard treatment (F(1,164) = 3.07, p = .06).

3.6. Secondary outcome measures

The ASI, HRBS, and URICA were included as measures of change in psychosocial problems, HIV risk behaviors, and intention to change, respectively. For the ASI composite scores, repeated measures ANOVA for the aggregate sample indicated significant reductions in intensity of problems in all seven areas (medical, legal, employment, alcohol, drug, family, and psychological) over time, for both the 28-day and 84-day assessment points. However, there were no significant effects of intervention or intervention by time. For the HRBS, there were significant reductions in both the drug-risk and sex-risk subscales at the 28-day and 84-day assessment points, but no significant effects of condition or condition by time. Finally, for the URICA, there were no significant effects of time, condition group, or condition by time at the 28-day follow-up for the precontemplation, contemplation, action, or maintenance scores. At the 84-day follow-up, there were significant effects of time only for the contemplation scale, indicating a significant decrease in contemplation scores for participants overall.

4. Discussion

This multisite randomized clinical trial evaluating the effectiveness of incorporating motivational interviewing techniques into the initial intake/orientation session in community treatment programs suggested the following: first, although treatment retention was comparatively high overall, participants assigned to MI were significantly more likely to still be enrolled in the program one month after randomization. This effect was seen across sites and was consistent with other indicators of retention, such as number of sessions completed. At the terminal follow-up, although retention in the clinics remained fairly high, and retention for those participants assigned to MI remained higher than for those participants assigned to the standard intervention, the difference was no longer statistically significant. Second, regarding substance use outcomes, for the group as a whole, there were significant reductions in frequency of substance use across time, but no significant differences by intervention condition. Among the secondary outcome measures (ASI composite scores, HIV risk behaviors), sustained reductions in these problem areas were seen, but there were no significant effects of intervention condition.

Data from this trial, which was among the first to evaluate the effect of implementing evidence-based therapies in ‘real world’ clinical settings and which randomized clinicians drawn from the staff of those settings to intervention condition, also suggest that the trial was implemented with acceptable internal validity (Carroll and Rounsaville, 2003). Analyses based on independent ratings of the session audiotapes suggested that, across sites, MI and the standard intervention were highly discriminable and thus the major aims of the trial were met and internal validity was protected, even in the context of comparatively high levels of variability in participant characteristics and the nature of the standard intervention across the sites. Second, both types of intervention were delivered comparatively consistently and skillfully, with larger proportions of the variability in treatment delivery accounted for by intervention condition, rather than by site or therapists. While the efficacy of the training model used in this trial was not assessed directly, these results do suggest that community-based clinicians can learn to deliver MI effectively, at least when required to demonstrate proficiency in implementing MI based on review of session tapes and provided with consistent, structured local monitoring and supervision. These findings are thus consistent with other recent studies evaluating strategies of training therapists in MI (Miller et al., 2004; Baer et al., 2004).

These data, suggesting that integrating MI techniques into only a single initial evaluation session was associated with positive effects on early retention in treatment, are nevertheless striking in that it was seen in the context of an effectiveness trial with a comparatively high level of variability across samples and site characteristics. It may be of some clinical significance, given consistent relationships between retention and outcome in drug abuse treatment. Although the effect of the single session of MI on retention was not statistically significant through the 84-day follow-up, it should also be noted that MI was delivered prior to comparatively intensive group and day treatment programs that may have diluted any intervention effect. Moreover, the beneficial initial effect of MI on retention occurred in the context of very good overall retention and outcome at the participating sites (which may in turn reflect selection effects among sites willing to participate in the CTN and in this protocol, clinicians willing to be randomized to training condition and to have their work audiotaped).

While MI was not associated with reduced substance use for the full study sample, there were some indications that it was most effective in enhancing retention for those who reported that alcohol was the primary substance they used. Although effectiveness research requires evaluation of treatment effects in heterogeneous samples of substance users, patterns of use may vary within types of drug use (e.g., alcohol, stimulants, and marijuana), and it may be difficult to detect change when there is a high level of variance within and across users of different types of substances (Rounsaville et al., 2003). Users of different types of substances may respond differently to different approaches; findings such as these imply it is important to understand the types of individuals for whom MI is effective, what mediators and moderators impact the process, including level of fidelity and skill to MI principles. These results are also consistent with recent effectiveness studies that suggest that empirical data for evidence-based practice are not universally positive and even interventions with strong empirical support may have weaker effects when evaluated in the context of the greater variability of community-based settings (Morgenstern et al., 2001; Miller et al., 2003). The impact on practice of evidence-based therapies should be evaluated carefully, and in a range of settings and populations.

This was, to our knowledge, the first behavioral therapy study in which clinicians were drawn entirely from the staff of community-based programs and which randomized them to intervention conditions to control for effects of clinician motivation, experience, and willingness to learn a new approach. The long-term impact of providing training and supervision in MI at the participating sites will be the subject of future reports; it was clear, however, that the clinicians approached the study with considerable enthusiasm and saw participation as a means of broadening their own skills and outcomes for the individuals with whom they worked. Moreover, in contrast to reports of high levels of turnover in community-based treatment settings (McLellan et al., 2003), turnover of supervisors and clinicians participating in the MI arm of the protocol was infrequent, suggesting that provision of training and supervision, at least in the context of a research protocol, may play a role in decreasing staff turnover. Future reports will address the relationship of therapist skill and treatment fidelity to outcome, specific participant characteristics (e.g., gender, referral by the criminal justice system), as well as the results of the independent, parallel trial evaluating the three-session individual treatment.

Several limitations of this study, many of which reflect its emphasis on effectiveness and its community-based context, should be noted. For example, because this was an effectiveness study, it was not feasible to monitor substance use via urine and breath specimen samples more frequently than at the major assessment points (baseline, 28-day, and 84-day follow-up) and thus substance use outcomes are based primarily on self-report. However, multiple methods previously demonstrated to enhance the validity of self-reports in clinical trials with substance-using populations were used (Brown et al., 1992; Babor et al., 2000; Darke, 1998; Maisto et al., 1990; Zanis et al., 1994), including assurances to participants that their self-reports were confidential, independence of clinical program versus research assessments, use of the Timeline Followback method, and confirmation of participant self-reports with results of urines and breath specimens. In addition, as an effectiveness study, the time spent in training was not balanced across conditions, and clinicians assigned to MI received more training and supervision throughout the trial by design. While some site effects were seen in this study, there was considerable consistency in findings across sites that included wide variations in client mix and severity, clinician characteristics, and procedures. Nevertheless, conclusions about intervention effects would have been somewhat different if findings were based on any single participating site, underlining the need for caution in interpreting the results of any single site study, even effectiveness studies (Carroll and Rounsaville, 2003; Klein and Smith, 1999; Beutler and Howard, 1998).

The strengths of this trial include its large, diverse sample, and its multisite nature, and that all treatments were conducted by clinicians drawn from the staff of the participating sites and who were randomized to intervention. In addition, minimal exclusions on study participants were intended to result in a diverse sample composed of ‘all comers’ to these sites, with follow-up rates that approached 80% in a study that had few barriers to participation. Finally, assessment of intervention discriminability and therapist skill by independent raters based on audiotapes of both MI and standard interventions sessions suggested that participating clinicians were able to implement MI at a high and consistent level, and that interventions consistent with MI were implemented comparatively infrequently in the standard intervention condition. This study thus suggests that community-based clinicians can effectively implement manual-guided approaches such as MI and underlines that even small adaptations to intake/triage procedures in community clinics can improve initial treatment retention.

Participating Nodes and Community Treatment Programs

• Oregon Node:

  • Principal Investigator: Dennis McCarty

  • Protocol Coordinator: Lynn Kunkel

  • Expert Trainers: Kathy Tomlin and Chris Farentinos

  • CTP Directors:

    • ADAPT, Inc., Roseburg, OR: Bruce Piper

    • Changepoint, Inc., Portland, OR: Richard Drandoff and Chris Farentinos

    • Willamette Family Treatment Services (WFTS), Eugene, OR: Hillary Wylie

  • Site Coordinators:

    • ADAPT: Lori Farmer

    • Changepoint: Chris Farentinos

    • WFTS: Lucy Zammarelli

  • Supervisors:

    • ADAPT: Susan Dimock, Steve Dimock, and Lori Farmer

    • Changepoint: Chris Farentinos

    • WFTS: Lucy Zammarelli

  • Clinicians:

    • ADAPT: Larry Fricano, Paul Farmer, Linda Watson, Patsy Smith, and Joan Anderson

    • Changepoint: Monica Rodrigues, Paula Mena, Dianna Kingdon, and Gustavo Picazo, Bob Nobile, Katy McLeod, Maddy O'Callin, Michael Davis

    • WFTS: Theresa Powell, Trina Renfrew, Doug Laird, Denise Zanetta, and Natalie Lock

  • Research Assistants:

    • ADAPT: Marisha Johnson and Janet Tribble

    • Changepoint: Joanne Weidemann and Mary Dyer-Romer

    • WFTS: Davina Jones

• Mid-Atlantic Node:

  • Principal Investigator: Maxine Stitzer

  • Protocol Coordinator: Gena Britt

  • Expert Trainer: Charlotte Chapman

  • CTP Director: Ned Snead, Chesterfield County CSB Substance Abuse Services

  • Supervisors: Janet Loving, Tom Mullins, and Denise Pyle

  • Clinicians: Tammy Bays, Blair Craig, Don Hanback, Debbie McNee, Shane Severt, Taresa Steele, Valerie Stolcis, and Robin Waggoner

  • Research Assistant: Jennifer Ferris

• New York Node:

  • Principal Investigator: John Rotrosen

  • Protocol Coordinator: Erin Conner

  • Expert Trainer: Jon Morgenstern

  • CTP Directors: Eileen Pencer and Larry Taub, Lower East Side Service Center

  • Supervisor: Jackie DeCarlo

  • Clinicians: Yueh Shan Chan, Susan D'Addario, Christopher Dustow, Jaime Holder, Arthur Maglin, and Devorah Silverman

  • Research Assistant: Erica Torres

• New England Node:

  • Lead Investigators: Kathleen Carroll and Samuel Ball

  • Project Coordinators: Melissa Gordon, Julie Matthews, and Bryce Libby

  • MI Trainers: Theresa Moyers and William R. Miller