Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 Jun 1.
Published in final edited form as: Addiction. 2009 Mar 13;104(6):959–971. doi: 10.1111/j.1360-0443.2009.02525.x

Results from Two Randomized Clinical Trials Evaluating the Impact of Quarterly Recovery Management Checkups with Adult Chronic Substance Users

Christy K Scott 1, Michael L Dennis 1
PMCID: PMC2695999  NIHMSID: NIHMS114394  PMID: 19344441

Abstract

Aims

Post-discharge monitoring and early re-intervention have become standard practice when managing numerous chronic conditions. These two experiments tested the effectiveness of Recovery Management Checkup (RMC) protocols for adult chronic substance users.

Intervention

RMC included quarterly monitoring; motivational interviewing to provide personalized feedback and to resolve ambivalence about substance use; treatment linkage, engagement, and retention protocols to increase the amount of treatment received.

Participants and Setting

Recruited from sequential addiction treatment admissions, participants in the two experiments were on average 36 and 38 years of age, mostly female (59% vs. 46%), African American (85% vs. 80%), and met past-year criteria for dependence (87% vs. 76%).

Design

Participants in both experiments were randomly assigned to the RMC or control condition and interviewed quarterly for 2 years.

Measurement

The Global Appraisal of Individual Needs (GAIN) was the main assessment instrument.

Findings

RMC participant outcomes were better than control participants in both experiments. Effect sizes were larger in the second experiment in terms of reducing days to readmission (Cohen’s d=0.41 vs. d=0.22), successive quarters in the community using substances (d= −0.32 vs. −0.19), past-month symptoms of abuse/dependence (d=−0.23 vs. −0.02) and increasing the days of abstinence over 2 years (d=+0.29 vs. 0.04).

Conclusion

RMC, which provided ongoing monitoring and linkage, is feasible to conduct and is effective for adults with chronic substance dependence.

Keywords: Chronic dependence, recovery management

INTRODUCTION

A growing body of evidence demonstrates that severe alcohol and drug use often constitutes a chronic condition marked by cycles of recovery, relapse, and repeated treatments often spanning many years before reaching either stable recovery, permanent disability or death [1]. While the majority of people with lifetime substance dependence eventually enter sustained recovery (i.e., no symptoms for the past year), most do so after participating in multiple episodes of treatment [2]. Of the people admitted to the U.S. public treatment system in 2003, 64% were re-entering treatment, including 23% for the second time, 22% 3rd–4th times, and 19% 5th or more times [3]. Moreover, in an evaluation of over a decade of treatment participation data in the United Kingdom, Beynon and colleages [4] found that the trend towards shorter lengths of stay was associated with increasing rates of continued drug use at discharge and readmission within the year.

Retrospective and prospective treatment studies also report that most participants who achieve stable recovery do so after 3 to 4 episodes of treatment over multiple years [2,5]. Dennis, Scott and colleagues [2] recently found that the median time from first use to a year of abstinence was 27 years, and the median time from first treatment to a year of abstinence was 9 years with 3 to 4 treatment episodes. Studies have also demonstrated the cyclical nature of addiction by documenting the transitions between relapse, treatment, and recovery over time. Scott and colleagues [6] looked at the frequency and direction of transitions between points in the relapse, treatment reentry, and recovery cycle over 2 years and found that approximately 33% moved from one point in the cycle to another each quarter; 82% transitioned at least once; and 62% transitioned multiple times.

These prolonged, complex service histories are especially prevalent when addiction is accompanied by one or more psychiatric or social pathologies—a significant finding given the high rate of co-occurring disorders among those entering addiction treatment [7]. The protracted course of severe dependence is in line with the course of other chronic conditions: only 27% of patients with hypertension have blood pressures under control, 46% of diabetics have hemoglobin Hba1c levels below 7, and similarly distressing statistics can be found for patients with congestive heart failure, chronic arterial fibrillation, asthma, and depression [3]. Unlike conditions or illnesses that are time-limited and treatable in single episodes of acute care, chronic conditions ebb and flow over long periods of time, and their course is not fundamentally altered by acute episodes of stabilization.

Historically, most health care systems, including addiction treatment, have been organized around an episodic relationship in which a person seeks treatment, receives an assessment, is treated and presumed cured—all in a relatively short period of time. In the addictions field, policy makers, clinicians, patients and their families, and the public often have the unrealistic expectation that all patients entering addiction treatment should maintain lifelong abstinence following a single episode of specialized treatment. The reality that the majority of persons leaving treatment will resume alcohol and drug use in the first year following treatment, most within the first 30–90 days, challenges these expectations [5,6,9,10]. Studies conducted in a wide range of countries including Australia [11,12], Sweden [13], Spain [14], Thailand [15], United Kingdom [16,17], and the U.S. [18,19] indicate that unfortunately, allowing people to continue using in the community has consistently been associated with 6 to 11 times higher risk of death. These post-treatment relapse rates and increased risk of mortality demonstrate the need for multiple treatments over long periods of time, invalidate the traditional assumption that a single episode of treatment should result in immediate and long-lasting positive outcomes and challenge the adequacy of an acute care model of treatment for individuals suffering from chronic substance use.

In other fields, ongoing management of chronic conditions has positively impacted the severity and progression of these conditions even when such conditions themselves cannot be prevented or cured [1,2,8]. In a recent review of 22 continuing care studies (conducted in the Hong Kong, Sweden, Taiwan, and the U.S.), only 38% of the studies that worked with clients for less than 3 months produced positive findings whereas 44% of the studies that worked with clients for 4 to 12 months, and 100% of the studies that worked with clients for more than 12 months [20] produced positive results. In addition to working with clients over longer periods of time, more effective interventions were also characterized as being more “assertive” with regard to contacting clients, flexible scheduling/placement to accommodate client choice, and including recovery support services (including self help). Kristenson and colleagues [13,18] experimented with the use of quarterly checkups up to 4 years as part of physician visits in Sweden. Participants who received such checkups reported fewer days of being sick and fewer hospital days in the first 6 years and had lower mortality rates over the 10–16 years.

Building on both general chronic care models and these addiction specific studies, Scott and colleagues developed and tested a Recovery Management Checkup Model (RMC) [6,9,22,23,24]. The core assumption underlying this work is that long-term monitoring through regular checkups and early re-intervention will facilitate early detection of relapse, reduce the time to treatment reentry and consequently improve long-term participant outcomes. This approach does not rely on participants to identify their symptoms and return to treatment. Instead, these checkups are pro-active and include quarterly assessments and personalized feedback for each participant on the status of their condition. Staff use motivational interviewing (MI) techniques to involve participants in the decision-making process about their care by helping them resolve their ambivalence about their dependence and move toward a commitment to change.

The initial Early Re-Intervention (ERI-1) experiment established the feasibility and efficacy (altered treatment-relapse cycle and improved outcomes) of conducting quarterly recovery management checkups (RMC) in cases of severe chronic addiction [6,22]. Results indicated that RMC participants were significantly more likely than those in the control group to return to treatment, to return to treatment sooner, and to spend more subsequent days in treatment. RMC participants also experienced significantly fewer total quarters in need of treatment and were less likely to need treatment 2 years after intake. However, in spite of the successful linkage rates, only 39% of the linked participants remained in treatment 14 or more days. Given that individuals who stayed 14 or more days were significantly more likely to end the quarter in recovery, improving engagement and retention rates is clearly a challenge for on-going management of this condition.

While results from ERI-1 were encouraging, it took 2 years for the intervention to significantly impact participant outcomes. Using data from a combination of sources (weekly management reports, quality assurance feedback, and participant outcome data), the strengths and weaknesses of the original protocol were evaluated and changes to the original RMC protocol were integrated into the second experiment (ERI-2). These evidence-based modifications to the RMC protocol included transportation to treatment and an Engagement Specialist to improve treatment engagement and retention and were incorporated in ERI-2. The goals of this paper are to evaluate our ability to replicate the RMC model, improve implementation and evaluate the effectiveness of the modified RMC protocol on participant outcomes.

METHODS

Overview

The target population for both experiments was people presenting for publicly funded addiction treatment on the Westside of Chicago, Illinois, USA. Participants were recruited at the time of their intake assessment and were referred to all levels of care depending on their need. All levels of care were also available to participants in the experimental group over the life of the study. At intake there were no significant differences in the percentage of clients placed into residential treatment (64% in ERI-1, 81% in ERI-2) or in the average length of stay of their index episode of treatment (32 days in ERI 1 and 45 in ERI 2).

The first experiment (ERI-1) has been completed and involved quarterly checkups for 2 years. The first 24 of 48 months of the second experiment (ERI-2) have also been completed. In this paper, the 24-month participant outcomes from ERI-1 and ERI-2 will be compared. The main data sources for the process or implementation measures included daily staff logs, weekly management reports, and participant self-report. The main data source for participant outcomes in both studies was the Global Appraisal of Individual Needs [25] administered at intake and each quarterly assessment thereafter for 2 years.

In both experiments, participants were recruited from sequential intakes at the largest addiction treatment agency in Illinois (intent to treat sample). In the first experiment, 84% (448/533) of the eligible people agreed to participate and 93% (446/480) agreed in the second experiment. Participants in both experiments were randomly assigned to the control group (quarterly follow-up interviews only) or to the experimental group (quarterly follow-up interviews plus RMC).

Close to 95% of the participants in both experiments completed at least one follow-up interview (96% in ERI-1 and 94% in ERI-2) and most completed all eight (82% and 87%). There were no significant differences by condition in the quarterly follow-up rates for both studies. In ERI-1 across all quarters, an average of 93% of the RMC participants (quarterly rates ranged from 91% to 96%) completed their interview, and an average of 95% of the control participants completed their interviews (range: 92%–98%). In ERI-2 across all quarters, an average of 95% of the RMC participants (range: 93%–97%) and an average of 95% of the control participants completed their interviews (range: 94%–98%).

All quarterly checkups were scheduled in 90-day intervals post-intake. For example, the second quarterly assessment was scheduled 180 days post-intake. In ERI-1, 88%–95% of the interviews were completed within plus or minus 14 days of the anniversary date. In ERI-2, 91%–98% were completed within plus or minus 14 days of the anniversary date. Of those assigned to the experimental condition, most were in need of and received the RMC intervention in one or more quarters (77% and 87%; 13% and 15% for one quarter; 64% and 72% for multiple quarters).

The first set of analyses focused on process measures to determine how well the intervention was implemented in both experiments in terms of: (1) locating participants for their quarterly interview and determining their need for treatment, (2) completing the intervention following the assessment (RMC only), (3) motivating participants to agree to an intake assessment for re-admission to treatment, (4) completing the intake assessment, (5) accessing treatment and (6) remaining in treatment 14 or more days. It was hypothesized that the process measures would be better in ERI-2 than ERI-1.

Within both experiments, it was hypothesized that relative to participants assigned to the control group, RMC participants would: (1) return to treatment sooner, (2) receive more total days of treatment, (3) experience more days of abstinence, (4) have fewer successive quarters of unmet need for treatment and (5) be less likely to be in need of further treatment at the end of the study period. Note that treatment participation is both a process measure and an intermediary outcome measure. To control for sample differences, the quasi-experimental comparison of the effects in both experiments measures the relative difference between conditions using Cohen’s effect size d. Sample sizes were designed to have 80% or more power to detect an effect size of d=.2 or more. To rule out the possibility that observed differences were due only to sample differences between the two cohorts, outcome analyses were replicated with a propensity score adjustment for participant characteristics.

Eligibility criteria

In both experiments, individuals were included if they: (a) met lifetime criteria for substance abuse or dependence, (b) used alcohol or other drugs during the 90 days prior to intake, (c) completed an intake assessment and received a referral to any level of addiction treatment at the collaborating treatment agency, and (d) were 18 years of age or older. Logistical constraints in providing the RMC intervention required that individuals be excluded if they (a) did not reside in the City of Chicago, or (b) did not plan to reside in the city during the ensuing 12 months, or (c) had been sentenced to jail or prison or a DUI program for most of the upcoming 12 months, or (d) were unable to speak and understand English or Spanish, or (e) were too cognitively impaired to provide an informed consent. Participation was voluntary after an informed consent process under the supervision of Chestnut Health System’s Institutional Review Board on Human Subjects, and the study was conducted under the protection of a Federal Certificate of Confidentiality issued by the National Institute on Drug Abuse (NIDA).

Randomization

The second author used an excel file to sort participants by interview date and research ID, and then randomly assigned them to the control or RMC conditions for the duration of the study. Assignments were blocked into groups of six to prevent runs to one condition. The field coordinator was notified of all assignments and research assistants communicated individual assignments to participants during the first quarterly follow-up interview. The study was not blind because staff needed to transfer participants to the Linkage Managers to conduct the RMC intervention following the interview and to schedule RMC participants for longer periods of time in order to accommodate the Linkage Meeting. To control for contamination, randomization was done externally by the second author, all interviews and linkage meetings were audio taped (and a random sample reviewed) and different conducted interviews and the intervention with RMC participants.

Sample characteristics

Table 1 shows the intake characteristics by intake cohort and experimental condition within cohort. All participants were presenting to publicly funded addiction treatment. The first cohort was younger (36 vs. 38 mean years of age) and significantly more likely to be female (59% vs. 46%), African American (85% vs. 80%), and to meet past-year criteria for dependence (87% vs. 76%), internalizing disorders (75% vs. 53%), externalizing disorders (45% vs. 33%), and violence/crime problems (60% vs. 54%). The first cohort was also more likely to have a history of addiction treatment (68% vs. 62%), and less likely to have had involvement with the criminal justice system (75% vs. 83%) or to be referred to residential treatment (65% vs. 81%). Randomization largely worked in both experiments (with less than 5% of differences significant). In ERI-1, the one significant difference was that the control group was more likely than the RMC group to meet past-year criteria for alcohol dependence (42% vs. 31%). In ERI-2, the two significant differences were that the control group was more likely than the RMC group to have been homeless in the month before intake (32% vs. 22%) and to report past-year acts of violence or crime (59% vs. 48%).

Table 1.

Participant Characteristics by Randomized Condition and Study Cohort

ERI-1
 ERI-2
Characteristics (Column % unless noted) Control (n=224) RMC (n=224) Total (n=448) Control (n=223) RMC (n=223) Total (n=446)
Female \c 58% 61% 59% 49% 42% 46%
Race/Ethnicity - African American \c 86% 84% 85% 78% 82% 80%
   Hispanic 5% 6% 6% 3% 1% 2%
   Caucasian 6% 9% 8% 9% 7% 8%
   Mixed/Other 3% 1% 2% 11% 10% 10%
Age (mean) \c 36 36 36 38 39 38
Past-month Employment 23% 24% 23% 24% 26% 25%
Past-month Homelessness \b 35% 27% 31% 32% 22% 27%
Lifetime Phys., Sexual or Emotional Victimization 74% 75% 75% 72% 71% 71%
 History of traumatic levels of victimization 60% 61% 61% 59% 52% 56%
 Victimized in 90 days before intake 15% 17% 16% 17% 17% 17%
Past-year Substance Use Disorder \c 92% 94% 93% 87% 87% 87%
 Any Dependence \c 86% 88% 87% 79% 73% 76%
  Alcohol Dependence \a, \c 42% 31% 36% 20% 21% 20%
  Cocaine Dependence \c 66% 68% 67% 60% 52% 56%
  Opioid Dependence \c 27% 30% 29% 21% 21% 21%
  Cannabis Dependence \c 9% 7% 8% 4% 4% 4%
  Any Other Dependence \c 1% 2% 1% 1% 1% 1%
 Any Abuse \c 22% 20% 21% 27% 31% 29%
Lifetime SUD Treatment \c 70% 67% 68% 64% 60% 62%
 Treatment in the past 90 days 24% 24% 24% 19% 19% 19%
 Referred to Residential at Intake\c 66% 69% 65% 81% 80% 81%
Any Past-year Internalizing Disorder \c 75% 74% 75% 57% 49% 53%
 Depression \c 63% 59% 61% 53% 45% 49%
 Anxiety \c 61% 59% 60% 29% 22% 26%
 Trauma \c 43% 45% 44% 32% 28% 30%
 Suicidal \c 34% 37% 36% 13% 10% 11%
Any Past-year Externalizing Disorder \c 46% 45% 45% 34% 31% 33%
 Attention Deficit/Hyperactivity \c 36% 32% 34% 28% 24% 26%
 Conduct Disorder \c 36% 37% 37% 26% 24% 25%
Lifetime Prior Mental Health (MH) Treatment 28% 26% 27% 32% 26% 29%
 MH Treatment in the past 90 days 13% 10% 11% 15% 13% 14%
Any Past-year Crime/Violence Problems \b 58% 63% 60% 59% 48% 54%
 Physical Violence towards others \c 43% 51% 47% 48% 39% 43%
 Drug Related Crime \c 32% 31% 32% 26% 21% 23%
 Property Crime \c 26% 32% 29% 21% 17% 19%
 Interpersonal Violent Crime 14% 17% 16% 13% 9% 11%
Lifetime Criminal Justice System Involvement\c 72% 77% 75% 81% 85% 83%
 CJ Involvement in past 90 days \c 31% 34% 33% 47% 42% 45%
Open in a new tab
\a

Significant difference between conditions in study ERI-1, p < .05

\b

Significant difference between conditions in study ERI-2, p < .05

\c

Significant difference between studies, ERI-1 total vs. ERI-2 total, p < .05

Control condition

Participants randomly assigned to the control condition in both experiments were interviewed at baseline and again quarterly thereafter. The majority of quarterly assessments were conducted face-to-face at the research office. They required approximately 30–45 minutes to complete, and all on-site assessments were audio taped for purposes of quality assurance. Once the assessment was completed, the research assistant updated the locator information and scheduled the next appointment.

Recovery Management Check-up (RMC) condition

As with the control group, research staff conducted baseline and quarterly interviews with participants assigned to the experimental Recovery Management Checkup (RMC) condition. The goals of the RMC manual-guided protocol [23,24] were to identify current substance users living in the community and to provide them with an immediate linkage to treatment, thus expediting the recovery process. In both experiments, RMC involved the following key steps: (1) locating the person for a follow-up interview, (2) determining the eligibility for the intervention (i.e., verifying that the person was not already in treatment or jail and was living in the community) and need for treatment, (3) transferring participants in need of treatment from the interviewer to the Linkage Manager for the intervention, (4) obtaining participant agreement for an intake assessment for treatment, (5) linking participants to the intake assessment, (6) linking participants to treatment, and (7) sustaining participation in treatment for at least 14 days.

During the intervention, Linkage Managers used motivational interviewing techniques to provide personalized feedback to participants on the status of their condition and related problems, helped participants resolve their ambivalence about their dependence and move toward a commitment to change by accessing additional care, address existing barriers to treatment, and schedule an assessment and facilitate reentry (reminder calls, transportation). Motivational Interviewing (MI) is an effective evidence-based method for resolving the ambivalence that prevents many individuals from making desired changes in their lives [26].

While the intervention was successful in ERI-1, it required 24 months and 7 checkups to impact participant outcomes. To improve the model, implementation data from the first experiment guided the design of two main modifications to the original RMC protocol. Data from ERI-1 indicated that providing transportation increased the chances that participants would complete the intake and the first treatment appointments. Therefore, transportation was a required component for ERI-2. In ERI-1, only 39% of the people who accessed treatment stayed 14 days or more. Moreover, clients who received 14 or more days of treatment were significantly less likely than those with less than 14 days to end the quarter in “need of treatment” (25% vs. 48%, odds ratio=0.36 [95% CI= .24 to .54], p<.05). While the RMC intervention significantly increased treatment participation, it did not impact retention rates. To address this challenge, a highly specified treatment engagement and retention protocol was implemented in ERI-2. The protocol included a specific telephone and face-to-face contact schedule and an agreement between treatment and research staff that the linkage manager would have the opportunity to conduct an intervention with participants who wanted to leave treatment or that staff wanted to ask to leave treatment prematurely.

Staff training, supervision and quality assurance

In both experiments, research staff completed a formal training program and participated in ongoing quality assurance to achieve high adherence to project protocols. To maintain fidelity of the MI intervention, all linkage meetings were audio taped and reviewed by an external MI expert until the Linkage Managers were certified prior to intervention implementation. Following certification, a random sample of tapes was reviewed throughout the study. In ERI-1 both linkage managers had General Educational Developmental (GED) and in ERI-2 both held master’s degrees. In both experiments, all interviewers were certified to use the Global Appraisal of Individual Needs (GAIN) and the Linkage Managers received the same MI training and similar quality assurance monitoring.

Instruments and measures

Process or implementation variables were measured through daily staff contact logs that were entered daily and reviewed weekly. The participant characteristics, diagnosis, and primary outcomes were measured using the GAIN [25]. The GAIN is a comprehensive, structured interview that has eight main sections (background, substance use, physical health, risk behaviors, mental health, environment, legal, and vocational) with good internal consistency (alpha over .90 on main scales, .70 on subscales), test-retest reliability (Rho over .70 on days/problem counts, Kappa over .60 on categorical measures), and is consistent with timeline follow-back, urine tests, collateral reports, treatment records, and blind psychiatric diagnosis (Rho of .70 or more; Kappa of .60 or more) [27–30]. Table 2 lists the definitions of the process and participant outcome measures reported in this paper. For the outcome measures it includes reliability in terms of the Cronbach’s alpha for a scale, Spearman’s rank order correlation (Rho) for test-retest of continuous variables, and Kappa (κ) for test-retest of dichotomous measures. The latter two are from a test-retest study done as part of the final (24-month) wave of ERI-1 data collection as reported in Dennis, Scott and Funk [22]. Copies of the full GAIN and more detailed information on the GAIN scales and created variables are publicly available at www.chestnut.org/li/gain.

Table 2.

Summary of Key Measures from GAIN

ADHERENCE AND IMPLEMENTATION MEASURES (From Staff Logs)

  1. Follow-up Rate: The number of RMC participants completing quarterly assessments divided by the number of “the number of RMC participants minus those who had died.”

  2. Treatment Need Rate: The number of RMC participants living in the Chicago metropolitan area and meeting “in need of treatment” criteria (see below) divided by the number of RMC participants “completing quarterly assessments.”

  3. Linkage Meeting Attendance Rate: The number of RMC participants attending a “Linkage Meeting” for the intervention divided by the number of RMC participants meeting “in need of treatment” criteria.

  4. Agreed to Assessment Rate: The number of RMC participants “agreeing to go to an intake assessment for readmission to addiction treatment” divided by the number of RMC participants meeting “in need of treatment” criteria.

  5. Assessment Completion Rate: The number of RMC participants that completed the intake assessment” divided by the number of RMC participants meeting “in need of treatment” criteria.

  6. Treatment Access Rate: showing for treatment: The number of RMC participants that went to treatment divided by the number of RMC participants meeting “in need of treatment” criteria.

  7. 14+Treatment Days Engagement Rate: The number of RMC participants that “stayed 14 or more days in treatment” divided by the number of RMC participants who “went to treatment” in the same quarter.

OUTCOME MEASURES (From Self Report on GAIN)

  1. Re-entered Treatment (%; test-retest kappa=.81). Whether the person re-entered treatment after the point of randomization (at 3 months) and the final interview (at 24 months or last interview).

  2. Total Days Received Treatment (M = 48 and SD=80.38; test-retest Rho=.93). Based on the sum of days an individual received outpatient, intensive outpatient, residential, or inpatient treatment reported at each interview for a given period.

  3. Total Days of Abstinence (M = 475 and SD=170.39; test-retest Rho=.99). Sum of days abstinent from months 4 to 24 (max=630 days) using the mean days per quarter for any missing observations.

  4. Successive Quarters of Unmet Need for Treatment (M = 2.20 and SD=2.32; test-retest Rho=.92). The number of quarters in which the individual started and ended the period “in need of treatment” (see definition below) where missing data were replaced with status at prior wave. Note that this is the primary measure of “clinically significant change.”

  5. Days of Abstinence in the last quarter. (M = 68 and SD=31.11; test-retest Rho=.94). Days abstinent (of 90) reported at the 24-month interview, using the last interview for any missing observation.

  6. Past-month Substance Problem Scale (SPS) (M = 2.5 and SD=4.16; alpha=.93; test-retest Rho=.81) Count of 16 past-month item related to weekly use, hiding using, complaints about use, 4 symptoms of abuse, 7 symptoms of dependence, substance induced health or mental health problems.

  7. In Need of Treatment (%; test-retest kappa=.78). Defined as a participant living in the community (vs. in incarcerated or in treatment) who was not already in treatment and answered yes to any of the following questions: 1.) During the past 90 days, have you used alcohol, marijuana, cocaine, or other drugs on 13 or more days? 2.) During the past 90 days, have you gotten drunk or been high for most of 1 or more days? 3.) During the past 90 days, has your alcohol or drug use caused you not to meet your responsibilities at work/school/home on 1 or more days? 4.) During the past month, has your substance use caused you any problems? 5.) During the past week, have you had withdrawal symptoms when you tried to stop, cut down, or control your use? 6.) Do you feel that you need to return to treatment? These criteria for need are internally consistent (alpha=.85) and the average person in need endorsed 3.3 of 6 of the items (80% endorsed 2 or more).

Open in a new tab

In both experiments urine samples were collected at 12 and 24 months in conjunction with on-site interviews. The urine results were used to improve the validity of self-report and not used to determine need for treatment. In ERI-1 the rate of false negatives (i.e., denying past-month use but positive on the urine test) was 9% at 12 months and 15% at 24 months. To reduce the rate of false negatives in ERI-2, the team adopted a new feedback protocol involving a review of substances reported in prior interviews, on-site urine testing, giving results immediately, and probing any inconsistencies of these two sources with subsequent self-reported recency of use. The ERI 2 feedback protocol was associated with significantly lower false negative rates at both 12 months (9% vs. 3%; d=.13, X2(1)= 13.89 p<.05) and 24 months (15% vs. 2% false negative; d=.25, X2(1)= 40.43, p<.05).

Analytic procedures

All analyses were done with SPSS [31] using an ‘intent to treat’ model [32]. This means that (1) people were analyzed as randomly assigned, regardless of whether they needed or received RMC services, and (2) for the small amount of missing data by wave, we imputed the answers as noted in Table 2. Data were used from all available interviews (98% of ERI-1 and 97% of ERI-2).

The first set of analyses focused on the process measures (e.g., % followed up, % in need of RMC, % completing linkage meeting, % agreeing to go to an assessment, % receiving an assessment, % accessing treatment, % engaging in treatment for 14 or more days), where the degree of implementation of RMC was evaluated by looking at the average across quarters (with increases indicating improvement) and the quarter-to-quarter variation (where a smaller range demonstrates better implementation). The average differences between RMC in ERI-1 and 2 were tested using a mixed model nominal regression with Hedeker [33] MIXNO and reported in terms of the Z-score of the Wald statistic.

The second set of analyses focused on participant outcomes that RMC intervention targeted for improvement. Note that treatment participation is both a process measure and an intermediary outcome measure. Cox Proportional Hazard survival analysis was conducted on the time from the 3-month interview (point of randomization) to first subsequent readmission, treating people who had not returned to treatment by their last interview as right censored (i.e., they had not re-entered treatment; 42% of ERI-1 and 54% of ERI-2 observations were right censored). For categorical outcome measures (e.g., % reentered treatment, % needing treatment at the last quarterly observation), the differences between conditions were tested using an exact chi-square. For most of the continuous participant outcome measures (e.g., total days of treatment, total days of abstinence, the number of successive quarters of unmet need for treatment, days of abstinence in the last quarterly observation, and past-month substance problem scale count of abuse/dependence symptoms), reliable differences were tested with analysis of covariance and reported in terms of an “F” statistic for the condition (experimental vs. control) with the baseline and 3-month values of the respective dependent variables as covariates.

Because the two experiments were conducted with separate time cohorts (2000 vs. 2004), the RMC vs. control differences for each experiment were compared conservatively in terms of their within study effect sizes in the same way that this issue is handled in a meta analysis. The process and outcome figures and tables include Cohen’s effect size “d”, where d=+/− 0.2 is considered small, d=+/− 0.4 is medium size and d=+/− 0.8 or more is large. If the d was .2 or greater but the alpha was greater than .05, we referred to this as a “trend.” To rule out the possibility that observed differences were due only to sample differences between the two cohorts, outcome analyses were replicated with a propensity score adjustment for participant characteristics. Specifically, the propensity (i.e., probability) of being in the second experiment (vs. the first) given the characteristics in Table 1 was calculated and saved to the combined data file. This propensity score was added to the outcome analyses related to Figures 2 and 3 and Table 3. If this propensity score adjustment “reduced” the differences between studies, it suggests the possibility that the improved outcomes might be due to subject differences in the sample. Conversely, the extent to which the propensity adjustment had no impact or increased the size of the difference between studies suggests that the observed improvements are more likely to be due to the differences in the RMC intervention and its implementation.

Figure 2.

Figure 2

Open in a new tab

Days to Re-Admission to Substance Treatment by Condition in ERI-1

Figure 3.

Figure 3

Open in a new tab

Days to Re-Admission to Substance Treatment by Condition in ERI-2

Table 3.

Effectiveness of RMC over Control by ERI Experiment

ERI-1 (months 4-24)
 ERI-2 (months 4-24)
Outcome Variable Good is Control RMC Cohen’s d Control RMC Cohen’s d
Between Months 4 to 24 (630 days)
Re-entered Treatment Up 51% 60% 0.21* 37% 55% 0.40*
Total Days of Treatment Up 40 63 0.27* 36 53 0.23*
Total Days of Abstinence Up 490 497 0.04 430 480 0.29*
# Successive Quarters of Unmet Need for Tx Down 2.31 1.86 (0.19)* 3.41 2.59 (0.32)*
At 24-Month Interview
Days of Abstinence (out of 90) Up 72 71 −0.05 61 68 0.23*
Past-month Symptoms on SPS (out of 16) Down 2.3 2.3 (0.02) 3.0 2.1 (0.23)*
In Need of Tx at last wave Down 44% 34% (0.21)* 57% 46% (0.24)*
Open in a new tab
*

P<.05

RESULTS

Adherence: how well was the model implemented?

Figure 1 shows the average and range by quarter for each variable used to assess the quality of implementation or adherence to the RMC model for ERI-1 and ERI-2. High follow-up rates were achieved across quarters in each experiment (93% in ERI-1 vs. 96% in ERI-2 of those randomized; d=0.18, Z = 1.79, n.s.d.). In ERI-2, significantly more participants were identified as needing treatment (30% vs. 44% of those interviewed; d=0.31, Z = 3.95, p<.001). Part of this improvement in identification was likely due to the reduced rates of false negatives associated with the ERI-2 feedback protocol (discussed earlier in 2.8). Of those who were eligible and in need, significantly more participants in ERI-2 attended the linkage meeting for the intervention (75% vs. 99% of those in need; d=1.45, Z = 5.86, p<.001). Of these participants in both experiments, a similar percentage agreed to schedule an intake assessment for re-admission to treatment (44% vs. 45% of those in need; d=0.02, Z = 0.43, n.s.d.). Relative to ERI-1, RMC participants in ERI-2 were significantly more likely to complete the intake assessment (30% vs. 42% of those in need; d=0.26, Z = 2.978, p < .01), access treatment (25% vs. 30% of those in need; d=0.18, Z = 1.98, p < .05), and remain in treatment for 14 or more days (58% vs. 39% of those in need; d=0.43, Z = 2.24, p < .05). It is important to note that across these variables, the quarter-to-quarter range narrowed in the second experiment, representing more model adherence and fidelity, which speaks to the feasibility of replicating the model.

Figure 1.

Figure 1

Open in a new tab

RMC Protocol Adherence Rate by Experimental Cohort

Participant Outcomes (Control vs. RMC) in ERI-1 vs. ERI-2

Time to readmission and treatment received

Figure 2 shows the time from the 3-month interview (point of randomization and the first opportunity for RMC) to the first subsequent readmission for control and RMC participants in ERI-1, and Figure 3 shows the same information for ERI-2. Unlike Figure 2, the next analysis compares RMC to the control group and everyone is included as assigned. After discharge from the index episode of care in ERI-1, RMC participants were significantly more likely than control group participants to return to treatment (51% vs. 60%) and return sooner (achieving 51% 200 days earlier) (Cohen’s d=+0.22; X2(1)= 5.15, p < .05). After discharge from the index episode of care in ERI-2, RMC participants were also significantly more likely than control group participants to return to treatment (37% vs. 55%) and to return sooner (achieving 37% 384 days earlier) (Cohen’s d=+0.41; X2(1)= 16.56, p < .0001). Treatment included both inpatient and outpatient programs. Adding the propensity score adjustment slightly reduced the significance and size of the effects in the first experiment and had no effect on the second experiment – suggesting that differences were more likely attributable to the protocol and implementation differences than the subject differences. In addition to achieving a more robust effect, the impact of ERI-2 shows up more prominently in the first 180 days and the horizontal gap to achieve the same percentage in treatment increases over time in both studies. Table 3 contains the outcomes by experiment and condition over the first 2 years. Relative to the control condition, RMC significantly increased the total average days of subsequent treatment received in months 4 to 24 in both ERI-1 (40 days in control vs. 63 days in RMC; d=.27; p<.05) and in ERI-2 (36 vs. 53 days, d=.23; p<.05).

Impact on substance use

In ERI-1, the impact of RMC on dependence was mixed: there was no significant difference in the total days of abstinence or the number of successive quarters of need across months 4 to 24. However, relative to people in the control group, the RMC participants had more total days in treatment across 4 to 24 months (41 vs. 62 days, F(1, 415)= 7.76, p<.05), fewer successive quarters of unmet need for treatment (2.31 vs. 1.86 quarters; d= −0.19, F(1,446)=4.30, p<.05) and the RMC participants were less likely to be in the community in need of treatment at month 24 (44% vs. 34%, d= −0.21, X2(1)= 4.23, p<.05). The improved implementation in ERI-2, in contrast, was associated with more consistent outcomes, with participants assigned to RMC reporting significantly more total days of treatment (36 vs. 53 days; d=0.23, F(1, 416)= 6.27, p<.05), more days of abstinence (430 vs. 480 days out of 680 days; d=0.29, F(1, 415)= 9.55, p<.05), and fewer successive quarters of unmet need for treatment (3.41 vs. 2.59 quarters; d= −0.32, F(1, 444)=11.85, p<.001) across months 4 to 24. In the last quarter, RMC participants reported significantly more days of abstinence than control participants (61 vs. 68 days out of 90; d=0.23, F(1, 422)= 5.48, p<.05), fewer past-month symptoms of abuse, dependence, or substance induced problems (3.0 vs. 2.1 out of 16, d= −0.23, F(1, 422)= 5.37, p<.05), and were less likely to be in the community in need of treatment at month 24 (57% vs. 46%; d= −0.24, X2(1)= 4.56, p < .05). Adding the propensity score adjustment to each analysis had no effect on the statistical significance for either experiment and only slightly decreased the effect sizes in the first experiment. This suggests that the differences reported in Table 3 were likely attributable to the protocol differences than the subject differences.

Impact on adverse events

As would be expected in this population, during the two-year period (across experiments), multiple instances of serious adverse events (SAE) occurred including medical (53%) or mental (11%) emergency room or hospital admissions, incarcerations (37%), death (2%) as well as a wide range of other adverse events. However, these rates were lower than the quarter prior to enrollment in the study (and initial treatment). There were no SAE or AE that appeared to be specifically related to randomization to RMC according to the principal investigator and data safety monitoring board, and generally no significant differences between RMC and the control group in rates of a wide range of SAE or AE. The exceptions in ERI-1 were that relative to the control group, RMC participants were significantly more likely to have a period of any incarceration (40% to 25%, X2(1)=10.38, p<.05, d=0.30), but were not significantly different in terms of the days of incarceration or days of illegal activity. In ERI-2 when compared to control group participants, RMC participants reported significantly fewer days of problems due to AOD use (32.2 to 48.3, F(1,434)=4.44, p<.05, d=−0.19) and significantly fewer days of arguing/fighting (29.1 to 40.1, F(1,435)=4.56, p<.05, d=−0.17). Thus, the SAE/AE rates appear to be primarily a function of the clinical severity of the samples.

DISCUSSION

The goals of this paper were to explore the replicability of the revised Recovery Management Checkup (RMC) model and to evaluate its impact on implementation and participant outcomes. Components of the RMC model were designed to address the duration and cyclical nature of dependence via quarterly monitoring and linkage to treatment. These regular checkups provided a pro-active approach to help participants learn to identify symptoms and resolve their ambivalence about their substance use; to offer the opportunity for multiple episodes of care in the context of chronic care management; and to include an engagement and retention component to retain participants in treatment.

Data from ERI-1 demonstrated the feasibility of implementing quarterly monitoring and recovery management checkups as well as the impact on participant outcomes two years after the initial treatment episode. Data from ERI-2 provided evidence regarding the feasibility of replicating the RMC model and improving its implementation. In both experiments, the team achieved high follow-up rates and high participant agreement to return to treatment. Relative to ERI-1, however, in ERI-2 RMC participants were more likely to be identified as in need of treatment (30 vs. 44%), attend the linkage meetings (75% vs. 99% of those in need), complete the assessment (30% vs. 42% of those in need), access treatment (25% vs. 30% of those in need), and remain in treatment for at least 14 days (39% vs. 58% of those starting treatment).

Results also indicated that modifications to the RMC model facilitated and improved the consistency with which the protocols were implemented (e.g., the quarter to quarter range in implementation narrowed in the second experiment). Thus, the RMC intervention likely contributed to more participants returning to treatment sooner and improved participant outcomes. The control group participants, in contrast, continued to use substances in the community for more successive quarters, which is consistent with prior findings on the natural cycle of relapse, treatment and recovery [6,9]. The current findings are also significant in light of prior research suggesting that people who do not return to treatment after relapse are likely to “deteriorate” further until they do [34]. It is also consistent with continuing care research that longer term and more assertive monitoring can improve outcomes [20]. Thus, the clinically significant reduction in the successive quarters of unmet need for treatment shown in these experiments demonstrates the effectiveness of RMC for managing dependence over time. Moreover, the effect size of these gains was as much as twice as large in ERI-2 than in ERI-1.

Strengths and Limitations

These two experiments have numerous strengths: the sample sizes, repeated observations, high follow-up rates, detailed measurement, randomization and replication across two cohorts. However, it is also important to note the limitations. First, the comparison of RMC in two experiments is based on two time cohorts (recruited in 2000 vs. 2004) and hence quasi-experimental in nature. While the samples were recruited from the same addiction treatment program using the same inclusion/exclusion criteria, shifts in the participant characteristics and treatment personnel confound the contrasts presented here. The finding that propensity score adjustments had either no effect or increased the differences between studies suggests that differences are not due to participant characteristics. Second, urine-testing procedures differed in the two experiments with the ERI-2 method resulting in significantly lower false negative rates. Thus, the days of abstinence reported for ERI-1 are probably an over estimate and, again, the improvement of ERI-2 over ERI-1 is probably larger than reported here. Third, for logistical reasons we limited recruitment to a single site with predominately African American inner city participants with high rates of co-occurring mental disorders, homelessness, and criminal justice system involvement. In the future, it would be useful to replicate this work with a more representative sample of the U.S. public treatment system, in other countries and/or less severe clinical samples.

Implications

Addiction like other chronic conditions is often marked by cycles of relapse, multiple treatments, and intermittent periods of abstinence over many years before reaching sustained recovery. Results from the two ERI studies provide evidence that ongoing monitoring, feedback and early re-intervention can be effective methods of managing recovery over time. Ideally, such services would become a requirement for treatment program licensure, accreditation and funding. Those requirements would be best linked to a larger strategy of re-orienting addiction treatment from a predominantly acute care model of intervention to a service model that provides services ranging from a brief intervention to long-term recovery management. However, the implications of shifting to a chronic care model are significant. That shift will require a radical redefinition of the continuum of care, new service philosophies, new service delivery technologies and a fundamental rethinking of systems of reimbursement for addiction treatment and recovery support services.

The shift will also require various stakeholders to address a number of critical questions. The first relates to the manner in which post-treatment monitoring, support, and early re-intervention services can be integrated into the current continuum of care. A second question is how to make these services accessible to all participants who enter treatment with high problem severity, complexity and chronicity.

Another critical question is where (organizationally) and by whom these post-treatment recovery support services will be best delivered. The data supporting recovery management checkups have been generated by services provided by clinical and non-clinical staff employed within a research organization. Additional research is needed to determine if the effectiveness of such services differs across types of organizations or differs based on whether they are delivered by clinical or non-clinical staff. Regardless of who ultimately delivers post-treatment recovery support services, considerable thought will need to be given to the recruitment, orientation, training and supervision of individuals performing these functions. Experience to date suggests the need for a substantial investment in articulating the ethics and etiquette of conducting recovery management checkups across diverse clinical populations and cultural contexts.

An important step in moving forward will be to estimate the costs associated with ongoing monitoring and early re-intervention. It may be that RMC will cost more than standard care (significantly more days of treatment by the RMC participants translates into higher costs for this group, at least for treatment provision). Even though the outcome data demonstrate the efficacy of RMC, cost offset (not just cost) data would be particularly important – it is much more likely that a program would choose to adopt RMC if it not only led to better outcomes but also could be shown to reduce subsequent costs. Other steps might include: (a) testing the model with different populations such as pregnant and postpartum women, male and female offenders leaving jail or prison, or adolescents; (b) determining when a participant can be transitioned from quarterly to bi-annual checkups based on need; (c) determining whether more frequent or even continuous monitoring would improve outcomes; and (d) testing the impact of linkages to less formalized types of care such as recovery coaches or faith-based interventions.

Acknowledgments

This work was done with support provided by the US National Institute on Drug Abuse Grant No. DA 11323. The authors would like to thank Rod Funk, Joan Unsicker, Melissa Ives, Victoria Coleman and Stephanie Guetschow Merkle for assistance preparing the manuscript; Bill White for his thoughtful comments, and the study staff and participants for their time and effort. The opinions are those of the authors and do not reflect official positions of the government.

Footnotes

Conflict of Interests: None

References

  • 1.Dennis ML, Scott CK. Managing addiction as a chronic condition. Addiction Sci & Clin Pract. 2007;4:45–55. doi: 10.1151/ascp074145. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Dennis ML, Scott CK, Funk R, Foss MA. The duration and correlates of addiction and treatment careers. Journal of Substance Abuse Treatment. 2005;28:S51–S62. doi: 10.1016/j.jsat.2004.10.013. [DOI] [PubMed] [Google Scholar]
  • 3.Office of Applied Studies. Abuse Treatment Services. Rockville, MD: Substance Abuse Mental Health Services Administration; 2005. Treatment Episode Data Set (TEDS): 2002. Discharges from Substance . (DASIS Series S-25; DHHS Publication No. (SMA) 04-3967) [Google Scholar]
  • 4.Beynon CM, Bellis MA, McVeigh J. Trends in drop-out, drug free discharge and rates of representation: A retrospective cohort study of drug treatment clients in the North West of England. BMC Public Health. 2006;6:205. doi: 10.1186/1471-2458-6-205. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Hser YI, Grella CE, Chou C-P, Anglin MD. Relationships between drug treatment careers and outcomes: Findings from the National Drug Abuse Treatment Outcome Study. Eval Review. 1998;22:496–519. [Google Scholar]
  • 6.Scott CK, Foss MA, Dennis ML. Pathways in the relapse, treatment, and recovery cycle over three years. J Subst Abuse Treat. 2005a;28:S63–S72. doi: 10.1016/j.jsat.2004.09.006. [DOI] [PubMed] [Google Scholar]
  • 7.Chan Y-F, Dennis ML, Funk RR. Prevalence and comorbidity co-occurrence of major internalizing and externalizing disorders among adolescents and adults presenting to substance abuse treatment. J Subst Abuse Treat. 2008;34:14–24. doi: 10.1016/j.jsat.2006.12.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.McLellan AT, Lewis DC, O’Brien CP, Kleber HD. Drug dependence, a chronic medical illness: Implications for treatment, insurance, and outcomes evaluation. Journal of the American Medical Association. 2000;284:1689–95. doi: 10.1001/jama.284.13.1689. [DOI] [PubMed] [Google Scholar]
  • 9.Scott CK, Foss MA, Dennis ML. Utilizing Recovery Management Checkups to shorten the cycle of relapse, treatment reentry, and recovery. Drug Alcohol Depend. 2005b;78:325–38. doi: 10.1016/j.drugalcdep.2004.12.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Simpson DD, Joe GW, Broome KM. A national 5-year follow-up of treatment outcomes for cocaine dependence. Arch Gen Psychiatry. 2002;59:538–44. doi: 10.1001/archpsyc.59.6.538. [DOI] [PubMed] [Google Scholar]
  • 11.Darke S, Ross J, Teesson M. The australian treatment outcome study (ATOS): What have we learnt about treatment for heroin dependence? Drug and Alcohol Review. 2007;26(1):49–54. doi: 10.1080/09595230601036986. [DOI] [PubMed] [Google Scholar]
  • 12.Gibson AE, Degenhardt LJ, Hall WD. Opioid overdose deaths can occur in patients with naltrexone implants. Med J Aust. 2007;186:152–3. doi: 10.5694/j.1326-5377.2007.tb01126.x. [DOI] [PubMed] [Google Scholar]
  • 13.Kristenson H, Osterling A, Nilsson JA, Lindgarde F. Prevention of alcohol-related deaths in middle-aged heavy drinkers. Alcohol Clin Exp Res. 2002;26:478–84. doi: 10.1111/j.1530-0277.2002.tb02564.x. [DOI] [PubMed] [Google Scholar]
  • 14.Brugal MT, Domingo-Salvany A, Puig R, Barrio G, Garcia de Olalla P, de la Fuente L. Evaluating the impact of methadone maintenance programmes on mortality due to overdose and aids in a cohort of heroin users in Spain. Addiction. 2005;100:981–9. doi: 10.1111/j.1360-0443.2005.01089.x. [DOI] [PubMed] [Google Scholar]
  • 15.Quan VM, Vongchak T, Jittiwutikarn J, Kawichai S, Srirak N, Wiboonnatakul K, Razak MH, Suriyanon V, Celentano DD. Predictors of mortality among injecting and non-injecting HIV-negative users in northern Thailand. Addiction. 2007;102:441–6. doi: 10.1111/j.1360-0443.2006.01709.x. [DOI] [PubMed] [Google Scholar]
  • 16.Farrell M, Marsden J. Acute risk of drug-related death among newly released prisoners in England and Wales. Addiction. 2008;103:251–5. doi: 10.1111/j.1360-0443.2007.02081.x. [DOI] [PubMed] [Google Scholar]
  • 17.Gossop M, Stewart D, Treacy S, Marsden J. A prospective study of mortality among drug misusers during a 4-year period after seeking treatment. Addiction. 2002;97:39–47. doi: 10.1046/j.1360-0443.2002.00079.x. [DOI] [PubMed] [Google Scholar]
  • 18.Saitz R, Gaeta J, Cheng DM, Richardson JM, Larson MJ, Samet JH. Risk of mortality during four years after substance detoxification in urban adults. J Urban Health. 2007;84:272–82. doi: 10.1007/s11524-006-9149-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Smyth B, Hoffman V, Fan J, Hser Y. Years of potential life lost among heroin addicts 33 years after treatment. Prev Med. 2007;44:369–74. doi: 10.1016/j.ypmed.2006.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.McKay J. Continuing care research: What we’ve learned and where we’re going. J Subst Abuse Treat. doi: 10.1016/j.jsat.2008.10.004. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Kristenson H, Ohlin H, Hulten-Nosslin MB, Trell E, Hood B. Identification and intervention of heavy drinking in middle-aged men: Results and follow-up of 24–60 months of long-term study with randomized controls. Alcohol Clin Exp Res. 1983;7:203–9. doi: 10.1111/j.1530-0277.1983.tb05441.x. [DOI] [PubMed] [Google Scholar]
  • 22.Dennis ML, Scott CK, Funk R. An experimental evaluation of recovery management checkups (RMC) for people with chronic substance use disorders. Eval Program Plann. 2003;26:339–52. doi: 10.1016/S0149-7189(03)00037-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Scott CK, Dennis ML. Preliminary findings from the Early Re-Intervention (ERI) experiment with chronic substance abusers (abstract) Drug Alcohol Depend. 2002;66 (Suppl 1):S161. [Google Scholar]
  • 24.Scott CK, Dennis ML. Recovery Management Checkups: An Early Re-Intervention Model. Chicago, IL: Chestnut Health Systems; 2003. (available from author) [Google Scholar]
  • 25.Dennis ML, Titus JC, White M, Unsicker J, Hodgkins D. Global Appraisal of Individual Needs (GAIN): Administration guide for the GAIN and related measures. 5. Bloomington, IL: Chestnut Health Systems; 2003. [Google Scholar]
  • 26.Lamb S, Greenlick MR, McCarty D, editors. Bridging the gap between practice and research: Forging partnerships with community-based drug and alcohol treatment. Washington, DC: National Academy Press; 1998. [PubMed] [Google Scholar]
  • 27.Godley MD, Godley SH, Dennis ML, Funk R, Passetti L. Preliminary outcomes from the assertive continuing care experiment for adolescents discharged from residential treatment. J Subst Abuse Treat. 2002;23:21–32. doi: 10.1016/s0740-5472(02)00230-1. [DOI] [PubMed] [Google Scholar]
  • 28.Dennis ML, Titus JC, Diamond G, et al. The Cannabis Youth Treatment (CYT) experiment: Rationale, study design, and analysis plans. Addiction. 2002;97:S16–S34. doi: 10.1046/j.1360-0443.97.s01.2.x. [DOI] [PubMed] [Google Scholar]
  • 29.Lennox RD, Dennis ML, Scott CK, Funk RR. Combining psychometric and biometric measures of substance use. Drug Alcohol Depend. 2006b;83:95–103. doi: 10.1016/j.drugalcdep.2005.10.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Shane P, Jasiukaitis P, Green RS. Treatment outcomes among adolescents with substance abuse problems: The relationship between comorbidities and post-treatment substance involvement. Eval Program Plann. 2003;26:393–402. [Google Scholar]
  • 31.SPSS. Statistical Program for the Social Sciences. Chicago, IL: 2006. Version 14.0. [Google Scholar]
  • 32.Dennis ML, Perl HI, Huebner RB, McLellan AT. Methodological challenges in study design and implementation: Twenty-five strategies for improving the design, implementation and analysis of health services research related to alcohol and other drugs. Addiction. 2000;95:s281–s308. doi: 10.1080/09652140020004241. [DOI] [PubMed] [Google Scholar]
  • 33.Hedeker D. MIXNO: A computer program for nominal mixed-effects nominal logistic regression. J Stat Sfw. 1999;4:1–92. [Google Scholar]
  • 34.Simpson DD, Joe GW, Bracy SA. Six-year follow-up of opioid addicts after admission to treatment. Arch Gen Psychiatry. 1982;39:1318–23. doi: 10.1001/archpsyc.1982.04290110070012. [DOI] [PubMed] [Google Scholar]

RESOURCES