An experimental evaluation of recovery management checkups (RMC) for people with chronic substance use disorders

Abstract

The majority of people presenting for publicly-funded substance abuse treatment relapse and receive multiple episodes of care before achieving long-term recovery. This Early Re-Intervention experiment evaluates the impact of a Recovery Management Checkup (RMC) protocol that includes quarterly recovery management checkups (assessments, motivational interviewing, and linkage to treatment re-entry). Data are from 448 adults who were randomly assigned to either RMC or an attention (assessment only) control group. Participants were 59% female, 85% African American, and 75% aged 30–49. Participants assigned to RMC were significantly more likely than those in the control group to return to treatment, to return to treatment sooner, and to spend more subsequent days in treatment; they were significantly less likely to be in need of additional treatment at 24 months. This demonstrates the importance of post-discharge recovery management checkups as a means to improve the long-term outcomes of people with chronic substance use disorders.

1. Introduction

Most people who use substances are eventually able to abstain or manage their use without the aid of either professionally-directed treatment or self help groups (Burman, 1997; Cunningham, 1999a,b; 2000; Granfield & Cloud, 1996; Hughes, 1996; Humphreys, Moos, & Finney, 1995; Kandel & Raveis, 1989; Sobell, Ellingstad, & Sobell, 2000; Toneatto, Sobell, Sobell, & Rubel, 1999). However, over the past 50 years there has been a growing recognition that a subset of substance users suffer from a chronic condition (Drummond, 1990; 1992; Edwards & Gross, 1976; Jellinek, 1960; Leshner, 1997; Leukefeld & Leukefeld, 1999; McLellan, Lewis, O'Brien, & Kleber, 2000; Muthen, Grant, & Hasin, 1993; Tims, Leukefeld, & Platt, 2001a,b). The American Psychiatric Association [APA] (1994, 2000) label this condition ‘substance dependence’ and define it as having three or more of seven symptoms: increased tolerance, withdrawal, loss of control, inability to cut down or stop, pre-occupation in terms of time spent using it or giving up other activities, and continued use despite persistent medical or psychological problems that are probably caused by it (the World Health Organization (1999) defines a slightly different but essential the same syndrome its international classification of diseases version 10). Approximately 7.1 million people in the US (3.2% of the population over age 12) met this definition of dependence in the year 2000, with only 1 in 5 getting any kind of treatment and about 1 in 10 ending up in publicly funded substance abuse treatment (Epstein, 2002; OAS, 2000).

Longitudinal studies have repeatedly demonstrated that substance abuse treatment (particularly for 90 or more days) is associated with major reductions in substance use, problems, and costs to society (French et al., 2000, 2002a,b; Hser et al., 2001a; Hoffman, Grella, & Anglin, 2001b; Hubbard et al., 1989; Salomé et al., 2003; Sells, 1974; Simpson, Joe, & Roway-Szal, 1997a; Simpson et al., 1997b; Simpson, Joe & Brown, 1997c; Simpson, Joe, Fletcher, Hubbard, & Anglin, 1999). However, post-discharge relapse and eventual re-admission are also the norm (Godley, Godley, Dennis, Funk, & Passetti, 2002; Lash, Petersen, O'Connor, & Lehmann, 2001; McKay et al., 1997, 1998). The risk of relapse does not appear to abate until 4 to 5 years of abstinence (Dawson, 1996; De Soto, O'Donnell, & De Soto, 1989; Jin, Rourke, Patterson, Taylor, & Grant, 1998; Nathan & Skinstad, 1987; Vaillant, 1996).

If we look at the cross section of people admitted to the US public treatment system in 1999, 60% have been in treatment before (including 23% 1 time, 13% 2 times, 7% 3 times, 4% 4 times, and 13% 5 or more times) (Office of Applied Studies, 2000). Ideally (according to ASAM (2001)), residential and intensive outpatient care should be followed by a step down to a less intensive level of care—though generally only about 1 in 5 actually attend aftercare or continuing care (see Godley et al. (2002) and McKay (2001) for reviews). Within 12 months after a given treatment episode, 25–35% of the clients return to treatment on their own—with the rates growing closer to 50% after 2–5 years (Hubbard et al., 1989; Peterson, Swindle, Ciaran, Recine, & Moos, 1994; Simpson and Savage, 1980a,b; Simpson et al., 2002). Retrospective and prospective treatment studies report that most clients undergo 3–4 episodes of care before reaching a stable state of abstinence (Anglin, Hser, & Grella, 1997; Grella & Joshi, 1999; Hser, Anglin, Grella, Longshore, & Prendergast, 1997; Hser, Grella, Chou, & Anglin, 1998). In a recent survival analysis for an intake cohort (n = 1326) from Chicago, we estimated the median time from first use to at least a year of abstinence in the community was 27 years and that the median time from first treatment entry to a year of abstinence in the community was 8 years (and 3–4 episodes of treatment) (Dennis, Scott, & Hristova, 2002). Observational studies that have examined treatment effects across episodes of care have found that the sooner after a relapse someone returns to treatment and the more subsequent treatment someone receives (particularly over 90 days total), the better their long-term outcomes (Joe, Chastain, & Simpson, 1990; Scott, Foss, & Dennis, 2003; Simpson, Savage, & Lloyd, 1979, Simpson & Savage, 1980a,b; Simpson & Sells, 1990, Simpson, Joe, & Broome, 2002; Stout, Rubin, Zwick, Zywiak, & Bellino, 1999).

In spite of this evidence of chronicity and multiple episodes of care, most substance abuse treatment continues to be characterized as relatively self-encapsulated, serial episodes of acute treatment with post discharge aftercare typically limited to passive referrals to self help groups (Dennis, Perl, Huebner, & McLellan, 2000; Godley et al., 2002; McLellan et al., 2000; White, 1996; Etheridge, Hubbard, Anderson, Craddock, & Flynn, 1997). Concern about these issues has led to several calls for new approaches modeled after the treatment of other chronic disorders (e.g. cancer, type 2 diabetes mellitus, hypertension, and asthma) which have similar kinds of relapse rates, readmission rates, and co-occurring problems that complicate treatment (Anglin et al., 1997; Davidson & Strauss, 1995; Dennis et al., 2000; Else, 1999; Godley et al., 2002; Lamb, Greenlick, & McCarty, 1998; Leshner, 1997; Leukefeld & Leukefeld, 1999; Leukefeld, Tims, & Platt, 2001; O'Brien & McLellan, 1996; McLellan, Lewis, O'Brien, & Kleber, 2000; White, Boyle & Loveland, 2003). While the field of substance abuse treatment has several studies looking at step down or continuing care, we know of no previous protocols focusing on post-discharge recovery management checkups for the explicit purpose of monitoring and early re-intervention per se.

While relatively new in behavioral health, models of on-going monitoring and early re-intervention occupy a central role in the long-term management of other chronic medical conditions (Dubar-Jacob, Burke, & Puczynski, 1995; Engel, 1977, 1980; Nicassio & Smith, 1995; Roter et al., 1998). Some of the common components of these models include (a) proactively tracking patients and providing regular ‘checkups’, (b) screening patients for early evidence of problems, (c) motivating people to maintain or make changes—including the return to more aggressive treatment when necessary, (d) assistance negotiating access to additional formal care and potential barriers to it, and (e) an emphasis on early formal re-intervention when problems do arise. The core assumption of these approaches is that earlier detection and re-intervention will improve long-term outcomes.

In developing a ‘Recovery Management Checkup (RMC)’ for chronic substance users, the protocol needed to overcome several behavioral problems frequently observed in this population. First, substance use is associated with a clandestine, chaotic and transient lifestyle that makes tracking difficult (Bale, 1984; Brown, Ridgely, Pepper, Levin, & Ryglewicz, 1989; Goldstein, Abott, Paige, Sobel, & Soto, 1977; MacKenzie et al., 1987). Second, we wanted to use a brief screener to identify the eligibility and need for re-intervention, but were also concerned that people might under-report or have difficulty reporting accurately because of a wide range of co-occurring problems. Third, we recognized the need for a motivational intervention to help prepare individuals to return to treatment, as well as assistance with individual, social and organizational barriers to accessing treatment. Our objectives were to overcome each of these challenges and, in so doing, to improve long-term outcomes. The purpose of this article is to briefly summarize the RMC protocol and report the results of an experimental evaluation of its relative effectiveness.

2. Materials and methods

2.1. Overview of design and hypotheses

The Early Re-Intervention (ERI) experiment evaluated the ability of a public health model of monitoring and early re-intervention to accelerate the identification and return to treatment after relapse. Specifically, we recruited 448 adults sequentially presenting to the largest central intake unit in Chicago between 2/00 and 4/00. Regardless of any prior treatment episodes, this treatment was considered the ‘index’ episode of care. At three months participants were randomly assigned to either quarterly interviews (an attention control group) or quarterly interviews with our Recovery Management Checkup (RMC) protocol (Scott, Dennis, Foss, & Hristova, 2003). In both condition, participants were located quarterly (averaging 94–96% per wave) for 2 years using our standardized follow-up protocol (Scott, under review) and interviewed with the GAIN-M90 (Dennis, 1999). Every quarter, participants assigned to the RMC condition were also screened for relapse while living in the community and, if appropriate, received additional early re-intervention that included personalized feedback with motivational interview techniques, identification and resolving of barriers to treatment, and linkage assistance to set up and keep readmission appointments. At 24 months we also did a 1–3 day test-retest study (n = 75) after the last interview and tested urine/saliva samples for biomarkers (n = 308) to assess the reliability and validity of the self reported data.

We hypothesized that relative to the participants assigned to the control group, participants assigned to RMC would:

• H1. return to treatment sooner after discharge from the index episode of care,

• H2. receive treatment on more total days after discharge from the index episode of care, and

• H3. be less likely to be in need of further treatment during the subsequent 24 months both in terms of (a) total quarters of need and (b) need at month 24.

Below is more detailed information on the intervention, instruments and measures, participants and procedures.

2.2. Interventions

Index episode of care

The index episode of care (immediately following the baseline interview) lasted an average of 27 days (with 11% still in treatment at 90 days). Approximately 60% of the participants received residential treatment and 40% outpatient. The treatment was provided by Haymarket Center, which is one of the largest substance abuse treatment provider in the state and which specifically operates programs for clients who are mentally ill substance abusers, pregnant and post-partum women, and/or homeless. The program is accredited by Medicaid, the State of Illinois and the Committee on the Accreditation of Rehabilitation Facilities (CARF). Diagnosis is based on DSM-IV (American Psychiatric Association, 1994) and placement is based on ASAM (1996) patient placement criteria. There were no significant differences by condition in treatment received during the index episode of care (which preceded randomization).

Control condition

Participants assigned to the ‘attention’ control condition were interviewed at baseline and scheduled for quarterly follow-ups during the next 2 years. The majority of interviews were face-to-face and conducted on site at the research office. The interview required approximately 30–45 min to complete, and all on-site interviews were audio taped for purposes of quality assurance. Once the assessment was completed, the research assistant updated the locator information and scheduled the next appointment. Referrals to treatment were provided only in emergency cases (less than a dozen times during over 1800 interviews).

Recovery management checkup (RMC) condition

Like the control group, research staff conducted quarterly interviews with participants assigned to the RMC condition. To minimize demand characteristics and contamination, different staff members were used for the interviews and the actual RMC. While it was impossible to keep staff blinded (interviewers had to send RMC clients to the linkage manager), the interview staff knew little about the experiment, every interview was audio taped, multiple biological tests were run to check for any bias, and both sets of staff were trained and under the supervision of the research staff. The RMC protocol (Scott & Dennis, 2002), including the forms and performance/adherence measures, is available on line (www.chestnut.org/li). Briefly, it involves the following steps: (1) determine ‘Eligibility’ (i.e. verify that the person is not already in treatment or jail and is living in the community), (2) determining ‘Need’ for Treatment based on self report (discussed further below), (3) transfer the participant to the Linkage Manager, and (4) Linkage Manager completes the intervention. The intervention utilized motivational interviewing techniques to: (a) provide personalized feedback to participants about their substance use and related problems, (b) help the participant recognize the problem and consider returning to treatment, (c) address existing barriers to treatment, and (d) schedule an assessment. The Linkage Manager provided both the motivational interviewing and linkage assistance for those individuals who agreed to participate. Linkage assistance often included scheduling, reminder calls, transportation, and in some cases being escorted to the intake appointment. In any given quarter, 60–80% of the participants self reported being eligible (i.e. in the community and not in treatment), and25–35% reported being ‘eligible and in need’ of treatment (based on definition presented earlier). From any given quarter to the next, the ‘needed for treatment’ status changed for 25% or more of the participants. Sessions were audio taped and the results of the screener (to determine eligibility and need), motivational interview and linkage assistance were all documented on the RMC worksheet; both were reviewed by the protocol supervisor for staff adherence and used to generate performance measures to monitor the protocol's implementation overall.

2.3. Instruments and measures

Data were collected from several sources including participant interviews, urine tests, saliva tests, and various process measures. Below is a short summary of each.

Global appraisal of individual needs

The participant characteristics and primary outcomes were measured with the Global Appraisal of Individual Needs (GAIN) (Dennis, 1999; Dennis, Titus, White, Hodgkins, & Unsicker, 2002)—which is a comprehensive, structured interview that has eight main sections (background, substance use, physical health, risk behaviors, mental health, environment, legal, and vocational). The GAIN has over 100 scales, with the main ones having alphas over 0.9 in this data set and the subscales generally having alphas over 0.7. In both this and other data sets, the pathological symptom counts produce a stable four factor solution (substance problems, internal distress, external behavior problems, crime/violence) (Dennis, Scott, Lennox, Funk, & McDermeit, under review). Diagnoses based on the GAIN have been shown to have good test-retest reliability for substance use disorders (kappa = 0.6; Dennis et al., 2002) and to accurately predict independent and blind staff psychiatric diagnoses of co-occurring psychiatric disorders including ADHD (kappa = 1.00), Mood Disorders (kappa = 0.85), Conduct Disorder/Oppositional Defiant Disorder (kappa = 0.82), Adjustment Disorder (kappa = 0.69), or the lack of a non-substance use diagnoses (kappa = 0.91) (Shane, Jasiukaitis, & Green, 2003). The test-retest data were also generally very good (0.7–0.9) on both the scales and individual questions. The values for the core dependent values are presented below.

Other self report measures

The GAIN was supplemented with several other self report measures, including (a) a study specific Participant Screener Form (PSF) that was used by the treatment staff on all sequential intakes to determine eligibility. Information was collected on demographics, frequency of use, and other inclusion/exclusion criteria and to document whether they agreed to participate in the study, and their index treatment assignment, (b) a variation of the Texas Christian University (TCU) treatment motivation scales (Knight, Holcom, & Simpson, 1994; Simpson & McBride, 1992; Sampl & Kadden, 2001) in which the questions were reorganized by subscales and integrated with similar items from the GAIN to simplify the RMC feedback session, and (c) (for the RMC group only) an RMC worksheet that included a short screener to determine the eligibility and need for RMC and, if applicable, document the linkage intervention. Across the eight follow-up observations, the two-minute RMC screener was consistent with the longer GAIN in terms of both eligibility (87% agreement, 87% sensitivity, 88% specificity, kappa = 0.70, p < 0:0001) and need (87% agreement, 74% sensitivity, 96% specificity, kappa = 0.72, p < 0:0001). Where there were differences, however, the longer assessment clearly identified more people in terms of both eligibility (65–71%, McNemar $X_{(1)}^2=47.7$, p < 0:0001) and need (30–37%, McNemar $X_{(1)}^2=61.4$, p < 0:0001).

Biological markers

To check for potential under reporting of substance use and differential demand characteristics, we also checked the validity of the terminal outcome (drug use at 24 months) with urine and saliva testing on all in-person interviews. Urine samples were checked for color and temperature, refrigerated, and then shipped overnight to a SAMHSA NLCP certified laboratory, MedTox (www.medtox.com). The laboratory conducted screening using kinetic interaction of microparticles in solution (KIMS) methodology at the SAMHSA standard cutoff levels (see www.drugfreeworkplace.gov/frames/frame_drugtest.htm) for a panel of five drugs: cannabinoids (marijuana/THC; 50 ng/ml); cocaine (300 ng/ml); amphetamines (1000 ng/ml); opiates (2000 ng/ml); and phencyclidine (PCP; 25 ng/ml). The laboratory also tested for adulteration by checking creatinine levels (less than 20 ng/ml suggests adulteration or high levels of kidney hydration), and if below the threshold, the specific gravity (less than 1.003 suggests dilution). (Two cases were discarded because of validity check problems). Saliva tests were done on-site with ORAL· Screen ™ 4 tests for THC, opiates, cocaine/crack and methadone (as well as a validity check) using a lateral-membrane immunoassay technique at the SAMHSA recommended cut offs. (While a dozen on-site tests had to be redone, the final test was always acceptable) Both approaches replicate well relative to gas chromatography/mass spectrometry (GC/MS) and will detect over 90% of use in the past two days, as well as some earlier use during the past 1–4 weeks.

Comparison of biomarkers

We included laboratory based urine tests as they are the most common validation tool in drug treatment research. However, urine tests are not without some significant problems as measures of change. It takes a few hours for very recent use to show up in urine samples, creating the potential for false negatives (i.e. negative urine when there was use). At the other extreme urine tests are imperfect measures of the recency of use. While tests for opioids, cocaine, and marijuana are accepted as reliable measures of use that occurred in the past 2–3 days and to a lesser extent 4–7 days ago, a subset of the people continue to show up positive for 30 or more days (Cone & Weddington, 1989; Ellis, Mann, Judson, Schramm, & Taschian, 1985), which is a false positive relative to the 48 h or past week use conversion that is often used in the literature. Handling delays in actually testing urine samples (even when refrigerated) have also been suspected as a source of the false negative rates in laboratory tests (relative to self report or GC/MS). In fact, Visher (1991) empirically estimated the error rate various urinalysis technologies relative to GC/MS on cocaine, opioids and marijuana; the best tests had false positive rates of 0.1–4.1% and false negative rates of 17.5–40.8% (see Foss, Dennis, and Scott (under review)). With a total error rate average 20%, this means the upper bound of any self report to urine test comparison would be around Rho = 0.8. Thus we were interested in seeing if an on-site test focusing on more recent use might improve our coverage. Towards this end we supplemented the urine test and self reports with on-site saliva tests to focus more explicitly on recent use (e.g. past 48 h, particularly the first 1–3 h before drugs show up in urine) and to provide a check on handling issues that can lead to false negatives (e.g. decay due to temperature exposure or delays).

Consistent with our expectations, these two biological measures were indeed different at 24 months—with the rates of agreement decreasing in the order of how long each substance takes to metabolize in urine samples. The results were moderately consistent for opioids (15% saliva only, 14% both, 5% urine only, 66% neither; kappa = 0.44, p < 0:05) and cocaine (11% saliva only, 38% both, 13% urine only, 38% neither; kappa = 0.52, p < 0:05), but very different for marijuana (8% saliva only, 2% both, 15% urine only, 75% neither; kappa = 0.04, n.s.d.) where THC is both slow to show up in urine and can be very slow to leave. In spite of refrigeration, the rates of false negative urines from laboratory tests (relative to on-site saliva) went up several fold as the total days from sample collection to testing went from 2 to 3–7 days to 1–4 weeks for cocaine (4–5–50%), opioids (7–33–50%), and marijuana (33–44–100%).

Check on validity of self reports

To assess the validity and potential bias of our substance use measures we looked at prevalence based on each method, prevalence across methods (i.e. either self reported use in the past month or positive urine or positive saliva), the rate of false negatives overall and by condition, and the kappa between each method and the combined estimate. While the prevalence rates by self report (56%), urine (62%) and saliva (57%) were not significantly different, they were not always produced by the same people and each was substantially lower than the combined estimate (76%). Relative to the combined estimate, urine appeared to miss the least people for cocaine (12% false negative), saliva for opioids (6% FN), and self report for marijuana (12% FN). For ‘any use’, each method was largely consistent with the combined estimate (Kappa of 0.59 for self report, 0.69 for urine, and 0.56 for saliva) and there was no significant difference in these Kappa by condition. Consistent with this data andrecommendations from the literature (Cone & Weddington, 1989; Kranzler, Tennen, Babor, Kadden, & Rounsaville, 1997), we based our final estimate of need on the combined data.

Missing data

One of the primary threats to the validity of randomized field experiments is missing data (Dennis et al., 2000; Figueredo, McKnight, McKnight, & Sidani, 2000; Hedeker & Gibbons, 1997; Little & Rubin, 1987). The state of the art in this area is to avoid (a) list wise deletion (which biases both the mean and variance estimates) and (b) mean or median replacement (which still biases the variance estimate). To do this, three things (in order of preference) are consistently recommended: (a) minimize attrition, (b) replace data within subject where correlated observations (e.g. repeated measures) or variables (e.g. scales) are available possible or (c) replace data across subjects with some kind of random noise (e.g. hot deck imputation) or via a regression formula. As will be discussed further below, we were very successful in minimizing attrition (100% have 1 or more follow-up interviews, including 95–97% per wave and 82% with all 8 wave). Because we have eight observations per person on the same measures and these measures either overlap (e.g. a treatment history) or are highly correlated over time (Cronbach alpha of 0.85 or higher), replacement feasible and done within individuals. The specific method for each dependent variable is described below after its definition.

Key outcomes

The first key outcome, ‘Time to First Re-admission to Treatment,’ was calculated two ways. First, it was calculated as the days from each interview to the next subsequent admission. Second it was calculated once across interviews as the days from the index episode of care discharge (or the point of random assignment at the 3 month interview) and the next subsequent admission to a formal substance abuse treatment program. Observations were censored at 90 days for the quarterly measure and 630 days (21 months) or the days to the last completed interview across waves. In both cases did not include self-help groups, recovery homes and detoxification in this outcome measure (though we do have measures of them). The complete treatment history was updated at each interview (including an periods covered by a missed interview) so there was no missing data through the last interview completed. At 24 months, the test-retest reliability was very good for both the time to first readmission (Rho = 0.88) and for any admission (Kappa = 0.81).

The ‘Total Days Received Treatment,’ the second key outcome measure, is based on the sum of days an individual received outpatient, intensive outpatient, residential, or inpatient treatment reported at each interview for the prior quarter. Note that this is the days of actual contact, not the duration between intake and discharge. For any given quarter, the number of days was capped at 90. If someone was missing an interview, the missing value was replaced based on the average of the other follow-up periods (see discussion above on the rationale of this approach). Again, there were two versions—one for each individual quarter and a second one across quarters. At 24 months, the test-retest reliability was acceptable for the days receiving formal treatment in a single quarter (Rho = 0.66) and excellent for the total days receiving formal treatment across quarters (Rho = 0.96).

The ‘Need for Treatment,’ the third key outcome was calculated two ways. The need to implement the RMC protocol for any given quarter and the number of quarters in need were based only on self-reported information. For the final outcome status, we combined this with additional biomarker data. Both measures defined someone as ‘in need of treatment’ if they were living in the community (vs. in jail or other controlled environment), were not already in treatment, and answered yes to any of the following questions: (a) During the past 90 days, have you used alcohol, marijuana, cocaine, or other drugs on 13 or more days? (i.e. about weekly use) (b) During the past 90 days, have you gotten drunk or been high for most of 1 or more days? (c) During the past 90 days, has your alcohol or drug use caused you not to meet your responsibilities at work/school/home on 1 or more days? (d) During the past month, has your substance use caused you any problems? (e) During the past week, have you had withdrawal symptoms when you tried to stop, cut down, or control your use? (f) Do you currently feel that you need to return to treatment? These criteria for need are internally consistent (alpha = 0.85) and the average person in ‘need’ endorsed 3.3 of 6 of the items (80% endorsed 2 or more).

For the 24-month measure of need only (i.e. the final outcome), we also included people who answered no to a-f, but were positive on either of the biomarkers (i.e. urine or saliva test). At 24 months, the test retest reliability on the self reported need for treatment was very good (Kappa = 0.78). While we used separate staff for interviewing and the RMC intervention, over time differential demand characteristics remained a potential threat to the study's validity (e.g. people becoming more honest in reporting; people denying use to avoid the intervention). To assess this possibility, we selected the subset of people who were in need of treatment at 24 months and looked at whether this was determined by self-report, biomarkers, or both. Importantly, there was no significant difference or even marginal evidence of such a bias.

The changing ‘pattern of need,’ the fourth key outcome was created as an 8-digit variable with each digit set to either 1 (eligible and in need of treatment) or 0 (not eligible and in need of treatment) according to their status at each of the 8 follow-up. To simplify the 2⁸ = 258 possible patterns, we then collapsed across runs (e.g. 00100000, 00010000, and 00110000 were all considered ‘not in need-in need-not in need’) to focus on the pattern of change. The majority (61%) of the participants had only 0–2 changes that could be categorized in one of five patterns. The rest could be categorized into either a pattern of intermittent need ending ‘in need’ or ending ‘not in need of treatment’—bringing the total to the seven patterns of need presented below in Section 3. At 24 months, the Kappa on the pattern of need variable was excellent (Kappa = 0.93).

2.4. Participants

Eligibility criteria

To be included in the study, individuals needed to: (a) meet lifetime criteria of substance abuse or dependence, (b) have used alcohol or other drugs during the past 90 days, (c) complete a central intake unit (CIU) assessment and receive a referral to substance abuse treatment at the collaborating treatment agency, Haymarket Center in Chicago, and (d) be 18 years of age or older. Logistical constraints in providing the RMC intervention required that individuals be excluded if they (e) did not reside in the City of Chicago, or (f) did not plan to reside in the City during the ensuing 12 months, or (g) had been sentenced to jail or prison or a DUI program for most of the upcoming 12 months, or (h) were unable to use English or Spanish, or (j) were too cognitively impaired to provide an informed consent. Participation was voluntary after an informed consent process under the supervision of Chestnut's Institutional Review Board on Human Subjects and the study was conducted under the protection of a Federal Certificate of Confidentiality issued by the National Institute on Drug Abuse.

Recruitment

Of the 796 individuals who presented for an assessment during the 3-month recruitment period (February to April, 2000), CIU clinical staff completed a participant screening form for 786 (99%) individuals. Of the 786 individuals, 533 (68%) met the eligibility criteria. The primary reasons for ineligibility were residing outside the city (N = 115) or planning to move outside the city in the next 12 months (n = 73). Over half of the people excluded were ineligible for multiple reasons. Of the 533 eligible participants, 448 (84%) completed the baseline interview and agreed to participate; 8% (n = 41) could not stay to complete the baseline interview and were not recaptured, and 8% (n = 45) refused to participate in the study.

Characteristics

Of the 448 people randomized, 59% were female, 85% African American, 8% Caucasian, 7% Hispanic, and 2% Other; 2% were between the ages of 18 and 20, 17% between 21 and 29, 47% between 30 and 39, 28% between 40 and 49, and 5% were 50 or older. All met criteria for lifetime dependence at the time of intake, including 7% for Alcohol only, 20% for Cocaine and Alcohol, 29% for Cocaine only, 8% for Cocaine and Opioids, 14% for Opioids only, and 17% for other patterns. The participants were initially referred to outpatient (7%), intensive outpatient (26%), short-term residential (45%), long-term residential (19%), or detoxification/other (2%). The vast majority (77%) reported additional co-occurring mental health problems including overlapping subgroups of Major Depression (61%), Generalized Anxiety Disorder (60%), Conduct Disorders (37%), or Attention Deficit Hyperactivity Disorder (34%), as well as a subgroup of 5% without these specific disorders, but who reported high rates of internal symptoms (somatic, depression, homicidal/suicidal, anxiety, and trauma related complaints). Over 26% reported health problems that bothered them daily or interfered with their responsibilities weekly. In addition, 25% of the females reported being pregnant in the past year. In terms of HIV risk behaviours in the past year, 6% reported needle use and 86% were sexually active; the latter includes 62% reporting unprotected sex and 42% reporting multiple sexual partners in the 90 days prior to intake. The participants also reported a wide range of other environmental issues related to recovery including weekly substance use by others in the home (40%), regular peer substance use (84%), being vocational engaged in school (5%) or work (32%), criminal justice system involvement (27%), spending time in an inpatient unit, jail or other controlled environment (64%), a history of physical, sexual or emotional abuse (75%), and a history of homelessness (54%, including 26% homeless at intake).

2.5. Procedures

Random assignment

After everyone was recruited, research staff used a list of participant IDs (in order of recruitment) and random numbers generated by Microsoft Excel™ to randomly assign each participant to either the control group or the RMC group. Post hoc analyses revealed no significant differences between participants in the two groups on 67–69 (97%) demographic, family, social, environmental, substance, health, mental health, and HIV risk variables. Though less than expected by chance, participants assigned to the control group were more likely than those assigned to RMC to be dependent on alcohol (46–33%, $X_{(1)}^2=6.9$, p = 0:009) and to initially be assigned to detox (6–2%, $X_{(1)}^2=5.8$, p = 0:028). No incidence of intervention crossover or contamination was detected throughout the study. Thus, overall we deemed the randomization process to be successful.

Assessment and follow-up procedures

All assessments were done by independent research staff trained and under the supervision of the research team. Once CIU assessors completed the intake assessment procedures and deemed a person eligible for the study, they physically transferred the study candidate to a research assistant (RA) housed on site. At this point, the RA reviewed with the individual the study requirements for participation (quarterly follow-up interviews) and the informed consent. Individuals who refused to participate were thanked for their time and transferred back to the CIU staff. Upon completion of the baseline interview, the RA scheduled the date and time for the 3-month follow-up interview, informed the participant about the location of the interview, and provided a schedule card. Each quarterly follow-up mirrored these procedures.

Upon completion of the first quarterly interview, participants received a total of $60–$30 for the baseline and $30 for completing the follow-up interview. For the remainder of the quarterly follow-ups, an individual received $30 for completing the interview plus a $5 bonus if it was completed within plus or minus 7 days of the anniversary date. Of the 448 people randomized, 100% completed 1 or more follow-ups, with the rates varying from 94 to 97% per wave and 82% completed all eight follow-up interviews over the 2-year period. Furthermore, 80% of the participants completed their follow-up interviews within plus or minus one week of their quarterly anniversary date.

During the 24-month follow-up interviews, members of the research team also collected urine and saliva samples. Participants were reimbursed $15 for providing both samples. Of the 421 people completing 24-month interviews, 333 (79%) provided saliva samples and 314 (75%) provided urine samples (303 provided both); most of the remainder were not attempted because the interviews were done by phone (49; 12%) or in jail (17; 4%). The remaining people either did not provide a valid urine or saliva sample (either because they were unable or because the control suggested the sample was invalid [n = 2; creatinine level <20 on urine sample]) or no sample was collected because the interview was done off site.

Analytic procedures

All analyses were done with SPSS (2002) using an ‘intent to treatment’ model (Dennis et al., 2000), meaning that (1) people were analyzed as randomly assigned, regardless of whether they needed or actually received RMC services, and (2) for the small amount of missing data by wave we assumed that status had not changed from the prior wave. The time to first readmission analysis used Cox Proportional Hazards Regression, treating people who had not returned to treatment by their last interview as right censored (42%). For the days of treatment received, we summed across all types of treatment (capped at 90 days per quarter). When a given quarterly interview was missed for someone, we replaced it with the average of the other quarters between 3 and 24 months. For the final outcome we used the 24-month or last-known status. For the analysis of patterns over 24 months we assumed that the status was unchanged from the prior quarter if an interview was missed. Analyses were done both with and without covariates for the two variables where there were significant differences in client characteristics by condition (dependent on alcohol, initial assignment to detox) and one requested by a reviewer (dependent on alcohol-only, drugs-only, both). None of these variables turned out to be significant or to impact the results, so they are not reported here.

3. Results

3.1. General pattern of change

Between intake to index episode of substance abuse treatment and the 3 month interview (the point of randomization), there was a statistically significant decreases in the days of using any substance (46–20 days), cocaine (26–8), alcohol (20–12), alcohol to intoxication (12–4), opioids (14–5) and marijuana (9–4), as well as increases in the percent totally abstinent (0–35%) and without past month abuse/dependence problems (13–53%). The ‘mean’ days and problems did not change significantly (over all or by group) from months 3 to 24. This stability, however, was an ecological illusion because an average of 32% of the people significantly changed their patterns of use, treatment and recovery each quarter. Below we will examine the impact of RMC on this cycle both within and across quarters.

3.1.1. Relative to the control group, participants assignedto RMC will return to treatment sooner after the indexepisode of care

After discharge from the index episode of care, RMC participants were significantly more likely than control group participants to return to treatment sooner (Median of 376 vs. 600 days; on the whole distribution in Fig. 1 Wald = 5.2, p < 0:05) and to return to substance abuse treatment at any point during the follow-up period (64–51%, Odds Ratio of 1.65 [95% CI:1.13–2.41], $X_{(1)}^2=6.8$, p < 0:01). Notice that the difference between the two groups grows over time as increasingly more people experience more quarters of needing and/or receiving RMC. This dramatic reduction in the time to re-admission and improvement in the likelihood of returning in the first two years is particularly important given the results of earlier work (Simpson & Savage, 1980a,b; Simpson, Joe, & Bracy, 1982) suggesting that people who do not initially respond to treatment will continue to deteriorate until they return to treatment.

Fig. 1.

Time to readmission to treatment by condition.

3.1.2. Relative to the control group, participants assigned to RMC will receive treatment on more total days after the index episode of care

Relative to participants assigned to the control group, RMC participants received treatment on significantly more days after the discharge from the index episode of care (mean of 62 vs. 40 days, t₍₃₉₀₎ = 2:65; p < 0:05). Even if we just compare the subset of people who received 1 + days of treatment, the mean days of subsequent treatment was significantly higher for participants assigned to RMC than for those assigned to the control group (mean of 100 vs. 74 days, t₍₂₅₃₎ = 3:61; p < 0:05). They were also more likely to receive subsequent treatment on 90 or more days during months 4–24 (25 vs. 17%, Odds Ratio of 1.61 [95% CI: 1.02–2.57], $X_{(1)}^2=4.1$, p < 0:05). This is important, as previous work suggests that 90 days is a key threshold for treatment effectiveness (Hubbard et al., 1989; Simpson et al., 1979, 1997a,b,c, 1999) and that the total amount of treatment across episodes of care (vs. just the index episode) may be the more important predictor of long-term outcomes (Hser et al., 1998; Simpson, Joe, & Broome, 2002).

3.1.3. Relative to the control group, participants assigned to RMC will be less likely to be in need of further treatment during the subsequent 24 months both in terms of (a) total quarters of need and (b) need at month 24

Based on self-report measures alone, 78% of all participants were eligible and in need of additional treatment in one or more quarters, with a mean of 3.1 of eight quarters in need and 33% self reporting need at 24 months. Expanding the need for treatment to include positive biomarkers (i.e. urine or saliva) increased the percent in need at 24 months to 49% across conditions (discussed further below by condition). To evaluate the impact of RMC on the need for treatment we compared the number of quarters in need (based solely on self report as this is the only time the RMC protocol would be used), the need for treatment at 24 months (based on self report or a positive biomarker), and then combined pattern of need to indicate their trajectory. The results are in Table 1 and summarized below.

Table 1. Impact of RMC-ERI on long term outcomes and trajectories.

Variable	Recovery management checkup (n = 224) (%)	Control (n = 224) (%)	Total (%)	Odds ratio (95% CI)	Statistic
Quarters of needing treatmenta
0–1 times	36	37	36	0.96 (0.65–1.41)	$X_{\left(2\right)}^2=6.6$, p < 0:05
2–4 times	41	31	36	1.56 (1.06–2.30)
5–8 times	23	32	28	0.63 (0.41–0.96)
Final status at month 24b
Need for treatment readmission	43	56	50	0.58 (0.39–0.85)	$X_{\left(1\right)}^2=7.7$, p < 0:01
24 Month pattern of needing treatmentc
Initial and sustained ‘No need’	4	4	4	1.12 (0.45–2.80)	Z = 2:4; p < 0:05
Need–‘No need’	9	7	8	1.37 (0.68–2.74)
No need–Need–‘No need’	33	26	29	1.35 (0.90–2.03)
Intermittent need ending in ‘No need’	11	7	9	1.63 (0.85–3.15)
No need–‘Need’	12	14	13	0.79 (0.45–1.37)
Intermittent need ending in ‘Need’	25	33	29	0.68 (0.45–1.02)
Intitial and sustained ‘Need’	6	8	7	0.71 (0.34–1.48)

^aNumber of eight possible quarters in which people were eligible (currently living in the community and not already in treatment) and in need of treatment at 24 months based on self report (any past month abuse/dependence sysmptoms, use interfering with responsibilities, weekly use or past week use).

^bPercent who were eligible and in need based on the above criteria at the time of the 24 month interview.

^cBased on quarterly pattern of need and final status from above; Z based on Mann–Whitney U.

Relative to control participants, RMC participants experienced fewer quarters during which they were eligible and in need of treatment ( $X_{(2)}^2=6.6$, p < 0:05); the main difference coming in a shift in the percent with ‘5–8 quarters of need’ (23 vs. 32%; Odds Ratio = 0.63) to the percent with ‘2–4 quarters of need’ (41 vs. 31%; OR = 1.56). The lack of difference (36 vs. 37%; OR = 0.96) in the percent with ‘0–1 quarters in need’ is to be expected since without at least 1 quarter there would be no RMC intervention or differences between the randomly assigned groups.

At 24 months, the RMC participants were also significantly less likely than control participants to need treatment based on self report and/or biomarkers (43 vs. 56%; $X_{(1)}^2=7.7$, p < 0:01). When assessing the changing ‘patterns of need’ over the 24 months, participants assigned to RMC were significantly more likely than control participants to demonstrate one of the positive trajectories that occur at the top of this list (Z = 2:4; p < 0:05). There is a prominent pattern of more RMC people who (a) first responded to treatment, needed to return, improved and maintained the improvement (33 vs. 26%; Odds Ratio 1.35) or (b) went in and out of being in need multiple times, but ended without being in need of treatment (11 vs. 7%; OR = 1.63), as well as less people who relapsed and were subsequently in need of treatment during every quarter (12 vs. 14%; OR = 0.79) or who were intermittently in need and ended in need of treatment (25 vs. 33%; OR = 0.68).

3.2. Quasi-experimental outcomes

To be true to a randomized design (see Dennis, 1988; 1994; Dennis et al., 2002), it is necessary to at least initially compare people exactly as randomized (as we have done above). For policy and program planning, however, it is also useful to quasi-experimentally look at the effects for the most relevant subset of people. To do this, we subsetted the data to the 1123 observations of people who entered the quarter in need of treatment (the target population for the RMC intervention) and examined what happened to them over the next 90 days. As one would expect, using this more liberal method all findings are in the same direction and larger. RMC participants were significantly more likely than control participants to return to treatment sooner (Wilcoxon–Gehan = 26.2, p < 0:0001), to return to substance abuse treatment at any point during the quarter (64 vs. 51%, Odds Ratio of 1.65 [95% CI:1.13–2.41], $X_{(1)}^2=6.8$, p < 0:01), and to have more average days of substance abuse treatment (7.75 vs. 4.68 days, Z(_Wilcoxon) ⁼ -4:12; p < 0:0001).

4. Discussion and conclusion

Proactive checkups to monitor and facilitate early re-intervention are increasingly recognized as an essential component for the effective long-term management for many chronic disorders (Health Care Management, 1997; McLellan et al., 2000; Nicassio & Smith, 1995; Roter et al., 1998; Wheatly, 2002; White et al., 2003). This study tested the effectiveness of a ‘recovery management checkup’ or RMC specifically designed for people with chronic substance use disorders. Our first objective was to track people in spite of the transient, chaotic and clandestine lifestyle that frequently accompanies addiction. Implementing our follow-up model (Scott, under review), with 100% followed up over the 2-year period (94–97% per wave and 82% completing all eight follow-up interviews), we have clearly demonstrated the feasibility of meeting this first objective.

The second objective was to see if we could identify people in need of further treatment through a relatively brief screener incorporated into the RMC contact log. Even after eliminating people who were out of the program's catchment area, already in treatment, and incarcerated; 25–35% met criteria for being in need of treatment in any given quarter (including at 24 months). If we consider the additional information in the full assessment this percentage rose to an average of 37% per quarter and when we include biomarkers in month 24, it increased to 49%. This clearly demonstrates the ‘need’ for on-going monitoring and suggests that we may need to extend the monitoring period even longer. Our one concern here was that the screening questions in the log missed several people who were identified in the more detailed assessment with the GAIN, saliva and urine testing. In the future we hope to extend the monitoring period and supplement the RMC screener with further questions and on-site saliva or urine testing.

The third objective was to link people in need back to treatment sooner and with greater certainty. On the upside, we significantly reduced the median time to readmission by 37% (600–376 days), increased the return to treatment rate (64 vs. 51%), and increased the average total time in subsequent treatment (mean of 62 vs. 40 days). On the down side, 2 out of 3 people who needed to return to treatment based on the RMC criteria did not successfully do so, and only 1 in 4 completed 90 days of subsequent treatment across the 21 month follow-up period. Thus there was still clearly room for more improvement, both in linkage and retention after readmission.

The fourth and final objective was to impact long-term outcomes. As hypothesized, participants assigned to RMC experienced significantly fewer quarters in need, better trajectories, and were less likely to be in need in the final quarter even after we included all data from self reports and biomarkers (43 vs. 56%). While better, there is still considerable room for further improvement. This also suggests the need for an RMC-type intervention to continue well past the 24 months involved in this study. This is consistent with natural history studies suggesting that reaching recovery is likely to take 3–4 admissions and a median of 8 years from first treatment (Anglin et al., 1997; Dennis et al., 2002; Hser et al., 1997, 1998, 2001).

This experiment had both strengths and limitations. Some of the key strengths included a large sample size, quarterly checkups over a long period of time, extremely high follow-up rates, a comprehensive standardized assessment covering a wide range of biopsychosocial domains, the inclusion of multiple biomarkers, and a manualized intervention with explicit performance (aka adherence) measures. While many experiments are criticized for creaming or excluding difficult cases, this study included a broad cross section of clients including a sizable number of relatively high severity cases. Although this recovery management model was successful overall, there were several limitations that we hope to address in the future. First, we believe that the RMC screener missed several people who might have benefited (as evidenced by the self report on the more detailed assessment and the results of the biomarkers). Second, in response to problems in the first 9 months, we modified the original protocol by adding more performance monitoring, making the intervention more culturally sensitive, and by providing more transportation to appointments (in some cases even escorting participants to their intake assessment). While these changes improved linkage rates during the later five quarters of the study, the improvements were confounded with time from intake and need to be replicated. Third, while linkage rates to intake and treatment improved over time, there were no assurances that once participants showed to treatment they would engage in treatment. Ideally, there would have been someone on the treatment side to facilitate their engagement and retention.

4.1. Implications for policy and practice

It is time for a major paradigm shift in both policy and practice related to the long term care of people with substance use disorders. Natural history studies have consistently demonstrated that there is a core group of chronic substance users who are cycling through periods of use, treatment, recovery and relapse that occur over a period that takes 8 years and 3–4 admissions on average. Economic studies have consistently demonstrated that this ordeal of use and relapse is associated with real social costs associated with crime, health problems, pregnancy, service utilization, and employment, as well as a personal costs in terms of quality of life. In the 21 months after discharge from the index episode of care, four out of five clients where found to be in need of treatment at one or more of their quarterly checkup (the median was 3 times). In the treatment as usual control group, less than 1 in 10 in need of treatment actually returned within the next 90 days. While far from perfect, the RMC intervention doubled the percentage of people getting into treatment in this first 90 days after detection and reduced the median time to readmission by over 7 months. Clearly it is time to shift our focus from acute episodes of care to long term recovery management across episodes of care.

At the program level, this means we need to be much more assertive in our approaches to continuing care and post-discharge follow-up. Like other chronic conditions, we must develop more realistic expectations and begin to think in terms of on going check-ups to detect relapse. We must stop placing the entire burden of returning on the client. We must recognize that people returning to treatment may need their own track or different approaches.

At the policy level, we need to change systems of care and financial support mechanisms from acute to chronic care models. This includes recognition that these are multi-problem clients who cross over several systems of care and need long term coordinated assistance. It also implies the need for insurance or service codes to cover the cost of recovery management type checkups.

In terms of research, it suggests the need for more longitudinal research that goes out further in time and that is capable of studying the cyclical nature of the treatment-recovery-relapse phenomena. To support the above program objectives it is important to examine the extent to which multi-morbidity interacts with the RMC's effectiveness. To support the above policy objectives it is also important to evaluate the costs and benefits of the RMC intervention and to provide a manual for its replication. For both objectives it is also important to extend the duration of the protocol to 4 or more years to examine if there is a point at which monitoring may be less important (e.g. 1 or 5 years sobriety).