A Practical Clinical Trial of Coordinated Care Management to Treat Substance Use Disorders among Public Assistance Beneficiaries

Abstract

This study tested whether coordinated care management, a continuity of care intervention for substance use disorders (SUD), improved rates of abstinence compared to usual welfare management for substance-using single adults and adults with dependent children applying for public assistance. The study was designed as a practical clinical trial and implemented in partnership with a large city welfare agency. Participants were 421 welfare applicants identified via SUD screening and assigned via an unbiased computerized allocation program to a site that provided either coordinated care management (CCM; n = 232) or usual care (UC; n = 189). Outcomes were assessed for one year post-baseline using self-reports and biological measures of substance use. As hypothesized, for participants not enrolled in methadone maintenance programs (n = 313), CCM clients received significantly more services than UC clients. Non-methadone CCM also showed significantly higher abstinence rates (OR = 1.75; 95% CI = 1.12, 2.76; Cohen’s d = .31) that emerged early in treatment and were sustained throughout follow-up. In contrast, no treatment services or outcome effects were found for methadone maintenance clients (n = 108). Findings suggest that CCM is promising as a wraparound to SUD treatment for welfare recipients.

A recent Institute of Medicine (IOM) report on the state of behavioral health services in the United States found the overall quality of care to be mediocre and noted the absence of evidence-based treatment in clinical practice (IOM, 2006). In addition, the report identified coordination across behavioral health, physical health, and social service systems as one of the major unaddressed challenges affecting quality of care for those with behavioral health problems. The primary aim of the current study was to examine whether coordinated care management (see Figure 2), an innovative intervention designed to increase engagement in substance use disorder (SUD) treatment, enhance linkage to other services, and foster a transition to employment, would improve abstinence outcomes among those with SUDs when compared to usual care for welfare-seeking single adults and adults with dependent children.

Figure 2.

Intervention Characteristics of Study Conditions: Coordinated Care Management (CCM) versus Usual Care (UC)

Coordination of Care Across Welfare Agencies and Substance Use Disorder (SUD) Treatment

Increasing coordination across welfare agencies and SUD treatment may be a particularly effective strategy for improving outcomes for vulnerable populations with SUD (Metsch & Pollack, 2005). SUDs are significantly more prevalent among welfare recipients than the general population (Meara, 2006). Prevalence estimates among mothers eligible for federal welfare or Temporary Assistance for Needy Families (TANF) range from 6% to 37%. The prevalence of SUD among single adults eligible for general assistance is thought to be as high as 40% to 60% (Schmidt, Weisner, & Wiley, 1998).

Subsequent to welfare reform legislation enacted in 1996, over half of states implemented SUD screen and refer programs in their public assistance agencies (SAMHSA & CSAT, 2002). Screen and refer programs contain a number of features identified as important aspects of systems level coordination of care (IOM, 2006), including: early detection of SUD via screening in welfare settings, active linkage mechanisms with treatment agencies, cross agency collaboration and information sharing, flexible use of welfare funds to support treatment, and provision of employment services to foster self-sufficiency (Morgenstern et al., 2003; Morgenstern & Blanchard, 2006). While screen and refer programs represent an important first step in coordination of care, they likely fall short of providing an optimum level of support. Specifically, to the best of our knowledge most welfare agencies refer SUD clients to treatment programs, but do not offer additional continuity of care services to foster engagement in SUD treatment, provide linkage to other needed services, monitor relapse, and assist clients in moving towards self-sufficiency, beyond what is available to other welfare clients.

In a prior effectiveness trial, we compared the benefit of two forms of continuity of care services for drug-dependent TANF mothers in New Jersey. One intervention, Care Coordination, offered minimal care monitoring by specially trained SUD counselors who were funded by the TANF agency. Care Coordination (CC) was the standard of care in New Jersey at the time of the study. The second intervention was intensive case management (ICM). ICM was a substantially more intensive intervention than CC, offering face-to-face case management services over a 24 month period as well as vouchers to reinforce treatment engagement. ICM yielded statistically and clinically significantly better SUD treatment engagement, abstinence, and employment outcomes than CC (Morgenstern et al., 2006; Morgenstern, Neighbors, et al., in press). Importantly, the hypothesized mechanisms of action of ICM were supported in mediational analyses (Morgenstern et al., 2008).

Despite positive findings, the prior trial had a number of limitations. First, although delivered by SUD counselors, ICM was implemented with intensive treatment fidelity monitoring procedures: ICM was manualized, and clinical staff were specially selected, intensively trained, and closely monitored for protocol adherence. Second, the sample was restricted to substance dependent TANF mothers who were not in treatment at baseline. This feature, while important for internal validity, limits the relevance of findings for policymakers because most welfare recipients in SUD screen and refer programs are single adults. Third, most screen and refer programs do not offer SUD-specific continuity of care. Rather, SUD clients receive the same monitoring services as others enrolled in welfare-to-work programs. Welfare monitoring services are provided by welfare case workers who have high caseloads and no special training in managing clients with SUD. Thus, the question of greatest relevance to decision makers is whether any SUD-specific continuity of care is better than none. In the New Jersey study, two different forms of SUD-specific continuity of care were tested.

The Current Study

The primary aim of the current study was to test whether coordinated care management, a SUD continuity of care intervention, improves rates of abstinence among individuals on public assistance identified via SUD screening in welfare settings. Coordinated care management was an innovative approach that focused on engaging clients in SUD treatment, linking clients to needed services, and fostering a transition to employment and training. Rather than solely providing direct case management services to clients, care managers also worked extensively with the agencies providing care to ensure quality and coordination across services. For example, care managers monitored the performance of SUD treatment programs and transferred clients from poorly performing programs to ones that provided higher quality services. Coordinated care management was compared to usual care: monitoring by welfare case workers as part of a routine welfare-to-work program.

Practical Clinical Trials (PCTs)

Despite dramatic advances in the development of efficacious treatments for SUDs (Carroll & Onken, 2005), the utilization of evidence-base practices (EBPs) in routine SUD treatment remains limited. One recent suggestion to foster adoption of EBPs is that more studies be conducted with immediate relevance to clinicians and policy makers (Zerhouni, 2003). The term “practical clinical trial” has been used to characterize this type of study (March, Silva, Compton, Shapiro, & Califf, 2005). PCTs have a number of essential features. The primary purpose of a PCT is to directly inform practice. Thus, questions asked by PCTs must be simple, clinically relevant, and of substantial public health importance. In addition, PCTs should be controlled trials, optimally with random assignment; be conducted under conditions that mirror clinical practice; in samples large enough to detect small to moderate effects; and use simple, clinically relevant outcome measures (March et al., 2005). PCTs differ from large-scale effectiveness trials in their limited use of elaborate quality assurance and research management strategies. For example, effectiveness trials for SUDs have implemented rigorous therapist training and monitoring procedures (Baer et al. 2007; Morgenstern, Morgan, McCrady, Keller, & Carroll, 2001). Such strategies are important to ensure treatment fidelity, but they limit a study’s value to directly inform practice because intensive therapist training and monitoring are very difficult to sustain outside a research context.

The current study was designed as a practical clinical trial. As noted below, the sample was heterogeneous, representative, and large enough to detect small to medium effects. Coordinated care management was delivered by an existing clinical program, supported by welfare funding, and treatment fidelity was assessed using quality assurance methods (March et al., 2005). Assessment methods minimally intruded on routine care and the outcome was simple (abstinence) and measurable in routine clinical care via self-report and biological confirmation.

Study Hypotheses

As described below in detail, coordinated care management was a continuity of care intervention designed to assess and link clients to needed SUD treatment, health, and social services and to manage their care over a 12-month period. Consequently, we hypothesized that individuals in coordinated care management (CCM) would receive more services in these domains than participants in usual care (UC). In addition, we hypothesized that CCM would yield higher monthly rates of abstinence than UC. Finally, we hypothesized that the effects of CCM relative to UC would be weaker for clients enrolled in methadone maintenance at baseline as opposed to those in drug-free treatment programs. We hypothesized a weaker experimental effect for methadone maintained clients because the ability of CCM to impact on the delivery of care for clients in methadone maintenance was more limited than for those in drug-free treatment. For example, an important goal of CCM is to improve treatment engagement. Methadone maintenance clients attend programs regularly in order to receive maintenance medication, regardless of receiving CCM. In addition, care managers had latitude to select appropriate programs for drug-free clients not enrolled in services and to transfer those already enrolled to more suitable placements. However, all clients in methadone maintenance were already enrolled in a program when they entered the intervention, and transfer to another program was difficult and hence unlikely to occur.

Method

Study Overview

Welfare workers at all welfare center offices in Bronx County screened welfare applicants for SUD. If applicants screened positive, workers assigned clients to receive further assessment at one of two assessment centers. One site provided coordinated care management (CCM) services and the other provided usual care (UC). The evaluation used a three-tiered sampling design. The Administrative Sample contained 8986 applicants assigned to study sites during a two-year period and evaluated via welfare administration records documenting welfare activity, employment, and treatment utilization. The Screened Sample contained 1519 representative participants from the Administrative Sample who were briefly assessed by research staff at baseline for current substance use, drug treatment history and motivation, and barriers to employment (see Treatment Assignment and Study Procedures). Characteristics of the Screened Sample are presented elsewhere (Morgenstern, Hogue, Dasaro, Kuerbis, & Dauber, 2008). The Interview Sample contained a targeted subgroup of 421 participants from the Screened Sample who were selected for comprehensive follow-up interviews based on reporting a substance use problem and need for SUD treatment. The current study examines between-condition differences for the Interview Sample in treatment services received and substance use outcomes at one year after baseline (cf Figure 1).

Figure 1.

CONSORT Diagram: Flow of participants through the study

Participants

Demographic, substance use, and other characteristics of the study sample (N = 421) are presented in Table 1. Participants were primarily men (66%) and either African American (49%) or Hispanic (43%). The sample averaged 39.6 years of age (SD = 8.5), and most were not married (91%). Fifty-five percent graduated high school or received an equivalency diploma. Severity and chronicity of substance use were high: at baseline the sample reported using alcohol or drugs on more than half the days of each month, and averaged 9.7 years of regular heavy alcohol use and 10.4 years of regular heroin or cocaine use. About 1 in 5 had unstable living conditions and almost half (46%) were involved in the criminal justice system. The vast majority (81%) had received public assistance prior to their application for benefits at baseline. We examined differences in demographics and substance use variables for the non-methadone versus methadone groups. Overall, clients in methadone treatment were significantly older, more likely to be Hispanic, and had greater chronicity and severity of drug problems, but lower chronicity and severity of alcohol problems. However, there were no significant differences at baseline in percent days substance use or prior treatment episodes. Thus, although clients differed in their use of alcohol versus drugs they appeared to have similar levels of severity and chronicity of substance use problems.

Table 1.

Demographics, substance use, and other issues by treatment: Coordinated Care Management (CCM), Usual Care (UC)

Variable	CCM	UC
N	55% (232)	45% (189)
Demographics
Age (M/SD)	40.2 (8.4)	38.9 (8.5)
Sex (male)	69% (160)	62% (117)
Race/ Ethnicity
African American	52% (120)	44% (81)
Hispanic	40% (92)	48% (87)
Other	8% (19)	8% (15)
Marital status (Not married)	90% (207)	93% (174)
Education (High school/GED/Some College)	55% (127)	54% (102)
Substance Use
Average percent days of use (alcohol and/or drugs) 6 months prior to baseline (M/SD)	53.0 (36.8)	55.5 (36.6)
Chronicity (years of regular use)
aHeavy alcohol (M/SD)	10.7 (11.4)	8.4 (10.5)
Heroin/cocaine (M/SD)	10.9 (9.0)	9.7 (8.7)
Severity (ASI Composite scores)
Alcohol (M/SD)	0.22 (0.26)	0.18 (0.23)
Drug (M/SD)	0.16 (0.14)	0.15 (0.13)
Number of prior AOD treatment episodes*	5.9 (7.0)	4.5 (5.4)
Treatment Status at Baseline
AOD Treatment Only	31% (71)	25% (48)
Some Methadone	24% (56)	28% (52)
No Treatment	45% (105)	47% (89)
Other Issues
Any criminal justice involvement	49% (114)	43% (81)
Unstable housing: Cannot stay beyond 6 months	21% (49)	21% (40)
Psychiatric medications or hospitalizations past 12 months	16% (37)	15% (29)
No employment at all in past 3 years	24% (55)	26% (49)
Had previous welfare case	80% (185)	82% (153)
bTANF Eligible	16% (36)	17% (32)

Note. Group differences were tested using the chi-square statistic for categorical variables and one-way ANOVA for continuous variables. Cell N’s and column percents are presented for categorical variables; means and standard deviations are presented for continuous variables.

^aHeavy alcohol use defined as five or more drinks for men and four or more drinks for women.

^bParticipants were categorized as TANF Eligible if they met at least 1 of the following conditions at the time of study intake: 1) Had an active TANF case; 2) Was the primary caretaker or legal custodian for a relative’s child under the age of 18; 3) Was pregnant; 4) Had a dependent child under the age of 18 living with them or the child lived with a non-parental relative or friend and parental rights had not been terminated; 5) Had a child under the age of 18 in foster care and had a reunification plan.

^*p < .05.

Due to interview time constraints at baseline and 1-month follow-up, data on current (within the last 12 months) DSM-IV psychiatric diagnoses (American Psychiatric Association, 1994) were collected at 3-month follow-up for 315 participants (75% of the sample) using the Mini International Neuropsychiatric Interview (Sheehan et al., 1998). A total of 78% met criteria for any SUD: 59% met criteria for substance dependence and 8% for substance abuse (primary drug was cocaine for 55%, heroin 33%, and marijuana 11%); 36% met criteria for alcohol dependence and 10% for alcohol abuse. Among the 22% who did not meet diagnostic criteria for SUD, participants reported an average of 6.7 days (SD = 9.9) of drug use in the previous 30, 6.3 years (9.3) of lifetime regular heavy alcohol use, 9.0 years (9.2) of lifetime heroin or cocaine use, and 2.7 (2.6) prior episodes of alcohol and/or drug treatment. Across the entire interview sample, 23% met criteria for a depressive disorder, 22% for an anxiety disorder, and 25% for antisocial personality disorder.

A total of 119 clients were in drug-free AOD treatment only, 108 were in methadone treatment only or in combination with drug-free treatment, and 194 were not in treatment for SUD. It should be noted that welfare applicants who screened positive on the SUD screen, regardless of whether they were or were not already in treatment, were sent for an assessment. For applicants already in SUD treatment, the assessment focused on whether the current level of SUD treatment was appropriate and whether the applicant was either (1) exempt from welfare work activities because of treatment placement or (2) needed to attend work activities in addition to treatment.

Participant Flow, Selection Criteria, and Study Refusal Rates

The CONSORT diagram (Figure 1) depicts the flow of participants into the study. A total of 1685 applicants (19% of the Administrative Sample) were approached by research assistants at the two study sites in a manner designed to recruit a representative sample and asked to complete a brief study eligibility screen. Of these, 1519 consented to participate in the Screened Sample, whereas 166 (10%) refused for the following reasons: not interested (60%), too personal (14%), no time to spare (11%), and various other reasons (15%). Screen refusal rates did not significantly differ between conditions. Analyses of New York City welfare administration data found no significant differences between the Administrative Sample and the Screened Sample on any of over 20 demographic and welfare status variables.

Selection criteria for the Interview Sample (N = 421) were designed to include a subset of participants from the Screened Sample most likely to benefit from drug treatment services provided in CCM and UC. We sought to include persons with problematic substance use at baseline (only half of the Screened Sample reported alcohol or drug problems; see below) and to exclude persons with severe mental illness or homelessness. Selection criteria were: At least one day of illicit drug use or heavy drinking in past month, or, one day of illicit drug use or heavy drinking in past 6 months and currently motivated to attend treatment; Not hospitalized for mental health problems more than once in past year; Not currently experiencing psychotic symptoms or prescribed antipsychotic medication; Not residing on the streets, in shelters, or in imminent danger of being homeless; Not planning to move from the area for six months.

Of the 1519 persons in the Screened Sample, 976 (64%) were ineligible for the Interview Sample for the following reasons: no problem substance use day in past 6 months and/or not motivated to attend treatment (50%), currently homeless or in imminent danger of being homeless (16%), currently on antipsychotic medications (13%) or psychiatrically hospitalized two more times in past year (6%), or planning to move (3%). Also some were excluded due to assessor concerns about reliability of the respondent’s self-report (7%) or because the respondent was assigned to UC but reported recent involvement in CCM (5%). Note that among the 50% who were ineligible due to lack of current substance use and/or treatment motivation, 34% had been incarcerated in the prior year. Finally, 122 (8%) of the Screened Sample were eligible for the Interview Sample but refused to participate in the study, citing no time to spare (57%), not interested (23%), and other reasons (20%). Refusal rates did not differ between conditions.

Treatment Assignment and Study Assessment Procedures

All persons applying for public assistance in all Bronx county welfare intake centers during the two-year study enrollment period were administered a modified version of the CAGE screening questionnaire for substance involvement (Ewing, 1984). Applicants who screened positive based on endorsing at least one item were assigned for further evaluation to one of two centralized assessment centers: one site housed the CCM program and the other housed the UC program. Assignment to condition occurred as follows: Welfare workers accessed an automated welfare management system via computer, which assigned the client to the next available assessment slot, either at CCM or UC. Slots turned over several times per week and clients were assigned regardless of geographical proximity to a center, client preference, or any other client characteristic. Welfare workers could not override the computer assignment. A check of administrative data during the three months prior to the start of data collection found no significant differences between the two sites for any of more than 20 demographic, welfare, or treatment-related variables, and no differences in show rates for assessment appointments. Thus, assignment to condition did not appear to be biased.

At each study site, research assistants approached welfare applicants and offered the opportunity to participate in a brief research interview. Applicants were assured that research data were strictly confidential and would not be shared with welfare officials of any kind. Those who signed informed consents were administered an eligibility screen in a private space; those eligible for the follow-up study were then asked to consent to a series of confidential interviews to be completed during the next year, and those who consented were administered a baseline interview. Participants were informed that a federal Certificate of Confidentiality from the National Institutes of Health was obtained to protect their confidentiality. Follow-up interviews were completed 1, 3, 6, and 12 months after baseline in research offices (81%), via mail (9%), by phone (6%), or in home (4%); there were no between-condition differences in type of interview. Participants received product vouchers after each interview. The study was conducted under approval by the governing Internal Review Board.

Treatment Conditions

At both study sites the clinical assessment goals were to determine the need for substance abuse treatment and assign an appropriate level of care. For clients needing any level of treatment, site assessors immediately selected a pre-approved treatment program based on client preference and proximity, phoned the program to arrange an intake appointment, and provided referral information to the client. Intervention characteristics of the two treatment conditions are described below in Figure 2.

Coordinated Care Management (CCM)

The CCM condition featured an innovative coordinated care management approach that focused on (1) coordinating services among multiple providers as a means to promote outcomes for individual clients and (2) assessment, referral, linkage, support, and monitoring client activity as in standard case management. CCM administrators and care managers (CMs) communicated directly with service providers about program features and service quality, the suitability of program activities for welfare clients, and program emphasis on sobriety and employability. CMs identified programs that matched specific client needs, withdrew clients from programs not serving their needs, and reduced referrals to underperforming programs. Each CM monitored program activities at 4-6 drug treatment sites via bimonthly site visits.

Assessment services were provided by a multidisciplinary team of psychiatrists, psychologists, licensed social workers, nurses, vocational rehabilitation specialists, and credentialed addiction counselors. Assessments lasted 2-3 hours and included structured diagnostic interviews, standardized health and mental health measures, employability questionnaires (e.g., work history, criminal history, family support, housing), substance use treatment motivation and stage of change measures, and a urine toxicology screen. Acute psychiatric cases were assessed by a psychiatrist for referral to a mental health clinic or emergency room. The multidisciplinary team made initial referrals to all indicated services, including drug treatment, work activity, medical and mental health care, domestic violence programs, housing, and childcare. Clients were then assigned to CMs for ongoing monitoring and service linkage.

CMs were stationed in the assessment site or a nearby field office and worked in specific catchment areas to foster cross-agency integration of available services. They maintained caseloads of 30-35 clients whom they contacted at least biweekly in office or by phone; they also visited each client at his/her drug treatment program every two weeks. Clients returned for re-evaluation with their CM every 90 days. During these visits clients were assessed for progress towards meeting treatment plan goals. SUD treatment type and intensity as well as other service linkages were modified accordingly. CMs attempted to match clients to suitable treatment programs and ancillary services. They consulted directly with drug program staff to monitor client progress and link clients to needed medical or mental health services, with welfare-sponsored job programs to help clients transition from treatment to work settings, and with other social systems (e.g., child welfare, criminal justice) to coordinate service plans.

Usual Care (UC)

The UC site was located in a city government building. Assessment services were provided by credentialed addiction counselors. Assessments lasted about 1 hour and focused on the nature and severity of substance use issues affecting employability. Mental health and physical disability issues were briefly evaluated; applicants with potential service needs in these areas completed a second assessment and referral session with a social worker on site the same day. Following assessment, clients were assigned to welfare eligibility workers (WEWs) stationed in field offices. WEWs maintained caseloads of 75-250 clients and made service referrals only during face-to-face meetings. They met with clients on an annual basis for recertification or whenever an emergency or non-compliance issue arose. WEW caseloads were not restricted to substance-using clients, and they did not receive training in managing substance-using populations. Clients were called back to the UC site every 90 days for a re-evaluation. The primary goal of this visit was to reassess whether clients who were exempt from work requirements because of the need for intensive SUD treatment should continue in this status or instead were ready for referral to a work activity.

Treatment Fidelity

To promote treatment integrity, an embargo was activated within the computerized appointment system at all Bronx welfare intake centers to ensure that applicants originally assigned to UC during the study period would again be assigned to UC (and not to CCM) if they re-applied for benefits up to three years after baseline. In addition, CCM administrators met monthly with welfare officials to review performance indicators related to coordinated care management activities and adherence to welfare regulations. Treatment fidelity was then assessed in two ways. First, a series of structured field interviews conducted with CCM administration and staff confirmed that behavioral assessment data were being incorporated into treatment planning activities at the assessment site and that CMs were receiving supervision from licensed staff, maintaining small caseloads, attempting regular contact with clients, and making monthly visits to drug programs in which their clients were enrolled. Second, in follow-up client interviews CCM clients reported more in-person contact with CMs during the prior 30 days than UC clients reported with WEWs at 1 month (CCM: M(SD) = 1.8(2.4); UC: M(SD) = 1.1(1.6); t(298) = 3.19, p < .01, Cohen’s d = .37), 3 months (CCM = 1.5(1.9); UC = .58(1.3); t(290) = 5.05, p < .001, d = .59) 6 months (CCM = 1.1(1.9); UC = .82(4.4); t(310) = .82, ns), and 12 months (CCM = .68(1.3); UC = .32(.80); t(324) = 3.12, p < .01, d = .35).

Measures

Demographics, substance use history, and other problems

Sex, ethnicity, age, marital status, education, welfare experience, and housing status were obtained via structured interview procedures. Information on substance use and criminal justice involvement was obtained using the Addiction Severity Index (ASI, 5th edition: McLellan, Kushner, et al., 1992).

Primary Outcome: Abstinence Rates

The primary outcome variable was abstinence from alcohol and drug use during each month over the one-year follow-up period. Abstinence rates were determined using self-report and biological measures. The Timeline Follow Back method (TLFB; Sobell & Sobell, 1996) is a structured interview technique that evaluates quantity and frequency of substance use. The interview involves constructing a retrospective, daily record of the respondent’s substance use over a fixed interval, using calendars and critical marker events as memory aids. The technique has demonstrated good reliability and validity (Sobell, Brown, Leo, & Sobell, 1996). The TLFB was also used to document periods of time during which respondents were in controlled environments (e.g., treatment centers, jails); abstinence rates were not calculated for these periods. Urine screens (Varian Inc., 2008) were conducted for the presence of five drug metabolites: marijuana, cocaine, morphine, phencyclidine (PCP), and amphetamines. In addition, hair samples were analyzed for the same five metabolites using radioimmunoassay (RIA) tests followed by mass spectrometry confirmation tests (Psychemedics Corporation, 1991). Hair analysis provides a much longer time record of drug use than other biological specimens and has low potential for evasion or manipulation of test results (Cone, 1997). It has been used to verify self-report data for a variety of substances, including heroin (Romano, Barbera, Spadaro, & Valenti, 2003) and cocaine (Tassiopoulos et al., 2004).

Intermediate Outcome: Services Received

We used a modified version of the Treatment Services Review (TSR; McLellan, Alterman, Cacciola, Metzger, & O’Brien, 1992), a companion instrument to the ASI that yields data on the number of services received in various domains in the prior 30 days. The TSR has high test-retest reliability and correspondence with independent measures of services provided (McLellan, Alterman, et al, 1992; McLellan et al., 1994; McLellan et al., 1998). The total number of various services received in each ASI domain were tallied: Medical (e.g., physical exam, prescription refill, meeting with Medicaid specialist), Legal (e.g., meeting with legal specialist, court appearance), Alcohol and Other Drug (AOD; e.g., treatment attendance, medication appointment, urine or breathalyzer test), Employment (e.g., job training, meeting with employment specialist), Housing (meeting with city housing agent or shelter caseworker), Mental Health (e.g., psychological testing, individual or group psychotherapy), and Basic Needs (meeting with specialist about childcare, food, clothing, etc.).

Follow-up Completion Rates and Collection of Timeline Abstinence Data

Of the 421 participants who completed baseline interviews, 27 (6%) did not provide data at any follow-up timepoint, leaving 394 participants (221 CCM, 173 UC) included in outcome analyses. Three persons died during the follow-up period; their missed interviews were not included in calculation of completion rates. Across treatment conditions, 87% of participants completed 1-month interviews, 71% 3-month, 75% 6-month, and 79% 12-month. The analyzed sample (n = 394) was compared to those lost to follow-up (n = 27) on demographic and baseline characteristics; no significant differences were found. Overall follow-up rates were 95% in CCM and 92% in UC and were not significantly different between conditions (χ² (1) = 2.41, ns). Chi-square tests were conducted to compare the drop-out subgroups from each condition to one another on baseline characteristics; because of the small size of the subgroups (n = 11 in CCM and 16 in UC), these analyses were considered exploratory. Results indicated that attrition characteristics were not differentially distributed across conditions.

Outcome analyses utilized abstinence data collected with timeline follow back procedures. Whenever a participant missed a 1-, 3-, or 6-month interview appointment, assessors extended the recall period at the next completed appointment to capture data from all prior months going back to the date of the last completed interview. Across treatment conditions, at 1 month, abstinence data were collected on 91% of the 421 participants, 84% of which derived from interviews completed at that timepoint (standard recall) and 16% from subsequent interview timepoints (extended recall). At 3 months, data were collected on 90% of participants, 80% of which were standard recall and 20% extended recalled. At 6 months, data were collected on 87% of participants, of which 82% were standard recall and 18% extended recall. At 12 months data were collected on 79% of participants, 88% of which were standard recall and 12% extended recalled. Note that monthly abstinence rates were not calculated for any month during which the participant either did not provide information on at least 15 days or was in a controlled environment for 15 days or more.

Verification of Drug Use Self-Reports

Urine samples were collected from 350 participants (83% of the sample) during at least one timepoint: n = 266 (73% of completed interviews) at 1 month, 219 (73%) at 3 months, 232 (74%) at 6 months, and 266 (76%) at 12 months. Agreement between urine screens and self-reported drug use at the corresponding month (defined as any outcome other than a positive urine screen combined with negative self-report) was high, ranging from 89% to 98% across all assessment points (Cohen’s kappa = .74—.94 (Cohen, 1960)). Also, hair samples were collected from 201 participants (48% of the sample) at 6 months only or, if missed, at 12 months. Agreement between hair analysis and self-report was also high: 91% (k = .58). Because agreement rates were high, self-report data were considered valid and were not recoded on the basis of biological results.

Statistical Analyses

Data were analyzed using Generalized Estimating Equations (GEE), an extension of the General Linear Model that permits a within-subject repeated measures examination of change over time as well as correction of variance estimates for correlated data within subject (Zeger & Liang, 1986; Zeger, Liang, & Albert, 1988). GEEs were used to examine condition differences in receipt of services in various domains as well as abstinence from substance use. Receipt of services within each domain was modeled as a binary outcome (0 = no services received, 1 = some services received) at each of the four study timepoints. Using the same procedures as in our previous work (Morgenstern et al., 2006), we modeled complete abstinence from all substances for each month from baseline through the 12-month follow-up. Abstinence was treated as a binary outcome (0 = not abstinent, 1 = abstinent) assuming a binomial distribution, logit link function, and exchangeable correlation matrix. Preliminary models were run using the autoregressive correlation structure; however, the exchangeable correlation structure provided a better fit to our data and was used in the final models. The independent variable of primary interest was condition assignment, coded as 0 = UC and 1 = CCM.

To account for factors that may confound an association between condition assignment and abstinence outcomes, we undertook a process of model building in which we included a variety of baseline sample characteristics as covariates in the initial model. These included gender (0 = female, 1 = male), age, race (1 = African American, 2 = Hispanic, 3 = Other), education (0 = less than 12 years, 1 = 12 years or more), employment history (0 = no work in past 30 days, 1 = some work in past 30 days), housing status (0 = unstable housing, 1 = stable housing), legal involvement (0 = criminal justice history, 1 = no criminal justice history), mental health problems (0 = mental health problems, 1 = no mental health problems), methadone treatment status (0 = not receiving methadone treatment, 1 = receiving methadone treatment), number of prior AOD treatment episodes, and abstinence during the 6 months pre-baseline (number of months completely abstinent, ranging from 0 to 6). In the initial full GEE model, the following covariates had a marginal statistical association (p < .10) with abstinence and were retained in the final model: pre-baseline abstinence and methadone treatment status. Condition by time interactions were also examined, but these were not significant and dropped from the final model. In the final reduced model, we tested the main effect of condition (CCM vs. UC) and the moderating effect of methadone treatment status (methadone vs. no methadone) to determine whether condition effects differed for clients receiving methadone. Given the lower statistical power for detecting interactions compared to main effects, we considered interaction effects significant at p < .10 (Aiken & West, 1991; Cohen, Cohen, West, & Aiken, 2003).

Results

Between-Group Baseline Differences in Covariates

Baseline condition differences in the covariates listed above, as well as other client characteristics presented in Table 1, were tested using F-tests for continuous variables and chi-square tests for categorical variables. Only one significant difference was found: Clients in CCM had more prior admissions to AOD treatment than those in UC (see Table 1). In addition, differences in baseline characteristics among the non-methadone group only (n = 313) were tested; no significant difference were found.

Condition Differences in Services Received

Condition differences in the percent of people receiving services in each ASI domain across the four follow-up assessments were tested using separate GEE models examining time, condition, and time X condition interactions. Condition differences were not significant in the domains of housing and legal services (ps > .35), but were statistically significantly different in the other five domains in the expected direction. Next, we examined condition differences within the methadone maintenance only and non-methadone groups in separate analyses. Within the methadone maintenance group only (n = 108) we found no significant differences (all p >.30) for any of the eight service domains. Next, we examined condition differences in the non-methadone group (n = 313). These results are presented in Table 2. Condition differences were not significant in the domain of housing services (p > .35). There were significant (p < .05) main condition effects for medical, mental health, and AOD services, but no significant time X condition effects: medical services (OR = 1.52, 95% CI = 1.08, 2.14), mental health services (OR = 1.63, 95% CI = 1.03, 2.58), and AOD services (OR = 1.60, 95% CI = 1.11, 2.31). However, time X condition effects were significant for three domains: employment and basic needs (p < .05) and legal services (p < .08).

Table 2.

Condition differences in services received across time for participants not enrolled in methadone maintenance at baseline (n = 176 CCM, 137 UC)

	Percent Receiving Any Services
	Month 1		Month 3		Month 6		Month 12		ORa (95% CI)
	CCM	UC	CCM	UC	CCM	UC	CCM	UC
Total N	159	106	137	87	137	93	145	94
AOD	88%	69%	68%	52%	61%	50%	38%	33%	1.60* (1.11, 2.31)
Medical	69%	59%	62%	42%	61%	49%	55%	53%	1.52* (1.08, 2.14)
Legal	23%	12%	17%	18%	18%	21%	18%	18%	1.47 (.77, 2.78)
Employment	26%	10%	23%	14%	20%	11%	19%	23%	2.45** (1.31, 4.59)
Mental Health	30%	16%	24%	20%	28%	17%	23%	21%	1.63 * (1.03, 2.58)
Housing	15%	12%	11%	9%	9%	10%	16%	16%	1.07 (.65, 1.75)
Basic Needs	14%	1%	7%	3%	8%	4%	8%	7%	5.95** (1.71, 20.63)

^**p < .01,

^*p < .05

Note.

^aOdds ratios are presented for significant main effects. For the three variables that had significant Condition X Time interaction effects (Legal, Employment, Basic Needs), the odds ratios presented are for the main effects of Condition for the initial study period (Months 1-3) to emphasize the time period during which condition differences were most pronounced, rather than for the full 12-month follow-up period. For all other variables, odds ratios are presented for the Condition main effect for the full follow-up period.

CCM = Coordinated Care Management, UC = Usual Care

Examining the plots of these three interactions revealed that the significant findings were largely due to early main effect differences favoring CCM at months 1 and 3 that subsequently dissipated over time. Based on these observed patterns, we decided to probe the interaction effects by exploring the magnitude of effects during the first three months after condition assignment. GEE models were re-run for these three service domains (employment, basic needs, and legal) using only scores from Month 1 and Month 3 (see Table 2). Significant condition effects were found for employment (OR = 2.45, 95% CI = 1.31, 4.59) and basic needs (OR = 5.95, 95% CI = 1.71, 20.63) Overall, within the non-methadone group a higher percent of CCM clients received AOD, medical, employment, mental health, and basic needs services during the first three months of the program compared to UC clients.

Abstinence Outcomes

The final reduced model for the full sample revealed no significant main effect of condition. However, a condition by methadone status interaction was found (OR = .51, 95% CI = .23, 1.13, p =.09), supporting the hypothesis that the effect of condition assignment on abstinence was weaker for clients receiving methadone maintenance at baseline. Subsequent analyses were conducted on the two subsamples separately: methadone clients (n = 108) and non-methadone clients (n = 313). There was no significant effect of condition among clients receiving methadone maintenance (OR = .56, 95% CI = .28, 1.14, ns). Analyses were then conducted on the non-methadone group as presented below.

The full model with all covariates included was then tested for the non-methadone group (n = 313), and covariates were trimmed following the procedures outlined above. Number of prior AOD treatment episodes was also included in the final model as a covariate, despite not reaching statistical significance, in order to control for between-condition baseline differences on this variable (see Table 1). The final reduced model included the following covariates significant at p < .10: pre-baseline abstinence (OR = 1.39, 95% CI = 1.21, 1.61), legal involvement (OR = 2.01, 95% CI = 1.27, 3.19), and treatment status (0 = not in treatment, 1 = in drug free treatment) (OR = 2.81, 95% CI = 1.78, 4.43). Clients with no history of criminal justice involvement had higher rates of abstinence over time, as did clients who were enrolled in drug-free treatment at baseline.

A significant main effect of condition was found for the non-methadone clients (OR = 1.75, 95% CI = 1.12, 2.76; Cohen’s d =.31)¹. Figure 3 depicts the abstinence rates by condition (dashed and dotted lines). As shown in the figure, abstinence rates were higher among CCM clients compared to UC clients. No significant time by condition interaction was found, suggesting that time did not moderate the effect of condition on outcome.

Figure 3.

Percent Abstinent in Each Study Month for Non-Methadone Sample: Coordinated Care Management versus Usual Care

Because legal involvement and treatment status predicted outcome, we tested each as a potential moderator of the condition differences in abstinence in a set of post-hoc analyses. No significant moderating effects were found, suggesting that CCM was equally effective across these subgroups.

Post Hoc Considerations

As described above, Figure 3 depicts the abstinence rates prior to and following assignment to condition for non-methadone clients. It appears in the figure that condition differences in abstinence rates begin to emerge approximately two months prior to participants being assigned to treatment conditions. These differences may reflect random fluctuation in abstinence rates across months or some pre-existing differences between conditions. We attempted to resolve the interpretation of this phenomenon in several ways. First, as shown in Table 1, no significant differences were found between the two conditions for nearly all baseline characteristics, including demographics, many measures of substance use chronicity and severity, and other issues. Thus, no systematic pattern of lower SUD severity appears in the CCM condition. Second, in the non-methadone group we conducted pairwise comparisons of abstinence rates at each of the 18 months using the TLFB data. No significant or trend differences were found at any of the six months prior to baseline. In contrast, during the 12-month follow-up period significant or trend-level differences in abstinence rates were found for the majority of months (months 3, 5, 6, 7, 9, 10, and 11).

Third, in the non-methadone group we re-conducted analyses for the condition effect by modeling two alternative covariates to control for pre-baseline differences in abstinence at the point in time where the conditions appear to diverge pre-baseline: (1) average percent days abstinent in the two months prior to baseline (2) completely abstinent (yes/no) in either or both of the two months prior to baseline. Both alternative analyses produced similar results to the original main analysis: a significant condition effect (p < .05) favoring CCM. Fourth, in the non-methadone group we examined condition differences across the follow-up period using a different measure of substance use outcome (sum of the ASI drug and alcohol composite scores) that demonstrated no significant differences between conditions at baseline. The main effect of condition for the ASI outcome was significant at the p < .05 level. Taken together, these findings support the conclusion that the main effect of condition on abstinence is due to real intervention effects rather than confounding factors, and that apparent pre-baseline differences in abstinence rates likely represent random fluctuation.

Confirmation of Condition Effect on Abstinence Using Biological Data

The significant condition effect on abstinence for the non-methadone group was verified using the biological measures. A GEE model was conducted to examine condition differences in negative urine screens (indicating complete abstinence) across the four study time points. Consistent with the self-report data, a significant main effect of condition was found (OR = 1.59, 95% CI = 1.04, 2.41), with CCM clients showing higher rates of abstinence across time compared to UC clients. We then used logistic regression to examine condition differences in abstinence using the hair samples collected at a single time point. Although the effect of condition was not statistically significant, the direction of the odds ratio was consistent with that found for the self-report data (OR = 2.40, 95% CI = .83, 6.98). Note that the reduced sample size (n = 142) and single time point reduced power to detect a significant effect. Taken together, these findings support the validity of the self-report measures of abstinence.

Discussion

Findings support the hypothesized effects of coordinated care management for individuals not enrolled in methadone maintenance. Specifically, among non-methadone maintenance clients, those in CCM received more services and had significantly higher rates of abstinence during the year following program entry when compared to clients in usual care (UC). Overall, the effects of CCM on receipt of services appeared to be broad, affecting not only AOD services but medical, mental health, employment, and basic needs. These positive effects were most evident during the first three months of the intervention.

The likelihood of abstinence in the CCM condition was 75% higher than UC for non-methadone clients, a small to medium effect size (Cohen’s d =.31; Cohen, 1988). For example, averaging across the last six months of follow-up, abstinence rates in CCM were 40% and in UC were 28%. Effects appear to emerge early and are sustained throughout the 12 months. Post hoc examination of individual follow-up timepoints, alternative covariates to control for pre-baseline differences in abstinence, a second measure of substance use, and biological data all confirmed main study findings for abstinence outcomes. In addition, CCM effects were consistent across clients in versus out of treatment at baseline and those with or without current criminal justice involvement. Thus, CCM effects were robust across subgroups where one might have anticipated weaker effects.

As expected, there were no between-condition differences for clients enrolled in methadone maintenance at baseline. Moreover, analyses indicate that methadone clients in CCM did not receive significantly more services than those in UC, a hypothesized intermediate outcome. It is not clear whether this explains the lack of condition effects. Further mediational analyses are required to provide a better explanation of these findings. In the absence of additional data, we speculate that case management programs for methadone clients may require greater tailoring to the specific features of methadone clients and programs. For example, clients who continue to relapse are discharged from drug-free treatment but continue to receive treatment in most methadone programs. These differences create different contingencies for abstaining and for meeting welfare activity participation requirements.

Findings in Context

In an earlier study, we found that intensive case management (ICM) significantly improved SUD treatment engagement and rates of abstinence among substance dependent TANF mothers when compared to care coordination (Morgenstern et al., 2006). The two studies have important similarities. Both studies examined whether elements of case management – service linkage, direct case managers contact, monitoring over time, and adjustments in treatment type and intensity when needed – would improve outcomes relative to usual care in the context of new systems level coordination across welfare agencies and SUD treatment programs. In both studies, a case management intervention yielded significantly higher rates of abstinence that were sustained across 12 months with a similar effect size for the intervention: a small-to-medium effect.

However, the two studies are different. As noted, the current study was designed as a practical clinical trial. The earlier study was designed as an effectiveness trial and greater internal rigor was implemented. Most important, the intervention, ICM, was substantially more structured and intensive than CCM, involving use of a manual and supervision, lower caseloads, more direct client contact, and vouchers to reinforce SUD treatment attendance. In addition, the earlier study examined a more homogenous but restricted sample (out-of-treatment mothers) and the control condition, care coordination, offered a somewhat more intensive set of services than UC in the current study.

A recent Cochrane review examined evidence for the efficacy of case management for SUD clients (Hesse, Vanderplasschen, Rapp, Broekaert, & Fridell, 2007). Only seven experimental studies reported on substance use outcomes and the overall effect size was not significant. The review concluded that given the small number, heterogeneity of interventions, populations, and control groups, and overall poor quality of many of the studies, case management should be considered promising but as yet unproven. We believe the positive findings of this and our prior case management study are likely attributable to substantially higher methodological quality. This explanation is consistent with the findings of the Cochrane review (Hesse et al., 2007) that studies with higher methodological quality yielded greater effect sizes for case management. Of special importance in our studies were the implementation of robust interventions and the ability to ensure the control condition did not receive a set of services similar to the experimental condition. We note that our studies were carried out in collaboration with the agencies overseeing the systems of care and that the experimental interventions were funded by them. Thus interventions were adequately resourced and systems of care, rather than specific programs, were studied.

Implications for Program Administrators, Policymakers, and Funders

Findings have most relevance for local governments who administer SUD screens to clients applying for public assistance and refer those who screen positive to assessment and SUD treatment. The most recent study (SAMHSA & CSAT, 2002) found that such services are being provided in over half of states, but few are offering additional case management services. Findings are also relevant to the broader population of SUD clients in treatment. About 40% of non-methadone clients in this study were already in SUD treatment. These clients applied for public assistance benefits in large part to receive Medicaid so they could pay for SUD treatment. Thus, this subgroup is similar to other low-income clients in publicly funded SUD treatment and not unique to those identified solely via screening in a welfare agency. Study findings provide the strongest evidence to date that clients already enrolled in SUD treatment would benefit from case management services, and they are consistent with a number of other studies supporting the efficacy of continuity of care interventions as a wraparound to existing care (Dennis, Scott, & Funk, 2003; McKay et al., 2004; McKay, Lynch, Shepard, & Pettinati, 2005).

Future Research Directions

The Cochrane review noted that case management for SUD is an understudied intervention compared to case management for mental health. Findings from this and our earlier study suggest that case management may be especially promising as a wraparound to SUD treatment, but more research is needed on what type of case management works for which type of clients and in what settings. Research examining the active ingredients of case management is especially important (cf. Morgenstern et al., 2008). For example, coordinated care management, when compared to UC, contained additional linkage to SUD treatment, other services, extended monitoring, direct client contact, and active management of SUD providers. It is not clear which of these ingredients are effective.

This study is also an early example of a practical clinical trial (PCT). PCTs have been recommended as a new strategy to facilitate adoption of evidence-based practice (March et al., 2005; Zerhouni, 2003). A novel additional feature of this study was a strong academic-government agency partnership. The New York City welfare agency collaborated with us in the design of the study and provided funding for all services. Our earlier study of ICM was conducted in the context of a similar academic-government agency partnership with the TANF agency in New Jersey. One major advantage of this feature is that government is often the sole funder of services and a primary stakeholder in accountability and quality of those services. Gaining stakeholder buy-in at the initial phases of a study increases the likelihood that study findings will be adopted at a systems level after the trial is completed. In New Jersey, positive findings (Morgenstern et al., 2006; Morgenstern et al., 2008) were followed by an expansion of case management from two to 14 additional counties. Coordinated care management is currently being expanded in New York City. Thus, PCTs especially in the context of academic-government collaboration may prove to be an effective strategy to speed adoption of evidence-base practice and should be considered by the broader behavioral health research community.

Study Limitations

One limitation of this study was that assignment to condition was not random. However, we do not believe our method of allocation introduced bias into the control or experimental conditions at baseline for the following reasons. First, allocation occurred when welfare workers scheduled assessment appointments for clients who were positive on an SUD screen. As noted, the scheduling system automatically assigned clients to one of the conditions on the basis of availability, and it could not be overridden by the welfare worker. Availability across conditions changed frequently because the welfare system in the Bronx is large with multiple offices, and as many as 3,000 appointments were scheduled each year.

Second, we examined condition differences on measured variables at three separate occasions: once prior to the study for the full Administrative Sample; next for the Screened Sample (Morgenstern, Hogue, et al., 2008); and finally, as noted above, for the current Interview Sample. Our analyses indicated that samples were not significantly different within several key domains, including those most relevant to bias: demographics, welfare history, SUD severity and chronicity, co-occurrence of other problems, baseline placement in treatment, and motivation to reduce substance use and seek employment. Thus, it seems unlikely that condition differences in outcomes can be explained by participants in CCM having less severe problems or more resources (e.g., education, motivation) than those in UC.

In addition, other design aspects of the study that served to increase clinical relevance decreased internal validity. For example, CCM was not protocol-driven and variability across case managers might have weakened effects. As noted earlier, PCTs by design are often, because of real-world constraints, required to sacrifice some internal rigor in exchange for clinical and policy relevance. Findings must be judged accordingly. In addition, the effectiveness of CCM may differ if caseload sizes are larger than those in the study. Current caseload sizes for CCM in New York City have been maintained at the 30-35 level.

Finally, findings are limited by study selection criteria. We excluded clients not in need of current SUD treatment and those whose housing status (homeless) or mental health problems (repeated psychiatric hospitalizations) might have limited their ability to attend or benefit from outpatient SUD treatment. It is important to note that in addition to promoting abstinence among clients needing treatment, the NYC SUD welfare special assistance programs also attempt to engage recovered (i.e., currently sober) SUD clients in employment services and monitor compliance to welfare-to-work regulations. Thus, many clients participating in UC or CCM do not have current SUD issues, although they are at risk for relapse. The extent to which CCM is effective for these clients is unknown. Findings are also limited to a report on services received and endpoint substance use outcomes. As noted earlier, given these limitations future studies of the effectiveness of care and case management interventions are important. Differences in other outcomes and testing of hypothesized mechanisms of action will be addressed in future studies. Finally, future research is needed to address the important issues of adoption and sustainability of CCM for this population.

Conclusion

Coordinated care management increased rates of service utilization and abstinence when compared to usual care among a large, heterogeneous sample of individuals with SUD identified in welfare settings, but not for those in methadone maintenance treatment at baseline. Among clients in methadone maintenance treatment, CCM was not superior to UC. The consistency of CCM effectiveness across those in or out of drug-free treatment as well as the PCT design of the study strengthen the relevance of its findings to multiple stakeholders.