Effectiveness and Cost-effectiveness of Four Treatment Modalities for Substance Disorders: A Propensity Score Analysis

Abstract

Objective

To assess the effectiveness and cost-effectiveness of four treatment modalities for substance abuse.

Data Sources

The study used data from the Services Research Outcomes Study (SROS), a survey of 3,047 clients in a random sample of 99 drug treatment facilities across the United States. Detailed sociodemographic, substance use, and clinical data were abstracted from treatment records. Substance abuse outcome and treatment history following discharge from index facilities were assessed using a comprehensive interview with 1,799 of these individuals five years after discharge. Treatment success was defined in two ways: as abstinence and as any reduction in substance use.

Study Design

Effectiveness and cost-effectiveness of four modalities were compared: inpatient, residential, outpatient detox/methadone, and outpatient drug-free. Clients were stratified based on propensity scores and analyses were conducted within these strata. Sensitivity analyses examined the impact of future substance abuse treatment on effectiveness and cost-effectiveness estimates.

Principal Findings

Treatment of substance disorders appears to be cost-effective compared to other health interventions. The cost per successfully treated abstinent case in the least costly modality, the outpatient drug-free programs, was $6,300 (95 percent confidence intervals: $5,200–$7,900) in 1990 dollars. There were only minor differences between various modalities of treatment with regard to effectiveness. However, modalities varied considerably with regard to cost-effectiveness. Outpatient drug-free programs were the most cost-effective. There was little evidence that relative effectiveness or cost-effectiveness of programs varied according to factors that were associated with selection into different programs.

Conclusions

Substance disorders can be treated most cost-effectively in outpatient drug-free settings. Savings from transitioning to the most cost-effective treatment modality may free resources that could be reinvested to improve access to substance abuse treatment for a larger number of individuals in need of such treatment.

Alcohol and substance use are common and affect the lives of many Americans. Approximately 14 million Americans currently abuse illicit drugs and a similar number abuse alcohol (Substance Abuse Chartbook 2001; Grant et al. 1994). Estimating the overall cost of substance disorders is difficult since these disorders impact not only the lives of the individual substance abusers and their families, but also the communities in which they live. One estimate puts the figure for the overall cost to the U.S. economy at about $414 billion a year (Substance Abuse Chartbook 2001). Only a small fraction of the total cost of substance abuse—about $12 billion a year—is directly spent on treatment of these disorders (Substance Abuse Chartbook 2001). The rest includes productivity losses due to premature death and disability and the cost of criminal activities to society. Although there is good evidence that, at a societal level, treatment of substance abusers is more cost-effective than enforcement efforts to reduce substance supply (Rydell, Caulkins, and Everingham 1996), policymakers often ask for evidence of effectiveness and cost-effectiveness for specific treatments.

A number of such cost-effectiveness studies have been conducted (Barnett and Swindle 1997; Barnett 1999; Flynn et al. 1999; Harwood et al. 1995; Holder et al. 1991; Pettinati et al. 1999; Schinka et al. 1998; Shepard, Larson, and Hoffman 1999) and excellent reviews of this literature have been published (French 1995; Cartwright 2000). A principal conclusion from many of these studies is that more intensive treatments are not necessarily more cost-effective than the less intensive alternatives (Annis 1986; McLellan et al. 1983; McCrady et al. 1986; Schinka et al. 1998). While effectiveness across modalities appears to be similar, outpatient care is generally less expensive than inpatient care. Many of the past studies, however, excluded clients with previous treatment failure, multiple drug use, history of homelessness, and psychiatric comorbidity. These exclusion criteria limit generalizability. Furthermore, these exclusion criteria limit our ability to examine the common assumption that the more severe and complicated cases may respond better to intensive treatments.

In the present study, we assess effectiveness and cost-effectiveness of treatment of substance disorders in usual treatment settings, addressing some of the limitations of previous studies. The data for this study come from the Services Research Outcomes Study (SROS; Substance Abuse and Mental Health Services Administration 1998), which is the first large-scale national study that drew a stratified random sample out of a pool of more than ten thousand enumerated drug treatment facilities across the nation.

In contrast to previous economic studies using national data (Harwood et al. 1995; Flynn et al. 1999), we focus on direct clinical outcomes of substance abuse treatment, rather than reduction in criminal activity. Furthermore, in our comparison of treatment modalities we employ propensity score analysis to control for the factors that influence selection into various modalities, such as type and severity of substance disorder and psychiatric comorbidity. Measures of effect are based on a five-year follow-up, hence addressing one of the major limitations of previous economic analyses—that is, follow-up periods that are too brief (Sindelar and Manning 1997). Finally, successful outcome was measured in two ways, as abstinence and as any reduction in use. To our knowledge, the present analysis is the first attempt at a comprehensive cost-effectiveness study of substance abuse treatment programs based on a large sample of clients from a nationally representative sample of facilities.

Methods

SROS Sample

The design of the SROS study has been described in more detail elsewhere (Substance Abuse and Mental Health Services Administration 1998) and is presented in Figure 1. Briefly, the sample of facilities for the SROS study was drawn from the 1990 Drug Services Research Survey (DSRS) conducted by the National Institute on Drug Abuse (NIDA). The NIDA's listing of all known substance abuse treatment facilities, consisting of more than ten thousand facilities, served as the universe for the DSRS. The DSRS was conducted in two phases. In phase one, a random sample of 1,803 of the enumerated facilities was approached. Of these, 1,183 responded, including inpatient, residential, outpatient detoxification/methadone, outpatient drug-free, and alcohol-only facilities as well as 317 facilities for which the modality was unknown. Phase two DSRS drew a sample of 146 facilities stratified according to modality from the phase one sample and abstracted records of 2,222 clients discharged from 120 cooperating facilities in the 12 months ending August 31, 1990. Since DSRS focused on drug treatment facilities, the alcohol-only facilities and those of unknown type were excluded from the second phase.

Figure 1.

The Design of the Services Research Outcomes Study (SROS)

Five years after DSRS, the SROS researchers recontacted the original 120 DSRS phase two facilities, 99 of which responded and became part of SROS. With the cooperation of these facilities, the clients were contacted and invited to participate in SROS. Five-year follow-up was sought for 1,706 of the original client sample plus a supplemental sample of 1,341 additional clients from the 99 facilities, totaling 3,047 clients. The criterion used for selection of the supplemental sample was discharge from the treatment facility in the same time frame as the original sample (i.e., discharged in the 12 months ending August 31, 1990). Of these, 2,489 (82 percent) were located in 1995 and 1996; the remaining 558 clients (18 percent) were not located before time and resources for fieldwork expired. Nine percent (277 clients) had died before follow-up. A total of 1,799 (59 percent) clients were interviewed.

Modalities

The programs sampled in the SROS fell into one of the following four categories: inpatient, residential, outpatient detox/methadone, and outpatient drug-free. A brief description of these modalities follows:

Inpatient programs

generally adopt a medical model of substance disorders. They provide intensive medication and counseling in a relatively short period of time. In the SROS sample, clients in inpatient settings had an average of 4.2 hours of contact per week with medical professionals (physicians, psychiatrists, and nurses) and 2.5 hours of contact with nonmedical professionals (Substance Abuse and Mental Health Services Administration 1998). Many inpatient programs are part of psychiatric or general hospitals.

Residential programs

are mostly modeled after the therapeutic communities of the 1950s, which aimed to change the lifestyle of the drug abuser through a long-term communal experience. Most residential programs are freestanding programs of limited capacity. The main distinguishing characteristics of these programs are the use of peer-counselors and less reliance on medical and nonmedical professional staff (Hubbard et al. 1989). In the SROS sample, clients in the residential settings had an average of 0.2 hours of contact per week with medical professionals and 1.0 hours of contact with nonmedical professionals.

Outpatient detox and methadone programs

in SROS are heterogeneous and include methadone detox and perhaps methadone maintenance programs for narcotic users as well as other detox programs. A common feature of many of these programs is reliance on pharmacological agents including methadone and other psychotropic agents. Thus, the proportion of clients receiving medications is generally higher than residential or outpatient drug-free programs. Psychosocial treatments are also often provided in such clinics (Hubbard et al. 1989). In the SROS sample, clients in the outpatient detox and methadone clinic had an average of 0.4 hours of contact per week with medical professionals and 0.8 hours of contact with nonmedical professionals.

Outpatient drug-free programs

emphasize counseling rather than medication as a major component of treatment. Clients continue to live in the community and go to clinics on a regular basis for counseling sessions. In the SROS sample, clients in the outpatient drug-free program had an average of less than 0.1 hours of contact per week with medical professionals and 0.6 hours of contact with nonmedical professionals. Many outpatient programs are affiliated with Community Mental Health Centers and others are free standing (Hubbard et al. 1989).

Sociodemographic Variables

Gender, age, race, education, and homelessness at the time of index admission were derived from medical records. History of homelessness, defined as homeless for two consecutive nights in the five years before index admission was ascertained in the follow-up interview.

Psychiatric Comorbidity

Psychiatric comorbidity was determined from chart reviews. Clients who were recorded as having a dual diagnosis or a “history of psychological disorders at admission (other than drug/alcohol related problems),” were rated as cases with psychiatric comorbidity. This is admittedly an inclusive definition of psychiatric comorbidity and is not limited to cases of severe mental disorder. Further analyses, however, indicated that this definition of psychiatric comorbidity is a relatively sensitive and specific measure of characteristics that may indicate severe psychopathology such as psychiatric hospitalization before index treatment and suicide attempts. Twenty-two percent of clients who were categorized as having psychiatric comorbidity had at least one psychiatric hospitalization in the five-year period before index substance treatment episode and 29 percent reported at least one suicide attempt. By contrast, only 3 percent of clients not categorized as having a psychiatric comorbidity reported a history of psychiatric hospitalization and 12 percent reported suicide attempts.

Future Treatments

Rating of treatments after discharge from index treatment program was based on response to the following question asked at the follow-up: “After you left [index program] in [date], and until now, have you received treatment or counseling for drinking or using drugs?”

Substance Abuse Variables

Ratings of substance use were based on questions at the five-year follow-up interview. The ratings were conducted separately for 14 substances or categories of substances, including alcohol, crack, cocaine, heroin, illicit methadone, other narcotics, marijuana, inhalants, downers, PCP, methamphetamine, hallucinogens, downers, and other drugs. For each, pretreatment use was ascertained by two questions: (1) “In looking at the calendar, in the five years before [index treatment], did you use [the substance] five times or more?” If the answer was positive, the client was asked: (2) “When you used [the substance] in the five years before [index treatment], on the average, about how many days did you use [the substance] during a month?” Clients were given an option to choose among six responses to the last question: 1 day, 2–5 days, 6–10 days, 11–20 days, 21–30 days, and everyday. Posttreatment use was ascertained with parallel questions covering the period from termination of index treatment to the time of follow-up interview.

Outcomes

Outcome was operationalized by two nonmutually exclusive measures: (1) abstinence during the five-year follow-up defined as a report of no use of any substances in the interval since discharge from the index treatment; and (2) any reduction in the use of substances from the five-year period before index treatment to the five-year period following treatment. These ratings were based on follow-up interviews with the clients as described above.

As most clients used several substances, data on the most frequently used substance before the index treatment episode was used as the pretreatment measure of severity and data on the substance most frequently used after the index treatment, as the posttreatment measure. This should provide a conservative estimate of the reduction in use that incorporates the substitution of one substance of abuse for another.

Costs

In individual interviews with program directors of all SROS collaborating facilities, they were asked detailed questions about salaries and benefits of staff, rent or the rental value of the facility if it was owned by the program, utilities, supplies, amortized cost of equipment, and the estimated salaries of volunteers. Missing data were estimated using the median cost from the rest of the programs in each modality. Total costs included these items and were calculated for each program. The average cost per client was the total cost of the program divided by the number of clients treated in that program. Individualized cost estimates were calculated by multiplying the average cost per client by the ratio of each client's length of treatment over the average length of treatment in that program. There were two outliers in the individualized cost data with estimated costs of $810,756 and $139,269—10 and 5 standard deviation above the mean, respectively. These were excluded from the analyses. All costs are in 1990 dollars.

Data Analysis

Data analyses were conducted in the following three stages:

1.Effectiveness of the four treatment modalities was compared using logistic regression analyses. In an observational study such as SROS, clients either self-select or are referred to treatment programs based on clients' preferences or referral biases of the clinicians. For example, homeless clients may prefer residential programs, which provide a place to live, and clinicians may refer clients with psychiatric comorbidities to inpatient settings where psychiatric treatment is more readily available. If such selection factors are associated with treatment outcomes, the results of treatment comparisons may be biased. For example, if clients with psychiatric comorbidity tend to have poorer outcomes, inpatient treatments that receive more of such clients would appear to have poorer outcomes compared with other modalities. Selection bias is a common concern in observational studies and various statistical methods have been introduced to minimize its effect. In this study we used the method of propensity score stratification (Rosenbaum and Rubin 1984; Rubin 1997). A propensity score is the probability that a client will seek treatment in one type of program rather than another and is derived from a regression model (often logistic) in which variables associated with selection into a particular program are entered as independent variables. When more than two treatments are compared, a separate propensity score is calculated for each pair of treatments.

Stratifying clients based on propensity scores creates strata in which the two treatment groups are balanced with regard to the selection variables. To use the previous examples, when comparing the residential treatment with any other treatment, similar proportions of homeless clients would be found in the treatment groups within each stratum. Similarly, when comparing the inpatient treatment with any other treatment, similar proportions of clients with psychiatric comorbidity would be found in treatment groups within each stratum. As the proportion of homeless clients or those with psychiatric comorbidity is similar across treatment groups within each stratum, these factors should not bias the comparison of treatments.

In addition, comparing the results across strata provides an opportunity to assess the appropriateness of the choices and preferences that select clients into programs. For instance, when Rubin (1997) used propensity scores based on age and size of tumor to compare the results of mastectomy and conservative treatment for breast cancer, he found that women in the propensity stratum most likely to receive conservative treatment did slightly better if they had received conservative treatment rather than mastectomy and vice versa for women in the stratum most likely to receive mastectomy. Rubin interpreted this finding as indicating that women and their physicians were making beneficial choices and treatment–patient matching improved results. Similarly, in our study, comparison of the effectiveness and cost-effectiveness of modalities across strata would indicate whether or not selection into treatments as it occurs in usual treatment settings improves outcomes of treatment.

To obtain propensity scores, we conducted a series of logistic regression analyses in which treatment modalities were compared in pairs. The sociodemographic and clinical variables that had shown significant variation across modalities were included as independent variables. These included gender, age, education level, race, history of homelessness in the five years before entering index treatment, type of drug, number of drugs abused, heavy use in the five years before index treatment (using substances more than 20 days per month on average), psychiatric comorbidity, and involuntary treatment. The estimated propensity scores were then rank ordered among clients in the two modalities and were divided into five almost equal strata. The distributions of covariates across the treatment modalities were compared within each stratum to assess whether stratification based on propensity scores had indeed created equivalent groups. If statistically significant differences (p<. 05) on a variable remained, the interaction terms of that variable with all other variables were entered into a logistic model along with all the previous variables and a new propensity score was calculated, on the basis of which new strata were formed. This procedure was repeated until no significant differences between the two treatments remained in any of the strata.

After deriving propensity strata, logistic regression analyses were conducted to compare effectiveness of the treatment modalities. These analyses used the sampling weights calculated by the SROS investigators, which adjust the data for the effect of various biases arising from sample selection, undercoverage of certain populations, nonresponse, and loss to follow-up (Substance Abuse and Mental Health Services Administration 1998).

In addition to analyses conducted within each stratum, we conducted analyses for all strata combined using conditional logistic regression analysis for matched sets in which propensity stratum was the matching variable (Breslow and Day 1980). These analyses are therefore stratified based on propensity scores. Separate analyses were conducted for the two outcomes of abstinence and any reduction in use.

2.Cost-effectiveness of the different modalities were estimated as the ratio of individualized cost data obtained as described above and the effectiveness data as defined by the proportion who were abstinent or had reduced use. Confidence intervals were calculated using the bootstrap method with one thousand replications and bias correction (Stata Corporation 2001).

In addition, relative cost-effectiveness analyses were conducted within propensity strata and for all strata combined. Relative cost-effectiveness was calculated as the ratio of the cost-effectiveness of one treatment modality over the cost-effectiveness of the other modality. Thus, this ratio represents how much greater (or smaller) the cost of successfully treating one client in one modality is compared with the other modality. Confidence intervals for these ratios were calculated using the bootstrap method.

3. Sensitivity analyses examined the impact of future treatments on assessment of treatment outcomes. Our analyses indicated that a large proportion of clients received further treatment after the index treatment episode. Unfortunately, the SROS data do not provide detailed information on the timing and reasons for subsequent treatment encounters. The main analyses, therefore, assumed that all clients who were abstinent or reduced substance use during the five years postdischarge were treatment successes. In other words, these analyses attributed the outcome at the five-year follow-up solely to index treatment—a “best case” scenario. However, for some of these individuals, their success might have been due to treatments that they received after discharge from the index treatment. To assess the potential impact of subsequent treatments on our conclusions, we conducted a series of sensitivity analyses in which the successful outcome was defined as abstinence or reduction in use without any future treatment. All those who received future treatments were counted as treatment failures—a “worst case” scenario.

An “intention to treat” approach was used for all analyses. Thus, clients who had not successfully completed the SROS treatment episodes were included in the analyses. All analyses were conducted using Stata 7.0 software (Stata Corporation 2001). The clogit subroutine was used for conditional logistic regression analyses and the bstrap subroutine for bootstrap estimation.

RESULTS

There were considerable variations across modalities in client and treatment characteristics, but not in unadjusted rates of abstinence or in reduction of use (Table 1). Logistic regressions conducted to derive propensity scores confirmed the differences in client and treatment characteristics among various modalities (Table 2).

Table 1.

Characteristics of the SROS Sample According to Treatment Modality

Characteristics	Inpatient	Residential	Detox/Methadone	Outpatient Drug-Free	Pa
Number of programsb	22	26	26	23	—
Number of subjects in the follow-up	441	464	423	471	—
Characteristics of the Clients
Female, %	30	25	37	31	**
Age, mean (SD)	39 (10)	35 (9)	38 (9)	35 (10)	**
Years of education, %
<12 years	41	49	40	46
12 years	28	27	31	30
>12 years	31	23	30	25	*
Race/ethnicity, %
White	65	54	61	66
Black	23	39	28	20
Hispanic	8	4	8	10
Other	3	3	3	4	**
Homeless, %
At admission as per chart	4	5	2	2	*
Ever in the 5 years before indexc	23	29	21	16	***
Characteristics of the Substance Disorder
Type of substance, %
Alcohol	72	64	55	73	***
Cocaine	23	27	30	19	**
Crack	19	37	12	12	***
Heroin	14	8	43	4	***
Marijuana	15	33	19	33	***
Multiple substances, %	42	56	50	44	**
Substance use severity in the 5 years before treatment
Mean days/month used (SD)d	21 (10)	21 (10)	21 (10)	18 (11)	***
Used >20 days/month, %	60	57	60	44	***
Psychiatric comorbidity, %	37	21	16	21	**
Characteristics of Treatment
Length of treatment,
Mean days (SD)	31 (99)	71 (104)	227 (373)	164 (189)	**
Median	14.5	29	101	122
Involuntary (court-ordered), %	5	17	13	31	**
Content of treatment, %
Any medication (not methadone)	90	57	32	39	***
Any individual counseling	89	94	86	91	**
Any group counseling	90	91	54	78	***
Drug education counseling	84	79	42	46	***
Average number of sessions for clients who received treatment
Individual counseling, mean (SD)	5.8 (5.3)	9.5 (11.6)	11.9 (14.5)	7.3 (9.1)	***
Median	5	5	6	4
Group therapy, mean (SD)	14.5 (12.8)	28.0 (18.6)	13.8 (14.2)	15.1 (12.4)	***
Median	11	29	8	12
Drug education counseling, mean (SD)	11.5 (11.9)	17.7 (17.7)	10.9 (16.6)	4.6 (3.6)	***
Median	6	8	2	4
Average cost per client, in $1,000's
Mean (SD)	3.2 (7.8)	3.1 (4.0)	2.2 (3.6)	1.2 (1.4)	*
Median	1.2	1.6	0.8	0.7
Completion of treatment, %
Completed as planned	73	51	33	43	***
Not completed, client choice	11	24	40	34	***
Not completed, other reason	13	22	23	17	***
Five-year course
Abstinent during the 5-year post-treatment, %	21	21	17	18	NS
Abstinent or reduced use during the 5-year post-treatment, %	53	47	48	49	NS
Received further treatment after the index treatment, %	58	58	55	41	**

^aP-values were determined by chi-square test for categorical data and F-test for continuous data.

^bOf the 99 programs whose directors were interviewed, two programs had no clients in the follow-up. The total number of programs is therefore 97.

^cDefined as homeless for two consecutive nights.

^dAssessment of the severity of substance abuse was based on one question asked for each type of substance: “When you used [the substance] in the five years before [index treatment], on the average, about how many days did you use [the substance] during a month?” Severity measure reported here was calculated as the maximum number for all the substances that the respondent reported using.

^*P<.05

^**p<.01

^***p<.001

NS=not significant.

Table 2.

Logistic Regression Analyses for Deriving Propensity Scores. The Odds Ratios Represent the Odds of Being Selected into the First Program versus the Second for Each Column

	Inpatient versus Outpatient Drug-free	Residential versus Outpatient Drug-free	Detox/Methadone versus Outpatient Drug-free	Inpatient versus Detox/Methadone	Residential versus Detox/Methadone	Inpatient versus Residential
	OR (95% CI)	OR (95% CI)	OR (95% CI)	OR (95% CI)	OR (95% CI)	OR (95% CI)
Female gender	0.79 (0.56–1.10)	0.57 (0.41–0.80)**	1.09 (0.77–1.54)	0.78 (0.55–1.08)	0.49 (0.34–0.71)***	1.32 (0.93–1.86)
Age≥31 yearsa	1.39 (0.99–1.95)	0.87 (0.64–1.19)	0.88 (0.63–1.24)	1.38 (0.98–1.95)	0.87 (0.61–1.23)	1.42 (1.02–1.97)*
Education<12 years	1.22 (0.85–1.73)	1.22 (0.88–1.68)	0.98 (0.68–1.41)	1.44 (1.00–2.07)*	1.29 (0.89–1.86)	1.08 (0.77–1.52)
Education>12 years	1.61 (1.06–2.43)*	0.88 (0.57–1.34)	1.45 (0.95–2.22)	1.11 (0.74–1.65)	0.61 (0.39–0.96)*	1.75 (1.15–2.66)**
Black	0.60 (0.38–0.94)*	1.04 (0.69–1.56)	0.68 (0.42–1.09)	0.87 (0.55–1.37)	1.25 (0.78–1.98)	0.71 (0.46–1.09)
Hispanic	1.45 (0.80–2.62)	0.40 (0.19–0.83)*	0.60 (0.31–1.15)	1.55 (0.86–2.78)	0.51 (0.22–1.17)	3.37 (1.65–6.90)**
Homelessb	1.23 (0.82–1.81)	1.74 (1.23–2.47)**	1.14 (0.75–1.74)	1.05 (0.71–1.53)	1.48 (1.00–2.19)*	0.70 (0.49–0.99)*
Abused alcohol	1.12 (0.77–1.62)	0.71 (0.51–0.98)*	0.83 (0.57–1.21)	1.43 (1.00–2.05)	0.91 (0.62–1.32)	1.36 (0.96–1.93)
Abused cocaine	1.12 (0.75–1.65)	1.12 (0.79–1.59)	1.27 (0.86–1.89)	0.83 (0.58–1.20)	0.96 (0.66–1.40)	0.98 (0.68–1.41)
Abused crack	1.46 (0.94–2.23)	3.45 (2.37–5.01)***	1.12 (0.69–1.81)	1.36 (0.87–2.13)	3.13 (2.06–4.75)***	0.48 (0.33–0.69)***
Abused heroin	3.82 (2.03–7.15)***	1.53 (0.80–2.93)	14.43 (8.12–25.65)***	0.22 (0.14–0.33)***	0.12 (0.07–0.21)***	2.15 (1.29–3.59)**
Abused marijuana	0.45 (0.31–0.66)***	0.95 (0.69–1.31)	0.70 (0.48–1.03)	0.52 (0.34–0.79)**	1.18 (0.80–1.74)	0.45 (0.31–0.66)***
Abused≥1 substance	0.69 (0.49–0.95)*	1.39 (1.02–1.90)*	0.45 (0.32–0.63)***	1.94 (1.40–2.69)***	3.41 (2.44–4.76)***	0.43 (0.31–0.59)***
Heavy usec	1.51 (1.09–2.08)*	1.32 (0.98–1.78)	1.16 (0.83–1.62)	1.32 (0.95–1.83)	1.20 (0.85–1.69)	1.23 (0.89–1.69)
Psychiatric comorbidity	2.34 (1.66–3.28)***	1.01 (0.71–1.43)	0.75 (0.50–1.13)	2.41 (1.67–3.48)***	1.40 (0.92–2.14)	2.41 (1.71–3.40)***
Involuntary treatment	0.12 (0.07–0.19)***	0.56 (0.39–0.78)**	0.61 (0.41–0.90)*	0.22 (0.12–0.40)***	0.96 (0.62–1.50)	0.25 (0.14–0.44)***

^aMedian age was 31 years.

^bEver homeless for two consecutive nights in the five years preceding index treatment.

^cAssessment of the severity of substance abuse was based on one question asked for each type of substance: “When you used [the substance] in the five years before [index treatment], on the average, about how many days did you use [the substance] during a month?” Severity measure used here was calculated as the maximum number for all the substances that the respondent reported using. For these analyses, the severity variable was dichotomized based on the median (>20 days of use per month=1, ≤20 days per month=0).

^*P<.05

^**p<.01

^***p<.001

Effectiveness of Modalities

The results of the effectiveness analyses adjusting for various selection factors mainly confirmed the unadjusted results (Table 3). With regard to abstinence, there was only one statistically significant finding. Residential modality clients with a low propensity for residential treatment compared to outpatient drug-free treatment (stratum one) had a lower rate of abstinence.

Table 3.

Comparisons of Effectiveness of Modalities, Defined as Abstinence or Reduction in Use and Stratified According to the Propensities of Receiving Treatment in One Modality versus Another

Propensity Score Strata	N in Modalitiesa	Abstinence OR (95% CI)	Reduction in Use OR (95% CI)
Inpatient (INP) versus Outpatient Drug-free (OP)
1 Propensity: ↓INP↑OP	NINP=17	NOP=160	1.42 (0.43–4.58)	3.38 (1.07–10.67)*
2	64	116	0.88 (0.41–1.88)	2.01 (1.04–3.87)*
3	82	92	1.08 (0.48–2.41)	0.83 (0.44–1.55)
4	125	52	0.92 (0.45–1.91)	0.75 (0.38–1.49)
5 Propensity: ↑INP↓OP	145	32	0.67 (0.26–1.71)	0.91 (0.40–2.06)
Conditional logistic regression	433	452	0.96 (0.67–1.38)	1.14 (0.84–1.56)
Residential (RES) versus Outpatient Drug-free (OP)
1 Propensity: ↓RES↑OP	NRES=34	NOP=138	0.27 (0.08–0.97)*	0.50 (0.20–1.24)
2	70	103	0.52 (0.20–1.33)	0.91 (4.50–1.85)
3	85	85	1.57 (0.63–3.89)	0.66 (0.34–1.26)
4	119	53	1.77 (0.71–4.40)	0.95 (0.48–1.89)
5 Propensity: ↑RES↓OP	136	35	1.52 (0.62–3.74)	1.27 (0.60–2.72)
Conditional logistic regression	444	414	0.95 (0.66–1.36)	0.81 (0.61–1.09)
Detox/Methadone (DT/MT) versus Outpatient Drug-free (OP)
1 Propensity: ↑DT/MT↓OP	NDT/MT=30	NOP=144	1.63 (0.44–6.03)	2.50 (0.90–6.96)
2	42	131	1.52 (0.61–3.74)	0.81 (0.36–1.82)
3	82	92	0.87 (0.41–1.84)	1.35 (0.70–2.59)
4	96	77	0.70 (0.33–1.45)	0.67 (0.35–1.29)
5 Propensity: ↑DT/MT↓OP	165	8	0.86 (0.10–7.56)	1.09 (0.24–5.00)
Conditional logistic regression	415	452	0.86 (0.58–1.26)	1.08 (0.78–1.49)
Inpatient (INP) versus Detox/Methadone (DT/MT)
1 Propensity: ↓INP↑DT/MT	NINP=21	NDT/MT=49	1.65 (0.51–5.35)	0.95 (0.36–2.47)
2	59	111	1.14 (0.49–2.63)	1.46 (0.74–2.90)
3	101	72	0.67 (0.30–1.50)	0.81 (0.41–1.59)
4	113	53	1.84 (0.77–4.42)	2.30 (1.07–4.91)*
5 Propensity: ↑INP↓DT/MT	139	30	0.69 (0.21–2.26)	0.72 (0.29–1.79)
Conditional logistic regression	433	415	1.32 (0.91–1.93)	1.11 (0.81–1.51)
Residential (RES) versus Detox/Methadone (DT/MT)
1 Propensity: ↓RES↑DT/MT	NRES=8	NDT/MT=167	1.34 (0.25–7.19)	0.37 (0.07–1.93)
2	59	115	0.82 (0.30–2.23)	0.63 (0.30–1.35)
3	107	68	0.49 (0.21–1.14)	1.01(0.50–2.03)
4	137	38	0.94 (0.32–2.71)	0.69 (0.28–1.72)
5 Propensity: ↑RES↓DT/MT	146	27	1.63 (0.52–5.06)	1.24 (0.49–3.11)
Conditional logistic regression	457	415	0.98 (0.65–1.48)	0.74 (0.53–1.03)
Inpatient (INP) versus Residential (RES)
1 Propensity: ↓INP↑RES	NINP=26	NRES=152	1.68 (0.68–4.16)	1.80 (0.73–4.49)
2	54	124	1.09 (0.49–2.44)	1.49 (0.75–2.96)
3	88	90	0.77 (0.33–1.84)	1.62 (0.84–3.15)
4	111	67	1.85 (0.74–4.59)	1.01(0.52–2.00)
5 Propensity: ↑INP↓RES	154	24	4.57 (0.84–24.88)	2.86 (0.85–9.55)
Conditional logistic regression	433	457	1.36 (0.94–1.97)	1.59 (1.17–2.17)**

Note: Analyses within each propensity quartile were weighted by the probability weights provided by SROS.

^aNumbers for some modalities vary across comparisons because of missing data in the independent variables used to derive propensity scores for those comparisons.

^*P<.05

^**P<.01

^***P<.001

Analyses in terms of reduction in use yielded few statistically significant findings (Table 3). There were significant but paradoxical differences between inpatient and outpatient drug-free programs in the two strata representing clients with the lowest propensity to be treated in inpatient programs. Clients in these strata had better outcomes in inpatient programs. This means that clients with less psychiatric comorbidity and less heavy substance use, for example, are more likely to reduce their use following treatment in inpatient settings compared with outpatient drug-free settings. The proportion of clients with psychiatric comorbidity in strata one and two were 10 percent and 20 percent respectively, compared to 60 percent in the fifth stratum. Similarly, the proportion of clients with heavy substance use (more than 20 days/month) in strata one and two were 31 percent and 41 percent respectively, compared to 75 percent in the fifth stratum.

There was also a statistically significant difference in reduction of use when comparing inpatient and detox/methadone modalities. This finding, however, was limited to stratum four of this comparison and did not follow a trend across strata, nor was it reflected in the conditional logistic regression analysis.

The conditional logistic regression analysis revealed a significant difference between inpatient and residential modalities (Table 3). Clients in the inpatient modality were more likely to reduce their use (odds ratio=1.59). This difference was also reflected in stratified comparisons of these modalities as all odds ratios were larger than one. However, the statistical tests for comparisons within strata were not significant (perhaps because of the smaller number of subjects in each stratum compared to the total sample used in conditional logistic regression).

Cost-effectiveness of Treatment Modalities

Nonstratified analyses revealed a number of differences across modalities in the cost per abstinent case. The costs were $15,600 (95 percent CI=$11,900–$20,700) in the inpatient modality, $14,900 (95 percent CI=$12,400–$17,400) in the residential modality, $12,400 (95 percent CI=$6,600–$17,000) in the detox/methadone modality, and $6,300 (95 percent CI=$5,200–$7,900) in the outpatient drug-free modality. Cost of successful treatment in the outpatient drug-free modality was lower than both inpatient and residential modalities at a statistically significant level but not lower than the detox/methadone modality. Average cost incurred across all modalities was $12,400 (95 percent CI=$10,700–$20,700) per abstinent case.

There were also considerable differences across modalities in cost per case of reduced use. The costs were $6,100 (95 percent CI=$4,600–$7,600) in the inpatient modality, $6,700 (95 percent CI=$5,700–$7,800) in the residential modality, $4,600 (95 percent CI=$3,800–$5,600) in the detox/methadone modality, and $2,400 (95 percent CI=$2,200–$2,800) in the outpatient drug-free modality. In this case, the cost of successful treatment in the outpatient drug-free modality was lower than all other modalities, including detox/methadone, at a statistically significant level. Average cost incurred across all modalities was $4,900 (95 percent CI=$4,400–$5,500) per case of reduced substance use.

Results of stratified comparisons of cost-effectiveness ratios supported the nonstratified cost-effectiveness analysis findings (Table 4). With regard to abstinence, inpatient, residential, and detox/methadone modalities were all about two times more costly than outpatient drug-free treatment, with no clear trend across propensity strata. Thus, for example, when comparing inpatient and outpatient drug-free modalities, treatment of a group of clients with more heavy substance use and a higher rate of psychiatric comorbidity (higher propensity for inpatient modality) was not more cost-effective in inpatient compared with outpatient facilities. There were no significant effects in comparing inpatient versus detox/methadone, residential versus detox/methadone, and inpatient versus residential modalities.

Table 4.

Comparisons of Cost-effectiveness of Modalities Stratified According to the Propensities of Receiving Treatment in One Modality versus Another.

Propensity Score Strata	N in Modalitiesa		Abstinence Ratiob (95% CI)	Reduction in Use Ratiob (95% CI)
Inpatient (INP) versus Outpatient Drug-free (OP)
1 Propensity: ↓INP↑OP	NINP=17	NOP=160	1.14 (0.51–4.10)	1.04 (0.62–1.79)
2	64	116	1.93 (0.98–3.65)	1.66 (1.11–2.42)**
3	82	92	2.58 (1.14–6.12)*	2.56 (1.50–4.64)***
4	125	52	3.40 (1.63–7.48)**	3.95 (2.48–6.67)***
5 Propensity: ↑INP↓OP	145	32	2.47 (0.92–6.69)	1.68 (0.74–3.45)
Total group	433	452	2.39 (1.71–3.44)***	2.37 (1.83–3.07)***
Residential (RES) versus Outpatient Drug-free (OP)
1 Propensity: ↓RES↑OP	NRES=34	NOP=138	1.64 (0.80–4.10)	2.20 (1.21–4.19)**
2	70	103	2.92 (1.41–8.28)*	2.49 (1.49–4.29)***
3	85	85	1.88 (0.76–4.27)	3.41 (1.99–5.68)***
4	119	53	2.06 (0.72–4.40)	3.36 (2.28–5.06)***
5 Propensity: ↑RES↓OP	136	35	3.96 (1.64–7.83)*	3.74 (2.20–6.42)***
Total group	444	414	2.35 (1.71–3.09)***	2.67 (2.19–3.23)***
Detox/Methadone (DT/MT) versus Outpatient Drug-free (OP)
1 Propensity: ↓DT/MT↑OP	NDT/MT=30	NOP=144	1.81 (0.53–9.39)	1.54 (0.55–4.28)
2	42	131	1.45 (0.57–3.86)	1.56 (0.82–2.98)
3	82	92	1.83 (0.87–3.72)	1.75 (1.05–3.02)*
4	96	77	2.37 (1.13–4.97)*	1.94 (1.21–3.27)*
5 Propensity: ↑DT/MT↓OP	165	8	1.12 (0.28–3.21)	0.99 (0.21–4.50)
Total group	415	452	1.97 (1.40–2.85)***	1.85 (1.47–2.32)***
Inpatient (INP) versus Detox/Methadone (DT/MT)
1 Propensity: ↓INP↑DT/MT	NINP=21	NDT/MT=149	0.37 (0.17–1.30)	0.55 (0.32–1.04)
2	59	111	0.98 (0.43–2.46)	1.03 (0.57–1.99)
3	101	72	1.77 (0.84–3.64)	1.50 (0.86–2.79)
4	113	53	0.73 (0.25–1.88)	1.06 (0.52–2.03)
5 Propensity: ↑INP↓DT/MT	139	30	1.28 (0.27–4.19)	1.08 (0.47–2.74)
Total group	433	415	1.21 (0.84–1.85)	1.27 (0.97–1.74)
Residential (RES) versus Detox/Methadone (DT/MT)
1 Propensity: ↓RES↑DT/MT	NRES=8	NDT/MT=167	0.43 (0.10–1.29)	1.25 (0.31–3.68)
2	59	115	1.33 (0.52–3.60)	1.64 (0.98–3.26)
3	107	68	1.89 (0.93–3.92)	1.66 (0.93–2.94)
4	137	38	1.12 (0.38–2.57)	1.46 (0.83–2.45)
5 Propensity: ↑RES↓DT/MT	146	27	1.81 (0.42–5.02)	2.91 (1.60–5.39)**
Total group	457	415	1.19 (0.85–1.71)	1.44 (1.13–1.80)**
Inpatient (INP) versus Residential (RES)
1 Propensity: ↓INP↑RES	NINP=26	NRES=152	0.60 (0.32–1.20)	0.62 (0.36–1.07)
2	54	124	0.66 (0.32–1.72)	0.60 (0.33–1.24)
3	88	90	1.48 (0.65–2.98)	0.89 (0.51–1.56)
4	111	67	0.88 (0.39–2.03)	0.92 (0.48–1.79)
5 Propensity: ↑INP↓RES	154	24	0.49 (0.16–1.39)	0.67 (0.25–1.56)
Total group	433	457	1.01 (0.71–1.44)	0.88 (0.68–1.14)

^aNumbers for some modalities vary across comparisons because of missing data in the independent variables used to derive propensity scores for those comparisons.

^bRatio of cost of successful treatment in the first modality over the second.

^*P<.05

^**p<.01

^***p<.001.

Statistical significance was assessed by repeating the analyses with confidence level set at 95%, 99% and 99.9% and comparing the resulting confidence intervals. If the 95% confidence interval did not include 1 but the 99% did, the P level was estimated at <.05, etc.

When the outcome was defined as reduction in use, the finding of superior cost-effectiveness of outpatient drug-free modality in comparison with the other three modalities was replicated (Table 4). In addition, residential programs appeared to be more expensive than detox/methadone programs, especially among clients who are more likely to be selected into residential programs (fifth propensity stratum). The better outcome for the inpatient modality compared to the residential modality with regard to reduced use of substances did not translate into more cost-effective care in inpatient modality due to higher average cost of care in the inpatient modality.

Sensitivity Analyses

Results of the sensitivity analyses confirmed the results of the main analyses reported above. The outpatient drug-free modality was the most cost-effective. The cost per abstinent case was $23,600 (95 percent CI=$17,600–$32,300) in the inpatient modality, $23,500 (95 percent CI=$18,800–$30,500) in the residential modality, $18,700 (95 percent CI=$14,300–$25,800) in the detox/methadone modality, and $8,000 (95 percent CI=$6,700–$9,800) in the outpatient drug-free modality.

The cost per case of reduced use was $11,800 (95 percent CI=$9,400–$15,500) in the inpatient modality, $12,600 (95 percent CI=$10,700–$14,900) in the residential modality, $8,300 (95 percent CI=$6,700–$10,300) in the detox/methadone modality, and $4,000 (95 percent CI=$3,500–$4,800) in the outpatient drug-free modality.

It is worth noting that even in the worst-case scenario, where any future treatment was counted as a treatment failure, the least cost-effective treatment (the inpatient modality) costs $23,600 per abstinent case and $11,800 per reduced-use case. If we assume that eliminating an individual's substance use provides that person with at least 0.50 discounted additional quality-adjusted-life-years (QALYs) over lifetime, this program is still below the generally accepted standard of $50,000 per QALY for health care decisions (Laupacis et al. 1992).

Discussion

When compared to other health interventions, all of the substance abuse treatment modalities examined appear to be cost-effective (Laupacis et al. 1992). The outpatient drug-free treatment modality, however, appears to be most cost-effective, even for clients who are more likely to choose (or be referred to) treatment in other modalities. While variations in settings, modalities, and outcomes make comparisons of cost-effectiveness estimates across studies difficult, our findings are, in general, consistent with the results of most prior cost-effectiveness studies of alcohol and substance abused treatment. Those studies also found that the less intensive treatment modalities, such as the outpatient drug-free modality, are more cost-effective than the more intensive ones (Longabaugh et al. 1983; Pettinati et al. 1999; Annis 1986; French 1995).

One exception is the cost-benefit analysis of the TOPS study which found that residential treatments had a higher benefit to cost ratio than other programs (Hubbard et al. 1989). The benefit estimates in that study, however, were based on reduced costs of criminal activities, making residential treatment, which reduces the opportunities for criminal activities, appear more attractive. This discrepancy between our findings and the results of the TOPS study highlight the variations in perspective and outcome domains in economic analyses of substance abuse treatment programs. While abstinence and reduction in substance use are central measures for assessment of effectiveness of treatments, other outcomes such as reduction in criminal activity may be more important when examining the societal benefits of treatments in terms of averted costs of crime (Reuter 2001).

Our findings have implications for current efforts to identify client characteristics that best predict successful outcome in one treatment setting versus another (McLellan et al. 1997). We found no evidence that, at least as it occurs in usual treatment settings, selection into programs improves effectiveness or efficiency of treatments. This is in contrast to a previous study in an HMO that found improved effectiveness among clients who “self-selected” themselves into a day-hospital setting rather than an outpatient setting (Weisner et al. 2000). This self-selected sample, however, was comprised of clients who were unable or unwilling to be randomized to the controlled trial arm of that study, raising questions about the representativeness of this sample population.

It is also noteworthy that attempts at client–problem matching in the past have produced mixed results (Pettinati et al. 1999; Shepard, Larson, and Hoffman 1999; McLellan et al. 1997). Commenting on these studies, McLellan et al. (1997) wrote, “This idea of ‘matching’ the right types of clients to the right kinds of programs has been as attractive to clinicians and administrators as it has been elusive to those who have tried to accomplish it” (p. 730). Furthermore, as these authors note, matching clients with treatments is often not feasible in the real world.

From our data, it is possible to derive a back-of-the-envelope estimate of the potential savings that would accrue had all individuals been treated in the most cost-effective treatment modality, that is, the outpatient drug-free modality. Using the sampling weights, the SROS sample represents about 950,000 clients who sought care in one of the four treatment modalities across the United States in the 12 months ending August 31, 1990. The cost of treatment in these settings amounted to approximately $2.4 billion based on our individualized cost data. Had all these individuals been treated in the outpatient drug-free modality (average cost of $1,204 per treated case), the overall cost of treatment would have amounted to only $1.1 billion—that is, a saving of more than 50 percent in the overall cost of treatment with no impact on overall treatment effectiveness!

We caution, however, about extrapolating from the SROS data to the present time. The mix and content of services has changed dramatically since the 1990s when SROS was conducted. The rate and length of inpatient treatments were reduced during this period and outpatient care replaced the more costly inpatient and residential care in many settings (Mark et al. 2000; Chen, Wagner, and Barnett 2001). Nevertheless, a 1999 survey of substance abuse treatment facilities showed that 25 percent of facilities continue to offer residential rehabilitation (Substance Abuse and Mental Health Services Administration 2001). This percentage was as high as 44 percent in some public settings. Furthermore, up to 28 percent of state facilities offered no outpatient rehabilitation programs. This may be, at least partly, attributable to the current payment systems that perversely encourage the provision of residential/inpatient care, even when it is not the clinically preferred modality (French 1995). Clearly, more attention should be paid to creating incentives that encourage the use of more cost-effective interventions.

The findings of this study should be interpreted in the context of the limitations of the SROS data and our analyses. The SROS was not a randomized clinical trial. While stratification according to propensity scores controls for the effect of observed confounders, it does not necessarily control for the effect of unobserved variables. Furthermore, only about 59 percent of the original SROS sample of 3,047 was followed-up. Rates of follow-up, however, were similar across modalities, and in analyses conducted by the SROS investigators, few baseline characteristics predicted loss to follow-up (Substance Abuse and Mental Health Services Administration 1998).

Assessment of outcomes was based on self-reports of the clients about behaviors in the distant past and was thus open to memory distortion and bias. To assess the validity of self-reports, SROS investigators conducted urinalysis for 76 percent of the sample interviewed in 1995–1996. Agreement between self-report and urinalysis ranged from 89.7 percent to 98.5 percent for drug use in the past week and from 86.2 percent to 99.0 percent in the past 24 hours. In addition, the SROS investigators tried to improve the reliability of self-reports of past events by various methods such as using color-coded calendars and interview techniques that facilitate discussion of sensitive subjects. We also note that outcomes defined for the present study (abstinence and any reduction in use) may be less prone to memory distortion than the actual number of days of use.

Lastly, our analyses were limited to the index treatment episode. Many clients continue treatment after discharge from a treatment episode and there is evidence that continuity of care may be associated with better outcomes (e.g., Ritsher, Moos, and Finney 2002). Also, a more complete picture of treatment costs should incorporate all treatments following the index treatment episode. Unfortunately, the SROS data do not provide information on the timing and the cost of treatment encounters following the index treatment episode.

In summary, this article demonstrated that, after adjusting for case-mix, various modalities of substance disorder treatment are more or less similarly effective. While all programs appear to be cost-effective, the outpatient drug-free modality is consistently the most cost-effective. Motivated by cost-containment, many health care systems are already shifting from the inpatient programs to the less costly outpatient programs. Nevertheless, it is of concern that the savings in the cost of care for individual clients are not matched by investments for extending services to a larger group of potential clients who are in need of such treatment (Chen, Wagner, and Barnett 2001; Galanter et al. 2000). The Office of National Drug Control Policy estimated that in 1996, only about 2 million of the 4.4–5.3 million Americans in need of treatment for substance abuse received such treatment (Mark et al. 2000). Savings from improved efficiency of current treatments for substance disorders may provide some of the needed funds to extend care to those who currently do not receive any such care.