Cost-Effectiveness Analysis (CEA) of Four Interventions for Adolescents with a Substance Use Disorder

Abstract

Alcohol, tobacco, and illicit drug use among adolescents in the U.S. continues to be a serious public health challenge. A variety of outpatient treatments for adolescent substance use disorders have been developed and evaluated. Although no specific treatment modality is effective in all settings, a number of promising adolescent interventions have emerged. As policy makers try to prioritize which programs to fund with limited public resources, the need for systematic economic evaluations of these programs is critical. The present study attempted a cost-effectiveness analysis (CEA) of four interventions, including family-based, individual, and group cognitive behavioral approaches, for adolescents with a substance use disorder. The results indicated that treatment costs varied substantially across the four interventions. Moreover, family therapy showed significantly better substance use outcome compared to group treatment at the 4-month assessment, but group treatment was similar to the other interventions for substance use outcome at the 7-month assessment and for delinquency outcome at both the 4-month and 7-month assessments. These findings over a relatively short follow-up period suggest that the least expensive intervention (group) was the most cost effective. However, this study encountered numerous data and methodological challenges in trying to supplement a completed clinical trial with an economic evaluation. These challenges are explained and recommendations are proposed to guide future economic evaluations in this area.

1. Introduction

National surveys show that adolescents in the U.S. continue to use alcohol, tobacco, and illicit drugs at high levels. The most recent statistics available from the Monitoring the Future survey (MTF) (www.monitoringthefuture.org) indicate that although adolescent substance use has gradually declined since 1996, the incidence of substance use remains high, with almost 50% of adolescents reporting having tried any illicit drug before completing high school (Johnston, O’Malley, Bachman, & Schulenberg, 2006). Relevant to the general public’s concern about adolescent drug use is the fact that teens themselves report being more concerned about drugs than any other issue, including crime, violence, social pressure, or academic pressure (The National Center on Addiction and Substance Abuse [CASA] at Columbia University, 2005).

Marijuana continues to be the most commonly used substance among adolescents. According to the National Survey on Drug Use and Health (NSDUH), 7.6% of youths aged 12–17 years were current marijuana users in 2004, and 536,000 youths reported using marijuana on 20 or more days in the past month (SAMHSA, OAS, 2005). In addition, marijuana use is correlated with substance abuse treatment admissions, emergency room admissions, and autopsies (Clark, Horton, Dennis, & Babor, 2002; Dennis, Babor, Roebuck, & Donaldson, 2002). Of particular concern is the fact that for many adolescents, substance use may lead to substance dependence and potentially long-term use and/or abuse (DiClemente, Hansen, & Ponton, 1996). In addition, several population-based studies suggest high rates of comorbid mental health and substance use disorders among adolescents or young adults (Kessler & Magee, 1994; Kessler, Nelson, McGonagle, Liu, Swartz, & Blazer, 1996; Lewinsohn, Rohde, Seeley, & Fischer, 1993). For example, rates of comorbid mental health conditions among adolescents entering substance abuse treatment range as high as 82% for DSM-IV criteria for Axis1 disorders and as high as 74% for two or more psychiatric disorders (Hovens, Cantwell, & Kiriakos, 1994; Rohde, Lewinsohn, & Seeley, 1996). Furthermore, adolescents with a substance use disorder are being identified at increasing rates across all sectors of health care (Aarons, Brown, Hough, Garland, & Wood, 2001). In response to the significant personal and societal costs of adolescent substance abuse and to the treatment needs of adolescents, considerable public resources have been devoted to the development of effective treatment models. Researchers have focused on developing specialized substance abuse treatment programs that target the constellation of problems commonly seen among adolescent clients, including delinquent behavior, peer drug use, school failure, social functioning and life skills, and family dysfunction (Rowe & Liddle, 2003; Waldron, Turner, & Ozechowski, 2006).

As researchers continue to explore the efficacy and effectiveness of substance abuse interventions for adolescents, gauging the economic impact of these programs is both necessary and difficult. Despite considerable methodological and empirical developments in the economic assessment of adult substance abuse programs (Barnett, Zaric, & Brandeau, 2001; Cisler, Holder, Longabaugh, Stout, & Zweben, 1998; Daley, Argeriou, McCarty, Callahan, Shepard, & Williams, 2000; French, Salomè, Krupski, McKay, Donovan, McLellan, & Durell, 2000; French, Salome, Sindelar, & McLellan, 2002a; McCollister, French, Inciardi, Butzin, Martin, & Hooper, 2003), economic evaluation techniques have not yet been systematically adopted in studies of adolescent addiction treatment. The economic evaluation of adolescent addiction treatment is considerably more complex than that of adult treatment because of the diversity of juvenile delivery systems, the absence of standardized economic instrumentation or modules, the integral participation of parents/guardians and other family members, the unique social and economic outcomes, and outcome measures with highly skewed distributions.

To improve and add to the scant number of economic evaluations of adolescent addiction interventions, the current study attempted a cost-effectiveness analysis (CEA) of a randomized clinical trial for adolescents with a substance use disorder conducted by Waldron and colleagues (2001). Four distinct interventions were compared, including individual cognitive behavioral therapy (CBT), functional family therapy (FFT), integrative treatment combining individual and family therapy (Joint), and a skills-focused psycho-educational group (Group). Resource use and associated costs for each intervention were collected and analyzed with the Drug Abuse Treatment Cost Analysis Program (DATCAP), www.DATCAP.com, (French, 2003a,b; French, Dunlap, Zarkin, McGeary, & McLellan, 1997). Two measures of effectiveness were assessed: the percent of days of marijuana use and the numerical score on a juvenile delinquency subscale.

The goals of this study began as empirical and evolved into primarily methodological because of the technical challenges that were encountered during the process of incorporating an economic evaluation (i.e., CEA) into a completed clinical trial. Despite being the most popular method in general health care program evaluation, CEA has not been widely applied as an economic evaluation method in addiction research (Gold, Siegel, Russell, & Weinstein, 1996; Drummond, O’Brien, Stoddart, & Torrance, 2005). This is mainly due to the variety and complexity of outcomes in addiction research, which make it difficult to express economic impact through one outcome, such as quality-adjusted life-years (QALYs) gained. The current study provided an opportunity to test the application of CEA methods to adolescent addiction treatment and to address three central questions: (1) What were the average weekly and episode costs of each intervention? (2) Were the reductions in percentage of days of marijuana use and delinquency score from baseline to 4 and 7 months post-baseline significantly different across the study conditions? (3) In comparing treatment costs and effectiveness measures across interventions, which intervention was the most cost-effective?

The findings of this study have both research and policy relevance because few studies have estimated the economic impact of adolescent substance abuse interventions and there is consequently no basis upon which to form a discussion or comparison of methods or results. The empirical results, documented methodological and data challenges of the economic evaluation, and corresponding research recommendations will provide important and timely information for treatment providers and health services researchers as they decide whether and how to conduct future economic evaluations of adolescent addiction interventions.

2. Background

Recent randomized field studies of adolescent outpatient treatment have evaluated a wide variety of treatment approaches, including cognitive behavior therapy (CBT), alone and in combination with a motivational interviewing approach (Kaminer, Burleson, & Goldberger, 2001; Waldron, Slesnick, Brody, Turner, & Peterson, 2001); family therapy approaches (Henggeler, Clingempeel, Brondino, & Pickrel, 2002, Henggeler, Borduin, Melton, Mann, Smith, Hall, Cone, & Fucci, 1991; Liddle, Dakof, Parker, Diamond, Barrett, & Tejada, 2001; Waldron et al., 2001; Joanning, Quinn, Thomas, & Mullen, 1992); group psychoeducational approaches (Kaminer, Burleson, & Goldberger, 2002; Waldron et al., 2001); and individual behavior therapy approaches (Diamond, Godley, Liddle, Sampl, Webb, Tims, & Meyers, 2002; Azrin, McMahon, Donohue, Besalel, Lapinski, Kogan, Acierno, & Galloway, 1994; Waldron & Kaminer, 2004). Empirical support for family-based treatments is well-established (Liddle & Dakof, 1995; Stanton & Shadish, 1997; Waldron 1997; Waldron, Turner, & Ozechowski, 2006). Family-based approaches include Functional Family Therapy (FFT) (Waldron et al., 2001), Multidimensional Family Therapy (MDFT) (Liddle et al., 2001), Multisystemic Therapy (MST) (Henggeler, Schoenwald, Borduin, Rowland, & Cunningham, 1998; Henggeler et al., 2002), Brief Strategic Family Therapy (BSFT) (Szapocznik, Perez-Vidal, Brickman, Foote, Santisteban, Hervis, & Kurtines, 1988), and Behavioral Family Therapy (BFT) (Azrin et al, 1994).

Several high-profile studies of adolescent substance abuse treatment came out of the Cannabis Youth Treatment (CYT) initiative (Dennis et al., 2002; Clark et al., 2002; Diamond et al., 2002; French, Roebuck, Dennis, Diamond, Godley, Tims, Webb, & Herrell, 2002b; French, Roebuck, Dennis, Godley, Liddle, & Tims, 2003; Petry & Tawfik, 2001; Tims, Dennis, Hamilton, Buchan, Diamond, Funk, & Brantley, 2002; Godley, Godley, Funk, Dennis, & Loveland, 2001). The goals of CYT were to develop and test short-term outpatient treatment programs for cannabis-using adolescents and to evaluate the costs, effectiveness, and economic impact of these interventions. A recent study by Dennis, Godley, Diamond, Tims, Babor, Donaldson, Liddle, Titus, Kaminer, Webb, Hamilton, & Funk (2004) evaluated the clinical effectiveness and cost effectiveness of 5 interventions during 2 trials at each of the 4 CYT sites. Interventions included Motivational Enhancement Treatment/Cognitive Behavior Therapy, 5 Sessions (MET/CBT5); Motivational Enhancement Treatment/Cognitive Behavior Therapy, 12 Sessions (MET/CBT12); Family Support Network (FSN); the Adolescent Community Reinforcement Approach (ACRA); and Multidimensional Family Therapy (MDFT). Client demographics and primary outcome measures were collected with the GAIN, www.chestnut.org/li/gain, (Dennis, Godley, & Titus, 1999) and program costs were evaluated with the DATCAP. Cost effectiveness for Trials 1 and 2 was assessed using the cost per day of abstinence and cost per person in recovery over a 12-month follow-up period.

The CYT interventions did not have a “no treatment” control condition. Intervention cost and effectiveness were therefore evaluated via site averages to estimate cost effectiveness ratios. In the first trial, the MET/CBT5 condition was the most cost effective in terms of cost per day of abstinence and cost per person in recovery. In the second trial, the ACRA intervention was more cost effective for both measures. The authors note that the clinical outcomes were similar across all interventions, but they differed considerably in treatment costs. Thus, while all interventions had desirable pre-post treatment outcomes, the less costly interventions were found to be more (economically) efficient. The authors note several limitations in their study, including the absence of a control group, and advocate for additional economic studies to better clarify the economic impact of adolescent treatment programs.

Empirical studies have demonstrated a strong relationship between family influences and adolescent substance use and have provided support for the efficacy of family-based treatments (Rowe and Liddle, 2003; Henggeler et al., 2002). In the original study (Waldron et al., 2001), on which this economic study is based, the investigators hypothesized that youths who complete either family therapy alone (FFT) or family therapy in combination with CBT (Joint) will show significantly greater reductions in marijuana use and delinquent behavior, compared to youths who did not receive family therapy (CBT or Group participants). It is also expected that the Joint intervention will be more cost effective than the other three interventions because more unique risk factors are addressed by the combination of family therapy and the individual skills training presented in CBT.

3. Methods

The current study supplements a randomized clinical trial for adolescent substance abuse funded by the National Institute on Drug Abuse. The parent study was designed to examine adolescent treatment outcomes for four interventions (Waldron et al., 2001). Participants included 120 adolescents (96 boys and 24 girls) and their families living in Albuquerque, New Mexico. Six of these adolescents did not complete any follow-up assessments and were dropped from the analyses, yielding a final sample of 114. The participants were referred to the University of New Mexico’s Center for Family and Adolescent Research for substance abuse treatment. Referral sources included the juvenile justice system (43%), public school system (31%), self or parent (21%), and other treatment agencies (5%). Most adolescents were mandated to treatment by court order, by probation officers in lieu of a court order, or by the schools in lieu of suspension or other consequences.

Youths between 13 and 17 years of age were eligible for the study if they were living at home with a primary caretaker willing to participate in the study and if they met Diagnostic and Statistical Manual of Mental Disorders diagnostic criteria for a primary substance abuse disorder (DSM-IV, 4^th ed., American Psychiatric Association, 1994). The focus of the study was illicit drug use, so youths that reported primarily abusing only alcohol and/or tobacco were excluded. Youths and family members were also excluded if the adolescent needed services other than outpatient treatment (e.g., was dangerous to self or others, needed monitored detoxification), if there was evidence of a psychotic or organic state, or if a sibling was participating in the study (Waldron et al., 2001). Consistent with national statistics, most youths reported marijuana as their primary drug of abuse.

All families referred to the center completed an intake interview to identify clinical concerns and to determine study eligibility. Eligible families were then scheduled for a pre-treatment assessment. After the initial assessment, adolescents were randomly assigned to one of four treatment conditions: individual cognitive behavioral therapy (CBT), Functional Family Therapy (FFT), integrative treatment approach combining individual and family therapy interventions (Joint), or a skills-focused psycho-educational group intervention (Group). Detailed description of the research design, clinical interventions, and treatment outcomes of this clinical trial at 4- and 7-month assessments can be found in Waldron et al. (2001).

3.1. Measures

For the purposes of the randomized clinical trial, substance use and family relationship outcomes were examined at 4 and 7 months after the initiation of treatment. The timing of these follow-up assessments was designed to correspond generally with completion of treatment and then with the time point 3 months after treatment completion. The primary measures of substance use were percentage of days of marijuana use and days of any drug use, which were obtained for all adolescents and parents with the Form 90D version (Miller & DelBoca, 1994) of the Timeline follow-back interview (TLFB) (Sobell, Maisto, Sobell, Cooper, Cooper, & Sanders, 1980). For all participants, substance use at pretreatment was examined for the previous 90-day period. At the 4-month and 7-month follow-up points, the calendar period for the TLFB interview extended back to the date of the last assessment. Collateral reports, urine screenings, and other measures were obtained to examine convergent validity of the TLFB.

We selected two outcomes for the cost-effectiveness analyses: adolescent marijuana use and delinquency scores at the 4- and 7-month follow-up assessments. Specifically, we examined the percent of days of marijuana use, as reported by the adolescent. The delinquency subscale score came from the Delinquent Behavior subscale of the Child Behavior Checklist, the Youth Self-Report (YSR) version (Achenbach and Edelbrock, 1982). Delinquency is a frequent concomitant of substance use, and the YSR provided an index of this behavior problem. This subscale is comprised of 13 items assessing the degree to which a set of delinquent behavioral tendencies characterizes a given adolescent. Each item receives a score ranging from 0 (not true) to 2 (very true or often true). Items include “I hang around with kids who get in trouble” and “I cut classes or skip school.” High scores indicate high levels of delinquent behavior.

3.2. Economic Evaluation

Although several complementary approaches exist for performing an economic evaluation of healthcare interventions (e.g., cost-effectiveness analysis, cost-utility analysis, benefit-cost analysis), cost-effectiveness analysis (CEA) was selected in the current study to examine the four interventions (CBT, FFT, Joint, Group) at 4 and 7 months after the initiation of treatment. Unlike benefit-cost analysis (BCA), which estimates the dollar value of multiple outcomes, CEA generally highlights one important outcome, such as reduced substance using days. This outcome is used to express treatment effectiveness, which is then compared with treatment cost. A cost-effectiveness ratio, typically with cost in the numerator and the effectiveness measure in the denominator (thus, a lower cost-effectiveness ratio is considered preferable), is the cornerstone of CEA. One can then compare the ratios of costs to outcome for two or more alternative programs to determine which programs are relatively more cost effective. While CEA is designed as an incremental analysis (e.g., incremental cost divided by incremental effectiveness), the interventions we examined did not include a no treatment condition and they were not strictly enhanced versions of each other. Thus, the analysis compared average costs and effectiveness rather than incremental ones.

3.3. Economic Cost Analysis

A comprehensive cost analysis provides a foundation for full economic evaluations that compares program costs and outcomes. Cost data for the four treatment conditions (CBT, FFT, Joint, and Group) were collected using the Drug Abuse Treatment Cost Analysis Program (DATCAP), www.DATCAP.com (French, 2003a,b). The DATCAP provides estimates of total program cost, annual cost per client, average weekly cost, and the average cost per treatment episode.

3.4. Effectiveness Analysis

In selecting measures of program effectiveness, two outcomes reported in the original Waldron et al. (2001) investigation stood out as representative of the primary goals of these interventions: reducing juvenile delinquency and the rate of marijuana use. We conducted two separate CEAs looking at differences in percentage of days of marijuana use and in the delinquency score during the 4-month and 7-month follow-up periods as measures of treatment effectiveness. Several other studies that have performed CEAs of substance abuse treatment have used measures of drug use such as “days of use” or “abstinence” as outcome measures of treatment success (Dennis et al., 2004; Cartwright, 1998; Kraft, Rothbard, Hadley, McLellan, & Asch, 1997; Zarkin, Lindrooth, Demiralp, & Wechsberg, 2001; Shepard, Larson, & Hoffmann, 1999).

3.5. Cost-Effectiveness Analysis (CEA)

CEA calculates and compares ratios of program cost and effectiveness across study conditions. We estimated and compared mean values for treatment cost and effectiveness. In this context, the cost-effectiveness ratios represent the average cost of achieving (1) a 1-percentage point reduction in the days of marijuana use and (2) a 1-unit reduction in the delinquency score in the respective treatment condition over the follow-up periods. As noted earlier, this is not an incremental CEA because the cost and effectiveness values are not computed relative to a control condition (i.e., a standard condition such as “treatment as usual”) or to other treatments.

4. Results

4.1. Baseline Descriptive Statistics

Table 1 presents mean values at baseline for all variables used in the economic analysis, organized by study condition. Statistically significant differences in variable means between study conditions are noted. The percent of days of marijuana use ranged from 52% (in the CBT condition) to 64.6% (in the Group condition). The delinquency subscale score ranged from 9.4 (in the FFT condition) to 11.3 (in the CBT condition). The results revealed no statistically significant differences at baseline among treatment conditions for either marijuana use or delinquency.

Table 1.

Baseline Characteristics, by Treatment Condition

	Treatment Condition
Variables	FFT (N=30)	CBT (N=31)	Joint (N=29)	Group (N=30)
Continuous Variables [mean, (sd)]
% days marijuana usea	53.1 (33.7)	52.0 (31.8)	56.7 (34.0)	64.6 (28.0)
YSR Delinquency Subscale Scoreb	9.4 (3.8)	11.3 (3.9)	11.3 (4.1)	10.3 (3.4)
Adolescent Education in Years	9.4 (0.9)	9.3 (1.2)	9.6 (1.0)	9.1 (0.8)
Parent education in years	14.1 (2.8)	13.7 (3.0)	14.7 (3.5)	13.5 (2.3)
Adolescent Age in years	15.6 (1.0)	15.6 (1.1)	15.7 (0.9)	15.5 (1.0)
Age of first drug usec,1	12.1 (2.1)2	12.0 (2.1)2	10.8 (2.7)	11.2 (2.2)
Categorical Variables (%)d
Gender
Female	20.0	16.1	24.1	16.7
Male	80.0	83.9	75.9	83.3
Ethnicity
Anglo	46.7	54.8	37.9	46.7
Hispanic	53.3	45.2	62.1	53.3
Family composition
Single-Parent	37.92	46.7	39.32	62.1
Two-Parent	62.1	53.3	60.7	37.9

FFT = Functional Family Therapy; CBT = Individual Cognitive Behavioral Therapy; Joint = combined FFT and CBT; Group = Group Cognitive Behavioral Therapy rank test

^aPast 90 days

^bRaw scores on the Delinquent Behavior subscale of the Child Behavior Checklist -- Youth Self-Report version. This subscale is comprised of 13 items assessing the degree to which a set of delinquent behavioral tendencies characterizes a given adolescent. Each item receives a score ranging from 0 (not true) to 2 (very true or often true). Items include “I hang around with kids who get in trouble” and “I cut classes or skip school.” High scores indicate high levels of delinquent behavior and vice-versa.

^cExcluding tobacco

^dReference category listed in italicized type

¹Statistically significant difference between groups, p < .10, Kruskal-Wallis equality of populations rank test

²Statistically significant difference from the GROUP condition, p < .10, Kruskal-Wallis equality of populations

The study sample included more males than females in each of the treatment conditions. The distribution of males was approximately equal across the study groups (ranging from 76% to 84%). The average age of all study participants was approximately fifteen years. The average adolescent client had completed approximately nine years of education, and the average parent had completed fourteen. The average age of first drug use ranged from 11 to 12, and this measure was statistically different between treatment conditions. Age, gender, ethnicity, and years of education for adolescents and parents did not significantly differ between the treatment conditions.

4.2. Treatment Cost

The upper part of Table 2 summarizes the results from the economic cost analysis for each treatment condition. All cost estimates are reported in 1998 dollars. Taking into account all services provided during the study year, the total annual economic cost was $16,877 for the FFT condition, $12,830 for CBT, $25,743 for Joint, and $9,471 for Group. The average weekly cost per client was $135 for FFT, $107 for CBT, $221 for Joint, and $118 for Group. Based on the median length of stay, the economic cost per client of the median treatment episode was $1,625 for FFT, $1,278 for CBT, $2,546 for Joint, and $885 for Group. Note that the Group treatment condition was the least costly on a per episode basis relative to the other treatment conditions.

Table 2.

Treatment Costs and Outcomes at 4- and 7-Months Post Baseline

	Treatment Condition
Variable	FFT	CBT	Joint	Group
Treatment cost statisticsa
Total annual cost	$16,877	$12,830	$25,743	$9,471
Annual cost per clientb	$7,061	$5,554	$11,544	$6,150
Average weekly cost per clientc	$135	$107	$221	$118
Average episode costd	$1,625	$1,278	$2,546	$885
4-month outcomes
% days marijuana use by adolescent1	25.3 (27.4)	50.6 (40.7)	38.1 (36.5)	54.8 (34.2)
YSR Delinquency Subscale Scoree	8.2 (3.4)	10.2 (3.8)	9.1 (4.2)	9.5 (3.5)
7-month outcomes
% days marijuana use by adolescent	39.8 (39.4)	51.8 (37.6)	35.4 (36.6)	40.7 (39.3)
YSR Delinquency Subscale Scoree,1	9.2 (3.8)	10.4 (4.7)	8.5 (4.2)2	9.4 (3.7)

FFT = Functional Family Therapy; CBT = Individual Cognitive Behavioral Therapy; Joint = combined FFT and CBT; Group = Group Cognitive Behavioral Therapy. Standard deviations in parentheses.

^aAll data and calculations were performed with the Drug Abuse Treatment Cost Analysis Program (DATCAP; www.DATCAP.com). All costs are reported in 1998 dollars. Some numbers may not add, divide, or multiply exactly due to rounding.

^bAnnual cost per client = total annual cost ÷ average daily census

^cAverage weekly cost per client = annual cost per client ÷ 52.14 weeks

^dAverage episode cost = average weekly cost per client x median length of stay (weeks)

^eRaw scores on the Delinquent Behavior subscale of the Child Behavior Checklist -- Youth Self-Report version. This subscale is comprised of 13 items assessing the degree to which a set of delinquent behavioral tendencies characterizes a given adolescent. Each item receives a score ranging from 0 (not true) to 2 (Very true or often true). Items include “I hang around with kids who get in trouble” and “I cut classes or skip school.” High scores indicate high levels of delinquent behavior and vice-versa.

¹Statistically significant difference between FFT and Group or CBT, p < 0.005, pair wise t-test comparisons.

²Significantly different from the CBT condition, p < 0.04, pair wise t-test.

4.3. Effectiveness

Table 2 also presents the unadjusted (i.e., for baseline differences) treatment outcomes at 4 and 7 months post-baseline by treatment condition, which were published in Waldron et al. (2001). The percent of days of marijuana use was significantly different between the conditions at 4 months, but not at 7 months. At 4 months, individuals in the Group condition reported the highest percentage of days of marijuana use (55%), while individuals in the FFT condition reported the lowest (25%). The delinquency subscale score was significantly different between the groups at 7 months, but not at 4 months. At 7 months, individuals in the CBT condition reported the highest delinquency subscale score (10.3), and individuals in the Joint condition reported the lowest delinquency subscale score (8.5).

Table 3 displays the results of the regression models for differences in the percentage of days of marijuana use and in delinquency score at the 4-month and 7-month follow-up periods. For the models measuring marijuana use, the baseline measure of marijuana use was included as a covariate to control for initial drug use. Similarly, the baseline delinquency score was included as a control in the regression models of delinquency. In both sets of analyses, the reference (i.e., omitted) category for the treatment conditions was Group, because this intervention had the lowest median episode cost (see Table 2). Accordingly, the parameter estimates for treatment conditions indicate the difference in the dependent variable (marijuana use or delinquency) for FFT, CBT, and Joint relative to the Group condition. Negative coefficient values represent lower levels of marijuana use or delinquency and suggest greater effectiveness relative to Group.

Table 3.

Regression Estimates for Percent Days of Marijuana Use and Delinquency Score from Baseline to Follow-up^a

	Outcome Variable
	% Days Marijuana Use		Delinquency Score
Explanatory Variables	4-Month Follow-Up	7-Month Follow-Up	4-Month Follow-Up	7-Month Follow-Up
Intercept	−14.11 (51.86)	57.60 (62.06)	12.68 (5.26)*	7.73 (5.93)
Treatment condition
FFT (vs. Group)	−20.11 (8.52)*	4.87 (10.19)	−0.60 (0.86)	0.15 (0.97)
CBT (vs. Group)	4.76 (8.47)	18.27 (10.14)	0.38 (0.85)	0.42 (0.96)
Joint (vs. Group)	−14.86 (8.83)	−2.00 (10.57)	−0.50 (0.89)	−1.50 (1.00)
Baseline measureb	−0.58 (0.10)**	−0.62 (0.12)**	−0.42 (0.09)**	−0.45 (0.10)**
Age	3.42 (3.15)	0.97 (3.77)	−0.34 (0.32)	−0.01 (0.36)
Age of first drug use	−3.43 (1.42)*	−1.89 (1.69)	−0.04 (0.15)	−0.13 (0.17)
Gender (Female vs. Male)	11.31 (7.90)	−4.29 (9.46)	−0.66 (0.80)	−0.39 (0.90)
Ethnicity (Anglo vs. Hispanic)	1.49 (6.45)	8.76 (7.72)	0.72 (0.66)	−0.89 (0.74)
Parent education	0.37 (1.10)	−1.00 (1.32)	−0.25 (0.11)*	−0.06 (0.13)
Single vs. two-parent family	−7.80 (6.08)	0.20 (7.27)	−0.32 (0.66)	0.22 (0.70)
R2	0.15	0.05	0.07	0.07

Note: Standard errors in parentheses.

FFT = Functional Family Therapy; CBT = Individual Cognitive Behavioral Therapy; Joint = combined FFT and CBT; Group = Group Cognitive Behavioral Therapy

^aEstimated by the method of iteratively reweighted least squares with robust standard errors.

^bBaseline measure is % days of marijuana use or the delinquency score for the corresponding dependent variables.

^*p < 0.05

^**p < 0.01

Regarding the difference in percentage of days of marijuana use from baseline to 4 months post-baseline, the FFT condition was significantly different from the Group condition, (B = -20.11, SE = 8.52, p< 0.025) and the Joint condition was marginally different from the Group condition (B = -14.86, SE = 8.83, p< 0.10). Baseline marijuana use (B= -0.58, SE = 0.10, p<0.01) and age of first drug use (B = -3.43, SE = 1.42, p< 0.05) were also significant predictors of reduced marijuana use. In the case of age of first drug use, this result indicates that adolescents whose first drug use occurred at a relatively older age exhibited lower levels of marijuana use at the 4-month follow up compared to those whose first drug use occurred at a relatively younger age. Higher levels of marijuana use at baseline were predictive of lower levels of marijuana use at the follow-up.

While there was a significant reduction in the mean delinquency score from baseline (mean = 10.41, sd = 3.66) to the 4 month assessment (mean = 9.17, sd = 3.64), F(1,110) = 12.71, p < .001, eta² =0.10) for the full sample, there were no significant between-treatment effects in the regression model at the 4-month follow-up on the delinquency measure. The findings only indicate a significant effect for years of parent education (B = -0.40, SE = 0.14, p< 0.01), suggesting lower levels of delinquent behavior for adolescents of parents with relatively more years of education.

For percentage of days of marijuana use at 7-months post-baseline, CBT exhibited marginally worse outcomes than the Group condition (B = 18.27, SE = 10.14, p < 0.10). The coefficient estimates for FFT and Joint were not significant, nor were any of the estimates for the covariates. Again, although the results revealed a significant reduction in the mean delinquency score from baseline to the 7 month assessment (mean = 9.28, sd = 4.09), F(1,110) = 9.75, p < .002, eta² =0.08) for the full sample, the multivariate analyses for delinquency score at 7-months post-baseline indicated that none of the between-condition comparisons was statistically significant. Similarly, no significant group differences were found for the substance use outcome at 7-months post-baseline.

4.4. Cost-Effectiveness

Normally, a formal CEA is appropriate and informative when more costly services or interventions produce better outcomes. Based on the rank order of median episode cost yielding Group as the least expensive ($885) and Joint the most expensive ($2,546), we expected Joint to produce the best outcomes and Group to produce the worst. The regression estimates in Table 3 did not support most of our expectations. In particular, although the FFT intervention showed significantly greater reduction in marijuana use than Group over the 4-month assessment, by the 7-month assessment, none of the more costly interventions (FFT, CBT, Joint) were significantly more effective than Group for marijuana use or delinquency outcomes. In this case, a more complex CEA reduces to a simple cost-minimization analysis, whereby the lowest cost intervention (Group) is deemed more cost effective than all other interventions given that it generated similar outcomes.

5. Methodological Challenges and Research Recommendations

The expectation for this study, consistent with Waldron et al. (2001), was that adolescents who completed either family therapy alone (FFT) or family therapy in combination with CBT (the Joint condition) would have better marijuana use and delinquency outcomes than those who did not receive family therapy. Results showed that over the initial assessment period (4-months post-baseline) FFT was associated with greater reductions in substance use compared to the Group intervention. By the 7-month assessment, however, the least expensive Group condition also showed significant reductions in marijuana use, as well as delinquency. These findings should be viewed in the context of numerous methodological challenges we encountered in attempting to supplement a randomized clinical trial with a CEA to study the economic impact of adolescent addiction programs. The main challenges are explained below.

The present study selected two outcomes (drug use and delinquency) and then analyzed them separately to quantify the relative effectiveness of the four interventions. Focusing on a single outcome to express treatment effectiveness is contrary to the nature of addiction treatment, which presumably will have multiple effects on program clients and their communities. A recent study by Sindelar, Jofre-Bonet, French, and McLellan (2004) provides a useful overview of the limitations associated with using CEA in addiction research. As a solution, the authors suggest conducting separate CEAs with multiple outcomes and then comparing them to see if the results (i.e., the cost-effectiveness ratios) are in conflict. If one intervention is found to be consistently more cost effective across a range of outcomes, then one can more confidently advocate for investing in that intervention. This advice resonates with many of the previous findings in the adolescent treatment literature, which have shown that there may be several effective treatment styles for the adolescents (e.g., Dennis et al., 2004).

Another important limitation pertains to sample size. The relatively small sample (N=120) did not provide substantial power for the multivariate analyses. Although we tried to follow the existing literature and include a common set of explanatory variables, the models may suffer from omitted variables bias. Unfortunately, we did not feel confident implementing any statistical techniques to improve power (e.g., bootstrapping) or deal with potential biases in other areas (e.g., proxy measures for omitted variables, instrumental variables) because the number of subjects in each of the four groups was very small. Although samples of this size may be fairly common for randomized trials and other clinical studies, it severely limits the range of economic analyses that can be performed.

The sample size and location of the trial could also limit the generalizability of the findings. Waldron et al. (2001) found that even though half the sample was Hispanic, the outcomes were similar for Whites and Hispanics, suggesting some degree of generalizablility to ethnic minority adolescents. Again, a larger number of subjects would have allowed us to explore generalizability to a greater extent.

Subjects were interviewed at baseline and then again at 4-months and 7-months post-baseline. Since all four of the interventions were structured to last approximately 16 weeks, the 4-month follow-up can be considered an “end of treatment” interview for many adolescents. Similarly, the 7-month follow-up corresponds to approximately 3 months post discharge. Thus, the outcomes analyzed in this study are very short-term measures of effectiveness for the CEA. If the interventions require a longer follow-up period to begin showing differences across the conditions, then this could explain why most of the between-group comparisons were not statistically significant in the regression models. Regardless, future economic evaluations of adolescent addiction interventions should strive for outcome data that are measured at 6-months post discharge and ideally at 12-months as well.

CEA is meant to be an incremental analysis that compares doing a little more or a little less of something to capture marginal variations in costs and effectiveness across standard and enhanced interventions (Gold et al., 1996). We did not have a strict “no treatment” or “treatment as usual” control group in this study, and thus the incremental association can only be inferred by comparing each intervention with baseline values for the effectiveness measures. Future economic evaluations should include a control condition, whenever possible.

The results of the current study indicate that FFT had better substance use outcome relative to the Group intervention at the 4-month follow-up. At the 7-month follow-up, Group was similar to the other conditions on both marijuana use and delinquency outcomes, rendering it more cost effective. These findings highlight two important issues. First, some researchers note that CEA is most useful for determining which programs are most cost effective at attaining primary clinical objectives, such as abstinence or days of continued drug use. From an economic perspective, this may leave out a number of other important outcomes related to drug use such as criminal activity, health care utilization, educational advancement, and productive social functioning. Second, the findings raise the issue of when treatment effects become apparent. The 3-month period immediately following a course of outpatient substance abuse treatment is a period of elevated risk for substance use relapse (Brown, Vik, & Creamer, 1989; Ramo, Anderson, Tate, & Brown, 2005). An intervention that produces more immediate effects has the potential for greater harm reduction and for the mitigation of negative consequences and costs during a critical period during which substance-abusing youth might be particularly at risk. For example, if an adolescent was on the verge of being placed outside the home (e.g., residential care, foster placement), continuing to live at home might be contingent on an immediate treatment response. Moreover, cost effective treatment for this youth might be different than cost effective treatment for a youth whom latency of treatment response is less critical. To understand the full range of costs and consequences associated with these interventions a broader approach to economic evaluation is required, such as benefit-cost analysis (BCA), which values multiple outcomes in monetary terms and estimates the net benefits associated with treatment alternatives.

Despite these shortcomings, CEA has certain advantages over other economic evaluation methods such as BCA or cost-utility analysis (CUA). First, CEA is the preferred method for evaluation of general health care programs and is thus widely recognized and understood by medical providers and policy officials. Second, CEA is somewhat easier to conduct than BCA because it uses only one outcome measure of effectiveness, expressed in its natural units, whereas BCA requires valuing all selected outcomes in monetary units to calculate economic benefits. Finally, cost effectiveness ratios are convenient in that they provide a concise measure of “cost per unit outcome.”

Because this is one of the first economic studies of adolescent addiction treatments, health economists and other researchers should develop opportunities to test CEA further in intervention studies of adolescents with larger samples. From a policy perspective, a CEA is desirable because it provides a bottom-line measure of the returns to treatment based on a specific outcome of interest. Future studies should plan to examine a number of cost- effectiveness ratios (as suggested by Sindelar et al., 2004) to more carefully evaluate various aspects of alternative treatment programs.

6. Conclusion

When considered in the context of the methodological challenges and data limitations noted earlier, we propose that the contribution of this study is more conceptual than empirical. The application of CEA to adolescent treatment programs is a relatively new approach and warrants further study. One possibility is that we will eventually be able to develop a composite index of effectiveness (similar to QALYs for primary medical care interventions) to represent a broader range of treatment outcomes. A critical variable that needs to be considered is the avoidable costs associated with juvenile justice services. Research by Aos and colleagues (Aos, Phipps, Barnoski, & Lieb, 2001) at the Washington State Institute of Public Policy suggests that the benefit-cost ratio of FFT is 14:1 when we consider avoidable juvenile justice costs. In the interim, additional studies by the investigative team are planned that will aim to advance both CEAs and BCAs of adolescent treatment interventions.