“Cost effectiveness of computer-assisted training in cognitive-behavioral therapy as an adjunct to standard care for addiction”

Abstract

Aim

To determine the cost effectiveness, from clinic and patient perspectives, of a computer-based version of cognitive-behavioral therapy (CBT4CBT) as an addition to regular clinical practice for substance dependence.

Participants, Design and Measurements

This cost-effectiveness study is based on a randomized clinical trial in which 77 individuals seeking treatment for substance dependence at an outpatient community setting were randomly assigned to treatment as usual (TAU) or TAU plus biweekly access to computer-based training in CBT (TAU plus CBT4CBT). The primary patient outcome measure was the total number of drug-free specimens provided during treatment. Incremental cost-effectiveness ratios (ICERs) and cost-effectiveness acceptability curves (CEACs) were used to determine the cost-effectiveness of TAU plus CBT4CBT relative to TAU alone. Results are presented from both the clinic and patient perspectives and are shown to be robust to (i) sensitivity analyses and (ii) a secondary objective patient outcome measure.

Findings

The per patient cost of adding CBT4CBT to standard care was $39 ($27) from the clinic (patient) perspective. From the clinic (patient) perspective, TAU plus CBT4CBT is likely to be cost-effective when the threshold value to decision makers of an additional drug-free specimen is greater than approximately $21 ($15), and TAU alone is likely to be cost-effective when the threshold value is less than approximately $21 ($15). The ICERs for TAU plus CBT4CBT also compare favorably to ICERs reported elsewhere for other empirically-validated therapies, including contingency management.

Conclusions

TAU plus CBT4CBT appears to be a good value from both the clinic and patient perspectives.

1. Introduction

Cognitive-behavioral therapy (CBT) has been shown to be highly effective in treating a wide range of substance use disorders (Carroll and Onken, 2005; Irvin et al., 1999; DeRubeis and Crits-Christoph, 1998). Nevertheless, despite evidence of positive and durable outcomes (Anton et al., 2006; Carroll et al., 1994), CBT has not been adopted widely in clinical practice due primarily to (1) limited availability of professional and specialty training programs that provide high-quality training, supervision, and certification in CBT (Weissman et al., 2006), (2) high rates of clinician turnover and lack of a CBT-trained workforce in many treatment settings (McLellan and Meyers, 2004; McLellan et al., 2003; McLellan et al., 2000), and (3) the relative complexity and cost of training clinicians in CBT (Sholomskas et al., 2005; Morgenstern et al., 2001).

To broaden the availability of CBT in clinical practice, Carroll and colleagues (Carroll et al., 2008) developed a computer-based version of CBT (CBT4CBT) that can be implemented as an adjunct to treatment as usual (TAU) in clinics lacking a CBT-trained workforce. In a recent randomized clinical trial, adding CBT4CBT to TAU was demonstrated to be both feasible and effective for improving outcomes among a heterogeneous sample of individuals seeking treatment for addiction at a community-based substance abuse treatment clinic (Carroll et al., 2008). Moreover, the effects of CBT4CBT compared with TAU alone were shown to be durable at a 6 month follow-up (Carroll et al., 2009). However, an important unanswered question is whether the additional costs of adding computer-assisted training to TAU are justified by improved outcomes. Like any new technology, integration of computer-assisted training into treatment will incur some costs, but without data on the cost-effectiveness of adding CBT4CBT, decision-makers have little guidance in determining whether the additional expenditures are worthwhile investments. While it is widely assumed that computer-assisted and other `e-therapies' have high potential as cost-effective enhancements of standard therapies, cost-effectiveness studies of e-therapies are rare and to our knowledge none have been done in the field of addiction.

In this study, we use incremental cost-effectiveness ratios (ICERs) and cost-effectiveness acceptability curves (CEACs) to define ranges of values over which CBT4CBT as a treatment adjunct would be considered likely to be cost effective compared to TAU alone, for increasing the number of drug-free specimens provided during treatment (i.e., negative for cocaine, opioids, marijuana and alcohol). Consistent with our previous work evaluating the cost-effectiveness of empirically-validated therapies for substance use disorders (Olmstead and Petry, 2009; Olmstead et al., 2007a; Olmstead et al., 2007b; Sindelar et al., 2007), analyses are conducted from the perspective of the clinic. In addition, because the CBT4CBT program requires a time commitment from patients above and beyond that required by treatment as usual, we conduct separate analyses from the perspective of the patient. Ideally, the study would have considered the cost-effectiveness of CBT4CBT from the societal perspective as well. However, due to a lack of data on societal outcomes (e.g., crime, spread of disease, family functioning), this broader analysis was not possible. Nevertheless, both of the perspectives considered in this study (i.e., clinic and patient) are important determinants of treatment uptake inasmuch as adoption decisions are made at the clinic level, while decisions to accept/attend treatment are made by the patient (Jones et al., 2009).

To check the robustness of our results, we (i) conduct sensitivity analyses on several key cost parameters and (ii) analyze a secondary objective patient outcome measure: the longest duration of abstinence during treatment. Use of these methods also permitted some comparison of the ICERs for CBT4CBT to the ICERs of other empirically-validated therapies for substance use disorders from our previous studies.

2. Methods

Cost-effectiveness analyses were conducted using patient outcomes and resource utilization data collected by the original effectiveness trial of CBT4CBT (Carroll et al., 2008). To these we added cost data obtained from the clinic and the patients where the trial took place. Methods and results of the effectiveness study are described in the main report of study design and outcomes (Carroll et al., 2008) and are thus summarized only briefly below, followed by a description of the analytical methods used for the cost-effectiveness analysis.

The randomized clinical trial evaluated the efficacy of CBT4CBT as an adjunct to standard outpatient treatment and delivered to a heterogeneous sample of individuals seeking treatment for addiction. All participants in the trial met DSM-IV criteria for any current substance dependence disorder (including cocaine, opioids, marijuana, or alcohol) and were recruited from individuals seeking treatment at Liberation Program's Mill Hill Clinic, a community-based outpatient substance abuse treatment provider in Bridgeport, Connecticut, USA. Exclusion criteria were kept to a minimum to facilitate recruitment of a clinically representative sample of individuals seeking treatment in a community setting. Specifically, individuals were excluded only if (i) they had not used alcohol or illegal drugs within the past 28 days or failed to meet DSM-IV criteria for a current substance dependence disorder, (ii) had an untreated psychotic disorder that precluded outpatient treatment, or (iii) were unlikely to be able to complete 8 weeks of outpatient treatment due to a planned move or pending court case from which incarceration was likely to be imminent. Study participation was voluntary and participants provided written informed consent as approved by the Yale University School of Medicine Human Investigations Committee. The study intervention lasted 8 weeks. The final study sample comprised 77 individuals who were randomly assigned to either TAU alone, or TAU plus biweekly access to computer-based training for CBT (TAU plus CBT4CBT). Of these 77 individuals, three were incarcerated and one dropped out immediately prior to the onset of treatment, resulting in 73 individuals who were exposed to the study treatments. No significant differences between treatment conditions were found on any of the participant demographic, substance use or psychosocial functioning variables measured at baseline.

2.1. Treatments

All participants received standard treatment (TAU) at the clinic, typically comprising one individual and one group session per week. On average, individual sessions lasted 45 minutes (plus an additional 15 minutes of administrative time for taking notes), while group sessions lasted 90 minutes (plus an additional 30 minutes of administrative time for taking notes). In addition, group sessions averaged 7 participants and were led by either one or two counselors 90% and 10% of the time, respectively. The counseling program's theoretical orientation is described as eclectic, encompassing principles of basic drug counseling, including facilitation of self-help involvement. While therapists said they were familiar with principles of evidence-based therapies such as cognitive behavioral therapy and motivational interviewing, they did not receive training or supervision in these approaches while the trial was ongoing. As suggested by our recent analysis of treatment as usual in this and other settings (Santa Ana et al., 2008), it is likely that participants randomly assigned to the TAU condition in this study had minimal exposure to CBT.

Participants randomly assigned to the TAU plus CBT4CBT condition, in addition to receiving treatment as usual as described above, also were provided biweekly access to the computer program in a small private room within the clinic. A research associate provided guidance to participants through their initial use of the CBT4CBT program and was available to answer questions and assist participants each time they used the program. The CBT4CBT program comprised six lessons, or modules, and was based closely on a CBT manual published by the U.S. National Institute on Drug Abuse (Carroll, 1998) used in several previous randomized controlled trials in a range of substance-using populations (Carroll et al., 2006; Carroll et al., 2004; Carroll et al., 1994). The program was designed to be user-friendly and required no previous experience with computers and only minimal use of text-based material. A range of formats was used to present the material, including graphic illustrations, videotaped examples, verbal instructions, audio voiceovers, interactive assessments, and practice exercises. The first module also included a brief explanation of how to use and navigate the program; upon completing the first module, participants were able to access the modules in any order they preferred and repeat any section or module as many times as desired. Each module concluded with the narrator summarizing the key points covered, followed by the actors demonstrating how they would complete the “homework” or practice assignment for that module. Participants were then given an identical homework assignment and a reminder sheet to take with them. Each module was designed to require about 45 minutes to complete, depending on how fast the user navigated the program and the amount of material he/she chose to access or repeat.

All participants were administered the Structured Clinical Interview for DSM-IV (SCID) (First et al., 1995) before random assignment to establish substance use and psychiatric diagnoses. Participants also met twice weekly during the study with an independent clinical evaluator who collected urine and breath specimens, assessed recent substance use (Varian CupKit 501), and monitored other clinical symptoms.

2.2. Cost-effectiveness analysis

Incremental cost-effectiveness analysis (ICEA) was used in the present study because CBT4CBT as a treatment adjunct was expected to add costs to TAU alone from both the clinic perspective (e.g., computer, office space) and patient perspective (e.g., time spent using the CBT4CBT program, time spent completing homework assignments). In addition, ICEA facilitates comparisons with our previous studies of ICERs of empirically-validated therapies (Olmstead et al., 2007a; Olmstead et al., 2007b; Sindelar et al., 2007). To calculate ICERs from each perspective, we first calculated the relevant unit costs using cost data obtained from the clinic (e.g., average counselor salaries during the study period, typical amount of time to conduct a urinalysis test, cost of a urine testcup) and the patients (e.g., hourly wage, employment status) where the effectiveness study took place. Each patient's hourly wage and employment status throughout the study period were obtained via interview during the effectiveness study. Then, for each participant, we multiplied the resources used by the unit costs. Data on resources used (e.g., number of counseling sessions attended, number of urine tests submitted, minutes spent using the CBT4CBT program, number of homework assignments completed) came from the effectiveness study (Carroll et al., 2008). Each patient accessed the CBT4CBT program through an ID/password system that was linked to an imbedded database, thereby allowing the computer to track, for each patient, time spent using the program and number of homework assignments completed. The average variable cost per participant in each condition was then calculated (for both the clinic and patient perspectives), followed by the incremental cost of TAU plus CBT4CBT over the cost of TAU alone.

The ICEA included only those costs that varied by treatment condition. Such costs included those related to counseling sessions, urine and breath specimen testing, and the CBT4CBT program. All clinic labor costs included fringe benefits (22.0%) and overhead (27.8%). The value of patient time in treatment was estimated as a function of employment status, reported hourly wage, and the Connecticut (CT) minimum wage during the study period. Specifically, patients reporting full-time employment were assigned a value of time equal to the maximum of their self-reported hourly wage and the CT minimum wage; unemployed patients were assigned the CT minimum wage; and patients reporting part-time employment were assigned a value of time equal to the weighted average of their self-reported hourly wage and the CT minimum wage.

2.2.1. Clinic perspective (cost of providing treatment)

2.2.1.1. Unit counseling costs

The unit counseling costs measure the average per participant cost of a counseling session and were estimated for individual and group therapies. These unit costs included the time spent by the counselor both in treatment and in administration (e.g., taking notes before or after the session); for group therapy this was prorated by the average number of patients in a session.

2.2.1.2. Unit testing cost

The unit testing cost measures the average cost per urine and breath test and included material costs (breathalyzer tubes and testcups) and time spent by staff administering the test.

2.2.1.3. CBT4CBT program costs

CBT4CBT program costs comprise four components: software costs, hardware costs, office space costs, and the cost of staff support to participants using the program. Our base cost-effectiveness analysis reflected the CBT4CBT program costs as implemented in the trial (with one exception noted below). However, because these costs may vary depending on how the program is implemented, we conducted sensitivity analyses to consider alternative implementation scenarios. Each of the four components of the CBT4CBT program costs is discussed below.

The software development costs for the CBT4CBT program were approximately $200,000. However, because these software development costs are “first-copy costs,” they were excluded from the base cost-effectiveness analysis. As defined in Gold et al. (1996), “first-copy costs” are costs incurred in developing the first copy of an item (e.g., the CBT4CBT program), independent of the number of units provided once the first unit is produced. Once the program has already been developed and the decision is whether to use the intervention, then first-copy costs should be excluded from the cost-effectiveness analysis (Babor et al., 2006; Gold et al., 1996). However, our sensitivity analyses explored the possibility that the CBT4CBT program may be commercialized in the future, thereby necessitating a per participant fee for using the software.

The outpatient clinic dedicated a computer and a small private office for the exclusive use of the CBT4CBT program for those assigned to that condition, for the duration of the effectiveness study. Due to the relatively small number of participants in this condition (compared to a non-research, clinical implementation of CBT4CBT), this decision resulted in a utilization rate for the dedicated resources (i.e., computer and office) of less than 1%. In non-research settings, clinics would likely choose to spread the fixed costs of the computer and office over a much larger number of users, either by making CBT4CBT available to more patients or by sharing these valuable resources with other clinic users. Therefore, for the purposes of estimating the per participant costs for the computer and office space in the present study, our base scenario assumed a throughput of 100 patients per year, resulting in a utilization rate for the dedicated resources of approximately 5%. In addition, sensitivity analyses examined how the results would likely change had these resources been shared with other clinic users, as opposed to being dedicated solely to use of the CBT4CBT program participants.

To estimate the per participant cost of the computer in the base scenario, we assumed the computer would last exactly 27.5 months (i.e., the duration of the study), and then prorated the cost of the computer ($2,000) over the total number of users for the lifetime of the computer (i.e., 100 patients per year × 27.5/12 years = 229.2 users).

To estimate the per participant cost of the office space in the base scenario, we estimated the annual cost of the space for the entire facility (including mortgage, utilities, and maintenance), and then prorated this annual cost by (1) the fraction of the facility occupied by the dedicated office (i.e., 64 ft²/3000 ft²), and (2) the number of annual users of the dedicated office (i.e., 100 patients). This approach is conservative inasmuch as the clinic's mortgage was higher than the fair market value of neighboring commercial properties during the time of the study.

Finally, staff spent a nominal amount of time showing participants assigned to the CBT4CBT condition how to get started using the computer program. This time averaged 10 minutes per participant and was valued at the equivalent counselor salary plus fringe benefits and overhead.

2.2.2. Patient perspective (cost of time in treatment)

2.2.2.1. Unit counseling costs

The unit counseling costs measure the average cost of a counseling session and were estimated for individual and group therapies. These unit costs included the time spent by the patient in treatment.

2.2.2.2. Unit testing cost

The unit testing cost measures the average cost per urine and breath test and included the time spent by the patient providing the specimens.

2.2.2.3. CBT4CBT program costs

CBT4CBT program costs comprise two components: time spent using the CBT4CBT program and time spent completing the CBT4CBT-generated homework assignments. Inasmuch as patients used the program on the same days as their regularly scheduled treatment sessions, no additional transportation-related costs were necessitated.

2.2.3. Resources used

To calculate the total variable cost of participants in each treatment arm, we multiplied the above unit costs by the number of units of each resource used. This was done separately for the clinic and patient perspectives. Resource utilizations for each participant were obtained from the effectiveness study (Carroll et al., 2008). Specifically, for each patient, the number of each type of counseling session attended and the number of urinalysis and breathalyzer tests provided were monitored closely by research assistants using the PACC-SAT (Program and Client Costs—Substance Abuse Treatment) instrument. In addition, each participant accessed the CBT4CBT program via an ID/password system that was linked to an imbedded database that tracked, for each patient, time logged into the program and completion of homework assignments. Variable costs per participant were then estimated by multiplying unit costs from each perspective by corresponding resource utilizations. Finally, the incremental cost of adding CBT4CBT to TAU was estimated by subtracting the average per participant cost of the TAU alone condition from the average per participant cost of the CBT4CBT plus TAU condition.

2.2.4. Incremental cost-effectiveness analyses

We conducted incremental cost-effectiveness analyses to evaluate the relative cost-effectiveness of CBT4CBT compared to TAU alone. The primary patient outcome used in the ICEAs was the total number of specimens provided during treatment that tested negative for cocaine, opioids, marijuana and alcohol. For each perspective (clinic and patient), we calculated incremental cost-effectiveness ratios (ICERs). The ICER is defined as the incremental cost divided by the incremental effect. We used incremental costs estimated as described above and incremental effects obtained from the effectiveness study. The ICERs measure the incremental cost of adding CBT4CBT to TAU to obtain an additional drug-free specimen.

We also developed cost-effectiveness acceptability curves (CEACs) to show the probability that adding CBT4CBT to TAU is cost-effective, given the observed data, under different assumptions about the value of an additional drug-free specimen. Costs and effects for each intervention were bootstrapped with 2,000 replicates to produce CEACs for each perspective (see Fenwick et al. (2001) for a detailed explanation of cost-effectiveness acceptability curves).

Finally, two additional analyses were performed to examine the robustness of the results. First, we conducted sensitivity analyses on several key cost parameters to assess how the ICERs and CEACs would likely change had the trial been implemented under alternative conditions. Second, we analyzed the longest duration of abstinence from cocaine, opioids, marijuana and alcohol that occurred during treatment.

3. Results

In the randomized effectiveness trial (Carroll et al., 2008), participants assigned to TAU plus CBT4CBT submitted, on average, 1.83 more drug-free specimens than their counterparts assigned to TAU alone (6.38 vs. 4.55, p = .032). Days retained in treatment did not differ significantly between the CBT4CBT and TAU conditions (40.5 vs. 41.4, p = .813) and was not a likely factor in the relative number of samples available for analyses. Moreover, no significant differences between treatment conditions were found on any of the participant demographic, substance use or psychosocial functioning variables measured at baseline. Therefore, observed differences in patient outcomes between the two conditions were likely associated with the interventions provided.

Unit costs were estimated following the methods described above using cost data obtained from the clinic and patients where the effectiveness trial took place. The average per participant cost of an individual counseling session was $28.84 from the clinic perspective (including administrative time for notes) and $6.64 from the patient perspective, while the average per participant cost of a group counseling session was $9.07 from the clinic perspective and $13.29 from the patient perspective. Each urinalysis and breathalyzer test cost the clinic an average of $12.97 (including materials and labor) and patients an average of $1.03 (including time only). From the clinic perspective, the average per participant cost of the CBT4CBT program was $8.73 for the computer hardware, $38.83 for the office space, and $4.81 for staff support. From the patient perspective, the average cost of completing a 45-minute CBT4CBT module and a 10-minute homework assignment was $6.50 and $1.45, respectively.

Average resource utilizations per participant were obtained from the effectiveness study and are summarized in Table 1 for both treatment conditions. Compared to TAU alone, participants in the CBT4CBT plus TAU condition attended, on average, slightly fewer individual counseling sessions and slightly more group counseling sessions (all differences NS).

Table 1.

Average and Incremental Resources Consumed Per Participant^a

	TAU plus CBT4CBT (N = 34)	TAU alone (N = 38)	(TAU plus CBT4CBT) – (TAU alone)
Counseling Sessions
Individual (#)	3.9 (2.9)	4.6 (3.1)	−0.7
Group (#)	7.2 (6.7)	6.5 (3.7)	0.7
Tests (#)	8.7 (4.9)	8.9 (4.0)	−0.2
Computer-related
Staff support (minutes)	10 (0)	0 (0)	10*
Modules (minutes)	151 (84)	0 (0)	151*
Homework (minutes)	24 (21)	0 (0)	24*

CBT4CBT = computer-based training for cognitive-behavioral therapy

TAU = treatment as usual

^aValues represent means and standard deviations (parentheses).

^*p-value < .05

Table 2 presents the average variable cost per participant for both treatment conditions, as well as the incremental cost of adding CBT4CBT to TAU. From both the clinic and patient perspectives, compared to TAU alone, participants in the CBT4CBT condition incurred significantly higher computer-related costs. Although adding CBT4CBT to TAU did not significantly increase the total variable cost of treatment from either perspective, incremental cost-effectiveness analysis is still warranted (as opposed to effect-maximization analysis) because cost effectiveness depends on the joint density of cost and effect differences, as opposed to individual differences in either cost or effect. In other words, separate and sequential hypothesis tests on differences in costs and effects are inappropriate for determining whether cost effectiveness should be estimated (Glick et al., 2007; Drummond et al., 2005; Briggs and O'Brien, 2001). Rather, the analysis should focus on estimating incremental cost-effectiveness ratios (ICERs), quantifying the uncertainty around the ICERs, and presenting this uncertainty in the form of cost-effectiveness acceptability curves (Briggs and O'Brien, 2001).

Table 2.

Average and Incremental Variable Cost Per Participant – Base Case^a

	TAU plus CBT4CBT (N = 34)	TAU alone (N = 38)	(TAU plus CBT4CBT) – (TAU alone)
CLINIC PERSPECTIVE
Counseling
Individual	114 (85)	131 (91)	−17
Group	66 (61)	59 (34)	7
Subtotal	180 (121)	190 (105)	−10
Testing
Labor	50 (28)	51 (23)	−1
Materials	62 (35)	64 (29)	−2
Subtotal	112 (63)	115 (52)	−3
Computer-related
Software	0 (0)	0 (0)	0
Hardware	9 (0)	0 (0)	9*
Office space	38 (0)	0 (0)	38*
Staff support	5 (0)	0 (0)	5*
Subtotal	52 (0)	0 (0)	52*
Clinic Total	344 (177)	305 (137)	39
PATIENT PERSPECTIVE
Counseling
Individual	26 (25)	31 (23)	−5
Group	91 (93)	86 (51)	5
Subtotal	117 (103)	117 (65)	0
Testing	9 (7)	9 (6)	0
Computer-related
Modules	23 (23)	0 (0)	23*
Homework	4 (3)	0 (0)	4*
Subtotal	27 (25)	0 (0)	27*
Patient Total	153 (123)	126 (69)	27

CBT4CBT = computer-based training for cognitive-behavioral therapy; TAU = treatment as usual

^aValues represent means and standard deviations (parentheses).

^*p-value < .05

Column 2 of Table 3 presents incremental cost-effectiveness ratios (ICERs) for the comparison of CBT4CBT to TAU alone. These ICERs were calculated using incremental costs from Table 2 and incremental effects as described above. Specifically, compared to TAU alone, the incremental cost of using CBT4CBT to obtain an additional drug-free specimen was $21 from the clinic perspective ($21 = ($344 - $305)/1.83) and $15 from the patient perspective ($15 = ($153 - $126)/1.83)).

Table 3.

Incremental Cost-Effectiveness Ratios for an Additional Drug-Free Specimen

Perspective	Base Casea ($)	Favorable Scenariob ($)	Unfavorable Scenarioc ($)
Clinic	21	−6	55
Patient	15	11	16

^aBase case corresponds to actual implementation of the CBT4CBT program and assumes (1) software cost = $0, (2) useful life of computer = 2.29 years, (3) computer and office are dedicated solely to TAU plus CBT4CBT patients, (4) staff spend an average of 10 minutes per participant explaining how to use the CBT4CBT program, (5) clinic overhead rate = 27.8%, and (6) local minimum wage = $7.36 per hour.

^bFavorable scenario assumes (1) software cost = $0, (2) useful life of computer = 4 years, (3) computer and office are shared with other clinic personnel, (4) no staff support time is required to show participants how to use the CBT4CBT program, (5) clinic overhead rate = 20.0%, and (6) local minimum wage = $5.15 per hour.

^cUnfavorable scenario assumes (1) software cost = $50 per participant, (2) useful life of computer = 2 years, (3) computer and office are dedicated solely to TAU plus CBT4CBT patients, (4) staff spend an average of 30 minutes per participant explaining how to use the CBT4CBT program, (5) clinic overhead rate = 35.0%, and (6) local minimum wage = $8.00 per hour.

To address the uncertainty inherent in the ICER point estimates, cost-effectiveness acceptability curves (CEACs) are presented in Figure 1a for the clinic and patient perspectives. For each perspective, the corresponding CEAC shows the probability that the addition of CBT4CBT is cost-effective compared to TAU alone, given the observed data (Fenwick et al., 2001). Note that the CEACs are a function of the threshold willingness-to-pay of the decision maker to obtain an additional drug-free specimen. Intuitively, as the threshold value of an additional drug-free specimen increases, the most effective intervention (i.e., CBT4CBT) becomes increasingly more likely to be cost effective, even though it adds incremental costs; and as the threshold value decreases, the least costly intervention (i.e., TAU alone) becomes increasingly more likely to be cost-effective. For example, from the clinic perspective, when the threshold value of an additional drug-free specimen is $0, then Figure 1a shows that CBT4CBT is only 14% likely to be cost effective (and by complementarity, TAU alone is 86% likely to be cost effective). As the threshold value increases from $0 to approximately $21, the likelihood that CBT4CBT is cost effective increases from 14% to 50% (and by complementarity, the likelihood that TAU alone is cost effective decreases from 86% to 50%). Note that at a threshold value of approximately $21, the two treatments are equally likely to be cost effective. Finally, as the threshold value of an additional drug-free specimen increases from $21 to approximately $75, the likelihood that CBT4CBT is cost effective exceeds 90%.

Figure 1.

Cost-effectiveness acceptability curves for number of drug-free specimens provided during treatment (a), and longest duration abstinent (LDA) during treatment (b). For a given threshold value, the probability that CBT4CBT is cost effective is equivalent to the proportion of the 2,000 bootstrapped replicates for which CBT4CBT had the highest net benefit (Fenwick et al., 2001).

3.1. Robustness Checks

3.1.1. Sensitivity analyses

To determine how the cost effectiveness of adding CBT4CBT to TAU would likely change had the effectiveness trial been implemented under alternative conditions, we conducted sensitivity analyses in which we considered two alternative scenarios—one favorable to CBT4CBT and the other unfavorable to CBT4CBT—in which we made different assumptions about (1) software costs, (2) the useful life of the computer used for the CBT4CBT program, (3) the dedication of the computer and office space for CBT4CBT use, (4) the staff support time required by the CBT4CBT program, (5) the clinic overhead rate, and (6) the minimum wage used in the calculation of the value of patients' time.

Table 3 summarizes the specific assumptions made for the base scenario as well as the two scenarios in the sensitivity analyses. Although most of the assumptions in the sensitivity analyses are straightforward, several may require additional explanation. In the unfavorable scenario, we assumed that the CBT4CBT program would require an annual site license costing $5000 which, when prorated over 100 patients per year, would result in an additional cost to the clinic of $50 per patient. In the favorable scenario, we assumed that the clinic would share the computer and private office used by CBT4CBT participants with other clinic personnel (patients or staff), thereby enabling the clinic to spread the fixed costs of these resources over a much larger pool of users. Said differently, under the “shared resources” assumption, the CBT4CBT participants were “charged” only for the amount of time they actually used the computer and the private office. The minimum wage used in the calculation of the value of patients' time was varied from a low of $5.15 per hour (the US Federal minimum wage during the study period) to a high of $8.00 per hour (the highest current State minimum wage).

Columns 3–4 of Table 3 present the results of the sensitivity analyses. In the unfavorable scenario, for example, the ICER from the clinic perspective more than doubled compared to the base scenario, due primarily to the additional per participant cost of the software and the cost of the additional staff support time. Conversely, in the favorable scenario, CBT4CBT dominated TAU alone from the clinic perspective (i.e., on average, CBT4CBT both cost less and led to more drug-free specimens than TAU alone). Given that the CBT4CBT program was implemented as a treatment adjunct, it may seem surprising that adding CBT4CBT to TAU could ever be less expensive than TAU. This is likely due to (1) the per participant cost of the computer in the favorable scenario decreased from $52 (Table 2) to $3 (results not shown), due primarily to the assumption that the clinic would share the computer and office space with other clinic users (and so CBT4CBT participants were charged only for the time that they actually used the computer and the private office), (2) participants in the CBT4CBT condition incurred lower overall counseling and testing costs than participants in the TAU alone condition (lower by $13 per participant, results not shown), and (3) therefore, the relatively small computer-related costs in the favorable scenario were more than offset by the lower overall counseling and testing costs in the TAU plus CBT4CBT condition. To be conservative, even if the overall counseling and testing costs were equated between the two conditions in the favorable scenario, the ICER from the clinic perspective would still be less than $2 (i.e., $3/1.83). Finally, the ICERs from the patient perspective did not vary much, due to the fact that the minimum wage was the only parameter in the sensitivity analyses that impacted the costs from the patient perspective.

3.1.2. Longest Duration of Abstinence

To determine how the relative cost-effectiveness of CBT4CBT might change for alternative patient outcomes, we calculated ICERs and CEACs for the longest duration of abstinence (LDA) from cocaine, opioids, marijuana and alcohol during treatment. As reported in the effectiveness study (Carroll et al., 2008), participants assigned to TAU plus CBT4CBT achieved, on average, longer LDAs than their counterparts assigned to TAU alone (2.88 weeks vs 2.18 weeks, p = .074). Compared to TAU alone, the incremental cost of using TAU plus CBT4CBT to lengthen the LDA by 1 week was $56 from the clinic perspective ($56 = ($344 − $305)/(2.88 − 2.18)) and $39 from the patient perspective ($39 = ($153 − $126)/(2.88 − 2.18)). Figure 1b shows the CEACs for the patient outcome LDA. The ICERs and CEACs for the patient outcome LDA reinforce those presented above for the patient outcome total number of drug-free specimens provided.

4. Discussion

In a population of substance-dependent individuals seeking treatment in a community-based outpatient substance abuse treatment clinic, CBT4CBT was associated with significantly more drug-free specimens than TAU alone, but required additional costs from both the clinic and patient perspectives. Whether CBT4CBT is likely to be cost-effective depends on the value that decision makers place on an additional unit of effect. At this time, no consensus threshold values exist for any patient outcomes in substance abuse treatment; that is, there are no generally accepted values associated with an additional drug-free specimen or any other treatment outcome, from either the clinic or patient perspectives. In the absence of consensus threshold values, we present ranges of values, defined by the ICERs and CEACs, over which each intervention (CBT4CBT versus TAU alone) is likely to be cost-effective. Specifically, from the clinic perspective, if the threshold value for an additional drug-free specimen is greater than approximately $21, then CBT4CBT is likely to be the most cost-effective intervention. However, if the threshold value is less than approximately $21, then TAU alone is likely to be the most cost-effective intervention. Similarly, from the patient perspective, if the threshold value for an additional drug-free specimen is greater than approximately $15, then CBT4CBT is likely to be the most cost-effective intervention, while TAU alone is likely to be the most cost-effective intervention if the threshold value is less than approximately $15. Thus, compared to TAU alone, CBT4CBT appears to be a good value for the money inasmuch as it is cost effective for relatively low threshold values for an additional drug-free specimen.

The results of the two robustness checks (i.e., the sensitivity analyses and the patient outcome LDA) were consistent with our main finding that CBT4CBT appears to be a good value for the money. In addition, the sensitivity analyses show that sharing the computer and office space with other clinic personnel, as would be expected in regular clinical practice (as opposed to dedicating these resources for the sole use of CBT4CBT), substantially improves the cost-effectiveness of CBT4CBT from the clinic perspective.

To put these results in perspective, Table 4 compares the ICERs for CBT4CBT from this study to the ICERs for other empirically-validated therapies reported in our earlier work on contingency management and clinician-administered CBT (Olmstead et al., 2007a; Olmstead et al., 2007b; Sindelar et al., 2007). All of the ICERs in Table 4 were estimated using similar methods; all are from the clinic perspective and measure the incremental cost of using the experimental treatment (i.e., CBT4CBT, contingency management, the combination of motivational enhancement therapy with CBT), compared to TAU alone, to obtain an additional drug-free specimen during treatment. Although direct “apples-to-apples” comparisons are not possible (the studies differ with respect to their target populations, primary target drugs, TAU practices, and specific parameters included in their respective sensitivity analyses), the cost effectiveness of CBT4CBT nevertheless looks promising inasmuch as its ICERs are considerably lower across all scenarios than those of the other empirically-validated therapies. Given CBT's established durability of effects, these results are particularly striking.

Table 4.

Comparison of Incremental Cost-Effectiveness Ratios (ICERs) for an Additional Drug-Free Specimen – Clinic Perspective

	ICERs
Treatment	Base Case ($)	Favorable Scenario ($)	Unfavorable Scenario ($)
TAU plus CBT4CBT	21	−6	55
TAU plus Prize CM (MM)a	70	51	92
TAU plus Prize CM (DF)b	146	78	153
MET/CBTc	159	121	n/a

TAU = treatment as usual

CBT4CBT = computer-based training for cognitive-behavioral therapy

^aPrize CM (MM) = prize-based contingency management in methadone clinics (Sindelar et al., 2007)

^bPrize CM (DF) = prize-based contingency management in drug free clinics (Olmstead et al., 2007b)

^cMET/CBT = motivational enhancement therapy combined with clinician-delivered cognitive-behavioral therapy (Olmstead et al., 2007a)

The present study has several strengths. First, it is based on a randomized clinical trial that relied on objective indicators of patient outcomes and included a heterogeneous sample of outpatients who used multiple substances concurrently (Carroll et al., 2008). Thus, our cost-effectiveness results are likely to be generalizable to substance-dependent populations. Second, we considered costs from the perspective of the clinic (cost of providing treatment) and the patient (cost of time in treatment), and all cost data were collected from the clinic where the trial took place. Third, in the absence of consensus threshold values for an additional drug-free specimen, we used ICERs and CEACs to present ranges of values, for each perspective, over which each intervention is likely to be cost effective. Decision makers can use this information in combination with their own evaluation of the value of an additional drug-free specimen to make policy decisions. Fourth, we examined the robustness of the results by (i) conducting sensitivity analyses on several key cost parameters and (ii) analyzing the LDA during treatment. Finally, the ICERs estimated in the present study can be used as thresholds for future studies.

There are also several limitations. First, given the relatively small study sample, additional studies are warranted to determine the reliability of our results. These additional studies should be powered appropriately with sample size calculations based on the results of this study. Second, this study evaluated the cost effectiveness of the CBT4CBT program in terms of one model of how it might be implemented in clinical practice, in this case, as an onsite addition to TAU. Although other implementation models are possible, the model evaluated in this trial is one likely means of making a version of CBT more available in substance abuse treatment. Third, due to a lack of data on societal outcomes, we did not consider broader perspectives such as costs to employers (e.g., lost work productivity) or the larger society (e.g., crime, spread of disease, family functioning). However, inasmuch as adoption decisions are made at the clinic level, while decisions to accept/attend treatment are made by the patient (Jones et al., 2009), both of the perspectives examined in this study (i.e., clinic and patient) are important determinants of treatment uptake. Finally, our cost estimates include only those costs that vary by treatment condition. Therefore, they do not measure the total cost of the interventions and should not be compared to “total cost” estimates in the literature nor used for reimbursement rate-setting purposes.

In conclusion, this study adds to the literature on cost effectiveness in substance abuse treatment by providing, to our knowledge, the first cost-effectiveness analysis of an e-therapy in the field of addiction. Cost-effectiveness analyses of e-therapies are critical because e-therapies are thought to add costs to treatment as usual, from both the clinic and patient perspectives. ICERs and CEACs suggested that compared to TAU alone (as well as to other empirically-validated therapies), CBT4CBT appears to be a cost-effective adjunct to substance abuse treatment. Given that computer-assisted delivery of CBT has recently been shown to be cost effective in the treatment of anxiety and depression disorders (McCrone et al., 2004), accumulating evidence suggests that providing computer-assisted therapies such as these does seem a cost-effective strategy for implementing CBT much more broadly. Further research is necessary to determine the cost-effectiveness of implementation models that differ from the one evaluated in the present study. For example, making the CBT4CBT program available offsite to patients with access to a computer (by putting the program on a DVD or a flash drive) or the internet (by hosting the program on a website) would likely affect patient outcomes as well as the magnitude and distribution of costs between the clinic and patients. It also would be interesting to determine which, if any, of the various models of implementing CBT4CBT are cost-effective compared to clinician-delivered CBT. In addition, it would be useful to determine the cost-effectiveness of CBT4CBT from the societal perspective. Inasmuch as there are many unanswered questions surrounding the “best” way to implement CBT4CBT and other computer-assisted approaches, the present study represents a first step in what promises to be a fertile area of future research.