A Pilot Economic Evaluation of Computerized Cognitive Behavioral Therapy for Alcohol Use Disorder as an Addition and Alternative to Traditional Therapy

Abstract

Background:

Computer-based delivery of cognitive behavioral therapy (CBT) may be a less costly approach to increase dissemination and implementation of evidence-based treatments for alcohol use disorder (AUD). However, comprehensive evaluations of costs, cost-effectiveness, and cost-benefit of computer-delivered interventions are rare.

Methods:

This study used data from a completed randomized clinical trial to evaluate the cost-effectiveness and cost-benefit of a computer-based version of CBT (CBT4CBT) for AUD. Sixty-three participants were randomized to one of the following treatments at an outpatient treatment facility and attended at least one session: (1) treatment as usual (TAU), (2) CBT4CBT plus treatment as usual (CBT4CBT+TAU), or (3) CBT4CBT plus brief monitoring.

Results:

Median protocol treatment costs per participant significantly differed between conditions, H(2) = 8.40, p = .02, such that CBT4CBT+TAU and CBT4CBT+monitoring each cost significantly more per participant than TAU. However, when non-protocol treatment costs were included, total treatment costs per participant did not significantly differ between conditions. Median incremental cost-effective ratios (ICERs) revealed that it cost $35.08 to add CBT4CBT to TAU to produce a reduction of one additional drinking day per month between baseline and the end of the 8-week treatment protocol: CBT4CBT+TAU was more costly and more effective than TAU. CBT4CBT+monitoring cost −$33.70 to produce a reduction of one additional drinking day per month because CBT4CBT+monitoring was less costly than TAU and more effective at treatment termination, yet not significantly so. Net benefit analyses suggested that costs of treatment, regardless of condition, did not offset monthly costs related to healthcare use, criminal justice involvement, and employment disruption between baseline and 6-month follow-up. Benefit-cost ratios were similar for each condition.

Conclusions:

Results of this pilot economic evaluation suggest an 8-week course of CBT4CBT may be a cost-effective addition and potential alternative to standard outpatient treatment for AUD. Additional research is needed to generate conclusions about the cost-benefit of providing CBT4CBT to treatment-seeking individuals participating in standard outpatient treatment.

Introduction

Alcohol use disorder (AUD) is the most prevalent substance use disorder (SUD) in the United States, with 12-month prevalence estimates ranging from 8.5% to 13.9% (American Psychological Association, 2013; Grant et al., 2015). According to the most recent estimates from the National Institute of Drug Abuse (NIDA), alcohol and illicit drug use cost the United States economy $442 billion annually through increased healthcare utilization, increased crime, and employment disruption; costs related to alcohol use alone accounted for $249 billion, or 56% of these expenditures (NIDA, 2020). Research suggests that specialized treatment for SUDs can offset these costs by reducing healthcare utilization and crime and increasing employment earnings (Ettner et al., 2006; Koenig et al., 2005). The National Institutes of Alcohol Abuse and Alcoholism (NIAAA) and NIDA have prioritized cost-inclusive evaluations because there is a need for randomized clinical trials of alcohol treatment with both cost and efficacy components (Institute of Medicine, 1990).

Despite the economic burden of alcohol use, only 6.7% of adults with an alcohol or substance use disorder in the United States who wanted treatment received specialty substance use treatment (SAMHSA, 2018). A review by Marsch and Dallery (2012) suggested that, from a provider’s perspective, the cost of providing treatment has become the greatest barrier to offering specialized and evidence-based substance use services (cf. Mark, Levit, Vandivort-Warren, Buck, & Coffey, 2011). Interviews with substance use treatment facility directors revealed that most facilities have neither the financial nor personnel resources to implement evidence-based practices, which require providers to have advanced education, complex therapeutic skills, and ongoing clinical supervision (McLellan, Carise, & Kleber, 2003). “A lack of insurance coverage and inability to afford the cost [of treatment]” (SAMHSA, 2014, p. 7) is the most commonly cited reason that treatment-seeking patients suffering from AUDs and other SUDs have not actually accessed specialized treatment. It appears that, from the patient’s perspective, the cost of receiving treatment has become the primary access barrier.

Research findings from the past several years suggest that computer-based delivery systems produce clinical outcomes comparable to traditional in-person treatment, but at a lower cost (Cuijpers et al., 2009; Kiluk et al., 2016; Olmstead, Ostrow, & Carroll, 2010; Portnoy, Scott-Sheldon, Johnson, & Carey, 2008). Computer-based delivery systems have been evaluated for the treatment of SUDs (Bickel, Christensen, & Marsch, 2011; Boumparis, Karyotaki, Schaub, Cuijpers, & Riper, 2017; Carroll et al., 2008; 2009; 2014; Kay-Lambkin, Baker, Kelly, & Lewin, 2011), AUDs (Kiluk et al., 2016; Riper et al., 2008), cannabis use (Budney et al., 2012), and tobacco use (Dallery et al, 2017). Although there have been relatively few direct comparisons to in-person treatment, computer-based interventions for SUDs have demonstrated efficacy by reducing substance use or increasing abstinence rates (Boumparis et al, 2017; Kiluk, Ray, Walthers, Bernstein, Tonigan, & Magill, 2019; Moore et al., 2011). Notably, computer-based cognitive-behavioral therapy (CBT) has demonstrated efficacy comparable to that of standard treatment for SUDs even though patients receiving computer-based CBT typically spent 50% to 80% less time interacting in-person with a provider (e.g., Bickel, Marsch, Buchhalter, & Badger, 2008; Kiluk et al., 2016; Kiluk et al., 2018). These reductions in provider time might reduce expenditures related to employing and training providers in evidence-based treatment practices (Bickel et al., 2008; Budney et al., 2012; Cuijpers et al., 2009; Kaltenthaler et al., 2006; Kay-Lambkin et al., 2011; Marks & Cavanagh, 2009; Marsch & Dallery, 2012; McLellan et al., 2003; Wright et al., 2005).

Computer-based CBT also may be less costly than in-person CBT by reducing the fiduciary and temporal costs of implementing CBT (Carroll & Onken, 2005). Therefore, computer-based interventions may be a less costly way to increase the dissemination of evidence-based treatments for AUD (Carroll & Rounsaville, 2010; McCrone et al., 2004; SAMHSA, 2009; Tito, 2007). However, evaluations of the cost-effectiveness and cost-benefit of computer-delivered CBT, compared to in-person therapy, are rare.

One computer-based CBT treatment for SUD that has demonstrated efficacy through randomized clinical trials and been subject to cost-effectiveness analysis is computer-based training for CBT (CBT4CBT; Carroll et al., 2008). According to Olmstead and colleagues (2010), CBT4CBT has the potential to be cost-effective as an adjunct to standard outpatient treatment for treating a general population of substance users, and to cost less per increment of symptom improvement than standard in-person treatment for providers and patients.

The present study is a cost-inclusive examination of CBT4CBT based on data from a recent clinical trial for AUD (Kiluk et al., 2016). This study addresses gaps in the alcohol use literature by (a) conducting a cost-effectiveness and cost-benefit evaluation of AUD treatment with a computer-based delivery component and (b) using data from a trial evaluating CBT4CBT as a virtual stand-alone treatment.

Materials and Methods

Participants

As detailed in Kiluk et al. (2016), participants were recruited at an outpatient substance use treatment clinic in New Haven, CT, from March 2012 until December 2014. To be included, participants were at least 18 years old, met diagnostic (APA, 1994) criteria for current (past 28 days) alcohol abuse or dependence, and were psychiatrically stable for outpatient treatment (i.e., no acute suicidality/homicidality or unstable psychotic disorder). Of the 87 individuals assessed for eligibility, 68 were deemed eligible and randomly assigned to a treatment condition.

Treatments

Participants were randomly assigned to one of three 8-week treatments: (1) Treatment as Usual (TAU), (2) CBT4CBT plus TAU, or (3) CBT4CBT plus brief monitoring. All participants were afforded access to emergency, pharmacologic, and psychiatric services at the clinic. Participants assigned to TAU received weekly group or individual psychotherapy, based on clinician discretion. Participants assigned to either of the CBT4CBT conditions were asked to complete modules of the program on a laptop computer at the clinic. Details about the individual treatment conditions can be found in the main study report (Kiluk et al., 2016).

Measures

The Structured Clinical Interview for DSM-IV (SCID-I; First et al., 1995) was administered at the time of study screening to determine eligibility for randomization. The Substance Use Calendar, a calendar-based Timeline Follow-Back method (Sobell and Sobell’s 1992), monitored frequency and quantity of drinking before randomization (i.e., at baseline) and weekly between treatment initiation and 6-month follow-up. The Addiction Severity Index (ASI; McLellan et al., 1992) evaluated participant medical status, education and employment history, sources of income, and psychological impairment. The ASI was administered before randomization, every 4 weeks during treatment, and at 1-, 3-, and 6-month follow-up. The Program and Client Costs-Substance Abuse Treatment form (PACC-SAT; Jofre-Bonet et al., 2004; Olmstead, Sindelar, Easton, & Carroll, 2007) was administered on the same schedule as the ASI and evaluated mental, substance use, and medical healthcare utilization, participant travel time to treatment, and justice system involvement. Breathalyzer and urine toxicology screenings were collected at each study visit.

Costs

Protocol healthcare services.

Costs of individual therapy, group therapy, CBT4CBT, and brief monitoring (i.e., costs that were related to the study protocol) included resources used by providers, the CBT4CBT web developer, and participants. All costs were calculated in U.S. dollars for January 2015, the last year of follow-up data collection.

Resources: Provider + developer perspective.

Provider costs included the personnel, facility, and equipment needed to provide treatment in the different conditions. Using a microcosting approach, provider costs were determined by multiplying the unit cost for each service by the amount of services provided. Personnel hourly wages were calculated by dividing annual wages plus fringe benefits by 2,087, the average number of work hours per year (§5504(b) of the U.S. Code). Personnel costs are presented in Table A1. Resources used to develop, advertise, and maintain CBT4CBT were valued to quantify treatment costs from the perspective of CBT4CBT developers and were included in provider costs. Detailed hourly network and hardware costs appear in Table A2. Projected 5-year developer costs appear in Table A3 and are described in Appendix B.

Resources: Participant perspective.

Round-trip transportation costs to protocol treatment services were estimated by doubling self-reported one-way bus fare, mileage, and travel time costs on the PACC-SAT to protocol treatment sessions (Jofre-Bonet et al., 2004). Costs of childcare related to healthcare use were excluded because they were only reported by one participant. Participant time was valued as wages lost during the time needed to travel to/from and participate in protocol treatment. Using Basu’s (2017) recommendation, to avoid underestimation, the hourly value of time for participants receiving public assistance, benefits (i.e., retirement and disability), or unemployment income was estimated as $12.23, which is the sum of the 2015 CT minimum hourly wage ($9.15) plus 33.7% for fringe benefits (Bureau of Labor Statistics [BLS], 2015). To increase pay equality between participants, employed participants’ time was estimated by taking the mean of $12.23 and their self-reported hourly employment wages including fringe benefits (Olmstead et al., 2010).

Non-protocol healthcare services

Healthcare services have been included as either treatment costs (Koenig et al., 2005) or monetary outcomes, i.e., benefits (French, Salomé, & Carney, 2002) in different cost-benefit evaluations of substance abuse treatment. We treated costs of non-protocol outpatient mental health/substance abuse treatment, outpatient medical treatment, inpatient mental health/substance abuse treatment, and inpatient medical treatment accessed during the 8-week treatment protocol as treatment costs to avoid under-estimating the costs of healthcare and medical services that may be contributing to substance use outcomes. Unit costs for healthcare services, along with criminal justice involvement and lost productivity, are presented in Table A4. Non-protocol healthcare costs are presented both separately from and combined with protocol treatment costs in Table 1.

Table 1.

Eight-Week Protocol and Non-Protocol Treatment Costs

	TAU			CBT4CBT+TAU			CBT4CBT+monitoring
Perspective	Mean (winsorized)$	Median $	SD	Mean (winsorized) $	Median $	SD	Mean (winsorized)$	Median $	SD
Protocol treatment costs	$285.01	$244.16	$144.62	$432.41	$395.24	$193.94	$295.70	$312.30	$124.72
Non-protocol treatment costsa
Outpatient substance use and mental health	$76.88	$0.00	$213.32	$6.97	$0.00	$30.36	$29.55	$0.00	$110.15
Outpatient medical	$69.67	$0.00	$139.39	$62.02	$0.00	$133.04	$30.99	$0.00	$60.78
Inpatient substance use and mental health	$250.44	$0.00	$1,091.63	$0.00	$0.00	$0.00	$22.19	$0.00	$108.70
Inpatient medical*	$531.58	$0.00	$1,801.04	$0.00	$0.00	$0.00	$0.00	$0.00	$0.00
Total protocol + non-protocol treatment costs	$1,213.58 ($739.73)	$371.90	$2,148.37	$497.95 ($498.06)	$417.58	$275.75	$378.44 ($379.27)	$335.21	$187.91

Note. N = 63 for protocol and total treatment costs. Non-protocol treatment costs and total (protocol plus non-protocol) treatment costs did not significantly differ between the 57 participants who completed the 6-month follow-up and the 6 who did not (ps > .73).

^aN = 62 for all non-protocol cost categories because 1 participant in CBT4CBT+TAU did not complete any post-baseline assessments. The missing non-protocol treatment values for this participant were treated as $0 when summing protocol and non-protocol treatment costs.

^*p ≤ .10

Efficacy¹

In the present analyses, as distinct from the original study, the outcomes of interest included percent days abstinent (PDA), drinking days per month (DDMs), and monetary outcomes (i.e., benefits). Efficacy analyses evaluated change in weekly and monthly PDA from baseline through treatment termination and 6-month follow-up, respectively, as well as change in DDMs between baseline and treatment termination and between baseline and 6-month follow-up. Some analyses were replicated here because the original study used the intent to treat sample (N = 68), whereas the present study started with a sample of participants who completed at least one session (N = 63).

Cost-Effectiveness

Incremental cost-effectiveness ratios (ICERs) divide differences in median or average costs by differences in median or average effectiveness between two treatment conditions (Neumann et al., 2017). Unlike average CERS, ICERs illustrate the average additional cost needed to produce an additional outcome unit (Hoch & Dewa, 2008). The pilot study evaluated whether the rate of change for PDA reductions significantly differed between conditions. Because using a percentage as a denominator in a fraction would be difficult to meaningfully interpret, the outcome of interest in the present cost-effectiveness evaluation was reduction in DDM between baseline and treatment termination and between baseline and 6-month follow-up.

Benefits and Net Benefits

Benefits are monetary outcomes, or the monetary value of activities that clinicians and policymakers hope will decrease after improvements in clinical outcomes attributed to treatment. We evaluated changes in criminal justice (arrests, court appearances, nights in jail) and lost productivity (i.e., missed work) costs from baseline through 6-month follow-up. We also evaluated changes in healthcare service costs from treatment termination through 6-month follow-up. A positive difference (benefits > $0) suggests a cost-savings, or that monthly costs of healthcare services, criminal justice involvement, and lost productivity were higher at baseline than at follow-up.

The baseline period covered one year (13 28-day months) before the baseline assessment and the 6-month follow-up covered eight 28-day months from baseline (two 28-day months of active treatment + six 28-day months of follow-up). Since the baseline and follow-up periods covered different time periods, we computed average monthly benefits at baseline and 6-month follow-up to make costs of these activities measured at each time point comparable.

We calculated both benefit-cost ratios, which divide benefits by treatment costs, and net benefits, which subtract treatment costs from benefits. Unlike net benefits, benefit-cost ratios determine how much benefit is generated for each dollar invested in treatment (Koenig et al., 2005). Both of these metrics have been reported in other cost-benefit evaluations of substance use treatment (French, Salomé, & Carney, 2002; Koenig et al., 2005) and can provide essential information to policy makers about how worthwhile it is to invest in substance use treatment (Institute of Medicine, 1990). A positive net benefit (i.e, net benefits > $0) suggests that benefits were greater than treatment costs; a net benefit of $0 suggests that benefits were equivalent to treatment costs; and a negative net benefit (i.e., net benefits < $0) suggest that a treatment cost more than the benefits it generated, i.e., more than the costs a treatment saved at follow-up in relation to baseline.

Data Analysis Plan

All analyses were conducted with IBM SPSS version 25. Since a vast majority of our missing data was attributed to attrition at different follow-up periods, we used available case analysis at each time point, which is sufficient in instances of low amounts (e.g., item-level) of missing data, particularly when conducting simple between-group comparisons (Parent, 2013). When data were normally distributed, multivariate generalized linear modeling (GLM) with post hoc Bonferroni-adjusted pairwise comparisons were used to determine if average costs, benefits, and net benefits differed between treatment conditions. When data for a variable were not normally distributed, Kruskal-Wallis H tests, which compare group medians, were conducted instead of GLM, and post-hoc pairwise comparisons used Dunn’s (1964) procedure with a Bonferroni adjustment. Because cost data tends to be positively skewed, 95% confidence intervals were obtained through bias-corrected accelerated bootstrapping using 1,000 resamples (Briggs, Wonderling, & Mooney, 1997; Campbell & Torgerson, 1999).

To reduce the impact of extreme outliers, we transformed the most skewed variables using winsorizaton, which replaces extreme values with less-extreme values at a pre-determined percentile (Wiechle et al., 2013). We winsorized protocol + non-protocol 8-week treatment costs and total benefits (differences in healthcare service use, criminal justice involvement, and lost productivity between baseline and 6-month follow-up), each at the 5^th and 95^th percentiles. Variables (net benefits and benefit-cost ratios) that relied on either these variables were calculated with and without winsorized cost and benefit values. Winsorized variables also were used in relevant statistical calculations. Non-parametric tests were conducted if winsorization did not normalize the distribution.

Longitudinal mixed effects regression models were used to evaluate a) weekly alcohol use frequency from baseline through treatment termination and b) monthly alcohol use frequency from baseline through 6-month follow-up. Wilcoxon signed-rank tests determined if median net benefits were significantly different from zero for the sample and each treatment condition.

Results

Demographics

As detailed in Kiluk et al. (2016), 68 participants were randomized to a treatment condition. Only 63 of the 68 participants began treatment, i.e., attended at least one treatment session (TAU = 19; CBT4CBT+TAU = 20; CBT4CBT+monitoring = 24). Since the aims of the present analyses were to examine the cost-effectiveness and cost-benefit of treatment, 5 participants were excluded because they did not complete any study protocol treatment and, therefore, had treatment costs of $0. For the 63 participants who attended at least one treatment session, mean age was 43 (SD = 12.08), 90.5% were unmarried, 79.4% had a high school diploma or more, and 74.6% were unemployed. ANOVAs and X² tests revealed no significant differences between conditions in marital status (married or unmarried), education level (less than high school diploma or high school diploma or more), employment status (employed or unemployed), gender (male or female), or age.

Six-month follow-up data were available for 57 (90%) of the 63 participants (TAU = 16; CBT4CBT+TAU = 19; CBT4CBT+monitoring = 22). Median baseline PDA was 64% in TAU, 59% in CBT4CBT+TAU, and 57% in CBT4CBT+monitoring and did not significantly differ between treatments, H(2) = 1.49, p = .48.

Cost Analysis

Below, we present the combined costs of protocol and non-protocol healthcare services accessed during the 8-week treatment period. Time spent in group therapy was not documented consistently for 8 participants assigned to TAU (42.1%), and 8 participants assigned to CBT4CBT+TAU (40%). Missing group therapy times were replaced by the average number of minutes spent in group by participants in TAU and CBT4CBT + TAU, combined (M = 58.59 minutes, Mdn = 60.00 minutes, SD = 7.13). Costs from the provider + developer and participant perspectives are presented in Tables A5 and A6, respectively.

Protocol treatment costs

In TAU, mean individual therapy sessions attended was 1.4 (SD = 2.0) and mean group therapy sessions attended was 2.8 (SD = 2.5); three participants completed only 1 individual or group session. In CBT4CBT+TAU, mean number of individual therapy sessions attended was 2.0 (SD = 2.5), mean number of group therapy sessions was 3.3 (SD = 2.4), and 1 participant completed only 1 group session and 1 CBT4CBT module. Mean number of modules completed was 5.6 (SD = 1.9) for CBT4CBT+TAU and 5.4 (SD = 1.9) for CBT4CBT+monitoring. In CBT4CBT+monitoring, all participants completed at least 1 CBT4CBT module. The number of group and individual therapy sessions did not significantly differ between TAU and CBT4CBT+TAU participants (ps > .42). Treatment completion was defined as attending at least 5 sessions during the 8-week period and differed significantly between conditions (Kiluk et al., 2016). As reported in Kiluk et al. (2016), participants were more likely to complete treatment when assigned to a condition including CBT4CBT: TAU = 26%, CBT4CBT+TAU= 65%, CBT4CBT+monitoring = 63%, Wald = 6.86, p < .01 (p. 1994).

Median protocol treatment costs per participant significantly differed between conditions, H(2) = 8.40, p = .02, such that CBT4CBT+TAU and CBT4CBT+monitoring each cost significantly more per participant than TAU, ps = .04: Protocol treatment costs per participant were highest for CBT4CBT+TAU (Mdn = $395.24, SD = $193.94, 95% CI [$356.33, $480.86]), followed by CBT4CBT+monitoring (Mdn = $312.30, SD = $124.72, 95% CI [$260.21, $340.28]) and TAU (Mdn = $244.16, SD = $144.62, 95% CI [$198.38, $371.90]). These per participant treatment costs were highest for CBT4CBT+TAU because participants and providers spent significantly more time in CBT4CBT+TAU than in TAU and in CBT4CBT+monitoring, F(1, 60) = 19.67, p < .001, contrast ps < .001; in addition, participants assigned to either CBT4CBT condition attended more treatment sessions than participants assigned to TAU.

Non-protocol treatment costs

Slightly less than half of the sample (42%) reported using non-protocol healthcare services (outpatient substance use/mental health treatment, outpatient medical treatment, inpatient substance use/mental health treatment, inpatient medical treatment) during the 8-week treatment period. Among those assigned to TAU, 58% reported non-protocol healthcare service use, whereas 32% assigned to CBT4CBT+TAU, and 38% assigned to CBT4CBT+monitoring reported such service use. Kruskal-Wallis tests revealed the sum of non-protocol treatment costs and individual categories of non-protocol treatment costs did not significantly differ between conditions (ps > .20), with differences in median inpatient medical costs trending towards significance p = .10.

Total protocol plus non-protocol treatment costs

Table 1 presents protocol, non-protocol, and combined treatment costs. Due to assumption violations, Kruskal-Wallis H tests were used to determine whether total treatment costs per participant significantly differed between treatments. Median total costs per participant were highest in CBT4CBT+TAU ($417.58, M = $497.95, SD = $275.75, CI [$350.50, $645.05]), followed by TAU ($371.90, M = $1,213.58, SD = $2,148.37, CI [$308.83, $625.30]) and CBT4CBT+monitoring ($335.21, M = $378.44, SD = $187.91, CI [$291.44, $404.21]). However, total treatment costs per participant did not significantly differ between conditions at the end of the 8-week protocol, p = .20. Total treatment costs per participant also did not significantly differ between the 57 participants who completed the 6-month follow-up and the 6 participants who did not, p = .16.

Efficacy

For the 63 participants who attended at least 1 treatment session, there was a significant reduction in weekly PDA for the sample from baseline through treatment termination, F(1, 498) = 7.11, p < .01, as well as a significant interaction between week and treatment condition, F(2, 496) = 4.81, p < .01. Participants in CBT4CBT+TAU reported greater weekly PDA increases during the treatment period compared to those in TAU, t(496) = 2.38, p < .02. Weekly PDA did not significantly differ between CBT4CBT+monitoring and TAU, t(495), = −.42, p = .67. These results are consistent with those reported in the main outcome paper (Kiluk et al., 2016).

For the 57 participants who began treatment and completed the 6-month follow-up, there was a significant main effect of time on PDA by month, F(1, 455) = 34.62, p < .001. From baseline through 6-month follow-up, there was not a significant difference in monthly PDA between CBT4CBT+TAU and TAU, p = .74, but those assigned to TAU reported greater monthly PDA increases compared to CBT4CBT+monitoring, t(453) = 2.52, p = .01. Change in monthly PDA over time is presented in Figure 1 and both PDA and DDM outcomes for all conditions and time points are presented in Table 2.

Figure 1.

Median Percentage of Days Alcohol Abstinent Across Baseline, Treatment Termination, and 1- and 6-month Follow-up

N = 57. Error bars represent 95% confidence intervals.

Table 2.

Monthly Alcohol Use Outcomes at Treatment Termination and 6-Month Follow-Up

	Baseline drinking days per month			Treatment termination drinking days per month			Baseline minus treatment termination drinking days per montha
	Mean	Median	SD	Mean	Median	SD	Mean	Median	SD
TAU	12.31	11.00	8.38	8.63	2.0	9.32	3.69	3.00	9.07
CBT4CBT+TAU	14.68	10.00	10.55	7.95	2.0	9.34	6.74	5.00	12.38
CBT4CBT+monitoring	12.77	12.5	7.12	8.36	2.5	8.78	4.41	4.00	9.78
	Baseline drinking days per month			6-month follow-up drinking days per month			Baseline minus 6-month follow-up drinking days per monthb
	Mean	Median	SD	Mean	Median	SD	Mean	Median	SD
TAU	12.31	11.00	8.38	5.13	0	7.82	7.19	6.50	9.43
CBT4CBT+TAU	14.68	10.00	10.55	7.74	3.00	9.60	6.95	8.00	12.03
CBT4CBT+monitoring	12.77	12.5	7.12	8.73	4.5	10.08	4.05	4.00	8.62

Note. N = 57 for all values so that treatment termination and 6-month follow-up ICERs in Table 3 are comparable.

^aDifference in drinking days per month between baseline and treatment termination did not significantly differ between treatment conditions (p = .66)

^bDifference in drinking days per month between baseline and 6-month follow-up did not significantly differ between treatment conditions (p =.60)

Cost-Effectiveness

ICERs are reported for the 57 participants who completed at least one treatment session as well as the 6-month follow-up. Baseline alcohol DDMs did not significantly differ between those who completed the 6-month follow-up and those who did not (p = .13). Although differences in DDMs were normally distributed at treatment termination and 6-month follow-up, total treatment costs were not; thus, ICERs with median costs and median effectiveness are reported to minimize the impact of outliers on cost-effectiveness findings (Zarkin et al., 2010). Mean, winsorized mean, and median ICERs at treatment termination and 6-month follow-up are reported in Table 3. ICERs compared CBT4CBT+TAU to TAU, and CBT4CBT+monitoring to TAU, between baseline at treatment termination and between baseline and 6-month follow-up.

Table 3.

Incremental Cost-Effectiveness Ratios (ICERs) at Treatment Termination and 6-Month Follow-Up

	Treatment termination				6-month follow-up
	CBT4CBT + TAU	TAU	CBT4CBT + mon	TAU	CBT4CBT + TAU	TAU	CBT4CBT + mon	TAU
Mean (winsorized) costs	$516.45 ($516.45)	$1,345.60 ($782.91)	$381.22 ($382.13)	$1,345.60 ($782.91)	$516.45 ($516.45)	$1,345.60 ($782.91)	$381.22 ($382.13)	$1,345.60 ($782.91)
Mean efficacya	6.74	3.69	4.41	3.69	6.95	7.19	4.05	7.19
Mean (winsorized) ICER	-$271.85 (-$87.36)		-$1,339.42 (-$556.64)		$3,454.79 ($1,110.25)		$307.13 ($127.64)
Median costs	$439.07	$368.91	$335.21	$368.91	$439.07	$368.91	$335.21	$368.91
Median efficacya	5.00	3.00	4.00	3.00	8.00	6.5	4.00	6.50
Median ICER	$35.08		-$33.70		$46.77		$13.48

Note. N = 57 so that ICERs are comparable at treatment termination and 6-month follow-up. Thus, total treatment costs in this table differ from total treatment costs presented in Table 1, where N = 63. ICER = [Treatment A mean (or median) cost - Treatment B mean (or median) cost] / [Treatment A mean (or median) drinking days per month - Treatment B mean (or median) drinking days per month).

^aMean or median efficacy of a treatment is calculated by subtracting mean (or median) 6-month (or treatment termination) follow-up drinking days per month from mean (or median) baseline drinking days per month. These drinking day per month difference values were obtained from Table 2 “Baseline minus treatment termination drinking days per month” columns.

At treatment termination, it cost an additional $35.08 per participant to implement CBT4CBT+TAU relative to TAU alone to produce an additional reduction of one DDM. Compared to TAU, CBT4CBT+TAU cost more ($439.07 vs. $368.91) and was more effective (median reduction of 5 vs. 3 DDMs). Relative to TAU, CBT4CBT + monitoring cost −$33.70 per additional reduction of one DDM, because CBT4CBT+monitoring was less costly ($335.21 vs. $368.91) and slightly more effective than TAU (median reduction of 4 vs. 3 DDMs). An exploratory analysis of variance (conducted since DDMs were normally distributed) revealed that differences in DDMs per month between baseline and treatment termination did not significantly differ between treatment conditions (p = .66).

At 6-month follow-up, it cost $46.77 to add CBT4CBT to TAU to produce an additional reduction of one DDM: CBT4CBT+TAU was more costly ($439.07 vs. $368.91) and more effective (median reduction of 8 vs. 6.5 DDMs) than TAU. It cost $13.48 more to implement CBT4CBT+monitoring vs. TAU to produce an additional reduction of one DDM because CBT4CBT+monitoring was less costly ($335.21 vs. $368.91) and less effective (median reduction of 4 vs. 6.5 DDMs) compared to TAU. An exploratory analysis of variance revealed that reductions in DDMs per month between baseline and 6-month follow-up did not significantly differ between treatment conditions (p = .60). Cost-effectiveness ratios (CERs) are presented in Appendix Table A7.

Benefits and Net Benefits

A Kruskal-Wallis test showed that total monetary outcomes did not significantly differ between conditions at baseline (ps = .19). Logistic regression and Kruskal-Wallis tests revealed that all benefits (i.e., monthly costs of healthcare services, criminal justice involvement, and lost productivity at baseline minus the monthly costs of these activities at follow-up), net benefits (benefits minus treatment costs), and benefit-cost ratios (benefits divided by treatment costs) did not significantly differ between conditions, with one exception: lost productivity. However, contrasts indicated that neither CBT4CBT+TAU nor CBT4CBT+monitoring, when compared to TAU, significantly predicted reductions in lost productivity (i.e, fewer days of missed work) at follow-up. Total monthly monetary outcomes (including non-protocol treatment costs during the 8-week treatment protocol) between baseline and 6-month follow-up are presented in Figures 2a and 2b.

Figure 2a.

Mean Total Monthly Monetary Outcomes Over Time by Treatment Condition

N = 57. Error bars represent 95% confidence intervals.

Figure 2b.

Median Total Monthly Monetary Outcomes Over Time by Treatment Condition

N = 57. Error bars represent 95% confidence intervals.

For the full treatment sample, net benefits were significantly less than zero, Mdn = −$350.71, SD = $903.91, standardized test statistic = −3.67, p < .001. This suggests that treatment costs outweighed cost-saving benefits generated between baseline and 6-month follow-up regardless of treatment assignment. Net benefits were not significantly different from zero for TAU, Mdn = −$326.93, SD = $14,291.17, standardized test statistic = −1.55, p = 0.12, or CBT4CBT+monitoring, Mdn = −$311.64, SD = $676.62, standardized test statistic = −1.90, p = .06. Net benefits were significantly less than zero for CBT4CBT+TAU, Mdn = −$363.30, SD = $465.11, standardized test statistic = −3.06, p < .01, suggesting that CBT4CBT+TAU treatment costs (protocol + non-protocol) were significantly greater than cost-savings.

Median benefit-cost ratios at the 6-month follow-up suggest that benefits (i.e., cost-savings) did not outweigh treatment costs and were similar between conditions. Benefit metrics are presented in Table 4. Additional benefit metrics, from multiple treatment cost perspectives, are presented in Table A8.

Table 4.

Summary of Monetary Outcomes, Benefits, Net Benefits, and Benefit-Cost Ratios

	TAU			CBT4CBT+TAU			CBT4CBT+monitoring
	Mean (winsorized)$	Median $	SD	Mean (winsorized)$	Median $	SD	Mean (winsorized)$	Median $	SD
Baseline monthly sum of monetary outcomes	$499.15	$434.78	$447.44	$318.61	$117.92	$624.25	$390.03	$211.30	$500.61
6-month follow-up monthly sum of monetary outcomes	$722.06	$86.89	$1,681.60	$146.32	$36.26	$386.92	$276.14	$56.48	$467.50
6-month cost-saving benefitsa	−$222.91 ($77.30)	$98.15	$1,797.83	$172.29 ($106.61)	$38.91	$702.53	$113.89 ($87.91)	$0.00	$723.51
Net benefits	−$1,568.51 (−$705.60)	−$326.93	$3,782.61	−$344.16 (−$409.84)	−$363.30	$676.90	−$267.33 (−$294.23)	−$311.64	$734.33
Benefit-cost ratio	.35 (.46)	.14	2.69	.13 (.04)	.08	1.68	.08 (.03)	.00	3.05

Note. N = 57 for all variables

^aBaseline monthly sum of monetary outcomes minus 6-month follow-up monthly sum of monetary outcomes.

Discussion

In this pilot cost-inclusive evaluation of a computerized CBT program for AUD using data from a completed randomized clinical trial (Kiluk et al., 2016), ICERs demonstrated that CBT4CBT may be a cost-effective addition, and CBT4CBT plus brief monitoring may be a cost-effective alternative, to standard outpatient psychotherapy during an 8-week treatment period. There were significant increases in the percentage of days alcohol abstinent during the duration of the trial regardless of treatment assignment; and the efficacy analysis showed those assigned to the combination of computerized CBT plus standard weekly counseling (CBT4CBT+TAU) demonstrated a greater rate of increase during the 8-week treatment period compared to standard psychotherapy only. Although CBT4CBT+TAU was the most effective treatment condition at termination, its treatment costs (protocol + non-protocol) were greater than its estimated cost-saving benefits by 6-month follow-up. One treatment did not emerge as more cost-beneficial than another.

Treatment Costs

Per participant protocol treatment costs differed significantly between conditions, such that CBT4CBT plus TAU was the most costly and TAU was the least costly. It is important to note that this is not simply due to the costs of adding CBT4CBT. Participants assigned to TAU attended fewer total protocol treatment sessions than those assigned to either of the CBT4CBT conditions (see Kiluk et al., 2016 for a more detailed explanation). In fact, those assigned to CBT4CBT plus TAU attended more standard treatment sessions (e.g., individual and group psychotherapy) than those assigned to TAU only. In other words, the addition of CBT4CBT may have increased attendance at standard treatment sessions, thereby increasing total costs for those assigned to the combination treatment. The lower treatment attendance for those assigned to TAU likely suppressed the per participant protocol treatment costs for that condition. In addition, over half of the participants assigned to TAU used non-protocol healthcare services, which was greater than the percentage of participants who accessed non-protocol services in the CBT4CBT conditions. Thus, total protocol + non-protocol treatment costs per participant were greater for TAU than the other CBT4CBT conditions, but not significantly so.

In a prior cost-effectiveness evaluation of CBT4CBT as an add-on to standard clinical practice for drug dependence (Olmstead et al. (2010), treatment costs were also higher for CBT4CBT+TAU (M = $497) than TAU (M = $431). Treatment costs were likely lower overall in the current study compared to costs reported in Olmstead et al. (2010) because it appears that participants in the current study attended fewer individual and group sessions: CBT4CBT+TAU M individual sessions = 2.0 vs. 3.9, M group session = 3.3 vs. 7.2; TAU M individual sessions = 1.4 vs. 4.6, M group session = 2.8 vs. 6.5.

Efficacy

By the end of the 8-week treatment protocol, all participants significantly increased their weekly rates of alcohol abstinence. However, participants in CBT4CBT+TAU improved their rates of abstinence more rapidly than participants in TAU by treatment termination. Although participants in CBT4CBT+TAU spent significantly more time in the protocol treatment compared to participants in the other conditions, participants in TAU accessed the most non-protocol healthcare services (e.g., detoxification, inpatient services, medical services, etc.). Thus, it is noteworthy that those assigned to CBT4CBT+TAU demonstrated more favorable clinical outcomes at treatment termination compared to those assigned to TAU who were more likely to access additional treatment services during the protocol period.

Cost-Effectiveness

Our results suggest the additional investment needed to add CBT4CBT to TAU results in positive treatment outcomes in the form of more rapid PDA improvements during the treatment period and reduced drinking days per month both between baseline and treatment termination and baseline and 6-month follow-up. Compared to TAU alone, the additional $35 per participant to add CBT4CBT in order to produce an additional reduction of one drinking day per month is comparable to prior cost-effectiveness results of adding CBT4CBT to TAU to obtain an additional drug-free urine specimen (Olmstead et al. 2010). It is important to note our ICER effectiveness metric relied on differences in DDM between baseline and follow-up, which did not significantly differ between conditions at either follow-up (treatment termination or 6-month follow-up). However, ICERs evaluate group level mean/median differences between two time points and cannot accommodate differences in slopes (such as in longitudinal mixed models). Rates of change in the form of weekly PDA did significantly differ between conditions, suggesting that CBT4CBT+TAU had superior efficacy compared to TAU at treatment termination. Treatments that can produce positive outcomes more quickly than a comparator are considered more cost-efficient by reducing the resources needed to deliver effective treatment and enabling interventions to serve more people, as a result (Yates, 2020). If superior efficacy would outweigh its higher cost per participant relative to the other conditions, a decision-maker would likely decide to implement CBT4CBT+TAU for AUDs in their practice.

Benefits, Net Benefits, and Benefit-Cost Ratios

Benefit-cost ratios were between 0 and 1 for all conditions during the 8-week protocol period, which suggests that costs of providing each treatment were not offset by increased benefits. In other words, costs related to monthly healthcare service use, criminal justice involvement, and lost productivity were approximately the same at baseline and 6-month follow-up for participants in each condition. Unsurprisingly, the total treatment costs for each condition were greater than $0, thus generating benefit-cost ratios less than 1 for each condition. It is possible that the small sample size and relatively brief follow-up time period may have impacted the benefit-cost ratios in this study. For example, in a 30-month study with a sample size and treatment costs that were each ten times greater than those in the present study, benefit-cost ratios of similar benefit variables (healthcare use, criminal justice involvement, and employment earnings) ranged from 1 to 6.8 (Koenig et al., 2005). The benefit-cost ratios in the present study are promising given the small sample size, short protocol treatment duration (8 weeks), and relatively short post-treatment follow-up period (6 months). In addition, given that 75% of the sample were unemployed at baseline and, as reported in the parent trial, 25% were court-referred to treatment, a longer study duration may have been needed to realize greater benefits.

Reconciling Cost-Effectiveness and Cost-Benefit Findings

Cost-effectiveness and cost-benefit data both provide information about the relationship between treatment costs and treatment effectiveness and can sometimes yield inconsistent findings. Our cost-effectiveness data suggest that CBT4CBT is a cost-effective addition to TAU, and CBT4CBT plus monitoring is a cost-effective alternative to TAU at the end of an 8-week treatment period. However, one of our cost-benefit analyses suggested the total treatment costs of CBT4CBT+TAU were significantly greater than the cost-savings it generated. Unlike net benefit analyses in the present study, benefit-cost ratios were similar for each treatment. Net benefits illustrate treatment benefits in proportion to treatment costs. Benefit-cost ratios illustrate treatment benefits for every $1 invested in treatment and assume that there is a linear relationship between treatment costs and treatment effectiveness, which is rarely the case.

It is important to note that economic outcomes, such as changes in healthcare service use, criminal justice involvement, and lost productivity, are not always correlated with clinical outcomes, such as measures of abstinence. For example, if increased healthcare service use can improve one’s physical health, increased healthcare service use could be viewed as something that providers want to increase, as opposed to decrease. Dismuke and colleagues (2004) demonstrated, for example, that clinical outcomes for a substance-using sample were not correlated with cost-saving benefits at a 6-month follow-up, but were at 24- and 36-month follow-ups. Thus, our cost-benefit findings should not discredit our cost-effectiveness findings, particularly if a decision-maker is most interested in clinical outcomes, such as increasing alcohol abstinence.

Limitations and Future Directions

This study had several limitations. Importantly, the trial was powered as a pilot study to evaluate safety, feasibility, and preliminary efficacy of CBT4CBT for AUD, but was not powered for cost-effectiveness or cost-benefit analyses. The pilot nature of the study, as reflected in the small sample size, as well as unequal retention across conditions, limits any definitive conclusions regarding cost-effectiveness or cost-benefit. In addition, some resource use data were incomplete. For example, information on CBT4CBT homework completion and time spent in group therapy had to be estimated due to incomplete data. Also, the study design did not include a formal wait list control group, which prevented firmer conclusions about the cost-effectiveness and cost-benefit of the three treatments for AUD as a whole, since some people might reduce their drinking without any treatment exposure. On the other hand, those who have not been exposed to treatment may not reduce alcohol consumption at all, which could make cost-effectiveness and cost-benefit data favor any treatment exposure compared to no treatment exposure. Lastly, because of the pilot nature of the present economic evaluation, we did not generate a cost-effectiveness acceptability curve (CEAC), which can provide information about how a treatment’s cost-effectiveness is influenced by different willingness-to-pay thresholds (Fenwick, Claxton, & Sculpher, 2001; Sanders et al., 2016). We recommend that CEACs be conducted for larger cost-effectiveness studies that are appropriately powered for economic evaluations.

Future research should aim to replicate the present cost-effectiveness and cost-benefit findings with a larger sample, a wait-list control group, and a longer follow-up period (≥ 12 months).