Cost-Effectiveness analysis of Recovery Management Checkups (RMC) for adults with chronic substance use disorders: evidence from a four-year randomized trial

Abstract

Aims

This study performs the first cost-effectiveness analysis (CEA) of Recovery Management Checkups (RMC) for adults with chronic substance use disorders.

Design

Cost-effectiveness analysis of a randomized clinical trial of RMC. Participants were randomly assigned to a control condition of outcome monitoring (OM-only) or the experimental condition OM-plus-RMC, with quarterly follow-up for four years.

Setting

Participants were recruited from the largest central intake unit for substance abuse treatment in Chicago, Illinois, USA.

Participants

446 participants who were 38 years old on average, 54 percent male, and predominantly African American (85%).

Measurements

Data on the quarterly cost per participant come from a previous study of OM and RMC intervention costs. Effectiveness is measured as the number of days of abstinence and number of substance-use-related problems.

Findings

Over the four-year trial, OM-plus-RMC cost on average $2,184 more than OM-only (p<0.01). Participants in OM-plus-RMC averaged 1,026 days abstinent and had 89 substance-use-related problems. OM-only averaged 932 days abstinent and reported 126 substance-use-related problems. Mean differences for both effectiveness measures were statistically significant (p<0.01). The incremental cost-effectiveness ratio for OM-plus-RMC was $23.38 per day abstinent and $59.51 per reduced substance-related problem. When additional costs to society were factored into the analysis, OM-plus-RMC was less costly and more effective than OM-only.

Conclusions

Recovery Management Checkups are a cost-effective and potentially cost-saving strategy for promoting abstinence and reducing substance-use-related problems among chronic substance users.

INTRODUCTION

Substance use disorders (SUDs) significantly contribute to morbidity and mortality among users and impose costly externalities on public health, public assistance, and criminal justice systems. Individuals with SUDs have a higher lifetime prevalence of comorbidities such as HIV infection and mental illness, and usually consume more healthcare resources as compared to the general population [1–6]. SUDs are also differentially distributed across race, gender, and ethnicity leading to notable disparities in treatment and recovery trajectories among high-risk population subgroups [7–13].

Treatment and recovery support for SUDs can lead to significant social, clinical, and public health benefits. Yet the current organization and delivery of substance abuse treatment services is not designed to fully realize these potential benefits. Most treatment clients complete an episode of residential or outpatient care without any link to continuing care services. Several recent studies demonstrate that SUD treatment followed by continued recovery monitoring may be an effective and economically viable approach for addressing SUDs [14–18]. The Recovery Management Checkups (RMC) model represents a promising strategy to managing addiction as a chronic disease. The RMC model is based on the premise that continued monitoring after treatment can identify barriers to abstinence, find ways to address these barriers, identify early relapse, and provide direct linkage to residential or outpatient treatment before relapse progresses [19]. RMCs have been shown to effectively promote linkage to treatment, engagement in treatment, and are associated with reduced substance use problems and relapse over time [15, 20].

Despite evidence supporting the effectiveness of RMCs, the economic impact of such interventions is largely unknown. Lack of economic evidence is a persistent concern when evaluating a long-term strategy for promoting recovery because policy makers and other stakeholders need to know how the costs of interventions like RMC balance with the health, criminal justice, workplace, and overall societal benefits generated by these interventions. Unlike standard treatment modalities (e.g., therapeutic communities, outpatient and day treatment) and processes (e.g., individual or group counseling, medication-assisted treatment) for which there is already a good base of economic data regarding program and client costs, cost effectiveness, and net economic benefits, few economic analyses have been conducted on continuing care and recovery management models for substance use disorders [21–24].

The present study addresses this important gap in the literature and builds on earlier cost [25] and effectiveness [20] studies of RMCs to conduct the first formal cost-effectiveness analysis (CEA) of RMCs for adults with chronic substance use disorders. We estimate the incremental cost effectiveness of adding the RMC component to standard quarterly assessments for two clinical outcomes: days of abstinence from drugs and alcohol and the number of substance-use-related problems. Intervention costs and outcomes are measured over a four-year period and provide unique insight into the long-term cost effectiveness of RMC. Results are presented for incremental cost, incremental effectiveness, and the incremental cost-effectiveness ratio (ICER) for both abstinence and substance-use-related problems. Sensitivity analyses examine the robustness of the cost effectiveness results.

Data

Data for the current study come from the second installment of the Early Re-Intervention experiments (ERI-2), which tested the feasibility and effectiveness of quarterly RMCs in improving recovery trajectories for adults with chronic SUDs [20]. Participants were recruited from sequential intakes at the largest addiction treatment agency in Illinois between February and April of 2004. To be eligible for the study, individuals had to be older than age 18, report any substance use in the past 90 days, and have any past-year symptoms of substance use disorders. Participants were randomly assigned to the control condition of quarterly assessments only [also called outcome monitoring (OM)] or the experimental condition involving quarterly assessments combined with RMCs (OM-plus-RMC). The full sample comprised 446 individuals—223 in each condition. Participants in both conditions were interviewed each quarter over four years with the Global Appraisal of Individual Needs (GAIN).[26] The GAIN is a comprehensive bio-psycho-social assessment designed to integrate research and clinical measures into one structured interview with eight main sections: background, substance use, physical health, HIV risk behaviors, mental health, environment, legal, and vocational.

The RMC intervention is characterized as follows. OM-plus-RMC participants who were identified as “in need” during a quarterly assessment were connected to linkage managers. Linkage managers performed a variety of activities including: (1) providing feedback to participants regarding their substance use and related problems; (2) discussing implications of managing substance abuse as a chronic condition; (3) discussing treatment barriers and solutions; (4) assessing and discussing motivation level for treatment; (5) scheduling treatment appointments; and (6) accompanying participants through the intake process. OM-only and OM-plus-RMC both received quarterly assessments of substance use, criminal activity, and participation in substance abuse treatment. Therefore, OM may be considered a “current practice” condition whereby participants can seek treatment voluntarily or are mandated to treatment by the criminal justice system or some other entity. Further information on intervention protocols, overall structure and delivery of the trial, participant selection, inclusion/exclusion criteria, and main findings are available in Scott and Dennis (2009), Dennis and Scott (2012), and Scott and Dennis (2003) [15,20,27].

Sample Characteristics

ERI-2 participants were 38 years old on average, mostly African American (85%), and 54% male. Most had never been married (63%), had experienced homelessness at least once (62%), were unemployed at the time of the trial (68%), and had children under 21 years of age (73%). In addition to having substance abuse and/or dependence, participants reported high rates of co-occurring psychiatric disorders (56%), infectious diseases (32%), and other health problems (25%) during the 90 days prior to intake. At intake, more than half reported engaging in illegal activity during the previous 12 months and in HIV risk behaviors during the past 90 days. A full description of all baseline sample characteristics is provided in Scott and Dennis (2003) [15, 27], but from the characteristics summarized here it is clear that these individuals are socially and economically disadvantaged, and at high risk for substance use relapse.

METHODS

Cost-Effectiveness Analysis (CEA)

CEA is one approach for conducting a full economic evaluation in which both the costs and outcomes of a program or intervention are compared [28, 29]. CEA does not require the effectiveness measure to be translated into dollars and thus allows for comparisons across a broad range of health care interventions sharing a common outcome of interest [29]. CEA is particularly useful for determining the cost at which competing programs achieve desired changes in a primary outcome, such as reduced substance use, which is difficult to translate into a monetary equivalent.

This study adopts the provider perspective for the core analysis, but explores the broader societal implications in a sensitivity analysis. Cost data come from a previously published cost analysis of the ERI-2 trial [25]. The per quarter (per participant) cost was $177 to deliver OM and $321 to deliver OM-plus-RMC. The average incremental cost to deliver the RMC portion of the intervention was therefore $144 per quarter. Response rates were very high over the 4-year follow-up: 79% of OM-plus-RMC participants and 82% of OM-only participants completed all 16 quarterly interviews, (including intake), with an average of 15.1 interviews completed across all participants. In both study conditions, only 2% of participants had 0–2 quarterly assessments and were dropped from the analysis [20]. The number of quarters a participant was successfully interviewed was multiplied by $177 to estimate the cost of conducting OM-only and by $321 to estimate the cost of OM-plus-RMC. This allowed us to calculate four-year episode costs for each participant in the trial.

Effectiveness measures were selected based on core outcomes from the ERI-2 trial. Follow-up data was collected during months 4 – 48 post-intake [20]. Days abstinent represents the sum of the number of days participants reported not using any alcohol or other drugs over follow-up, with a possible range of 0 to 1,350 days (90 days × 15 quarters post-intake). Missing observations (6% of all person quarters) were replaced with the average (within person) value across all quarters. On-site drug testing (urinalysis) was also conducted on all participants who claimed to be abstinent for the past seven days and probed for clarity when inconsistent results arose. The false negative rate (i.e., claiming abstinence but positive on urine test) was 1.7% on average with no significant differences by condition [20].

Substance use problems reflect the sum of 16 possible symptoms per month related to weekly use, complaints about use, hiding use, four symptoms of abuse, seven symptoms of dependence, and two sets of symptoms related to substance-induced health or mental health problems. The substance-use-problems measure was created by multiplying the participant’s reported number of problems per month times 45 months, replacing missing observations with the mean within-person value across all quarters. The possible range was 0 to 720 problems over all intervention months. Missing data had to be imputed for 5.2% of the quarters for days abstinent and 4.8% of the observations for past-month substance problems. Follow-up attrition was equivalent for both OM-only and OM-plus-RMC.

To estimate cost effectiveness, mean differences in program costs are divided by mean differences in program effectiveness to calculate the ICERs. One can compare the ratios of cost to outcome for two or more alternative programs to determine which program(s) is relatively more cost-effective as determined by a lower cost-effectiveness ratio or an ICER below a specified budgetary threshold. In the present study, the ICER reports the marginal cost of achieving one more day of abstinence or one fewer substance-use-related problem for subjects participating in OM-plus-RMC relative to OM-only.

A priori, we expect OM-plus-RMC to be cost-effective because it was designed to augment the OM process by targeting specific challenges and needs in sustaining recovery and was already shown to be effective in the ERI trials described above [15, 20]. This hypothesis is tested by calculating the incremental cost to achieve a day of abstinence (or one fewer substance-use-related problem) in OM-plus-RMC relative to OM-only, and then comparing this estimate to the value society places on this “per day” improvement or “per problem” reduction. Such a value is known as societal willingness-to-pay (WTP) and is used to designate the cost-effectiveness threshold.

Cost-effectiveness Acceptability Curve (CEAC)

The process of calculating the ICER is relatively straightforward, but there remains some degree of uncertainty associated with this ratio due to the typically skewed distributions of costs and outcomes (i.e., sampling variability). [30] Although it isn’t known exactly what society is willing to pay for an additional day of abstinence or a reduced substance-use-related problem, the analyst can project a value or range of values for WTP per unit of outcome to interpret the ICER in terms of net benefit (NB) and incremental net benefit (INB). For instance, if society values a day of abstinence at $100, the NB of each study condition is the number of days abstinent over follow-up multiplied by $100 minus intervention costs. The INB is simply the difference in NB between study conditions. [31]

To address the uncertainty surrounding the ICER (a point estimate), non-parametric bootstrap methods are used to produce 1,000 replicates of mean costs and effects; each one generating unique ICER, NB, and INB estimates. Based on those 1,000 replicates, the proportion for which OM-plus-RMC generates a positive INB over the range of potential WTP values is used to plot a cost-effectiveness acceptability curve (CEAC). The CEAC represents the likelihood that OM-plus-RMC is cost-effective as compared to OM-only over a range of WTP values. Therefore, it is not known what society will pay for one more day of abstinence or to prevent one more substance-use-related problem, but the CEAC illustrates how likely OM-plus-RMC is to be cost-effective relative to OM-only over a range of values where the WTP may lie.

Sensitivity Analysis

Additional robustness checks to the cost effectiveness results are warranted. We conducted a sensitivity analysis of OM-only vs. OM-plus-RMC by incorporating the costs to society from health services, social services, criminal activity, criminal justice system involvement, and employment problems to examine how results of the CEA change when costs in these other areas are added to the direct intervention costs. Societal costs were estimated using data from the GAIN [26] by multiplying each unit of outcome (e.g., visit to the emergency room, inpatient hospital day, robbery offense committed) by a corresponding monetary conversion factor representing the cost per unit of outcome. Monetary conversion factors for physical health, mental health, and substance abuse treatment services came from the American Medical Association (2010) [32], Agency for Healthcare Research and Quality (2009) [33], and French et al. (2008) [34]. Criminal activity costs were estimated using monetary conversion factors from McCollister et al., 2010 [35]. Public assistance and employment earnings were based on self-reported information from participants. Criminal justice system costs were obtained from the Illinois Department of Corrections. Criminal activity and criminal justice system costs were included in separate calculations of total societal costs because the monetary conversion factors for criminal activity already contain estimates of policing, legal and adjudication, and corrections costs per offense [35].

Expected Value of Perfect Information (EVPI)

As part of the sensitivity analysis, we consider the value to reduce the uncertainty associated with the ICERs using the EVPI [36, 37]. A decision maker’s ability to consistently choose the optimal intervention would require having perfect information a priori regarding the costs and effectiveness of competing interventions. Since perfect information is not usually attainable a priori, there is a tangible cost savings from being able to reduce the uncertainty surrounding the decision; also called the expected value of perfect information (EVPI) [37, 38]. The EVPI essentially describes the value of further research to minimize the uncertainty associated with the costs and effectiveness of the competing interventions and is generally reported as a dollar value per research participant (i.e., dollars per client). Readers are directed to works by Koerkamp et al. (2006) [38], Claxton (1999) [39], and Barton et al. (2008) [37] for additional conceptual and empirical details on EVPI.

RESULTS

Table 1 shows bootstrapped averages of intervention costs and effectiveness as well as incremental differences and the ICERs for each outcome. It cost on average (per participant) $4,889 to deliver OM-plus-RMC and $2,705 to deliver OM-only over the 4-year trial period. The incremental cost of the RMC component was therefore $2,184. Participants in OM-plus-RMC averaged 1,026 days abstinent and had 89 substance-use-related problems over the same period. OM-only participants were abstinent for 932 days and reported 126 substance-use-related problems. Thus, the incremental effectiveness of OM-plus-RMC was 94 additional days abstinent and 37 fewer substance-use-related problems over four years. The ICER for OM-plus-RMC relative to OM-only is $23.38 per additional day of abstinence and $59.51 per fewer substance-use-related problem.

Table 1.

Summary of intervention costs, outcomes, and incremental differences over four-year follow-up

Condition	Intervention Cost ($)	Days Abstinent	Number of Substance Use-Related Problems
OM plus RMC (N = 223)	4,889 (52.56) [4,784; 4,993]	1,026 (20.94) [983; 1,068]	89 (7.29) [74.72; 104]
OM only (N = 223)	2,705 (23.84) [2,640; 2,749]	932 (25.05) [881; 983]	126 (9.20) [107; 145]
Incremental Differencea	$2,184* (57.19) [2,075; 2,313]	94* (33.18) [27.56; 160]	37* (11.55) [13.03; 60.36]
ICERb	--	23.38 (68.37) [−46.53; 93.29]	59.51 (84.97) [−1,791; 1,910]

Notes: Bootstrapped means and standard errors reported (1,000 replications). Standard errors in parentheses and 95% confidence intervals in brackets.

^*p<0.001

Incremental differences may not be exact due to rounding errors. Intervention costs based on study by Dennis et al., (2011) [25] in which costs are reported in 2007 dollars.

^aThe difference in cost or outcome measured between the two conditions: OM-plus-RMC vs. OM-only.

^bIncremental cost-effectiveness ratio describing the cost per day abstinent and per reduced substance-use-related problem.

Figures 1 and 2 show the CEACs for days of abstinence and reduced substance-use-related problems. The vertical axis shows the probability that OM-plus-RMC is cost-effective relative to OM-only. The horizontal axis presents the range of WTP values per day of abstinence or per reduced substance-use-related problem. OM-plus-RMC achieves an extremely high likelihood of being cost-effective at a relatively low WTP per day of abstinence. If society values a day of abstinence at $35, for example, the probability that OM-plus-RMC is cost effective exceeds 80%. At values above $60, this probability approaches 100%. The CEAC for substance use-related problems shows that OM-plus-RMC has a very high probability (>90%) of being cost-effective at a WTP per reduced substance problem of $100 or greater.

Figure 1.

Cost-effectiveness acceptability curve (CEAC) for Abstinence

Figure 2.

Cost-effectiveness acceptability curve (CEAC) for Reduced Substance Use Problems

Results of the Sensitivity Analysis

In Figure 3, the EVPI shows that the decision maker will be unlikely to finance additional research to reduce the uncertainty surrounding intervention costs and days abstinent if WTP is very low (i.e., < $20 per day abstinent). However, as the societal value per day of abstinence increases (approaching the value established by the ICER), the uncertainty regarding the optimal intervention choice also increases. The decision maker’s expected value of reducing this uncertainty by conducting further research is $300 per participant at a WTP per day abstinent of $23. As WTP per day abstinent increases above this level, the uncertainty regarding the optimality of OM-plus-RMC relative to OM-only diminishes (EVPI decreases). In Figure 4, the EVPI for reduced substance-use problems is highest at a WTP per reduced substance problem of $59. This is the point where the expected value to the decision maker of conducting additional research to reduce the uncertainty regarding which condition is optimal is approximately $275 per participant.

Figure 3.

Expected Value of Perfect Information (EVPI) for Abstinence

Figure 4.

Expected Value of Perfect Information (EVPI) for Substance Use Related Problems

Incorporating Societal Costs

Table 2 presents the results of the sensitivity analysis where the societal costs of physical and mental health services, substance abuse treatment, social services, and criminal activity as well as earnings from employment are factored into the calculation of incremental costs. Over the duration of the trial, OM-only generated $24,048 in societal costs and OM-plus-RMC generated $21,763 in societal costs. When these societal costs are combined with direct intervention costs, total intervention-related cost becomes $26,652 for OM-plus-RMC and $26,750 for OM only. Although the incremental cost of RMC now becomes negative (i.e., −$98 per RMC participant), the quantitative difference between OM-only and OM-plus-RMC is small and not statistically different from zero. In a separate calculation of societal costs where criminal justice services (incarceration and days of probation/parole) replace criminal activity, OM-only generated $9,971 in costs over four years and OM-plus-RMC generated $8,300, again producing a negative incremental cost of RMC (i.e., −$1,671 per participant). This implies that OM-plus-RMC dominates OM-only as it is less costly and more effective in decreasing substance-use-related problems and increasing abstinence. All of these monetary values were highly skewed, however, and none of the group differences in costs were statistically significant.

Table 2.

Sensitivity analysis of intervention costs plus societal costs over four years

Economic Outcomes	OM-plus-RMC (N=219)	OM-only (N=217)
Physical and Mental Health Services	$11,637 (24,882)	$12,711 (22,153)
Social Services	$2,224 (5,477)	$2,460 (3,548)
Criminal Activity	$21,078 (82,619)	$19,438 (59,291)
Criminal Justice System Involvement	$2,739 (5,010)	$2,493 (4,644)
Treatment of Substance Use Disorders	$4,380 (4,927)	$3,765 (4,435)
Employment Earnings	(−) $17,556 (29,133)	(−) $14,238 (21,546)
Direct Intervention Costs	$4,889 (52.56)	$2,705 (23.84)
Total Societal Costs – criminal activity	$26,652 (97,981)	$26,750 (73,097)
Difference in total societal costs OM-plus-RMC minus OM-only	(−)$98 (122,243)
Total Societal Costs – criminal justice system	$8,300 (44,041)	$9,971 (36,118)
Difference in total societal costs OM-plus-RMC minus OM-only	(−)$1,671 (57,097)

Notes: Costs reported in 2007 dollars. Total costs include direct intervention costs plus societal costs (minus employment earnings). Total societal costs were calculated two ways: first, using self-reported criminal activity for which the monetary conversion factors already include criminal justice system costs, and second, using criminal justice system costs only. Standard deviations in parentheses. Incremental differences may not be exact due to rounding errors.

DISCUSSION

This study represents the first formal economic evaluation of the RMC approach for long-term management of SUD. Results suggest that, at very modest WTP thresholds, quarterly RMCs are a cost-effective strategy for promoting recovery from SUD. The estimated cost per day of abstinence ($23.38) and cost per reduced substance-use-related problem ($59.51) are relatively low, particularly considering that this intervention targets a high-risk/high-cost population with a history of chronic substance use. Additionally, in measuring the uncertainty associated with the incremental cost effectiveness results, we find that OM-plus-RMC shows a very high likelihood (>80%) of being cost-effective relative to OM-only, particularly for a value per day of abstinence (per reduced substance-use-related problem) of greater than $35 ($75).

These results are reinforced when including the societal costs of health services use, criminal activity (or criminal justice system involvement), public assistance, and employment earnings. Incorporating these additional costs (minus employment earnings) into the calculation of incremental cost rendered OM-plus-RMC less costly than OM-only. Although differences in total societal costs were not statistically significant, the fact that both conditions generated similar costs across health, social, and criminal justice systems is an important finding. Even if the full economic impact is roughly equivalent between OM-only and OM-plus-RMC, participants that received RMC services still had significantly more days of abstinence and fewer substance-use-related problems than those receiving OM-only. Thus, these comparisons strongly support the overall cost effectiveness of OM-plus-RMC relative to OM-only.

This study has a few notable limitations. First, intervention costs are based on group averages from a previous economic study [25]. On an individual basis, costs could be higher or lower depending on the amount of time each participant spends with a linkage manager or any other additional resources they may consume. Similarly, in a non-clinical-trial setting, quarterly assessments would not include a full GAIN, which takes an average of 63 minutes to complete. Although this was partially adjusted for in the cost analysis, we acknowledge that the average OM costs might be somewhat inflated.

Second, the sample was mostly African American from an urban setting, which raises questions regarding generalizability of results. Moreover, results from this study are based on a single trial of RMC and its implementation in an urban minority population. Costs and effectiveness of the RMC model might differ if implemented with alternative demographic groups and/or in other geographic settings. We aim to shed light on some of these questions as we are currently evaluating the economic impact of RMC for women offenders [20].

Third, outcome measures are based on self-report. To support the validity of participants’ reported abstinence, we were able to compare the reported number of days abstinent with urine drug screens conducted during assessments. As mentioned previously, the rate of false negatives across waves was only 2.4%, which dropped to 0.9% when adjusting for everyone reporting use of medications.

Finally, we have not addressed a recommended length of involvement in OM and RMC. Our analysis represents a four-year horizon, but if OM-plus-RMC is to be recommended as a long-term chronic disease approach to managing addiction, average length of participation could extend much longer. To rank OM-plus-RMC in terms of cost effectiveness among the broader list of chronic diseases such as hypertension or diabetes, we would need to estimate quality-adjusted life years (QALYs) or some other standard metric (e.g., life expectancy) that can be compared across disparate treatments and population subgroups.

In conclusion, results of this economic evaluation support OM-plus-RMC as a cost effective recovery management strategy, particularly if society values a day of abstinence at more than $23 or a reduced substance use-related problem at greater than $60. When the additional costs associated with substance use consequences are factored in, results suggest that OM-plus-RMC could actually be cost-saving. Furthermore, the longitudinal nature of this study supports the notion that long-term investments in recovery management strategies are economically viable and effective for achieving desired changes in substance use behaviors and related consequences.