Net Benefits of Recovery High Schools: Higher Cost but Increased Sobriety and Increased Probability of High School Graduation

Abstract

Background:

Recovery high schools (RHS) provide a supportive educational and therapeutic environment for students subsequent to treatment for substance use disorders (SUDs). Most students served by RHSs have concurrent mental health disorders and are at risk for school failure or dropout and substance use relapse.

Aims of the Study:

The central question addressed is whether RHSs are economically efficient alternatives to other high school settings for students in recovery. The aim is to estimate the incremental cost-benefit of RHSs.

Methods:

A quasi-experimental non-equivalent pretest-posttest comparison group design was used. We compared substance use and educational outcomes for adolescents who had received specialty SUD treatment; 143 who enrolled in an RHS were compared to 117 who enrolled in a non-RHS school. Groups were balanced by use of a propensity score to drop students who were not similar to those in the other group. The propensity score was also used as a covariate in multiple regression to estimate cost and outcome parameters and standard errors. To take account of uncertainties in impacts and shadow prices, we used Monte Carlo simulations to estimate the distribution of incremental benefits of RHS relative to non-RHS schooling.

Results:

Two beneficial impacts of statistical and substantive importance were identified: increased probability of high school graduation and increased sobriety. RHS students had significantly (p <.05) less substance use during the study period—at 12-month follow-up, 55% of RHS and 26% of comparison students reported 3 month abstinence from alcohol and drugs. Urinalysis confirmed abstinence from THC (cannabis) for 68% of RHS versus 37% of comparison students. RHS students’ high school graduation rates were 21% to 25% higher than comparison students. Adopting a societal perspective, incremental benefits of RHSs were estimated by monetizing the increased probability of high school graduation and comparing it to incremental costs. Mean net benefits ranged from $16.1 thousand to $51.9 thousand per participant; benefit-to-cost ratios ranged from 3.0 to 7.2.

Discussion:

Monetizing the benefits and the incremental costs of RHS relative to conventional schooling show substantial positive net benefits from RHS participation. Two factors lend credibility to the results. First, the RHS improvement in substance use indicates a mechanism through which the increased probability of high school graduation can plausibly occur. Second, the estimated increases in the probability of high school graduation were large and statistically significant. As the productivity gains from high school graduation are also large, the dominant benefit category is very plausible. Limitations include the non-randomized design; selection bias into the study conditions not fully controlled by the propensity scores; generalizability only to young people with treated behavioral health disorders; lack of estimates for direct monetization of reduced substance use among adolescents; possible attenuation of the value of education among individuals with behavioral health issues; and uncertainty in calculation of school costs.

Implications for Behavioral Health Policies:

This research provides evidence that the recovery high school model provides cost beneficial support for high school students after primary SUD treatment. The students who enroll in RHSs typically have co-occurring mental health and substance use disorders, adding complexity to their continuing care. Funding policies recognizing the multiple systems of care (behavioral health, education, child and family services, juvenile justice) responsible for these young people are called for.

Introduction

Recovery high schools (RHSs) provide educational and therapeutic support to students with treated substance use disorders (SUDs) who are at risk for school failure or dropout and substance use relapse (1–4). Students typically enter RHSs voluntarily after primary treatment for SUDs, and may be referred to RHSs by treatment centers, schools, or parents. About 40 RHSs are currently in operation; at least 90 have existed at some point since 1979 (5). Research using a controlled quasi-experimental design has demonstrated that RHS attendance is associated with positive outcomes of reduced substance use at six months (6). These findings of positive behavioral effectiveness call for closer examination of the social efficiency of RHSs. Thus, the central question addressed in this manuscript is whether RHSs are economically efficient alternatives to conventional high school for students recovering from substance use disorders (SUDs). That is, do RHS impacts justify their higher incremental costs relative to conventional schooling?

Background

Alcohol and other drug use by high school students is a recognized problem in many schools and communities. Drug and alcohol use not only have adverse effects on the health and safety of these youth, but also affect school performance––longitudinal studies indicate that substance use during adolescence is associated with poor academic performance, dropout, and lower college attendance (7–9). These students also have a very high prevalence of diagnosed or diagnosable co-occurring mental health disorders (10).

An estimated 1.1 million youth in the United States (4.4 percent of U.S. adolescents age 12 to 17) needed treatment for a substance use problem in 2016. Of these, an estimated 180,000 youth received treatment in 2016 (11). The progress of patients with SUDs is marked by cycles of recovery and relapse (12). Abstaining from alcohol or drugs represents a major challenge for students during and after SUD treatment, and young people are especially vulnerable to relapse during the period right after completion of a treatment program, whether residential or outpatient (13, 14). In particular, the first three months after returning from residential treatment are particularly risky for relapse (13).

During the particularly fragile period after SUD treatment, the school is a critical social environment for adolescents with SUDs. Schools provide crucial social networks which influence general development as well as academic success (15). For an adolescent in recovery from an SUD, the school context is likely to involve interactions with peers who are actively using substances. For example, in 2015, an estimated 21.7 percent of high school students were offered, sold, or given an illicit drug on school property (16).

RHSs support students recovering from SUDs by providing academic services in a structured environment that promotes recovery. The most complete picture of these schools and their operative program models comes from a descriptive study of 17 RHSs (3). RHSs are typically small schools (15–80 students) that offer both academic and therapeutic services. Services ancillary to the academic curriculum typically include individual and group counseling, chemical dependency education, family support, and random drug testing. Academic supports include transitional help for students to earn credits and catch-up for missed opportunities, as well as the provision of complete (although limited) high school curricula, often with on-line and non-traditional experiential components. Most recovery schools are either public charter schools, alternative school programs within or funded by public school systems, or private non-profit schools.

Study Design

Methods

This study used a prospective longitudinal quasi-experimental design to assess the outcomes and cost-benefit of recovery school attendance. It was not feasible to assign adolescents at random to attend RHSs. Instead, a quasi-experimental comparison group design was used. Design details are available in other publications (6,17–19), so are only summarized here. The study used a non-equivalent pretest-posttest comparison group design to compare outcomes for adolescents who had received specialty SUD treatment and subsequently enrolled in an RHS (intervention group) with outcomes for adolescents who had received specialty SUD treatment and subsequently enrolled in a non-RHS (the comparison group). We set an operational threshold of one month of RHS attendance for inclusion in the intervention group, based on expert opinion that little benefit would be expected from less than one month participation.

Baseline selection bias may introduce systematic differences between groups and is a primary threat to internal validity in quasi-experimental designs. To address this potential bias, we used a propensity score based selection of comparison cases and subsequently dropped non-comparable participants from both groups (17). The primary goal of dropping non-comparable cases in a quasi-experimental design is to assure that the groups are as similar as possible in the absence of randomization.

Participants

Adolescent patients were recruited from 10 SUD specialty treatment facilities in Minnesota, Wisconsin, and Texas. These locations were selected because RHSs were accessible nearby at the time of study recruitment, and local research teams were available to coordinate recruitment and data collection. Adolescents and their families were recruited into the study upon discharge from SUD treatment (baseline data collection, retrospective to the period immediately prior to the last treatment episode), and were followed at 3-months, 6-months, and 12-months after the baseline interview. Students in the intervention and comparison groups were recruited from the same treatment programs, which included acute psychiatric hospitalization, intensive outpatient, and residential SUD treatment facilities. After the first year of recruitment, this plan yielded a smaller number of RHS enrollees than expected; the recruitment strategy was modified to also identify adolescents who had recently enrolled in RHSs and had participated in SUD treatment within the past 12 months.

Students enrolled in the study between December 2011 and January 2016. A total of 294 adolescents enrolled during this period. At baseline 55 percent were male, 82 percent Caucasian, the mean age was 16.3 (SD=1.09), and median grade in school was 10.6. All participants had a history of treatment for an SUD (averaging 4.3 treatment episodes in a variety of modalities); 91 percent also reported various mental health specific treatment experiences. Parents reported that 37 percent had also received special school services for attention, behavior or learning problems. Thus this sample of participants can be characterized as a dually-diagnosed population with complex behavioral health needs. Additional details regarding the characteristics of these participants is available in other publications (1,6,10,19).

Of the 294 enrolled, 237 (80.6 percent) completed 6-month follow-up interviews and 210 (71.4 percent) completed 12-month interviews. Attrition analyses indicated that there were no significant differences between study completers and dropouts in terms of the baseline measures or study condition at the 12-month follow-up.

An additional brief on-line or telephone follow-up was conducted to assess status prior to the close of the study for all students and/or their parent if they were at least one year past the 12-month follow-up interview (i.e., 24 or more months follow-up), with successful contact for 68 percent (148 of 218) of the families attempted.

Propensity score estimation and sample pruning

In order to adjust for non-equivalence of RHS and comparison groups, a propensity score predicting the likelihood of RHS enrollment was used as a primary covariate. Baseline measures included in the propensity score were derived from prior research on the predictors of adolescent substance use treatment outcome (20). Measures contributing to the propensity score included: adolescents’ self-reported age, race, gender, comorbid mental health conditions (collected using the M.I.N.I. Structured Clinical Interview for major Axis I psychiatric disorder diagnostics)(21); alcohol use, tobacco use, other substance use, and mental health treatment service receipt (all collected using the Timeline Followback method)(22); physical health treatment receipt, life satisfaction, sources of life stress, criminal justice system involvement, and school problems (all collected using the GAIN-Q3) (23); perceived consequences of drug use (collected using the Personal Experiences Inventory) (24); interest in attending an RHS, perceived teacher support for substance use recovery, prior year school attendance and grades, negative attitudes toward school (collected using the Behavior Assessment System for Children) (25); perceived academic abilities, family income, parental education level, parental history of substance use treatment, and family history of mental health problems.

Use of this estimated propensity score (and its squared value to account for non-linear effects) served to control for potential baseline differences between the intervention and comparison conditions. It also provided a range of common support that required dropping of 34 extreme outliers from the analytic sample. There were no similar cases in both the intervention and comparison conditions for the dropped participants, defined as any case with a propensity score higher or lower than the highest or lowest score in the other group (RHS or comparison). This yielded a final analytic sample of 260 adolescents, 143 in the RHS group and 117 in the comparison group.

Data Analytic Procedures

Analysis for the present paper involved two major phases. First, the identified benefit and cost variables were identified, descriptive statistics (percentages and means) concerning the RHS and comparison students computed, and OLS regression coefficient estimated. The regression models included the dependent variable of interest, a binary independent variable designating condition (RHS or Comparison), and adjustment (covariate) variables consisting of the linearized propensity score and the propensity score squared (to control for non-linear effects). Early analyses included additional control variables, including baseline measures of the dependent variables, age and gender of the student, and site. However, the robustness of the propensity score was such that these additional control variables did not substantively affect the coefficients of interest and were dropped in the final modeling. These analyses yielded adjusted structural (B) regression coefficients and standard errors for the effect of interest (RHS participation). The coefficients provided in the tables are thus interpreted as the adjusted mean effect of the recovery school, in the original (unstandardized) metric of the dependent variable. Subsequently, these coefficients and standard errors were used in the cost-benefit analyses described in the next section.

Cost-Benefit Estimation

Public resources can be used in many ways to reduce the social costs of behavioral health disorders. Thus, a critical question is whether RHSs are good investments for society. Specifically, taking account of the impacts of RHSs on all members of society, do they provide a larger excess of benefits over costs than alternative investments. A necessary condition for a positive answer to this question is that RHSs provide positive net benefits relative to other high school settings for adolescents subsequent to primary treatment. That is, do recovery high schools provide incremental benefits in excess of their incremental costs? To answer this question, incremental impacts were estimated and monetized with plausible shadow prices adapted from the literature. Incremental institutional and parental resource costs required for RHSs were also taken into account. Positive net benefits, relative to other school placements for similar students, indicate that investment in RHSs is an efficient expenditure relative to conventional schooling. The magnitude of the positive net benefits provides a basis for comparing the efficiency of investments in RHSs with those of other programs to support students in recovery after primary treatment for substance abuse.

To facilitate potential comparison with other interventions, net benefits were estimated on a per-participant basis. As impact estimates and shadow prices are uncertain, a distribution of net benefits was derived from a Monte Carlo simulation based on assumptions about the distributions of estimated costs and benefits. Although for illustrative purposes point estimates are provided for each component of cost and benefit, the mean of the simulated distribution of net benefits provides the most appropriate assessment of net benefits per participant.

Incremental School Costs

Relative to conventional high schools, RHSs typically provide more therapeutic services and smaller class sizes that involve greater per student use of resources, largely due to low staff to student ratios. RHSs may also involve greater school-related parental expenditures and time costs.

Budget information and average attendance from three representative RHSs was used, with permission, to estimate typical costs of RHS programs. These data and supplementary interviews provide a basis for estimating the incremental average per pupil cost to school districts of RHSs. Doing so, however, poses several challenges. First, the effort faces a common shortcoming of evaluations of social programs––reliance on administrative data to estimate real resource costs. Second, and specifically relevant to RHS evaluations, the variety of alternative schooling options within a large school district complicates the identification of the counterfactual to program participation. Since many students in RHSs have Individual Education Plans (IEPs) recognizing their behavioral health disabilities, direct comparison of average costs is further compromised. Third, as one purpose of the exercise is to use the experience of the RHSs included in the study to predict the efficiency of future adoptions, avoiding any cost advantages or disadvantages that would not be replicable is appropriate.

Recovery school budgetary and attendance data were provided by the three RHSs which accounted for the most students in this research. Based on annual average budgets, reported number of students attending on a typical day, and local school district data from the National Center for Educational statistics, we calculated incremental RHS per pupil school costs relative to the average cost in the local public school districts. Recovery schools report higher total numbers of students served per year; however enrollment is dynamic and the average daily attendance is thus the appropriate number to use in these calculations. Dollar amounts were inflated to 2017 dollars for comparability in calculating incremental costs. Our estimates (Table 1) vary with incremental additional costs of $16,362 per student at Archway Academy in Houston, TX; $7,626 at Horizon High School in Madison, WI; and $1,238 at PEASE Academy in Minneapolis, MN. These estimates vary due to inclusions and exclusions from annual reported school budgets in both the RHS and public school reports. To compensate for this we used instructional costs only for PEASE and Minneapolis, and total per pupil costs for Archway and the Houston district as well as for Horizon and the Madison district in the analysis.

Table 1:

Estimate of Average Cost of Recovery Schools

School	Mean Annual Budget		Mean Daily Attendance	Mean Annual per Pupil Cost*		Incremental Annual RHS per Pupil Cost Difference from Local District Mean*
	Amt	Year(s)		RHS	Local District
Archway Academy (TX)	$1,935,189	2015–16	80	$25,135	$ 8,773	+ $16,362
Horizon High School (WI)	$ 223,191	2014–16	12	$19,326	$11,700	+ $ 7,626
PEASE Academy (MN)	$ 681,299	2012–15	64	$11,300	$10,062	+ $ 1,238

^*Inflated to 2017 dollars.

Students attend RHSs for varying lengths of time. Estimating incremental cost thus requires a per student per day incremental cost, which was obtained by dividing the annual per student cost by 180 days, the typical statutorily mandated number of days in the academic calendar in the United States. Assuming the aforementioned annual incremental per student costs for the three RHSs yields per student per day cost differences from non-RHS schools of $6.68 for PEASE, $42.37 for Horizon, and $90.90 for Archway Academy. This estimated cost difference is largely due to variation in staff to student ratios in these small schools, but also reflects imprecision in administrative reporting of school enrollments, attendance and expenditures.

The subsequent Monte Carlo simulation assumed a triangular distribution of per student per day costs with extreme points at $6.68 (the PEASE estimate) and $90.90 (the Archway Academy estimate), with a mode at $42.37 (the estimate for Horizon). The per student per day incremental cost was multiplied by the average number of days spent in RHS schools in this study (174 days), to obtain incremental participant cost.

Parental educational and time costs

Parents were asked about their education and behavioral health care costs related to the focal student. Table 2 summarizes means for the RHS and non-RHS comparison group as well as differences estimated in regressions with propensity score controls. Note that none of the differences in Table 2 are statistically significant at the α < .05 level. The estimated coefficients are nonetheless the best estimates of incremental impacts. Treating them as zero (and their standard errors as zero) by excluding them from the analysis is inconsistent with testing the “hypothesis” of interest, namely whether or not the net benefit of RHS is positive (26). Thus, net benefits were estimated in two ways: first, inclusive of all effects, whether or not statistically significant at the .05 level; and second, including just statistically “significant” effects.

Table 2:

Estimated Costs Related to Youth’s Substance Use Disorder as Reported by Parents

	Mean (SD)		Regressions adjusted for Propensity Scores
	RHS Students (n=107)	Comparison Students (n=81)	B Coefficient (SE) for RHS
Educational Costs 12 (Annual)	$1557 (2718)	$529.47 (887.9)	$856 (603)
Annualized Hours Missed Work	53.2 (95.0)	73.54 (116.83)	−21.8 (19.5)

Parental interview data indicate that RHS parents spent on average more than comparison group parents. The Monte Carlo simulations draw from a normal distribution centered at the adjusted mean difference, $856, and with a standard deviation of $603. On average RHS parents missed fewer hours of work due to their child’s SUD over the course of the year, resulting in a negative cost, relative to parents of comparison students. A normal distribution centered at the mean difference provides an estimate of −21.8 hours, with a standard deviation of 19.5 hours. Lost wages were monetized at the median US national wage in 2016 for all occupations as reported by the Bureau of Labor Statistics, $17.81, times 1.426 to capture fringe benefits (27,28).

Benefits of Reduced Substance Use

The proximate incremental impact of RHS attendance of most interest is reduction in substance use. As shown in Table 3, both in terms of raw numbers and regressions adjusting for propensity scores, RHS students on average have significantly far less substance use during the study period than students in other school settings. Urinalysis at the 12 month interview confirmed the large difference between groups for recent use of cannabis, consistent with self-reports.

Table 3:

Substance Use Outcome Variables

	Mean (SD) or Percent		Regressions adjusted for Propensity Scores
Variable (based on TLFB)	RHS Students (n=107)	Comparison Students (n=81)	B Coefficient (SE) for RHS
Percent abstinent 3 months at 12 months	54.7	25.6	.268* (.086)
Percent abstinent full 12 months	34.6	12.3	.235* (.078)
Days of alcohol use past 12 months	11.5 (28.4)	21.8 (39.8)	−17.94* (6.11)
Days of marijuana use past 12 months	43.3 (76.7)	119.4 (130.7)	−91.46* (18.86)
Days of other drug use past 12 months	16.5 (37.0)	21.3 (54.4)	−15.38 (8.37)
Percent with positive urine analysis for marijuana past 12 months	31.8	62.7	−.291* (.091)

^*p < .05

Although there is certainly a benefit from sobriety, there are no shadow prices available in the literature for adolescent sobriety and it was not monetized directly. Instead, the benefits of reduction in substance use are captured through expected impacts on criminal behavior and medical expenses. Further, sobriety’s long-term benefit is incorporated through school attainment as measured by increased probability of graduating from high school.

Productivity gains from school attainment

The protocol for this study called for a twelve-month follow-up interview with the participating students and a parent. However, at the 12-month point only about half of the participant youth had reached the typical high school graduation age of 18. With IRB (ethics board) approval, a brief follow-up contact was added during the final year of data collection for all participants who were at least two years past their baseline interview and had agreed to be re-contacted. At this point, the mean age of those reached was 20.5 (SD=1.45); all were past the typical high school graduation age. Participants (youth and parents) were asked via email to complete a brief on-line survey; those who did not complete the survey on-line were called by telephone and, if they consented, were interviewed using the on-line survey protocol. These interviews included questions regarding high school graduation status asked of both parents and students. Longer term (2+ years) follow-up attempts were made with 199¹ families; 129 (64.8 percent) responded. Response rates were higher for parents (66.1 percent) than students (36.6 percent). Data on high school graduation from this long-term follow-up were combined with any prior interview reports of graduation or drop-out from either parents or students at any of the prior interviews to determine high school graduation status at final contact. As there are plausible alternative ways of analyzing these data, several alternative estimates of the impact of RHSs on the probability of high school graduation were calculated

To assess the impact of RHS attendance on high school graduation, linear regression equations were estimated with graduation status as the dependent variable and four independent variables: RHS attendance, age at intake into the sample, propensity score at intake, and the square of the propensity score at intake. Results from the four regressions are shown in Table 4. The first regression used the entire analytic sample (n=260) assuming that all students with missing data were non-graduates. The estimated adjusted impact of RHS was 13.6 percentage points (B=.136) with a standard error of 7.7 percentage points (SE=.077). The second regression included only those cases for which graduation or non-graduation status was ascertained and the student was sufficiently old enough to be expected to graduate (n=164). The estimated impact of RHS was 23.9 percentage points with a standard error of 8.8 percentage points. The third regression excluded the Texas cases, which had both low and asymmetric response rates between RHS and comparison students (n=153). The estimated impact of RHS was 19.1 with a standard error of 8.0 percentage points. The fourth regression, which excludes both the unknown and Texas cases, estimated the RHS impact as 26.8 percent with a standard error of 8.5 percent.

Table 4:

High School Graduation (Combined parent and youth reports through 24+ month follow-up)

	Percentages		Regression Adjusted for Propensity Scores and Age
	RHS Students	Comparison Students	B Coefficient (SE) for RHS
HS graduate by last contact (assumes unknowns/missing did not graduate)	53.8 (n=143)	34.2 (n=117)	.136 (.077)
HS graduate by final contact-unknown cases excluded	81.9 (n=94)	57.1 (n=70)	.239* (.088)
Excluding Texas: HS graduate by last contact (assumes unknowns did not graduate)	60.9 (n=115)	38.8 (n=103)	.191* (.080)
Excluding Texas: HS graduate by final contact—unknown cases excluded	83.3 (n=84)	58.0 (n=69)	.268* (.085)

^*p < .05

The Monte Carlo simulation initially used results from the third regression (designated “preferred educational impact” in Table 7). Specifically, the percentage point increase in the probability of high school graduation was assumed to be normally distributed with mean 19.1 points and standard deviation of 8.0 points. However, as these estimated effects are large and drive the benefits, the analysis was repeated using the smallest (most conservative) impact estimate from the first regression, 13.6 percent with the 7.7 percent standard error. This smaller estimate is designated as “low educational impact” in Table 7.

Table 7:

Per Participant Net Benefits, Probability of Positive Net Benefits, and Benefit/Cost Ratios from Monte Carlo Simulation (10,000 trials)

Models*	Net Benefits per Student in 1000s of Dollars and (SD)	Fraction of Trials with Positive Net Benefits	Ratio of Benefits to Costs*
Full Model: Includes all categories of costs and benefits (preferred educational impact)	51.9 (25.2)	.99	7.2
Productivity with externalities: Includes only high school graduation benefit and incremental school costs (preferred educational impact)	39.1 (22.6)	.97	5.8
Productivity without externalities: Includes only high school graduation benefit and incremental school costs (preferred educational impact)	25.9 (16.8)	.96	4.2
Full Model: Includes all categories of costs and benefits (low educational impact)	38.3 (23.6)	.96	5.6
Productivity with externalities: Includes only high school graduation benefit and incremental school costs (low educational impact)	25.5 (20.8)	.91	4.1
Productivity without externalities: Includes only high school graduation benefit and incremental school costs (low educational impact)	16.1 (15.4)	.86	3.0

^*Full Model: ratio of productivity, criminal justice, and health benefits to incremental school and parental costs

Reduced Model: ratio of productivity benefit to incremental school cost

To monetize the value of a high school diploma, the analysis used a shadow price that updates one developed by the Washington State Institute for Public Policy (WSIPP, 29) as a component of its cost-benefit model (30, pp. 504–509). The estimate of the shadow price used national earnings data for individuals in four educational attainment groups: less than high school diploma, high school diploma, some college (including associates degrees), and a four-year college degree or advanced degree taken from the national Community Population Survey for years 2002 through 2014. These data allow estimation of the relationship between earnings and age over the working life (assumed to be 18 through 65 years) for each educational group. Earnings were converted to total compensation by taking account of the value of fringe benefits (28) and the predicted growth in real earnings and fringe benefits estimated by WSIPP. The earnings profiles were adjusted to take account of mortality prior to retirement (31). The resulting age-income profiles for the attainment groups were converted to present values using a 3.5 percent real discount rate.

The present values of total compensation for the educational attainment groups were adjusted for estimates of the causal effect of the education attainment on compensation––as those with greater cognitive ability are both more likely to attain higher levels of education and more likely to have higher earnings without it (32). Attributing all gains in compensation to the earned degree ignores the contribution of ability to productivity. Thus, although assessing the independent effect of educational attainment is challenging, it is necessary and has been addressed by economists whose work provides a basis for making causality adjustments (32, 33). The causality adjusted earnings by educational attainment were then used to calculate the expected present value of educational attainment taking account of educational paths following the earning of a high school diploma. This was estimated using data from the 2014 Current Population Survey: 34 percent obtain no further education; 31 percent obtain some college; and 35 percent go on to earn a college degree.

The resulting benefit of the high school diploma was adjusted for the present value of the average cost of higher education for those who attain it (34). Again, following the WSIPP approach, the resulting gains were adjusted to take account of positive externalities associated with higher total compensation (35–37).

To take account of the various uncertainties in the development of this shadow price, distributions were assumed for key parameters and used in a Monte Carlo simulation to estimate both the mean value of a diploma and its standard deviation. This procedure resulted in an estimated shadow price for a high school diploma of $264 thousand (SD = $56.5 thousand). Excluding externalities, the shadow price is $190 thousand (SD = $41.3 thousand). As these present values are realized on average two years after program participation (the mean age for RHS students was approximately 16), they were discounted by two years. The Monte Carlo results we present for the net benefits of RHS employ the estimated impact (and standard error) of RHS attendance on the probability of high school graduation and the high school diploma shadow price (and standard error) assuming normal distributions. Specifically, draws from the distribution of the increase in percentage points of graduation were multiplied by draws from the distribution of shadow prices.

Crime reduction benefit

Table 5 shows reduced numbers of reported days of illegal behavior and likelihood of arrest for RHS participants relative to comparison students. In the Monte Carlo simulation, values were drawn from normal distributions centered at −13.2 days and −5.9 percent with standard deviations of 10.4 and .069, respectively. Days of illegal behavior were monetized at $1,050, based on an estimate made of the “weighted average cost of an ‘undefined’ crime for drug abusers” (38, p.441). An arrest was monetized at $1,150 based on an estimate of the marginal operating cost for a misdemeanor arrest made by WSIPP (29, p.55).

Table 5:

Criminal and Legal Involvement Outcome Variables

	Mean (SD) or Percent		Regressions adjusted for Propensity Scores
Variable	RHS Students (n=107)	Comparison Students (n=81)	B Coefficient (SE) for RHS
Annualized self-reported days of illegal behavior	25.9 (49.3)	30.4 (61.1)	−13.2 (10.4)
Percent with any arrest reported at 6 or 12 months	12.0	20.3	−.059 (.069)

Avoided medical expense benefit

RHS students were less likely to have an emergency room visit, but more likely to have inpatient/residential SUD treatment, than were comparison students. As shown in Table 6, these effects are small in magnitude. As with parental costs, in the Monte Carlo simulations draws were made from normal distributions centered at the regression coefficient estimates (−.052 and .095, respectively) and with standard deviations equal to the standard errors (.090 and .067, respectively). Emergency department visits were monetized at $1,526 per visit based on an estimate made by WSIPP (29, p. 150). Inpatient/residential SUD treatment was included at $8,945 per episode (39). Note that future SUD treatment beyond that incurred in the one year of follow-up is not included in these models; presumably, with continued reduction in substance use, long-term treatment costs are likely to be less among the RHS students.

Table 6:

Use of Medical Care

	Mean (SD) or Percent		Regressions Adjusted for Propensity Scores
Variable	RHS Students (n=107)	Comparison Students (n=81)	B Coefficient (SE) for RHS
Any Emergency Room Visit	30.5	27.2	−.052 (.090)
Inpatient/Residential SUD Treatment	25.0	20.4	.095 (.067)

Excluded benefits

As reported in Table 3, RHS participants had statistically significant lower alcohol, marijuana, and other substance use relative to non-RHS participants at 12 months. Although the greater sobriety is itself a primary goal of recovery schools and extremely beneficial to the student, and therefore to society, it was not directly monetized for two reasons. First, there was no plausible mechanism to reliably estimate a shadow price for a day of adolescent abstinence; no estimated monetary savings for adolescent sobriety was located in literature searches. Second, sobriety or abstinence can be considered an intermediate outcome contributing to the monetized benefits of school attainment, reduced criminal behavior, and avoided medical expenses––including sobriety as an additional benefit would thus likely involve double-counting of benefits.

Results: Monte Carlo Simulation of Net Benefit Distributions

Net benefits and sensitivity to assumed parameter values were estimated through 10,000 trials, each randomly drawing parameter values from their assumed distributions as described in the preceding sections. Figure 1 shows the results for a model including all the discussed cost and benefit categories and using the preferred estimate for the impact of RHS on the probability of high school graduation. As noted in the figure, 99 percent of the trials yielded positive net benefits. The mean net benefit of RHS relative to other high school placements for students in recovery after primary SUD treatment was $51.9 thousand with a standard deviation of $25.2 thousand.

Figure 1:

Monte Carlo Simulation of Net Benefits (n=10,000 trials)

The calculation of incremental benefit/cost ratios depends on the division of monetized effects into benefits and costs (costs can be negative benefits and vice versa). For the full model, it is the ratio of the mean sum of productivity, criminal justice, and health benefits divided by the mean sum of school and parental costs, or 7.2 for the simulation presented in Figure 1. For the reduced models, it is the mean productivity benefit divided by the mean incremental school cost.

The first row of Table 7 summarizes these results and subsequent rows present Monte Carlo results for alternative assumptions. The first three rows present results for the preferred estimates of the impact of RHS on graduation rate; the second three rows present results for the low estimate. Recognizing that the only RHS impact with statistical significance (at p < .05) was on the probability of high school graduation, rows 2 and 4 include only the productivity gain from increased high school graduations and incremental school costs. Rows 3 and 5 also limit benefits to the productivity gain from high school graduations but exclude the externality benefits in the derived shadow price.

The range of net benefits across the modeling assumptions is substantial, ranging from the high estimate of $51.9 thousand per student to the low estimate of $16.1 thousand per student. Benefit-to-cost ratios ranged from 3.0 to 7.2. Even in the most conservative case, only 14 percent of trials produced negative net benefits. Based on the analysis presented here across a range of underlying assumptions, the conclusion is that RHSs are very likely to produce positive net benefits for students in recovery from SUDs after primary substance use treatment.

Discussion

This analysis of adolescents participating in Recovery High Schools subsequent to SUD treatment, compared to similar adolescents in other school settings, shows significantly greater relative gains in sobriety for the RHS students. Sobriety gains are presumed to reduce criminal behavior during schooling and increase the probability of high school graduation. Monetizing these benefits and the incremental costs of RHS relative to conventional schooling show substantial positive net benefits from RHS participation.

Estimating the impacts and efficiency implications of RHSs posed substantial challenges in terms of recruiting a sufficient number of RHS and similar comparison students, obtaining data from students and parents over an extended period, and making assumptions about appropriate modeling strategies. Consequently, some caution is required in interpreting the large estimated net benefits as a clear indication of the efficiency of RHS over other forms of schooling. Two factors lend credibility to the results. First, although not monetized, the substantial and statistically significant RHS improvement in substance use indicates a mechanism through which increases in the probability of high school graduation can plausibly occur. Second, the estimated increases in the probability of high school graduation were large and statistically significant, regardless of assumptions about missing data. As the productivity gains from high school graduation are also large, and widely recognized as such, the dominant benefit category is very plausible.

Limitations of this research include the non-randomized design, small sample size and possible selection bias into the two study conditions (19). Despite efforts to strengthen the internal validity of the quasi-experimental design by using propensity scores to balance conditions, the results from this study may not generalize to all young people departing SUD treatment. Many young people in behavioral health treatment do not reach the severity threshold where RHS attendance is considered (19). Another limitation of this study is the lack of estimates for direct monetization of the value of abstinence or reduced substance use among adolescents. Had such estimates been available, they likely would increase the estimate of net benefit of the RHS intervention over the estimates provided in this analysis. Nonetheless, the benefits of educational attainment, estimated using data from general populations, may be attenuated among individuals with behavioral health issues. Further, in our analysis the costs and benefits of the mental health component for these young people with co-occurring disorders has not yet been adequately explored (10). In addition, our analysis does not consider the prior treatment costs since these are not incrementally different between groups. Finally, there is uncertainty and instability in calculation of school costs. RHSs, as small programs, have per pupil costs that fluctuate greatly with dynamic enrollments; attendance and costs are not uniformly documented; and public school district costs are not consistently reported.

Recent recommendations (40) for rigorous economic analyses emphasize standards in (1) framing the evaluation, (2) estimating the costs of the intervention, (3) determining the magnitude and monetary value of the intervention’s effects, (4) developing metrics that summarize effects, (5) incorporating uncertainty into the estimate, and (6) reporting the findings of the study. Overall, this cost-benefit analysis meets these recommendations for scientifically rigorous investigation. The weakest component is the estimation of incremental costs. Although it could conceivably have been measured in terms of the specific additional resources required for RHSs relative to traditional schools, the assessment would also be prone to error because of variation in utilization rates caused by changing numbers of enrolled students as well as variation across locations in terms of prices. The Monte Carlo simulations take account of the uncertainty in the estimates of incremental costs.

One of the strengths of our analysis lies in the extensive data collected during the five-year study, which allowed for detailed benefit monetization. As recommended by Zavala et al. (41), the economic benefits of RHSs are calculated by using self-reported information, collected through interviews and surveys at baseline and at several follow-up points. The benefits that proved statistically and substantively significant were monetized, converted to present value, and incorporated into this study. As recommended, this study included sensitivity analyses, in the form of the Monte Carlo simulations, to account for variability and uncertainty. It is worth noting that our results demonstrate the positive net benefits and apparent efficiency of RHSs in general; this does not imply that every RHS is utilizing resources in the most efficient manner possible.

Implications for Behavioral Health Policies

This analysis indicates that the recovery high school model provides cost beneficial support for behavioral health recovery subsequent to primary specialty SUD treatment. The students who enroll in RHSs are typically diagnosed with co-occurring mental health and substance use disorders, adding complexity to their education and continuing care. Evidence from this study suggests that RHS students have significantly better substance use outcomes and high school graduation rates relative to similar students not participating in RHSs. Societal costs of taking the RHS model to scale may be large relative to the immediate benefits; however, the long-term societal benefits would appear to outweigh the high immediate costs. Further research is needed to estimate the precise costs, geographic distribution and reach of RHSs that would be required to meet the needs of communities by bringing the model to scale.

The net benefits of RHSs can be compared to those of other interventions intended to improve the outcomes of SUD treatment for adolescents. Relative to other cost-benefit estimates for adolescent SUD treatment, our results are quite favorable. WSIPP has estimated net treatment benefits ranging from negative $5,827 to positive $2,370 for youth substance abuse treatment (42). Even our most conservative estimate for RHSs ($16,100) exceeds these estimates. In our analyses, the societal benefits of RHS participation are spread over the life cycle of the participants, while the costs measured are concentrated in the one year after the baseline interviews. These costs are additive to the already incurred costs of SUD (and mental health) treatment in both study groups; thus, the initial family and societal investment is large, but with a substantial long-term benefit likely under-estimated in the analysis. With the observed differential abstinence rates, long-term reductions in the future costs of SUD treatment among the RHS students not captured in the analysis can also be expected. Thus, the findings from the current study suggest that RHSs are an efficient use of social resources to improve the behavioral health of young people with substance use (and often co-occurring mental health) disorders. Funding policies recognizing and braiding the multiple systems of care (behavioral health, education, child and family services, juvenile justice) responsible for these young people should be considered.