Economic Evaluation of the Juvenile Drug Court/Reclaiming Futures (JDC/RF) Model

Abstract

Juvenile drug court (JDC) programs are an increasingly popular option for rehabilitating juvenile offenders with substance problems, but research has found inconsistent evidence regarding their effectiveness and economic impact. While assessing client outcomes such as reduced substance use and delinquency is necessary to gauge program effectiveness, a more comprehensive understanding of program success and sustainability can be attained by examining program costs and economic benefits. As part of the National Cross-Site Evaluation of JDC and Reclaiming Futures (RF), an economic analysis of five JDC/RF programs was conducted from a multisystem and multiagency perspective. The study highlights the direct and indirect costs of JDC/RF and the savings generated from reduced health problems, illegal activity, and missed school days. Results include the average (per participant) cost of JDC/RF, the total economic benefits per JDC/RF participant, and the net savings of JDC/RF relative to standard JDC.

Introduction

Although a number of evidence-based practices have been established for adolescents with substance use disorders, these practices have not been widely adopted in juvenile justice and behavioral health settings due to concerns regarding staff and other resource burdens, and budget impact.^1–3 For instance, adopting standardized screening and assessment tools requires staff time to be trained on using the tools, additional time with clients to administer the tools, data management and storage, and licensing fees. Other issues contributing to the disconnect between juvenile justice agencies and community treatment providers include the lack of open and consistent communication between these agencies, resulting in reduced referrals and linkages to care among juveniles with substance use and mental health treatment needs. Alleviating unmet needs in the juvenile justice population requires a coordinated effort between juvenile justice and behavioral health systems to support evidence-based screening, assessment, referral, and linkage to care. Calling upon these agencies to change current practices requires not only clinical evidence of best practices, but also consideration for the additional costs and reallocation of existing resources that will occur as a result.

The integrated Juvenile Drug Courts: Strategies in Practice (JDC: SIP) and Reclaiming Futures (RF) (JDC/RF) model fulfills this need and coordinates the overlapping goals of JDCs, RF, and behavioral health treatment.^4,5 Each of these entities strives to reduce the harmful use of alcohol and illicit drugs and associated illegal behavior, while promoting a healthy transition to adulthood. An important aspect of forming a more integrated system of juvenile justice and behavioral health services is the ability to demonstrate that such an approach is financially viable in systems where resources are already stretched thin. To contribute to the limited information on the economic impact of integrating juvenile justice, behavioral health, and community services, this paper describes the methods and results of a multisite economic evaluation of JDC/RF. As part of the National Cross-Site Evaluation of Juvenile Drug Courts and Reclaiming Futures,^4,6,7 this study estimates both the costs and economic benefits of integrating RF into existing JDC programs. Findings are reported on the specific costs associated with the integration of RF into an existing JDC, the economic benefits generated by reduced criminal activity and improved health and educational outcomes, and the net economic benefits of the JDC/RF program relative to standard JDCs.

Background

To date, relatively more studies have examined the economic impact of adult drug court programs than juvenile drug court programs.^8–11 The findings are generally supportive of adult drug courts, showing that the savings to the criminal justice, health, and employment sectors tend to offset the cost of drug courts. However, less is known about the costs and net economic impact of JDCs. This section summarizes findings from existing economic studies of JDCs, provides a summary of the JDC/RF model, and provides some background information on the different approaches to estimating program costs and benefits.

Previous economic studies of JDCs

While there are a few studies examining costs and cost-effectiveness of substance use treatment for adolescents, including juvenile offenders, they have been limited in scope and typically focus on a single treatment episode.^12–18 They are also difficult to compare given differences in analytic methods and perspectives, quality of data, and lengths of follow-up. Even less is known about the full economic impact of these programs, where the cost of JDCs is compared to the savings generated by these programs to the juvenile justice and public health sectors.¹⁹

Within the scant literature in this area, a recent study in the Journal of Family Psychology examined the costs and benefits of Multisystemic Therapy for Problem Sexual Behaviors (MST-PSB) in a sample of juvenile sexual offenders relative to standard community services.²⁰ The authors considered the direct costs to the criminal justice system as well as the broader societal costs to crime victims. Compared to standard community services, the group receiving MST-PSB generated significantly greater economic benefits from reduced re-arrest following treatment over an approximately 9-year follow-up period. An estimated $49 in economic benefits were generated per $1 invested in providing MST-PSB.

Among the more widely referenced work in this area is a collection of reports by researchers at NPC Research describing an evaluation of JDC costs and outcomes in Maryland, Indiana, and Oregon. A transactional and institutional cost analysis (TICA) approach was used to gather data from JDC agency financial records, budget reports, and interviews with JDC stakeholders. The consensus across these JDC outcome and cost evaluations is that JDCs appear to be cost-saving, generating reductions in arrests, court cases, probation days, detention days, and days in treatment.^21–24

Researchers at the Washington State Institute for Public Policy have also conducted several benefit-cost analyses of programs for adolescents with substance use disorders, including JDCs and other juvenile justice programs.^25,26 On average, about $2 in benefits were generated for every $1 invested in JDCs, demonstrating a positive net benefit to society.

To further evaluate the impact of JDCs on criminal activity specifically, McCollister et al. conducted an analysis comparing criminal activity costs among JDC and Family Court participants in South Carolina.¹³ The authors focused on criminal activity as this is a costly outcome to society and has been shown to generate the greatest contribution to total economic benefits of substance use-related interventions.²⁷ Results showed that all participants reduced criminal activity over the 12-month follow-up, but JDC participants had a slightly greater reduction in crime costs than Family Court participants. These results were not statistically significant, however, and must be interpreted with that limitation in mind.

Overview of JDC/RF model

One goal of the Reclaiming Futures initiative was to improve the operations and impact of JDCs by creating an integrated framework with the established Juvenile Drug Courts: Strategies in Practice (JDC:SIP) model. JDC:SIP was developed by the Bureau of Justice Assistance (BJA), the National Council of Juvenile and Family Court Judges (NCJFCJ), and the Office of Juvenile Justice and Delinquency Prevention (OJJDP) and features a 16-point plan to improve the planning, implementation, and operation of JDCs. Reclaiming Futures expanded on JDC:SIP by adapting the systems of care approach from children’s mental health to provide a model of juvenile justice reform with a specific focus on improving SUD treatment access, quality, and continuing care.²⁸

RF is a system-wide change intervention to 1) increase the performance of a variety of service delivery partners in identifying, engaging, and facilitating successful completion of system requirements by youth; 2) cultivate community readiness to engage these same young people in an increased array of positive youth development and longer term “recovery” activities that boost their prospects for long-term success; and 3) provide training and fellowship with similar staff from other sites.^29,30 To achieve these goals, each RF site utilizes a five person leadership team called the Change Team, which consists of a juvenile court judge, a juvenile probation officer, an adolescent substance use and mental health treatment professional, a community member (either a successful youth and/or family member, a representative of the faith community, an elected official or another person not employed by a formal helping system), as well as a project director. The project director’s unique role is to conceptualize, create, and execute a multi-system change strategic impact plan along with these diverse cross-disciplinary teams.³¹

Although change needs can vary by community, all RF plans have in common three core, quantifiable goals: 1) advocate for more treatment for young people in trouble with drugs and crime (e.g., addressing the shortage of care and identifying/assessing young people so that they can access appropriate treatment); 2) assuring better treatment (e.g., assuring that evidence-based, developmentally appropriate and culturally relevant treatment is operating successfully); and 3) building pathways beyond treatment (e.g., connecting youth to caring adults and peers, as well as positive youth development, pro-recovery activities in the community – some of which have to be specifically created as a result of the presence of RF). These goals allow for comparison between JDC programs that do and do not implement RF, and examination of the relationship between these factors and client outcomes.

In order to accomplish these goals, leadership teams work together to build six local and community-specific mechanisms, including: 1) standardized screening (to identify substance use and co-occurring problems); 2) more detailed standardized assessment (to make treatment/service plan recommendations); 3) service coordination (strengths- and needs-based plan developed by the RF team); 4) initiation (to assure that treatment begins within 2 weeks of being assessed); 5) engagement (to assure that youth receive at least 3 or more sessions within 30 days); and 6) transition (to assure that completion of treatment and withdrawal of court monitoring and agency services with concurrent connections to long-term community supports has occurred).

The purpose of the current study is to expand the national cross-site evaluation of JDC/RF to incorporate a benefit-cost analysis. The five sites that participated in the National Cross-Site Evaluation of JDC/RF had the same basic grant requirements to implement the integrated JDC/RF model into their existing juvenile drug courts as an enhancement, but there are few other similarities. The sites are diverse both geographically and in the populations they serve. Two sites were in the West, one in the Midwest, one in the South, and one in the Southeast. The urban, suburban, and rural settings create great differences between the sites in terms of the behavioral health and community services available to youth, accessible public transportation, and other unique community-level factors such as the quality and number of middle and high schools. These characteristics provide important contextual information for interpreting results of the economic analysis of JDC/RF. The next section describes the economic evaluation methods followed by the results of the analyses.

Methods

Cost analysis

The economic analysis of JDC/RF was conducted primarily from the juvenile justice agency perspective, with additional consideration for some of the broader societal costs associated with service delivery. Reported costs reflect the direct and indirect costs incurred by the juvenile justice system, behavioral health care providers, and community agencies to provide JDC/RF services. The juvenile justice agency perspective is represented by the direct expenditures of the JDC/RF program, whereas the societal perspective includes the indirect costs attributable to volunteer time and other subsidized resources. These “unpaid” resources are considered opportunity costs (i.e., costs that consider the foregone value of an alternative use for program resources). For instance, the time volunteered by a counselor to offer a meditation class could be spent on an alternative activity, including working for pay. There is a forgone opportunity to earn an additional hour or more working, and this value represents the opportunity cost of choosing to volunteer his/her time. The full societal perspective for a cost analysis would also include costs incurred by JDC/RF participants and their families (e.g., out-of-pocket expenses associated with the program, travel costs to attend program activities or drug court meetings, lost work time by parents to attend meetings). Estimating participant/family costs associated with JDC/RF was not possible, thus the perspective for the cost analysis is somewhat narrower than the full societal perspective.

Reported costs for JDC/RF are in 2012 dollars, which represents a single fiscal year when the project was at full capacity (typically year 3 of the grant, with some overlap into year 4). This also corresponds with the timeframe of the main outcome evaluation. Cost data were collected and organized using the Drug Abuse Treatment Cost Analysis Program (DATCAP) and came primarily from probation, court, and treatment provider financial records.³² Additional cost estimates were obtained using expert judgment from key program personnel and using self-report data available from the Global Appraisal of Individual Needs (GAIN) assessment instruments. The GAIN integrates clinical and research measures into one comprehensive structured interview with eight main sections: background, substance use, physical health, risk behaviors, mental health, environment risk, legal involvement, and vocational correlates.³³

Where direct salary information was not available (i.e., to estimate the value of volunteer time), the Occupational Information Network (O*Net) was used to identify occupations with similar job duties and provide relevant salary estimates.³⁴ Cost data are organized into the following general categories: 1) criminal justice system (including court and probation), 2) substance use and mental health treatment, 3) community services and volunteers, and 4) miscellaneous resources. Within each of these categories, costs included personnel (salary and benefits), the estimated value of volunteers’ time, contracted services and/or consultant fees, building/facilities, equipment and supplies, and miscellaneous resources (e.g. laboratory services, security, utilities). All cost estimates are reported in 2012 dollars, meaning the costs reflect the value of a dollar in that year.

Criminal justice system costs

Criminal justice system costs include time invested by court personnel, the prosecuting attorney, the defense attorney, and, when applicable, the guardian ad litem. Court personnel included the presiding judges, magistrates, clerks, bailiffs, judicial assistants, and administrative/office staff. The costs associated with the time invested by these criminal justice and legal staff were calculated using information on the salaries and benefits of each position and the estimated time spent in the court room for JDC/RF cases (typically 10–20% of the judges and lawyers’ time). In a few cases, the costs of court personnel and legal staff were included as part of the indirect rate for all court services/operations as the detailed information on salaries and benefits was not available. Additional costs were those associated with building (space) use, transportation, and shared administrative personnel and supplies (i.e., overhead).

Criminal justice system costs also include the costs of probation and JDC services. Probation staff consisted of probation officers and their supervisors. Most of the probation officers for these drug court programs dedicated 100% of their time to JDC/RF participants. Personnel in this category also included drug court staff such as the drug court coordinator, case managers, and the project director. Salary and benefits were provided for all staff in this category. All programs conducted drug screens (breath and/or urine) regularly, and most could provide actual paid or contracted costs for the tests and lab services. For one site, cost information was not directly available for these services, so costs were estimated using two sources: 1) participants’ self-reported frequency of drug testing from the GAIN at follow-up, and 2) the average cost of a drug screen as reported in state corrections expenditure reports available on the internet. Costs of participant incentives provided to encourage compliance with program requirements (e.g., gift cards) and transportation vouchers are grouped under a category “other participant assistance.” For one site, the costs of incentives and other assistance could not clearly be distinguished from costs for other JDC program supplies and therefore were not broken out for this category.

Sanctions for failing to meet JDC/RF program requirements are also included in criminal justice system costs. All sites used detention time as a sanction. The cost of days in detention was either provided directly, or, when not available as an aggregate estimate, was calculated using participant-reported data in the GAIN based on follow-up interviews conducted during the fiscal year of interest. The number of days reported were multiplied by the cost of a day in detention as provided by probation staff, or from state corrections expenditure reports available online. Other sanction-related costs included time in detention alternative programs and canine searches.

Substance use and mental health treatment costs

Substance use and mental health treatment costs primarily include personnel (i.e., administrative and clinical staff), miscellaneous supplies related to treatment provision, building/facility costs, staff benefits, and administrative services. Cost data for these services were obtained from financial records provided by the treatment agencies, based on actual payments or contracted rates to treatment agencies from court/probation records, or based on electronic records of types of services received, number of participants receiving these services, and existing cost-per-service estimates. Because few sites could separate the cost of staff time from the rest of the paid or contracted services costs, this category is mostly presented here as an overall contract rate or indirect rate. For two of the five sites, specific information regarding staff time spent administering the GAIN instruments was provided allowing the research team to report assessment costs separately from the contracted rates for treatment services. Participants primarily received treatment using the Seven Challenges® Program, the Adolescent Community Reinforcement Approach (A-CRA), and/or Motivational Enhancement Therapy/Cognitive Behavioral Therapy (MET/CBT). Within a single site, participants may have received treatment from one or more agencies. Most programs provided outpatient or intensive outpatient care, with a few participants receiving residential services.

Community services and volunteer costs

Community services and volunteer costs include contracted services from community agencies as well as the estimated value of donated community services and volunteer time. A variety of community services were available to JDC/RF participants to provide educational opportunities, life skills development, art, music, and other pro-social activities. The cost analysis included a subset of selected community services that were an integral part of each program during the fiscal year considered. Because participation in many of these services was somewhat unpredictable and subject to availability, they are captured as secondary costs to the core operations of the JDC/RF programs.

Volunteers were an important part of all the JDC/RF programs. A key volunteer cost reflects the value of time invested by the Community Fellows, committed members of the JDC/RF team who served as liaisons between the court and community agencies providing services and pro-social activities to participants. The opportunity cost of the Community Fellows’ time was valued at an hourly rate equivalent to that of a Public Relations Specialist (according to O*Net). The rate is based on a median annual salary of $54,170 and an hourly rate of $26.04.³⁴ Other JDC/RF volunteers served as community members on the Change Team or Advisory Boards. Change Team meetings lasted about one hour and were typically held once per month. To estimate the value of the Community Fellows and other volunteers’ time, the hourly rate for a Public Relations Specialist was applied to the number of attendees multiplied by the number of meetings held during the fiscal year. When compensated, the actual amount paid was included as part of the Community Services category. The services of mentors who volunteered their time were similar to those of a Social and Human Services Assistant from O*Net, and were valued based on an annual salary of $28,850 ($13.87/hour). Volunteers also conducted educational seminars as part of JDC/RF, including life skills and health classes. The cost per hour applied to these positions depended on the topic of the course and was obtained using salary information from local job postings for related types of positions or using estimated salaries from O*Net. The hourly rates ranged from $9.67 (i.e., teaching assistants) to $23.66 (i.e., health educators).³⁴

Most sites had an unpaid student intern who worked in the probation department doing various levels of administrative office tasks. Based on compensation data from O*Net for a General Office Assistant, the value of this time was calculated based on an annual salary of $27,470 ($13.21/hour).³⁴

Miscellaneous resource costs

Miscellaneous resource costs include training and technical assistance from the Reclaiming Futures-National Program Office (RF-NPO) and the National Council of Juvenile and Family Court Judges (NCJFCJ). The RF-NPO and NCJFCJ funded these services for the sites, and provided financial records of the value of the training/technical assistance services offered to each site during 2012. In addition to these training requirements, all sites were directed to use the GAIN as part of their grant requirements. Since assessment of all participants is a key element of the Reclaiming Futures Model, costs to license and use the GAIN are also included here. Certain supplies and indirect costs (e.g., travel, cell phones, office supplies) were presented as an aggregated overhead rate, and are included under ‘Other supplies/indirects’. There were other contracted services available at some of the sites that did not fit into the categories above. These include support groups for the participants’ parents and formal education provided by the school district (e.g., a charter middle and high school attached to an IOP treatment program). Most sites also paid for some type of program evaluation services. While this cost could be considered a requirement of the grants each site received (and therefore research-related as opposed to service-delivery related), every site indicated that they would typically pay for these services regardless of the grant requirements because of their value to the program. In the case of shared resources (e.g., administrative services), these costs were allocated to the JDC/RF programs based on juvenile case flow (proportion of juveniles in the JDC/RF program vs. other juvenile justice programs), or in the case of personnel involved in multiple programs, participants were asked to report the percentage effort (% full time effort) dedicated to JDC/RF.

Site-specific cost analysis reports were generated and shared with each site for their review and approval. Site-specific analyses allowed the research team to capture the variability in the services provided and resources available among the diverse sites. Each site reviewed and approved their individual cost reports, further validating the cost analysis results. Each report stated total annual program cost, annual cost per participant, weekly cost per participant, and average cost per participant over the duration of the program (based on participants’ average length of stay). For the cross-site comparison presented here, only aggregate (de-identified) summary statistics (mean, minimum, maximum, and standard deviation) are reported.

Indirect costs and RF costs

Because the economic analysis was framed from juvenile justice agency and societal perspectives, it is possible to represent the total annual cost per participant as two separate pieces: direct expenditures, which represent direct costs incurred by the JDC/RF programs, and opportunity costs, which include the value of all volunteer and donated resources that were invested during the selected cost analysis year. These leveraged resources are important in the assessment of JDC/RF’s net economic benefits due to the model’s emphasis on developing and maintaining community partnerships to provide youth with a full continuum of services. Presenting the results in this way allows a comparison of direct expenditures for the JDC/RF program with the full value of all resources invested by the sites, volunteers, and other stakeholders.

The specific costs attributable to RF were also estimated separately to highlight the additional resources that were required to incorporate RF into existing JDC programs. With a model utilizing a systems-level approach, it is challenging to determine the full range of RF-specific costs, which in some cases could have been a reallocation of staff time from one set of activities to another. To best identify these costs, feedback was solicited from multiple staff at each site regarding financial and other implications of adopting RF. A summary of the costs and/or additional responsibilities specifically attributable to RF reported by sites include:

1. Time – Staff and volunteer time was the largest opportunity cost resulting from the integration of RF. This included additional time spent by all criminal justice staff involved in the JDC/RF court programs, probation officers and other probation personnel, and substance use treatment providers on Change Team meetings, additional time spent by probation officers and other probation personnel on direct probation services, and time dedicated by the Community Fellow. For the purposes of this evaluation, the only time that could consistently be tracked as distinctly associated with RF was time spent in the Change Team meetings by the fellows and the community volunteers.

2. Evidence-based treatment – The RF model states that “interventions should include whatever mix of services is appropriate for each youth, perhaps including (alcohol and other drug) treatment.”³⁵ The RF model focuses on three major goals: more treatment, better treatment, and beyond treatment. To implement the goal of offering youth better treatment, sites awarded these grants were required to replace their treatment as usual with an evidence-based model such as the Seven Challenges® Program or Adolescent Community Reinforcement Approach (A-CRA). Thus, additional costs associated with using an evidence-based treatment model over other treatment services typically provided to drug court participants was also considered an RF cost. Providing these evidence-based treatments typically involves some costs for training, quality assurance, and certification not required for other modalities. The costs of implementing evidence-based treatment was not able to be estimated here, but note that it would likely be a tangible cost to a new site adopting RF for the first time. Due to this limitation, it is acknowledged that the estimate of RF-specific costs is conservative.

3. Assessment – The RF model recommends the use of reputable screening and assessment tools. For the purposes of these grants, sites were required to use the GAIN-I (initial assessment) and GAIN-M90 (quarterly follow-up assessment). Costs for use of the GAIN assessments can be broken down into 4 components: licensing, software, training and quality assurance (QA), and data management and reporting. While any treatment program would typically require the use of some standard assessment and a software system to maintain the data, the extensive training and QA processes, data management, and reporting services that are part of the GAIN service package are key components to using it as an evidence-based instrument. Therefore, costs of data management and training and QA are included here as RF costs.

4. Community services - All sites had strong ties with a diverse set of community organizations. Members of community organizations regularly attended Change Team meetings and provided youth with pro-social activities and opportunities to complete community service requirements. Some of the activities included music and art-oriented programs; working with animals; and building/construction. The cost analysis includes a representative sampling of these services in addition to formal/contracted arrangements between community organizations and the JDC programs. The challenge here is determining what portion of these services should be attributed to RF. For the purposes of this evaluation, all contracted and donated community services were designated as RF. The reasoning behind this was twofold: first, sites were being asked specifically to describe their RF programs, and second, a major goal of the RF model is to extend services beyond treatment, including involvement with the community. Therefore, these community programs are excellent examples of enacting this component of the model.

5. Training and technical assistance - The RF-NPO offered several tools and training opportunities to facilitate the integration of RF into JDCs and assure its proper implementation at the sites. Each site was offered coaching calls, on-site technical assistance, fellowship meetings, and opportunities to attend conferences. While these services were provided at no charge to the sites, there is clearly value-added by the integration of the RF model.

Benefit-cost analysis

Benefit-cost analysis (BCA) describes a full economic evaluation where both the costs and consequences of an intervention are estimated and compared to determine the net economic impact. A positive net economic benefit indicates the benefits generated by the intervention outweigh the costs (i.e., the intervention is cost-saving). Economic benefits represent positive consequences like increased earnings, or reductions in negative consequences such as criminal activity or drug-related emergency department visits. The relevant outcome categories and specific measures will depend on the chosen perspective for the analysis (e.g., societal, treatment provider, criminal justice system), as well as the estimation approach and availability of data. Units of service (e.g., hospital day) and other outcomes (e.g., robbery offense committed) will then be multiplied by their corresponding monetary conversion factor (also known as unit cost or unit price) to estimate societal costs. Monetary conversion factors (MCFs) are usually assembled from a variety of sources including published studies, government statistics, and administrative claims data. Domain-specific costs are then summed to estimate the total economic consequences of substance use.

To determine the savings associated with the provision of JDC/RF services, the research team selected four outcomes that are considered measures of success regarding adolescent substance use and juvenile justice interventions: the number of crimes committed, days missed school or work, days bothered by mental health problems, and days bothered by physical health problems. Each of these measures has been included in previous studies describing the economic consequences of substance use disorders.^14,36–39

The societal cost per crime varies depending on the severity of the offense, victim medical losses, pain and suffering, lost property values, and risk of homicide. For instance, the monetary conversion factor for a household burglary offense is $6,462 (in 2008 dollars) per act, whereas the societal cost per robbery offense is $42,310 because a robbery carries a small risk of victim death. A day missed from school is valued at $14.78 (in 2001 dollars), estimated from increased earnings associated with an additional year of schooling. MCFs for a “day experiencing medical/psychological/psychiatric problems” come from a study by French et al. (2002),⁴⁰ which outlined a methodology for using the Drug Abuse Treatment Cost Analysis Program (DATCAP) and measures from the Addiction Severity Index (ASI) to conduct economic evaluations of SUD interventions. These MCFs draw from the literature on the value of a statistical life (VSL) and quality-adjusted life years (QALYs) to project the cost per one-unit decrement in a quality-adjusted life day based on a VSL of $1 million.^41,42 A day experiencing physical health problems is valued at $27.02 and a day experiencing mental health problems is valued at $10.94. As JDC/RF programs are expected to impact health, juvenile justice, and educational outcomes, these measures are appropriate for examining the economic benefits associated with JDC/RF.

Data on these outcomes were collected from individual clients at each site via self-reported responses from the GAIN instrument administered to all youth upon and after program intake. The reported numbers (i.e., days or crimes) were then multiplied by MCFs derived to value the societal costs of these problems.^36,38 Outcome data collected for the year prior to intake to JDC/RF was compared to data collected in the year after clients entered the JDC/RF program. Examining changes over time and the associated costs allowed the research team to assess the economic benefits of JDC/RF as an intervention. Summary estimates include total savings attributable to JDC/RF and net savings of JDC/RF (savings minus program costs).

Since the purpose of the RF model is to enhance the effectiveness of JDCs, it is important to understand the net economic benefits of JDC/RF relative to JDC programs not integrating the RF model. To make this comparison, 1,080 clients were selected from 17 JDC sites that received grants from the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center on Substance Abuse Treatment (CSAT) and were required to use the GAIN for data collection. Thus, the same GAIN outcomes and 12-month analysis timeframes were available to examine the economic benefits of standard JDCs. The resources to collect detailed cost data from these comparison JDC sites were unavailable; instead, estimates from a series of JDC outcome and cost evaluations performed by NPC Research in the states of Maryland, Indiana, and Oregon were used.^23,43,44 NPC Research investigators used the Transactional and Institutional Cost Analysis (TICA) approach, which incorporated data from agency financial reports and expert judgment to estimate the costs of JDCs. The cost analysis of JDC/RF programs used similar sources of resource and expenditure data, thus it was determined that the JDC program cost estimates were comparable to this study’s cost estimates for JDC/RF.

Results

Cost analysis

Table 1 reports the summary statistics for the cross-site descriptive measures. Bracketed values reflect the direct expenditures incurred by JDC/RF programs (not including the value of donated or subsidized resources). The average (mean) total annual cost of JDC/RF programs during the selected year for the analysis (years 3–4 of operations) was $1,712,482, with a standard deviation of $1,063,901. The range in total annual cost ranged from $782,001 minimum to $3,442,448 maximum. This variation is in part due to the fact that some sites had only one program option available for all participants, whereas other sites had two or three “treatment tracks.” Sites with multiple treatment tracks had much larger participant case-flow and typically worked with a greater number and variety of treatment providers. Based on participant case-flow information, the average annual cost per participant was $50,216 with a range from $27,196 to $65,167. The total number of new admissions in fiscal/calendar year 2012 across the five sites was 245, and the average daily census (average number of participants accessing services per day) was 35.2, with a range from 12 to 56. In terms of the average length-of-stay in treatment, the shortest duration in JDC/RF was 32.3 weeks and the longest was 56.7 weeks. The mean length-of-stay across all JDC/RF programs was 40.9 weeks. The average weekly cost per participant was $963 with a range from $522 to $1,250. Based on the mean length-of-stay, the average cost per participant over the duration of the intervention was $38,288, with the lowest average cost per JDC/RF episode being $19,299 and the highest being $49,369 per participant.

Table 1.

Summary Costs of JDC/RF Across the Five Sites

	Total annual program cost	Annual program cost per participant	Average daily census	Average Length-of- stay (weeks)	Weekly cost per participant	Episode cost per participant
Mean	$1,712,482 [$1,538,166]	$50,216 [$43,654]	35.2	40.9	$963 [$837]	$38,288 [$33,098}
Standard Deviation	$1,063,901 [$1,086,805]	$17,639 [$16,726]	16.42	9.28	$338 [$321]	$11,716 [$10,447]
Minimum	$782,001 [$594,057]	$27,196 [$22,096]	12	32.3	$522 [$424]	$19,299 [$15,680]
Maximum	$3,442,661 [$3,302,646]	$65,167 [$58,976]	56	56.7	$1,250 [$1,131]	$49,369 [$43,177]

Note: Summary measures of direct costs presented in brackets. Total new admissions in fiscal/calendar year 2012 across all five sites was 245.

The costs for each of the main categories contributing to total JDC/RF costs (i.e., criminal justice system, substance and mental health treatment, community services and volunteers, and miscellaneous) are shown in Table 2. On average, the highest costs were associated with the criminal justice system (i.e., court and probation). The average (mean) criminal justice costs for JDC/RF programs in fiscal year/calendar year 2012 were $867,350 (standard deviation $519,857), with a wide range from $322,951 minimum to $1,663,504 maximum. Substance use and mental health treatment comprised a substantial portion of the costs at most sites, ranging from a minimum of $65,165 to a maximum of $1,320,309, with an average of $401,169. The average annual cost (including both direct expenditures and the value of unpaid resources) of community services/volunteer efforts was $120,035 with a range from $7,117 to $322,587. Miscellaneous costs ranged from a minimum of $136,049 to a maximum of $762,124, with an average of $323,886.

Table 2.

Costs of JDC/RF Across Criminal Justice, Behavioral Health, Community Services, and Miscellaneous Categories

	Criminal Justice System	Substance and Mental Health Treatment	Community Services and Volunteers	Miscellaneous Costs
Mean	$867,350	$401,169	$120,035	$323,886
Standard Deviation	$519,857	$524,876	$119,699	$255,207
Minimum	$322,951	$65,165	$7,117	$136,049
Maximum	$1,663,504	$1,320,309	$322,587	$762,124

When considered as a portion of the total value of services provided at each site, criminal justice system expenses comprised the highest percentage of total costs, ranging from 41% to 67% (though for one site, this was equivalent to the portion of costs associated with miscellaneous resources). The distribution of costs to the other three categories (substance use and mental health treatment, community services/volunteer efforts, and miscellaneous resources) varied widely across sites. Treatment made up as little as 8% to as much as 38% of total cost, community and volunteer services between 0.5% and 11%, and miscellaneous resources between 4% and 42%.

Direct expenditures vs. opportunity costs

The cost estimates presented in Table 1 and Table 2 represent the full value of all resources invested in the JDC/RF programs in their third to fourth year of operation. It should be noted that, while an attempt was made to capture volunteer time and other opportunity costs, there was a great deal of variability in the amount of community involvement and donated effort and resources reported across sites. Whether this was an actual difference in provision of materials and services or a reporting bias cannot be conclusively determined.

Given that the cost analysis incorporated the value of volunteer time and other resources, the difference between direct expenditures by JDC/RF sites to run the programs (i.e., standard operating costs) and the opportunity cost of the programs (i.e., full value of all resources invested in the program, regardless of cost or funding source) is highlighted. Of the $1,712,480 cross-site average annual cost of JDC/RF, 90% ($1,540,166) represent direct expenditures and 10% ($172,316) represent donated time and other resources. Bracketed values in Table 1 report the direct costs attributable to JDC/RF.

Based on direct expenditures only, the average annual cost per JDC/RF participant across all sites was $45,320. The average weekly cost per participant was $869. Based on an average length of stay of 40.9 weeks in the JDC/RF programs, the average direct cost over the duration of the intervention was $34,448. The additional cost per participant associated with opportunity costs was: $4,895 per year; $94 per week, and $3,840 over the duration of the intervention (i.e., per average JDC/RF episode).

Costs associated with Reclaiming Futures

The specific costs attributable to RF are also estimated separately to highlight the additional resources that were required to incorporate RF into existing JDC programs. As noted above, it is challenging to isolate RF-specific costs, especially those costs associated with the reallocation of resources across and within the juvenile justice, treatment, and community systems resulting from adopting RF. To best identify these RF costs, feedback was solicited from multiple staff at each site regarding financial and other implications of RF to the JDC programs. In these discussions, four cost components were consistently identified as RF-related: staff and volunteer time, evidence-based assessment, community services, and training and technical assistance. These costs averaged 15% of total JDC/RF program costs, ranging from 8% to 26% of total costs across sites.

Benefit-cost analysis

Table 3 shows the results of the benefit-cost analysis of JDC/RF and standard JDC. Desired changes in days missed school or work, days experiencing physical health problems, days experiencing mental health problems, and number of crimes are translated to dollars using monetary conversion factors reported in row 3 of Table 3. Across these outcomes, the average savings in the JDC/RF program was $170 per youth for days missed school or work, $267 savings per youth for days experiencing mental health problems, and a $122,565 average savings per youth for crimes committed. Physical health problems increased during this timeframe, generating an additional $145 in societal costs for reported days experiencing physical health problems. Adding up these values translates to an average savings of $122,857 per JDC/RF youth. Once the average costs associated with providing JDC/RF services are subtracted out ($38,288), a net savings of $84,569 per youth remains. Extrapolating these average savings over a larger (potential) cohort, for every 50 youths served by the JDC/RF program, there is a potential net savings of $4,228,469.

Table 3.

Costs to Society and Net Savings in JDC/RF

	Days Missed School or Work		Days Physical Health Problems		Days Mental Health Problems1		Number of Crimes2		Total per Youth
	JDC/RF	JDC	JDC/RF	JDC	JDC/RF	JDC	JDC/RF	JDC	JDC/RF	JDC
Monetary Conversion Factor3	$19.58		$27.02		$10.94		varies
Days/Times year before intake to treatment	23.70	28. 76	13.15	10.29	115.74	97.68	39.96	27.97
Cost year before intake to treatment4	$464.	$563	$355	$278	$1,266	$1,069	$168, 125	$145, 792	$170, 210	$147, 701
Days/Times year after intake to treatment	15.05	22.25	18.52	12.26	91.33	72.76	9.93	11.64
Cost year after intake to treatment5	$294	$434	$500	$330	$999	$796	$45,560	$68,764	$47,353	$70,323
Change in days/times from year before to year after (Pre-Post)	8.65	6.51	−5.37	−1.97	24.41	24.92	30.03	16.33
Mean Savings (Pre-Post)	$170	$129	($145)	($51.58)	$267	$273	$122, 565	$77,208	$122, 857	$77,558
					Treatment Episode Cost				$38,2 88	$25,445
					Net Savings (Total savings - Treatment costs)				$84,569	$52,113

¹Self-reported days bothered by nerve, mental, or psychological problems; disturbed by memories of things from the past; or problems paying attention and controlling behavior.

²Self-reported number of times committed each of 12 crimes (in the past year before intake or in the past 90 days after intake); value based on specific crime in 2012 dollars.

³Estimates for days missed school or work, days physical health problems, and days mental health problems from French et al.⁴⁰ Estimates for costs of crime from McCollister et al.³⁶

⁴Costs for missed school or work, physical health problems, and mental health problems in the year before intake are based on the value for the past 90 days multiplied by 4.

⁵Costs for all variables in the year after intake are based on an estimate of the year following intake to treatment (average of available responses at 3, 6, 9, and 12 months post-program intake about the past 90 days multiplied by 4).

To compare with results of the JDC/RF evaluation, similar analyses were conducted for standard JDCs, using an average JDC episode cost per client of $25,445 (from NPC Research reports discussed above). Analyses showed that the average savings in standard JDC programs was $129 per youth for reductions in missed school or work, $77,208 per youth for reduced criminal activity, and $273 per youth for fewer days of mental health problems. As with JDC/RF youth, physical health problems in the standard JDC cohort increased during this timeframe, generating an additional $52 in societal costs for reported days experiencing physical health problems. Combining these values generates an average savings of $77,558 per JDC youth, which is $45,299 less than the average (per participant) JDC/RF savings. While the average episode cost of standard JDC services was lower than JDC/RF services ($25,445 vs. $38,288), once the reduction in societal costs is factored in, a net savings of $52,113 per JDC youth remains, compared to $84,569 for JDC/RF. Thus, despite the higher cost per JDC/RF episode, the incremental net benefit in JDC/RF relative to standard JDC was $32,456.

Discussion

This economic analysis of JDC/RF had several objectives. First, it is an important contribution to the emerging research on the effectiveness of JDC/RF programs, which has limited information on the implementation and standard operating costs of JDC/RF across different settings. Second, results showing the cost savings associated with implementing RF, including net savings after factoring in the costs of providing program services, are presented here for the first time. This is important for understanding how JDCs essentially pay for themselves when their impact on key juvenile justice, health, and educational outcomes is factored in. And finally, a benefit-cost comparison between JDC/RF programs and standard JDC programs illustrates the incremental benefits of integrating RF into a JDC.

Results of the cost analysis found that the average cost for an episode of treatment in a JDC/RF program (including court, probation, incentives, sanctions, community services, and substance and mental health treatment) was $38,288 per youth based on an average length of stay in the program of 40.9 weeks. This is an estimate of the average cost of JDC/RF during a selected cost analysis year when the program was running at full capacity, and does not include start-up costs. There was large variation across sites in the average cost per JDC/RF episode (ranging from $19,299 to $49,369).

The estimated cost per JDC/RF episode was not clearly associated with the number of clients served or metropolitan size of the area served. All cost categories had wide standard deviations, and for both substance and mental health treatment and community services, the standard deviation was greater than the mean. Cost data are often highly skewed, and the authors hypothesize that variation in substance and mental health treatment costs may be related to the variation found between sites in the severity of substance and mental health problems at intake to the program. The variation in JDC/RF episode costs is also driven by the (sometimes limited) availability of community services in each area. Also, some sites expressed a clear preference to keep as many services in-house as possible and made little effort to integrate external community services. However, it must also be noted that collection of data on community services was one of the most difficult, and likely to be one of the most inconsistent, elements of this study.

The economic analysis also assessed if JDC/RF costs are offset by program-generated savings. To evaluate this, reductions in costs to society associated with health problems, criminal activity, and missing school or work among JDC/RF and standard JDC participants were examined. JDC/RF clients generated an average savings of $122,857 from fewer days of missed school or work, fewer days bothered by mental health problems, and fewer crimes committed. Both JDC/RF and standard JDC participants reported slight increases in physical health problems following treatment. The authors hypothesize that this finding may reflect the fact that withdrawal from substance use while participating in treatment may produce temporary uncomfortable physical symptoms. Factoring in the average cost per JDC/RF episode of $38,288, net economic benefits to society were $84,569 per JDC/RF client.

Standard JDC programs generated average savings of $77,558 and a net savings of $52,113 per client. These findings highlight the fact that standard JDCs, whether integrating a RF framework or not, result in cost savings to society; a finding which has not received consistent support in the literature. Further, results indicate that, despite higher episode costs, JDC/RF programs produce larger cost savings, more than offsetting higher program costs relative to standard JDC. Specifically, relative to standard JDC, the additional or incremental net benefit of JDC/RF was $32,456 (for the average JDC/RF client).

Limitations

This study is one of the most comprehensive of its kind available for JDC programs, and one of the only ones to consider the integration of RF into a JDC from an economic perspective. However, there are several important limitations to consider when interpreting the results. Regarding the cost analysis, there were some inconsistencies in reporting of costs across sites resulting from different levels of detail provided in financial records and other data. For instance, there were very detailed records on juvenile justice costs, but in some sites, less detail on substance use treatment or community-based services costs. As a result, assumptions were made about the average cost per episode of substance use treatment, and assumptions were also made regarding the average cost per day receiving services in a community agency like Big Brothers Big Sisters. The relatively small number of JDC/RF sites considered here coupled with the large variation in program costs suggests that caution is warranted when comparing the costs of a JDC/RF episode to alternative forms of monitoring and treatment for juvenile justice youth.

Calculating and comparing the cost savings of JDC/RF and JDC programs relied upon several assumptions as well. First, four measures were available in the GAIN at both intake and follow-up assessment points that could be monetized to estimate savings. These measures are all self-reported by clients, and thus are subject to recall, underreporting, and other biases. However, the GAIN has been shown to be valid, reliable, and consistent across numerous studies and the authors do not expect that reporting bias would systematically differ across JDC/RF and JDC clients. Additionally, for standard JDC program costs, the authors had to rely on data from JDC outcome and cost evaluations produced by investigators at NPC Research. These JDC programs are not part of the SAMHSA/CSAT cohort of JDC programs with available GAIN data that were used to estimate JDC savings, so the assumption was made that the average cost of JDC estimated by NPC Research is an appropriate proxy for any standard JDC program. The important limitation here is that there are many unmeasureable factors that may influence both program costs and outcomes across all of these sites, which cannot be accounted for in the analyses.

Finally, the authors did not have complete expenditure information for all substance use disorder treatment received by JDC/RF participants. The authors relied on information reported in the GAIN and standard treatment cost estimates from the literature to impute the value of SUD treatment. Leaving out these costs would have reduced the overall cost of JDC/RF, but would not have accurately reflected the full range of services that were provided as part of the JDC/RF approach. Despite these important limitations, this study is the first to examine the costs and benefits of JDC/RF relative to standard JDC and serves as a foundation for future studies in this area. Additional research is needed with experimental or quasi-experimental study designs, larger samples, and the ability to collect economic data to support continued funding and new implementation of JDC and JDC/RF programs.

Implications for Behavioral Health

While the juvenile justice system bears the burden of balancing punishment and rehabilitation among juvenile offenders, this burden extends beyond the traditional roles of incarceration and community supervision, as many youth enter the system with multiple clinical problems including SUD, mental illness, and HIV-risk behaviors. In a funding climate where available resources are already stretched thin, recommending that agencies take on additional responsibilities to integrate services more effectively with behavioral health providers and other community services agencies is a challenging proposition. Agencies need to be aware of what sort of investments will be required to sustain a more integrated system of care. Equally important is to understand how these investments pay off in terms of improved services delivery, reduced substance use, reduced recidivism, and other youth outcomes.

This study completed a detailed assessment of the costs and net economic impact of five JDC/RF programs. Results are supportive of JDC/RF as a cost-saving protocol given that JDC/RF costs were greatly offset by the savings from reduced crime, mental health problems, and days missed from school or work. When compared to standard JDCs, JDC/RF generated, on average, greater economic benefits per dollar invested in the program.

These results have important implications for juvenile justice and behavioral health policies and practices. First, although JDCs are known to be expensive programs that can be difficult to implement in settings with limited resources or without formal reimbursement mechanisms, this study demonstrated that these programs generated significant economic benefits to society in a relatively short timeframe (i.e., one year following intake to JDCs or JDC/RF). Second, the detailed accounting of the additional resources invested to adopt RF into existing JDC settings can also serve as a template for other JDCs interested in adopting RF. Finally, the ability to track changes in criminal activity, health services, and educational outcomes over time is an important strength for JDC agencies wishing to document the financial impact of their programs.

Figure 1.

Total Cross-Site Average Economic Cost of JDC/RF

Figure 2.

Proportion of Total Annual Costs Attributable to Elements of the Reclaiming Futures Model