Comparative Effectiveness of California’s Proposition 36 and Drug Court Programs Before and After Propensity Score Matching

Abstract

California’s voter-initiated Proposition 36 (Prop 36) program is often unfavorably compared to drug courts, but little is empirically known about the comparative effectiveness of the two approaches. Using statewide administrative data, analyses were conducted on all Prop 36 and drug court offenders with official records of arrest and drug treatment. Propensity score matching was used to create equivalent groups, enabling comparisons of success at treatment discharge, recidivism over 12 months post-treatment entry, and magnitude of behavioral changes. Significant behavioral improvements occurred for both Prop 36 and drug court offenders, but while more Prop 36 offenders were successful at discharge, more recidivated over 12 months. Core programmatic differences likely contributed to differences in outcomes. Policy implications are discussed.

Introduction

California’s voter-initiated Substance Abuse and Crime Prevention Act, more commonly known as Proposition 36 (Prop 36), went into effect in July 2001, allocating $120 million annually for 5 years to provide community-based substance abuse treatment to nonviolent drug offenders in lieu of incarceration. Approximately 50,000 drug offenders are eligible for the Prop 36 program each year, and over its 8 years of operation thus far, there have been more than 300,000 admissions to drug treatment services (Urada et al., 2009). Research on Prop 36 has shown that the program successfully routed many drug-abusing offenders to treatment within a very short time period (Hser, Teruya, et al., 2007), resulted in favorable outcomes, especially among treatment completers (Longshore et al., 2005), and yielded a noteworthy cost savings to state and local governments (Longshore et al., 2005, Longshore, Hawken, Urada, & Anglin, 2006). Despite the successes of Prop 36, the program has been widely criticized for being ill-equipped to adequately treat and supervise the various needs of many offenders, especially those considered to have the most severe problems (Farabee, Hser, Anglin, & Huang, 2004; Hardy, Teruya, Longshore, & Hser, 2005). As the Prop 36 program matured, stakeholder groups increasingly voiced unfavorable comparisons between Prop 36 and other criminal-justice-supervised drug treatment options already operating in California. In particular, Prop 36 was often perceived to be less effective than drug courts.

In California, both drug courts and the Prop 36 program provide an opportunity for routing substance-using offenders from incarcerated settings into the community, where they can access and benefit from substance abuse treatment. As California’s budget deficit continues to worsen (California Department of Finance, 2009), placing the state at the top of 10 states recently identified as being in “fiscal peril” (Pew Center on the States, 2009), legislators debate whether to solve the state’s overcrowded prison problem through early prisoner release programs that could presumably route even more offenders to community-based substance abuse treatment programs. Yet plans submitted to the legislature have proposed to eliminate Prop 36 funding altogether (California Department of Finance, 2009) and signal that the Prop 36 program may be in jeopardy of permanent defunding. Such policy decisions may be prompted, in part, by perceptions among some key stakeholders that Prop 36 is less effective than drug courts. This paper provides empirical evidence on the comparative effectiveness of Prop 36 and drug courts, focusing on the primary desired social outcomes.

Drug Courts

Drug courts have been an integral component of a national trend toward “therapeutic jurisprudence” or “collaborative justice” in the United States. Soon after the inception of the drug court model more than 20 years ago, drug court professionals identified necessary key components (U.S. Department of Justice, 1997) and drug court programs proliferated, as did effectiveness studies (Turner et al., 2002). Drug courts are generally thought to be more effective than routine criminal justice case-processing at reducing rates of recidivism and drug use among offenders (Galloway & Drapela, 2006; Gottfredson, Najaka, Kearley, & Rocha, 2006; Krebs, Lindquist, Koetse, & Lattimore, 2007). However, citing variation in drug court operations, weak study designs, and mixed results, researchers have also identified the need for more rigorous evaluations (Belenko, 2002; Merrall & Bird, 2009; Wilson, Mitchell, & Mackenzie, 2006; Wiseman, 2005). Nevertheless, the drug court model has truly “gone mainstream” (Belenko, DeMatteo, & Patapis, 2007; Hora, 2002), and is now operating in all 50 states, including in all of California’s 58 counties (American University, 2009), and several other countries (Bean, 2002; Freeman, 2003; Makkai, 2002; McIvor, 2009; Shanahan et al., 2004; Werb et al., 2007). Few rigorous studies have compared drug court programs to other offender diversion programs.

California Drug Courts

More than 150 drug courts operate in California (American University, 2009), accounting for approximately 12% of all drug courts nationwide (Judicial Council of California, 2006). Minimum standards for California drug courts were adopted in 1998. Typically, a pre-plea diversionary model is applied, but in recent years, state funding has supported various pre- and post-plea collaborative justice efforts (California Department of Alcohol and Drug Programs, 2005, 2010; Judicial Council of California, 2010). The general goal of California drug courts is to provide access to treatment for substance-abusing offenders while minimizing the use of incarceration. The majority of courts include intensive treatment services with frequent monitoring and continuing care (Burns & Peyrot, 2003; Judicial Council of California, 2010). Practices associated with positive outcomes have been identified including: a non-adversarial approach, graduated sanctions and incentives, frequent testing for alcohol and drug use, a single overseeing treatment provider, volunteer judges with no mandatory rotation off the bench, and a minimum 6-month abstinence period prior to program graduation (Judicial Council of California, 2006).

Proposition 36

Since its passage 10 years ago by 61% of California voters, Prop 36 has matured into an established criminal justice diversion option for offenders in all counties in the state, and in the past decade, more than 20 states have considered, and some have implemented, legislation that is similar to Prop 36 (The Avisa Group, 2005). Under Prop 36, adults convicted of nonviolent drug possession offenses can choose to receive drug treatment in the community in lieu of incarceration. Offenders on probation or parole who commit nonviolent drug possession offenses or who violate drug-related conditions of probation or parole can also opt to receive treatment. To qualify for admission to drug treatment under Prop 36, eligibility determination is made based on the offender’s current offense and past criminal history, after which eligible offenders are offered treatment in lieu of routine criminal justice processing. Offenders who choose to participate are ordered to complete a treatment assessment and then enter the assigned treatment. Offenders who successfully complete Prop 36 requirements can petition the court to expunge the criminal arrest and conviction that initially qualified them for the Prop 36 program (California Penal Code 1210.1, section 5).

The policy and programmatic implications of Prop 36 have been much discussed (Appel, Backes, & Robbins, 2004; Ehlers & Ziedenberg, 2006; Klein, Miller, Noble, & Speiglman, 2004; Marlowe, Elwork, Festinger, & McLellan, 2003; Riley, Ebener, Chiesa, Turner, & Ringel, 2000; Speiglman, Klein, Miller, & Noble, 2003). Research has shown that Prop 36 has increased the number of criminal offenders accessing California’s drug treatment system (Hser, Teruya, et al., 2007; Longshore et al., 2005), many of whom were treatment-naïve individuals with multiple and complex problems that were unexpectedly severe (Hser et al., 2003; Wiley et al., 2004), resulting in several system-wide impacts (Hardy et al., 2005; Hser, Teruya, et al., 2007; Niv, Hamilton, & Hser, 2009). Most offenders (about 85%) receive substance abuse treatment in outpatient drug-free (non-methadone) settings, where their median length of stay is approximately 6 months, about 10% receive long-term residential treatment (with a median length of stay of 90 days), and relatively few (< 5%) receive short-term residential or narcotic replacement therapies (Urada, Longshore, & Conner, 2007). Prop 36 research has documented offender characteristics and treatment needs (Longshore et al., 2005; Urada et al., 2009), barriers to treatment entry (Evans, Li, & Hser, 2008), patterns of treatment services utilization (Evans, Hser, & Huang, 2009; Fosados, Evans, & Hser, 2007), treatment outcomes (Evans, Li, & Hser, 2009; Farabee et al., 2004; Hser, Evans, et al., 2007), and economic costs and benefits (Carey, Pukstas, Waller, Mackin, & Finigan, 2008; Longshore et al., 2006). While the design of the Prop 36 program was informed by the experiences and successes with drug court models in California (Hardy et al., 2005), significant differences between the two programs exist.

Programmatic Differences Between California’s Prop 36 and Drug Court Programs

A primary difference between Prop 36 and drug courts is the criteria for selecting offenders for participation. Drug courts typically utilize both offense-based eligibility criteria and standards of offender suitability (e.g., treatment motivation level, outcome of prior treatment experiences, addiction severity level, criminal history) to select participants who are most likely to succeed in the program. In contrast, the Prop 36 program, by law, must be made available to all offenders who meet the conviction-based eligibility criteria regardless of treatment motivation level or other indicators of program suitability. Accordingly, Prop 36 handles many more offenders than do drug courts. Each year about 40,000 of the 50,000 eligible offenders are admitted to drug treatment under Prop 36, while fewer than 3,000 offenders are admitted to treatment under drug courts (Urada et al., 2009). Another key difference is that drug courts emphasize offender accountability and, to that end, typically include intensive judicial supervision and court monitoring (e.g., frequent status hearings with the judge), frequent and random drug testing, and brief incarcerations for program noncompliance. Prop 36 specifies that probation supervision and court monitoring occur, but the type and level of supervision and monitoring are unspecified and under-resourced; for example, there were no funds allocated for urine testing in the annual allocations, a deficiency remediated by separate legislative action (i.e., Senate Bill 223; California Department of Alcohol and Drug Programs, 2001). Moreover, incarceration of offenders for program noncompliance is prohibited. Finally, Prop 36 provides up to three opportunities for offenders to re-enter treatment without incarceration despite initial failures (e.g., as demonstrated by a pattern of no-shows to treatment or court, positive urine tests, re-arrest, or other acts of program noncompliance), whereas under drug courts, continued treatment following program noncompliance is at the discretion of the judge (Carey et al., 2008; Cosden et al., 2006).

Common Criticisms of the Prop 36 Program

When the performance and outcomes of Prop 36 and drug courts are compared, perceived limitations of the Prop 36 program are often cited. Prop 36 is primarily criticized as having exceedingly broad and inflexible criteria for determining offender eligibility; insufficient judicial interactions with offenders; insufficient monitoring and supervision of offenders; overprovision of opportunities for noncompliant offenders to access treatment repeatedly; slow, inconsistent, or non-existent provision of rewards for beneficial behavior and sanctions for non-compliant behavior; and inadequate resources to provide the type and amount of substance abuse treatment, oversight, and ancillary services often needed by offenders (Little Hoover Commission, 2008). There have also been concerns that Prop 36 may have inadvertently hampered the effectiveness of drug courts (Carey et al., 2008; Hardy et al., 2005; Urada et al., 2009). This concern is founded on the practice that many drug offenders first receive treatment through Prop 36 (about half of all Prop 36 offenders have had no prior drug treatment [Hser, Teruya, et al., 2007]) and must fail the program before drug court is considered as an alternative. Inherent in the design of Prop 36 is that offenders may access treatment up to three times with relatively few criminal justice sanctions. Consequently, it is argued, some offenders with several treatment failures under Prop 36 might later enter treatment under drug court having developed resistant behaviors and increased cynicism toward the effectiveness of substance abuse treatment. While drug courts reportedly use criteria to select participants who appear most likely to succeed, some stakeholders have indicated that Prop 36 may have inadvertently “hardened” drug offenders to treatment and decreased offender motivation to take drug court provisions seriously (Evans, Anglin, Urada, & Yang, 2010). Others hold an opposing view, arguing that prior treatment exposure can produce incremental attitude change that favors later treatment participation. As yet, little research has been conducted to understand whether prior Prop 36 treatment experiences among offenders who later participate in drug court may harm or enhance treatment readiness and outcomes.

Many leading stakeholders have expressed a preference to make Prop 36 programming more like drug court procedures (Urada et al., 2009; Little Hoover Commission, 2008). Underlying many of these discussions is the presumption that drug courts perform better and have better outcomes than Prop 36, despite limited empirical information on the comparative effectiveness of the two programs. Drug courts and Prop 36 share similar goals but, as discussed above, the methods for achieving these goals differ considerably. Additionally, some data indicate that there are consequential differences between the programs in the characteristics of offenders routed to treatment and in the type and amount of treatment received.

Differences in Offender Characteristics, Treatment Received, and Outcomes

The limited research indicates that offenders who participate in substance abuse treatment through the Prop 36 program have different characteristic profiles, different types and amounts of treatment received, and different outcomes from those who enter treatment via drug courts. The only published comparison of drug courts and Prop 36 found that for offenders in two court-based treatment programs, there were differences by treatment-referral source in offender histories, perceived importance of treatment, and predictors of program completion (Cosden et al., 2006). Cosden reported that compared to drug court participants, Prop 36 offenders tended to be older and more ethnically diverse, more used methamphetamine, fewer had prior treatment, and their rated need for treatment was lower as was their treatment completion rate. By contrast, an unpublished report comparing Prop 36 and drug courts as implemented by two courts located in two different counties reported that recidivism over 3 years did not differ significantly between Prop 36 and drug court offenders (Carey et al., 2008). Aside from this work, the only other source of information on differences between Prop 36 and drug courts is contained in UCLA’s annual Prop 36 evaluation report, which showed that there were differences between Prop 36 and drug courts in offender characteristics, treatment received, and outcomes (Urada et al., 2009). Findings were informative but restricted by several notable limitations. Importantly, differences in offender characteristics were not statistically tested, outcome analyses were not adjusted for apparent differences in offender characteristics and treatment received, and outcomes beyond those assessed at treatment discharge were not examined. As another limitation, analyses and results were not subjected to scientific peer review, a process that is critical for establishing and disseminating evidence-based knowledge about the effectiveness of behavioral interventions.

In summary, the performance and outcomes of Prop 36, a relatively new mechanism for treating drug offenders in California, have often been unfavorably compared to drug courts, a well-established and popular criminal justice model. Extant data indicate that the two programs serve different types of offenders and provide different treatment experiences. These dissimilarities indicate differential prognostic criteria that must be considered when comparing outcomes across groups (Kalich & Evans, 2006; Merrall & Bird, 2009). Yet little empirical evidence exists on how the two programs compare statewide, and few studies have examined short- and longer-term outcomes after accounting for the offender differences demonstrated at treatment admission and for differing treatment experiences.

To fill this knowledge gap, we examined the following research questions. How are Prop 36 and drug court offenders different in offender characteristics at treatment admission and in drug treatment received? After creating equivalent groups with propensity score matching, are there differences between groups in rates of success at treatment discharge, arrest over 12 months after treatment admission, and degree of behavioral change over time? We hypothesized that before propensity score matching, Prop 36 offenders would exhibit a more moderate level of drug use and criminal severity at treatment entry and they would receive a less intense level of care, but that after propensity score matching, both groups would show improved functioning over time and the drug court group would exhibit better outcomes.

Methods

Data Sources

Data were collected as part of the evaluation of the Substance Abuse and Crime Prevention Act of 2000 (Urada et al., 2009), conducted since 2001 by the UCLA Integrated Substance Abuse Programs (ISAP), which obtained existing administrative data compiled by several state agencies. The analyses utilized individually matched data from two administrative databases: the California Outcomes Measurement System (CalOMS), maintained by the California Department of Alcohol and Drug Programs (ADP), and the Automated Criminal History System (ACHS), maintained by the California Department of Justice (DOJ). CalOMS contains self-reported and official information on all individuals admitted to publicly funded drug treatment. Data are recorded at treatment entry and discharge, regardless of whether treatment is successfully completed or not, and submitted by treatment provider staff. Arrest histories were acquired from ACHS on all Prop 36 and drug court offenders in CalOMS. The Institutional Review Boards at UCLA and at the California Health and Human Services Agency approved all study protocols.

In fiscal year 2006–07, 46,754 unique offenders entered drug treatment statewide via referral by Prop 36 or by drug court. Our analysis identified all offenders who: (1) were discharged from treatment as of December 31, 2008, and (2) had a criminal history record on file in DOJ’s ACHS as of December 31, 2008. This cut-off date was chosen to accommodate the 3 months that is generally needed for an arrest to appear in the DOJ database (Hser & Evans, 2008) and, given that most Prop 36 and drug court offenders receive ≥ 90 days of treatment but very few receive > 12 months of treatment (Urada et al., 2009), to allow each offender an adequate amount of time for treatment to occur and for a treatment discharge record to appear in CalOMS. In addition, guided by prior cross-system data linkage research conducted with multiple administrative data systems maintained by the State of California (Hser & Evans, 2008), a combination of personal identifiers, including offender name, Social Security number, and date of birth served as the primary variables for applying a deterministic method to link CalOMS and DOJ records. Because a deterministic method accepts as a match only those cases that completely fulfill the matching criteria, under-linkage of data was expected; however this concern was considered to be outweighed by the high certainty of linkage associated with the deterministic method and the corresponding level of confidence in resulting findings. Of all offenders with a CalOMS admission record, about 88% had discharge data on file by the cut-off date and 75% were matched to DOJ data.

Criteria resulted in an analytic sample of 27,911 Prop 36 offenders and 1,320 drug court offenders, totaling 29,231 offenders with a complete set of data needed for analyses. The characteristics of offenders omitted from (n = 17,523) and included in (n = 29,231) analyses were compared at p < 0.05 on variables at treatment entry including: age, sex, race/ethnicity, education level, employment status, and primary drug problem type. All differences noted here were statistically significant at p < 0.001, unless otherwise indicated. Among drug court referrals, there were no differences in age or sex, but the analytic sample did include more African American offenders (23.0% vs. 19.6%; p = 0.02), fewer employed offenders (14.7% vs. 20.0%), fewer heroin users (9.0% vs. 12.5%; p = 0.002), and offenders with slightly more mean years of education (11.4 vs. 11.2 years; p = 0.01). Among Prop 36 offenders, the analytic sample included offenders who were slightly younger (35.6 vs. 36.2 years old), fewer women (25.6% vs. 27.8%), more White offenders (44.7% vs. 37.6%) and fewer African Americans (13.4% vs. 14.5%) and Hispanics (34.0% vs. 40.0%), more users of methamphetamine (58.1% vs. 53.2%) and marijuana (13.2% vs. 12.0%) and fewer users of cocaine (12.2% vs. 16.1%) and heroin (6.9% vs. 8.5%), and slightly more mean years of education (11.5 vs. 11.3 years). Notably, some raw percentages were very similar (e.g., 13.4% vs. 14.5% African American; 13.2% vs. 12.0% marijuana users; 6.9% vs. 8.5% heroin users) but nevertheless were found to be different statistically, illustrating how large sample sizes increase statistical power, making it possible to detect even minor differences between groups. However, these small percentage differences were not considered substantively significant to unduly bias subsequent analyses.

Of the total sample (n = 29,231), mean age was 35.6 years at treatment entry, 25.9% were women, race/ethnicity was 44.4% White, 33.9% Hispanic, 13.9% African American, and 7.9% “other” race/ethnic group, mean years of education was 11.5, 57.4% reported methamphetamine as their primary drug, 32.4% were employed full- or part-time, and 49.9% had no prior drug treatment. In the 30 days prior to treatment entry, 24.6% had been arrested, 33.8% incarcerated, and 54.7% had used their primary drug. Offenders were distributed across all counties in California and their distribution was proportional to county-level adult drug treatment populations.

Propensity Score Matching

A randomized controlled study design is considered to be the optimal method for evaluating the efficacy of an intervention. Within this design, research participants are randomly assigned to receive either the “new” treatment that is being tested (i.e., the experimental group) or treatment as usual (i.e., the control group), thereby balancing potential self-selection biases or other confounding variables between the groups being compared. Despite methodological strengths, the randomized controlled study design is rarely used to evaluate the effectiveness of drug courts and other court-mandated substance abuse treatment programs (Merrall & Bird, 2009).

Offenders are not randomly assigned to participate in either Prop 36 or drug courts. Instead, offense history determines eligibility for the Prop 36 program, whereas offense history, substance abuse history, motivation level, and other factors thought to increase the chance of successful outcomes are typically used to select offenders for drug court. Furthermore, offenders may choose to not participate in Prop 36 or a drug court and receive routine criminal justice processing instead, thereby potentially introducing uncontrolled and often unknown self-selection biases. Real-world complexities like these pose challenges for health-related effectiveness research. Accordingly, some researchers have emphasized the need to go beyond randomized controlled study designs toward the development of a broader range of suitable methodological techniques for conducting such comparative research (Tucker & Roth, 2006). Propensity score matching is an analytical approach that corrects for selection biases and, as such, is an effective tool for achieving comparison groups that are adjusted to be statistically unbiased (D’Agostino, 1998; Rosenbaum & Rubin, 1983, 1985). A growing number of substance abuse and criminology research studies have applied propensity score matching techniques (e.g., DeLisi, Barnes, Beaver, & Gibson, 2009; French, Fang, & Fretz, in press; Slade et al., 2008; Ye & Kaskutas, 2009).

We used propensity score matching to create equivalent groups that could be compared on rate of success at treatment discharge and longer-term arrests. Propensity scores were estimated using 30 variables as predictors, including 27 measures of offender characteristics at treatment admission and 3 indicators of drug treatment received: (a) Seven demographic and living-situation variables: age; sex; race/ethnicity; education level; employment status; has children aged 17 or younger; and homelessness. (b) Eight substance abuse and treatment history variables: primary drug problem type; age first used primary drug; recent primary drug use; frequency of recent primary drug use; recent injection of drugs; prior admission to alcohol or drug treatment; recent participation in social support activities (e.g., self-help meetings, support from family or friends); and lives with an alcohol or drug user. (c) Six mental health and medical status variables: ever diagnosed with mental illness; ever diagnosed with tuberculosis, hepatitis C, or a sexually transmitted disease; currently pregnant; stayed overnight in a hospital or had an emergency room visit in the 30 days prior to admission. (d) Six criminal justice status variables: arrest or incarceration in the 30 days prior to admission; type of offense (violent, property, drug) and number of arrests in the 12 months prior to treatment admission. (e) Three indicators of drug treatment received: treatment modality (outpatient, residential, narcotic replacement treatment); treatment retention; and treatment completion.

Propensity scores were estimated using multivariate logistic regression models predicting drug court membership. Variables were first entered into the regression and then a forward-selection approach was used to sequentially select quadratic terms of continuous variables and interaction terms between each pair of variables. Only significant terms (p < 0.05) were retained for the next step and as a result, the final propensity model included 4 quadratic terms and 49 interaction terms. Next, based on the logit form of the estimated propensity scores (D’Agostino, 1998; Rosenbaum & Rubin, 1985), each offender in the drug court group without missing values on the 30 predictors (n = 1,154) was matched with an offender in the Prop 36 group that also had no missing values on those same predictors (n = 25,388) and had the nearest available matching propensity score, with the maximum distance (or “caliper”) limited to 0.10. The caliper width was equivalent to 7.8% of the standard deviation of the logit propensity scores for the drug court and Prop 36 groups combined, which is much less than the 20% recommended by Rosenbaum and Rubin (1985). As a result, 1,062 subjects from each group were selected for the matched sample.

Measures

Treatment retention, or number of days in drug treatment, was defined as the number of days from the date of CalOMS admission during fiscal year 2006–07 to the date of CalOMS discharge or December 31, 2008, whichever date occurred earlier. If an individual had multiple treatment events during this time period, events that occurred within 30 days of one another (meaning the date of first treatment discharge and the date of second treatment admission were ≤ 30 days apart) were “tied together” into one episode of care, consistent with the CalOMS definition of a treatment episode (California Department of Alcohol and Drug Programs, 2007), and all days of treatment within this episode, whether successfully completed or not, were analyzed.

Treatment completion is indicated on the CalOMS discharge record by treatment staff when an individual has successfully completed all treatment program requirements and recovery goals. Completion was determined if it was marked at discharge from the given episode of care.

Treatment success included drug use, arrest, and incarceration status as assessed at treatment discharge and recorded in CalOMS by treatment provider staff. Although each measure is analyzed separately, a composite index of success on the three indicators was also constructed, defined by no use of any illicit drugs, no arrest, and no incarceration (in jail or prison) in the 30 days prior to treatment discharge.

Recidivism was calculated using Department of Justice records on post-treatment admission arrests. The arrest percentage and total number included any arrests that occurred in the 12 months following the date each individual entered treatment through December 31, 2008, uncorrected for time-at-risk (i.e., months of incarceration during this period).

Statistical Analyses

Offenders were compared before and after propensity score matching. Variables were analyzed by using chi-square or Fischer’s exact test (for categorical variables) and t tests (for continuous variables) on socio-demographic characteristics at treatment admission, experiences during treatment, three measures of success at treatment discharge and the composite indicator of success, and recidivism 12 months after treatment admission. In addition, behavioral changes in outcomes (three indicators of success in the 30 days prior to treatment entry and discharge; arrests over 12 months pre-post treatment entry) and their interaction with treatment group (drug court vs. Prop 36) were tested by generalized estimating equations (GEE). The significance level for all statistical tests was set at p < 0.05.

Results

Group Differences Before Propensity Score Matching

Analysis of unmatched data (Table 1) showed that at admission to treatment there were many significant differences between Prop 36 and drug court offenders on most variables examined. With respect to demographics, compared to drug court offenders, Prop 36 offenders were older (mean 35.6 vs. 34.6 years), fewer were female (25.6% vs. 31.4%), more were White (44.7% vs. 37.7%) and fewer were African American (13.4% vs. 23.0%), more were employed (33.3% vs. 14.6%), fewer were homeless (11.2% vs. 18.9%), and Prop 36 offenders lived for more days with an alcohol or drug user (2.5 vs. 1.6 days). There were no differences between groups in years of education or percentage with a child aged 17 or younger.

Table 1.

Offender Characteristics at Treatment Entry, Treatment Received, and Outcomes

	Unmatched		Matched
	Prop 36 (n = 27,911)	Drug Court (n = 1,320)	Prop 36 (n = 1,062)	Drug Court (n = 1,062)
Demographics
Age, Mean (SD)	35.6 (10.5)c	34.6 (11.1)c	36.3 (10.2)	36.1 (10.4)
Female, %	25.6c	31.4c	32.2	32.6
Race/ethnicity, %
White	44.7c	37.7c	38.0	38.9
African American	13.4c	23.0c	23.0	22.9
Hispanic	34.0	32.0	31.7	30.9
Other	7.9	7.3	7.3	7.3
Years of education, Mean (SD)	11.5 (2.1)	11.4 (2.3)	11.6 (1.9)	11.6 (2.2)
Employed-past 30, %	33.3c	14.6c	17.3	15.6
Has children aged ≤17, %	47.7	44.9	45.1	45.1
Homeless, %	11.2c	18.9c	21.0	20.1
Days lived with alc/drug user-past 30, Mean (SD)	2.5 (7.8)c	1.6 (6.2)c	1.6 (6.4)	1.6 (6.1)
History of substance abuse and treatment
Primary drug type, %
Methamphetamine	58.0c	43.6c	46.8	47.6
Cocaine	12.2c	25.6c	26.6	26.5
Heroin	6.9b	9.0b	9.8	10.1
Marijuana	13.2	12.1	7.6	7.6
Alcohol	7.9	8.1	6.9	6.8
Other	1.8	1.7	2.4	1.5
Age first used primary drug, Mean (SD)	21.0 (8.1)	21.1 (8.2)	21.3 (8.0)	21.5 (8.3)
Used primary drug-past 30, %	55.5c	36.4c	34.1	37.8
Days used primary drug -past 30, Mean (SD)	5.5 (9.2)c	4.2 (8.2)c	3.6 (7.4)	4.2 (8.1)
Injected drugs-past 30, %	14.6c	18.0c	19.3	19.4
Prior alc/drug admits-lifetime, Mean (SD)	1.1 (2.5)c	1.4 (2.4)c	1.3 (3.8)	1.3 (2.3)
Days of social support-past 30, Mean (SD)	5.4 (9.1)	5.5 (9.9)	5.5 (9.6)	5.9 (10.2)
Mental health and medical status
Ever been diagnosed with, %
Mental illness	14.9	13.1	12.6	13.6
Tuberculosis	2.2	2.6	2.1	2.5
Hepatitis C	5.0	6.0	5.4	5.7
Sexually transmitted disease	3.1	3.1	3.6	3.1
Pregnant, %	5.4	8.1	9.7	8.7
Overnight hospital stay/ER visit-past 30, %	8.5	6.9	6.5	7.3
Criminal justice status
Arrested-past 30, %	24.4c	28.6c	29.4	29.9
Incarcerated-past 30, %	32.3c	67.7c	64.3	65.3
Arrested-past 12 mos, %	75.9	75.1	75.5	75.2
Violent	2.6b	3.9b	3.0	2.9
Property	9.1c	19.6c	18.0	16.6
Drug	58.8c	52.7c	57.2	57.0
# arrests-past 12 mos, Mean (SD)	1.54 (1.46)c	1.79 (1.82)c	1.75 (1.72)	1.75 (1.68)
Violent	0.03 (0.19)b	0.04 (0.23)b	0.03 (0.19)	0.03 (0.21)
Property	0.11 (0.38)c	0.31 (0.80)c	0.25 (0.64)	0.23 (0.63)
Drug	0.87 (0.97)	0.91 (1.17)	0.98 (1.20)	0.99 (1.18)
Treatment received
Modality, %
Outpatient	85.6c	55.5c	57.9	56.6
Residential	13.5c	44.0c	41.5	42.8
Narcotic replacement	0.9	0.5	0.6	0.6
Treatment retention, Mean (SD)	118.1(108.4)c	135.7(137.3)c	132.9 (136.0)	133.2 (131.8)
Completed treatment, %	36.1a	33.4a	33.0	33.5
Outcomes
In 30 days prior to treatment discharge, %
Used primary drug	26.0c	18.5c	16.3	18.7
Arrested	5.7	7.1	5.1	6.0
Incarcerated	6.4c	33.0c	9.2c	31.4c
Successful on three indicators, % d	69.7c	51.7c	77.3c	53.4c
Arrested-12 mos post-treatment admit, %	48.0b	44.0b	49.7b	43.1b
# arrests-12 mos post-treatment admit, Mean (SD)	0.87 (1.2)b	0.77 (1.1)b	0.89 (1.19)b	0.74 (1.12)b

^ap < 0.05.

^bp < 0.01.

^cp < 0.001.

Indicators of drug use and substance abuse treatment history showed that more Prop 36 offenders reported their primary drug problem to be methamphetamine (58.0% vs. 43.6%) and fewer reported cocaine (12.2% vs. 25.6%) or heroin (6.9% vs. 9.0%). In the 30 days prior to admission, more Prop 36 offenders had used their primary drug (55.5% vs. 36.4%) and for more days (5.5 vs. 4.2 days), fewer had injected drugs (14.6% vs. 18.0%), and Prop 36 offenders reported a fewer number of prior lifetime drug treatment admissions (1.1 vs. 1.4 admits). There were no differences in age at first primary drug use or number of days having participated in social support activities.

As for mental and medical health at treatment admission, all measured health indicators revealed similar proportions reporting a diagnosis of mental illness, tuberculosis, hepatitis C, or a sexually transmitted disease, current pregnancy, and recent overnight hospital stay or emergency room (ER) visit.

Analysis of criminal justice status showed that in the 30 days prior to treatment admission, fewer Prop 36 offenders had been arrested (24.4% vs. 28.6%) or incarcerated (32.3% vs. 67.7%). About 75% in both groups had been arrested in the 12 months prior to treatment admission; however, the number of prior arrests over that time period was significantly different, with Prop 36 offenders exhibiting fewer arrests overall (1.54 vs. 1.79 arrests). When arrest data were analyzed by type of arrest, fewer Prop 36 offenders had been arrested for a violent (2.6% vs. 3.9%) or property (9.1% vs. 19.6%) related arrest, more had been arrested for a drug offense (58.8% vs. 52.7%), and Prop 36 offenders had fewer violent (0.03 vs. 0.04 arrests) and property arrests (0.11 vs. 0.31 arrests).

Additional group differences were revealed by analysis of drug treatment received. More Prop 36 offenders received treatment in an outpatient setting (85.6% vs. 55.5%) and fewer were treated in a residential setting (13.5% vs. 44.0%). Very few offenders in both groups received narcotic replacement treatment (e.g., methadone maintenance). Treatment retention rates were different by group, with Prop 36 offenders receiving a fewer number of treatment days overall (a mean of 118.1 vs. 135.7 days), but exhibiting a slightly higher treatment completion rate (36.1% vs. 33.4%).

Finally, in the 30 days prior to treatment discharge, more Prop 36 offenders had used their primary drug (26.0% vs. 18.5%), but substantially fewer had been incarcerated (6.4% vs. 33.0%). There was no significant difference in the percentage arrested during this time period (5.7% vs. 7.1%). As a composite of these three outcome measures, more Prop 36 offenders were successful at discharge (69.7% vs. 51.7%). However, over 12 months after treatment admission, slightly more Prop 36 offenders were arrested (48.0% vs. 44.0%) and they had a greater number of arrests (0.87 vs. 0.77 arrests).

Group Differences After Propensity Score Matching

Propensity score matching created two groups that were comparable on all offender characteristics at treatment entry and on all treatment received items that were examined (Table 1). As for outcomes, in the 30 days prior to treatment discharge, differences in primary drug use disappeared. As seen in the unmatched data, there continued to be no difference between groups in arrests, but many fewer Prop 36 offenders were incarcerated during this time (9.2% vs. 31.4%) and also more were successful based on the composite indicator (77.3% vs. 53.4%). During the 12 months after treatment admission, more Prop 36 offenders were arrested (49.7% and 43.1%) and their number of arrests was greater (0.89 vs. 0.74).

Differences in Degree of Behavioral Change Over Time

Both Prop 36 and drug court offenders made significant improvements after treatment, as indicated by significant decreases in primary drug use, arrest, and incarceration, and patterns of improvement were similar before and after propensity score matching (Table 2).

Table 2.

Amount of Behavioral Change Over Time

	Unmatched		Matched
	Prop 36	Drug court	Prop 36	Drug court
Change 30 days before treatment discharge vs. 30 days before treatment entry
Used primary drug, %	−29.5a b**	−17.9a b**	−17.8a	−19.1a
Arrested, %	−18.7a	−21.5a	−24.3a	−23.9a
Incarcerated, %	−25.9a b***	−34.7a b***	−55.1a b***	−33.9a b***
Change 12 months after treatment entry vs. 12 months before treatment entry
Arrested, %	−27.9a	−31.1a	−25.8a b*	−32.1a b*
# arrests, Mean (SD)	−0.67(1.66)a b***	−1.02(1.81)a b***	−0.86(1.79)a b*	−1.01(1.72)a b*

^aDifferences across time (pre, post) are significant at p < 0.001, GEE.

^bInteraction effect between time (pre, post) and groups (Prop 36, drug court) is significant (b*p < 0.05; b** p < 0.01; b*** p < 0.001), GEE.

However, unmatched data showed that decreases in primary drug use were larger among Prop 36 offenders (−29.5% vs. −17.9%) and improvements in incarceration were smaller (−25.9% vs. −34.7%). There was no group difference in change in arrests from treatment entry to discharge, but the decrease in number of arrests over the 12 months after admission was much smaller among Prop 36 offenders (−0.67 vs. −1.02).

Matched data showed that both Prop 36 and drug court offenders made similar degrees of improvement from treatment admission to discharge in primary drug use (e.g., −17.8% and −19.1%) and arrests (e.g., −24.3% and −23.9%), and Prop 36 offenders showed a greater reduction in incarceration (e.g., −55.1% vs. −33.9%). There was also a notable decrease in the percentage of both groups arrested in the 12 months after treatment admission compared to the 12 months before treatment admission (e.g., −25.8% and −32.1%) and in the number of arrests that occurred (e.g., −0.86 and −1.01), but these decreases were significantly smaller among Prop 36 offenders.

Discussion

As hypothesized, analyses of unmatched data confirmed that there are differences between Prop 36 and drug court participants in offender characteristics, treatment received, and outcomes. Prop 36 offenders exhibit less severe drug use and criminal behaviors on some indicators; more Prop 36 offenders are treated in an outpatient setting, for a fewer number of days, and more complete treatment; at treatment discharge more Prop 36 offenders report use of their primary drug but many fewer are incarcerated and more are successful; and over the 12 months following treatment admission, more Prop 36 offenders are arrested and they are arrested more frequently. When offenders are propensity-score matched, differences in drug use at discharge disappear, fewer Prop 36 offenders are incarcerated, and even more are successful, but patterns of longer-term recidivism remain similar to those seen in the unmatched data. Finally, both Prop 36 and drug court offenders make significant improvements in all areas measured; however, Prop 36 offenders exhibit a greater reduction in incarceration from admission to discharge and, in the 12 months after treatment admission, Prop 36 offenders show a smaller reduction in both the percentage arrested and in the number of arrests.

The present study has several limitations. Data on periods of incarceration beyond treatment discharge were not available and thus outcomes were not adjusted to account for possible group differences in periods of time-at-risk (i.e., “time in the community”). More drug court offenders were incarcerated at treatment discharge and this difference most likely affects other outcome indicators that occur concurrently and also subsequently in time. This limitation, and its potentially biasing effects, poses an important area for further research. Second, the study utilized administrative data, a data source that can be vulnerable to over- or under-reporting of behaviors due to data linkage errors and missing data, among other factors (Duran, Wilson, & Carroll, 2005; McCarty, McGuire, Harwood, & Field, 1998; Saunders & Heflinger, 2004). Omitted from our analysis were offenders who were admitted to drug treatment during the time period of interest but who were missing drug treatment discharge data by the cut-off date (approximately 18 months after treatment admission) or DOJ criminal history data. In particular, offenders with extremely long lengths of treatment (e.g., some heroin users receiving methadone maintenance treatment) might have been excluded. However, administrative data represent a valuable resource for the evaluation of substance abuse treatment outcomes (Evans, Grella, Murphy, & Hser, 2010; McCarty et al., 1998) and use of this data allowed analyses on a large and unprecedented proportion of offenders admitted to treatment statewide under Prop 36 and drug courts. Third, success at treatment discharge may have differed with examination of other indicators, but the composite indicator was congruent with prior published work (Hser, Evans, Teruya, Huang, & Anglin, 2007) and, more important, it permitted constructive examination of how well offenders met all of the intended goals of these programs. Also, success was measured at treatment discharge, a relatively short time period for examining outcomes, but information on arrests over 12 months after treatment admission is also presented. Fourth, variation by region and by local courts in program operation, treatment practices, and supervision of offenders were not examined, nor were offender attitudes and perceptions that research has identified as being associated with treatment outcomes among drug offenders, such as levels of perceived coercion and treatment motivation (Farabee, Prendergast, & Anglin, 1998) and subjective stress levels (Garrity et al., 2008). Where feasible, possible confounders such as these should be included in future work that uses propensity score matching to compare outcomes between different mechanisms for routing offenders into drug treatment. Additional research is also needed to identify which type of offender is best served by Prop 36 versus drug court. Despite these limitations, this study provides new and valuable information on the comparative effectiveness of Prop 36 and drug courts.

Study results may be used by criminal justice policymakers and program planners to improve the design and evaluation of similar court-mandated substance abuse treatment programs. For example, outcome differences in longer-term arrest rates are likely attributable, in part, to differences in incarceration rates and accompanying variation in time-at-risk. For the present analysis, data on incarceration were not available beyond the 30 days prior to treatment admission and discharge. Additional research is needed to examine the extent to which inclusion of incarceration time affects longer term patterns of outcomes.

A second study implication is that both Prop 36 and the drug courts diverted drug offenders from jail and prison and into community-based treatment settings, and both programs reduced drug use, arrests, and incarcerations. Prop 36 has been criticized for being ill-equipped to handle offenders considered to have the most severe problems, yet findings suggest that for this type of offender, Prop 36 does just as well as drug courts at reducing short-term drug use and arrests. This finding may be a reflection of adjustments that were made to ameliorate problematic Prop 36 program elements as implementation proceeded. For example, after initial implementation efforts, funds were allocated to administer drug tests (California Department of Alcohol and Drug Programs, 2001), some counties altered processes to enhance treatment show rates by implementing “drug-court like” procedures, as recommended by Longshore and colleagues (2003), and judges found creative ways to exercise judicial sanctioning (including short incarceration stays) with noncompliant Prop 36 participants (Burns & Peyrot, 2008).

Despite such efforts to make Prop 36 more like drug courts, Prop 36 does not do as well as drug courts in reducing longer-term recidivism. The lower short-term incarceration rate among Prop 36 offenders and their higher longer-term recidivism rate are most likely attributable to persistent differences between the two programs. Drug courts continue to place an emphasis on holding noncompliant offenders accountable through increased supervision and incarceration, whereas Prop 36 prohibits the use of incarceration as a sanction, allowing instead the provision of additional or more intensive treatment. A brief incarceration stay for program noncompliance is thought by some stakeholders to be therapeutic, but its use as a rehabilitative sanction and its impact on offender behavior and program outcomes are poorly understood. Also, some offenders are likely to be career criminals who will continue to recidivate and engage in a range of drug- and non-drug-related crime (DeLisi, 2003; Durose & Mumola, 2002; Sullivan, McGloin, Pratt, & Piquero, 2006), despite receiving interventions to treat their substance abuse problems. These issues underscore the need for additional research to identify which program elements are associated with enhancing the effectiveness of court-supervised treatment.

A third study implication is that Prop 36 and drug courts operate on different scales, target different types of drug offenders, and provide different types and amounts of drug treatment. Many more offenders participate in Prop 36 and more of them exhibit characteristics that have been associated with increased likelihood of better treatment outcomes such as higher rates of employment at treatment entry (Hser, Evans, et al., 2007; Roll, Prendergast, Richardson, Burdon, & Ramirez, 2005). Fewer offenders are involved in drug courts and more drug court offenders receive treatment in a residential setting, a setting characterized as being more intensive and comprehensive (Grella et al., 2007). Prop 36 has been seen as a diversion program that has incorporated a public health approach into penal drug law (Appel et al., 2004; Klein et al., 2004), but there has been little dialogue regarding overall costs and benefits. Is it best to involve fewer but needier offenders in a system that provides intensive supervision and treatment or should criminal justice diversion programs seek to involve more offenders with less severe problems in a system that provides less intensive supervision and treatment? It may be possible and desirable to do both.

Some counties report that Prop 36 and drug court programming can be structured as a continuum such that the programs are complementary rather than competitive. Under this approach, Prop 36 is tried first, as a kind of “drug court lite” experience for offenders with a lower severity level, and only the more severe drug offenders, identified by program noncompliance, are moved into drug court. Doing so effectively “triages” access to services such that drug courts become a program on the more intense end of a treatment and supervision continuum. While this practice of rationing more intensive care and supervision to more severe offenders may enhance efficient use of limited resources in the short run, its potential effects on offender motivation for treatment and long-term treatment outcomes requires further examination.

Substance abuse continues to be a major contributor to rising incarceration rates (California Department of Corrections and Rehabilitation, 2009), and rehabilitation of offenders in lieu of continued incarceration is a priority in California, as evidenced not just by drug courts and Prop 36 but also as re-affirmed by recent legislative efforts to grant early release to inmates and route them, along with other types of offenders, to community-based substance abuse treatment (e.g., the Public Safety and Offender Rehabilitation Services Act of 2007; California Senate Bill X318 of 2010). A valuable lesson learned is that the debate over which program performs better, Prop 36 or drug courts, is not as meaningful as understanding the role that each program plays within California’s larger criminal justice system. A more constructive question is how to best integrate the range of available court-mandated programs to produce a continuum of care and supervision that is appropriately matched to offender risks and needs. Offenders may alter their drug use behaviors and criminal offending patterns over their life course (Hser, Evans, Huang, Brecht, & Li, 2008) and presumably can be exposed to a range of criminal justice diversion options (e.g., juvenile drug court, mental health court, DUI/DWI court, Prop 36, drug court). More research is needed to determine if better long-term outcomes result when drug offenders are better assessed and matched to the most appropriate drug diversion program according to their treatment and supervision needs (Marlowe, 2003; Taxman, Thanner, & Weisburd, 2006) and at the right time in the course of their criminal and drug-using careers.

Biographies

Elizabeth Evans, M.A., is a Project Director at the UCLA Integrated Substance Abuse Programs where she has contributed to ISAP’s statewide evaluation of California’s Proposition 36 since 2001. Currently, Ms. Evans directs a longitudinal follow-up study of substance-abusing mothers and their children and she contributes to the Center for Advancing Longitudinal Drug Abuse Research (CALDAR). Her interests include health services utilization and outcomes among substance-abusing offenders and longitudinal substance abuse research.

Libo Li, Ph.D., is a statistician at the UCLA Integrated Substance Abuse Programs. His work includes developing, evaluating, and applying advanced statistical methods for longitudinal drug abuse research.

Darren Urada, Ph.D., is currently working on California’s evaluation of Proposition 36. Previously he was the Principal Investigator on a project to advance research and cooperation in the Middle East, served as project director for the California State Treatment Needs Assessment Program, and directed a study on substance abuse and welfare reform. He has also contributed to the California Treatment Outcomes Project (CalTOP), meta-analytic studies on substance abuse and HIV/AIDS, and research on treatment expansion. He has worked for the UCLA Integrated Substance Abuse Programs since 1998.

M. Douglas Anglin, Ph.D., is an Associate Director of the UCLA Integrated Substance Abuse Programs. Dr. Anglin has been conducting research on substance abuse epidemiology, etiology, treatment evaluation, and social policy since 1972. He has been Principal Investigator on more than 25 federally funded studies and numerous state- and foundation-supported studies.