Decomposing Racial Disparities in Prison and Drug Treatment Commitments for Criminal Offenders in California

Abstract

Blacks convicted of drug-related offenses in the U.S. have higher prison-commitment rates than Whites. Studies have been largely unsuccessful in explaining these disparities. This study uses administrative data from a random sample of individuals arrested for drug offenses in California to examine this issue. We use a decomposition model to estimate whether Black-White disparities in commitments to prison or diversions to drug treatment are attributable to differences in the characteristics of criminal cases and whether case characteristics are weighed differently by race. We also examine whether the influence of case characteristics changes after California implemented Proposition 36, which was a mandatory prison diversion program for eligible drug offenders. Our results suggest that Black-White differences in prison commitments are fully explained by criminal case characteristics, but that a significant portion of the differences in treatment diversions remain unexplained. The unexplained variation in drug treatment also does not change after Proposition 36. These findings suggest that case characteristics play a larger role in explaining prison commitments for drug offenders than the discretion of prosecutors and judges. By contrast, diversion to drug treatment appears to be driven more by the discretion of court officials and Black-White disparities remain prominent.

1. INTRODUCTION

Historically, Black criminal offenders have been more likely to be incarcerated than their White counterparts. Some have argued that differences in incarceration rates for Blacks versus Whites in the modern context can be explained primarily by differences in their involvement in offenses that are more eligible for prison (Kleck 1981; Wilbanks 1987). However, substantive Black-White disparities have been documented in a number of aggregate and individual-level studies examining criminal justice outcomes, especially for drug offenses where there is ample room for discretion in determining whether someone convicted is eligible for prison (Sellin 1928; Blumstein 1982; 1993; Langan 1985; Tonry 1995; Mustard 2001; Bourassa and Andreescu 2009). The issue of racial disparities in sentencing for drug offenders is especially relevant in light of the tremendous rise in drug offending arrests that have occurred over the past 25 years (Raphael and Stoll 2009).

Leading scholarly commentators have noted that the racial disparities in incarceration rates for drug offenses may be connected to changes in the criminal justice policy environment that uniquely affected Blacks. The “get tough on crime” movement and associated sentencing reforms, which started in the 1970s and hit their peak in the 1990s (Tonry 1995), have been shown to be associated with an increasing Black-White disparity in prison rates (Western 2006; Tonry and Melewski 2008).

States, however, are starting to amend sentencing statutes to address concerns with prison overcrowding caused by the increasing use of prison for drug offenses. Until now, however, there has been little empirical examination of whether state policies that attempted to reduce the use of prison for drug-related offenses may reverse racial disparities in prison commitments. California’s Proposition 36, as well as similar legislative changes in other states such as Arizona, is one such example of a change in sentencing policy. Under Proposition 36 (hereafter, Prop36), which was implemented in July 2001, adult non-violent drug offenders charged with drug possession, drug use, or the transport of illicit drugs for personal use are required by statute to be sentenced to probation with drug treatment in lieu of incarceration.¹ By mandating that all non-violent drug offenders be offered the opportunity to obtain treatment in lieu of prison, Prop36 substantially reduced discretion in sentencing. To the extent that court dispositions to prison or diversion to drug treatment were determined with some degree of court discretion that unduly influenced Blacks, Prop36 could potentially reduce racial disparities in prison by standardizing diversion from prison for drug offenders. However, if racial disparities remain in criminal history background and contemporaneous charge decisions made by prosecutors for drug-related offenses, then such policy changes will likely yield little impact on the Black -White disparity in prison commitments.

Prior research points to a number of stages where there is evidence of racial disparities in criminal justice processing. At the same time, racial disparities in criminal court sanctions appear to shrink when one takes into account criminal history and current charge seriousness. While it is true that background features are not independent of racial bias, as police decide who to arrest and prosecutors decide what charges to file in court, it remains unclear the extent to which legally relevant factors are treated the same for Blacks relative to Whites. Importantly, we have little knowledge of whether legislative changes to sentencing statutes that emphasize diversion from prison for drug-related offenses has any impact on shrinking Black-White disparities in prison commitments (see Nicosia, MacDonald, and Arkes 2013).

In the present study, we improve on previous research that examines racial disparities in criminal justice outcomes for drug offenders by examining detailed data from California courts on prison admissions and drug treatment commitments. With these data we apply decomposition models to examine the extent to which case features related to charge seriousness and criminal history are weighed differently between Black and White males.² We also examine whether these Black-White disparities change under a new sentencing regime for drug offenders enacted under California’s Prop 36, which was purposively designed to reduce incarceration for all drug offenses.

This paper contributes to the research in three important ways. First and foremost, it provides a rigorous demonstration of the importance of criminal history in explaining Black-White disparities in prison commitment. With a much richer set of criminal history characteristics than is typically used in sentencing research, our decomposition analysis shows that the average Black-White disparity in commitments to prison for drug offenders is completely explained by current charge and criminal history characteristics, a finding which stands in contrast to recent examinations of federal criminal sentencing data that indicate disparities remain after conditioning on similar factors (see Sorensen, Sarnikar, and Oaxaca 2012; Starr and Rehavi 2012). Our findings suggest, at least within the context of California, that omitted variables are an important reason that the existing literature reports lingering effects of racial disparities in prison commitments for drug offenders. Second, we find that the observed Black-White disparity in diversion to drug treatment includes a large unexplained portion, even with our rich set of control variables. Third, we find that the passage of a California state law that took discretion away from prosecutors by mandating drug treatment rather than prison for nonviolent drug offenders had no consequence on Black-White disparities in diversions. These findings suggest that at a lower margin-of-punishment discretion among criminal justice actors plays a larger role in court dispositions to the disadvantage of Black drug offenders. Even when sentencing policies are shifted to encourage community drug treatment, Blacks remain underserved. This finding highlights the importance of considering multiple dispositions for criminal offenders when considering the importance of Black-White disparities in criminal court outcomes, as different margins of sentencing outcomes may show disparate impacts

While this paper focuses findings on just one other disposition targeted by the change in state law, diversion, other lower dispositions are also possible, including jail, probation and community service. The fact that we find prison commitments to be explained by our rich set of offender characteristics but not diversion suggests that disparities are likely to exist across these alternative dispositions as well.

2. PRIOR LITERATURE

A number of hypotheses have been put forth to explain why Blacks are more likely to be committed to prison after being charged with a criminal offense. One potential explanation is that Blacks engage in more serious crimes and have lengthier criminal histories. There is evidence that Black-White disparities in state and federal court dispositions to prison in the 1980s and 1990s are substantially reduced after one takes into account criminal history and the seriousness of arrest charges (Langan 1985; 2001; Klein, Petersilia, and Turner 1990). More recent studies using Pennsylvania (Harris, Steffensmeier, Ulmer, and David 2009) and federal sentencing data (see Sorensen, Sarnikar, and Oaxaca 2012; Starr and Rehavi 2012) from the 2000s reaffirm earlier work and suggests that differences in offense severity and criminal history continue to explain a substantial share of Black-White disparities in prison rates. A meta-analysis of research on racial disparities in criminal court dispositions finds that studies that controlled for offense severity or criminal history estimated lower racial disparities (Mitchell 2005).

While studies have attempted to address omitted variables bias by statistically controlling for the differences in criminal history and current charges between Blacks and Whites, it is important to acknowledge that including these legally relevant factors does not address whether Blacks and Whites would be treated the same if they had similar charges and criminal histories. There are a number of reasons to suspect that case characteristics related to the current offense and criminal history are weighed differently for Black and White defendants by the criminal justice system (e.g. district attorneys, judges, and juries) during the adjudication process (e.g. plea, conviction, or sentence). Blacks are more likely to live in urban counties, where courts may be less sympathetic to mistakes due to assembly-line justice and anonymity between prosecutors and defendants (Bibas 2012). Blacks may also have a lower quality of counsel, which could lead to a greater willingness to plead guilty and a greater likelihood of being incarcerated (Wilbanks 1987). There is some evidence indicating that Blacks are less likely than Whites to plead guilty (Petersilia 1985; Albonetti 1990), which typically leads to more severe sentences (Albonetti 1997). But, it is worth recognizing that plea bargaining happens in over 90% of felony convictions (Bibas 2012), suggesting that differences in guilty pleas alone are not likely to explain the bulk of racial disparities in prison commitments. One of the more rigorous studies on racial disparities in prison conducted by Mustard (2001) found that 55% of the Black-White difference in sentencing for federal offenders (for various crimes) is due to departures from federal guidelines. For drug-trafficking crimes specifically, Mustard finds that 66% of the racial difference comes from departures from federal guidelines, with Blacks being more likely than Whites to receive the few upward departures from guidelines and less likely to get the downward departures. When no prison time is an available option, Blacks are less likely than Whites to receive this option. The disparities also appear to be greatest for Black men and those in the lowest income bracket.³

A more recent examination of criminal cases in the federal system by Starr and Rehavi (2012) finds that case characteristics that include charge seriousness and criminal history account for almost 80% of the observed racial disparity in sentence lengths for non-drug related offenses. Despite the contribution of arrest offense and criminal history to explaining the bulk of the racial gap in prison sentences, Blacks appear to receive sentences that are on average 10% higher than Whites. Almost half of this difference appears to be explained by the fact federal prosecutors are almost twice as likely to file charges that carry a mandatory-minimum sentence if the defendant is Black. This paper also finds that mandatory-minimum sentence charges appear to account for a substantial share of the Black-White disparity in drug sentences, though missing data negates any firm conclusions.

There is substantial evidence that racial disparities in prison rates are greatest among drug offenders (Blumstein and Beck 1999, 2005; Mitchell 2005; Raphael and Stoll 2009). There are a variety of explanations why these disparities might occur, including differences in police deployment to minority neighborhoods and a focus on low-level drug arrests (Beckett et al. 2006; Blumstein 1993) and differences in charging decisions and sentencing outcomes conditional on criminal history (Bushway and Piehl 2001; Mustard 2001; Spohn and Fornango 2009). Law enforcement priorities and sentencing policies may cause minorities to accumulate lengthier criminal histories for drug-related offenses compared to Whites, especially if there is a focused effort at “cracking down” on particular drugs and their distribution.

There is some empirical support for these claims, including a study by Reuter and colleagues (2000) that showed increases in racial disparities in drug arrests in Maryland following a policy focus on cracking down on drug markets that was disproportionately concentrated in inner-city minority neighborhood police beats. Beckett et al. (2006) arrived at similar conclusions from a case study of law enforcement concentration in minority areas of Seattle, WA, which targeted crack cocaine possession and distribution. Ramchand et al. (2006) used data from the National Survey on Drug Use and Health to estimate a model that found Blacks were significantly more likely than Whites to report buying marijuana outdoors and from a stranger, even after controlling for a number of background factors. Scholars have also suggested that law enforcement priorities to crack down on open air drug markets were purposively designed to target Blacks (Bobo and Johnson 2004). These sort of strategic law enforcement activities, which disproportionately target the illegal activities of Blacks, may cause minorities to accumulate lengthier criminal histories of drug-related offenses compared to Whites, which coupled with changes in sentencing to make prison more mandatory for drug offenses may create greater Black-White disparities in prison commitments (Tonry and Meleweski 2008).

Research consistently finds that the size of the racial disparities in prison commitments is sensitive to the inclusion of characteristics such as current offense type and prior criminal history (Klein et al. 1990; Kramer and Steffensmeier 1993; Steffensmeier and Demuth 2000; Langan 2001; Harris, Steffensmeier, Ulmer and Davis 2009; Starr and Rehavi 2012). To properly identify Black-White disparities in prison commitments, one needs to appropriately account for the confounding effect that current case characteristics and criminal histories may have in determining prison dispositions. Even after holding constant criminal history and charge seriousness factors, several studies do show evidence of racial disparities in: (1) prosecutors’ decisions to bargain down charges (Spohn and Fornango 2009) or file mandatory-minimum charges (Starr and Rehavi 2012); (2) judges’ downward departures from prison time required under sentencing guidelines (Engen et al. 2003; Bushway and Piehl; 2001; Mustard 2001); and (3) the decision to incarcerate someone in jail or state prison (Bailey and Piquero 2012).⁴

There is ample empirical research supporting the hypothesis that tougher sentencing regimes, particularly increased penalties for drug violations, led to an increase in the number of Blacks sent to prison relative to Whites (Jacobs and Carmichael 2001; Bushway and Piehl 2001; Golembeski and Fullilove 2005; Western 2006; Tonry and Melewski 2008). A number of investigators have shown that the primary factors driving the increase in incarceration rates for Blacks between the 1970s and 1990s was the increase in prison commitments for drug offenses, as Black-White differences in violent offending remained fairly constant during this time frame (Tonry 1995). There appears to be a continuation of these trends in Black-White disparities in prison commitments for drug offenses in the recent decade (Raphael and Stoll 2009; Blumstein 2011).

However, we know of no studies that have examined how sentencing policies for drug offenders mandating diversion to treatment in lieu of prison influence the context of racial disparities in prison commitments when one compares Blacks to Whites in similar contexts.⁵ This is an important omission given that several key states, including California and Arizona, have passed mandatory diversion policies and numerous other states have well-established drug courts. The effect of these diversion policies, particularly mandatory diversion, has an unclear impact on observed racial disparities due to background differences that may influence eligibility in the programs. While racial disparities in prison in the United States are at the forefront of scholarly commentaries (Western 2006; Blumstein 2011), other studies have noted that unexplained disparities are also evident for non-custodial sanctions like probation (Petersilia 1985) and diversion to drug treatment among criminal offenders (Nicosia, MacDonald, and Arkes 2013), suggesting that careful consideration of some of these other criminal sanctions should also be considered.

Finally, research has not given enough attention to examining how Black-White disparities are driven by current charges or criminal histories. Instead, these legally relevant factors are typically included as control variables to remove their confounding effect. Only a handful of recent studies have specifically modeled what the outcomes from criminal courts would look like if Blacks had similar charges and criminal backgrounds as Whites. These studies have specifically focused on federal criminal cases (Sorensen, Sarnikar, and Oaxaca 2012; Starr and Rehavi 2012). We could identify only one small-scale study of a single county that examined this question for cocaine possession charges but it included only a limited number of variables on case characteristics (Bourassa and Andreescu 2009). As a result, we know little about how Black-White disparities in prison commitments for drug offenses would be similar or different if they had substantively similar case attributes in state court systems. This is an important omission as the bulk of growth in drug-related prison commitments in the United States occurs in state courts.

3. METHODS

3.1. Data

Our analyses focus on arrests made in the state of California. California provides an ideal setting for this research as it has a large population of drug arrestees, allowing us to rely on a random sample of cases. By focusing on a single state, we ensure that all arrestees are subject to the same laws at any given time. This is a key consideration given that aggregating offenders from different states (with different laws and proclivities to use prison) may overwhelm any within-state analysis of racial disparities (Crutchfield et al. 1994). Particularly important for our purposes, we have sufficient diversity to consider how Blacks and Whites differ in court dispositions to prison and to drug treatment for drug-related offenses. We purposively exclude Hispanics and the smaller group of Asians from this analysis in order to focus on the greatest disparities, which occur between Blacks and Whites (see Nicosia, MacDonald, and Arkes 2013). For example, the ratio of prison commitments to arrests among drug-related arrests in the period of 1995 to mid-2001 is 0.045 for Whites and 0.088 for Blacks, suggesting that Blacks are nearly twice as likely to be sentenced to prison for a drug charge relative to the share of arrests for drugs in the population. The observed disparities in imprisonment rates between Blacks and Whites in California are also consistent with other states. California is also an ideal study location because it allows us to examine the effects of implementing a potentially important policy change that mandated diversion from prison into community-based treatment for drug offenses.

We use administrative data from California’s Automated Criminal History System (ACHS), which provided us with a random sample of approximately 200,000 individuals who were arrested at any time between 1980 and 2009 on a drug-related offense. The ACHS system provides the complete criminal court history of each arrest made by police, including the arresting charges, court filings by prosecutors, and resulting court dispositions processed in the state. Information on individual race/ethnicity, age at arrest, gender, and the exact date for each criminal justice “event” (e.g. arrest, court charge, and court disposition) and location (county code) were extracted, from which we created an event-based dataset.⁶ For each of individual in this random sample we have their complete criminal justice history over their lifetime in California. The records contain approximately 1.4 million arrests for drug and non-drug offenses for these 200,000 individuals. The offense categorizations (criminal justice information system codes) include 611 different offenses that qualify as drug-related.⁷ Generally offense codes do not have indications of the amount of drugs, as the majority of health and safety codes of California do not specify the amount of drugs under possession or for intent to sell that distinguish between a felony and misdemeanor charge.⁸ Police officers can charge under broad statutes of felony or misdemeanor possession (e.g., possession of a controlled substance – felony; possession of a controlled substance – misdemeanor). Therefore, we categorized the exact offense codes (a total of 5,130 potential penal codes) for the current arrest charge into thirteen separate variables that capture meaningful variation in the seriousness of the arrest offense charges on the current case. We included count variables identifying number of felony and misdemeanor offenses by offense type (e.g., drug-related, property, violent, other offenses); as well as indicator flags for whether the offense had a weapons or sex charge. We also included a measure of the arrest cycle; which is a count of the number of arrests the person has accumulated up to and including the current arrest. We also constructed a complete criminal history for each arrestee, which was based on all events that occurred prior to the current arrest and included eleven separate measures. Our criminal history measures characterized previous arrests and dispositions as well as any warrants or other court violation qualifiers on an individual’s criminal record. Thus, we measure both the seriousness of the current charge, length and type of criminal history charges, and prior experiences in the criminal justice system that include prison stays and failures to obey court orders. We discuss the variables used in our analysis in detail in Section 3.3.

To generate a comparable sample of arrests, we narrowed our analysis sample to drug-related arrests among adult White and Black male offenders whose entire criminal record appeared to be confined to California.⁹ We define a drug-related arrest as an arrest that includes at least one drug-related offense (but may also contain other offenses). However, all recorded offenses and arrests were used to construct both the current arrest and criminal history characteristics. We also restricted our analysis to drug-related arrests that occurred between 1995 and 2005. This period restriction ensures that the only major change in sentencing regime was the implementation of Proposition 36 in 2001. Prior to this policy change, the most significant change was the 1994 implementation of the Three Strikes Law in California. The final analytic sample consisted of 97,507 total drug-related arrests (70,889 Whites and 25,618 Blacks) of which 42,980 were felony drug arrests (28,023 Whites and 14,957 Blacks).

From those arrests, we then created separate samples for non-Hispanic Whites and non-Hispanic Blacks. The first sample includes all drug-related arrests. In a second sample, we include only felony drug arrests that did not have a violent felony charge. This second sample has the advantage that it compares arrests among Blacks and Whites with more similar drug offenses and without violent felony charges, which are the prototype prison-eligible offenses that Prop36 intended to divert to drug treatment in lieu of prison.

3.2. Outcomes

The two outcomes analyzed in this study are: 1) whether the offender was committed to prison, and 2) whether the offender was diverted to drug treatment. These dispositions are the product of several steps in the judicial process, but we focus on the final court disposition outcome rather than the process because this has been the focus of the literature and because data for that stage are most available (e.g. not everyone has a court event).

3.3. Explanatory variables

The explanatory variables are grouped into legally relevant and other background characteristics. Legally relevant factors are measured by a rich array of twenty-six variables that capture current arrest characteristics, criminal history characteristics, and offense qualifiers. Arrest characteristics for the current charge include thirteen variables that include counts of the number of felony and number of misdemeanor charges, respectively, for drug- and alcohol-related offenses, violent-crime offenses, property-crime offenses, and other offenses; indicators (yes=1; no=0) for whether any of the offenses could be categorized as weapon- or sex-related misdemeanors and felonies; and indicators for whether this is the first, second, and so on arrest for the offender.¹⁰ It is important to emphasize that our charging categories refer to arrest charges and not charges filed with the court by prosecutors. Recovering the charge filed in court for each arrest event is complicated by the fact the prosecutors have the discretion to collapse arrests together into single court filings, or to simultaneously resolve multiple arrest charges in a single court charge. In addition, we wanted to guard against the known possibility that prosecutors may up-charge or down-charge an offense to negotiate a plea agreement, and the fact that these decisions may be confounded with the change in sentencing law imposed under Proposition 36. Thus, the actual arresting charge we feel is the closest administrative record of the actual offense that occurred (see Starr and Rehavi 2012).

With respect to criminal history characteristics, we include a set of eleven indicators that capture the extensiveness and nature of prior arrests. We created indicators for whether an individual had previously been arrested for drug, property, violent or other felonies. We also include whether the individual had any prior convictions, any prior prison sentence, and whether there were any custody-related events for drug, property, violent or other felonies. We also include an indicator for whether the individual had any juvenile arrests, but do not include the nature of those offenses because they are not reliably available in the data. Finally, we include a set of variables representing “qualifiers” to those arrests, such as whether the individual, when arrested, had an open warrant for his/her arrest or was on supervised probation and would be under violation of their court order. ¹¹ These are important indicators of whether someone failed to comply with court conditions before. With these measures, we capture both prior behavior and experience in the criminal justice system.

General background factors are measured to take into account the county location, time period of arrest (month and year), and age at the time of arrest of the defendant. These background factors allow us to control for location, age, and time differences between Blacks and Whites that may influence court dispositions to prison or diversions to drug treatment.

3.4. Empirical model

Our empirical model is based largely on the seminal work of Oaxaca (1973) and Blinder (1973), with the following two equations for Whites (w) and Blacks (b):

$$\overline{Y_w}={\beta }_{w,0}+{\sum }_{j=1}^n{\mathrm{X}}_{\mathrm{w},\mathrm{j}}{\widehat{\mathrm{\beta }}}_{\mathrm{w},\mathrm{j}}$$ (1)

$$\overline{Y_b}={\widehat{\beta }}_{b,0}+{\sum }_{j=1}^n{\mathrm{X}}_{\mathrm{b},\mathrm{j}}{\widehat{\mathrm{\beta }}}_{\mathrm{b},\mathrm{j}}$$ (2)

where Ȳ is the mean outcome for the particular racial group, n is the number of explanatory variables, β̂ is the vector of coefficients particular to each race, and X is the vector of the observed legally relevant and background characteristics described above. We can then write the difference in mean outcomes for the two racial/ethnic groups as:

$$\overline{Y_b}-\overline{Y_w}=({\widehat{\beta }}_{b,0}-{\widehat{\beta }}_{w,0})+{\sum }_{j=1}^n{\mathrm{X}}_{\mathrm{b},\mathrm{j}}({\widehat{\mathrm{\beta }}}_{\mathrm{b},\mathrm{j}}-{\widehat{\mathrm{\beta }}}_{\mathrm{w},\mathrm{j}})+{\sum }_{j=1}^n({\mathrm{X}}_{\mathrm{b},\mathrm{j}}-{\mathrm{X}}_{\mathrm{w},\mathrm{j}})({\widehat{\mathrm{\beta }}}_{\mathrm{b},\mathrm{j}})$$ (3)

The first two of the three terms on the right hand of side of Equation (3) represent the contribution to the racial disparity due to differences in the intercepts and coefficients between Blacks and Whites, or how different values for the X variables are weighted for Blacks versus Whites in determining the court disposition. These two terms are considered the “unexplained” portion of the racial difference. The third term on the right hand side of Equation (3) represents the contribution to the disparities from differences in the vector of observed X characteristics between Blacks and Whites. These latter differences are considered the “explained” portion of the racial difference. An important analytic contribution of the decomposition model we estimate is that we can calculate the contribution from Black-White differences in coefficients (unexplained contribution) versus differences in characteristics (explained contribution) in total, for each “set” of variables and for individual variables.

Given the dichotomous nature of the outcomes (i.e. commitment to prison or diversion to treatment) we could use Fairlie’s (2005) extension of the Oaxaca-Blinder decomposition for logistic regression models. This has an advantage over linear probability models when the underlying probability of the outcome is near 0 or 1 (when the cumulative distribution function is non-linear). On the other hand, logistic approximations cause problems in our model because there are a relatively small number of Blacks in many California counties, making the models unstable and unable to find a solution in maximum-likelihood. Thus, our primary models are based on linear probability models (LPM) and include all counties. But we do compare the results from the LPM to those from logistic regression models based on only the 15 counties in California with the largest number of Black drug-related arrests in our data over the analysis period.¹² The substantive conclusions are similar with both sets of models, so we focus our discussion on the LPM results.

We first report the magnitude and statistical significance of the total explained and unexplained contributions. Given that the “unexplained” contribution is hardly ever statistically significant, we focus our discussion on which sets of characteristics make the greatest contributions to the “explained” contribution. That is, we consider the contribution from differences between Blacks and Whites in the county of arrest, the time period when the arrest was made, their age, and more importantly, the sets of variables representing the current arrest charges and criminal history characteristics. Because many of our explanatory variables are often categorical, the results can be sensitive to the choice of excluded category unless categories are normalized. Consequently, we normalize the year, month, age and county categories.¹³

The models are estimated separately for the pre- and post-Proposition 36 periods in order to consider whether the contribution of these factors changed under the new policy regime that mandated diversion from prison to drug treatment in the community under probation. Such changes are critical to our understanding of the changes associated with the policy. From these models, it is not possible to observe what caused the prison and diversion rates to change from the pre- to post-Prop36 period within each racial group.

To gain some insight into this issue, we also estimated another set of decomposition models. Specifically, for Blacks and Whites separately, we decompose the differences in prison rates (diversion rates) from pre- to post-Prop36. In effect, we estimate a difference-in-differences approach that examines the shifts in prison (diversion) rates pre-Prop36 versus post-Prop 36 for Blacks and then for Whites.

3.5. Samples

Tables 1a and 1b show the unadjusted differences between Blacks and Whites in the explanatory variables for the various samples in the pre and post Prop 36 periods and their differences. First, among “all drug arrests,” Blacks have a higher number of drug-related felony counts (e.g., 0.777 vs. 0.590 during the pre-Prop36 period) and fewer drug-related misdemeanor counts (e.g., 0.454 vs. 0.769 during the pre-Prop36 period). But among “felony drug arrests,” the average felony counts between Whites and Blacks are more comparable. There is no clear pattern in the rest of the current arrest characteristics that differentiate Blacks from Whites (Table 1a). For some characteristics, Whites have higher counts (e.g., property felony counts) while Blacks have higher counts for other characteristics (e.g., violent felony counts and other felony counts). And, still other characteristics are very similar between Blacks and Whites, such as whether there was a sex felony charge. The patterns are more consistent for the criminal history characteristics (Table 1b). In every single Black-White comparison, Blacks had higher rates or counts than Whites in both the pre-Prop36 and post-Prop36 periods on criminal history factors. For example, while 29 to 34% of Black arrestees in either period had a prior prison sentence, the corresponding shares among White arrestees was only 10 to 16%. The important point to emphasize here is that criminal histories differ by race and time in meaningful ways that will likely influence prison or drug diversion commitments.

Table 1a.

Descriptive Statistics of Case Characteristics for Blacks and Whites

	All drug arrests				Felony drug arrests without a violent felony count
	Pre-Prop36		Post-Prop36		Pre-Prop36		Post-Prop36
	Black	White	Black	White	Black	White	Black	White
	12,205	31,911	13,413	39,978	7,858	14,912	7,099	13,111
Current Arrest characteristics
Drug/Alcohol Felony Count	0.777	0.590	0.664	0.443	1.175	1.233	1.219	1.280
Drug/Alcohol Misdemeanor Count	0.454	0.769	0.596	0.912	0.129	0.349	0.190	0.453
Violent Felony Count	0.030	0.023	0.036	0.023	0.000	0.000	0.000	0.000
Violent Misdemeanor Count	0.014	0.012	0.014	0.012	0.010	0.009	0.010	0.006
Property Felony Count	0.043	0.060	0.047	0.060	0.037	0.074	0.049	0.092
Property Misdemeanor Count	0.019	0.025	0.018	0.022	0.014	0.025	0.014	0.021
Other Felony Count	0.080	0.059	0.134	0.063	0.085	0.077	0.177	0.107
Other Misdemeanor Count	0.176	0.193	0.351	0.299	0.133	0.141	0.255	0.263
Weapons Felony Flag	0.018	0.020	0.020	0.016	0.022	0.029	0.027	0.029
Weapons Misdemeanor Flag	0.007	0.013	0.006	0.007	0.009	0.017	0.006	0.010
Sex Felony Flag	0.001	0.001	0.001	0.001	0.001	0.001	0.001	0.002
Sex Misdemeanor Flag	0.002	0.002	0.003	0.002	0.001	0.002	0.002	0.001
Arrest Cycle (#)	10.954	6.090	13.155	6.976	10.609	6.207	13.841	8.044

Table 1b.

Descriptive Statistics of Criminal History and Other Characteristics for Blacks and Whites

	All drug arrests				Felony drug arrests without a violent felony count
	Pre-Prop36		Post-Prop36		Pre-Prop36		Post-Prop36
	Black	White	Black	White	Black	White	Black	White
	12,205	31,911	13,413	39,978	7,858	14,912	7,099	13,111
Criminal history
Juvenile Arrest (in past)	0.264	0.089	0.311	0.132	0.304	0.092	0.356	0.143
Prior Arrest Drug Alcohol Felony	0.697	0.493	0.685	0.429	0.751	0.622	0.802	0.636
Prior Arrest Violent Felony	0.539	0.255	0.548	0.248	0.531	0.257	0.564	0.276
Prior Arrest Property Felony	0.567	0.352	0.546	0.316	0.567	0.382	0.593	0.402
Prior Arrest Other Felony	0.354	0.233	0.469	0.244	0.364	0.257	0.520	0.311
Prior Conviction	0.810	0.655	0.826	0.647	0.816	0.695	0.862	0.736
Prior Prison	0.295	0.112	0.297	0.107	0.290	0.131	0.336	0.160
Prior Custody Step for Drug/Alcohol	0.219	0.086	0.232	0.090	0.221	0.104	0.271	0.134
Prior Custody Step for Violent	0.113	0.026	0.122	0.028	0.108	0.025	0.131	0.032
Prior Custody Step for Property	0.154	0.079	0.167	0.073	0.142	0.082	0.187	0.103
Prior Custody Step for Other	0.059	0.025	0.066	0.027	0.058	0.027	0.073	0.038
Qualifiers
Warrant Qualifier	0.206	0.144	0.314	0.193	0.209	0.157	0.343	0.237
Violation Qualifiers	0.101	0.065	0.041	0.027	0.104	0.071	0.044	0.029

4. RESULTS

In this section, we discuss the results of our decomposition models. We start by modeling differences in the probability that Blacks receive a prison disposition relative to Whites and then look at how these differences change after the enactment of Prop 36. We then do the same for a second outcome, the probability that Blacks receive a disposition that involves drug treatment diversion (hereafter, diversion) relative to Whites. Although other dispositions are also possible, including jail, probation and community service, diversion is the specific outcome targeted by the state policy and thus expected to be the most likely impacted by it. All analyses are conducted on the full set of drug-related cases using LPM and using a logistic model for only the 15 selected counties with the largest number of black drug-related arrests. We then estimate a separate LPM model on the subsample of felony drug cases that did not have a violent felony charge on the arrest (hereafter, non-violent felony drug offenders). The last subsample is important because it was these offenders whom Prop36 was supposed to specifically divert from prison to drug treatment. The full set of results showing the share of explained and unexplained differences due to each individual independent variable for prison and diversion to treatment dispositions are not shown for ease of exposition. We provide only the summary results from the decomposition models.

4.1. Prison

4.1.1. Black-White Decomposition

Table 2 shows the results of the Black-White decomposition for prison dispositions. The explained and unexplained shares of the variances are shown for each grouping of variables (e.g. current arrest, criminal history, etc.). The first two columns in Table 2 show the Black-White decomposition for the pre-Prop36 period (Column 1) and the post-Prop36 period (Column 2) for all drug-related cases. The third and fourth columns show the corresponding results for the 15 selected counties. The fifth and sixth columns show the results for the subsample of those cases where the accused committed a felony but had no violent felony charges in their current arrest.

Table 2.

Decomposition of Prison Between Blacks and Whites Before and After Proposition 36

	All drug arrests (LPM)		15 select counties (Logit)		Felony drug arrests without a violent felony count (LPM)
	Pre-P36 (n=44,116)	Post-P36 (n=53,391)	Pre-P36 (n=34,477)	Post-P36 (n=39,992)	Pre-P36 (n=22,770)	Post-P36 (n=20,210)
White	0.045 (0.004)	0.024 (0.003)	0.045 (0.007)	0.025 (0.003)	0.082 (0.008)	0.054 (0.007)
Black	0.088 (0.017)	0.050 (0.009)	0.089 (0.022)	0.051 (0.013)	0.122 (0.027)	0.081 (0.015)
Difference	0.043** (0.014)	0.026** (0.008)	0.040* (0.016)	0.025* (0.013)	0.040+ (0.022)	0.026* (0.013)
Explained	0.043** (0.010)	0.027** (0.005)	0.039* (0.017)	0.025* (0.008)	0.036+ (0.019)	0.024* (0.010)
Unexplained	0.000 (0.008)	−0.001 (0.004)	0.001 (0.003)	0.000 (.005)	0.003 (0.008)	0.003 (0.005)
	Contribution to the disparity due to differences in X’s (Explained Difference)
Year	0.000+ (0.000)	0.000 (0.000)	0.006 (0.000)	−0.00 (0.00)	0.000 (0.001)	0.000 (0.000)
Month	0.000 (0.000)	0.000* (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)
County	−0.004 (0.008)	−0.004 (0.004)	−0.012 (0.011)	−0.006 (0.006)	−0.010 (0.015)	−0.009 (0.009)
Age groups	0.000** (0.000)	0.000 (0.000)	0.000 (0.000)	−0.000 (0.000)	0.000 (0.000)	0.000 (0.001)
Current arrest characteristicsa	0.008* (0.004)	0.006** (0.002)	0.017** (0.002)	0.008** (0.002)	−0.007* (0.003)	−0.008** (0.003)
Criminal history characteristicsb	0.042** (0.006)	0.027** (0.004)	0.037** (0.007)	−0.027** (0.005)	0.057** (0.009)	0.046** (0.006)
Qualifiersc	−0.003** (0.001)	−0.003** (0.001)	−0.003** (0.001)	−0.004** (0.001)	−0.003* (0.001)	−0.005** (0.001)

Note: Standard errors in parentheses are clustered at the county level.

LPM=linear probability model; Logit=logistic regression

^athirteen separate variables listed in Table 1

^beleven separate variables listed in Table 1

^ctwo separate variables listed in Table 1

⁺P < .10;

^*P < .05;

^**P < .01.

The first two rows show the unadjusted rate of prison dispositions among White and Black arrestees. Among all drug arrests, 8.8% of Blacks and 4.5% among of Whites in the pre-Prop36 period were sentenced to prison. As intended by law, the probability of prison for a drug-related offense was substantially lower after Prop36 was implemented. The rate at which drug-related cases were sentenced to prison post-Prop36 (Column 2) declined to 5.0% among Blacks and 2.4% among Whites, a 43% and 46% reduction. We see similarly significant declines post-Prop36 among non-violent felony drug cases for both Whites and Blacks (Columns 5 and 6). These reductions for Blacks were not significant improvements vis-à-vis Whites, as the proportion of prison dispositions declined by approximately one-third for both groups.

Despite the decline in prison dispositions after the implementation of Prop36, the magnitude of the differences in Table 2 shows there remains statistically significant differences between racial groups among all drug-related cases and among non-violent felony cases. While proportionally, Blacks and Whites had similar reductions in prison disposition rates, the absolute difference decreased from 4.3 percentage points to 2.6 percentage points after Prop36 but remained statistically significant. Similarly, we see in Columns 5 and 6 (for non-violent felony cases) that the magnitude of the unadjusted difference between Black and White prison commitments is reduced from 4.0 to 2.6 percentage points between the pre- and post-Prop36 periods. These results indicate that the residual share of the Black-White differences in probability of receiving a prison disposition from the courts shrank after the passage of Prop36, suggesting that the law had some effect on reducing unexplained racial disparities.

As discussed in our review of the literature, previous studies have only been able to explain 26–50% of the racial disparity in prison rates for drug offenses. The fourth and fifth rows of Table 2 shows that, with our richer set of covariates, our model attributes the entire estimated difference between groups to differences in observable characteristics (i.e. the “explained” component). Among all drug arrests in both periods (columns 1 and 2), Table 2 (row 4) shows that the “explained” differences (4.3 percentage pre-Prop36; 2.7 percentage post-Prop36) are basically identical to the unadjusted differences, indicating that we are fully explaining the residual racial disparities in prison dispositions for Blacks relative to Whites. This explained contribution is statistically significant and is constructed as the product of the coefficients from the pooled model multiplied by the difference in the observed characteristics between Blacks and Whites. In other words, if Whites were to have the same current arrest and criminal history characteristics as Blacks in this sample we would expect a similar probability of prison disposition for the average case. The unexplained contribution from the models in Table 2 (row 5) is small and not statistically different from zero in either period (0.0 and −0.1 percentage points). The model does similarly well for describing the average Black-White differences in the probability of a prison disposition for non-violent felony drug arrests (columns 5 and 6). Again, the explained variation from the model based on average Black-White differences in prison dispositions taking into account case characteristics are nearly the same magnitude as the unadjusted differences. As in the model for all drug arrests, the unexplained variation is statistically indistinguishable from zero (0.3 percentage points for both).

The second panel of Table 2 shows contributions from each set of observable characteristics to the “explained” portion of the decomposition. The set of thirteen criminal history characteristics makes the largest contribution to the explained share of the average higher probability of prison for Blacks compared to Whites for all drug arrestees both before and after Prop36 (4.2 percentage points and 2.7 percentage points, respectively, p < 0.01 for both). The share of the variation between average Black and White cases in the probability of prison explained by criminal history factors is nearly equal to the entire unadjusted difference in each period. The next largest contribution comes from Black-White differences in the current arrest characteristics in both periods: 0.8 percentage points (p < 0.05) and 0.6 percentage points (p < 0.01) respectively. The current arrest characteristics, however, explain a small share relative to criminal history factors. The greater contribution of criminal history makes intuitive sense, as criminal background is a key factor in whether someone is prison eligible on a current charge under California Penal Codes. Qualifiers to the current offense charge, like a warrant, make smaller though still significant off-setting adjustments to the explained contribution (−0.3 percentage points for both). The negative value for qualifiers indicates that for Whites qualifiers contribute more to a greater average probability of prison than for Blacks.

The decomposition results for the non-violent felony drug offenders shown in columns 5 and 6 are generally similar in nature to those for all drug-related offenders with minor modifications. Racial differences in the criminal history characteristics remain the single most important set of explanatory characteristics, explaining 5.7 and 4.6 percentage points (p < 0.01 for both) of the racial disparity in prison dispositions for the average case before and after Prop36. In this case, however, they explain more than the unadjusted average differences and are offset by the negative contributions from racial differences in current arrest characteristics and qualifiers. That is, criminal history characteristics contribute to the higher average probability of a prison disposition among Blacks relative to Whites, but that for the average White case, current arrest characteristics and qualifiers contribute more to the probability of prison. These results suggest that, based on criminal history alone, there should be a greater Black-White disparities in prison dispositions than is actually observed, but this is counterbalanced by the contribution that current arrest characteristics and qualifiers have in increasing the probability of prison for Whites relative to Blacks charged with a felony non-violent drug offense.

4.1.2. Within-Race Pre-Post Decomposition

Table 3 considers the influence of observable characteristics, including the effect of Prop36, from a slightly different perspective. Here, we assess what contributes to the decline in prison commitments across time periods within each race. This analysis allows us to examine whether the contributors to the decline are similar for each group. Similar to the Black-White comparisons presented, only the explained contributions are significant among both Black drug-related offenders and felony offenders (−3.5 percentage points; −3.7 percentage points) and their White counterparts (−1.8 percentage points; −3.0 percentage points). The vast majority of the downward shift in the probability of prison for the average case after the implementation of Prop36 can be explained by differences in the observable factors, while the unexplained share of the variance remains small and statistically insignificant for Blacks and Whites.

Table 3.

Decomposition of Prison Before and After Proposition 36 for Blacks and Whites Separately

	All arrests (LPM)		Felonies without a violent felony count (LPM)
	Black (n=25,618)	White (n=71,889)	Black (n=14,957)	White (n=28,033
Pre-Prop36	0.088 (0.017)	0.045 (0.004)	0.122 (0.027)	0.082 (0.008)
Post-Prop36	0.050 (0.009)	0.024 (0.003)	0.081 (0.015)	0.054 (0.007)
Difference	−0.038** (0.009)	−0.021** (0.003)	−0.041** (0.014)	−0.028** (0.006)
Explained	−0.035** (0.008)	−0.018** (0.004)	−0.037* (0.015)	−0.030** (0.009)
Unexplained	−0.003 (0.011)	−0.002 (0.005)	−0.004 (0.018)	0.002 (0.010)
Year	−0.032** (0.008)	−0.011** (0.003)	−0.052** (0.014)	−0.038** (0.008)
Month	0.000 (0.000)	0.000* (0.000)	0.000 (0.000)	0.000 (0.000)
County	0.002 (0.002)	0.001+ (0.001)	0.003 (0.002)	0.002+ (0.001)
Age groups	−0.001 (0.001)	0.000 (0.000)	−0.003* (0.001)	−0.001 (0.001)
Current arrest characteristicsa	−0.005** (0.002)	−0.006** (0.001)	0.004* (0.002)	0.002 (0.001)
Criminal history characteristicsb	0.002 (0.001)	−0.001 (0.001)	0.016** (0.003)	0.009** (0.003)
Qualifiersc	−0.001 (0.002)	−0.001* (0.001)	−0.004 (0.002)	−0.005** (0.002)

Note: Standard errors in parentheses are clustered at the county level.

LPM=linear probability model

^athirteen separate variables listed in Table 1

^beleven separate variables listed in Table 1

^ctwo separate variables listed in Table 1

⁺P < .10;

^*P < .05;

^**P < .01.

This decomposition does a better job of highlighting the extent to which differences in some of our observable controls, such as current arrest or criminal history characteristics, are the primary factors driving the reduction in prison commitments or whether it is the policy change itself. In the case of all drug arrests among Blacks (Column 1), we see that the reduction in prison commitments pre and post-Prop36 (−3.8 percentage points) is largely explained by the year dummy variables included in the model (−3.2 percentage points, p < 0.01) rather than changes in criminal history characteristics, age groups, county effects, or qualifiers. For Whites, the year dummy variables contribute to over one-half (−1.1 percentage points, p < 0.01) of the difference between pre- and post-Prop36 (−2.1 percentage points). The year dummy variables (fixed effects) will capture the effects of the implementation of the drug courts (e.g., DCPA) and Prop36. This suggests that these programs were the primary driver of reductions in incarceration rates for both Blacks and Whites. However, it is also possible that the year effects capture, to some extent, reduced incarceration rates for both Blacks and Whites over time due to changing factors that would reduce the probability of incarceration (such as an aging sample and being less likely to commit felonies as they get older). What also contributed to the reduction in prison for both Blacks and Whites was current arrest characteristics being weighed less heavily in prison commitment dispositions.

When we look at explanations for declines in prison commitments among those arrested for a felony drug offenses (Columns 5 and 6 of Table 3), we see that more of the observable factors are relevant for explaining the declines among both Blacks and Whites, although the year effects again dominate these other factors, suggesting that there are secular trends dominating the changes for everyone in the sample. Indeed, if changes in the other observables characteristics had not occurred, there would have been an even larger reduction in commitments to prison for both Blacks and Whites, as indicated by the fact that the year effects (−5.2 percentage points, −3.8 percentage points, p < 0.01 for both) are larger than the unadjusted differences (−4.1 percentage points; −2.8 percentage points). Specifically, changes in criminal history characteristics contributed to a greater share of the within-race changes in average probability of prison commitment for both Blacks and Whites. These findings make sense given that Prop36 specifically disqualified individuals from prison if they had no violent criminal history and were simply repeat drug and property involved offenders.

There are a few salient points from these analyses on incarceration. First, whereas prior studies had only been able to explain about a one-quarter to one-half of the Black-White difference in incarceration rates for drug arrests, the more detailed information we have on current arrest and criminal history characteristics fully explains the Black-White residual variation in the probability prison commitments for the average case with these current models. Second, there does not appear to be any clear shift in unexplained discretion in the use of prison for the average case with the implementation of Prop36, as there is no increase in unexplained share of the variation between periods for Blacks that looks different than that that occurred for Whites. Still, this analysis does not rule out the existence of judge or prosecutor discretion contributing to the racial disparity in incarceration rates, as our model may miss aspects of charge bargaining between prosecutors and defendants that influence racial disparities in outcomes. However, by relying on the arresting charge rather than the formal charge filed by the prosecutor, our analysis is protected from any influence that the change in sentencing under Prop36 would have on charges filed by prosecutors.

4.2. Diversion to Treatment

4.2.1. Black-White Decomposition

The results thus far suggest that observable characteristics explain a large share of the unadjusted differences between racial groups with respect to prison dispositions. Our data and the change ushered in under Prop36 also allow us to consider racial disparities and their change with regard to diversion to drug treatment. This is an important outcome if diversion in lieu of incarceration changes the likelihood of future drug use and criminal involvement.

Table 4 replicates our previous analysis in Table 2 with the diversion outcome. For Black males approximately 4.0% of all drug-related cases and 5.4% among non-violent felony cases were diverted to treatment in the pre-Prop36 period. This is substantially lower than that observed for Whites, for which 7.9% for all drug-related cases and 12.1% of non-violent felony cases were given diversion. By mandating drug treatment for eligible offenders, Prop36 substantially reduced the absolute magnitude of the differences between Blacks and Whites from 3.9 percentage points to 2.1 percentage points among all drug-related cases and from 6.7 percentage points to 5.9 percentage points among non-violent felony cases. After Prop36, White offenders are slightly less likely to be diverted to treatment (−1.5 percentage points), while their Black counterparts are more likely to be diverted (+0.3 percentage points). The same pattern holds among felony offenders where Whites are less likely to be diverted to treatment (−0.6 percentage points) and Blacks are more likely to be diverted (+1.4 percentage points) after Prop36.

Table 4.

Decomposition of Diversion to Treatment Between Blacks and Whites Before and After Proposition 36

DIVERSION	All drug arrests (LPM)		15 select counties (Logit)		Felony drug arrests without a violent felony count (LPM)
	Pre-P36 (n=44,116)	Post-P36 (n=53,391)	Pre-P36 (n=34,442)	Post-P36 (n=40,089)	Pre-P36 (n=22,770)	Post-P36 (n=20,210)
White	0.079 (0.006)	0.064 (0.006)	0.081 (0.009)	0.064 (0.010)	0.121 (0.009)	0.127 (0.018)
Black	0.040 (0.003)	0.043 (0.007)	0.040 (0.004)	0.043 (0.010)	0.054 (0.004)	0.068 (0.014)
Difference	−0.039** (0.005)	−0.021** (0.006)	−0.041** (0.007)	−0.021** (0.007)	−0.067** (0.009)	−0.059** (0.012)
Explained	−0.023** (0.004)	−0.007 (0.005)	−0.028** (0.006)	−0.008 (0.006)	−0.048** (0.006)	−0.040** (0.013)
unexplained	−0.016** (0.005)	−0.015** (0.003)	−0.012** (0.004)	−0.012** (0.005)	−0.019* (0.008)	−0.019* (0.009)
	Contribution to the disparity due to differences in X’s (Explained Difference)
Year	0.001 (0.001)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	−0.001 (0.001)	0.000 (0.000)
Month	0.000 (0.000)	0.000 (0.000)	−0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)
County	0.001 (0.003)	−0.002 (0.005)	−0.000 (0.002)	−0.001 (0.004)	0.005 (0.006)	0.003 (0.011)
Age groups	0.001 (0.001)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	−0.003* (0.002)	0.000 (0.001)
Current arrest characteristicsa	0.005* (0.002)	0.006+ (0.003)	−0.009** (0.002)	0.001 (0.003)	−0.009** (0.002)	−0.011** (0.003)
Criminal history characteristicsb	−0.027** (0.003)	−0.006** (0.002)	−0.016** (0.003)	−0.004* (0.002)	−0.036** (0.004)	−0.025** (0.005)
Qualifiersc	−0.003** (0.000)	−0.005** (0.001)	−0.004** (0.001)	−0.005* (0.001)	−0.003** (0.001)	−0.007** (0.002)

Note: Standard errors in parentheses are clustered at the county level.

LPM=linear probability model; Logit=logistic regression model

^athirteen separate variables listed in Table 1

^beleven separate variables listed in Table 1

^ctwo separate variables listed in Table 1

⁺P < .10;

^*P < .05;

^**P < .01.

Among all drug-related offenders, the majority of the difference between Blacks and Whites in the pre-Prop36 period is explained by differences in characteristics (2.3 percentage points). As shown in the bottom part of Table 4, the largest contributor to the lower diversion rate for Blacks are worse criminal history characteristics in the pre-Prop36 period. Current arrest characteristics represent the next largest contributor, but their contribution offsets that of the criminal history characteristics for all drug arrests. After Prop36, the explained portion of residual variation in the average probability of diversion is not statistically significant largely due to the offsetting effect that criminal history characteristics (−0.6 percentage points, p < 0.01) has on the contribution of current arrest characteristics (+0.6 percentage points, p < 0.10). Stated simply, in the post-Prop36 period, Blacks with criminal histories are less likely than Whites to get diversion whereas Whites with similar current cases are more likely to get diversion.

Unlike the results for prison commitments, a statistically significant component of the difference between Black and White diversion rates remains unexplained in both the pre-Prop36 period and the post period (−1.6 percentage points and −1.5 percentage points, respectively, p<0.01). Thus, there is some evidence of a racial disparity in diversion to treatment that is not explained by observable characteristics, but rather by differences in how Blacks and Whites with the same observed characteristics are adjudicated by the courts. There appears to be some evidence that Blacks are being treated less favorably for diversion even after Prop36 expanded the requirements and availability for diversion among those charged with a drug offense.

Results for the 15 selected counties and those for the sample of felony offenders demonstrate a similar pattern to what we observed for all drug arrests in Table 4. Most of the unadjusted differences shown in the third row of columns 5 and 6 for felony offenders, for example, are explained primarily by observed criminal history case characteristics. But, there remains a statistically significant share of this difference that is unexplained in both periods.

How it is possible that our rich set of offender and case characteristics completely explain Black-White differentials for prison commitments and not diversion to treatment? Diversion is but one of several alternative dispositions that could be considered in lieu of prison for these offenders. Other dispositions that might also be considered include short jail stays, probation, and community service (many of which are not mutually exclusive). Our results suggest that while Black-White differences in incarceration can be explained by observable case and offender characteristics, other factors seem to explain decisions regarding adjudication of offenders to the primary aim of diversion to drug treatment as an alternative to prison. As diversion to treatment was the specific target of Prop36, and because treatment programs have been shown to reduce recidivism of participants in the future, we were particularly interested in this specific alternative disposition. Our findings clearly show that future work needs to give greater attention to the entire range of alternative dispositions so as to better understand the extent to which Blacks are differentially treated in the criminal justice system.

4.2.2. Within-Race Pre-Post Decomposition

Table 5 shows the within-race decomposition across time periods. While the diversion rates among all arrestees appears to slightly increase for Blacks while they slightly decrease for Whites after the implementation of Prop36, only the decrease among Whites is statistically significant. There is a within-race reductions in diversion among Whites (−1.5 percentage points, p < 0.01), meaning that it is getting more difficult for White males with prior criminal histories to receive diversion to drug treatment after Prop36 compared to White male counterparts before the passage of this new sentencing reform. But these changes are very small in magnitude. Furthermore, they may reflect shifts, for White offenders, to other dispositions, such as probation and community service. These within-race differences in unadjusted rates between periods are quite small making it difficult to identify whether the “explained” or “unexplained” contributions are statistically significant. Indeed, only the models for White drug-related offenders (−2.6 percentage points, p<0.01) identify statistically significant explained contributions and none of the models identifies statistical significance for the unexplained shares.

Table 5.

Decomposition of Drug Treatment Before and After Proposition 36 for Blacks and Whites Separately

	All arrests (LPM)		Felonies without a violent felony count (LPM)
	Black (n=25,618)	White (n=71,889)	Black (n=14,957)	White (n=28,033
Pre	0.040 (0.003)	0.079 (0.006)	0.054 (0.004)	0.121 (0.009)
Post	0.043 (0.008)	0.064 (0.006)	0.068 (0.014)	0.127 (0.018)
Difference	0.003 (0.007)	−0.015** (0.004)	0.013 (0.013)	0.005 (0.012)
Explained	−0.005 (0.013)	−0.026** (0.009)	−0.002 (0.021)	−0.019 (0.023)
Unexplained	0.008 (0.014)	0.011 (0.010)	0.015 (0.025)	0.024 (0.026)
Year	0.001 (0.013)	−0.017+ (0.009)	0.009 (0.021)	0.005 (0.023)
Month	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)	0.000 (0.000)
County	0.000 (0.001)	0.000 (0.001)	−0.001 (0.002)	−0.009* (0.004)
Age groups	0.001* (0.001)	−0.002** (0.000)	0.004** (0.001)	−0.003* (0.001)
Current arrest characteristicsa	−0.006** (0.001)	−0.008** (0.002)	−0.004* (0.001)	−0.003+ (0.002)
Criminal history characteristicsb	0.000 (0.001)	0.003** (0.001)	−0.007** (0.001)	−0.005* (0.003)
Qualifiersc	−0.002 (0.001)	−0.001 (0.001)	−0.004* (0.002)	−0.004+ (0.002)

Note: Standard errors in parentheses are clustered at the county level.

LPM=linear probability model; Logit=logistic regression

^athirteen separate variables listed in Table 1

^beleven separate variables listed in Table 1

^ctwo separate variables listed in Table 1

⁺P < .10;

^*P < .05;

^**P < .01.

The overall pattern of the within-race changes before and after Prop36 suggests that there is only a small relative change for Blacks and Whites with similar case characteristics. These findings also suggest that the changes observed in the between-race comparisons for Blacks and Whites on the average probability of diversion before and after the implementation of Prop36 were not driven by compositional changes that affected one group more than the other with the switch in sentencing law.

5. DISCUSSION

This study provides very detailed individual-level information on the adjudication process for drug offenders in California. Whereas previous studies on drug offenders were only able to explain 26–50% of the Black-White disparity in incarceration rates, we are able to fully explain the average Black-White probability in receiving a prison commitment based on observable characteristics. For the average probability of being diverted to drug treatment, which is often used as an alternative to incarceration among drug offenders without significant criminal histories, a statistically significant component of the Black-White difference (ranging from 28 to 68%) remains unexplained. The large unexplained share of the average Black-White difference in diversion may also be due to the ambiguous nature of how courts determine whether someone is in “need of treatment” in lieu of prison. On the other hand, the shift to mandatory diversion under Prop36 should have reduced the unexplained share of the Black-White difference in the average probability of diversion to treatment if there was a mechanical effect of the law. After all, this would have automatically qualified a large share of drug offenders for treatment. While it appears that indeed a greater share of Blacks and Whites received a diversion disposition after the implementation of Prop36, this law did not materially shrink the share of unexplained variation. It is uncertain why this is the case. But, one possible explanation is that, with the shift away from incarceration, relatively more Whites than Blacks received other non-diversion dispositions such as probation and community service. One could speculate that Prop36, by opening up greater access to treatment, may have actually benefited Whites who would have had their cases dropped in the past or diverted in some other reason. It is also possible that other characteristics beyond those used to model dispositions to prison may be relevant for diversion to treatment in lieu of jail or probation.

The probability of a prison commitment for drug offenders decreased substantively after Prop36 for both Blacks and Whites, but there was no evidence that Prop36 reduced unexplained racial disparities. Thus, the law appears to have shrunk Black-White disparities in this sample based on observable case characteristics. This suggests that the law did in fact reduce prison use for those qualified for other outcomes, regardless of race. For diversion to drug treatment, the average Black-White probabilities converged from the pre- to post-Prop36 period due in part to a reduction in diversion rates for White offenders relative to Blacks, not greater access for Blacks. The contribution to the disparities that remained unexplained was statistically significant and remained the same from pre- to post-Prop36. Thus, again there was no evidence that Prop36 had any effect on reducing unexplained share of the differences between Blacks and Whites in the probability of receiving a drug treatment diversion disposition from the courts. It is worth noting that the exclusion criteria for eligibility to Prop36 may have made it too restrictive to expand access to a greater share of Black offenders in this sample, given their average lengthier criminal histories. It is possible that racial disparities in drug diversions may have declined more if the criteria for Prop36 did not exclude individuals with prior violent convictions or concurrent felony charges on non-drug related offenses.

The finding that racial differences in prison commitments are fully explained by observable characteristics does not mean that there is no discrimination in the adjudication process. Average Black-White differences in criminal history may be a byproduct of earlier discrimination or other policies that inadvertently target Blacks. Our work cannot speak to that. Furthermore, it is clear from this study that unexplained racial disparities remain in criminal dispositions to drug treatment, which warrants careful examination and further consideration in future work seeking to explain why some individuals receive community-based sanctions in lieu of prison.