Assessing the Long-Term Impact of Drug Court Participation on Recidivism with Generalized Estimating Equations

Abstract

Drug courts are one of the most common strategies for dealing with the large proportion of criminal offenders who are drug-involved, yet methodological limitations limit the conclusions that can be drawn from many existing evaluations of their effectiveness. The current study examined the long-term impact of drug court participation compared to regular probation on the recidivism of 475 drug-involved offenders under supervision in Hillsborough County, Florida. Using a combination of self-reported data (collected through in-person interviews at baseline, i.e., the beginning of supervision) and administrative records, the study employed a repeated measures framework (examining five six-month time periods from baseline to 30 months post-baseline) and generalized estimating equations to compare the likelihood of being arrested between drug court participants and a matched sample of comparison offenders. The results indicate that participation in drug court was associated with a significant decrease in the likelihood of being arrested in the 12 to 18 months post-baseline time period. Although the drug court effect was somewhat delayed (it was not significant prior to 12 months) and short-lived (it was not significant after 18 months), the fact that significant program effects were observed during a time period that coincides with the conclusion of drug court participation for graduates and a time period well beyond initial program exposure, suggests that drug court participants are more likely than comparable offenders not exposed to drug court to remain arrest free when no longer under community supervision.

1.0 Introduction

In 2003, there were nearly 7 million criminal offenders under correctional supervision in the United States, and almost 4.8 million of them were being supervised in the community (Glaze, 2004). A large proportion of these offenders are drug-involved. The Arrestee Drug Abuse Monitoring (ADAM) Program reported that over 70 percent of adult males arrested in 2003 tested positive for at least one illicit drug (Zhang, 2003).

The high prevalence of drug use among criminal offenders and the increase in the proportion of offenders who are drug-involved have been well-documented—24% of the 4 million adults on probation have a drug law violation (Bureau of Justice Statistics (BJS), 2003a) and approximately 20% of state prisoners and 55% of Federal inmates are being held for drug offenses (BJS, 2003b). Policy makers have responded to the large numbers of drug-involved offenders who end up in and cycle through the criminal justice system, sometimes multiple times, with a variety of strategies intended to curb substance use and criminality. These strategies have included self-help groups and therapeutic communities, TASC (Treatment Alternatives to Street Crimes, also known as Treatment Approaches for Safer Communities) models, and specialized caseloads for supervising drug-involved offenders (e.g., drug-offender probation).

Another common approach to dealing with the threats that these offenders potentially pose to public safety and public health involves providing drug-involved offenders with court-supervised treatment as an alternative to traditional criminal sanctions. This approach is typically packaged in the form of a drug court program that utilizes a judge and other court officials in a non-adversarial environment to sanction, supervise, and provide alcohol and drug treatment services to offenders. In exchange for the possibility of dismissed charges or reduced sentences, defendants are diverted (from the traditional justice system process) to the drug court program in which they agree to participate in judicially monitored substance abuse treatment. The model for drug courts assumes that a combination of judicial monitoring and supervised treatment can be more effective in reducing drug use and criminality than treatment or judicial sanctions operating alone or independently. As of April 2006, there were over 1,557 drug court programs in operation and another 394 being planned (Office of National Drug Control Policy, 2006) in the United States. The drug court model is also being used or is planned in numerous other countries, including Canada, Australia, Jamaica, Ireland, Brazil, Cayman Islands, Bermuda, Trinidad, Barbados, New Zealand, Scotland, Norway, Italy, and Macedonia (International Association of Drug Treatment Courts, 2006). Given the prevalence of drug courts in the United States and abroad, it is important that policy and operational decision makers are provided with the information they need to evaluate drug courts for efficacy, particularly with respect to long-term effects.

This study examines the long-term impact of drug court participation (compared to regular probation) on the recidivism of drug-involved offenders under supervision in Hillsborough County (Tampa), Florida. We recruited 475 drug-involved offenders, 274 of whom (58%) were enrolled in drug court, to participate in the impact evaluation study. Study participants were interviewed at baseline and 9-months following baseline¹, and administrative data that provide information on the participants prior to and 30 months after the arrest that made them eligible to participate in the study were obtained. As is often the case, the selection process for drug court and other treatment interventions often results in observable and unobservable differences between those who are offered drug court and those who are not offered drug court, as well as those who accept and do not accept the offer to participate. Many evaluations ignore the selection process and its methodological implications, which makes it difficult to attribute impact to the actual intervention an not the differences between those receiving and not receiving treatment (Manski, Pepper, and Petrie, 2001; Horowitz, MacCoun, and Manski, 2002). Although ignoring these issues makes for a simpler and ‘cleaner’ study, we decided to employ methods that would take into account the process of selection and enable the most rigorous evaluation possible. Our analytic approach helps control for biases associated with differences between those who did and those who did not enroll in drug court.

In order to examine the impact of drug court participation on recidivism among our sample of offenders, we utilize longitudinal marginal (population-averaged) models using Generalized Estimating Equations (GEE). We utilize these models instead of generalized linear mixed effects (subject-specific) models for two reasons. First, marginal GEE models describe how the average rates of arrests (in odds) would increase or decrease in the study population if offenders were in drug court, whereas the estimated treatment effect from mixed effects models would describe how the odds of arrest increase for any individual offender in drug court (Fitzmaurice, et al. 2004). Our intent is to calculate population-averaged effects of drug court on recidivism as opposed to subject specific because the former yields more generalizable and policy-relevant findings. Secondly, even though marginal interpretations can be made from subject-specific models, we are utilizing a non-linear link function, and thus it is not possible to average random effects over individuals (from subject-specific models) to accurately make marginal population interpretations (Fitzmaurice, et al 2004).

The next section provides a brief summary of the literature on the impact of drug courts on recidivism. Subsequently, we provide a description of the Hillsborough drug court, our study methodology, our sample, and our results. In the final section, we discuss our findings and their policy relevance. Generally, our results suggest that participating in drug court reduces the likelihood of recidivism, after controlling for selection and maturation bias, although the reduction is not significant until 12 months after entry into drug court.

1.1 Previous Research

Drug court programs vary in their specific policies and practices because of, among other things, differences in local jurisdictions and criminal justice system practices, but they typically share several general characteristics. Generally, judges preside over drug court sessions (often called status hearings), and in collaboration with prosecutors, defense attorneys, and treatment providers, monitor defendants' progress with drug testing, and impose sanctions and rewards. Drug court programs vary in terms of the duration, type, and intensity of substance abuse treatment that is required; however, most programs offer a range of treatment options and require 12 or more months of participation before a defendant can graduate from the program.

Although not all drug court evaluations report reductions in recidivism for drug court participants (Meithe et al., 2000; Wolfe et al., 2002), a summary of drug court evaluation studies conducted between 1999 and April 2001 (Belenko, 2001) identified several studies reporting significantly lower recidivism rates among drug court participants than comparison offenders (Brewster, 2001; Deschenes et al., 2001; Truitt et al., 2000; Gottfredson & Exum, 2002). In addition, the United States Government Accountability Office (GAO) report on the efficacy of drug courts (GAO, 2005), most of the adult drug court programs evaluated demonstrated reductions in recidivism, at least during the program participation period (i.e., 12 months post-baseline). The GAO analysis was based on 27 “methodologically strong” adult drug court evaluation studies (reporting on 39 individual programs, as some of the studies involved evaluating multiple courts) published between May 1997 and January 2004. Studies were defined as “methodologically rigorous” if they:

• used either an experimental design in which (1) eligible offenders were randomly assigned to different programs or conditions and (2) there was an acceptable level of attrition5 or a quasi-experimental design in which (1) all drug court program participants were compared with an appropriate group of comparable offenders who did not participate in the drug court program, and (2) appropriate statistical methods were used to adjust, or control, for group differences. If random assignment was not used, in an attempt to ensure that the groups were similar, aside from program participation (the intervention), the comparison group(s) should have been as alike as possible on a range of important characteristics. Statistical analyses can be used to further minimize differences between the program and comparison groups. (GAO, 2005, pp. 10)

The GAO analysis suggested that fewer drug court program participants were rearrested or reconvicted than comparison offenders; drug court participants had fewer recidivism events than comparison group members during the time of program participation, for both any felony offenses and for drug offenses (felony or misdemeanor); and that the evidence for the efficacy of specific drug court components, such as the behavior of the judge or the amount of treatment received, on “during-program” recidivism was inconclusive. GAO also concluded that recidivism reductions occurred for some period of time after program participation, when sufficient follow-up data were presented (GAO, 2005).

The generally positive findings of previous drug court evaluation results are encouraging because the drug courts that have been evaluated to date vary considerably in terms of program location (the 27 methodologically rigorous evaluations reviewed in the GAO report include programs located in 9 states and the District of Columbia and represent all regions of the country except the Northeast), program size, target population (with many program evaluations focusing on urban programs containing large proportions of non-white participants), judicial stage (i.e., whether the program is designed to be a diversion or post-adjudication program), and drug court program components (with substantial variability in treatment intensity, supervision intensity, and judicial leverage). The fact that most evaluation findings have generated positive findings for a program model that is implemented in an extremely diverse way across hundreds of jurisdictions, certainly suggest the promise of the drug court model. In addition, the general pattern of positive evaluation results being generated from studies with substantial variation in terms of evaluation methodology (with the GAO report including 3 experimental design evaluations, as well as numerous quasi-experimental designs employing either historical or contemporaneous comparison groups), outcomes examined, and the measured magnitude of their impacts, suggests a surprising degree of consistency.

In addition, although many of the existing drug court evaluations are methodologically rigorous, design or data issues limit the conclusions one is able to draw from many such inquiries. Importantly, out of 117 published drug court evaluation studies, GAO selected only 27 (23%) had enough methodological rigor to merit inclusion in their report. Common methodological limitations of the studies not selected include short follow-up periods, small sample sizes (i.e., limited statistical power), and reliance on inappropriate comparison groups.

Many drug court evaluations are characterized by short follow-up periods, which in many cases do not extend beyond treatment participation. Although it is necessary to assess whether drug court participation has positive effects during or shortly after provision, studies that examine effectiveness over multiple years are needed to determine whether the investment in drug court and its many features provides benefits at the individual and societal level over an extended period of time. Among the methodologically rigorous evaluations included in the GAO report, only 5 studies followed participants up to 2 years after drug court program entry (with the longest follow-up period being 3 years after program entry).

Many evaluations use small samples because program capacity or evaluation funds are limited. Although smaller studies may have an advantage in monitoring intervention fidelity and minimizing variability (Weisburd et al., 1993), the cost of small samples is limited statistical power to detect relatively small effects (e.g., a 0.1 or 0.2 reduction in the likelihood of arrest). Small effects may be the best that current approaches can obtain with criminal justice populations in which substance use is often co-occurring with other problems. With small sample sizes, the potential to identify intervention effects is reduced. From a policy perspective, however, even small effects may be substantial—and worth the investment of public resources—if they can be extended to larger populations. The 27 drug court evaluations reviewed in the GAO report found reductions in rearrest rates by drug court participants that were 10-30 percentage points lower than rearrest rates for comparison group members. For actual differences at the lower end of this range (or lower), larger samples are necessary for significant differences to be detected between groups.

Finally, and perhaps most importantly, many quasi-experimental evaluations use inappropriate comparison groups and/or methods. Of particular concern are evaluation designs that involve using intervention dropouts as comparison subjects. Since those who drop out of drug court are likely to be ‘self selecting’ not only into the dropout group but also the outcome failure group, the selection bias in such a scenario poses a serious threat to the validity of findings. Most of the 27 evaluations deemed by GAO to be rigorous used designs in which all drug court program participants were compared with an appropriate group of similar offenders who did not participate in the drug court program. Specifically, to ensure that the treatment and comparison groups were similar in virtually all respects aside from the intervention, GAO selected evaluations in which the comparison group was matched to the program group as closely as possible on a number of characteristics or that used statistical models to adjust, or control for, preexisting differences between the treatment and comparison groups. However, several of the previous evaluations have used historical comparison groups with which to compare the drug court group (i.e., the comparison group consists of individuals processed via “business as usual” prior to the implementation of the drug court program), which has some limitations. Of most concern is the possibility that changes other than the drug court program itself occurred over the time period, such as political climate, procedures for processing, etc. Worth noting is the fact that the present study both matched “contemporaneous” drug court and non-drug court participants and uses analytic techniques that help control for differences between the treatment and comparison groups.

Other researchers have utilized longitudinal marginal GEE models in the drug court and drug-involved offender literature to measure binary repeated measure outcomes. Binary outcomes utilizing GEE were used for measuring pass/fail urinalysis and treatment scale results (Marlowe, et al., 2003), as well as self-reported illicit drug use or alcohol intoxication (Marlowe, et al., 2006). It has also been used for dichotomized Addiction Severity Index (ASI) composite outcome measures in studies of drug-involved offenders (Chan, et al., 2005).

Although many studies of the impact of intervention on recidivism suffer from methodological limitations, several studies have addressed selection bias issues by using a variety of strategies, including randomization (e.g., Harrell, 1998; Deschenes et al., 2001), the instrumental variable approach (e.g., Truitt et al., 2002), and propensity score matching (e.g., Banks & Gottfredson, 2003; Rempel et al., 2003). These studies constitute improvements in methodological rigor, but additional studies addressing the issues that complicate research of this kind are needed to create a better understanding of the potential impact of drug court on the likelihood and severity of offender recidivism. This evaluation of the Hillsborough drug court did not employ randomization because the Hillsborough officials were simply unwilling to randomize offenders to receive or not receive treatment, as is often the case in the criminal justice system. We did not use the instrumental variable approach because we could not locate or find a variable or variables that would sufficiently represent the decision to assign a subject to treatment but not be associated with the outcome. In other words, no measure enabled us to isolate the effect of treatment independent of unobserved subject characteristics. We did not use propensity scores to equate treatment and comparison subjects because our sample size was not large enough to justify potentially losing cases from the analysis for whom an appropriate match could not be found. Instead, we employ an analytic alternative that helps adjust or control for both selection and maturation bias, which the GAO report supports as a methodologically rigorous quasi-experimental approach for handling the complications associated with having non-equivalent comparison subjects.

2.0 Methodology

The current analysis builds upon previous research, offering findings on the impact drug court participation has on recidivism. We use a quasi-experimental design and a sample consisting of 475 drug-involved offenders who entered supervision in Hillsborough, Florida, 274 of whom were enrolled in drug court. The sampling frame consisted of all adult offenders who entered community supervision in Hillsborough during our study enrollment period (February 2002 through November 2002), were determined to be drug-involved, had one or no prior prison commitments, had two or fewer supervision terms, and were on either felony probation or drug offender probation were included in the sampling frame. The drug court and non-drug court samples were thus matched in that both samples consisted of offenders who were drug-involved (described in more detail below), who had similar criminal histories, and who were under the same types of supervision, because we excluded offenders who were not similar on these dimensions. Additionally, we employed analytic methods to control for differences between the drug court and non-drug court samples.

The Hillsborough drug court being evaluated in this analysis has a pre-trial intervention program, as well as a post-conviction track. Participation in the pre-trial intervention program involved 12-to-18 months of status hearings, urine testing at least once a week, and several treatment components (screening, individual counseling, group counseling, acupuncture, and outside self-help meetings). Participation in the post-conviction program involved 24-to-36 months of status hearings, drug-offender probation or community control, random drug testing, and a range of treatment components. Drug court participants in this study were enrolled in either the pre-trial or the post-conviction program. No attempt is made to distinguish between these two treatment tracks descriptively or analytically in the current study, however, because the programs are similar in function and because comparison subjects met eligibility criteria for both programs.

Offenders were considered drug-involved if they had a drug-related charge other than drug trafficking, a charge that renders someone ineligible for drug court in Hillsborough. This restriction was essential to our efforts to identify a viable comparison group. Certainly there were offenders with drug or alcohol problems who were not enrolled in drug court and had charges that were not drug-related, but since it was not possible to determine from the administrative data if someone with a property charge was drug-involved, for example, we sampled only those with drug-related offenses. Similarly, there are likely some subjects with drug-related charges who were not drug-involved or did not have a problem with drugs or alcohol (e.g., a dealer who does not use). This, however, was not likely a common phenomenon and was not something for which we could control. Once concern with this approach relates to whether it resulted in the proportion of treatment subjects who use drugs being higher than the proportion of comparison subjects who use drugs. It is made clear later in the paper that drug court participants were more likely to report using drugs in the six months prior to the arrest that led to them being eligible for the evaluation, but this difference is not dramatic and is controlled for in our analyses.

We restricted our sample (both the drug court and comparison group members) to offenders with one or no prior prison commitments and two or fewer prior supervision terms because historical data from the Florida Department of Corrections (FL DOC) indicated that drug court participants rarely had more than one prison stay or two supervision commitments in their correctional histories. We also restricted our sample to offenders who were under either felony probation or drug offender probation because other forms of supervision (e.g., community control) were rarely used with drug court participants.

By excluding offenders who did not have drug-related charges, had more extensive criminal and incarceration histories, and were under unique or specialized forms of community supervision, we eliminated some of the bias that initially differentiated drug court and non-drug court participants, and thus were left with treatment and comparison samples that were essentially matched on these dimensions.

The sampling frame generated by FL DOC contained a flag for drug court program status. Individuals were classified as drug court participants if they participated in drug court for at least one day during the 30 days following the arrest that led to their current supervision. We used an intent-to-treat approach in our analyses, in that an individual's classification as drug court at baseline did not change over the 30-month follow-up period of our study, regardless of whether the individual exited drug court after initial classification. However, we dropped from our analyses 17 comparison group subjects who subsequently entered drug court during the 30-month follow-up period.² Our intent-to-treat approach is conservative in that we do not remove cases from our treatment group for not finishing the program. Some previous studies drop these cases from the evaluation or count drop-outs as comparison subjects, which likely inflates the impact of the intervention and in our opinion results in inappropriate conclusions about the efficacy of the program being evaluated.

2.1 Data Collection

The sampling frame was provided by FL DOC on a rolling basis (weekly) and included individuals who had entered supervision within the previous 7 days. We conducted baseline interviews with offenders entering supervision for approximately nine months, between February 2002 and November 2002. Subjects were recruited via telephone and household visits, and were interviewed in person in the community by trained field interviewers. Subjects received $50 cash incentives for participating in the baseline interview. Nobody else was present during the interviews, which were conducted using laptop computers and computer-assisted personal interview (CAPI) technology. The interviews took, on average, 75 minutes to administer and were typically conducted in private areas of the respondents' residences. The interview covered a variety of topics including demographics (race/ethnicity, marital status, parental status, employment, income, housing), supervision conditions (drug court participation, supervision status/conditions, supervision contacts, drug tests, court contacts, attitudes toward supervision), alcohol and drug use (including lifetime and current use [frequency and intensity] of 14 categories of substances, problems associated with use), treatment history (including treatment readiness), mental health, social support, and criminal history (arrests, incarcerations, and commission of crimes).

In addition to the baseline interview data, we obtained administrative data from the FL DOC and the Florida Department of Law Enforcement (FDLE) on all members of our sampling frame. The FL DOC data included a variety of measures of criminal history, the instant offense or crime for which the offender was charged that led to eligibility for the evaluation study, and the type of community-based supervision on which the offender was placed. We also obtained data on time at risk (i.e., the time offenders are not incarcerated in state prisons or in residential treatment facilities); however, we did not use the time at risk data in the analysis because after analyzing the distribution of this measure, we recognized that its variability was limited (i.e., most subjects had either 100% time at risk or no time at risk) and that inclusion did not improve the models.

The FDLE data included all arrests (by local and state police departments) experienced by the offender. The coupling of the interview and administrative data enables us to control for criminal history, include numerous covariates, and assess the impact of drug court participation on recidivism, the outcome of interest in this paper. We obtained administrative data on recidivism from FDLE as recently as July 2005, which provide at least 30 months of follow-up data for all evaluation study participants.

In Hillsborough, we fielded a total of 1,100 subjects. Of the 1,100 subjects fielded, 475 (43%) were successfully recruited and interviewed, 462 (42%) could not be located and reached within one month of entering supervision (our established “window” by which the baseline interview had to be completed),³ 106 (10%) refused to participate, 40 (4%) were incarcerated or institutionalized for the month after entering supervision, 15 (1%) could not speak English (the only language in which we offered interviews), and 2 (<1%) had died.

Our ability to recruit and interview subjects did not vary much across our drug court and non-drug court samples: we interviewed 274 of the 640 drug court subjects (42.8%) and 201 of the 460 non-drug court subjects (43.7%). Although our response rate (43%) was lower than expected, the actual refusal rate (10%) was quite low and cooperation did not differ across our treatment and comparison groups. The large majority of subjects we did not interview could simply not be located and reached within one month of entering supervision.

The administrative data allow for the comparison of respondents and non-respondents on a number of dimensions, including demographics (age, gender, and race), substance abuse treatment received prior to baseline, and criminal and incarceration history (prior arrests and prior prison stays). Because we expected drug court and non-drug court participants to differ on some dimensions, we conducted non-response bias analyses within the drug court and comparison groups (i.e., we compared drug court subjects who participated in the evaluation study to drug court subjects who did not participate in the evaluation study). Within the drug court group, the only significant difference was on the age variable. Drug court participants who were enrolled in the evaluation study were, on average, 2.2 years older than drug court participants who did not participate in the evaluation study. In the non-drug court (comparison) group, two significant differences were found between respondents and non-respondents: males were less likely to participate (86% of the comparison group non-respondents were male, compared to only 75% of the comparison group respondents) and offenders with more prior arrests were less likely to participate (comparison group non-respondents had, on average, 1.3 more arrests prior to baseline than comparison group respondents). Although our overall response rate was lower than what we hoped to achieve, the number of refusals was quite low. In addition, those who were enrolled in the study do not appear to be substantially different from those who did not participate, and the few differences that were identified are controlled for in the analyses.

2.2 Study Subjects

Data describing our study sample are presented in Table 1. The total sample had an average age of 30.4; was 77.5% male; was 36% white, 46% black, 14% Hispanic, and 4% other race. Over 46% of the sample did not graduate from high school, 36% graduated from high school, and 19% attended or finished college. Less than 24% of the sample was married, 19% were unemployed at the time of arrest, and 35% reported having annual incomes above $10,000. Additional data on the full sample are presented in Table 1.

Table 1.

Characteristics of the Total, Drug Court, and Non-Drug Court Samples

	Total Sample		Drug Court Sample		Non-Drug Court Sample		t-test/chi - square p level
	n=475		n=274		n=201
	n	%	n	%	n	%
Age (mean)	30.36		30.65		29.965		0.5007
Male	368	77.47%	218	79.56%	150	74.63%	0.2042
African American	218	45.89%	122	44.53%	96	47.76%	0.4855
Hispanic	66	13.89%	34	12.41%	32	15.92%	0.2753
Other race	19	4.00%	13	4.74%	6	2.99%	0.3347
White	172	36.21%	105	38.32%	67	33.33%	0.2647
Education < high school graduate	219	46.11%	118	43.07%	101	50.25%	0.1213
High school graduate, <some college	167	35.16%	95	34.67%	72	35.82%	0.796
Some college or greater education	89	18.74%	61	22.26%	28	13.93%	0.0215
Married	112	23.58%	68	24.82%	44	21.89%	0.4589
Employed full/part-time	264	55.58%	166	60.58%	98	48.76%	0.0103
Employed other	122	25.68%	67	24.45%	55	27.36%	0.4742
Unemployed	89	18.74%	41	14.96%	48	23.88%	0.0138
Income $10,000 +	164	34.60%	104	38.10%	60	29.85%	0.0624
Prior arrests (mean)	6.451		6.547		6.318		0.7052
Total prior prison sentences (mean)	0.3696		0.3447		0.4031		0.4987
Drug offender probation supervision	340	71.58%	208	99.27%	68	33.83%	<.0001
Felony probation supervision	135	28.42%	2	0.73%	133	66.17%	<.0001
Felt need for drug treatment	111	23.47%	77	28.21%	34	17.00%	0.0044
Received treatment before baseline	182	38.48%	121	44.32%	61	30.50%	0.002
Drug problem recognition scale	2.098		2.249		1.887		0.0003
Desire for help with drugs scale	2.898		2.982		2.78		0.0042
Pressure to get help with drugs scale	2.93		3.192		2.561		<.0001
Social support scale	3.773		3.776		3.768		0.938
Primary care responsibilities for child	144	30.32%	73	26.64%	71	35.32%	0.0421
Alcohol use past 6 months	355	74.74%	215	78.47%	140	69.65%	0.0289
Marijuana use past 6 months	274	57.68%	165	60.22%	109	54.23%	0.1925
Illicit non-marijuana use past 6 mos.	151	31.79%	97	35.40%	54	26.87%	0.0485
Drinking scale (10 indicator mean)	0.0888		0.1047		0.0672		0.0079
Drug use scale (7 indicator mean)	0.2271		0.2596		0.1827		0.0004
Mental health diagnosis	121	25.47%	72	26.28%	49	24.38%	0.6397
Public attorney	337	71.10%	199	72.89%	138	68.66%	0.3156
Colorado Symptom Index (CSI)	27.124		26.996		27.299		0.7565

Some significant differences between our drug court and non-drug court participants are evident. Drug court participants are more likely to have attended college, be employed, be on drug offender probation (compared to regular felony probation), believe they need substance abuse treatment, have received treatment before, recognize they have a problem with drugs or alcohol (problem recognition), want help for their drug or alcohol problem (desire for help), feel external pressure to get help with their drug or alcohol problem (external pressures)⁴, have used alcohol and illicit drugs (other than marijuana) in the 6 months before baseline, and score higher on problematic drinking and problematic drug use scales.⁵ Drug court participants are less likely to be on felony probation (compared to drug offender probation) and have primary care responsibilities for a child than comparison group members. No differences were observed in the remaining variables, including most demographic variables, criminal history, and mental health diagnoses and symptoms.

Thus, we observe in our data what we might expect from non-experimental data, despite efforts to control for important characteristics during the comparison selection process. In particular, we observe differences between our treatment and no-treatment groups along dimensions that might reasonably expect to be related to our outcome measure (e.g., arrest). The drug court participants are slightly more drug-involved, but also seem to be more ready for treatment. Controlling for differences of this kind between treatment and comparison subjects is important if one is to have confidence in any findings related to the impact of the intervention.

2.3 Data Analysis

We estimated multivariate statistical models to determine whether drug court participation effectively reduced the likelihood of recidivism for participating offenders compared to non-drug court participants, while controlling for covariates and both selection and maturation bias. Selection bias is most problematic when groups differ on characteristics that are correlated with expected differences in the outcomes of interest—the circumstance of the evaluation presented herein.

Maturation bias occurs when the outcome naturally evolves or changes over time irrespective of the presence of an independent variable of interest such as drug court participation. Arrests sometimes occur at the peak of an individual's criminal career since engaging in many criminal acts increases the likelihood of arrest. The arrest, and any subsequent intervention, may in turn serve as a catalyst for change, or the offender may return to more conventional (i.e., less criminal) behavior simple because they are coming down from their peak of offending, and the latter may have happened in the absence of arrest or subsequent intervention. Either will lead to observed lower rates of arrest following the instant arrest. Testing the impact of an intervention like drug court without accounting for the potential effect of maturation bias can confound the intervention effect with the impact of maturation.

Our statistical approach follows methodology similar to that outlined in Heckman and Hotz (1989) and used by several authors in criminal justice research. For example, Broner et al. (2004) used similar methods to estimate the impact of a program to divert people with histories of substance abuse and mental illness from jail into community treatment. The model exploits the information from repeated measures on each subject by comparing the pre-/post-intervention change experienced by participants in the intervention group to the change experienced by those in the comparison group, in a sense allowing a subject to serve as his or her own comparison. For this study, seven 6-month periods represent the repeated measures. Administrative data for two 6-month periods (12 months) prior to baseline and five 6-month periods post-baseline (30 months) are used in these analyses. Exhibit 1 presents the 6-month time periods and how they fit together relative to the baseline interview (conducted within one month of supervision entry).

Exhibit 1.

Repeated Measures Spanning 42 Months: Seven 6-Month Periods (0 represents baseline)

Months
-12	-6	0	6	12	18	24	30
Time 1	Time 2	Time 3	Time 4	Time 5	Time 6	Time 7
6 months	6 months*	6 months	6 months	6 months	6 months	6 months

^*Indicates time period that is the reference period

	The following equation shows the form of our outcome model.
	Let
	ARit = 1 if offender i was arrested during time t and 0 otherwise
	DCi = 1 if offender i was a drug court participant and 0 otherwise
	TimeTit = 1 if the observation is at time T = t and 0 otherwise
	Zit = a vector of covariates
	εit = error term
	Then
	ARit = β0 + β1DCi + β2Time1it + β3Time3it + β4Time4it + β5Time5it + β6Time6it + β7DCi * Time7it + β8DCi * Time1it + β9DCi * Time3it + β10DCi * Time4it + β11DCi * Time5it + β12DCi * Time6it + β13DCi * Time7it + γZit + εit

We estimated this model using generalized estimating equations (GEE). Each subject contributed seven observations to the dataset; an arrest was observed for 1,156 of the 3,097 observations (37.3%).⁶ The drug court indicator is included to control for selection bias due to omitted variables associated with treatment status (under random assignment the expected value of β₁ would be 0). The time indicators capture individual trend levels over time and are intended to help control for maturation bias; if the likelihood of arrest is the same in all periods then the coefficients on these variables would also have an expected value of 0. The most important variables in the above equation are the interaction terms, DC_i * TimeT_it where T = 3, 4, 5, 6, and 7 which capture the effect of the drug court program at 6, 12, 18, 24, and 30 months post-baseline (DC_i * Time1_it refers to the period 6-to-12 months prior to the period that includes the instant arrest).⁷ The corresponding coefficients measure the degree to which drug court is associated with arrest in the previous six months. Time 2 (the 6 months immediately preceding baseline) served as the reference category. If the drug court program is effective at reducing the likelihood of arrest during each of the five 6-month periods following baseline, we would expect the coefficients to be negative and statistically significant. This model allows us to examine the change in offending over time, relative to baseline, for the treatment and comparison subjects. Thus, we can examine, for example, whether those in the drug court improved more during a period than those not in drug court, relative to baseline.

Z represents covariates that are included to control for selection bias potentially associated with observed variables. These covariates are derived from either the baseline interview data or administrative data. The variables available for inclusion in Z were socio-demographics (e.g., age, gender, race/ethnicity, marital and parental status, education, employment, and income); social support (Lin Social Support Scale [range = 1-4, Cronbach alpha = 0.74]; substance use (including scales of problematic drug use [range = 0-1 Cronbach alpha = 0.75] and problematic alcohol use [range = 0-1 Cronbach alpha = 0.75]), readiness/motivation for treatment (TCU scales of problem recognition [range = 1-5, Cronbach alpha = 0.90], desire for help [range = 1-5, Cronbach alpha = 0.58], and external pressure for treatment [range = 1-5, Cronbach alpha = 0.77]), and treatment received; mental health diagnoses and psychological symptomatology (Colorado Symptom Index [CSI, Shern et al., 1994], range = 15-75, Cronbach alpha = 0.89); criminal and incarceration history (i.e., number of arrests and having a prison stay prior to baseline); whether the offender had a public attorney; and supervision type and intensity (e.g., number of completed and incomplete contacts with probation, and number of drug tests taken and number of positive tests). Some of these covariates, specifically measures of supervision and drug test results change over time; all other measures reflect an offender's situation at baseline and are therefore fixed in each time period.

3.0 Findings

The multivariate statistical models were designed to determine whether drug court participation reduced the likelihood of recidivism for drug court participants compared to non-drug court participants, while controlling for covariates and potential sources of selection and maturation bias. The models exploit the information from repeated measures on each subject by comparing the pre-/post-intervention change experienced by participants in the intervention group to the change experienced by those in the comparison group, thus allowing a subject to serve as his or her own comparison. For this study, seven 6-month periods (42 months total), including two 6-month periods (12 months) prior to baseline and five 6-month periods post-baseline (30 months) were examined. The 6-month period immediately preceding baseline was selected as the reference period because the purpose of the repeated measures model is to compare the pre-/post-intervention change experienced by drug court participants to that experienced by comparison group members. Within this framework we regressed a binary arrest indicator on drug court participation, as well as a rich set of covariates, using generalized estimating equations (GEE). We used logistic regression to estimate the likelihood of any arrest.⁸

We utilized data for 458 subjects for the reduced model, for a total of 3206 time intervals. 57 subjects had missing data, for a total of 93 time intervals (2.9%). Of these 93 intervals, 13 (0.4%) were missing due to right censoring of arrest data (intervals that included time occurring after the arrest file was generated). Another 80 (2.5%) intervals were set to missing due to the subjects having missing or no time at risk, because the subject was in prison or in residential drug treatment for the entire 6-month time interval, or incarceration and/or treatment information was right censored. It was assumed that if a subject had no time at risk, the subject could not have been arrested, and leaving such observations in the analysis could potentially bias results.

The missingness of these dropped intervals is questionable. Retaining actual arrest information for these records could potentially bias results, justifying their removal. However, these observations would not normally be considered missing completely at random (MCAR) nor missing at random (MAR). These data could be considered informative (Diggle, Liang, Zeger, 2004) or similarly termed ‘not missing at random’ (Fitzmaurice, et al., 2004), since the probability that arrest information is missing could be considered related to the values that should have been obtained. For certain individuals, it is possible that some intervals may have had no time at risk as a direct result of an arrest during the prior interval (which led to incarceration, for example). These missing intervals are irrelevant observations since during these intervals subjects were not able to be arrested, for either part or all of the period. We, therefore, analyzed the data treating the 93 intervals as missing, but we also attempted a more conservative approach by performing worst-case confirmatory models that assumed all intervals with missing arrest data had actual arrests occur during the interval.

Table 2 presents the logistic results from the reduced model, treating missing intervals as missing. The results suggest that drug court participants are at higher risk of arrest than comparison subjects overall, while the time measures reflect that for both drug court and non-drug court participants the period immediately prior to the baseline interview was one of elevated risk of arrest compared to the period immediately prior and all periods after. The interaction terms are, however, the variables of interest, particularly those associated with times 3 through 7, as the coefficients on these variables provide measures of the association between drug court participation and recidivism (relative to baseline) in each of the five 6-month time post-baseline periods. As shown in Table 2, all of the post-baseline drug court-time interactions (3, 4, 5, 6, and 7) have negative coefficients, suggesting that participating in drug court reduces the likelihood of recidivism as measured by arrest. The coefficient is significantly different from zero, however, only for the drug court-time5 measure (12 to 18 months post-baseline). During this time period, drug court participants are 2.04 times less likely to be arrested than their non-drug court counterparts, a statistically significant finding at the 0.01 level.

Table 2.

Reduced Logit Model of Drug Court Impact on Recidivism

	Odds Ratio	Standard Error	95% CI
Drug Court Impact Measures
Drug court indicator	1.55*	0.322	1.03	2.33
Time 1	0.47**	0.121	0.29	0.78
Time 3	0.26***	0.053	0.17	0.39
Time 4	0.28***	0.061	0.19	0.43
Time 5	0.28***	0.056	0.19	0.41
Time 6	0.18***	0.040	0.12	0.28
Time 7	0.17***	0.038	0.11	0.27
Drug court*time 1	0.84	0.278	0.43	1.60
Drug court*time 3	0.80	0.217	0.47	1.37
Drug court*time 4	0.87	0.245	0.50	1.51
Drug court*time 5	0.49**	0.134	0.29	0.84
Drug court*time 6	0.69	0.203	0.39	1.23
Drug court*time 7	0.71	0.206	0.41	1.26

^*= significant at the 0.05 level

^**= significant at the 0.01 level

^***= significant at the 0.001 level

When missing arrest outcome information was forced to represent the ‘worst case scenario’ (i.e., missing data was set to indicate the presence of an arrest), the results were very similar to when missing arrest data were ignored. As was the case in Table 2, Table 3 shows all drug court-time interactions with negative coefficients; however, the drug court indicator is no longer significant (although this is in the model for control purposes only and does not represent the association between drug court participation and recidivism), and the odds ratio for the drug-court time interaction for period 5 changes to 0.49 (p=.011).

Table 3.

Reduced Logit Model of Drug Court Impact on Recidivism

	Odds Ratio	Standard Error	95% CI
Drug Court Impact Measures
Drug court indicator	1.36	0.280	0.91	2.04
Time 1	0.49**	0.117	0.31	0.78
Time 3	0.23***	0.046	0.15	0.34
Time 4	0.25***	0.052	0.17	0.38
Time 5	0.28***	0.054	0.19	0.41
Time 6	0.20***	0.042	0.14	0.31
Time 7	0.18***	0.038	0.12	0.27
Drug court*time 1	0.81	0.260	0.43	1.52
Drug court*time 3	0.91	0.245	0.54	1.55
Drug court*time 4	0.99	0.273	0.58	1.70
Drug court*time 5	0.51*	0.136	0.30	0.86
Drug court*time 6	0.71	0.196	0.41	1.22
Drug court*time 7	0.83	0.229	0.48	1.42

^*= significant at the 0.05 level

^**= significant at the 0.01 level

^***= significant at the 0.001 level

Table 4, ignoring intervals with missing arrest data, extends the reduced model by including a number of covariates that are theoretically or empirically associated with recidivism.⁹ As with the reduced model, the drug court-time interactions for periods 3, 4, 5, 6, and 7 are the independent variables of interest and reflect the association between drug court participation and recidivism (relative to baseline) in each of the five 6-month post-baseline time periods. Similar to the reduced model, the coefficients for all of these variables are negative, but drug court participation is statistically significant in only time period 5 (12 to 18 months post-baseline). During this time period, drug court participants are 2.16 times less likely to be arrested than their non-drug court counterparts, controlling for baseline characteristics and covariates. The log likelihood of this model was -1765.7, which suggests that the addition of these covariates to the model resulted in an improvement over the previous model, which had a log-likelihood of -1980.6.

Table 4.

Backward Selection Logit Model of Drug Court Impact on Recidivism

	Odds Ratio	Standard Error	95% CI
Drug Court Impact Measures
Drug court indicator	1.44	0.351	0.89	2.32
Time 1	0.46**	0.129	0.26	0.79
Time 3	0.20***	0.048	0.13	0.32
Time 4	0.24***	0.056	0.15	0.37
Time 5	0.24***	0.054	0.16	0.37
Time 6	0.15***	0.036	0.09	0.24
Time 7	0.15***	0.036	0.09	0.24
Drug court*time 1	0.80	0.291	0.39	1.63
Drug court*time 3	0.95	0.292	0.52	1.74
Drug court*time 4	1.01	0.308	0.56	1.83
Drug court*time 5	0.46**	0.138	0.26	0.83
Drug court*time 6	0.75	0.243	0.40	1.41
Drug court*time 7	0.72	0.225	0.39	1.33
Socio-Demographics
Age (mean)	0.98***	0.004	0.97	0.99
Male	1.00	0.114	0.80	1.25
Black non-Hispanic	1.19	0.128	0.96	1.47
Hispanic	1.28*	0.158	1.01	1.63
Other race	1.05	0.254	0.65	1.69
High school graduate	1.00	0.099	0.82	1.21
Some college or greater education	0.86	0.105	0.68	1.10
Employed full/part-time	0.86	0.106	0.68	1.10
Employed other	0.81	0.115	0.61	1.07
Annual income of $10,000 +	0.85	0.086	0.70	1.03
Social support scale	0.89*	0.041	0.82	0.98
Substance Use and Mental health
Felt need for drug treatment at baseline	0.82	0.102	0.64	1.04
Drinking scale at baseline	1.45	0.373	0.87	2.40
Drug use scale at baseline	1.40	0.332	0.88	2.23
Colorado Symptom Index (CSI)	0.98***	0.006	0.97	0.99
Criminal and Incarceration History
Arrests prior to baseline	1.08***	0.012	1.05	1.10
Prison stay prior to baseline	0.66**	0.086	0.51	0.85
Public attorney	1.40***	0.141	1.15	1.70
Probation Supervision Measures
Drug offender probation supervision at baseline	1.13	0.174	0.84	1.53
Incomplete probation contacts1	1.07*	0.032	1.01	1.13
Completed probation contacts1	1.01	0.004	1.00	1.02
Number of drug tests1	0.95***	0.015	0.92	0.98
Number of positive drug tests1	1.06	0.054	0.96	1.17

¹Indicates a measure that changes over time, from time period to time period

^*= significant at the 0.05 level

^**= significant at the 0.01 level

^***= significant at the 0.001 level

The findings presented in Table 4 show that a number of covariates are associated with the likelihood of arrest during the 30 month follow-up period. Age is negatively associated with the likelihood of arrest, with older offenders being less likely to be arrested. Hispanic subjects have a higher likelihood of arrest (compared to white subjects). Both of these findings are consistent with literature and therefore not surprising. Social support appears to be somewhat protective against arrest, with offenders reporting higher levels of social support having a lower likelihood of arrest. Interestingly, psychiatric symptomatology (measured by the CSI) appears to be negatively associated with likelihood of arrest. One would have hypothesized that subjects with higher scores on a scale of psychiatric symptomatology would have been more likely to be arrested; however, it is possible that those who score high on the CSI were more likely to be in some sort of residential mental health placement (an indicator for which we do not have data) and therefore had less time at risk (i.e., time in the community to be arrested) and a lower likelihood of being arrested. One surprising finding is that none of the substance use measures were significantly associated with likelihood of recidivism during the 30-month follow-up period. We explored the possibility that our measures suffered from multicollinearity and that perhaps the association between one or more substance use measure and recidivism was being attenuated, but no evidence of this possibility could be found.

Several criminal history measures were associated with arrest, although not always in the expected direction. Although arrest history was associated with a greater likelihood of arrest (consistent with the notion that past behavior may be the best predictor of future behavior), offenders who served at least one prison sentence prior to baseline were less likely to be arrested during the 30 month follow-up period. Although somewhat surprising, this result is consistent with findings from an analysis that examined re-arrest for nearly 140,000 drug-involved probationers in Florida (see Lattimore et al., 2005). It is also consistent with previous research (e.g., Andenaes, 1968; Fabelo, 1995) that seems to support specific deterrence theory which posits that individuals who experience more severe punishment, in this case a prison sentence, are likely to reduce their criminal activities in the future. The idea that incarceration reduces recidivism is, however, not without detractors (e.g., von Hirsch et al., 1999; Andrews & Bonta, 1994). Interestingly, being represented by a public attorney (as opposed to private counsel) for the charge that led to the current supervision term was associated with increased arrest likelihood. It may be that having a public attorney is a proxy for other risk factors for recidivism, such as low socio-economic status or limited social support, however since we have measures that control for income and social support, it may be that having a public attorney has an independent impact on likelihood of arrest.

Finally, several supervision measures were significantly associated with arrest. Measures of both uncompleted and completed probation contacts are significant and positive, illustrating that the more probation contacts (or attempted contacts) an offender has, the more likely he or she is to be arrested. Although having more contacts may simply increase the likelihood that a probation officer will detect an offender doing something illegal, probation officers may have more contacts with offenders whom they perceive to be (and who may actually be) at higher risk for recidivism. The number of drug tests administered to an offender is associated with a decrease in arrest likelihood, suggesting that testing drug-involved offenders for drugs may deter them from using drugs and/or committing crimes.

Similarly, when missing data was assumed to represent the ‘worst case scenario’ (i.e., missing data was set to indicate the presence of an arrest) in the full model, the results are very similar to those generated by the model that ignored missing arrests. The same results hold, except incomplete contacts becomes insignificant (p=.07), having a prior prison stay becomes insignificant (p=.16), having a public attorney becomes significant at the 0.01 level (p=.003). Since the results do not change very much depending on whether missing arrest data are treated as missing or as actual arrests, only the model ignoring missing arrest information is presented.

4.0 Discussion and Conclusions

This study examined the long-term impact of drug court participation (compared to regular probation) on the recidivism of drug-involved offenders under supervision in a large county in Florida. Our study was based on a relatively large sample (n=475), a rich dataset comprised of both self-reported and administrative data, an extended follow-up period, and an analytic approach that helped control for selection and maturation bias. The results indicate that participation in drug court is associated with a significant decrease in the likelihood of recidivism in the 12 to 18 months following baseline.

Although drug court was associated with decreased likelihood of recidivism in the other time periods, the association was not significant. The impact of drug court appears to be delayed (particularly since the 12-18 month time period reflects a time period that often follows the conclusion of drug court participation for program graduates, and a time period well beyond initial program exposure for program drop-outs). In addition, the “drug court effect” seems short-lived, in that it is not sustained in subsequent time periods.

Interpreting the delayed and somewhat short-lived drug court effect observed in this study is also difficult because few previous evaluations have analyzed long-term effects (for all members of a large sample) and none that we are aware of have examined effects by various time intervals in a repeated measures framework such as the one employed in the current study. The GAO review identified 17 drug courts for which post-program recidivism was analyzed, and found lower rearrest or reconviction rates for drug court participants compared to comparison group members for 13 of these programs (GAO, 2005). The pattern of a “delayed” drug court effect observed in the current study is somewhat supported by the Maricopa County evaluation (reviewed in the GAO report), which did not find a drug court effect during the one-year period associated with program participation, but did find lower rearrest rates over a three-year follow-up period for drug court participants compared to comparison group members.

When a single study produces a finding it is sometimes viewed as an anomaly until it is replicated. The fact that this other study also found a delayed impact lends credibility to our finding. Relatively few drug court evaluations have looked beyond the program period, so it is unclear whether what we found is common or not, but certainly two studies finding the same delayed impact of drug court on recidivism suggests there may be something about drug courts that keep them from producing an immediate impact on client recidivism. Many drug courts, including Hillsborough, impose strict supervision conditions on clients and it might be that these conditions make it difficult for clients to succeed (i.e, remain arrest free) during the program period, but that these conditions are somewhat effective at discouraging offending long-term. Of course, we see that the impact drug court seems to have on recidivism goes away 18 months after program enrollment. One implication of our finding that the impact of drug court on recidivism is delayed and rather short-lived is that drug courts and researchers should explore what about drug court might be producing these results. By interviewing drug court participants and staff, we might be able to determine what it is about drug court that makes it difficult for participants to make any gains in terms of recidivism compared to their comparison counterparts during the program period and why they are not able to sustain any subsequent gains long-term. We might find that the some changes can be made to the drug court model that would result in more immediate and longer lasting impacts. For example, maintaining some features of drug court beyond the typical program period (e.g., random drug testing) could result in longer term effects.

Although this evaluation makes a contribution to the literature on the efficacy of drug courts and presents some methodological options for researchers faced with quasi-experimental conditions, it is not without limitations. First, the baseline response rate was relatively low and our data were collected in a single county in Florida, which limit the generalizability of our data and results to other jurisdictions in the United States and other countries. It is encouraging, however, that some of our results are consistent with other research and that our nonresponse bias analyses did not uncover any major differences between those who did and did not participate in our evaluation. Second, the research design we employed is quasi-experimental in nature, which makes the evaluation of the intervention more complicated logistically and analytically. Because we believe accounting for bias that typically affects non-experimental studies is important and methodologically responsible, we did what we could to match treatment and comparison subjects and analytically control for observed differences using a unique analytic approach. Unfortunately, randomizing subjects to receive treatment or not, was not possible, which is often the case under evaluation scenarios of this kind (Manski, Pepper, and Petrie, 2001). Third, as is the case with any data that uses self-reported data, threats to data quality, such as issues related to recall and honesty, are always a concern; however, we used self-reported data for both treatment and comparison subjects and only at baseline. All outcome data on recidivism were based on administrative data. Although we would have liked to use the follow-up interview data, we noticed differential attrition, as well as differential rates of recidivism associated with attrition, across our treatment and comparison groups. Using only administrative data for the recidivism outcome thus constituted the more methodologically responsible evaluation approach and enabled us to study the impact of drug court over a longer follow-up period (i.e., 30 months).

The nature of the outcome data, as well as the analysis framework utilized in this study, had limitations. The fact that the nature of the arrest outcome and the time at risk measure were potentially associated created analytic challenges. Alternative methods could have been employed to treat the missing data in another way, for example, multiple imputation techniques, Last Observation Carried Forward (LOCF), or principal stratification techniques. However, these methods would not necessarily help address the decreased opportunity for arrest that individuals were experiencing during an interval of no risk (i.e., when they were incarcerated or in residential treatment). An alternative to longitudinal models using 6-month intervals, that we plan to utilize in future studies, would be to employ survival models with time dependent covariates to assess exactly when an individual became at risk and or recidivated. This time dependent covariate approach might also be an effective alternative to dropping cases who started out in the comparison group but entered drug court during the follow-up period.

In sum, the large number of offenders who are arrested and supervised in the community and who are drug-involved has been widely documented in the United States and abroad. Some proportion of these offenders are referred to and enter drug treatment courts, which seem to have the potential to reduce the likelihood of recidivism. A substantial proportion of prison inmates are drug-involved and/or being held on drug charges¹⁰, and the costs associated with imprisonment are substantial. At a time when budgets are stretched, it is possible that drug courts may yield a number of societal and criminal justice system benefits--both in terms of improvements in public safety and potential cost savings. Providing drug court as an option to drug-involved offenders around the world who might otherwise be supervised in the community may in fact be a feasible approach to reducing the likelihood and collateral impacts of recidivism, and thus enhancing public safety. This evaluation provides some support for the role of drug courts in our mission to reduce substance use and criminality, but additional research is certainly needed to better understand the long-term impact of drug courts, which components are most effective, and for whom drug court is most appropriate.