Drug Court Effectiveness: A Review of California Evaluation Reports, 1995–1999

Abstract

Over the past two decades, drug courts have emerged as a viable alternative for addressing drug cases within the criminal justice system. In California, the Drug Court Partnership Program (DCPP) was created in 1998 and has supported and funded the development of drug courts throughout the State. This article reports on a review of California drug court evaluations through January 2000 conducted as part of an evaluation of the California DCPP. A total of 23 evaluations were collected. Seventeen were reviewed in detail, and six were excluded because they were internal reports rather than evaluations. A standardized review process was initiated which led to a scored rating of the evaluation reports. Results of this review support previous findings that drug court participants may experience reduced rearrest rates by 11% to 14% compared to nonparticipants. The largest reduction in rearrest rates appears among graduates. The graduation rates were between 19% and 54%. Costs and savings associated with drug courts were discussed but no conclusions were possible based on the findings from these evaluations. The evaluation of the effectiveness of drug courts presents unique challenges. This review concludes with a discussion of evaluation methods (e.g. standardizing rate calculations, term definitions) that would strengthen drug court research.

In the past two decades, the jail and prison systems of the United States have experienced unparalleled growth, with much of this growth attributable to drug-related crime. From 1980 to 1997 the number of persons incarcerated in state prisons for violent offenses doubled, the number incarcerated for nonviolent offenses tripled, and the number incarcerated for drug offenses increased eleven-fold (1040% increase) (Beatty, Holman & Shiraldi 2000).

Early efforts to address drug issues in court systems occurred between the 1950s and 1970s, when a few courts dedicated themselves to addressing drug cases. Specialized to meet the growing heroin problem of the time and in response to harsher penalties for drug-related offenses, these courts met with limited success. The crack cocaine epidemic of the 1980s, and the legislative and judicial response to this epidemic, resulted in changing demographics of the drug-offender population to include more minorities and women. Incarceration alternatives such as diversion programs and treatment as a condition of probation were designed to meet the needs of this growing and more diverse offender population. However, these programs had limited supervision, varied in approach and structure, and did not seem to stem the growth in jail and prison populations (Drug Courts Program Office 1999; Belenko 1990).

The first structured drug court began in Dade County, Florida, in 1989 (Goldkamp & Weiland 1993). Subsequent drug courts emerged as a result of a nationwide grassroots effort that evolved from prior attempts at integrating substance abuse treatment and criminal justice systems, and out of the frustrations of judicial leaders who saw “revolving door” court strategies as ineffective. Several early drug courts were developed in the wake of the Miami Drug Court, and the experiences of these early courts were distilled in a guide entitled “Defining Drug Courts: The Key Components” (Drug Court Program Office 1997). The 10 key components of drug courts include such elements as immediate sanctions, a nonadversarial approach, frequent drug testing, immediate access to a continuum of substance abuse treatment services, and a partnership between drug courts, public agencies and community-based organizations.

Since 1989, in addition to the practice standards reflected in the key components, the drug court movement has been supported and assisted in its development by a federal Drug Court Programs Office, a dedicated professional association (National Association of Drug Court Professionals), and a training institute (National Drug Court Institute). There are currently over 700 drug courts nationwide, and an additional 400 drug courts in planning stages (National Drug Court Institute 2001). Drug courts vary in terms of whether they are pre- or post-plea programs, and whether they are sentence-based or diversion model courts. Recently, other specialized courts such as juvenile, family, domestic violence, mental health, driving under the influence, and prostitution courts have developed from the original drug court model.

THE CALIFORNIA DRUG COURT PARTNERSHIP PROGRAM (DCPP)

In California there are over 128 drug courts, with 49 out of the 58 counties operating at least one (California Administrative Office of the Courts 2001). About 30 counties have more than one drug court, with Los Angeles County having 11 adult drug courts. The California Drug Court Partnership Act of 1998 (Health and Safety Code 11970), allocated $4 million to support the development of drug courts statewide. The initial funding, available in May 1999, was supplemented by an additional $4 million in July 1999. Under this initiative, 34 counties received funding to implement or expand (post-plea) drug courts. The legislation provided that the Department of Alcohol and Drug programs and the California Judicial Council would conduct an evaluation of the DCPP. One component of the statewide DCPP evaluation involved the collection and review of existing California-based drug court evaluation reports.

This article reports on a review of all California drug court evaluations available through January 2000. Each evaluation was reviewed and abstracted to produce a descriptive summary, and was then rated for its scientific strength using a 25-point scale. In this article, findings from the 11 highest-rated outcome studies are summarized to offer conclusions about the effectiveness of California drug courts. Drawing from the experience of conducting this review, the authors discuss considerations for designing and reporting future drug court evaluations.

METHODS

Collection of Reports for Review

The authors first collected all California-based drug court evaluation reports described in the national literature review by Belenko (1998). Next, for each of the 58 California counties, either the Drug Court Coordinator or the Alcohol and Drug Program Director was contacted by phone. Respondents were asked to describe the number of drug courts in their county, types of courts operating (e.g., pre-plea, post-plea), and whether any drug court evaluations had been conducted in the county. At the time of the survey, 41 of the 58 counties operated at least one drug court. A total of 16 counties reported having completed at least one evaluation, and 17 counties reported having evaluations in progress. Respondents were asked to send completed reports to the review team. Twenty-three evaluations were collected through literature review and county survey procedures. Six were later excluded because they took the form of internal or quarterly reports without explanation or interpretation. The remaining 17 reports were reviewed in detail.

Review Procedures

Each evaluation was reviewed using a two-stage process. In the first stage, each report was reviewed, key information was abstracted, and a summary description was completed. In the second stage, each report was scored for scientific quality using a rating scale developed for this study. Quality was generally defined in terms of comprehensiveness (inclusion of process, outcome, and/or cost information), and the strength of study design. Each report was reviewed twice, first for descriptive purposes and then again to assign the quality rating.

Descriptive review

To systematically describe the collected reports, a standardized data abstraction tool was developed by the review team. The tool included identifying information (title, author, agency, county, year), the type of evaluation as given in the report (process, outcome, cost), a summary of main findings, and study limitations. For each type of evaluation activity reported, additional information was abstracted. For process evaluations information was abstracted about program description (type of drug court, length of program, treatment options), and data collection strategies (e.g., interviews with key informants or drug court participants). For outcome evaluations, abstracted information included study design, demographics of the drug court and comparison samples, outcome measures used in the study, and follow-up design. For cost evaluation efforts, abstracted data included type and amount of costs, savings, and averted costs associated with the drug court.

Each report was read by at least three reviewers who independently completed the written descriptive review form. Reviewers met weekly to discuss each report, clarify questions and inconsistencies between reviewers, and generate a single summary for each evaluation reflecting the collective judgement of the review team.

Quality Rating

After completing the descriptive review for all reports, the review team developed a rating scale for use in the second review stage. The rating scale assigned points in each of four component areas: Type of Evaluation (six points), Design (six points), Follow-up (six points), and Measures (seven points).

In the first component, Type of Evaluation, one point was given for each of the following if included in the evaluation: description of the drug court; process information such as interviews with key informants, or feedback from drug court participants; descriptive data about study cohorts (e.g., demographics, criminal history); outcome data; cost information for the drug court or associated treatments; or an analysis of cost information.

In the Design component, studies having stronger designs received more points. Pretest and posttest designs, defined as measuring the same group before and after an intervention, were divided into two groups. Pre-post designs with no comparison group were assigned one point. Pre-post designs with a comparison group, and within group comparison designs, were assigned two points. Unmatched comparison group designs received three points, matched comparison designs received four, and random assignment designs received five points. In practice, there were no pre-post designs or no random assignment designs in this group of evaluation reports. Reports received one additional point if they showed that the comparison group used in the study was similar to the drug court group.

In the Follow-Up component, one point was given for each of the following areas: each participant followed for the same time period; a follow-up period of at least one year; follow-up time frame was the same for drug court and comparison groups; follow-up included both drug court and comparison participants; follow-up data from administrative sources (e.g., arrest records) were collected for more than 50% of the sample; and if interviews were conducted, more than 50% of the sample was interviewed.

A total of seven points could be given under the Measures component, with one point assigned for each of the following: sample size greater than 40 per group; reporting of arrest or conviction measures; reporting program graduation or retention rates; reporting of other program outcome measures beyond arrest, conviction, or graduation information; reporting demographic information for the drug court sample and for the comparison sample; and reporting statistical analyses of data for between-group comparisons.

At least three reviewers read the initial summary and rated each review component. The team met for a second series of weekly meetings to review the ratings, revisit the full evaluation report when necessary, and agree on the scores included in the rating scale. While the total score for any single report could range from zero to 25, actual values ranged from one to 21 points. At the conclusion of the review process, each evaluation report was summarized in terms of a written description and overall quality rating. The descriptive review form and rating scale are available from the first author.

RESULTS

Descriptive Summary of California Drug Court Evaluations

Table 1 summarizes general descriptive information for the 17 California drug court evaluation reports reviewed. These reports, completed between 1995 and 1999 and representing 13 counties, are arranged in descending order of quality rating. Five counties completed two reports during this time period (Alameda, Ventura, Santa Clara, Contra Costa, and Los Angeles). The dots in the table reflect how the authors of each report described the type of evaluation. Of the 17 reports, 11 were described as including process information, 15 as including outcome information, and seven as including cost or savings information. Two reports (Bedrick 1997; Deschenes & Torres 1996) included process information only, while the others were described as some combination of process, outcome, and/or cost evaluations.

TABLE 1.

Summary of California Drug Court Evaluations, 1995–1999

Author	Year	County	Process	Outcome	Cost	Score
Deschenes et al.	1999	Orange	*	*		21
Sechrest et al.	1998	Riverside		*	*	17
Tauber	1995	Alameda		*	*	16
Cosden, Crothers & Peerson	1999	Ventura		*		16
Roehl	1998	Monterey	*	*	*	15
Cosden, Peerson & Crothers	1999	Santa Barbara	*	*		14
Criminal Justice Research Foundation	1999	Sacramento	*	*	*	13
Cabrera et al.	1997	San Francisco		*	*	12
Oberg	1996	Ventura	*	*		12
Hicks, Hicks & Bautista	1999	Mendocino	*	*	*	11
Santa Clara County	1998	Santa Clara		*		11
Jrapko et al.	1999	San Joaquin	*	*		09
Community Crime Prevention Associates	1998	Santa Clara	*	*	*	09
Cherry	1999	Contra Costa		*		08
LA Cty Municipal Courts Planning & Research Unit	1996	Los Angeles	*	*		08
Deschenes & Torres	1996	Los Angeles	*			02
Bedrick	1997	Alameda & Contra Costa	*			01

The use of the scores given in the last column of Table 1 enables discrimination between reports based on their quality rating. To focus review efforts on a smaller number of reports, the authors selected those having a quality rating of 10 points or above. The use of a cut-off allows greater focus on those reports that were, in our view, most informative with respect to the effectiveness of California drug courts. This cut-off reflects the extent to which reports included information we were interested in, and is not a measure of how well the reports met their intended goals.

Characteristics and Ratings of California Drug Court Evaluations

Table 2 summarizes key characteristics of the 11 reports rated 10 points or above. In this table, the counties where reports originated are listed across the top of the table, beginning with Orange County (Deschenes et al. 1999; 21 points) and ending with Santa Clara County (Santa Clara County 1998; 11 points). The left hand column of the table lists the four component areas on which each report was assessed (Evaluation Type, Design, Follow-up, and Measures). Under each component heading are the characteristics rated by the review team for that component, followed by the number of points assigned.

TABLE 2.

Summary of Evaluation Characteristics and Ratings for Reports Rated 10 or Above

	Points (25 pts)	Orange	River- side	Alameda	Ventura 1999	Monterey	Santa Barbara	Sacramento	San Francisco	Ventura 1996	Mendocino	Santa Clara
Evalution Type
Program Description	1	•	•	•		•	•	•		•
Process Evaluation	1	•				•					•
Descriptive Data	1	•	•	•	•	•	•	•	•	•	•	•
Outcome Data	1	•			•		•					•
Cost Data	1		•	•		•		•	•		•
Cost Analysis	1		•						•
Design*
Random Design	5
Matched Comparison Group	4	•
Unmatched Comparison Group	3		•	•	•	•			•	•	•	•
Within Group Comparison**	2						•	•
Pre/Post w/ Comparison Group	2
Pre/Post w/o Comparison Group	1
Comparable Comparison Group	1	•
Follow-Up
Time frame consistent across cases	1	•	•	•	•	•	•	•
Follow-up at least one year	1	•	•	•	•	•	•	•
Time frame consistent across groups	1	•		•	•	•		•		•
Unbiased***	1	•	•	•	•	•			•	•	•	•
Data on more than 50%	1	•	•	•	•	•	•	•	•	•	•	•
Interview more than 50%	1
Measures
N > 40 per group	1	•	•	•	•		•	•	•	•		•
Re-arrest /conviction	1	•	•	•	•	•	•	•	•	•	•	•
Graduation/Retention	1	•	•	•	•	•	•	•	•		•	•
Other Measures	1	•					•
Demographics - Drug Court Group	1	•	•	•	•	•	•	•		•		•
Demographics - Comparison Group	1	•	•	•	•					•
Statistical analysis	1	•	•		•		•		•		•

^*Design category –maximum of six points – highest ranking design totaled with one bonus point if there was a comparable comparison group.

^**Most frequent design of within group comparisons were graduates versus nongraduates.

^***Follow-up included all drug court participants or drug court graduates and the comparison group.

Reading down any column shows the characteristics for that evaluation report. The column for Orange County, as an example, indicates that this report (Deschenes et al. 1999) included a drug court program description (one point), process evaluation information (one point), descriptive data about the drug court (one point), and outcome data (one point). The study employed a matched comparison group design (four points), and demonstrated that the comparison group was similar to the drug court group (one point). The follow-up strategy employed a consistent time frame for each participant (one point) of at least one year duration (one point), and the same time frame was used for drug court and comparison groups (one point). Follow up included drug court participants (sometimes graduates only) and comparison group participants (one point), and outcome data were collected for more than 50% of those included in the study sample (one point). Information under the Measures component shows that the sample size was greater than 40 cases per group (one point), and that data were reported for arrests or convictions (one point), for drug court graduation or retention (one point), and for other outcome measures (one point). Demographic characteristics were reported for both the drug court (one point) and comparison (one point) groups, and analyses were used to compare differences between study groups (one point).

Reading across the columns indicates which characteristics of these reports are more and less common. For example, all of these relatively stronger evaluation reports included descriptive data about the study cohorts (e.g., demographics or criminal history), reported follow-up information on more than 50% of the sample, and reported arrest and/or incarceration rates as outcome measures. All of the reports, excepting Ventura County (Oberg 1996), provided some information on drug court graduation rates. Nine of the reports collected follow-up information for both drug court and comparison group participants, conducted follow-up for more than 40 cases per group, and provided demographic information for drug court participants. Most of the outcome evaluation efforts employed unmatched comparison group designs.

Main Outcome Findings of California Drug Court Evaluations

Table 3 summarizes main findings for those reports with a quality rating of 10 points or above. Included for each study is the sample size for the drug court group, the size and type of comparison group, rearrest rates for each group, and the drug court graduation rate. A visual comparison of rearrest rates (proportion rearrested during the follow-up period) for drug court and comparison groups in each study shows that rearrest rates were uniformly lower in the drug court group. This is consistent with other drug court literature reviews which suggest that, while participating in drug court, criminal activity is less (Belenko 2001).

TABLE 3.

Outcome Findings of California Drug Court Evaluations

County	Author	Year	Drug Court N	Comparison Group N, type	Outcomes
					Rearrest Rate	Graduation rate
					Drug Court Group	Comparison Group
Orange	Deschenes et al.	1999	236	234, probation	22%	34%	31%*
Riverside	Sechrest et al.	1998	102	243, eligible but prior to start of DC	14.7%	25.5%	50%**
Alameda	Tauber	1995	110	110, diversion	41%	55%	54%*
Ventura	Cosden, Crothers & Peerson	1999	235	66, eligible but not accepted in DC	20%***	−65%*	47%**
Monterey	Roehl	1998	41	51, nonviolent offenders	6% felony-11% misdemeanor ***	37% misdemeanor, 47% felony	23%*
Santa Barbara	Cosden, Peerson & Crothers	1999	120, graduates	196, nongraduates	27% ***	76%	38%*
Sacramento	Criminal Justice Research Foundation	1999	70, graduates	210, nongraduates, terminated	12.9%***	31%	11% – 15.3%*
San Francisco	Cabrera et. al.	1997	251	2801, possession & petty theft charges	12.8%***	46%	31%*
Ventura	Oberg	1996	75	219, applied for DC but not accepted	12%	32%	—
Mendocino	Hicks, Hicks & Bautista	1999	75	109 dropouts, others	16%*** (3 rearrests out of 19 graduates)	17%	19%**
Santa Clara	Santa Clara County	1998	338	325, eligible for DC but did not enter; 2713, deferred entry of judgment	13%	47% for eligible; 27% for deferred entry of judgment	61%**

^*As reported in paper.

^**Graduation rate calculated as the number of graduates divided by the number of graduates plus dropouts.

^***Graduates only

The difference in rearrest rates between the two groups is informative. In the Orange County study (Deschenes et al. 1999) for example, the difference between the drug court group rearrest rate (22%) and the comparison group rearrest rate (34%) is 12%. The difference between these rearrest rates was 10.8% in the Riverside study and 14% in the Alameda study. Some of the largest differences, however, occurred in studies where the drug court rearrest rate was either calculated or presented for drug court graduates only (Ventura 1999, Monterey, Santa Barbara, Sacramento, and San Francisco). In some instances (e.g., Ventura 1999) where the proportion rearrested was reported for drug court graduates and nongraduates separately, without giving the number of persons in each subgroup, we could not calculate an overall drug court rearrest rate. Rearrest rates reported in the last three studies (Ventura 1996, Mendocino, and Santa Clara) are less informative because they apparently applied inconsistent follow-up periods within and across study groups, and followed at least some of the sample for less than one year (see Table 2). Consequently, the Orange, Riverside, and Alameda county studies offer the best estimates of the difference in rearrest that drug courts may expect to produce, ranging from 10.8% to 14%.

The last column shows that drug court graduation rates vary widely, from a low of 11% to 15% in Sacramento County (Criminal Justice Research Foundation 1999) to a high of 61% in Santa Clara County (1998). Eliminating the highest (61%; Santa Clara County 1998) and lowest (11% to 15%; Criminal Justice Research Foundation 1999) graduation rates as likely outliers, graduation rates in these reports ranged from 19% to 54%. Interpreting graduation rates is hampered by two considerations. First, six studies reported graduation rates without mention of how these were calculated. For the four reports indicating how the rate was calculated (the number of graduates divided by the number of graduates plus drop-outs), graduation rates varied from 19% (Mendocino) to 61% (Santa Clara). Second, drug court programs sometimes use an initial “wash-out” period, during which drug court participants may be removed either by their own choice or by the drug court team. Such cases would depress the graduation rate if included in the calculation, and would inflate the graduation rate if excluded. The use of a wash-out period was mentioned in six of the 11 reports, with durations generally varying from seven to 30 days.

Main Cost Findings of California Drug Court Evaluations

Six reports offered estimates of costs or savings associated with drug courts. Table 4 summarizes estimated program costs, savings, and averted costs as reported in these evaluations. Wherever possible, information provided in the reports has been recalculated as an annualized figure. Reading down the Estimated Costs column shows that costs were reported for treatment only in Riverside, for both court and treatment in Monterey but using different metrics, as an annual total program cost (Sacramento and San Francisco), or as an average cost per enrollee (Sacramento and Mendocino). Estimated savings and averted costs are typically based on criminal justice indicators such as police and adjudication costs, and on jail, prison, probation, and parole costs. Two reports (Monterey and San Francisco) included other savings (or costs averted) to society, such as those for drug exposed infants, employment, health, and productivity. Two reports (Alameda and Sacramento) also reported on fees collected by the drug courts (data not shown).

TABLE 4.

Cost Findings of California Drug Court Evaluations

County	Author	Year	Estimated Costs	Estimated Savings or Costs Averted
Riverside	Sechrest et al.	1998	Annual treatment cost -$310,710	Annual net savings – $1,512,945
				Costs Averted for 102 Cases
				Prison-      $2,519,400
				Parole -       374,650
Alameda	Tauber	1995		Annual total savings - $1,047,412*
				Jail -       $646,412
				Police-         $61,000
				Probation -       $300,000
Monterey	Roehl	1998	Annual court costs - $125,000	Annual total savings -  $609,160
			Daily treatment costs –$6.79 - $61.47 per person	Costs Averted for 18 Graduates:
				Police-         $13,441
				Adjudication–    $34,681
				Corrections–      $50,094
				Jail days-        $10,944
				Drug-free babies – $500,000 +
Sacramento	Criminal Justice Research Foundation	1999	Annual total cost- $892,900
			Treatment- $519,168
			Court costs- $373,732
			Cost/enrollee – $3,247
			Cost/day/enrollee – $8.90
San Francisco	Cabrera et al.	1997	Annual total cost- $2,379,595	Annual total savings - $1,067,883
				Crime-related -     $565,369
				Health, productivity –$348,807
				Efficiency-     $153,707
Mendocino	Hicks, Hicks & Bautista	1999	Cost/enrollee - $3,900	$2,460 savings per drug court compared to nonparticipant
			Cost/day/enrollee – $6.50client

^*Dollar amounts given in evaluation reports for longer time periods have been adjusted to a one-year time-frame in the table.

Two reports provided sufficient data to calculate a cost versus savings ratio (San Francisco and Mendocino). In these two reports, each dollar expended may have resulted in respectively, savings of 45 cents and 63 cents. Numerous caveats are needed in interpreting these data. As seen in the table, different authors applied different approaches to estimating costs and savings and, with the exception of the San Francisco report, authors did not themselves offer a direct cost/savings comparison. Nevertheless, the table shows the nonstandardized nature of calculating and reporting costs and savings, and show that the cost-effectiveness of California drug courts is not demonstrated in reports up to January 2000.

DISCUSSION

This review points to four conclusions about the effectiveness of drug courts. First, drug courts may reduce rearrest rates among participants by 11% to 14%, as compared to nonparticipants. This is supported by findings from the three methodologically strongest studies reviewed (Deschenes et al. 1999; Sechrest et al. 1998; Tauber 1995), which suggest that drug courts have a positive, if modest, effect on outcomes. Second, the largest reductions in rearrest rates may be expected for drug court graduates, so that the effects of drug court may be dose-related (i.e., the degree of success is associated with the degree of exposure). Participants who remain in drug court to criterion (defined as graduation) tend to have better outcomes (lower rearrest rates). This is supported where rearrest rates of graduates were compared to those of nongraduates (e.g., Criminal Justice Research Foundation 1999), and where rearrest rates of graduates were compared to those of non-drug court samples (e.g., Cosden, Crothers & Peerson 1999). Third, based on the 11 studies shown in Table 3 and eliminating the highest and lowest graduation rates as likely outliers, drug courts may be expected to graduate 19% to 54% of participants. Because there is evidence that outcomes are better for program graduates, the overall effectiveness of drug courts may be enhanced by interventions to retain participants and increase graduation rates. Fourth, cost analyses in these reports do not permit conclusions concerning whether drug courts are or are not cost-effective.

Most of the studies reviewed were open to two sources of systematic bias: the use of comparison groups that differed from drug court groups, and the use of program wash out periods. Systematic differences between drug court and comparison groups at baseline, rather than the effects of the drug court itself, may account for differences observed at follow-up. If comparison group participants have more severe criminal histories at baseline, for example, they may also have more severe criminal histories at follow-up. A difference in rearrest rates at follow-up, in such cases, may be erroneously attributed to the drug court. None of the studies in this review applied a randomized design, and only one employed a matched comparison group design (Deschenes et al. 1999). While several studies did employ non-drug court comparison groups, only Deschenes and colleagues (1999) showed that the comparison and drug court groups were similar at baseline.

The use of “wash-out” periods in many drug courts may also bias evaluation findings. Those who leave drug court programs early on, whether by personal choice or by direction of the court, may also have more severe criminal and/or substance abuse histories. If they are eliminated from analyses, the overall severity of problems in the drug court group is decreased, and the likelihood of positive outcomes for the group is increased. This is more problematic to the degree that comparison groups have no similar early attrition period during which certain participants are removed.

In addition to problems of systematic bias, a number of evaluations neglected to report on important considerations. As given in Table 2, four reports offered no program description. Four reports did not apply consistent follow-up periods either across all cases or across all groups. Four did not report demographic characteristics for the non-drug court comparison group. Five reports, while providing information for arrest or conviction rates, did not perform statistical techniques to test the difference between groups. Outcome evaluation efforts were focused narrowly on rearrest and/or conviction measures, with only two studies considering additional outcomes. As given in Table 3, rearrest rates for drug court participants were sometimes calculated for program graduates only, or calculated for graduates and nongraduates separately without offering an overall rearrest rate for drug court participants. Graduation rates were calculated differently in different studies, limiting ability to compare rates across studies. Most cost analyses the authors reviewed were relatively undeveloped. Cost estimates did not regularly include both court costs and treatment costs, and did not consider opportunity costs associated with in-kind commitment of county resources. Estimates of savings also employed differing measures and calculation methods, limiting ability to compare findings across studies.

Deschenes and colleagues (In press) have offered a context for both the strengths and weaknesses of drug court evaluation literature nationally. A main strength was that evaluations were conducted early in the history of drug courts, under mandates from funding sources, enabling rapid development of descriptive, process, and outcome data. Many such evaluations were not well funded, so that weaker evaluation designs were often applied. In consequence, the drug court evaluation literature often suffers from methodological limitations, meta-analyses are prevented by a lack of standards in reporting outcomes, and the evidence for the effectiveness of drug courts is open to challenge. A related issue is that drug court evaluation reports are generally not available in journals, and must be obtained from other sources. Deschenes and colleagues (In press) note that, of 100 evaluations listed in the third edition of the Drug Court Publications Resource Guide (Freeman-Wilson & Wilkosz 2001), less than 25% were published. The lack of a published evaluation literature means that drug court research is relatively inaccessible, and that much of this literature has not met the quality assurance standard of peer-review.

This review of California drug court evaluations has important limitations. Some of the earliest California drug courts (with the exception of Alameda) started in 1995, and the reports reviewed here are from 1995 through January 2000. Many of the reports were first generation evaluations, conducted on relatively new programs that may not have achieved peak effectiveness. Since this review was begun, more recent and methodologically stronger evaluations likely have been conducted in California. Last, broader national literature reviews are now available (Belenko 2001, 1999, 1998) which may enable additional conclusions.

This review points to strategies for strengthening the evidence base supporting drug courts. As Deschenes and colleagues (In press) have commented, increasing funding for drug court evaluations would likely support stronger evaluation efforts and enhance the quality of findings. Those engaged in funding, leading, operating and evaluating drug courts may wish to consider the feasibility of randomized designs, and support such evaluation designs wherever possible. Randomized designs address directly the issue of systematic bias due to noncomparability between drug court and comparison groups. Evaluations using comparison groups would be strengthened by comparing groups at baseline on demographics and criminal history and, where differences are found, controlling for these in the outcome analysis. The conservative approach to comparisons of rearrest rates between study groups would retain in the analysis, for the drug court group, those who left drug court during any wash-out period. If such an intention-to-treat analysis is not possible, it remains important for evaluators to report on any wash-out period and the number of cases falling out during this period, so that the reader can assess the potential effect.

As drug courts benefited from the development of general practice standards included in the key components (Drug Court Programs Office 1997), so drug court evaluations may benefit from general evaluation standards. Such standards would be developed at a national level and with broad input from the field. The present review of California drug court evaluations, however, suggests issues that might be included in all evaluation reports. A description of the drug court program is vital to understanding the type of drug court being evaluated, and how findings from a given evaluation may relate to findings from other evaluations. Outcome evaluation reports will be stronger if they provide demographic information for both drug court and comparison groups, apply consistent follow-up observation periods, and use statistical analyses to test differences in outcomes between groups. In this review, some studies collected rearrest data at a single time point for all participants. Rearrest rates calculated using these data then reflect a different time period for each participant, and may reflect a different mean follow-up time period between drug court and comparison groups. Drug court evaluations may also benefit by using standard approaches to calculating rearrest and graduation rates, which would facilitate summarizing or comparing rates across studies. In the absence of accepted standards, evaluation reports should describe how these rates were calculated.

Cost analyses are needed to assess whether drug courts represent good investment of public funds, but there are few models to guide drug court cost evaluation efforts. The studies reviewed here were consistent in making efforts to estimate costs averted in the criminal justice system, but were not consistent in making efforts to estimate both court and treatment costs associated with the drug court intervention. Last, as the drug court movement graduates from a novel and promising intervention to a standard intervention deployed in community, criminal justice, and treatment systems, the evidentiary base supporting its effectiveness will be questioned. An accessible, peer-reviewed, and published drug court evaluation literature is needed to provide credible evidence of effectiveness, and to suggest strategies for improving intervention effects.