Abstract
This meta-analysis compares recidivism reduction in problem-solving courts employing judicial supervision in Australia and New Zealand to traditional processes. Using a four-phased search strategy, 16 studies totalling a treatment sample of 6588 individuals and 32,147 comparison participants were identified from 7161 unique records. Meta-analyses indicate that the problem-solving courts significantly reduced both the odds and incidences of recidivism compared with standard justice processes but that the heterogeneity observed within the latter analysis plus reliance on weak methodologies limits the strength of these conclusions. Studies at risk of bias may have had an undue influence on the odds of recidivism analysis. Additionally, the benefits of treatment on the incidence of recidivism are closely linked to the overlap of measurement and treatment periods. The findings suggest a positive impact from judicial supervision but further rigorous research is needed that closely matches experimental samples, strictly measures participants post-intervention and meticulously reports pertinent information.
Keywords: judicial supervision, judicial monitoring, problem-solving courts, recidivism, therapeutic jurisprudence
Problem-solving courts are those that aim to address both the illegality of an offence and its precipitating psychosocial issues (Casey & Rottman, 2005). Despite considerable variation in their aims and the populations they serve, there are several traits common to all problem-solving courts (Berman & Feinblatt, 2001). These include a focus on tangible outcomes, an overarching shift in the systemic response to social issues, a collaborative approach between the parties, the use of non-traditional roles that deemphasise the adversarial nature of the court process and finally judicial supervision, which comprises the review and monitoring of an offender’s compliance with a treatment programme or court order at designated ‘status review hearings’ conducted by a judicial officer.
The purpose of the current investigation is to systematically review and quantitatively synthesise extant research on problem-solving courts in Australia and New Zealand.
Although much of the problem-solving court movement in Australia and New Zealand has emerged through ‘grassroots’ judicial innovation (Richardson et al., 2013), these courts have required significant government expenditure to fund specialist staff and resource-intensive programme components such as access to medical and social services that are not offered in traditional justice processes (Bartels, 2009). For this reason, many of the courts in Australia and New Zealand have been subjected to some form of public evaluation (e.g. Harris, 2006; Knaggs et al., 2008), and a limited number of peer-reviewed scholarly articles have independently evaluated Australian problem-solving courts (e.g. Lim & Day, 2014). Much of this research has been reviewed in narrative form by Payne (2006) and Klinkum (2019), who examined problem-solving courts in Australia and New Zealand, respectively. Both authors argue that the literature generally shows that these courts appear to have been successful in achieving the common aim of reducing recidivism, with limited evidence also indicating that they are as cost effective as – if not cheaper than – the alternative of imprisonment. Nonetheless, each review is relatively brief and does not, for instance, detail the methodological quality of the existing problem-solving court research (Klinkum, 2019; Payne, 2006).
Conversely, reviews of Australian drug courts and Indigenous courts by Kornhauser (2018) and Marchetti (2017), respectively, have concluded that the existing quantitative assessments of Australian problem-solving courts possess numerous methodological flaws, such as poorly matched comparison groups (or no comparison group), treatment groups being represented by programme graduates and not reported on an intention-to-treat (ITT) basis and assessing participants based on time elapsed rather than time spent outside custody (time-at-risk, hereafter). Importantly, these issues have the potential to confound the results of the studies examined.
Both Kornhauser’s (2018) and Marchetti’s (2017) reviews nevertheless note numerous barriers to conducting rigorous investigations of the problem-solving courts in Australia. In some instances court programmes possess limited treatment places, which restricts sample numbers, or data is insufficient for selecting an appropriate comparison group. Kornhauser (2018) further concludes that because randomised studies – considered the ‘gold standard’ of outcome investigations (see Weisburd et al., 2001) – require significant governmental funding, planning and implementation well in advance of any evaluation, they have rarely been undertaken. To our knowledge, no attempt has been made to systematically review the overall quality of problem-solving court research in either Australia or New Zealand, nor has there been any effort to synthesise their findings.
The impact of problem-solving courts on recidivism has been subjected to extensive review internationally, and the results are varied. Several meta-analyses have found meaningful reductions in recidivism among intervention participants as compared with treatment-as-usual individuals in drug courts (Wilson et al., 2007), mental health courts (Sarteschi et al., 2011) and domestic violence courts (Gutierrez et al., 2016). Nonetheless, there is some suggestion from the same evidence base that these findings generally only hold among studies that are more susceptible to bias, whereas the analyses of more rigorous studies posit weaker or non-significant treatment effects (Gutierrez & Bourgon, 2012; Lowder et al., 2018).
Meta-analyses of problem-solving courts also tend to find considerable heterogeneity between the included studies (Sarteschi et al., 2011; Tanner-Smith et al., 2016). As such, researchers have attempted to unearth the sources of variance between courts via moderator analyses, with components such as the frequency of judicial supervision review hearings in the initial stages of drug court programmes found to moderate reductions in substance-related recidivism (Mitchell et al., 2012).
This last point is notable given that the growing body of literature examining judicial supervision in problem-solving courts reports conflicting results. Experiments in drug courts have found, for example, that participants who are randomly assigned to more frequent judicial supervision status hearings are less likely to accrue sanctions or return positive urinalysis tests in the initial stages of treatment (Jones, 2011, 2013), but that these results do not replicate when intensive supervision is compared with supervision ‘as needed’ (Marlowe et al., 2003).
With this research considered, three important questions remain unanswered regarding Australian and New Zealand problem-solving courts: what is the methodological study quality of the research on these courts, are these courts more effective at reducing recidivism compared with treatment-as-usual processes and how are the treatment effects of these courts moderated by programmatic characteristics – particularly judicial supervision? Within this study, we operationally define problem-solving courts as those that feature some form of judicial supervision in combination with a court-led intervention on an offending-related psychosocial issue.
Method
This review follows the Preferred Reporting Items for Systematic reviews and Meta‐Analyses (PRISMA) guidelines (Moher et al., 2009).
Protocol and registration
The systematic review protocol is registered with PROSPERO (registration number CRD42020187417; see Centre for Reviews and Dissemination, n.d.).
Eligibility criteria
Published and unpublished empirical studies in the English language reporting original research were accepted for inclusion.
Population
The population consists of individuals criminally charged with or convicted of any criminal offence who were made subject to a court list, court order or court programme.
Intervention
Any specialist or mainstream court list in an Australian or New Zealand jurisdiction that employed judicial supervision as part of its programme, sentence or court order was considered a valid intervention. Individuals could be accepted as participants before submitting a plea, after submitting a plea or after sentencing.
Comparator
Investigations that compared individuals who received a target intervention with a sample that did not via a randomised controlled trial or quasi-experiment were accepted.
Outcomes
Research that featured a measure of recidivism operationalised as rearrests, convictions, days incarcerated or adjudicated crimes was included. Outcomes were measured as incident counts, proportions or means. Self-report and qualitative measures were excluded. The measurement period could coincide with the start of the intervention period or could begin at exit.
Search strategy
A four-phased search strategy was employed to identify all records that met the above eligibility criteria. The first three phases were performed as part of a larger systematic review and meta-analysis of the international problem-solving court literature that is described in detail elsewhere (Trood et al., 2021). Briefly, these stages involved a preliminary database search to establish a search syntax (exemplified in Table A1) and a search of review repositories (e.g. The Cochrane Library) for existing systematic reviews in the topic area, a search of 9 bibliographic database platforms resulting in the extraction of 6001 records and a manual search of records from court research repositories and problem-solving court publications resulting in the individual extraction of a further 5035 records.
The additional stage performed as part of this review involved a search of sources of both published and unpublished literature specific to Australia and New Zealand. This was mainly comprised of a systematic search of records from relevant publication sections of governmental websites that led to the extraction of 921 records. An additional 201 records were extracted from the reference lists of three review articles of Australian or New Zealand problem-solving courts or diversion programmes, and another 45 records were obtained from other sources (e.g. found inadvertently). Together, 1167 records were extracted during the additional search performed for the present study, bringing the total records extracted to 12,203.
Study selection
The selection criteria were created, refined and adjusted by the first and second authors on three randomly selected batches of 200 records, with the third author resolving any eligibility disagreements where needed. The inter-rater reliability (IR) was calculated after each batch and the selection criteria were refined (κ = .43, n = 199, z = 6.84, p < .001, 73% agreement; κ = .44, n = 200, z = 7.17, p < .001, 78% agreement; and κ = .51, n = 200, z = 7.33, p < .001, 77% agreement). The first author reviewed the remaining records from the first three phases of the search strategy using the refined criteria.
The selection criteria were further modified in two ways for the present study: studies on populations outside Australian or New Zealand jurisdictions were excluded and post-test only measures were included. Using the modified selection criteria, the first author reassessed the 525 records excluded from the parent review at the ‘outcome’ stage, the final 62 records included in that review and the additional 1167 records extracted in the fourth search phase.
Data collection process
The first author extracted the information from the primary studies using an electronic coding form developed for the parent review that was adapted to the Australian and New Zealand context. The form sought information from the following subject areas: sample characteristics (e.g. age), methodological traits (e.g. follow-up length), publication details (e.g. year of publication), programmatic characteristics (e.g. jurisdiction), treatments offered or required (e.g. substance abuse treatment) and effect sizes (e.g. means, dichotomous measures).
Study quality
Study quality was assessed using a modified version of the Maryland Scientific Methods Scale (MSMS; Friendship et al., 2005) and is available in Table A2. Each included study was rated on the MSMS as either ‘3: Unmatched or Poorly matched’, ‘4: Well-matched’ or ‘5: Randomized’. A study was classed as ‘3: Unmatched or Poorly matched’ for one of four reasons: the study did not report sample matching; the study did not describe its matching technique; the study reported using a valid and reliable matching technique (i.e. one that is commonly employed in the research literature and considered a valid and reliable statistical technique for sample matching) but did not match participants on most of the important confounding variables; or the study reported employing an unreliable or invalid technique (e.g. simply testing samples for statistically significant differences on measured variables). For a study to be classed as ‘4: Well-matched’ it needed to report that a reliable and valid statistical matching technique had been used to match participants, that most of the important confounding variables had been controlled and that participants had not been randomised into their condition. Lastly, for a study to be rated ‘5: Randomized’ it had to report that the participants had been randomly allocated to the treatment and control conditions. The confounding domains were identified as likely predictors of the outcome of interest (e.g. criminal history) and are provided in Table A3. MSMS criteria were applied to the outcome, rather than the study, where the outcomes included in this review’s analysis employed controls for sample differences. The first and second authors blindly rated each included study and the IR was calculated (κ = .87, n = 20, z = 5.14, p < .001, 95% agreement). Differences were discussed and an agreed rating was established with assistance from the third author where required.
Synthesis of results
The odds ratio (OR) was chosen as the effect size where studies reported the number of each sample that reoffended during the follow-up period. In this context, an OR of 1 reflects no difference in the odds of recidivism between the treatment and control groups, an OR of less than 1 indicates that the treatment group possessed lower odds than the control group and the opposite is true for an OR of greater than 1. A recidivism rate was calculated from the OR for ease of interpretation using a formula employed by Wilson et al. (2007) based on the assumption that approximately 50% of individuals convicted in Australia or New Zealand eventually reoffend within 12 to 24 months (see Payne, 2007; Sullivan & Povey, 2015). Probit coefficients and their standard errors were rescaled into ORs using a scaling factor of 1.6 where necessary (Agresti, 2013). A random-effects model with inverse variance weighting and τ2 estimated using restricted maximum likelihood was selected to provide a summary effect for studies where the outcome reported was (or could be converted to) the number of each sample that reoffended during the follow-up period (Borenstein et al., 2009; Viechtbauer, 2005).
An additional effect size, the incidence rate ratio (IRR), was included to compare studies that reported sufficient information to calculate the number of recidivist incidents per group during their respective follow-up periods. An IRR of 1 similarly reflects no difference in the rate of recidivist events, an IRR of less than 1 indicates that the rate of reoffending was lower among the treatment group and an IRR of greater than 1 means that the rate of reoffending was lower among the comparison group. An IRR of 0.90, for example, indicates the estimated incidence of recidivist events was 10% lower among the treatment group (1 − 0.90 = 0.10). In line with Stijnen et al. (2010), free days were transformed to represent the number of events per 1000 free days. The summary IRR was modelled using a conditional generalised linear mixed model (GLMM) with exact likelihood (Stijnen et al., 2010).
The I2 statistic, signifying the proportion of variance in the observed effect sizes reflected in the variance in the true effects, was used as a measure of heterogeneity in all models (Borenstein, 2019). The Q statistic was used as a measure of residual heterogeneity among all linear and mixed-effects meta-analytic models and the Wald-type and likelihood ratio tests for residual heterogeneity were calculated on all GLMMs, with a significant p value for any of the three generally reflecting a meaningful amount of variance between the effect sizes modelled (Borenstein et al., 2009; Viechtbauer, 2007). Lastly, credibility intervals, which estimate a range of values where 95% of future effects would occur, were calculated on all GLMMs and random-effects models (Borenstein et al., 2009). All effect sizes, summary effects and their respective confidence intervals and credibility intervals were calculated in logarithmic units and exponentiated for ease of interpretation.
Moderator analyses
Meta-regressions and subgroup analyses were performed on all continuous and categorical moderators, respectively. Subgroup differences were tested for statistical significance via the Z-test method (Borenstein et al., 2009). A Benjamini–Hochberg procedure was employed to reduce the familywise error rate among moderators which were grouped into the following categories irrespective of whether they were categorical or continuous: programmatic, methodological, treatments and sample characteristics (Benjamini & Hochberg, 1995).
Results
Study selection
Figure 1 illustrates the flow of study selection. The search strategy resulted in the combined extraction of 7161 unique records, of which 1570 records were screened for inclusion in the present study. From this figure, 1249 records were excluded at the screening stage. A total of n = 301 records were excluded after full-text assessment for the following reasons: population (1.33%), intervention (35.88%), no comparison group (9.30%), outcome (4.98%), insufficient information reported (0.66%), jurisdiction (47.18%) or the record presented secondary research (0.66%). The remaining 20 records – representing 16 independent studies – were included in the review, with a subset of 15 studies (k = 19 outcomes) included in the meta-analyses.
Figure 1.
Prisma diagram (Moher et al., 2009) illustrating the flow of study selection.
Study characteristics
The characteristics of the 16 independent studies are presented in Table 1. The investigations were conducted between 1999 and 2020 and the majority have not been published in peer-reviewed journals (68.75%). The most commonly studied jurisdiction is Victoria (25.00%), while 18.75% were each conducted in New South Wales, New Zealand and Queensland, 12.50% in Western Australia and 6.25% in South Australia. Drug courts are the most common setting (56.25% of studies), followed by diversionary mainstream court lists (18.75%), community courts (12.50%) and family violence court and youth drug court (6.25% each).
Table 1.
Key characteristics of the 16 independent studies included in the current study.
| Study | Jurisdiction | Programme duration (months) | Court intervention | Intention-to-treat | Treatment sample | Comparison sample(s) | Post-test period | Measures | MSMS score |
|---|---|---|---|---|---|---|---|---|---|
| Crime Research Centre (2003) | WA | 6 (min.) | Post-plea DC; charge level NR; did not accept violent charges | Graduates (53.42%); terminates (46.57%) | n = 219 offenders on either STIR or DCR who were dependent on illicit drugs | n = 219 offenders apprehended on a drug offence | Up to 24 months from enrolment; measures taken on free time | Rearrests | 3 |
| Department of the Attorney General (WA) (2006) | WA | NR | Post-plea DC; charge information NR | Graduates only | n = 194 programme graduates for whom there was sufficient information |
n = 150 CJS offenders identified as having a substance use problem who had completed a CBO n = 214 offenders identified as having a substance use problem who had exited prison |
Each participant tracked for 24 months from treatment exit; measures taken on elapsed time | Offences | 3 |
| Heale and Lang (1999, 2001) | VIC | 2.3 (min.) | Pre-plea MCL; only accepted indictable offences | NR | n = 89 offenders accepted into the programme | n = 92 offenders referred to the programme who did not participate | Each participant tracked for 2.76 months; time status of measure NR | Offences | 3 |
| Lind et al. (2002), Shanahan et al. (2004) | NSW | 12 (min.) | Post-plea DC; accepted summary and indictable offences; did not accept violent charges | Graduates (4%); terminates (43%)* | n = 309 offenders dependent on illicit drugs randomly allocated to DC | n = 191 offenders eligible for DC for whom there was no place available in detox | Treatment: 243 days (avg) from programme referral; Comparison: 145 days (avg) and comparison respectively for treatment; measures taken on both free and elapsed time | Theft or drug offences | 5 |
| Lulham (2009) | NSW | 3 (min.) | Pre-plea (bail stage) of MCL; only accepted summary offences; did not accept violent charges | Graduates (68.36%); terminates (31.64%) | n = 2396 offenders with a demonstrable illicit drug problem | n = 23,960 offenders who had not participated in MERIT and had not been charged with driving offences | Each participant tracked for 24 months from index hearing; measures taken on elapsed time | Offences | 4 |
| Makkai and Veraar (2003) | QLD | 15.4 (avg) | DC; charge level and stage of disposition NR; accepted violent charges | Graduates (34.36%); terminates (42.86%)* | n = 259 offenders predominantly charged with property offences and few prior violent offences |
n = 89 offenders who refused to participate in the intervention n = 107 individuals released from prison who had an eligible qualifying arrest |
Treatment: 523 days (avg) from treatment entry; Comparison (refused): 611 days (avg); Comparison (prisoners): 575 days (avg) from treatment exit; measures taken on free time | Offence episodes | 3 |
| New Zealand Ministry of Justice Tāhū o te Ture (2019) | NZ | NR | Post-plea DC; charge level NA; accepted violent charges | Graduates (45.52%); terminates (54.48%) | n = 290 offenders with a moderate to severe substance use disorder and/or charged with their third or a subsequent drink driving offence | n = 553 offenders released from prison (141 sentenced in Auckland; 412 national sample sentenced outside Auckland) | Up to 48 months programme post-entry (treatment) and post-exit (comparison); measures taken on free time | Proven offences | 4 |
| Payne (2005) | QLD | 7.9 (avg) | Post-plea DC; charge level NR; accepted violent charges | Graduates (20.0%); terminates (37.5%)* | n = 120 drug dependent offenders whose dependency contributed to the commission of their index offence and who were admitted to treatment | n = 36 drug dependent offenders whose dependency contributed to the commission of their index offence and who refused admission to treatment | Treatment: 313 days (avg) from treatment enrolment; Comparison: 312 days (avg); measures taken on free time | Offences | 3 |
| Payne (2008) | QLD | 10.8 (avg) | Post-plea DC; charge level NR; accepted violent charges | Graduates (50%); terminates 50%) |
n = 200 individuals charged with a relevant offence who were assessed as drug dependent | n = 107 offenders released from prison whose offending was seemingly drug related | Treatment: 1280 days (avg) from treatment exit; Comparison: 1848 days (avg); some measures taken on free time | Offences | 3 |
| Ross (2009), PricewaterhouseCoopers (2009) | VIC | 3.6 (avg) | Pre-plea (bail stage) MCL; accepted both summary and indictable offences; accepted violent charges | Graduates only | n = 200 offenders screened and accepted as having an identifiable problem (e.g. homelessness) | n = 200 offenders sentenced at other venues | Treatment: 670 days (avg) from treatment exit; Comparison: 720 days (avg); measures taken on free time | Proven offences | 3 |
| Ross (2015) | VIC | NR | CC; accepted summary offences; accepted violent offences | NR | n = 187 offenders | n = 187 offenders | Each participant tracked for 24 months from enrolment; measures taken on elapsed time | Proven offences | 3 |
| Ross et al. (2009) | VIC | NR | Post-plea CC; accepted summary offences | Graduates only | n = 100 offenders sentenced in NJC who had received an intervention through NJC client services | n = 200 offenders sentenced at other courts with similar index offences | Each participant tracked for 8 months from treatment exit; time status of measure NR | Proven offences | 3 |
| Searle and Spier (2006) | NZ | 10 (avg) | Post-plea JDC; charge level NA; accepted violent charges | Graduates (57%); terminates (43%) | n = 30 recidivist offenders aged 14 to 16 years with a moderate to severe substance dependency linked to their offending behaviour |
n = 952 offenders sentenced in the National Youth Court n = 120 offenders comprising a partially matched subset of those sentenced in the National Youth Court |
Each participant tracked for 12 months from treatment exit (treatment) or case finalisation date (comparisons); measures taken on free time | Proven offences | 3 |
| Soboleva and Su-Wuen (2008) | NZ | NR | Post-plea DVC; accepted summary and indictable offences; accepted violent charges | NR |
n = 835 offenders from 2001 to 2005 convicted or discharged without conviction for FV offences; n = 274 offenders from 2005 to 2006 convicted or discharged without conviction for FV offences |
n = 719 offenders from 1998 to 2001 convicted or discharged without conviction for FV offences in Waitakere; n = 3327 offenders discharged without conviction for FV offences (national sample) |
Each participant tracked for 12 months from case finalisation date; time status of measure NR | Proven offences | 3 |
| Weatherburn et al. (2008), Weatherburn et al. (2020b)♦ | NSW | 12 (min.) | Post-plea DC; accepted summary offences; accepted violent charges | Graduates (37.46%); terminates (55.66%)* | n = 645 offenders accepted into the eligibility assessment phase of the DC | n = 329 offenders rejected from the DC due to an ineligible index offence or residence outside the catchment area | Participants followed for approximately 1350 days (max.) from index hearing; measures taken on free time | Proven offences | 3 |
| Graduates (39.90%); terminates NR | n = 604 treatment subjects that participated in Weatherburn et al. (2008) | n = 306 comparison subjects that participated in Weatherburn et al. (2008); | Participants followed for 3864 days (avg) from index hearing; measures taken on free time | Proven offences | 4 | ||||
| Ziersch and Marshall (2012) | SA | 12 (min.) | Post-plea DC; charge level NR; violent charges not accepted | Graduates (39.83%); terminates (60.17%) | n = 241 offenders charged with an offence related to their drug use that would likely result in prison time | n = 176 prisoners who had tested positive for illicit drugs during their time in prison | Each participant was tracked for 24 months from treatment exit; measures taken on free time | Offences | 3 |
Note. *The intention-to-treat percentages exclude active participants and do not sum to 100%; ♦Weatherburn et al. (2020b) has been published since the completion of the search phase of the present study and the updated citation is included in the reference list (Weatherburn et al., 2020a); avg = average; CBO = Community Based Order; CC = community court; CJS = community justice services; DC = drug court; DCR = drug court regime; DVC = domestic violence court; FV = family violence; graduates = individuals that completed the court programme; JDC = juvenile drug court; MCL = mainstream court list; MERIT = Magistrates Early Referral Into Treatment; NA = not applicable; NJC = neighbourhood justice centre; NR = not reported; NSW = New South Wales; NZ = New Zealand; post-plea = programme entered after submitting a plea; pre-plea = programme entered before submitting a plea; QLD = Queensland; SA = South Australia; STIR = supervised treatment intervention regime; terminates = individuals that did not complete the court programme; VIC = Victoria; WA = Western Australia.
With respect to treatment components, 25.00% of programmes reported measuring criminogenic risk among the treatment sample, 87.50% reported individualising treatment, 100.00% included a drug treatment component (although it was only mandatory in 75.00%), 25.00% offered mental health treatment, 68.75% included counselling, 12.50% offered a domestic violence programme, 37.50% offered or required a component of vocational or educational training, 31.25% offered anger management and 31.25% included released prisoners as a comparison sample.
The combined treatment sample includes 6588 individuals with an approximate mean age of 30.06 years. The treatment sample is approximately 83.10% male, 22.84% Aboriginal or Torres Strait Islander (ATSI) for Australian studies or Maori for those conducted in New Zealand, 51.19% programme graduates and 33.28% programme terminates. The combined comparison sample totals 32,147 persons with an approximate mean age of 30.16 years, of whom 81.87% are male and 23.48% are ATSI or Maori.
The majority of included studies (75.00%) were rated ‘3: Unmatched or Poorly matched’, which can be further broken down into 31.25% that did not report matching the samples, 25.00% that did not describe their matching technique, 12.50% that used a valid and reliable matching technique but did not match participants on most of the important confounding variables and 6.25% that employed an unreliable or invalid matching technique. Of the remaining studies, three (18.75%) were rated as ‘4: Well-matched’ and one (6.25%) was rated as ‘5: Randomized’.
Treatment effects
Recidivism outcomes from all but 1 study were able to be included in the odds of recidivism analysis (n = 12, 75.00%) or the incidence of recidivism analysis (n = 7, 43.75%), with 4 studies providing an outcome for both analyses. Within the odds of recidivism analysis, k = 1 (8.33%) outcome was converted to an odds ratio from a probit coefficient and k = 1 (8.33%) outcome was calculated with data extracted from a survival analysis plot using WebPlotDigitizer software (Rohatgi, 2019). Of the k = 12 outcomes included in the odds of recidivism analysis, 25.00% included only graduates, 50.00% included both graduates and failures and the remaining 25.00% did not report the ITT of the treatment participants. Furthermore, 25.00% reported that the follow-up period was based on elapsed time, 41.67% reported that the follow-up was based on time-at-risk and 33.33% did not report the time status of the follow-up. Conversely, all k = 7 (100.00%) outcomes in the incidence of recidivism analysis included both graduates and failures, and measured participants during free time. All descriptive information for the odds of recidivism and incidence of recidivism analyses are available in Tables A4, A5 and A6 of the supplementary materials, respectively.
Table 4.
Comparison of meta-analytic models: odds of recidivism.
| Model | OR | 95% CI | p value | Credibility interval | Q | I 2 |
|---|---|---|---|---|---|---|
| Random-effects model | 0.80 | (0.67, 0.95) | .01* | (0.53, 1.20) | 23.33* | 45.61 |
| Fixed-effect model | 0.92 | (0.83, 1.01) | .09 | – | 23.33* | 52.85 |
| Random-effects model (graduates only) | 0.50 | (0.29, 0.85) | .01* | (0.11, 2.25) | 42.67*** | 88.51 |
Note. *p < .05; ***p < .001; CI = confidence interval; OR = odds ratio.
Table 5.
Comparison of meta-analytic models: incident rate ratio of recidivist events per 1000 days.
| Model | IRR | 95% CI | p value | Credibility interval | Wald-type test | Likelihood ratio test | Df | I 2 | Deviance | p value |
|---|---|---|---|---|---|---|---|---|---|---|
| GLMM: Conditional model with exact likelihood | 0.80 | (0.65, 0.99) | .036 | (0.46, 1.41) | 94.70*** | 93.02*** | 6 | 94.38 | 0.66 | .995 |
| GLMM: Unconditional model with fixed study effects | 0.80 | (0.65, 0.98) | .034 | (0.46, 1.40) | 94.70*** | 93.02*** | 6 | 94.27 | 0.68 | .995 |
| GLMM: Unconditional model with random study effects | 0.80 | (0.65, 0.98) | .034 | (0.46, 1.40) | 94.70*** | 92.76*** | 6 | 94.30 | 26.83 | <.001*** |
Note. ***p < .001; CI = confidence interval; Df = degrees of freedom; GLMM = generalised linear mixed model; IRR = incidence rate ratio;
The pooled estimate for the odds of recidivism in addition to each study’s effect size is depicted in Figure 2. The effect sizes range from 0.55 to 1.13 and the pooled OR is 0.80 (95% CI [0.67, 0.95], p = .01). This difference is statistically significant and, compared to a 50.00% recidivism rate of a typical offender sample, is analogous to a reduction to 44.45%. There is a moderate amount of heterogeneity (Q = 23.33, p = .02, I2 = 45.61, credibility interval [0.53, 1.20]) between the k = 12 outcomes, with 45.61% of the variance between effect sizes reflecting the variance in the true effects.
Figure 2.
Meta-analysis of the odds of recidivism.
Note. OR values below 1 favour treatment.
Figure 3 illustrates the pooled estimate and effect sizes for the k = 7 outcomes included in the IRR analysis of recidivist events. The effect sizes range from 0.54 to 1.27 and the pooled IRR is 0.80 (95% CI [0.65, 0.99], p = .036), suggesting that the incidence of recidivist events is 20.00% lower among treatment participants compared with non-exposed individuals and that this estimate is marginally statistically significant. Furthermore, there is marked heterogeneity between the studies modelled (I2 = 94.38, credibility interval [0.46, 1.41]).
Figure 3.
Meta-analysis of the recidivist incidents.
Note. IRR values below 1 favour treatment.
Moderator analysis
Tables 2 and 3 present the combined results of the moderator analyses for both the odds and incidence of recidivism outcomes, with the former showing the methodological and programmatic moderators and the latter depicting the sample characteristics and treatment moderators. Among the odds of recidivism analysis, significantly weaker treatment effects were found for New Zealand courts (OR = 1.11, 95% CI [0.97, 1.27]) compared to both Victoria (b = 0.65, adjusted p < .05) and Australia (b = 0.64, adjusted p < .001) generally. Surprisingly, stronger treatment effects were also observed among the studies that did not report offering anger management to the treatment participants (OR = 0.71, 95% CI [0.61, 0.83]) compared with those that did (b = 1.50, adjusted p < .05). This is a peculiar result and will be explicated in the discussion.
Table 2.
Methodological and programmatic moderators: odds of recidivism & incidence of recidivism.
| Odds of recidivism |
Incidence of recidivism |
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Moderator | Level | k | OR | 95% CI | I 2 | p value b/w | k | IRR | 95% CI | I 2 | p value b/w |
| Methodological | |||||||||||
| Intention-to-treat | Graduates (ref) | 3 | 0.67 | (0.52, 0.88) | 20.22 | – | – | – | – | – | – |
| Graduates/Terminates | 6 | 0.82 | (0.65, 1.02) | <0.001 | .28 | – | – | – | – | – | |
| Not reported | 3 | 0.84 | (0.56, 1.27) | 73.40 | .37 | – | – | – | – | – | |
| Measure | New offence (ref) | 4 | 0.67 | (0.53, 0.85) | 3.77 | – | – | – | – | – | – |
| New proven offence | 7 | 0.86 | (0.69, 1.07) | 47.28 | .14 | – | – | – | – | – | |
| Measured during or post treatment | During (ref) | 5 | 0.74 | (0.59, 0.93) | <0.001 | – | 5 | 0.71 | (0.59, 0.85) | 90.95 | – |
| Post | 7 | 0.82 | (0.65, 1.04) | 58.71 | .53 | 2 | 1.07 | (0.85, 1.35) | 74.13 | .006* | |
| MSMS rating | Beta weight | 12 | 1.04 | (0.63, 1.69) | 48.03 | .89 | 7 | 0.81 | (0.64, 1.02) | 91.45 | .07 |
| MSMS rating recode (a) | Beta weight | 12 | 1.38 | (0.90, 2.13) | 35.98 | .14 | 7 | 0.84 | (0.64, 1.10) | 92.84 | .21 |
| MSMS rating recode (b) | Beta weight | 12 | 1.54 | (1.24, 1.92) | 1.95 | < .001** | 7 | 0.90 | (0.68, 1.20) | 93.76 | .47 |
| Outcome | Offences (ref) | – | – | – | – | – | 5 | 0.78 | (0.67, 0.91) | 82.37 | – |
| Proven offences | – | – | – | – | – | 2 | 0.82 | (0.45, 1.48) | 97.24 | .88 | |
| Post-treatment (months) (Con) | Beta weight | 7 | 0.99 | (0.98, 1.01) | 3.24 | .66 | – | – | – | – | – |
| Post-treatment (months) (Tr) | Beta weight | 7 | 0.98 | (0.95, 1.01) | <0.001 | .21 | 4 | 1.05 | (1.02, 1.09) | 72.60 | .003* |
| Post-test period (months) | Beta weight | 11 | 0.99 | (0.98, 1.01) | 43.85 | .39 | – | – | – | – | – |
| Publication type | Evaluation (ref) | 10 | 0.82 | (0.68, 0.99) | 46.75 | – | – | – | – | – | – |
| Journal article | 2 | 0.70 | (0.52, 0.95) | <0.001 | .40 | – | – | – | – | – | |
| Study groups | Combined comparison (ref) | 2 | 0.80 | (0.39, 1.63) | 92.70 | – | – | – | – | – | – |
| Standard | 10 | 0.77 | (0.65, 0.90) | <0.001 | .91 | – | – | – | – | – | |
| Time status of measure | Elapsed time (ref) | 3 | 0.63 | (0.50, 0.80) | 1.83 | – | – | – | – | – | – |
| Free time | 5 | 0.82 | (0.65, 1.04) | <0.001 | .13 | – | – | – | – | – | |
| Not reported | 4 | 0.92 | (0.69, 1.21) | 45.45 | .05 | – | – | – | – | – | |
| Year of publication | Beta weight | 12 | 1.11 | (0.43, 2.87) | <0.001 | .83 | 7 | 0.99 | (0.96, 1.02) | 92.89 | 0.42 |
| Programmatic | |||||||||||
| Accepted violent offenders | No (ref) | 3 | 0.79 | (0.60, 1.03) | <0.001 | – | 2 | 0.77 | (0.63, 0.95) | 78.23 | – |
| Yes | 7 | 0.87 | (0.70, 1.10) | 44.82 | .57 | 5 | 0.81 | (0.61, 1.08) | 95.17 | .77 | |
| Charge level | Summary (ref) | 3 | 0.71 | (0.55, 0.92) | <0.001 | – | – | – | – | – | – |
| Summary/indictable | 3 | 0.96 | (0.73, 1.27) | 49.17 | .12 | – | – | – | – | – | |
| Court setting | Community court (ref) | 2 | 0.66 | (0.48, 0.91) | <0.001 | – | – | – | – | – | – |
| Drug court | 6 | 0.71 | (0.57, 0.89) | 4.56 | .69 | – | – | – | – | – | |
| Mainstream court | 3 | 0.80 | (0.62, 1.05) | <0.001 | .35 | – | – | – | – | – | |
| Frequency of review hearings | Fortnightly (ref) | – | – | – | – | – | 2 | 0.82 | (0.45, 1.48) | 97.24 | – |
| Weekly | – | – | – | – | – | 3 | 0.79 | (0.65, 0.97) | 85.00 | .91 | |
| Jurisdiction | New Zealand (ref) | 3 | 1.11 | (0.97, 1.27) | <0.001 | – | 2 | 0.82 | (0.45, 1.48) | 97.24 | – |
| Victoria | 4 | 0.72 | (0.57, 0.91) | <0.001 | .001* | – | – | – | – | – | |
| Queensland | 2 | 0.80 | (0.50, 1.26) | <0.001 | .17 | 2 | 0.82 | (0.57, 1.20) | 79.55 | 1.00 | |
| Australia | 9 | 0.71 | (0.61, 0.83) | <0.001 | < .001*** | 5 | 0.78 | (0.67, 0.91) | 82.37 | .88 | |
| New South Wales | – | – | – | – | – | 2 | 0.73 | (0.66, 0.80) | 54.98 | .69 | |
| Programme duration average (months) | Beta weight | 4 | 1.01 | (0.92, 1.10) | <0.001 | .90 | 3 | 0.91 | (0.86, 0.96) | 16.52 | < .001** |
| Programme duration minimum (months) | Beta weight | 4 | 0.99 | (0.94, 1.06) | <0.001 | .92 | – | – | – | – | – |
| Reported accepting indictable charges | No (ref) | 9 | 0.71 | (0.60, 0.84) | <0.001 | – | – | – | – | – | – |
| Yes | 3 | 0.96 | (0.73, 1.27) | 49.17 | .07 | – | – | – | – | – | |
| Reported accepting violent offenders | No (ref) | 5 | 0.70 | (0.57, 0.85) | <0.001 | – | – | – | – | – | – |
| Yes | 7 | 0.87 | (0.70, 1.10) | 44.82 | .15 | – | – | – | – | – | |
| Reported frequent review hearings | No (ref) | 8 | 0.78 | (0.64, 0.96) | 59.20 | – | 3 | 0.77 | (0.67, 0.88) | 66.07 | – |
| Yes | 4 | 0.84 | (0.60, 1.17) | <0.001 | .72 | 4 | 0.83 | (0.59, 1.18) | 95.96 | .68 | |
| Reported frequent team meetings | No (ref) | 8 | 0.78 | (0.63, 0.96) | 58.65 | – | 2 | 0.80 | (0.74, 0.86) | <0.001 | – |
| Yes | 4 | 0.84 | (0.63, 1.12) | <0.001 | .66 | 5 | 0.79 | (0.59, 1.05) | 95.08 | .94 | |
| Reported single judicial officer | No (ref) | 9 | 0.85 | (0.70, 1.02) | 41.25 | – | 5 | 0.76 | (0.62, 0.93) | 92.78 | – |
| Yes | 3 | 0.66 | (0.48, 0.91) | 18.24 | .19 | 2 | 0.90 | (0.57, 1.44) | 93.45 | .51 | |
| Reported team training | No (ref) | 10 | 0.80 | (0.66, 0.97) | 50.69 | – | – | – | – | – | – |
| Yes | 2 | 0.78 | (0.57, 1.07) | <0.001 | .91 | – | – | – | – | – | |
| Reported use of graduated sanctions | No (ref) | – | – | – | – | – | 4 | 0.87 | (0.65, 1.17) | 92.25 | – |
| Yes | – | – | – | – | – | 3 | 0.72 | (0.56, 0.92) | 92.90 | .32 | |
| Reported use of rewards | No (ref) | 9 | 0.79 | (0.65, 0.97) | 55.35 | – | 2 | 1.07 | (0.85, 1.35) | 74.13 | – |
| Yes | 3 | 0.80 | (0.56, 1.16) | <0.001 | .95 | 5 | 0.71 | (0.59, 0.85) | 90.95 | .006* | |
| Reported use of sanctions or graduated sanctions | No (ref) | 7 | 0.78 | (0.62, 0.98) | 63.18 | – | – | – | – | – | – |
| Graduated | 2 | 0.79 | (0.56, 1.11) | <0.001 | .95 | – | – | – | – | – | |
| Sanctions | 3 | 0.85 | (0.57, 1.27) | <0.001 | .71 | – | – | – | – | – | |
| Sanctions or graduated sanctions | 5 | 0.81 | (0.63, 1.06) | <0.001 | .81 | – | – | – | – | – | |
| Stage of disposition | Post-plea (ref) | 7 | 0.92 | (0.75, 1.13) | 31.98 | – | – | – | – | – | – |
| Pre-plea | 3 | 0.80 | (0.62, 1.05) | <0.001 | .44 | – | – | – | – | – | |
Note. *p < .05; **p < .01; ***p < .001; Con = variable measured among control condition; post-plea = programme entered after submitting a plea; pre-plea = programme entered before submitting a plea; ref = reference group; Tr = variable measured among treatment condition; p value b/w asterisks denote significance level of the difference between the moderator and reference category after Benjamini–Hochberg adjustment.
Table 3.
Sample characteristics and treatment moderators: odds of recidivism & incidence of recidivism.
| Odds of recidivism |
Incidence of recidivism |
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Moderator | Level | k | OR | 95% CI | I 2 | p value b/w | k | IRR | 95% CI | I 2 | p value b/w |
| Sample characteristics | |||||||||||
| Mean age (Con) | Beta weight | 3 | 0.95 | (0.86, 1.05) | <0.001 | .35 | 5 | 1.05 | (1.02, 1.08) | <0.001 | < .001** |
| Mean age (Tr) | Beta weight | 3 | 0.96 | (0.87, 1.05) | <0.001 | .34 | 5 | 1.05 | (1.02, 1.09) | <0.001 | .001** |
| Pre-treatment mean (Con) | Beta weight | – | – | – | – | – | 3 | 1.19 | (1.07, 1.32) | <0.001 | .001** |
| Pre-treatment mean (Tr) | Beta weight | – | – | – | – | – | 3 | 1.05 | (0.94, 1.18) | 57.40 | .35 |
| Proportion ATSI or Maori (Con) | Beta weight | 7 | 1.00 | (0.99, 1.02) | 58.32 | .59 | 4 | 0.99 | (0.98, 1.01) | 92.92 | .56 |
| Proportion ATSI or Maori (Tr) | Beta weight | 6 | 1.01 | (0.99, 1.02) | 23.29 | .39 | 4 | 0.99 | (0.98, 1.01) | 92.65 | .53 |
| Proportion male (Con) | Beta weight | 8 | 1.00 | (0.98, 1.03) | 55.35 | .70 | 6 | 0.98 | (0.96, 0.99) | 84.44 | .002** |
| Proportion male (Tr) | Beta weight | 8 | 1.02 | (1.01, 1.04) | <0.001 | < .001*** | 6 | 0.97 | (0.96, 0.99) | 85.35 | .004** |
| Proportion that completed order (Tr) | Beta weight | 7 | 0.99 | (0.99, 1.00) | <0.001 | .15 | 3 | 1.01 | (1.00, 1.02) | 60.23 | .03 |
| Proportion that were graduates (Tr) | Beta weight | 9 | 0.99 | (0.99, 1.00) | <0.001 | .30 | 7 | 1.01 | (0.99, 1.02) | 92.89 | .34 |
| Proportion that were terminates (Tr) | Beta weight | 8 | 1.00 | (0.99, 1.01) | <0.001 | .28 | 6 | 0.99 | (0.96, 1.02) | 93.14 | .44 |
| Treatment | |||||||||||
| Reported anger management (Tr) | No (ref) | 8 | 0.71 | (0.61, 0.83) | <0.001 | – | 4 | 0.71 | (0.59, 0.85) | 91.50 | – |
| Offered | 4 | 1.06 | (0.87, 1.29) | 9.94 | .002* | 3 | 0.96 | (0.68, 1.35) | 88.50 | .12 | |
| Reported any treatment offered (Con) | No (ref) | 8 | 0.86 | (0.70, 1.05) | 43.69 | – | – | – | – | – | – |
| Yes | 4 | 0.68 | (0.54, 0.86) | 4.57 | .14 | – | – | – | – | – | |
| Reported CBT treatment (Tr) | No (ref) | 9 | 0.80 | (0.65, 0.98) | 54.69 | – | 4 | 0.77 | (0.57, 1.04) | 96.88 | – |
| Offered or required | 3 | 0.79 | (0.58, 1.07) | <0.001 | .93 | 3 | 0.85 | (0.66, 1.08) | 77.58 | .62 | |
| Reported counselling (Tr) | No (ref) | 3 | 0.61 | (0.47, 0.78) | <0.001 | – | – | – | – | – | – |
| Required | 2 | 0.77 | (0.55, 1.08) | <0.001 | .27 | – | – | – | – | – | |
| Offered | 7 | 0.92 | (0.76, 1.12) | 31.89 | .009 | – | – | – | – | – | |
| Reported detox treatment (Tr) | No (ref) | 8 | 0.79 | (0.63, 0.99) | 58.08 | – | 4 | 0.90 | (0.65, 1.25) | 93.70 | – |
| Offered | 4 | 0.80 | (0.63, 1.02) | <0.001 | .97 | – | – | – | – | – | |
| Offered or required | – | – | – | – | – | 3 | 0.71 | (0.65, 0.78) | 50.29 | .17 | |
| Reported domestic violence programme (Tr) | No (ref) | 10 | 0.71 | (0.61, 0.84) | <0.001 | – | – | – | – | – | – |
| Offered | 2 | 1.01 | (0.74, 1.38) | 57.37 | .05 | – | – | – | – | – | |
| Reported drug or alcohol testing (Tr) | No (ref) | 7 | 0.86 | (0.69, 1.06) | 48.24 | – | – | – | – | – | – |
| Offered or required | 5 | 0.69 | (0.56, 0.86) | 1.28 | .17 | – | – | – | – | – | |
| Reported drug or alcohol treatment (Tr) | Offered (ref) | 4 | 0.84 | (0.62, 1.14) | 67.19 | – | – | – | – | – | – |
| Required | 8 | 0.73 | (0.61, 0.88) | <0.001 | .43 | – | – | – | – | – | |
| Reported housing support (Tr) | No (ref) | 6 | 0.83 | (0.64, 1.08) | 62.76 | – | 3 | 0.95 | (0.75, 1.20) | 88.64 | – |
| Offered | 6 | 0.74 | (0.60, 0.91) | <0.001 | .49 | 4 | 0.69 | (0.55, 0.88) | 90.58 | .06 | |
| Reported life-skills training (Tr) | No (ref) | 7 | 0.83 | (0.66, 1.04) | 58.45 | – | 3 | 0.95 | (0.75, 1.20) | 88.64 | – |
| Offered | 5 | 0.72 | (0.57, 0.92) | <0.001 | .40 | – | – | – | – | – | |
| Offered or required | – | – | – | – | – | 7 | 0.80 | (0.65, 0.99) | 94.38 | .28 | |
| Reported material aid (Tr) | No (ref) | 8 | 0.79 | (0.63, 0.99) | 57.83 | – | 4 | 0.75 | (0.64, 0.88) | 81.03 | – |
| Offered | 4 | 0.80 | (0.63, 1.01) | <0.001 | .94 | 2 | 0.82 | (0.45, 1.48) | 97.24 | .78 | |
| Reported measuring risk (Con) | No (ref) | 10 | 0.85 | (0.71, 1.01) | 38.92 | – | – | – | – | – | – |
| Yes | 2 | 0.62 | (0.43, 0.89) | 20.97 | .13 | – | – | – | – | – | |
| Reported measuring risk (Tr) | No (ref) | 8 | 0.82 | (0.66, 1.04) | 52.60 | – | 5 | 0.85 | (0.67, 1.07) | 93.38 | – |
| Yes | 4 | 0.74 | (0.59, 0.92) | <0.001 | .50 | 2 | 0.69 | (0.48, 0.99) | 92.65 | .36 | |
| Reported mental health treatment (Tr) | No (ref) | 8 | 0.83 | (0.67, 1.04) | 52.65 | – | – | – | – | – | – |
| Offered | 4 | 0.72 | (0.57, 0.91) | <0.001 | .39 | – | – | – | – | – | |
| Reported pharmacotherapy (Tr) | No (ref) | – | – | – | – | – | 3 | 0.85 | (0.57, 1.26) | 95.70 | – |
| Offered | – | – | – | – | – | 4 | 0.75 | (0.64, 0.88) | 81.03 | .59 | |
| Reported prison intervention (Con) | No (ref) | 9 | 0.80 | (0.65, 0.98) | 54.71 | – | – | – | – | – | – |
| Yes | 3 | 0.79 | (0.58, 1.05) | <0.001 | .93 | – | – | – | – | – | |
| Reported relapse prevention (Tr) | No (ref) | 8 | 0.79 | (0.64, 0.97) | 58.53 | – | 2 | 0.65 | (0.50, 0.83) | 92.78 | – |
| Offered or required | 4 | 0.81 | (0.61, 1.09) | <0.001 | .85 | – | – | – | – | – | |
| Offered | – | – | – | – | – | 3 | 0.96 | (0.68, 1.35) | 88.50 | .07 | |
| Required | – | – | – | – | – | 2 | 0.77 | (0.63, 0.95) | 78.23 | .29 | |
| Reported support groups (Tr) | No (ref) | 8 | 0.78 | (0.62, 0.97) | 58.18 | – | – | – | – | – | – |
| Offered or required | 4 | 0.82 | (0.64, 1.07) | <0.001 | .74 | – | – | – | – | – | |
| Reported vocational or educational services (Tr) | No (ref) | 7 | 0.81 | (0.64, 1.03) | 57.52 | – | 4 | 0.82 | (0.69, 0.97) | 77.22 | – |
| Offered or required | 5 | 0.75 | (0.60, 0.94) | <0.001 | .61 | 3 | 0.77 | (0.51, 1.15) | 97.15 | .77 | |
| Total number of treatments offered (Tr) | Beta weight | 12 | 1.01 | (0.96, 1.06) | 47.75 | .67 | 7 | 1.01 | (0.93, 1.10) | 94.40 | .77 |
| Total number of treatments required (Tr) | Beta weight | 10 | 0.96 | (0.85, 1.09) | 41.71 | .51 | 7 | 0.93 | (0.81, 1.06) | 93.23 | .27 |
Note. *p < .05; **p < .01; ***p < .001; ATSI = Aboriginal and Torres Strait Islander; CBT = cognitive behavioural therapy; Con = variable measured among control condition; ref = reference group; Tr = variable measured among treatment condition; p value b/w asterisks denote significance level of the difference between the moderator and reference category after Benjamini–Hochberg adjustment.
The incidence of recidivism methodological moderator analysis indicates that the studies which measured participants’ reoffending whilst they were in treatment (IRR = 0.71, 95% CI [0.59, 0.85]) possess significantly stronger treatment effects compared to those which measured reoffending after treatment exit (b = 1.51, adjusted p < .55). Further, the participants' annual incidence of recidivism was associated with an expected increase of 5% relative to comparisons for every month after they had exited the problem-solving court (b = 1.05, 95% CI [1.02, 1.09], adjusted p < .05). In a similar vein, the average duration of the treatment programme was found to significantly moderate the estimated treatment effects, indicating that as the average duration of the programme studied increases, the estimated incidence of recidivist events decreases among the treatment participants relative to the comparison participants (b = 0.91, 95% CI [0.86, 0.96], adjusted p < .01).
Sensitivity analysis
It has been noted by Agresti (2013) that the rescaling factor between ORs and probit coefficients ranges from 1.6 to 1.8. For this reason, the odds of recidivism meta-analysis was iteratively repeated 400 times using every combination of probit coefficient and standard error rescaled by 1.6 to 1.8 (in increments of 0.01) for the single outcome that reported a probit coefficient, whilst holding the effect sizes and variances for the remaining k = 11 outcomes constant. The rescaling factor of 1.6 (for both coefficient and standard error) that was chosen for the primary analysis produced the most conservative pooled estimate (OR = 0.80, p = .01, 95% CI [0.67, 0.94]), with the remaining 399 models indicating stronger treatment effects. Furthermore, every iteration was significant at the p = .01 level after rounding, and every I2 value fell between 45.00 and 45.70. Similarly, a ‘leave-one-out’ analysis was performed on the analysis of odds of recidivism which revealed that the removal of k = 1 outcome (Department of the Attorney General [WA], 2006) left the pooled estimate marginally non-significant (OR = 0.85, p = .52, 95% CI [0.72, 1.00], Q = 14.68 (p = .14), I2 = 35.20).
Two additional analyses were conducted on the odds of recidivism outcomes: one that included the k = 12 outcomes in a fixed-effect model and another that used k = 8 outcomes which only included problem-solving court graduates (graduates only hereafter) in the treated sample compared to controls. Both models are presented in Table 4 below the random-effects model chosen for the primary analysis. The fixed-effect model was found to produce a smaller pooled estimate (OR = 0.92, p = .09, 95% CI [0.83, 1.01]) than the random-effects model but is marginally non-significant, while the graduates only analysis estimates the odds of recidivism as 50% lower among problem-solving court graduates compared with controls (OR = 0.50, p = .01, 95% CI [0.29, 0.85]).
The meta-analysis of the incidence of recidivism was additionally modelled using two alternative GLMMs: an unconditional model with fixed study effects and an unconditional model with random effects. As can be seen in Table 5, neither model possesses any meaningful difference compared to the conditional model with exact likelihood in terms of the estimate (IRR = 0.80 in all three models) and its associated precision, although the deviance in the unconditional model with random effects is notably significant at the p < .001 level.
Lastly, two additional variables related to the MSMS rating were created and included in the moderator analyses to test the decision to assign the rating ‘3: Unmatched or poorly matched’ to studies that did not match participants on most of the important confounding variables despite employing a valid and reliable matching technique. The first variable, MSMS rating recode (a), recoded any of these studies to a category halfway between ‘3: Unmatched or poorly matched’ and ‘4: Well-matched’, while the second variable, MSMS rating recode (b), recoded them to ‘4: Well-matched’. As shown in Table 2, the results of the odds of recidivism moderator analysis reveal that MSMS rating recode (a) failed to reach statistical significance (b = 1.38, 95% CI [0.90, 2.13], adjusted p > .05). However, it can be seen that MSMS rating recode (b) negatively correlates with the treatment effects and reaches statistical significance (b = 1.54, 95% CI [1.24, 1.92], adjusted p < .01). The same analyses were performed within the incidence of recidivism moderator analysis and both variables were found to be non-significant, respectively (b = 0.84, 95% CI [0.64, 1.10], adjusted p > .05; b = 0.90, 95% CI [0.68, 1.20], adjusted p > .05).
Discussion
This investigation systematically sought out and reviewed research evaluating problem-solving courts in Australia and New Zealand. The results indicate that judicial supervision interventions throughout both jurisdictions are associated with meaningful reductions in both the frequency and likelihood of reoffending compared to standard judicial practices. Over a period of 15.0 to 19.5 months, participation in Australian and New Zealand problem-solving courts translated to approximately 1 in 10 fewer recidivists, and those who did reoffend committed 20% fewer offences than non-exposed individuals. However, the majority of the independent studies reviewed did not adequately match treatment and comparison participants, placing their results at significant risk of bias. The meta-analytic findings reported here should therefore be considered with an appropriate amount of caution.
Meta-analytic investigations of problem-solving courts from the international literature base consistently find that the majority of the outcome evaluations they review possess significant methodological flaws (e.g. Gutierrez & Bourgon, 2012; Gutierrez et al., 2016). For example, only 25.00% (k = 23) of the k = 92 evaluations reviewed by Mitchell et al. (2012) were considered ‘rigorous quasi-experiments’ or stronger after the authors applied a modified version of the MSMS rating scale similar to that employed here. In the present study, 16.67% (k = 2) of the k = 12 outcomes in the odds of recidivism analysis and 42.96% (k = 3) of the k = 7 outcomes in the incidence of recidivism analysis were derived from treatment groups considered well matched or randomised. Consequently, the outcomes – particularly within the odds of recidivism analyses – are overwhelmingly represented by those that possess significant risk of bias.
Indeed, meta-analytic research on problem-solving courts routinely reveals stronger results among studies considered low in methodological quality (e.g. Gutierrez & Bourgon, 2012; Lowder et al., 2018). In the present investigation, 'well-matched' studies negatively correlated with treatment effects in the odds of recidivism analysis when the definition of 'well-matched' was expanded to include studies that reported using an appropriate matching technique but did not match participants on most of the important confounding variables. In other words, under a broader definition, methodologically stronger studies associated with significant results favouring the comparison group. The sample-matching procedures in question arguably sit between poorly matched and well matched, and when they are treated as such no meaningful relationship presents. This effect is absent from the incidence of recidivism analysis, where the MSMS (transformed or otherwise) does not meaningfully predict any change. At the very least, the possibility remains that the studies at greatest risk of bias had an undue influence on estimates of the effectiveness of problem-solving courts at reducing the odds of recidivism, calling for additional rigorous research.
In a different vein, an extreme amount of heterogeneity was observed among the outcomes included in the incidence of recidivism analysis. The principal implication of this finding is that it places considerable restrictions on generalising the results to future studies. This result is somewhat surprising when considered against the effect sizes within the odds of recidivism analysis, which incorporate a greater variety of courts but are considerably less dispersed. One possible explanation for this difference is that the measure used in the odds of recidivism analysis is a relatively simple dichotomous split between those in each experimental group that did and did not reoffend, whereas the incidence of recidivism measure tallies the frequency of offences and could thereby be more sensitive to differences between courts.
The moderator and subgroup analyses that were employed in an attempt to investigate the inconsistency among the effect sizes of both meta-analyses provided some clarity. The heterogeneity between the outcomes included in the incidence of recidivism analysis, for example, is partly explained by the timing of the problem-solving court treatment in relation to the follow-up period. The treatment effects are considerably stronger when at least some of the follow-up period was concurrent with participation in the problem-solving court, such that studies which exclusively measured participants after they had exited the program found a lower incidence of recidivism among the comparison group. Furthermore, the longer participants spent inside the problem-solving court programme during the follow-up period, the stronger the treatment effect on the incidence of recidivism. Ultimately, the comparison of individuals measured during treatment with controls who have exited treatment raises serious methodological concerns. It could be argued, for example, that there is no meaningful comparison to be made between problem-solving court participants active in treatment with individuals who have exited prison as they are simply not at a comparable stage of treatment.
The studies conducted in Victoria, or Australia taken together, possess significantly lower odds of recidivism favouring treatment compared to those conducted in New Zealand. The explanation for this difference may rest in the types of courts investigated in each jurisdiction. Two of the three New Zealand studies examined a youth drug court and a family violence court, which is notable because the former tend to produce the same results as traditional juvenile justice processes, and a similar observation has been made among methodologically rigorous studies of the latter (Gutierrez et al., 2016; Tanner-Smith et al., 2016). Two of the investigations from Victoria were of a community court that targets lower-level offences with a broad range of services, and four of the other Australian studies were conducted in drug courts, which have consistently been found more effective than standard processes (Mitchell et al., 2012). Any comparison of these groups of problem-solving courts based on jurisdiction then was perhaps destined to be skewed.
Similarly, the finding that studies which did not report offering anger management are significantly correlated with lower odds of recidivism can likely be attributed to the sample characteristics – such as criminogenic risk of reoffending – of the populations processed by those courts. Violent or aggressive populations, for instance, tend to possess various treatment needs deserving of a multi-modal approach that features anger control among other cognitive behavioural treatments (see Papalia et al., 2019). Moreover, the separation of studies into those that did and did not report a treatment component inherently brings a risk of mischaracterisation, although in the current study it is unavoidable due to under-reporting.
To the extent that it is possible, we urge the authors of future problem-solving court investigations to adopt rigorous standards in two critical areas: accounting for baseline variables that are likely to confound the results of any sample comparison, and detailed reporting. In the first case, it is vital that the causality of results can be confidently attributed to an intervention when a comparison is made. This is particularly true when experimental results inform cost evaluations and ultimately help to shape public policy. Acknowledging that randomised trials are not always practical in the domain of criminal justice, robust statistical matching on variables that possess a well-documented relationship with the outcome of interest should be implemented when quasi-experiments are undertaken (see Iacus et al., 2012). In the second case, we recommend that authors of future research on problem-solving courts take care to report sufficient information for assessing the impacts of the respective interventions. Meticulous guidelines for performing outcome evaluations that include reporting instructions are available elsewhere (Department of Premier and Cabinet [Qld], 2015; Morgan & Homel, 2013). Authors should additionally report detailed information on sample, methodological, programmatic and treatment characteristics. Consistent and detailed reporting of sample demographics and the treatments received in both the control and treatment samples will have the dual benefits of improving the precision of comparisons within these studies in addition to those between them.
The results of the current study should be considered in light of the limitations of the work. The recidivism measures in the included studies were restricted to those, such as rearrests, that are based on the detection of crime. As such, they represent an unknown proportion of the total crimes committed. Some avenues for mitigating this problem are to employ a longer follow-up period or adjust the outcome based on multiple historical measures of recidivism (e.g. New Zealand Ministry of Justice Tāhū o te Ture, 2019). Despite both approaches falling victim to the same detection bias, we can assume that they will provide a more accurate indication of offending behaviour as the probability of detection increases with time. A longer follow-up period also has the added advantage of providing some indication of the longer-term impacts of the interventions in question.
It is lastly worth noting that recidivism in isolation provides an incomplete picture of the overall effectiveness of problem-solving courts, given that they aim to address both offending and its precipitating psychosocial issues. For this reason, future investigations of problem-solving courts would do well to include appropriate measures of the socio-health outcomes targeted within the respective problem-solving court models. Future reviews should then assess individual problem-solving court models (e.g. drug courts) on comparable quality-of-life outcomes alongside recidivism measures in order to gauge the overall impact on the target population.
Conclusion
The results of this review tentatively suggest that problem-solving courts which incorporate judicial supervision across Australia and New Zealand are proving more effective at reducing recidivism than traditional justice processes. These findings need to be considered cautiously given the likely presence of methodological bias in some of the included studies. Furthermore, the findings do not provide any substantial evidence as to the influence that programmatic variables have on the effects of these courts in reducing recidivism. The implications of this study primarily relate to those commissioning or conducting research into problem-solving courts in Australia and New Zealand, advocating for the use of stronger methodologies in terms of matching treatment and comparison subjects, measuring them at an equivalent stage post-treatment and meticulously reporting the treatments that they receive.
Supplementary Material
Acknowledgements
The authors are thankful to all individuals who supplied or assisted with obtaining requested information.
.
Table A1.
Search syntax executed for EBSCOhost.
| Syntax | Sources included | Sources excluded | Retrievable records |
|---|---|---|---|
| (TI judicial W1 monitoring) OR (AB judicial W1 monitoring) OR (TI judicial W1 supervision) OR (AB judicial W1 supervision) OR (TI special* W1 court W1 program*) OR (AB special* W1 court W1 program*) OR (TI drug W1 court*) OR (AB drug W1 court*) OR (TI DWI W1 court*) OR (AB DWI W1 court*) OR (TI DUI W1 court*) OR (AB DUI W1 court*) OR (TI mental W1 health W1 court*) OR (AB mental W1 health W1 courts*) OR (TI veteran* W1 court*) OR (AB veteran* W1 court*) OR (TI domestic W1 violence W1 court*) OR (AB domestic W1 violence W1 court*) OR (TI family W1 violence W1 court*) OR (AB family W1 violence W1 court*) | Academic journals, Journals, Conference materials, Books, Dissertations | Magazines, Reviews, News, Trade publications | 1368 |
Table A2.
Modified Maryland Scientific Methods Scale (MSMS) rating protocol.
| ‘3: Unmatched or poorly matched’ | ‘4: Well-matched’ | ‘5: Randomised’ |
|---|---|---|
| Rated 3 if any of the following criteria are met: | Rated 4 if all of the following criteria are met: | Rated 5 if the following criterion is met: |
|
|
|
Table A3.
Confounding variables.
| Confounding domain | Example s of measured variable(s) |
|---|---|
| Demographic characteristics | Age, race, sex, education, income, marital status |
| Programme eligibility | Groups were selected using the same eligibility criteria; comparison participants were rejected from the intervention |
| Criminogenic risk | Level of Service Inventory – Revised scores |
| Criminal history | Lifetime arrests; lifetime convictions; convictions for person offences; arrest severity |
| Diagnostic profile | Primary diagnosis (e.g. schizophrenia, bipolar, depression, other) |
| Substance use profile | Addiction Severity Index scores; a history of substance use; a history of failed substance treatment |
Ethical standards
Declaration of conflicts of interest
Michael D. Trood has declared no conflicts of interest
Benjamin L. Spivak has declared no conflicts of interest
James R. P. Ogloff has declared no conflicts of interest
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Data availability statement
The raw data and accompanying code script may be accessed online at https://github.com/mtrood/effects-of-judicial-supervision-on-recidivisim-of-offenders-in-Australia-and-New-Zealand.
Supplemental material
Supplemental material is available via the “Supplementary” tab on the article’s online page (https://dx.doi.org/10.1080/13218719.2021.1956385).
References
- Agresti, A. (2013). Categorical data analysis. John Wiley & Sons. [Google Scholar]
- Bartels, L. (2009). Challenges in mainstreaming specialty courts (Trends & Issues in Crime and Criminal Justice No. 383). https://www.aic.gov.au/sites/default/files/2020-05/tandi383.pdf [Google Scholar]
- Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300. 10.1111/j.2517-6161.1995.tb02031.x [DOI] [Google Scholar]
- Berman, G., & Feinblatt, J. (2001). Problem-solving courts: A brief primer. Law & Policy, 23(2), 125–140. 10.1111/1467-9930.00107 [DOI] [Google Scholar]
- Borenstein, M. (2019). Common mistakes in meta-analysis and how to avoid them. Biostat. [Google Scholar]
- Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2009). Introduction to meta-analysis. John Wiley & Sons. [Google Scholar]
- Casey, P. M., & Rottman, D. B. (2005). Problem-solving courts: Models and trends. Justice System Journal, 26(1), 35–56. 10.1080/0098261X.2005.10767737 [DOI] [Google Scholar]
- Centre for Reviews and Dissemination . (n.d.). PROSPERO: International prospective register of systematic reviews. https://www.crd.york.ac.uk/PROSPERO/
- Crime Research Centre. (2003). Evaluation of the Perth Drug Court Pilot Project. https://www.parliament.wa.gov.au/publications/tabledpapers.nsf/displaypaper/4010923c794d6a896780c09a482581e800133453/$file/tp-923.pdf
- Department of the Attorney General. (2006). A review of the Perth Drug Court. Government of Western Australia. https://www.parliament.wa.gov.au/publications/tabledpapers.nsf/c41d5695f20b386348256b0200183f75/199175e34b12c3b3482581e8001854d7/$FILE/TP-924.pdf
- Department of the Premier and Cabinet (Qld). (2015). Criminal justice evaluation framework: Guidelines for evaluating criminal justice initiatives. Queensland Government. https://www.premiers.qld.gov.au/publications/categories/guides/assets/criminal-justice-evaluation-framework-2015.pdf
- Friendship, C., Street, R., Cann, J., & Harper, G. (2005). Introduction: The policy context and assessing the evidence. In Harper G. & Chitty C. (Eds.), The impact of corrections on reoffending: A review of ‘what works’ (2nd ed., pp. 1–16). The Research, Development and Statistics Directorate, Home Office, Government of the United Kingdom. [Google Scholar]
- Gutierrez, L., Blais, J., & Bourgon, G. (2016). Do domestic violence courts work? A meta-analytic review examining treatment and study quality. Justice Research and Policy, 17(2), 75–99. 10.1177/1525107117725012 [DOI] [Google Scholar]
- Gutierrez, L., & Bourgon, G. (2012). Drug treatment courts: A quantitative review of study and treatment quality. Justice Research and Policy, 14(2), 47–77. 10.3818/JRP.14.2.2012.47 [DOI] [Google Scholar]
- Harris, M. (2006). ‘A sentencing conversation’: Evaluation of the Koori courts: Pilot Program: October 2002–October 2004. Department of Justice, Victoria State Government. [Google Scholar]
- Heale, P., & Lang, E. (1999). Court Referral and Evaluation for Drug Intervention and Treatment (CREDIT): Final evaluation report. Turning Point Alcohol and Drug Centre. [Google Scholar]
- Heale, P., & Lang, E. (2001). A process evaluation of the CREDIT (Court Referral and Evaluation for Drug Intervention and Treatment) pilot programme. Drug and Alcohol Review, 20(2), 223–230. 10.1080/09595230120058500 [DOI] [Google Scholar]
- Iacus, S. M., King, G., & Porro, G. (2012). Causal inference without balance checking: Coarsened exact matching. Political Analysis, 20(1), 1–24. 10.1093/pan/mpr013 [DOI] [Google Scholar]
- Jones, C. G. A. (2011). Intensive judicial supervision and drug court outcomes: Interim findings from a randomised controlled trial (Crime and Justice Bulletin No. 152). NSW Bureau of Crime Statistics and Research. https://www.bocsar.nsw.gov.au/Publications/CJB/cjb152.pdf [Google Scholar]
- Jones, C. G. A. (2013). Early-phase outcomes from a randomized trial of intensive judicial supervision in an Australian drug court. Criminal Justice and Behavior, 40(4), 453–468. 10.1177/0093854812449215 [DOI] [Google Scholar]
- Klinkum, O. (2019). Taking New Zealand’s specialist criminal courts ‘to scale’ for better criminal justice outcomes. Te Wharenga – New Zealand Criminal Law Review, Article 2. https://ojs.aut.ac.nz/new-zealand-criminal-law-review/index.php/tewharenga/article/download/65/55 [Google Scholar]
- Knaggs, T., Leahy, F., Soboleva, N., & Ong, S.-W. (2008). The Waitakere and Manukau Family Violence Courts: An evaluation summary. Ministry of Justice Tāhū o te Ture. https://www.justice.govt.nz/assets/Documents/Publications/report-1.pdf [Google Scholar]
- Kornhauser, R. (2018). The effectiveness of Australia’s drug courts. Australian & New Zealand Journal of Criminology, 51(1), 76–98. 10.1177/0004865816673412 [DOI] [Google Scholar]
- Lim, L., & Day, A. (2014). Mental health diversion courts: A two year recidivism study of a South Australian mental health court program. Behavioral Sciences & the Law, 32(4), 539–551. 10.1002/bsl.2126 [DOI] [PubMed] [Google Scholar]
- Lind, B., Weatherburn, D., Chen, S., Shanahan, M., Lancsar, E., Haas, M., & De Abreu Lourenco, R. (2002). New South Wales Drug Court evaluation: Cost-effectiveness. NSW Bureau of Crime Statistics and Research. https://www.bocsar.nsw.gov.au/Publications/Legislative/l15.pdf [DOI] [PubMed] [Google Scholar]
- Lowder, E. M., Rade, C. B., & Desmarais, S. L. (2018). Effectiveness of mental health courts in reducing recidivism: A meta-analysis. Psychiatric Services (Washington, D.C.), 69(1), 15–22. 10.1176/appi.ps.201700107 [DOI] [PubMed] [Google Scholar]
- Lulham, R. (2009). The Magistrates Early Referral Into Treatment Program: Impact of program participation on re-offending by defendants with a drug use problem (Crime and Justice Bulletin No. 131). NSW Bureau of Crime Statistics and Research. https://www.bocsar.nsw.gov.au/Publications/CJB/cjb131.pdf [Google Scholar]
- Makkai, T., & Veraar, K. (2003). Final report on the South East Queensland Drug Court. Australian Institute of Criminology. https://www.courts.qld.gov.au/__data/assets/pdf_file/0007/93913/m-drug-court-seq-final-report.pdf [Google Scholar]
- Marchetti, E. (2017). Indigenous sentencing courts in Australia. In Deckert A. & Sarre R. (Eds.), The Palgrave handbook of Australian and New Zealand criminology, crime and justice (pp. 379–392). Springer. [Google Scholar]
- Marlowe, D. B., Festinger, D. S., Lee, P. A., Schepise, M. M., Hazzard, J. E., Merrill, J. C., Mulvaney, F. D., & McLellan, A. T. (2003). Are judicial status hearings a key component of drug court? During-treatment data from a randomized trial. Criminal Justice and Behavior, 30(2), 141–162. 10.1177/0093854802250997 [DOI] [Google Scholar]
- Mitchell, O., Wilson, D. B., Eggers, A., & MacKenzie, D. L. (2012). Assessing the effectiveness of drug courts on recidivism: A meta-analytic review of traditional and non-traditional drug courts. Journal of Criminal Justice, 40(1), 60–71. 10.1016/j.jcrimjus.2011.11.009 [DOI] [Google Scholar]
- Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151(4), 264–269. 10.7326/0003-4819-151-4-200908180-00135 [DOI] [PubMed] [Google Scholar]
- Morgan, A., & Homel, P. (2013). Evaluating crime prevention: Lessons from large-scale community crime prevention programs (Trends & Issues in Crime and Criminal Justice No. 458). Australian Institute of Criminology. https://www.aic.gov.au/publications/tandi/tandi458 [Google Scholar]
- New Zealand Ministry of Justice Tāhū o te Ture. (2019). Alcohol and other drug treatment court quantitative outcomes evaluation 2018–19: Reduced reoffending, imprisonment, and benefit dependency: Final report. Ministry of Justice Tāhū o te Ture. https://www.justice.govt.nz/assets/Documents/Publications/AODTC-Quantitative-Outcomes-Evaluation-June-2019.pdf [Google Scholar]
- Papalia, N., Spivak, B., Daffern, M., & Ogloff, J. R. (2019). A meta‐analytic review of the efficacy of psychological treatments for violent offenders in correctional and forensic mental health settings. Clinical Psychology: Science and Practice, 26(2), e12282. 10.1111/cpsp.12282 [DOI] [Google Scholar]
- Payne, J. (2005). Final report on the North Queensland Drug Court. Australian Institute of Criminology. https://apo.org.au/sites/default/files/resource-files/2005-10/apo-nid2648.pdf [Google Scholar]
- Payne, J. (2006). Specialty courts: Current issues and future prospects (Trends & Issues in Crime and Criminal Justice No. 317). Australian Institute of Criminology. https://www.aic.gov.au/sites/default/files/2020-05/tandi317.pdf [Google Scholar]
- Payne, J. (2007). Recidivism in Australia: Findings and future research (Research and Public Policy Series No. 80). Australian Institute of Criminology. https://www.aic.gov.au/sites/default/files/2020-05/rpp080.pdf
- Payne, J. (2008). The Queensland Drug Court: A recidivism study of the first 100 graduates. Australian Institute of Criminology. https://www.aic.gov.au/sites/default/files/2020-05/rpp083.pdf [Google Scholar]
- PricewaterhouseCoopers. (2009). Economic evaluation of the Court Integrated Services Program (CISP): Final report on economic impacts of CISP. Department of Justice, Victoria State Government. https://www.magistratescourt.vic.gov.au/sites/default/files/Default/cisp_economic_evaluation_final_report.pdf [Google Scholar]
- Richardson, E., Thom, K., & McKenna, B. (2013). The evolution of problem-solving courts in Australia and New Zealand: A trans-Tasman comparative perspective. In R. L. Wiener & E. M. Brank (Eds.), Problem solving courts (pp. 185–210). Springer. 10.1007/978-1-4614-7403-6 [DOI]
- Rohatgi, A. (2019). WebPlotDigitizer: Web-based plot digitizer (Version 4.2) [Computer software]. https://automeris.io/WebPlotDigitizer
- Ross, S. (2009). Evaluation of the Court Integrated Services Program: Final report. Melbourne Consulting & Custom Programs. [Google Scholar]
- Ross, S. (2015). Evaluating neighbourhood justice: Measuring and attributing outcomes for a community justice program (Trends & Issues in Crime and Criminal Justice No. 499). Australian Institute of Criminology. https://www.aic.gov.au/sites/default/files/2020-05/tandi499.pdf [Google Scholar]
- Ross, S., Halsey, M., Bamford, D., Cameron, N., & King, A. (2009). Evaluation of the Neighbourhood Justice Centre, City of Yarra: Final report. Department of Justice, Victoria State Government. [Google Scholar]
- Sarteschi, C. M., Vaughn, M. G., & Kim, K. (2011). Assessing the effectiveness of mental health courts: A quantitative review. Journal of Criminal Justice, 39(1), 12–20. 10.1016/j.jcrimjus.2010.11.003 [DOI] [Google Scholar]
- Searle, W., & Spier, P. (2006). Christchurch Youth Drug Court pilot: One year follow-up study. Ministry of Justice Tāhū o te Ture. https://www.justice.govt.nz/assets/Documents/Publications/chch-youth-drug-court-follow-up.pdf [Google Scholar]
- Shanahan, M., Lancsar, E., Haas, M., Lind, B., Weatherburn, D., & Chen, S. (2004). Cost-effectiveness analysis of the New South Wales Adult Drug Court Program. Evaluation Review, 28(1), 3–27. 10.1177/0193841X03257531 [DOI] [PubMed] [Google Scholar]
- Soboleva, N., & Su-Wuen, O. (2008). The Waitakere Family Violence Court: Outcome, sentencing and reconviction analysis. Ministry of Justice Tāhū o te Ture. [Google Scholar]
- Stijnen, T., Hamza, T. H., & Ozdemir, P. (2010). Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data. Statistics in Medicine, 29(29), 3046–3067. 10.1002/sim.4040 [DOI] [PubMed] [Google Scholar]
- Sullivan, C., & Povey, L. (2015). Measuring re-offending with court data: Proposed tier 1 specification [Unpublished manuscript]. Ministry of Justice Tāhū o te Ture. 10.13140/RG.2.1.3287.2087 [DOI] [Google Scholar]
- Tanner-Smith, E. E., Lipsey, M. W., & Wilson, D. B. (2016). Juvenile drug court effects on recidivism and drug use: A systematic review and meta-analysis. Journal of Experimental Criminology, 12(4), 477–513. 10.1007/s11292-016-9274-y [DOI] [Google Scholar]
- Trood, M. D., Spivak, B. L., & Ogloff, J. R. P. (2021). A systematic review and meta-analysis of the effects of judicial supervision on recidivism and well-being factors of criminal offenders. Journal of Criminal Justice, 74, 1–23, Article 101796. 10.1016/j.jcrimjus.2021.101796 [DOI] [Google Scholar]
- Viechtbauer, W. (2005). Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics, 30(3), 261–293. 10.3102/10769986030003261 [DOI] [Google Scholar]
- Viechtbauer, W. (2007). Hypothesis tests for population heterogeneity in meta-analysis. The British Journal of Mathematical and Statistical Psychology, 60(1), 29–60. 10.1348/000711005X64042 [DOI] [PubMed] [Google Scholar]
- Weatherburn, D., Jones, C., Snowball, L., & Hua, J. (2008). The NSW Drug Court: A re-evaluation of its effectiveness (Crime and Justice Bulletin No. 121). NSW Bureau of Crime Statistics and Research. https://www.bocsar.nsw.gov.au/Publications/CJB/cjb121.pdf [Google Scholar]
- Weatherburn, D., Yeong, S., Poynton, S., Jones, N., & Farrell, M. (2020a). The long-term effect of the NSW Drug Court on recidivism (Crime and Justice Bulletin No. 232). NSW Bureau of Crime Statistics and Research. https://www.bocsar.nsw.gov.au/Publications/CJB/2020-The-Long-term-effect-of-the-NSW-Drug-Court-on-recidivism-CJB232.pdf [Google Scholar]
- Weatherburn, D., Yeong, S., Poynton, S., Jones, N., & Farrell, M. (2020b). The long-term effect of the NSW Drug Court on recidivism [Manuscript submitted for publication]. NSW Bureau of Crime Statistics and Research.
- Weisburd, D., Lum, C. M., & Petrosino, A. (2001). Does research design affect study outcomes in criminal justice? The Annals of the American Academy of Political and Social Science, 578(1), 50–70. 10.1177/000271620157800104 [DOI] [Google Scholar]
- Wilson, D. B., Mitchell, O., & MacKenzie, D. L. (2007). A systematic review of drug court effects on recidivism. Journal of Experimental Criminology, 2(4), 459–487. 10.1007/s11292-006-9019-4 [DOI] [Google Scholar]
- Ziersch, E., & Marshall, J. (2012). The South Australian Drug Court: A recidivism study. Attorney General’s Department, Government of South Australia. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The raw data and accompanying code script may be accessed online at https://github.com/mtrood/effects-of-judicial-supervision-on-recidivisim-of-offenders-in-Australia-and-New-Zealand.