Racial/Ethnic Disparities of the Positive Achievement Change Tool in Predicting Recidivism and Court Dispositions for Justice-Involved Youths

Abstract

Objective:

Responding to the concern about racial/ethnic disparities in the use of risk assessment instruments (RAIs) in justice systems, previous research has overwhelmingly tested the extent to which RAI scores consistently predict recidivism across race and ethnicity (predictive bias). However, little is known about racial/ethnic disparities in the association between RAI measures and court dispositions (disparate application) for justice-involved youths. This study investigated predictive bias and disparate application of three risk measures—criminal history, social history, and the overall risk level—produced by the Positive Achievement Change Tool (PACT) for White, Black, and Hispanic justice-involved youths.

Hypotheses:

Given the mixed evidence in existing research for predictive bias and lack of evidence for disparate application, we did not make any specific hypothesis but conducted exploratory analyses. From a clinical perspective, however, we anticipated little or no evidence to support predictive bias and disparate application of the PACT among White, Black, and Hispanic youths in the jurisdiction we examined.

Method:

The sample consisted of 5,578 youths (11.4% White, 43.9% Black, and 44.7% Hispanic) who completed the PACT while in the Harris County Juvenile Probation Department, Texas. The outcome variables included recidivism (general and violent reoffending) and court dispositions (deferred adjudication, probation without placement, and probation with placement). We ran a series of moderating binary logistic regression models and moderating ordinal logistic regression models to evaluate predictive bias and disparate application.

Results:

Race and ethnicity influenced how the criminal history score related to violent recidivism: This compromised the validity of the score as a predictor of recidivism. Moreover, evidence showed that the overall risk of reoffending was associated with harsher sanctioning decisions for Black and Hispanic youths than for White youths.

Conclusion:

Ensuring that RAI results are consistently interpreted and used in informing decisions is as important as ensuring that RAI scores function equally well in predicting recidivism regardless of race and ethnicity.

A collective body of research focusing on risk assessment instruments (RAIs) has strongly supported the use of such actuarial tools to inform decisions in the juvenile justice system. This support derives from the well-established validity of RAIs in predicting the likelihood that a youth will engage in delinquency in the future (Hoge, 2002; Olver et al., 2014; Petkus et al., 2022; Schwalbe, 2007). In addition, it has been widely recognized that structured, research-based RAIs produce more consistent and valid judgments regarding risk and need for justice-involved youths than do intuitive, clinical judgments made by juvenile justice professionals, such as probation officers and court judges (Desmarais & Zottola, 2020; Hoge, 2002). Given the accuracy and objectivity of RAIs, as of 2020, all 50 U.S. states and the District of Columbia have adopted them in juvenile probation to inform dispositional planning, with 42 states mandating the use of such tools statewide, according to Juvenile Justice Geography, Policy, Practice & Statistics (n.d.).

Nevertheless, critics have expressed concern that the use of RAIs may exacerbate racial/ethnic disparities (R/ED) in correctional settings (Freeman et al., 2021; Holder, 2014; Moore & Padavic, 2011). The primary reason for this concern is that static risk factors included in various RAIs, such as criminal history, may disadvantage racial/ethnic minorities in sanctioning decisions (Harcourt, 2015). These static risk factors arguably reflect structural racism instead of actual criminal propensity. As a result, the use of RAIs may inadvertently lead to harsher punishment for racial/ethnic minorities, which in turn may reinforce R/ED in jurisdictions (Freeman et al., 2021; Onifade et al., 2019).

In response to this controversial concern, a substantial body of research has evaluated the degree to which justice-involved youths from different racial/ethnic groups have the same risk score but differ in their predicted likelihood of reoffending (Baglivio & Jackowski, 2013; Campbell et al., 2018, 2020; McCafferty, 2018; Onifade et al., 2008, 2009; Perrault et al., 2017; Rembert et al., 2014; Schwalbe et al., 2006, 2007; Vincent et al., 2011). Yet findings do not fully support or dispute any side of the conflicting debate about the racial/ethnic bias of RAIs. Even when the same RAI has been applied in different jurisdictions, the extent to which the likelihood of recidivism predicted by an RAI score was invariant across race and ethnicity often exhibited heterogeneity. Beyond that, investigating R/ED in the association between RAIs and recidivism makes an oblique contribution to the debate because it mainly reveals whether RAI scores have the same meaning in predicting recidivism for different groups. Yet the debate largely revolves around R/ED in the use of RAIs to inform actual sanctioning decisions—the extent to which court judges consistently interpret and use RAI results to inform decisions regardless of race and ethnicity. Indeed, an RAI score provides an estimate of the risk of reoffending, which will eventually be used to inform sanctioning decisions. However, R/ED in the association between RAIs and court dispositions (e.g., the level of supervision and residential placement) for youths has remained unclear.

In this study, we used data from Harris County Juvenile Probation Department (HCJPD), Houston, Texas, to investigate the effects of three risk measures—criminal history, social history, and overall risk level of reoffending—produced by the Positive Achievement Change Tool (PACT) on predicting recidivism and court dispositions for White, Black, and Hispanic youths. The PACT was designed to inform placement and supervision alternatives and guide service allocation (Baird et al., 2013; Parsons Early et al., 2012). Therefore, a better understanding of how the PACT measures perform across race/ethnicity can help policymakers and court professionals promote social justice and reduce R/ED in the juvenile justice system.

Predictive Bias for RAIs

One of the primary functions of RAIs is to estimate the likelihood that an individual will reoffend in the future, and these estimates can be used to assist justice professionals in making key decisions, including treatment and service allocation, supervision, and various types of sanctions (Hoge, 2002). In the context of R/ED in RAIs, therefore, it is fundamentally important to ensure that a given RAI score has the same meaning regardless of race and ethnicity (Lowder et al., 2019, 2022; Skeem & Lowenkamp, 2016). This is commonly tested by examining the extent to which the predicted recidivism matches the observed recidivism when using a common regression line (Cleary, 1968). Conversely, predictive bias occurs when there are differences in the patterns of association between test scores (RAI scores) and the criterion (recidivism) for different groups (American Educational Research Association et al., 2014). If the predicted likelihood of recidivism for an RAI score varies as a function of race and ethnicity, this may lead decision-makers who use the RAI to favor certain groups over others.

Voluminous research has been conducted to examine the extent to which the predictive validity of a variety of RAIs in the likelihood of recidivism varies as a function of race and ethnicity for justice-involved youths. Table S1 in the Supplemental Material summarizes the findings regarding various RAIs’ predictive bias by race and ethnicity. One line of research has shown that RAI scores (continuous) and the overall risk level (ordinal) predict recidivism equally well by race (Campbell et al., 2018, 2020; McCafferty, 2018; McKenzie, 2018; Onifade et al., 2009; Perrault et al., 2017) and, to a lesser extent, by ethnicity (McKenzie, 2018; Vincent et al., 2011). Another line of research has found that RAIs operate differentially by race and ethnicity. For example, Onifade et al. (2008) found that risk scores derived from the Youth Level of Service/Case Management Inventory (Flores et al., 2003) were correlated with the number of charges for White youths (r = .22, p < .05, n = 142) but not for Black (r = .13, p > .05, n = 101) and Hispanic (r = .25, p > .05, n = 30) youths. Rembert et al. (2014) reported that risk scores calculated using the Los Angeles County Needs Assessment Instrument (Turner & Fain, 2003) were related to recidivism for Hispanic youths (area under the receiver operating characteristic curve [AUC] = .66, 95% confidence interval [CI] [.59, .71], n = 341) but not for Black youths (AUC = .58, 95% CI [.48, .67], n = 139). Additionally, Schwalbe et al. (2007) found that the effect of static risk scores assessed by the Joint Risk Matrix on recidivism was smaller for Black youths (AUC = .66, 95% CI [.58, .74], n = 264) than for non-Latino White youths (AUC = .79, 95% CI [.70, .86], n = 269).

Considerable variation in the risk factors composing different RAIs, sample size and composition, and operationalization of recidivism across studies makes it difficult to paint a comprehensive picture of predictive bias across existing studies. Different jurisdiction contexts add an additional layer of complexity to forming a cohesive understanding of predictive bias. In fact, the degree of predictive bias by race and ethnicity also varied even when the same RAI was implemented by practitioners in different jurisdictions (Campbell et al., 2018; Onifade et al., 2008; Perrault et al., 2017). For example, in a study based on a large sample (N = 15,168) from the Florida Department of Juvenile Justice, Baglivio and Jackowski (2013) found no evidence supporting predictive bias in the overall risk level and criminal history derived from the PACT; however, the association between social history and subsequent referral and arrest was stronger for Hispanic youths than for White and Black youths, suggesting predictive bias. In a later study (N = 549) conducted in Montgomery County Juvenile Probation Department, Texas, McKenzie (2018) found that the PACT results, including criminal history, social history, and overall risk, consistently predicted the likelihood of committing a new offense or rearrest within 12 months for White non-Hispanic youths and minority youths (Hispanic, African American, and other minority ethnicities), suggesting the absence of predictive bias.

There are inconsistent findings regarding the elaborate associations between race/ethnicity, RAIs, and recidivism, highlighting the need for evidence validating the use of an RAI in a new setting. Indeed, according to the Standards for Educational and Psychological Testing, it is the test user’s responsibility to “evaluate the validity of a test in the particular setting where the test is used” (American Educational Research Association et al., 2014, p. 13).

Disparate Application for RAIs

Beyond predicting recidivism, the overall objective of most RAIs is to inform decisions in the juvenile justice system, such as intervention, supervision, and sanctioning. Disparate application occurs when RAI measures are not consistently applied to inform decisions across groups, which is likely to lead to socially undesirable consequences (Lowder et al., 2019; Warne et al., 2014). Like predictive bias, disparate application can also be tested by applying moderation models. Predictive bias and disparate application are different in that the former concentrates on recidivism, whereas the latter concentrates on actual decisions. Thus, predictive bias highlights the meaning of RAI scores associated with recidivism, whereas disparate application emphasizes how test users interpret and use RAI scores in making decisions. A test with predictive bias may not necessarily lead to socially undesirable consequences because test decision-makers do not make decisions solely on the basis of a risk score or a risk level derived from an RAI. However, an RAI exhibiting disparate application is likely to exert an adverse impact on racial/ethnic minority youths.

Compared with predictive bias, much less attention has been paid to how RAI measures may perpetuate R/ED in the juvenile justice system through harsher sanctions. As illustrated above, critics contend that RAIs exacerbate R/ED in correctional settings because racial/ethnic minorities are assumed to have higher RAI scores, which in turn results in harsher sanctions (e.g., being placed in a residential facility vs. community supervision) compared with their White counterparts (Freeman et al., 2021; Harcourt, 2015). This proposition implies a direct association between RAI measures and court dispositions. Yet this proposition has been understudied, despite its explicit contribution to the racial/ethnic debate regarding the use of RAIs.

To date, only one study focused on disparate application of RAIs between racial/ethnic groups. In a large sample consisting of 11,792 adult probationers (age: M = 32.37 years, SD = 11.26) in Kansas, Lowder et al. (2019) investigated the extent to which the association between risk scores, derived from the Level of Service Inventory–Revised (LSI-R; Andrews & Bonta, 2001), and sentence length varied as a function of race (Black vs. White). They found that White probationers had longer sentence lengths (M = 22.08 months, SE = 0.26) at low-risk levels relative to Black probationers (M = 20.39 months, SE = 0.44), whereas there were little differences in the average sentence lengths at other risk levels—moderate–low, moderate, moderate–high, and high—between the two groups. Their finding suggests that LSI-R risk scores were not related to harsher punishment for Black adult probationers. Yet this conclusion may not be generalizable to justice-involved youths because of the obvious heterogeneity in brain, cognitive, and psychological development between youths and adults (Steinberg, 2009). Beyond that, only a small proportion of justice-involved youths receive a certain length of sentencing, whereas the vast majority receive treatment, rehabilitation, and supervision. Thus, a better understanding of how risk scores are related to different types of court dispositions between racial/ethnic groups has important practical implications.

The Present Study

In response to the Texas Administrative Code (37 TAC §341.20, 2014) regarding the adoption of risk/needs assessment to guide case management and inform decisions, HCJPD approved the use of the PACT for assisting in dispositions in 2017. Although a recent study evaluated the predictive validity of the PACT in this jurisdiction (Mueller et al., 2022), racial/ethnic bias of the PACT has not been systematically evaluated. In this study, we aimed to investigate predictive bias and disparate application of three types of risk indicators—criminal history, social history, and overall risk level of reoffending—derived from the PACT for White, Black, and Hispanic youths.

We ran a series of moderated regression models to evaluate potential racial/ethnic differences of the PACT in predicting recidivism (predictive bias) and court dispositions (disparate application). Previous research related to predictive bias suggests the context-dependent feature of RAIs. There has been little evidence for the PACT, and more broadly RAIs, in terms of disparate application for justice-involved youths. Thus, we did not make any specific hypothesis for predictive bias and disparate application but conducted exploratory analyses. From a clinical perspective, however, we anticipated little or no evidence to support predictive bias and disparate application of the PACT among White, Black, and Hispanic youths in HCJPD.

Method

Sample

Participants were drawn from a sample of 8,175 youths who completed the PACT during the intake process from September 2017 through March 2021. We restricted our sample to youths who had at least one court disposition that was made after the implementation of the PACT (n = 7,040). Participants had to be less than 17 years old at the time of intake in order to allow us 1 year to follow up with them (n = 5,705). The small proportion of Asian American youths (n = 47) and youths with unknown race/ethnicity (n = 13) would result in unreliable parameters for the primary analyses, so we excluded them. Finally, an additional 67 youths were excluded because grouping their court dispositions—dismissed (n = 6), adjudicated with no disposition (n = 25), certified as an adult (n = 3), and transferred with no disposition (n = 33)—would lead to small classifications. The final sample size for the study was 5,578.

We examined the degree of differences between the included and excluded youths. There was a larger proportion of female youths in the analytical sample (25.9%) than the excluded sample (19.3%), χ²(1) = 40.89, p < .001, but the effect size was very small (Cramér’s V = .07), according to guidelines by Lovakov and Agadullina (2021). The racial/ethnic distributions between the included youths (11.4% White, 43.9% Black, 44.7% Hispanic) and the excluded youths (12.8% White, 46.1% Black, and 41.1% Hispanic) were similar, χ²(2) = 9.96, p = .007, Cramér’s V = .04. Youths included in the current study had a lower average criminal history score (M = 5.40, SD = 2.95) compared with those who were excluded (M = 6.17, SD = 3.90), t(3860) = 8.85, p < .001, Cohen’s d = 0.22, representing a small effect size (Cohen, 1992). Youths included in the current study also had a lower average social history score (M = 5.26, SD = 3.20) compared with those who were excluded (M = 5.59, SD = 3.17), t(8092) = 4.36, p < .001, Cohen’s d = 0.11, which again represented a small effect size (Cohen, 1992). Results of ordinal logistic regression (OLR) with proportional odds showed that the odds of having a lower level of overall risk (i.e., low vs. moderate or high) was 1.38 (95% CI [1.27, 1.51]) times greater for the analytical sample than for the excluded sample, which is considered a small effect size (Chen et al., 2010). The very small to small effect sizes suggested that the sample retained for the current study was slightly different from the excluded sample. Ethical and administrative approvals were obtained from the Institutional Review Board of Baylor College of Medicine (STUDY00000134: LD in JD) and HCJPD. The information needed to reproduce all of the reported results is openly accessible through the following link: https://doi.org/10.17605/OSF.IO/FCYHR.

Measures

Positive Achievement Change Tool

The PACT comprises a prescreen (46 items) and a full assessment (126 items). The prescreen, and the full assessment both produce a continuous criminal history score and social history score for each youth. The two assessments generate identical criminal history and social history scores because the same items on both assessments are used in scoring (Baglivio, 2009). The criminal history score consists of 12 indicators, and scores range from 0 to 31, reflecting the severity of prior offending histories and justice system placements. Information for criminal history risk was auto-populated from the Juvenile Management Information System (Hutchins, 2019). The social history score consists of 21 indicators, and scores range from 0 to 18, reflecting individual, family, and environmental criminogenic factors. The overall risk level (ordinal) can be derived from a matrix of the criminal history score and social history score (Table 1). The PACT prescreen, and full assessments (except items for criminal history risk) can be administered by nonclinical staff in juvenile justice intake, diversion, probation, detention, residential commitment, and aftercare settings in semi-structured interviews by using motivational interviewing techniques to obtain information from both the youths and their caregivers, when possible (Parsons Early et al., 2012; Rollnick & Miller, 1995).

Table 1.

Scoring Matrix for the Positive Achievement Change Tool

	Social history score
Criminal history score	0–5	6–9	10–18
0–2	Low	Low	Moderate
3–7	Low	Moderate	High
8–31	Moderate	High	High

Recidivism

Recidivism was defined as having a new record of arrest that is a misdemeanor B or higher severity based on county court records. Furthermore, adult records and offenses taking place outside the state of Texas were not tracked by the jurisdiction and thus were not included in the study. The new arrest was categorized as a violent offense (e.g., homicide, assault, violent sexual offenses, and other delinquency against persons) or any type of reoffending (e.g., violent offenses, property offenses, drug law violations, and delinquency against public order). Youths younger than 17 years old were tracked for 1 year from the date they received the court disposition or completed a stay or commitment at a residential facility according to the court disposition. A 1-year follow-up period has been used in prior research when studying the predictive validity of the PACT (Baglivio & Jackowski, 2013). Regarding recidivism, 1,198 (21.5%) youths were rearrested for any type of delinquency during the 1-year follow-up period and 563 (10.1%) youths committed violent delinquency against a person.

Court Dispositions

Eight types of court dispositions were grouped into four categories according to disposition severity. The first category was deferred adjudication (n = 3,129, 56.1%), which refers to a voluntary dispositional alternative to adjudication wherein the youth, parent/guardian, and court agree on supervision conditions. The second category, probation without placement (n = 1,710, 30.7%), constituted adjudication to probation (i.e., an adjudicated youth is placed on probation in the community), determinant sentence probation (i.e., a youth is ordered to determinate sentence probation by a court or jury for specific violent or habitual conduct), and modified or extended probation (i.e., an extension of a prior court order wherein the youth continues probation in the community). The third category, probation with placement (n = 646, 11.6%), included adjudication to probation with placement (i.e., an adjudicated youth is ordered to placement in a special facility, i.e., an in-county facility or a private facility—in or out of state—paid for by the county, which provides services that the local facilities cannot) and determinate sentence probation with placement (i.e., an adjudicated youth is ordered to determinate sentence probation for specific violent or habitual conduct in a special facility). The fourth category was commitment (n = 93, 1.7%), which included an indeterminate and determinate commitment to the Texas Juvenile Justice Department. Given the relatively small sample size for commitment, we collapsed commitment into probation with placement (n = 739, 13.3%). Thus, probation with placement is the most severe disposition, followed by probation without placement and deferred adjudication.

Analytical Plan

We explored racial/ethnic differences in the PACT measures, recidivism, and court dispositions by running a series of bivariate statistics. We performed one-way between-subjects analyses of variance (ANOVAs) to evaluate observed mean differences in the criminal history and social history scores between White, Black, and Hispanic youths. When significant group differences were detected, we conducted a Tukey’s honestly significant difference (HSD) post hoc test for pairwise comparisons (Tukey, 1949). Pearson’s χ² test was used to assess the association between race/ethnicity and recidivism. Two dummy-coded variables (Black and Hispanic) were created to capture race/ethnicity with White youths as the reference group. Given the ordinal nature of the overall risk level and court dispositions, we applied OLR with proportional odds to evaluate the association between each of these variables and race/ethnicity, whereby the overall risk level/court dispositions were regressed on Black and Hispanic.

We used logistic regression models to examine the predictive bias of the PACT. In Model 1.1, general recidivism was regressed on Black, Hispanic, criminal history, social history, Black × Criminal History, Black × Social History, Hispanic × Criminal History, and Hispanic × Social History. Model 2.1 had the same set of independent variables as Model 1.1, except that the outcome variable was violent recidivism. In Model 3.1, general recidivism was regressed on Black, Hispanic, overall risk, Black × Overall Risk, and Hispanic × Overall Risk. Model 4.1 was similar to Model 3.1, except that general recidivism was replaced with violent recidivism. These four conditional models allowed for evaluating slope differences between the racial/ethnic groups. When there was a lack of strong evidence to support any slope differences, unconditional models without interaction terms were used to evaluate intercept differences. Thus, an unconditional model was developed for each corresponding condition model (Models 1.2, 2.2, 3.2, and 4.2) by excluding the interaction terms. We used the glm command in the R programming environment to perform the logistic regressions (R Core Team, 2021).

We used OLR with proportional odds to evaluate disparate application of the PACT on court dispositions. In Model 5.1, dispositions were regressed on Black, Hispanic, criminal history, social history, Black × Criminal History, Black × Social History, Hispanic × Criminal History, and Hispanic × Social History. In Model 6.1, dispositions were regressed on Black, Hispanic, overall risk, Black × Overall Risk, and Hispanic × Overall Risk. Except for the exclusion of the interaction terms, unconditional Models 5.2 and 6.2 corresponded to Models 5.1 and 6.1, respectively. OLR was performed using the VGAM package (Yee, 2010) embedded in R (R Core Team, 2021).

When a moderating effect was detected, we calculated the simple effect of the PACT on the likelihood of recidivism/dispositions by racial/ethnic groups. When a moderating effect was not detected, the estimates for Black and Hispanic in the corresponding unconditional model represent intercept differences. To facilitate interpretation, we plotted the predicted probability for recidivism/dispositions depending on whether slope differences or intercept differences were detected. These group regression lines portray what the actual likelihood is. For plots related to the predicted probability of recidivism, a combined-group regression line was included, which was derived from a regression model in which recidivism was regressed on the PACT scores based on the entire sample (Cleary, 1968). This combined-group regression line/common regression line served as a reference that allowed us to capture overprediction or underprediction of recidivism for certain groups (Cleary et al., 1975).

Following the recommendations of Cumming (2014), we refrained from taking the conventional approach of making inferences or conclusions about the presence of interactions between race/ethnicity and the PACT using p-value thresholds. Instead, we relied on an estimate of the odds ratio (OR) with 95% CIs. When a 95% CI crosses 1, it suggests that the moderating effect is inconclusive. Conversely, when a 95% CI does not cross 1, it suggests the presence of a moderating effect. In addition, the width (margin of error) of a 95% CI allows for evaluating the uncertainty of the OR. To facilitate interpretation, we plotted the ORs and their corresponding 95% CIs for all interaction terms (see Figures S1 and S2 in the online supplemental materials).

Results

Bivariate Statistics

Results of the one-way between-subjects ANOVA revealed statistical differences in criminal history among White, Black, and Hispanic justice-involved youths, F(2, 5575) = 48.41, p < .001, η_p² = .017. A Tukey’s HSD test (p < .001) indicated that Black youths (M = 5.83, SD = 3.23) had significantly higher criminal history scores than White youths (M = 4.89, SD = 2.56; Cohen’s d = 0.32) and Hispanic youths (M = 5.10, SD = 2.69; Cohen’s d = 0.24), whereas no statistically significant difference was found between White and Hispanic youths (p = .236, Cohen’s d = 0.08). There was little evidence supporting racial/ethnic differences in social history, F(2, 5575) = 1.637, p = .195, η_p² < .001. Specifically, the average social history score was similar among White (M = 5.04, SD = 3.21), Black (M = 5.27, SD = 3.22), and Hispanic (M = 5.30, SD = 3.18) youths. For social history, Cohen’s d was 0.07 for White and Black youths, 0.08 for White and Hispanic youths, and 0.01 for Black and Hispanic youths. Although the residual normality for the aforementioned one-way ANOVA results was not met (see “Test Normality and Homogeneity” in the online supplemental materials), a sensitivity analysis based on the robust one-way ANOVA showed that violation of the residual normality assumption had a minor impact on the inference of the results (see “Racial/Ethnic Differences in Criminal History and Social Scores by Using Robust Methods” in the online supplemental materials). Thus, the ANOVA results presented here were robust against the violation of ANOVA assumptions. Results of OLR showed that the odds of having a moderate or high level of risk versus a low level of risk were 1.40 (95% CI [1.18, 1.66]) times greater for Black youths than for White youths and were 1.15 (95% CI [0.97, 1.36]) times greater for Hispanic youths than for White youths.

A larger proportion of Black youths (24.2%) fell into the general recidivism category, followed by Hispanic (22.1%) and White (8.6%) youths, χ²(2) = 73.32, p < .001, Cramér’s V = .11. Similarly, a larger proportion of Black youths (12.8%) fell into the violent recidivism category, followed by Hispanic (9.1%) and White (3.6%) youths, χ²(2) = 50.95, p < .001, Cramér’s V = .10. The odds of having probation without placement or probation with placement versus deferred adjudication were 2.14 (95% CI [1.78, 2.57]) times greater for Black youths than for White youths and were 1.83 (95% CI [1.52, 2.20]) times greater for Hispanic youths than for White youths.

Predictive Bias

Table 2 presents the results of the conditional and unconditional logistic regression models predicting general and violent recidivism from race/ethnicity, criminal history, and social history. For general recidivism (Model 1.1), ORs for the four interaction terms were all close to 1 (ORs = 0.98–1.01), and their 95% CIs all crossed 1, indicating that the slopes for the two continuous risk scores on general recidivism did not vary as a function of race/ethnicity. Their small margins of error indicated relatively high certainty on the point estimate for the four interaction terms (see Figure S1 in the online supplemental materials). Results of the unconditional model (Model 1.2) showed that Black (OR = 3.28, 95% CI [2.46, 4.46]) and Hispanic (OR = 2.98, 95% CI [2.23, 4.05]) were positively related to the likelihood of general recidivism, suggesting that the intercepts for Black and Hispanic youths were greater than those for White youths. We plotted the predicted probability of general recidivism according to Model 1.2. Figure 1A presents the predicted probability of general recidivism as a function of criminal history and race/ethnicity with a median social history score (Mdn = 5), whereas Figure 1B presents the predicted probability of general recidivism as a function of social history and race/ethnicity with a median criminal history score (Mdn = 5). In both panels, regression lines for Black and Hispanic youths lie above and overlap with the combined-group line (i.e., common regression line), whereas the regression line for White youths lies below the combined-group line, suggesting that the likelihood of general recidivism was overpredicted for White youths at any given criminal history and social history. Thus, one will systematically interpret White youths as having a higher level of risk of reoffending measured by the two types of risk scores than is actually the case. Thus, we found intercept differences in the form of overprediction of general recidivism for White justice-involved youths but no differences in slopes across race and ethnicity.

Table 2.

Results of Conditional and Unconditional Logistic Regression Models Predicting Recidivism From Race/Ethnicity, Criminal History, and Social History

	General recidivism		Violent recidivism
Predictor	Model 1.1 OR [95% CI]	Model 1.2 OR [95% CI]	Model 2.1 OR [95% CI]	Model 2.2 OR [95% CI]
Intercept	0.03 [0.01, 0.06]	0.03 [0.02, 0.05]	0.01 [0.00, 0.02]	0.01 [0.01, 0.02]
Black	3.45 [1.64, 7.51]	3.28 [2.46, 4.46]	7.29 [2.55, 22.93]	3.66 [2.42, 5.83]
Hispanic	3.23 [1.53, 7.05]	2.98 [2.23, 4.05]	3.33 [1.15, 10.63]	2.59 [1.70, 4.13]
Criminal history	1.05 [0.95, 1.15]	1.04 [1.02, 1.06]	1.23 [1.09, 1.40]	1.05 [1.02, 1.08]
Social history	1.16 [1.07, 1.26]	1.16 [1.14, 1.19]	1.07 [0.94, 1.20]	1.16 [1.13, 1.19]
Black × Criminal History	1.00 [0.90, 1.11]		0.84 [0.73, 0.95]
Black × Social History	0.99 [0.91, 1.08]		1.07 [0.94, 1.22]
Hispanic × Criminal History	0.98 [0.88, 1.09]		0.86 [0.75, 0.98]
Hispanic × Social History	1.01 [0.92, 1.10]		1.11 [0.98, 1.26]
Residual deviance (df)	5,447 (5569)	5,448 (5573)	3,421 (5569)	3,431 (5573)

Note. OR = odds ratio; CI = confidence interval.

Figure 1.

Probability of Recidivism Predicted by the Positive Achievement Change Tool for White, Black, and Hispanic Justice-Involved Youths. Note. Results are shown separately for the probability of general recidivism as a function of (A) criminal history and (B) social history (both plotted using estimates from Model 1.2), the probability of violent recidivism as a function of (C) criminal history and (D) social history (both plotted using estimates from Model 2.1), the probability of general recidivism as a function of overall risk (E; plotted using estimates from Model 3.2), and the probability of violent recidivism as a function of overall risk, (F; plotted using estimates from Model 4.2).

Results of Model 2.1 indicated that Black and Hispanic moderated the association between criminal history and violent recidivism because their 95% CIs did not cross 1. The high precision of the estimates for the two interactions was evidenced by the short 95% CIs (Figure S1). Simple slopes for the effect of criminal history on violent recidivism were 1.23 (95% CI [1.09, 1.40]), 1.03 (95% CI [0.99, 1.07]), and 1.06 (95% CI [1.01, 1.11]) for White, Black, and Hispanic youths, respectively. Figure 1C, plotted using estimates from Model 2.1, shows that when the criminal history score was below 13, for youths with a typical social history score (Mdn = 5), the probability of violent recidivism was slightly overpredicted for White youths but slightly underpredicted for Black youths. However, when the criminal history score was above 13, the probability of violent recidivism was underpredicted for White youths but adequately predicted for Black youths. For Hispanic youths, the probability of violent recidivism was slightly overpredicted regardless of the level of criminal history. Figure 1D, plotted using estimates from Model 2.1, shows the predicted probability of violent reoffending along with the social history score when the criminal history score was set to the median. It shows that the probability of violent reoffending was overpredicted for White youths relative to Black and Hispanic youths at any given social history score. Thus, findings revealed slope differences for criminal history and intercept differences for social history in predicting the probability of violent reoffending between the three racial/ethnic groups.

Table 3 shows results regarding the associations between the overall risk and recidivism. According to Models 3.1 and 4.1, the 95% CIs for the two interactions consistently crossed 1 and had sizable width (Figure S1), suggesting that the moderating effect of race/ethnicity on the association between the overall risk and general/violent recidivism was inconclusive with relatively large uncertainty. Thus, we evaluated intercept differences from the unconditional Models 3.2 and 4.2. Black (OR = 3.16, 95% CI [2.37, 4.28]) and Hispanic (OR = 2.95, 95% CI [2.21, 4.00]) youths had a higher likelihood of general recidivism than White youths after we accounted for the overall risk. This pattern held true for violent recidivism (Black youth: OR = 3.57, 95% CI [2.36, 5.67]; Hispanic youth: OR = 2.58, 95% CI [1.70, 4.11]). Figures 1E and 1F were plotted using estimates from Models 3.2 and 4.2, respectively. For any given overall risk, one will systematically interpret White youths as having a higher likelihood of general and violent reoffending than is actually the case. For Black and Hispanic youths, no systematic predictive bias was detected for the two types of recidivism.

Table 3.

Results of Conditional and Unconditional Logistic Regression Models Predicting Recidivism From Race/Ethnicity and Overall Risk

	General recidivism		Violent recidivism
Predictor	Model 3.1 OR [95% CI]	Model 3.2 OR [95% CI]	Model 4.1 OR [95% CI]	Model 4.2 OR [95% CI]
Intercept	0.08 [0.05, 0.11]	0.06 [0.05, 0.08]	0.03 [0.01, 0.04]	0.03 [0.02, 0.04]
Black	2.67 [1.82, 4.06]	3.16 [2.37, 4.28]	3.84 [2.12, 7.65]	3.57 [2.36, 5.67]
Hispanic	2.40 [1.64, 3.65]	2.95 [2.21, 4.00]	2.17 [1.18, 4.37]	2.58 [1.70, 4.11]
Overall risk	1.45 [1.00, 2.07]	1.82 [1.68, 1.97]	1.80 [1.05, 3.02]	1.86 [1.68, 2.07]
Black × Overall Risk	1.25 [0.86, 1.83]		0.93 [0.54, 1.62]
Hispanic × Overall Risk	1.31 [0.90, 1.92]		1.20 [0.70, 2.10]
Residual deviance (df)	5,500 (5572)	5,502 (5574)	3,455 (5572)	3,460 (5574)

Note. OR = odds ratio; CI = confidence interval.

Disparate Application

Results of OLR models testing the moderating effect of race/ethnicity on the associations of court dispositions with a criminal history and social history are presented in Table 4. The results for Model 5.1 show that the four interaction terms were all close to 1 (ORs = 0.95–0.98), and their corresponding 95% CIs all crossed 1. According to Figure S2A, 95% CIs for the four interaction terms had a relatively narrow width, indicating relatively high precision. Taken together, slopes for criminal history and social history did not vary as a function of race and ethnicity. Thus, the corresponding unconditional model (Model 5.2) was evaluated and interpreted. The estimated odds of Black youths receiving a less severe disposition rather than a hasher disposition were 0.53 (95% CI [0.43, 0.65]) times the estimated odds for White youths. In other words, Black youths were 1.88 (95% CI [1.53, 2.30]) times more likely to receive probation with placement rather than probation without placement or deferred adjudication (or probation with or without placement rather than deferred adjudication) compared with White youths. Hispanic youths were 1.91 (95% CI [1.56, 2.34]) times more likely to receive a harsher disposition compared with White youths. A 1-unit increase in criminal history and social history was related to odds of 1.39 (95% CI [1.36, 1.43]) and 1.29 (95% CI [1.26, 1.32]), respectively, of receiving a disposition in the harsher direction rather than, the less harsh direction.

Table 4.

Results of Conditional and Unconditional Ordinal Logistic Regression Models Predicting Court Dispositions From Race/Ethnicity, Criminal History, and Social History

Predictor	Model 5.1 OR [95% CI]	Model 5.2 OR [95% CI]
Intercept 1	36 [21, 60]	49 [39, 63]
Intercept 2	365 [212, 627]	504 [379, 671]
Black	0.71 [0.40, 1.25]	0.53 [0.43, 0.65]
Hispanic	0.82 [0.46, 1.45]	0.52 [0.43, 0.64]
Criminal history	0.73 [0.68, 0.79]	0.72 [0.70, 0.73]
Social history	0.80 [0.76, 0.85]	0.77 [0.76, 0.79]
Black × Criminal History	0.98 [0.91, 1.06]
Black × Social History	0.97 [0.91, 1.03]
Hispanic × Criminal History	0.97 [0.90, 1.05]
Hispanic × Social History	0.95 [0.90, 1.01]
Residual deviance (df)	8,433 (11146)	8,436 (11150)

Note. Disposition was treated as an ordered-categorical variable with three categories: 1 = deferred adjudication, 2 = probation without placement, and 3 = probation with placement. Intercept 1 represents the log odds of being equal to 1 (deferred adjudication vs. probation without placement and probation with placement). Intercept 2 represents the log odds of being less than or equal to 2 (deferred adjudication and probation without placement vs. probation with placement). OR = odds ratio; CI = confidence interval.

Figure 2 was plotted using estimates from Model 5.2. The three regression lines in each of the six plots have identical shapes, echoing the homogeneity of slopes across race/ethnicity. Although the regression line for White youths intersects with the regression lines for Black and Hispanic youths for probation without placement, this does not mean that the slopes are significantly different. If the regression line for probation without placement for White youths was shifted in a parallel manner toward the left, it would overlap with the regression lines for Black and Hispanic youths, suggesting intercept differences across race and ethnicity. The almost indistinguishable regression lines for Black and Hispanic youths mirrored the similar estimates in Model 5.2 (Black youths: OR = 0.53, 95% CI [0.43, 0.65]; Hispanic youths: OR = 0.52, 95% CI [0.43, 0.64]). When the criminal history score was lower than 10, Black and Hispanic youths were less likely to receive deferred adjudication but more likely to receive probation without placement and probation with placement compared with White youths. When the criminal history score was above 10, Black and Hispanic youths were less likely to receive deferred adjudication and probation without placement but still more likely to receive probation with placement compared with White youths. The same pattern of predicted probability for dispositions was also observed for social history, except that the lower or higher predicted probability for probation without placement occurred at a score of 12. Taken together, these results showed that Black and Hispanic youths were more likely to receive harsher dispositions than White youths when they had the same criminal history score or social history score.

Figure 2.

Predicted Probability of Court Dispositions with Criminal History (Top Row) and Social History (Bottom Row) for White, Black, and Hispanic Justice-Involved Youths. Note. Results are shown separately for (A, D) deferred adjudication, (B, E) probation without placement, and (C, F) probation with placement. All panels were plotted using estimates from the unconditional Model 5.2.

Last, we investigated the extent to which overall risk was associated with the likelihood of court dispositions conditional on race/ethnicity. Results are presented in Table 5. Given that the 95% CIs for Black × Overall Risk (OR = 0.76, 95% CI [0.59, 0.98]) and Hispanic × Overall Risk (OR = 0.75, 95% CI [0.58, 0.97]) did not cross 1, the association between the overall risk level and court dispositions depended on race and ethnicity. In addition, the narrow margins of error, as illustrated in Figure S3 in the online supplemental materials, demonstrated relatively high precision for the estimates of the two interaction terms. The simple slopes were 0.30 (95% CI [0.24, 0.38]), 0.23 (95% CI [0.21, 0.25]), and 0.23 (95% CI [0.20, 0.25]) for White, Black, and Hispanic youths, respectively, suggesting that the association between the overall risk level and court dispositions was stronger for Black and Hispanic youths than for White youths.

Table 5.

Results of Conditional and Unconditional Ordinal Logistic Regression Models Predicting Court Dispositions From Race/Ethnicity and Overall Risk Level

Predictor	Model 6.1 OR [95% CI]	Model 6.2 OR [95% CI]
Intercept 1	4.73 [3.69, 6.07]	5.72 [4.73, 6.91]
Intercept 2	40 [31, 52]	48 [39, 59]
Black	0.60 [0.45, 0.78]	0.49 [0.40, 0.59]
Hispanic	0.65 [0.50, 0.86]	0.53 [0.43, 0.64]
Overall risk	0.30 [0.24, 0.38]	0.23 [0.22, 0.25]
Black × Overall Risk	0.76 [0.59, 0.98]
Hispanic × Overall Risk	0.75 [0.58, 0.97]
Residual deviance (df)	8,991 (11149)	8,996 (11151)

Note. Disposition was treated as an ordered-categorical variable with three categories: 1 = deferred adjudication, 2 = probation without placement, and 3 = probation with placement. Intercept 1 represents the log odds of being equal to 1 (deferred adjudication vs. probation without placement and probation with placement). Intercept 2 represents the log odds of being less than or equal to 2 (deferred adjudication and probation without placement vs. probation with placement). OR = odds ratio; CI = confidence interval.

Figure 3 shows the predicted probability of court dispositions as a function of overall risk and race/ethnicity according to the parameters derived from Model 6.1. The shapes of regression lines for Black and Hispanic youths were nearly identical, but they were substantially different from the shape of the regression line for White youths. The shape differences corresponded to the slope differences for the three racial/ethnic groups. With a low or moderate level of overall risk, Black and Hispanic youths had a lower probability of receiving deferred adjudication but a higher probability of receiving probation without placement and probation with placement compared with White youths. With a high level of overall risk, the probability of receiving deferred adjudication and probation without placement was lower for Black and Hispanic youths, whereas the probability of receiving probation with placement remained higher for Black and Hispanic youths than for White youths. Thus, Black and Hispanic youths were more likely to receive harsher dispositions compared with White youths when they were matched on the overall risk produced by the PACT.

Figure 3.

Predicted Probability of Court Dispositions with the Overall Risk Level for White, Black, and Hispanic Justice-Involved Youths. Note. Results are shown separately for (A) deferred adjudication, (B) probation without placement, and (C) probation with placement. All panels were plotted using estimates from the conditional Model 6.1.

Discussion

The findings of the current study expand the existing literature on the debate surrounding R/ED in RAIs. We used county-record data from HCJPD to investigate predictive bias and disparate application of the PACT among White, Black, and Hispanic justice-involved youths. We found that the criminal history score assessed by the PACT was problematic in predicting violent recidivism for justice-involved youths because the association between these factors was not equivalent across race and ethnicity. Moreover, there was evidence showing that the overall risk derived from the PACT was associated with harsher sanctioning decisions for Black and Hispanic justice-involved youths than for White justice-involved youths.

Predictive Bias

We found racial/ethnic differences in the intercept with respect to the associations between criminal history and general recidivism, social history and general recidivism, social history, and violent recidivism, overall risk and general recidivism, and overall risk and violent recidivism. The intercept differences consistently showed that the risk of reoffending was underpredicted for Black and Hispanic youths but overpredicted for White youths. Therefore, using the social history score or the overall risk in making sanctioning decisions would mean that Black and Hispanic youths would have a lower likelihood of receiving harsher sanctions. The same conclusion also holds true for criminal history and general recidivism. It means that fewer Black and Hispanic youths would receive harsher sanctioning decisions. To that end, the PACT would have the potential to reduce R/ED in the juvenile justice system. However, Black and Hispanic youths who did not receive sanctions would have a higher probability of reoffending in the future compared with White youths who have similar PACT scores/risk levels. In this sense, the use of the PACT would increase the risk to public safety. Our findings are consistent with the consensus made for fairness in RAIs—it is almost impossible to simultaneously maximize accuracy and fairness (Berk et al., 2021; Chouldechova, 2017; Kleinberg et al., 2016). Thus, the focus of the debate about R/ED in RAIs should be shifted toward how to reach a compromise between public safety and fairness.

However, intercept differences identified in the current study are not sufficient to conclude that the PACT is biased (Meade & Tonidandel, 2010). The predictive bias, as indexed by intercept difference, may merely be a statistical artifact (Warne et al., 2014). Generalized linear models, including logistic regression and OLR, which we performed in the current study, are built on the assumption that the predictors are free of measurement error. In practice, criminal history, social history, and overall risk level generated by the PACT are unlikely to be free of measurement error because the information has to be solicited from various sources, including court records, youths, parents, teachers, and social workers. There is evidence in education testing showing that when alternative methods are used by taking measurement errors into account, the intercept differences disappear (Warne et al., 2014). Beyond measurement errors, biased criterion, omitted covariates, and inflated Type I error are all associated with intercept differences. Thus, the underprediction of risk for Black and Hispanic justice-involved youths should not be over-interpreted.

Apart from intercept differences, our results also revealed slope differences between White justice-involved youths and racial/ethnic minority justice-involved youths. These slope differences indicate that the criminal history scores function differentially across racial/ethnic groups. Specifically, when a criminal history score was below 13, the likelihood of violent recidivism was underpredicted for Black youths but overpredicted for White and Hispanic youths; when a criminal history score was above 13, the likelihood of violent recidivism was slightly overpredicted for Black and Hispanic youths but underpredicted for White youths. This is concerning because it suggests that the meaning of criminal history is different for White, Black, and Hispanic youths when predicting violent recidivism. Thus, using the PACT criminal history score to predict violent recidivism is problematic.

Our findings are consistent with those of a previous study (McKenzie, 2018), in which the author found that the slopes of criminal history and social history in predicting general recidivism were similar between White non-Hispanic and minority youths in the Montgomery County Juvenile Probation Department, Texas. Yet Baglivio and Jackowski (2013) found that the slope of social history in predicting general recidivism was greater for Hispanic youths than for White and Black youths in the Florida Department of Juvenile Justice. The inconsistencies were difficult to reconcile because of the numerous confounders, such as jurisdiction cultures and sample size and composition. Nevertheless, our findings, together with those of previous research (Baglivio & Jackowski, 2013; Campbell et al., 2018, 2020; McCafferty, 2018; Onifade et al., 2008, 2009; Perrault et al., 2017; Rembert et al., 2014; Schwalbe et al., 2006, 2007; Vincent et al., 2011), highlight the importance of recognizing the context-dependent feature of RAIs. It is important to systematically appraise the predictive bias between racial/ethnic groups when an RAI is applied in a new context.

Disparate Application

Concerns surrounding the use of RAIs in the justice system have primarily been centered on the socially inequitable consequences caused by the RAI results (Freeman et al., 2021; Holder, 2014). Our study provides empirical evidence that explicitly contributes to the debate. Although the moderating effect of race/ethnicity on the association of criminal history and social history on court dispositions was inconclusive, we found that Black and Hispanic youths were more likely to receive harsher dispositions than White youths (intercept differences). This means that judges interpreted Black and Hispanic youths as more threatening to public safety and delivered hasher sanctioning decisions, even when they had similar criminal history and social history scores as White youths. Again, intercept differences should be interpreted with caution because they may represent statistical artifacts (Meade & Tonidandel, 2010; Warne et al., 2014).

We found that the association between overall risk and court dispositions depended on race and ethnicity. Specifically, Black and Hispanic youths were more likely to receive probation with placement (most severe) but less likely to receive deferred adjudication (least severe) compared with White youths, irrespective of the risk level. When the overall risk level was low or moderate, Black and Hispanic youths were more likely to receive probation without placement (medium severe); however, when the overall risk level was high, Black and Hispanic youths were less likely to receive probation without placement. In general, judges assigned more blame to Black and Hispanic youths and therefore sanctioned them more harshly. In this regard, our findings support the proposition that the use of the overall risk produced by the PACT disadvantages Black and Hispanic youths and therefore exacerbates R/ED in the jurisdiction. Nonetheless, one should bear certain caveats in mind while reaching a conclusion. Court judges make the decisions, not RAIs (Perrault et al., 2017). The PACT was not the sole measure that courts used to reach their judgments (Lowder et al., 2022). A court disposition is a product of legal factors (e.g., charge severity, charge type) and extralegal factors (e.g., RAI results and sociopsychological reactions of court judges). It is likely that those unmeasured confounders play a profound role in determining court dispositions.

Taking a step forward, however, we argue that it would be premature to conclude that the use of RAIs leads to socially undesirable consequences that are unfavorable to racial and ethnic minorities. There is evidence showing that an RAI sometimes produces socially undesirable consequences for White offenders. For instance, Lowder et al. (2019) also found evidence that LSI-R scores were related to harsher treatment (longer sentence length) for White adult probationers relative to Black probationers. If one took a relatively liberal perspective that intercept differences of criminal history and social history on court dispositions identified in the current study were a statistical artifact, one could even claim that the two continuous risk scores would not result in biased consequences. In all, it is safe to say that racial/ethnic disparities in the consequences of applying RAI are context-sensitive.

Implications for Research, Policy, and Practice

Our findings have several implications for research, policy, and practice surrounding the debate of whether RAI results exacerbate R/ED in the juvenile justice system. First, results related to the PACT reported here and in prior research (Baglivio & Jackowski, 2013; Hutchins, 2019; McKenzie, 2018), in combination with studies focusing on other RAIs (Campbell et al., 2018, 2020; McCafferty, 2018; Onifade et al., 2008, 2009; Perrault et al., 2017; Rembert et al., 2014; Schwalbe et al., 2006, 2007; Vincent et al., 2011), reveal substantial variability in predictive bias across race and ethnicity. This variation can be ascribed to numerous reasons, such as the quality of RAIs, the climate and culture of the juvenile justice system, the adherence to RAIs, the reliability of recidivism, and the heterogeneity of the youths with whom RAIs were implemented, among others (Andrews et al., 2011). The variable predictive bias identified across studies underscores the importance of systematically evaluating the validity evidence of RAIs (e.g., predictive validity, predictive bias, and disparate application by demographic groups) when they are implemented in a new setting. It would be unwise to exaggerate the generalizability of the validity evidence of RAIs across settings. Even when the same RAI is administered in the same jurisdiction, its performance may evolve over time. Thus, it is the responsibility of researchers advocating for RAIs to continue to shed light on the validity and fairness of these measures. Note that the dynamic adaptability of RAIs across settings does not mean that these research-based actuarial tools are problematic; rather, it renders evidence and provides the opportunity to update our understanding of RAIs and, ultimately, promote social justice.

Second, an RAI exhibiting predictive bias can be considered a sufficient but not a necessary condition for determining whether it leads to harsher sanctions against racial/ethnic minorities. As our results illustrate, although race and ethnicity moderated the association between criminal history and violent recidivism, this moderating effect did not translate into inequitable sanctioning decisions. In fact, the association between criminal history and court dispositions did not depend on race and ethnicity. Despite the absence of a moderating effect on the association between overall risk and general and violent recidivism, race and ethnicity moderated the association between overall risk and court dispositions. An RAI free of predictive bias does not guarantee that it will be applied equivalently across different racial/ethnic groups. Given that the debate about racial/ethnic disparities in RAI is focused on sanctioning decisions, future researchers should extend their investigations to disparate application.

Third, shifting attention to disparate application should not simplify the complex processes of court decision-making. In practice, RAI results are not the only piece of information necessary to determine probation and disposition decisions. This process also involves statutes, administrative guidelines, operating procedures (Hoge, 2002), and the feelings, perceptions, and emotional reactions of professionals (Peck & Jennings, 2016). Those are all sources that may compromise the fairness of RAIs in practice. It is crucial to bear in mind that probation officers and court judges should not base decisions exclusively on RAI results (Viljoen et al., 2019).

Fourth, concern about RAIs has overwhelmingly revolved around the risk of overclassifying racial/ethnic minorities into high-risk groups on the basis of certain biased static risk factors. Indeed, gauging the likelihood of reoffending is central to many RAIs. However, RAIs also assess risk factors amenable to change through treatments and interventions aiming to reduce delinquency (need) and identify personal and environmental characteristics relevant to youths’ response to interventions (responsivity; Andrews & Bonta, 2010). Given that rehabilitation is the cornerstone of the juvenile justice system, reliably assessing youths’ needs and appropriately delivering interventions targeting these needs become equally or even more important than evaluating how dangerous a youth may be. Accordingly, it is urgent that we evaluate racial/ethnic disparities regarding their needs and interventions. The debate about RAIs and race/ethnicity in the juvenile justice system is incomplete without considering the other two essential functions of modern RAIs—need and responsivity.

Limitations

It is important to note that the current study has limitations. First, given the small sample size of American Indian/Alaskan Native and Asian/Pacific Islander youths, we did not examine whether the PACT was biased toward them relative to other racial/ethnic groups. In fact, all existing research focusing on the predictive bias of RAIs has not involved American Indian/Alaskan Natives and Asian/Pacific Islanders. This is in part because R/ED in the justice system are more obvious for Black and Hispanic youths than for American Indian/Alaskan Natives and Asian/Pacific Islanders. However, Indigenous people are also overrepresented in criminal justice systems in the United States (U.S. Sentencing Commission, 2013). It is important to bear in mind that the ultimate goal of the adoption of RAIs is to promote public safety by replacing incarceration with rehabilitation for justice-involved youths. The bottom line for accomplishing this goal is to ensure social justice. To that end, it is important to ensure that RAIs have the same meaning and may be applied consistently in practice for all racial/ethnic groups.

Second, we acknowledge that gender may play a role in the differential bias of the PACT on recidivism and court dispositions. In fact, there is heated debate about how female youths should be assessed in the juvenile justice system: The gender-neutral hypothesis posits that empirically derived risk factors work equally well for male and female youths, whereas the gender-specific hypothesis contends that certain risk factors (e.g., mental health, relationship dysfunction) should be weighted more for female youths than for male youths (Scott & Brown, 2018). There is evidence that the association between the criminal history score derived from the PACT and recidivism was conditional on both race/ethnicity and gender (Baglivio & Jackowski, 2013). Although our sample size was large enough to test the two-way interactions between race/ethnicity and the PACT, it was not large enough to provide robust estimates for much more complicated models, including additional multiple two-way interactions (Gender × PACT) and multiple three-way interactions (Race/Ethnicity × Gender × PACT). Indeed, tests of a moderating effect have been notoriously underpowered (Meade & Tonidandel, 2010). Efforts to uncover the complicated associations among gender, race/ethnicity, recidivism, and dispositional outcomes will be useful and important.

Third, research on racial/ethnic disparity in decision-making has revealed that race/ethnic effects are more prominent at some stages than others, and many factors contribute to such differences (Peck & Jennings, 2016; Rodriguez, 2010). In the current study, we focused only on the stage of judicial disposition. Further research would greatly benefit from examining the application of RAIs at different stages, such as intake and adjudication.

Fourth, given that the criminal history score, social history score, and overall risk level were all accessible to court actors in HCJPD, we did not know which components were used in informing the court dispositions and how the components were weighted when legal factors were considered. Thus, this study was not able to test whether the racial/ethnic differences in court dispositions were due to the bias of the instrument or to certain external factors in the jurisdiction. This limitation highlights the complexity and challenges of integrating RAIs into the decision-making process. The greater weighting of legal factors (e.g., charge type, charge severity) has the potential to undermine the impact of RAIs in court decisions in the juvenile justice system but possibly at the cost of failing to fulfill rehabilitation goals for justice-involved youths. On the other hand, the greater weighting of extralegal factors, including RAI results, is likely to benefit rehabilitation; however, there is still room for prejudice and discrimination when practitioners interpret RAI results. Last, we would like to reiterate that findings in the current study may not be generalizable to jurisdictions in the same state or different states because the varying predictiveness of RAIs is typical rather than exceptional.

Conclusion

Racial and ethnic minority overrepresentation has been observed in nearly every court proceeding for both adolescents and adults. Some researchers have raised concerns that R/ED in RAI results may exacerbate disproportionate minority contact. Despite substantial efforts to examine the predictive bias of various RAIs by race and ethnicity, little is known about the R/ED of RAI results in actual juvenile court decisions. Our findings suggest that the criminal history score of the PACT is problematic in predicting violent recidivism for youths in HCJPD because the association between criminal history and violent recidivism varies as a function of race and ethnicity. In addition, there is evidence showing that the overall risk derived from the PACT is associated with harsher sanctioning decisions for Black and Hispanic youths compared with White youths. Our study highlights that ensuring that RAI results are consistently interpreted and used in informing decisions is equally as important as ensuring that RAI scores function well in predicting recidivism regardless of race and ethnicity. Because RAIs are usually characterized by within-measure variation (variable performance of risk indicators derived from the same RAI) and between-measure variation (variable performance of RAIs in different contexts), researchers should continue to investigate potential sources of racial/ethnic bias in RAI results across different jurisdictions.

Public Significance Statement:

For White, Black, and Hispanic justice-involved youths, the criminal history score of the Positive Achievement Change Tool (PACT) inconsistently predicts the likelihood of violent reoffending, and the overall risk of reoffending determined by the PACT may be interpreted and used inconsistently when making sanctioning decisions. The predictive bias and disparate application of the PACT may exacerbate racial and ethnic disparities in the jurisdiction. Therefore, the PACT measures should be used with caution.