Evaluating the impact of a street outreach intervention on participant involvement in gun violence

Significance

Community violence interventions (CVIs) are promising approaches to reducing community violence, but little research has examined their effectiveness for mitigating individual-level violence. This quasiexperimental study provides evidence for the impact of CVIs on participant involvement in gun violence across three treatment levels. No statistically significant change in victimization was detected across treatment groups. While program effects on arrests begin to emerge across all treatment levels, only program alumni experienced statistically significant differences from their controls: Alumni are 73.4% less likely to have an arrest for a violent crime in the 2 y following program enrollment. This reduction is large considering the complex systemic factors contributing to gun violence, including situational, contextual, and neighborhood factors outside of programmatic or individual control.

Abstract

The past several years have witnessed increased calls for community violence interventions (CVIs) that address firearm violence while centering local expertise and avoiding the criminal legal system. Currently, little evidence exists on CVI effectiveness at the individual level. This study presents an evaluation of the impact of a street outreach-based CVI [Chicago CRED (Create Real Economic Destiny)] on participant involvement in violence. We used a quasiexperimental design with a treatment sample of 324 men recruited by outreach staff from 2016 to 2021 and a balanced comparison sample of 2,500 men from a network of individuals arrested in CRED’s service areas. We conducted a Bayesian survival analysis to evaluate CRED’s effect on individual violence-related outcomes on three levels of treatment: All enrolled participants, a subsample that made it through the initial phase, and those who completed programming. The intervention had a strong favorable effect on the probability of arrest for a violent crime for those completing the program: After 24 mo, CRED alumni experienced an 11.3 percentage point increase in survival rates of arrest for a violent crime relative to their comparisons (or, stated differently, a 73.4% reduction in violent crime arrests). The other two treatment levels experienced nontrivial declines in arrests but did not reach statistical significance. No statistically significant reduction in victimization risk was detected for any of the treatment levels. Results demonstrate that completion of violence intervention programming reduces the likelihood of criminal legal involvement for participants, despite the numerous systemic and environmental factors that impede personal success.

Gun violence remains a pressing public health issue in the United States, responsible for more than 19,000 homicides in 2020 (1) and an average of more than 70,000 injuries and deaths each year (2). Between 2019 and 2020, the most recent data available, gun homicides increased nearly 35%, representing the highest firearm homicide rate since the 1980s (3). While firearm homicide rates increased across most demographics, non-Hispanic Black males aged 10 to 44 saw the largest increases (3), and firearm-related injuries were the leading cause of death of children aged 1 to 19 in 2020 (4).

America’s most recent increase in gun violence occurred as the country wrestled with the harms and racial disparities caused by mass incarceration and aggressive policing (5, 6). Over the last several years, community violence interventions (CVIs) have been centered in policy and practice to address gun violence without relying on criminal legal responses known to exacerbate racial disparities. CVIs are efforts to address gun violence reduction which center local knowledge and community organizations that identify individuals most at risk for involvement in gun violence and provide targeted interventions (7). One popular CVI approach focuses on street outreach programs that employ individuals with lived experiences [sometimes referred to as “credible messengers” (8)] who insert themselves into the conflicts and social networks at the center of gun violence to mediate disputes, broker peace, and provide life-saving services (8). While attention to such programs—as well as other efforts such as trauma-informed local responses (9) and hospital-based interventions (10)—has grown in recent years, the evidence base on program impact is nascent (11, 12).

This paper presents the results from the evaluation of a pilot study on Chicago CRED (Create Real Economic Destiny), an organization operating CVI programming on the West and South Sides of Chicago, Illinois, where the risk of homicide and gunshot victimization is more than 2.3 times as high as in Chicago as a whole (Fig. 1). CRED employs individuals from the community with lived experience with gun violence to act as outreach staff. Outreach staff then identify and recruit men from their service areas who are at the highest risk of involvement in gun violence into structured and phased violence intervention programming. CRED programming includes mental health counseling and mentoring, educational assistance and tutoring, and job training and professional development (SI Appendix). CRED participants who complete programming then enter an “employment and training” and “alumni” phase. Using an engaged research approach, our team codesigned with CRED an evaluation that captured participant progress through the program with data collected jointly by CRED and embedded research staff. This study presents the individual-impact portion of the evaluation.

Fig. 1.

CRED participants display exceedingly high risk of gunshot victimization and homicide compared to CRED neighborhood peers and Chicago as a whole. CRED participants are 4.1 times as likely to be the victim of a gunshot compared to others in their neighborhood and 9.4 times as likely as Chicago residents at large.

Study design included frank discussions about the realistic impact that street outreach-based CVIs in their current capacity can have on reducing gun violence. Communities experiencing high levels of gun violence often also suffer from systemic disinvestment, racial segregation, underfunded schools, over-policing, and limited opportunities for sufficient legal employment (13–16). CVI interventions typically operate on the individual level, focused on changing participants’ behaviors and attitudes towards violence, and unfold without similar attention and investments in the larger community contexts in which individuals live, learn, work, and play. As such, interventions focusing solely on individual behavior (like trying to reduce gun carrying or enhancing decision-making) without considering or changing situational, contextual, and neighborhood-level factors (like the easy access to firearms or the legacies of segregation) are likely to have limited impact.

Furthermore, street outreach-based CVIs frequently lack the resources to employ a workforce that meets the overwhelming need for services or intervention. There are approximately 230 street outreach workers currently employed throughout Chicago (17) to try and quell gun violence in social networks with approximately 140,000 individuals (18) and, by some estimates, approximately 900 or so street crews, cliques, and gangs (19). Such figures underscore the dire need for violence prevention efforts but also the limited scopes of even the best funded programs; any effort at parsing out possible program effects should take such factors into consideration.

Trying to establish realistic program effects of CVI efforts vis-à-vis larger contextual issues is even more difficult as most evaluation efforts to date have examined community (20, 21), city (22), or group (22) level outcomes, in part because of a reticence to collect individual identifying data among a population that is frequently targeted by the criminal legal system (23, 24). Results from impact studies of CVI and outreach efforts on neighborhood gun violence are mixed at best. A recent study of the CureViolence program in Baltimore, for example, found an association with a reduction in gun violence indicators in some neighborhoods but not others, and in at least one neighborhood, the program was associated with increases in gun violence (25). A study of Philadelphia’s CeaseFire program also found a significant reduction in aggregate shooting rates in some, but not all, program service areas (26). Recent studies of individual impact are rare, but reports on the READI Chicago program (which resembles Chicago CRED’s phased programming structure) (27) and the Gang Reduction Initiative of Denver (28) suggest a strong relationship between CVI programming and reductions in gun-related and shooting arrests but not necessarily shooting victimizations.

The nature of gun violence more generally further complicates CVI evaluation efforts. The majority of assaultive gun violence is nonrandom, concentrating within small geographic spaces (29) and social networks (30, 31). Concentration within social networks is especially problematic for traditional research designs, including those involving randomization, given the extensive interdependence among individuals involved in gun violence. In Chicago, for example, approximately 70% of all fatal and nonfatal shootings occur within a network comprising less than 6% of the city’s population (30) and, on average, members of such cooffending networks in Chicago are less than three handshakes away from an individual linked to a recovered firearm used in a crime (32). This concentration within networks means that even selecting randomly from high-risk populations potentially produces events that are directly or indirectly connected. Further, the smaller the selection criteria (e.g., from a small number of neighborhoods), the greater the likelihood of interdependence among cases. As such, even the best attempts at randomization may likely lead to interdependence within the treatment group and across treatment and control groups (33). Separately, if the risk of gunshot victimization is used as a selection criterion and someone has already been shot, the likelihood of repeat victimization is even smaller than overall victimization rates—an already statistically rare occurrence; by our estimates, fewer than 10,500 individuals in Chicago have reported multiple gunshot injuries since 2011 (0.39% of the average population).

Given the nonrandom concentration of gun violence in small social networks in Chicago (18, 30) and CRED’s nonrandom selection of participants, we utilized a quasiexperimental approach that allowed us to model the selection process as accurately as possible while also leveraging known information about networks of gun violence, thereby excluding possible interdependencies both within the treatment group and between the treatment and control groups. Participant information (n = 410) was linked with administrative data on arrests, victimizations, and gang membership from the Chicago Police Department (CPD) to follow participant outcomes over time (final sample n = 324) and to create a well-balanced statistical comparison group (n = 2,500). See SI Appendix, Table S3 and Methods for sample characteristics and comparison group selection; additional information on selection processes and potential biases are presented in the SI Appendix. We then used Bayesian survival analysis, controlling for various network interdependencies, to estimate the effect of the CRED intervention on violence-related outcomes for all participants. Although nonrandomization prohibits us from calculating precise “intent to treat” (ITT) or “treatment-on-the-treated” (TOT) estimates, CRED’s phased programming allows for an analysis of individual outcomes after defined treatment levels of services, including participants who were selected because staff believed that they were ready to be treated and those that actually advanced to the full treatment phase. To approximate such estimates, we conducted analyses along three subsets of participants: 1) “All Participants” who were selected by CRED to be initiated into the outreach phase of programming (n = 324); 2) “Post Outreach” participants (n = 232) who advanced past the initial outreach phase and into at least programming phase 1; and 3) a group consisting only of CRED Alumni (n = 111) who completed all program phases. In our quasiexperimental design, the All Participants sample and Post Outreach sample might roughly approximate the ITT sample in randomized studies and the Alumni sample approximates the TOT. Additionally, parsing the groups in this way allows us to evaluate how a partial and full treatment with the intervention impacts violence-related outcomes.

The primary outcomes for the survival analysis were arrest for a violent crime (homicide, manslaughter, criminal sexual assault, robbery, aggravated assault, aggravated battery, simple assault, and simple battery; see SI Appendix for discussion of arrest types), homicide or nonfatal gunshot victimization, and an indexed outcome for either victimization or arrest. Separate analyses were run for each outcome. Failure in survival analyses was modeled as one victimization or violent arrest 6, 12, 18, and 24 mo after the enrollment date with CRED (or imputed start date for the comparisons).

To address confounding in baseline risk of victimization and arrest, the effect of enrollment on victimization outcomes conditional on covariates was estimated using Bayesian Additive Regression Trees (BART) (34, 35). BART is advantageous for causal inference in quasiexperimental designs because it requires no a priori information about the shape of the response surface, thereby mitigating a source of bias in the model and allowing for the inclusion of multiple covariates (34). Settings with many possible covariates with differing parametric and nonparametric relationships between predictors and outcomes, as in the study of a program like CRED, benefit from the BART method’s ability to flexibly model such relationships without requiring researchers to specify each relationship’s shape (34). BART has been used to estimate treatment effects in survey research (36), a medical intervention (37), and more recently in social networks (38). For this analysis, we estimate the likelihood of survival —that is, not experiencing—victimization and/or violent arrest for the comparison group and the CRED-enrolled treatment group to assess the effect of the intervention on participants’ involvement in gun violence (38). Covariates included number of arrests and victimizations in the 3 y prior to the start of services or imputed start date; binary code for gang involvement; number of co-arrestees and number of co-arrestees' victimizations in the 3 y prior to start; the proportion of first- and second-order peers receiving street outreach treatment; and year of birth, sex, and race. We examined the probability of each outcome in three separate models.

We hypothesized that CRED participants would show an increased probability of survival on outcomes relating to gun violence (defined as homicide or gunshot victimization and arrest for a violent crime) after recruitment into the program, in relation to a comparison group of demographically similar peers with comparable levels of involvement in gun violence who had no recorded direct or indirect connections to participants. Further, because appropriate dosage of CVI programming is likely essential to violence reduction (39–41), we hypothesized that participants who complete CRED programming (hereon referred to as alumni) would show the strongest increases in the probability of survival on violence-related outcomes relative to the comparison group, with the Post Outreach group effects somewhere in the middle of the All-Sample and Alumni effects.

Results

Treatment Sample and Control Pool Balancing.

CRED participants presented with similar baseline characteristics across programming phases (see SI Appendix, Table S1 for demographics and SI Appendix, Fig. S1 and Table S2 for attrition and retention information). Balancing of the control pool was successful across most covariates of interest (see SI Appendix, Table S3 for sample characteristics) and participants and eligible comparisons had similar proportions of arrests for violent crime types (SI Appendix, Table S4). The comparison pool and treatment sample only differed on one selected covariate in the alumni sample, the number of arrests for violent crime in the 3 y before start date, for which the comparison pool had, on average, 0.7 more arrests than the treatment sample (SI Appendix, Tables S5–S7). Due to the available sample and the collinearity between age and number of arrests, it was not possible to balance completely on both number of previous arrests and year of birth. Because age is a well-known strong predictor of criminal-legal involvement (42, 43), we chose to focus on the models that balance on year of birth. Results from models balanced on previous arrests, but not year of birth, show similar effects to that of the models balanced on year of birth and are provided in SI Appendix, Fig. S2 and Table S12.

Program Effect on Victimization.

Results from the BART models indicate that the survival probability for gunshot victimization or homicide was marginally lower for all three groups of participants relative to the comparison pool. This suggests that program participants, over time, are slightly more likely to experience victimization than geographically and demographically similar comparisons. As seen in Fig. 2, however, the credible intervals overlap, suggesting that program participation had no discernable effect on victimization rates amongst participants, whether considering program completion or not.

Fig. 2.

Victimization survival analysis. Relative to comparisons, all CRED participants (A), participants who advanced past the Outreach phase (B), and alumni (C) were equally likely to survive on the victim of a gunshot or homicide outcome.

Program Effect on Arrests for Violent Crimes.

CRED participation had a modest effect on arrests for violent crimes amongst the All-Participant and Post Outreach participants (Figs. 3 A and B and 4 A and B and SI Appendix, Tables S8 and S9) but a strong effect over 24 mo for Alumni participants (Fig. 3C and SI Appendix, Table S10). While the credible intervals on the survival curves for the All-Participant and Post Outreach samples overlap with their comparisons’ curves, they do so marginally, indicating a trend towards increased survival on arrests for violent crimes for these samples. For program alumni, model results show an increased survival on arrest for violent crime which also appears to get larger over time. BART estimates suggest that, for alumni, program participation resulted in a mean increase in survival on the outcome of arrest for a violent crime of 4.22 percentage points at 6 mo postprogram start (mean estimate of survival = 0.986, CI = 0.978 to 0.994, difference in mean survival = 4.48%, difference in mean failure = −75.43%), 7.06 percentage points at 12 mo (mean = 0.976, CI = 0.960 to 0.989, difference in mean survival = 7.80%, difference in mean failure = −74.74%), 9.34 percentage points at 18 mo (mean = 0.967, CI = 0.946 to 0.985, difference in mean survival = 10.69%, difference in mean failure = −74.02%), and 11.30 percentage points at 24 mo postprogram start (mean = 0.959, CI = 0.936 to 0.981, difference in mean survival = 13.36%, difference in mean failure = −73.38%, SI Appendix, Table S10). As such, CRED alumni participants are nearly 74% less likely to be arrested for a violent crime than untreated individuals in the comparison group.

Fig. 3.

Arrest for violent crime survival analysis. Relative to comparisons, all CRED participants (A) and those who advanced past the Outreach phase (B) were marginally more likely to survive on having an arrest for a violent crime after program start. CRED participation had a large increasing effect of likelihood of survival on arrest for a violent crime for program alumni (C).

Fig. 4.

Indexed outcome survival analysis. relative to comparisons, CRED participants (A) were marginally more likely to survive on having an arrest for a violent crime or a gunshot or homicide victimization after program start, while CRED participation had no impact on this outcome for the group of participants who advanced past the Outreach phase (B). CRED participation had a large increasing effect of likelihood of survival on arrest for a violent crime or gunshot victimization for program alumni (C).

Program Effect on Indexed Outcome of Either Arrest for Violent Crimes or Victimization.

Results present intervention effects for the All-Participant sample (Fig. 4A) as well as increases in survival probability among Alumni (Fig. 4C) but not for the Postoutreach participants (Fig. 4B). For CRED alumni, BART estimates suggest increases in survival for the combined outcomes of 3.41 percentage points at 6 mo postprogram start (mean = 0.965, CI = 0.952 to 0.978, difference in mean survival = 3.66%, difference in mean failure = −49.28%), 5.87 percentage points at 12 mo (mean = 0.938, CI = 0.916 to 0.960, difference in mean survival = 6.68%, difference in mean failure = −48.33%), 7.85 percentage points at 18 mo (mean = 0.915, CI = 0.885 to 0.944, difference in mean survival = 9.39%, difference in mean failure = −47.85%), and 9.44 percentage points at 24 mo postprogram start (mean = 0.895, CI = 0.860 to 0.930, difference in mean survival = 11.79%, difference in mean failure = −47.24%, SI Appendix, Table S10). For the largest group of all CRED participants, a reduction in the probability of either arrest for a violent crime or gunshot or homicide victimization is realized after the 24-mo benchmark, suggesting longer-term effects of CRED programming on violence-related outcomes (Fig. 4A and SI Appendix, Table S11).

Discussion

Evaluation of CVIs presents unique challenges for researchers due to the statistically rare and nonrandom nature of community gun violence, the corollary nonrandom selection of program treatment areas and participants, and the deep situational and historical contexts of communities facing extensive disinvestment, segregation, and marginalization. Past evaluations of CVIs have shown mixed results at the community level, and few have examined individual-level outcomes. Few contemporary studies to our knowledge have assessed the effect of a violence prevention intervention on individual risk for victimization or arrest; however, recent papers on similar programs, READI Chicago (27) and the Gang Reduction Initiative of Denver (28), using different sampling and methodological approaches, also find a relationship between program participation and reductions in gun-related arrests (27, 28). Results from the evaluation of READI Chicago report a statistically significant 79% reduction in arrests for homicides and shootings for the sample of men who came in through the outreach referral pathway (27), which is quite like CRED’s recruitment process (SI Appendix). Similarly, the Denver evaluation reported that treated clients were 70% less likely to perpetrate violence relative to their untreated peers (28). READI’s and Denver’s evaluators report their results in terms of likelihood of arrest, while we present our results in terms of likelihood of survival, meaning likelihood of no arrest. When we convert our 11.3 percentage point increase in survival for CRED alumni to the percentage change in the likelihood of failure, we determine an effect similar in magnitude to READI: a 73.4% reduction in the likelihood of CRED participants being arrested for a violent crime relative to comparisons. Like our results, READI’s evaluators also report a nonstatistically significant change in likelihood of gunshot victimization for participants relative to controls. Despite their similarities in findings, direct comparisons between effect sizes of the READI evaluation and this evaluation should be approached with caution, given differences in study design, sampling, definitions of dosage and treatment, and how the interdependencies among cases are addressed (SI Appendix). Nonetheless, that two different studies of different programs in the same city during the same period found convergence in the directionality, magnitude, and significance of one key outcome—impact on gun-related arrests—is compelling on its own.

The present study used an engaged, quasiexperimental approach to evaluate the effect of Chicago CRED on violence-related participant outcomes. Importantly, our analysis considers program dosage by differentiating participants who complete programming—CRED alumni—separately from the participant sample at large and a subset who progressed to the full program phase. While no statistically significant effects were detected on gunshot victimization outcomes, we estimated a substantial reducing effect of CRED on alumni participants for violence related arrests, who were on average 73.4% less likely than comparisons to have an arrest for a violent crime in the months following initiation of the intervention. In Chicago, a ten percent reduction in violent crime is estimated to reduce direct costs by nearly $43 million per year (44); as such, CRED’s ability to reduce gun-related crimes has a direct financial benefit to society as well.

Critically, our finding of modest effects in the All-Participant and Post Outreach participant samples but a strong favorable effect for alumni on probability of violent arrest speaks to the important question of dosage when evaluating CVI impact. Our results and others (39–41) suggest that the amount of exposure to services is likely an important consideration for assessing the success of CVIs. With a full dosage of services, participants may have stronger tools with which to modify their decision-making related to high-risk street behaviors compared to those in earlier phases of programming. Program retention is an urgent area of investigation and support.

While the BART models estimated increases in the probability of not having an arrest for CRED participants, no such effect was found in the victimization risk outcome. While this finding is disheartening, it is perhaps unsurprising for several reasons (see SI Appendix for extended discussion). First, gunshot victimization is a statistically rare and decidedly nonrandom event that concentrates in space and in social networks. Disentangling such interdependencies is complicated by issues of selection bias inherent in how CVI organizations select participants more generally (45)—a factor which also likely plagues all such programs and evaluations, even those attempting randomization. Second, while statistical models often capture the risk of victimization as a binary condition, qualitative research emerging from CRED, along with previous work, suggests that risk is a dynamic combination of individual-, situational-, and neighborhood-level factors (46). In other words, CVI programs that focus on solely changing individual behaviors are not enough to mitigate the risk conferred from prior involvement in group conflict, situational and network contexts, or external factors associated with historically marginalized neighborhoods experiencing high levels of gun violence. As one CRED participant told researchers: “Playing defense (disengaging from group conflicts and gun violence) can only go so far if the rest of the neighborhood is still playing offense.” Even within the CRED area, the program reached only a fraction of individuals involved in high-risk networks: For every CRED participant, there were nearly three other individuals with similar needs who received no such programming, many of which are likely in opposing crews and gangs. Third, and relatedly, reducing gun violence victimization requires extraindividual interventions that focus on other social factors contributing to violence outside of the participant’s direct control, such as community disinvestment and availability of firearms. With all these factors at play, along with the nationwide rise in gun violence and interruption in programming that occurred during the early days of the COVID-19 pandemic, a 73.4% reduction in individual probability of arrest for violent crimes among alumni is a remarkable impact and a promising indicator of Chicago CRED’s ability to reduce violence.

These results highlight the importance of considering programming dosage in the evaluation of interventions for community violence. Though the impact of any given intervention presumably varies with the amount of exposure to services, it is unlikely that this variation follows linear trends for all participants. Future evaluations of CVIs should consider research designs that assess for a “critical mass” or “peak” in programming effects, providing insight into how much exposure to CVI services is necessary for the average participant to achieve the goals of the program. Further, knowing that desired program effects are dosage-dependent should encourage CVI programs to take steps to better understand participant retention and prevent attrition. Finally, future evaluations of program completers should consider participant outcomes outside of the criminal legal system including employment, education, and family relationships to assess program impact on individuals’ economic and social well-being.

Frontline staff and violence prevention practitioners have their own assessments of participant “readiness” that inform their decisions about participant selection. The similarities between our results and the results from the READI Chicago Evaluation emphasize the important role of the street outreach worker in selecting participants who may be “ready” to take full advantage of programming opportunities; this is of special note as the READI Evaluation only described significant changes in arrests for the group brought in through the outreach staff pathway, not through their other two pathways (reentry and selection algorithm). Unfortunately, due to the limitations of our dataset and the binary outcomes of violence involvement used in this analysis, we are unable to answer the question of why some participants are ready to benefit from programming, while others fail to show the same benefits. However, our research group conducted interviews of street outreach staff for the qualitative arm of our evaluation, providing some insight into this question. According to street outreach staff, the assessment of participant readiness is dynamic and involves a variety of signals, including changes in communication styles or frequency, building trust with and initiating contact with outreach staff without prompting, openness to seeing a different way of life, and curiosity about alternatives. The role of the street outreach professional in gauging a participant’s readiness rests within their own lived experience, as one outreach worker stated: “As outreach workers, we went through all these changes too.” Future work, especially studies designed in engagement with and prioritizing the lived experience of staff and participants, should more closely evaluate this concept of readiness and how it relates to selection into and success with programming.

This study grappled with limitations in three primary areas: sample characteristics and selection bias, administrative data quality, and COVID-19. First, the nature of the population recruited into programming by CRED inherently biases the sample towards participants who are high-risk for involvement with gun violence. While the BART models addressed confounding due to baseline risk, CRED participants are difficult to track and are highly likely to drop out of programming (SI Appendix, Fig. S2). Consequently, the sample of those who make into the post outreach and alumni phases may be biased toward relatively lower-risk participants in comparison to those who do not complete programming, specifically when it comes to involvement in street gangs (SI Appendix, Table S1). In addition, alumni are participants who remained in the program long enough to graduate, meaning that they cannot have been fatally shot or dropped out permanently due to risk-related concerns. We provide a more thorough discussion of the issue of selection bias in CVI evaluations in SI Appendix.

Second, this analysis relied on the arrest, gang membership, and victimization administrative records provided by the CPD and therefore is subject to biases in police behavior and reporting (see SI Appendix for further discussion). Particularly of concern is the CPD’s records of gang affiliation, as by our estimates, the CPD only correctly identifies participants’ gang affiliations about 48% of the time, underrepresenting CRED-defined gang affiliations in 43% of gang-affiliated participants. However, because Chicago CRED recruits participants based on gang involvement and CPD-defined gang members have, on average, more arrests, victimizations, and coarrestees in the 3 y prior to their imputed start dates compared to nongang members, we felt it was imperative to balance our comparison group on CPD-defined gang involvement. Additionally, because CPD clearance rates for violent crimes have declined sharply over time (SI Appendix, Table S13 and Fig. S3), we are almost certainly under-counting crimes in our coverage area; however, this decline in clearance rates should impact comparisons and CRED participants equally and therefore should not impact our estimated treatment effects.

Third, our evaluation period includes the unprecedented nationwide increase in gun violence and the COVID-19 pandemic, which has had an especially pronounced impact on Black communities. CVI programming also had to accommodate pandemic-era restrictions (47, 48). While the gains attributable to CRED even under these circumstances are impressive, future research should continue to focus on the impacts of CVIs under nonpandemic conditions.

Firearm violence is a pervasive public health problem in need of immediate solutions for ensuring community safety, and CVIs are promising approaches for reducing gun violence. We presented an evaluation of one such CVI, Chicago CRED, and results show that participation in CRED drives an 73.4% reduction in the probability of arrests for violent crimes for alumni relative to a comparison group of similar peers 24 mo postenrollment. This finding, alongside the modest effects in the larger sample of CRED participants, emphasizes the importance of a complete dosage of wraparound CVI services for reducing violent behavior. This requires reliable systems for tracking and maintaining participants in programming and proper funding for organizations to do so. Determining the “sufficient” amount of service dosage to prevent violence-related outcomes for each individual participant will be critical for service providers to achieve their violence prevention goals, as will understanding how the focus on individual change interacts with larger neighborhood contextual factors that cause gunshot victimization.

Materials and Methods

Data.

All data collection and procedures were approved under Northwestern University IRB #STU00210364 and informed consent procedures were not necessary because only administrative records were used. The data used in the present study were compiled by merging CRED participant information with official arrest and victimization records. Identifiable participant information was provided by CRED in a secure data-sharing process and then matched to arrest and victimization records provided by the CPD before anonymization for analytic use. The anonymized CPD arrest and victimization databases were also used to select the comparison pool. Records encompass arrests, homicides, and gunshot victimizations from January 1, 2011 through December 31, 2021 and include demographic information such as arrestee or victim year of birth, race, and sex, and incident geographic location.

Methods.

Given the nonrandom concentration of gun violence in small social networks in Chicago (18, 30) and CRED’s nonrandom selection of participants, we utilized a quasiexperimental approach that allowed us to model the selection process as accurately as possible while also leveraging known information about networks of gun violence, thereby excluding possible interdependencies both within the treatment group and between the treatment and control groups.

Participant selection.

The treatment sample consisted of adult males (n = 410) enrolled in CRED through their South Side outreach area beginning on January 1, 2016 and selected based on outreach workers’ evaluation of risk for involvement in gun violence combined with a program review of official shooting data. Enrolled participants were deemed “high-risk” by CRED staff due to a previous arrest for violent crime, previous gunshot victimization, past or current involvement in group/gang activity, or proximity to community violence, and ongoing neighborhood or group disputes. Participants were excluded from our analysis of the treatment group if they lacked tracked arrest history, victimization, or gang membership according to available records (n = 67) or if records lacked demographic covariate information (year of birth, sex, and race, n = 7). Individuals were also excluded from the analyses if they enrolled in CRED services less than 6 mo before analyses were conducted (n = 12). This left a final treatment sample of 324. All eligible participants were included in initial analyses, regardless of CRED program phase completion or attrition. To assess the impact of program progress and completion, participants who had completed the Outreach phase and advanced to at least Programming Phase 1 (“Post Outreach”) and participants who had completed all phases of programming (Alumni) were also considered separately in additional analyses.

Comparison pool selection.

The comparison group was randomly selected using Bernoulli random sampling (49) from a pool of individuals arrested by CPD on or after January 1, 2016 who did not receive services through CRED or other known outreach organizations, and who also had no coarrest peers receiving services from the other outreach organizations. Borrowing from insights learned about the cascading of gun violence in social networks (18), we limited any possible immediate spillover by excluding from the pool of eligible comparisons all individuals who were directly connected to a CRED or other outreach participant through a coarrest tie. Further, comparisons were only selected from the geographic areas that overlapped with CRED outreach zones to reduce confounding by neighborhood context. Individuals were also excluded from analyses if they lacked demographic covariate information. Finally, the comparison pool was balanced to the treatment group on race, sex, year of birth, gang affiliation, and gunshot victimizations in the 3 y before service start date. From this pool, 2,500 comparison units were selected. Service start date was randomly imputed for each comparison individual based on the distribution of real start dates from the treatment group, and eligible comparison individuals were dropped from analyses if they experienced a fatal victimization before the imputed start date. See SI Appendix for balancing metrics.

Survival Analysis.

Estimating the effects of program participation.

CRED participants are selected by outreach workers due to their exceedingly high risk of involvement in gang/group activities or known group or neighborhood disputes that lead to gun violence; as such, CRED participation is not randomized and may be confounded with the risk of victimization. To address confounding, the effect of enrollment on victimization outcomes conditional on covariates was estimated using BART (34, 35). BART is a sum-of-trees model that allows for the relaxation of linearity assumptions in the modeling of relationships between outcomes and their interactions (35). Using Markov Chain Monte Carlo modeling and Bayesian backfitting on a set number of trees, BART iteratively fits the residuals and predicts outcomes (35) and models the response surface in victimization or arrest nonparametrically as a function of all available observed covariates, automatically detecting interactions and nonlinearities. This guards against researcher-imposed choices regarding which covariates might be confounders and the functional form of the relationship between arrest or victimization and these confounders. Additionally, BART naturally estimates heterogeneous effects across participants given their covariates. We estimated the model:

$$y_i=\mu \left(x_i\right)+\tau \left({\tilde{x}}_i\right)z_i+{\epsilon }_i,$$

Where y_i is victimization or arrest status, µ(x_i) is the observed covariates function which captures the effects of covariates x on the outcome, $\tau \left({\tilde{x}}_i\right)$ is the treatment effect function, which is distinct from the covariate effects function, z_i is the treatment status indicator, and ${\epsilon }_i$ an error term. Recent developments in causal inference using BART recommend separating the µ and τ functions so that the treatment function can be regularized independently.

We used the BART R package (50) to conduct our survival analysis. Covariates included number of arrests and victimizations in the 3 y prior to the start of services or imputed start date; binary code for gang involvement and coarrestees; number of co-arrestees and number of co-arrestees’ victimizations in the 3 y prior to start; the proportion of first- and second-order peers receiving street outreach treatment; and year of birth, sex, and race. We set the number of trees to 50 and the number of cut points to 100 and examined the probability of survival from arrest for violent crime, gunshot victimization or the indexed combined outcome in three separate models.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

Due to the highly sensitive nature of the data used in this analysis (arrest and victimization records and program data from a violence prevention organization with potentially identifiable participant information), data cannot be shared. The data used in this report represent highly sensitive criminal legal-related information on participants from a vulnerable group of individuals at risk of gun violence. As such, it would be highly unethical to share these protected data without the correct precautions and secure data sharing agreement in place. However, in the repository linked below, we have included code for generating a demonstration dataset with the same variables we used in this paper. The repository with code for generating a demo dataset and running the BART survival analysis can be found here on GitHub: https://github.com/cornersresearch/Ross-et-al_PNAS2023/tree/main (51).