Technologies of Crime Prediction: The Reception of Algorithms in Policing and Criminal Courts

Abstract

The number of predictive technologies used in the U.S. criminal justice system is on the rise. Yet there is little research to date on the reception of algorithms in criminal justice institutions. We draw on ethnographic fieldwork conducted within a large urban police department and a midsized criminal court to assess the impact of predictive technologies at different stages of the criminal justice process. We first show that similar arguments are mobilized to justify the adoption of predictive algorithms in law enforcement and criminal courts. In both cases, algorithms are described as more objective and efficient than humans’ discretionary judgment. We then study how predictive algorithms are used, documenting similar processes of professional resistance among law enforcement and legal professionals. In both cases, resentment toward predictive algorithms is fueled by fears of deskilling and heightened managerial surveillance. Two practical strategies of resistance emerge: foot-dragging and data obfuscation. We conclude by discussing how predictive technologies do not replace, but rather displace discretion to less visible—and therefore less accountable—areas within organizations, a shift which has important implications for inequality and the administration of justice in the age of big data.

In recent years, algorithms and artificial intelligence have attracted a great deal of scholarly and journalistic attention. Of particular interest is the development of predictive technologies designed to estimate the likelihood of a future event, such as the probability that an individual will default on a loan, the likelihood that a consumer will buy a specific product online, or the odds that a job candidate will have a long tenure in an organization. Predictive algorithms capture the imagination of scholars and journalists alike, in part because they raise the question of automated judgment: the replacement – or at least the augmentation – of human discretion by mechanical procedures. Nowhere are these questions more salient than in the context of criminal justice. Over recent decades, the U.S. criminal justice system has witnessed a proliferation of algorithmic technologies. Police departments now increasingly rely on predictive software programs to target potential victims and offenders and predict when and where future crimes are likely to occur (Brayne 2017; Ferguson 2017). Likewise, criminal courts use multiple predictive instruments, called “risk-assessment tools,” to assess the risk of recidivism or failure to appear in court among defendants (Hannah-Moffat 2018; Harcourt 2006; Monahan and Skeem 2016).

Predictive technologies, in turn, raise many questions about fairness and inequality in criminal justice. On the positive side, advocates emphasize the benefits of using “smart statistics” to reduce crime and improve a dysfunctional criminal justice system characterized by racial discrimination and mass incarceration (Brantingham, Valasik, and Mohler 2018; Milgram 2012). On the negative side, critics argue that algorithms tend to embed bias and reinforce social and racial inequalities, rather than reducing them (Benjamin 2019; Eubanks 2018; O’Neil 2016). They note that predictive algorithms draw on variables or proxies that are unfair and may be unconstitutional (Ferguson 2017; Starr 2014). Many point out that predictive algorithms may lead individuals to be surveilled and detained based on crimes they have not committed yet, frequently comparing these technologies to the science-fiction story Minority Report by Philip K. Dick and its movie adaptation, which evoke a dystopian future.

To date, studies of criminal justice algorithms share three main characteristics. First, existing work tends to focus on the construction of algorithms, highlighting the proprietary aspect of most of these tools (which are often built by private companies) and criticizing their opacity (Angwin et al. 2016; Pasquale 2015; Wexler 2017). Second, they tend to treat the criminal justice system as a monolith, lumping together the cases of law enforcement, adjudication, sentencing, and community supervision (O’Neil 2016; Scannell 2016). Third, and most importantly, most studies fail to analyze contexts of reception, implicitly assuming – usually without empirical data – that police officers, judges, and prosecutors rely uncritically on what algorithms direct them to do in their daily routines (Harcourt 2006; Hvistendahl 2016; Mohler et al. 2015; Uchida and Swatt 2013).

In this article, we adopt a different perspective. Building on a growing body of literature that analyzes the impact of big data in criminal justice (Hannah-Moffat, Maurutto, and Turnbull 2009; Lageson 2017; Lum, Koper, and Willis 2017; Sanders, Weston, and Schott 2015; Stevenson and Doleac 2018), as well as existing ethnographic work on the uses of algorithms (Brayne 2017; Christin 2017; Levy 2015; Rosenblat and Stark 2016; Shestakovsky 2017), we focus on the reception of predictive algorithms in different segments of the criminal justice system. Drawing on two in-depth ethnographic studies – one conducted in a police department and the other in a criminal court – we examine two questions. First, to what extent does the adoption of predictive algorithms affect work practices in policing and criminal courts? Second, how do practitioners respond to algorithmic technologies (i.e., do they embrace or contest them)?

Based on this ethnographic material, this article provides several key findings. First, we document a widespread – albeit uneven – use of big data technologies on the ground. In policing, big data are used for both person-based and place-based predictive identification, in addition to risk management, crime analysis, and investigations. In criminal courts, multiple predictive instruments, complemented by digital case management systems, are employed to quantify the risk of the defendants. Second, similar arguments are used in policing and courts to justify the use of predictive technologies. In both cases, algorithms are presented as more rational and objective than “gut feelings” or discretionary judgments. Third, we find similar strategies of resistance, fueled by fears of experiential devaluation and increased managerial surveillance, among law enforcement and legal professionals—most importantly, foot-dragging and data obfuscation. Despite these resemblances, we document important differences between our two cases. In particular, law enforcement officers were under more direct pressure to use the algorithms, whereas the legal professionals under consideration were able to keep their distance and ignore predictive technologies without consequences, a finding we relate to the distinct hierarchical structures and levels of managerial oversight of the police department and criminal court we compared.

We conclude by discussing the implications of these findings for research on technology and inequality in criminal justice. Whereas the current wave of critical scholarship on algorithmic bias often leans upon technological deterministic narratives in order to make social justice claims, here we focus on the social and institutional contexts within which such predictive systems are deployed and negotiated. In the process, we show that these tools acquire political nuance and meaning through practice, which can lead to unanticipated or undesirable outcomes: forms of workplace surveillance and the displacement of discretion to less accountable places. We argue that this sheds new light on the transformations of police and judicial discretion – with important consequences for social and racial inequality – in the age of big data.

DECISION-MAKING ACROSS A VARIETY OF DOMAINS

As a growing number of daily activities now take place online, an unprecedented amount of digital information is being collected, stored, and analyzed, making it possible to aggregate data across previously separate institutional settings. Harnessing this rapidly expanding corpus of digitized information, algorithms – broadly defined here as “[a] formally specified sequence(s) of logical operations that provides step-by-step instructions for computers to act on data and thus automate decisions” (Barocas et al. 2014) – are being used to guide decision-making across institutional domains as varied as education, journalism, credit, and criminal justice (Brayne 2017; Christin 2018; Fourcade and Healy 2017; O’Neil 2016; Pasquale 2015). Advocates for algorithmic technologies argue that by relying on “unbiased” assessments, algorithms may help deploy resources more efficiently and objectively. Yet recent research casts doubt on the idea that algorithmic technologies are always more efficient, objective, and accountable than human judgment. Across fields, scholars have showed that algorithms can be biased in ways that mirror or even amplify the discriminatory features of the existing social system (Benjamin 2019; boyd and Crawford 2012; Eubanks 2018; Noble 2018; O’Neil 2016). Such questions play out in particularly important ways in the case of predictive algorithms in the criminal justice sector.

Algorithmic Technologies in Policing and Criminal Courts

This section provides an overview of the main predictive algorithms used in policing and criminal courts. The term “predictive policing” refers to the use of analytical techniques to make statistical predictions about potential criminal activity (van Brakel and De Hert 2011). The basic underlying assumption of predictive policing is that crime is not randomly distributed across people or places. Rather, predictive policing draws from canonical theories in criminology that analyze crime as a function of environmental conditions (Brantingham and Brantingham 1981; Ratcliffe 2008; Sampson, Raudenbush, and Earls 1997), situational decision-making (Keizer, Lindenberg, and Steg 2008; Matsueda, Kreager, and Huizinga 2006), chronic offenders (Uchida and Swatt 2013) and social networks (Papachristos, Wildeman, and Roberto 2015).

Predictive technologies affect two main areas of police work. First, patrol officers rely on “risk-based deployment” to target police resources on the “hottest” people and places (Bennett Moses and Chan 2018). Note, however, that algorithmic forecasts alone do not meet the threshold of reasonable suspicion or probable cause (Ferguson 2017). Thus, even though data drive deployment, what officers do once they reach the person or place the algorithm identified as higher risk remains within officers’ discretion. Second, in investigations, detectives now conduct automated data grazing to flag potential crime series that span jurisdictional boundaries and are, therefore, difficult for any one person to identify.

In criminal courts, there has also been an exponential deployment of predictive technologies (Harcourt 2006). Starting in the 1980s, criminal courts turned to “evidence-based” approaches to risk prevention (Mehozay and Fisher 2018). This entailed using risk-assessment tools, that is, actuarial techniques that forecast criminal justice outcomes – most commonly an offender’s risk of recidivism or failure to appear in court when on bail. The first wave of risk-assessment tools relied on “static” or predetermined risk factors, such as history of substance abuse and age of first offense. Over time, risk analysis switched to “dynamic factors” such as age or employment status (Hannah-Moffat, Maurutto, and Turnbull 2009; Kehl, Guo, and Kessler 2017). Recent risk-assessment tools have been described as specifically “algorithmic,” in the sense that they rely on omnivorous data collection and machine-learning models to identify relevant patterns, becoming opaque and “black boxed” in the process (Berk 2012; Hannah-Moffat 2018; Kehl, Guo, and Kessler 2017; Mehozay and Fisher 2018).

Risk-assessment instruments are explicitly designed to “structure” decision-making and curtail judicial discretion by providing a clear set of guidelines, scores, and recommendations to legal professionals throughout the adjudication and incarceration process. Pre-trial risk assessment instruments evaluate the probability that a defendant is a threat to public safety or will fail to appear in court. During adjudication, they can be used for sentencing decisions. Post-adjudication, they are used to predict recidivism for probation and parole decisions. Risk scores also serve as correctional instruments to determine the security classification of incarcerated individuals.

Both predictive policing and risk-assessment algorithms have faced growing criticism because of their potential to reinforce preexisting biases and inequalities. Journalists and scholars have documented the “ratchet effect” of predictive technologies and the “disparate impact” that they tend to have on protected groups (Angwin et al. 2016; Barocas and Selbst 2016; Harcourt 2006). As Sandra Mayson writes, “In a racially stratified world, any method of prediction will project the inequalities of the past into the future. This is as true of the subjective prediction that has long pervaded criminal justice as of the algorithmic tools now replacing it” (Mayson 2019:2218).

A Longer History: Quantification in Criminal Justice

Although it has received increased attention in recent years, data-driven decision-making is far from new in the criminal justice context. Indeed, sorting and scoring technologies have existed in policing and courts since the early years of the twentieth century. Yet the current tools differ from these historical precedents in several important ways.

Over the past century, criminal courts have experienced many quantification initiatives in their attempts to increase efficiency, reduce disparities, and improve crime prediction. In 1928, Ernest Burgess of the University of Chicago designed an actuarial model that predicted the probability of parolees’ reoffending (Harcourt 2006). In the 1980s, the Sentencing Reform Act was passed, creating mandatory sentencing guidelines that constrained the range of sentences that judges could adopt (Espeland and Vannebo 2007; Lynch 2017). Over the past decades, the criminal justice system experienced a shift towards “actuarial justice,” drawing on statistical techniques derived from insurance and risk management to assess criminal risk (Ericson and Haggerty 1997; Feeley and Simon 1992; Garland 2002; Lyon 2003; Mehozay and Fisher 2018; Rothschild-Elyassi, Koehler, and Simon 2019).

In policing, the quantitative turn took a different path. Until the 1970s, policing was largely reactive, involving random patrols, response to 911 calls, and reactive investigations. However, practitioners and researchers observed that these strategies had little effect on crime rates. Over time, policing shifted to proactive and data-driven practices (Braga and Weisburd 2010; Sherman, Gartin, and Buerger 1989). In 1994, CompStat – a managerial model that involved analyzing crime and enforcement patterns and holding accountability meetings with officers of various ranks – was established in New York City and other places (Weisburd et al. 2003). The attacks on 9/11 catalyzed another shift towards “intelligence-led policing,” emphasizing prediction and preemption (Ratcliffe 2008; Sanders, Weston, and Schott 2015; van Brakel and De Hert 2011; Waxman 2009).

Hence, data-driven and predictive technologies have long been promoted as technical solutions to complex social issues relating to efficiency and accountability in policing and criminal justice. But today’s technologies differ from their predecessors in several important ways. First, the scale, range, and granularity of available data is greater than ever before. Software programs now rely on high-frequency observations, records are increasingly detailed, and data collection is largely automated – a process primarily driven by the capabilities of digital technologies in terms of automation and information production. Data itself has become a form of capital that is bought, sold, and traded between public and private agencies, including third-party data brokers (Lageson 2017).

Second, computational techniques have also become more refined, leading to the emergence of predictive tools that are heralded as more accurate than their earlier counterparts (Mohler et al. 2015). The increasing complexity of modelling techniques has led to the emergence of new forms of algorithmic “fetishism” (Monahan 2018) and Big Data “mythologies” (boyd and Crawford 2012), in particular surrounding predictive and machine-learning algorithms (Berk 2012; Hannah-Moffat 2018). As a result, even though the economic and institutional infrastructure for data collection and analysis has grown exponentially more complex over the past decades, it has paradoxically become less visible than ever before: machines, not humans, appear to be doing all the work. As Fourcade and Healy (2017:4) explain, “If the recorded individual has come into full view, the recording individual has faded into the background, arguably to the point of extinction.”

Analyzing the Reception of Algorithms

Given the rationalizing impetus that guides the adoption of algorithmic technologies in the criminal justice context, these profound changes lead us to raise the question of the reception of predictive algorithms in the context of law enforcement and criminal courts. Although there is strong theoretical work in surveillance studies that focuses on the possibilities, good and bad, of new forms of algorithmic decision-making, there is a dearth of empirical work on the social contexts of their reception in policing and courts.

Here we draw on the few existing studies that examine the reception of either quantification instruments or information and communication technologies (iCTs) in police and criminal courts to document several intriguing findings. First, scholars report that digital tools are often “translated” in order to fit local priorities and concerns, both in police departments and criminal courts, which leads practitioners to ignore the tools that they find “inefficient” (Brayne, Levy, and Newell, 2018; Lum, Koper, and Willis 2017; Sanders, Weston, and Schott 2015; Stevenson and Doleac 2018). This means that many of the tools that are praised by hierarchical superiors as “revolutionary” end up not being used by rank-and-file officers or front-line legal professionals (Christin 2017). Second, existing studies suggest that the implementation of quantitative instruments leads to “reactive” processes among police officers and legal professionals, who adjust their daily practices as a result of the new metrics and standards through the emergence of what Lynch calls “narratives of the numbers” (Brayne 2017; Espeland and Vannebo 2007, Hannah-Moffat, Maurutto, and Turnbull 2009; Lynch 2017; Sanders, Weston, and Schott 2015). Drawing on these studies, this article examines how predictive algorithms are used in the criminal justice system. Because predictive technologies have become ubiquitous throughout the criminal justice process, we compare the uses of algorithms in decision-making at two key institutional passage points.

DATA AND METHODS

To date, most research on criminal justice remains siloed: some focus on policing, others on courts, and others on incarceration. Few consider individuals’ processing through the entire system or analyze multiple stages of the criminal justice process. Yet predictive technologies now permeate the entirety of the criminal justice process. At the entry point into the criminal justice system – police contact – algorithms are used to guide how law enforcement agencies focus their resources. After police contact, if an individual is charged and booked, they may encounter algorithms again in the pretrial assessment stage, during plea bargaining, and at sentencing. During and after incarceration, algorithms may be used to determine security levels, eligibility for parole, and conditions of community supervision.

Hence, this project examines the uses of predictive algorithms in two key institutions of the criminal justice process: police departments and criminal courts. Before introducing the cases at the center of this analysis, it is important to note that the two ethnographic projects on which this analysis is based were conducted independently. In both cases, we relied on “abductive analysis” (Tavory and Timmermans 2014) to organize the respective ethnographic material: we started not with a “blank slate” but with preliminary questions and a toolkit of theories, which we refined and redefined in an iterative manner as we gathered more data. Over the course of our fieldwork, we noticed the parallels between our respective cases, which led us to conjointly elaborate a broader argument about the uses of predictive technologies in criminal justice. We then reanalyzed the data for the purpose of the present analysis (see Lara-Millán and Gonzales 2017 for a similar analytic strategy).

Policing

The first case study is the Los Angeles Police Department (LAPD), a site that was selected not because it is representative, but rather because this agency is a leader in its field in the use of data analytics. Over the course of two and a half years, the first author conducted interviews and observations with 75 individuals, complemented by 31 follow up interviews. Intensive fieldwork was conducted between 2013 and 2015, and follow-up field site visits were conducted between 2016 and 2018. Interviewees included sworn officers of various ranks (captains, sergeants and officers) and civilian employees working in patrol, investigation, and crime analysis across four area divisions of the LAPD, as well as individuals in specialized divisions. To better understand how data were used in the field, we conducted observations on ride-alongs and shadowed data analysts. Additional interviews and observations were conducted within the L.A. County Sheriff’s Department (LASD) and the Joint Regional Intelligence Center (JRIC). Last, we attended surveillance industry conferences and interviewed individuals working at technology firms that collaborate with the LAPD.

During our fieldwork, the LAPD used two main types of predictive policing models: person-based and place-based. In the person-based models, individuals on the street were assigned a points value and given a numerical rank according to that value. Individuals were assigned five points for a violent criminal history, five points for known gang affiliation, five points for parole or probation, and one point for every police contact. The Crime Intelligence Detail (CID) then generated “Chronic Offender Bulletins”—which include a person’s name, date of birth, CII number (rap sheet number), driver’s license number, physical descriptors, physical oddities (such as tattoos or scars), arrest history, CalGang designation, parole and probation status, warrants, vehicles, recent stops, and police contacts—for individuals with the highest numbers of points. Officers were directed to seek out Chronic Offenders while on patrol.

In 2012, the LAPD also began using the place-based predictive software program PredPol to identify areas were crime was most likely to occur in the future. PredPol uses an algorithm that relies on three types of inputs – past time, place, and type of crime – to identify 500 by 500 square foot areas in which crime is most likely to occur in the immediate future. Officers received printouts at the beginning of their shift that showed predictive boxes overlying small areas of division maps; they were encouraged to spend at least 10 percent of their time in predictive boxes, a strategy referred to as “risk-based deployment.” Officers recorded their self-reported minutes in predictive boxes on their in-car laptops.¹

Criminal Courts

The ethnographic project for the criminal court segment of this research began in 2015, when the second author started conducting interviews about the uses of analytics and big data technologies in criminal courts, with a specific focus on risk-assessment tools. In May–June 2016, intensive ethnographic fieldwork was conducted in Marcy County (the name of the county, as well as individuals, have been changed), an urban jurisdiction located in a southern state that has both district and county criminal courts.

As in the policing study, Marcy County was chosen because it was at the forefront of a new analytics initiative. We conducted more than 70 hours of in-depth ethnographic observations, following judges, prosecutors, clerks, court administrators, and data analysts in their daily activities, including arraignments and hearings. In addition to the observations, we conducted open-ended interviews with 22 people, including probation officers, judges, defense attorneys, clerks, court administrators, and technology developers. These interviews were complemented by seven follow-up interviews. In addition to the data collected in Marcy County, we conducted observations in two large criminal courts in metropolitan areas on the East and West coasts, complemented by 12 interviews with prosecutors, judges, and data analysts in the two jurisdictions. We also compiled technical literature on risk-assessment instruments and attended industry conferences on the topic, during which we conducted three interviews with technology developers involved in constructing risk-assessment tools.

During our fieldwork, Marcy County used three main risk-assessment tools: one for pretrial decisions, one for domestic violence (used in pretrial and sentencing), and one for probation. We focused on the pretrial tool. When a new defendant entered the system, pretrial officers completed their risk-assessment profile. They would then print out the summary pages produced by the risk-assessment instrument, which were added to the defendant’s paper file and brought to hearings. These print-outs provided risk assessment scores along different axes. The “assessment” page featured a table on the front page summarizing the “risk factors” for several categories (education and employment, family and social support, criminal history, etc.). Risk thresholds were set to separate individuals into “low,” “medium,” or “high” categories.

Hence, both the police department and the criminal court analyzed in this project relied on several predictive algorithms to assess risk. In spite of these similarities between our two cases, however, we note that the comparability between the two sites studied here is not perfect. Most importantly, the police department under consideration is larger than the criminal court, which, in turn, is likely to influence the findings, as we examine in the discussion. While we acknowledge these limitations, we also argue that the benefits associated with collecting data on the reception of predictive algorithms at these two key points of the criminal justice process outweigh the methodological limitations intrinsic to the comparison of ethnographic field sites.

RESULTS

Algorithms on the Ground

Based on these two ethnographic studies, we turn to the reception of predictive technologies in policing and criminal courts. Specifically, we compare the motivations and arguments deployed to justify their implementation and the ways in which they are used on the ground. Both in the policing and criminal justice contexts, we find that algorithms are presented as objective and efficient instruments. Yet when we analyze the actual ways in which predictive technologies are used and interpreted, a different picture emerges. We find strong processes of resistance fueled by fear of professional devaluation and threats of performance tracking in the two domains.

Justifying Predictive Technologies: Between Accountability and Cost-Cutting

How do law enforcement officers and legal professionals justify the implementation of predictive technologies? In both of our case studies, several factors came into play, including pressures for accountability and new budgetary constraints.

To understand the motivation behind the adoption of predictive technologies in policing, it is important to consider the LAPD in historical context. In recent decades, the department faced criticism over civil rights violations, corruption, and training deficiencies. In 2001, the Department of Justice entered into a consent decree with the LAPD that mandated, among other things, the creation of a new data-driven employee risk management system, which paved the way for more data-driven decision-making within the organization in general. As one officer explained when asked why predictive instruments were used in his division:

The code of federal regulations. They say you shouldn’t create a—you can’t target individuals especially for any race or I forget how you say that. But then we didn’t want to make it look like we’re creating a gang depository of just gang affiliates or gang associates … We were just trying to cover and make sure everything is right on the front end.

A second factor was state legislative decisions. In response to Brown v. Plata (a 2011 U.S. Supreme Court Decision holding that the overcrowding of California prisons and lack of access to adequate healthcare violated prisoners’ constitutional rights), the California Legislature passed Assembly Bill 109. The bill aimed at reducing the state prison population and shifting the responsibility of supervising released non-serious, non-violent, non-sexual offenders from state to local law enforcement. As a result, local law enforcement agencies became responsible for hundreds more individuals per month than they could afford to conduct compliance checks on. Thus, they decided where to allocate resources by sorting the post-release community supervision population according to risk. Last, predictive technologies were often positioned as a means to increase objectivity. As one captain explained, relying on data rather than human interpretation helped avoid misallocating resources: “There’s an emotional element to it … . Because, there’s, you know, I mean it’s a human doing it. And they cannot sort out what’s noise.”

In criminal courts, the impetus for using predictive algorithms was comparable to the one in policing: risk-assessment tools were usually presented as more “objective,” accountable, and efficient than subjective judgements. In Marcy County, many legal professionals and administrators lamented their voters’ “lack of trust.” As one defense attorney put it, “In criminal justice there is a huge chasm between what people believe is happening and what is actually happening.” Many thought that data-driven technologies could help address what they experienced as a disconnect between the perceptions of local publics and their internal dynamics. For instance, the director of the technology team cited accountability and transparency to justify the use of algorithms in Marcy County: “This is a bright light that we’re putting on ourselves to see if the kitchen is dirty before anyone else sees it.”

Legal professionals and administrators also had high hopes that predictive algorithms would help them cut costs in a context of budgetary constraints. In particular, many were concerned about the local jail population, which kept growing, leading to multiple safety and administrative issues. District attorney and judges kept trying to reduce the number of jail detainees in order to match the existing number of cells. Many focused their efforts on pretrial detention, which they felt could be better addressed by relying more extensively on predictive risk-assessment tools to release defendants presenting a low risk of “failure to appear in court” (FTA) and a low risk to public safety.

Hence, a dual justification emerged for using predictive algorithms in policing and criminal courts: first, an “objectivity” argument, which presented algorithms as a means to increase accountability and mitigate bias; second, an “efficiency” argument, which described predictive algorithms as a costcutting device at a time of funding and budgetary constraints.

Institutional Actors Object to Their Own Surveillance

We now turn to the reception of predictive technologies among law enforcement and legal professionals. Far from obedient adoption, the analysis documents processes of professional resistance fueled by similar fears in the two contexts, as respondents raised issues with what they perceived to be threats to their professional autonomy.

In the policing case, the proliferation of data collection sensors associated with algorithmic policing resulted in the police themselves feeling that they were under increased surveillance. For instance, during a ride-along, we observed an officer type on the in-car computer that he was “Code 6 at [address].” Code 6 is a code indicating that the unit has arrived at a location and the officer is conducting a field investigation. We were surprised that there was not an automated mechanism for tracking patrol cars and asked the officer why he manually placed himself at the location. He explained that there was, in fact, an automated way of knowing where cars were – every police unit was equipped with an automatic vehicle locator (AVL). However, police officers usually did not turn the AVLs on because of resistance from the police union. Even though AVLs were originally created in order to make deployment faster and more efficient, officers expressed concern over the possibility for “function creep” – whenever data originally collected for one purpose ends up being used for another purpose, in this case managerial surveillance. In another example, a captain explicitly stated that he wanted to use GPS technology in conjunction with place-based predictive algorithms in order to better track his officers:

The thing I’m really interested in with the GPS is the tracking of how much time they’re spending on missions … . We give them a mission in the predictive boxes [and] instead of getting self-reported [data], via the computer—and I don’t think they’re lying to us ‘cause they’re all paranoid about getting in trouble—but I’d like to have GPS tracks.

The same captain complained, “The problem with the police officers is they think we’re gonna nickel and dime them on where you at? Oh, you guys are at Starbucks too long, or you know?” And, indeed, the officers often expressed this specific criticism of how their data could be used.

In criminal courts, judges also raised concerns about the potential of risk-assessment tools for function creep and managerial surveillance. In particular, they worried that predictive algorithms could be used by administrators and defense attorneys to directly compare their sentencing decisions, incarceration rates, and productivity. They were not wrong, in the sense that risk-assessment tools were used in conjunction with digital case management systems, which, in turn, were mobilized both for the prediction of risk and for tracking the productivity and output of legal professionals. For instance, in Marcy County, the implementation of a new digital case management system went hand in hand with the gathering of detailed analytics comparing the output of different procedures and the productivity of different judges. As a judge explained:

We often compare the numbers for our courts. This is what I have [he opens a Word document with a table on it; it’s broken down by court]. I have the number of cases on the docket, how many pleas, how many people in jail, and I have the percentage change compared to last month. We get this every two weeks. I try not to get caught up in that. Some of that stuff you can’t control – I have this case, this guy has been in jail for two years, it brings my numbers up, but it’s also the biggest case of the county… . It’s helpful to get feedback on my management as a judge, but it’s not the only thing that matters!

Most judges and prosecutors vehemently resisted the data collection initiative and argued in favor of the incommensurability of their decisions. As an older judge put it:

There are things you can’t quantify. […] You can take the same case, with the same defendant, the same criminal record, the same judge, the same attorney, the same prosecutor, and get two different decisions in different courts. Or you can take the same case, with the same defendant, the same judge, etc., at a two week interval and have completely different decision. Is that justice? I think it is.

Hence, both in the police and in the criminal court cases, the rise of predictive and data-driven technologies – together with the function creep that followed – came with an uncomfortable inversion of the usual surveillance relationship: police officers and legal professionals saw their own performance being mechanically assessed and quantified. Although most of the surveillance literature focuses on the unidirectional extraction of information for the purposes of control and risk management, we found that the police and legal professionals were not exempt: as surveillance tools multiply, the surveillors are being increasingly surveilled.

The Devaluation of Experiential Knowledge

A second complaint we encountered is that predictive technologies devalue experiential knowledge in favor of algorithms, which respondents described as deterministic, inflexible, opaque, and often “dumb.”

In the law enforcement case, patrol officers regularly expressed frustration that “pointy heads” working on computers were out of touch with how things actually played out on the streets. Predictive policing, some officers complained, was subverting their on-the-job knowledge and placing greater value on the theoretical skills of civilian crime analysts. Such frustration between front-line workers and officer managers is not new (e.g., Burawoy 1979). Yet it has become more acute in the age of algorithms: whereas supervisors were previously only able to tell officers where to go; now they can track where officers actually go. As one captain who was trying to incorporate predictive analytics into his daily operations lamented, “there’s so much resistance.” He explained how a typical exchange would play out:

They’re like, “You know what, I know where the crime’s occurring” … And I show them the [predictive policing] forecast and they say, “Okay, so [at intersection], I know there are crimes, I could have told you that. I’ve been working here ten years! There’s always crime there.” I go, “Okay, you’re working here ten years on that car, why is there still crime there if you’re so knowledgeable?”

Patrol officers frequently asserted that they did not need an algorithm to tell them where crime occurs. Such resistance was not limited to patrol: it occurred in investigations as well. For instance, after a detective found a suspect using Palantir – one of the platforms individuals in the LAPD use for merging , structuring, and analyzing data – he explained that although Palantir may have helped him find the suspect faster, that he “probably would have still found [the suspect] eventually.”

Similar fears of devaluation emerged in criminal courts. Managers and administrators were often enthusiastic about risk-assessment tools. Yet judges and prosecutors were more skeptical, suggesting the tools failed to tell them anything they did not already know. As a judge told us, “In some cases, the algorithm is just silly. For instance, here [he pulls up a file and looks at a printed sheet of paper] I see 29 failures to appear … I mean, a “high risk” score is just silly in this case.” Another one explained: “When I look at [risk-assessment tools], the question I ask is, “does it pass the sniff test?” Seems like it could be useful, but I’m not sure.” A former prosecutor concurred: “I didn’t put much stock in [risk-assessment instruments], I’d prefer to look at actual behaviors. I just didn’t know how these tests were administered, in which circumstances, with what kind of data … With these tools the output is only as good as the input. And the input is controversial.”

One interpretation of these critical attitudes towards predictive technologies is that police officers, judges, prosecutors, and defense attorneys have worked hard to become professionals. For them, the technocratic oversight associated with big data analytics represents a threat of deskilling. Thus, their criticisms may stem from their concern that their experiential knowledge and professional discretion may become devalued, turning them into line workers, instead of autonomous actors with specific expertise. Previous work suggests that part of why it might be difficult for officers and legal professionals to accept data-driven decision-making – and “commensuration” processes more broadly – is that analytics threaten an identity that is held sacred. In our cases, autonomy and discretion are important components of police officers’ and legal professionals’ occupational identities.

Practical Strategies of Resistance

Such feelings of defiance toward predictive technologies were frequently accompanied by practical strategies of resistance, both in the policing and in the criminal courts under consideration.

The first widespread strategy was foot-dragging, or simply ignoring the tools in one’s daily work (Espeland and Sauder 2016; Scott 1998). In the policing case, for instance, the extent to which officers spent time in predictive boxes varied widely depending on the division, the officer, the shift, and the ride along. On ride alongs when there was considerable uncommitted time, some officers would drive to those boxes, manually check in to the location on their in-car laptop, look around, and then manually check out of the box and leave; others would not. On ride alongs when officers were responding to calls for service, they spent very little time in the predictive boxes (unless they incidentally found themselves in a box while responding to a call for service). Overall, officers retained a significant amount of discretionary power in deciding whether and when to rely on predictive policing technologies.

Similarly, in criminal courts, the observations of hearings revealed that most judges and prosecutors did not typically rely on the risk scores at their disposal: scores were added to the defendants’ files by pretrial and probation officers, but judges, prosecutors, or attorneys almost never mentioned them during the hearings. When we were able to read through the defendants’ files, we realized that the risk scores sheets were often placed towards the end of the file; they were generally not annotated, in contrast to the first 50 pages of the file. When we asked a judge whether he considered risk scores when making decisions, he said that he usually did not, paused, and added: “Anyways there is always a lot of delays in getting the reports from Pretrial Services… . Usually I only get them 15 minutes before the beginning of the hearings, and I don’t have time to read them.” Follow-up interviews revealed that this kind of delay was frequent and well-known, but no one had mentioned it until then, even though it meant that most judges did not have access to risk scores during hearings.

Law enforcement and legal professionals also relied on more data-centered forms of resistance, which often involved some form of obfuscation – making things obscure either by blocking data collection or by deploying more data (Brunton and Nissenbaum 2011). For instance, there was a “rash of antennae malfunction” in South Bureau of the LAPD. A subsequent internal investigation discovered that officers deliberately removed the antennae from cars in order to tamper with the voice recording equipment and prevent management from hearing what they were saying in the field. Conversely, some individuals were producing their own data. For example, one officer carried his own audio recorder everywhere, as he did not trust the department’s own recording equipment. He explained that his own audio recorder was always turned on, so that in case anyone accused him of wrongdoing, he would have his own evidence.

Along similar lines, an ongoing struggle took place in criminal courts regarding the scheduling and sharing of data between different departments. Specific departments – most notably Pretrial Services and the prosecutorial offices – vehemently refused to share their data with the central data analytics team. As a member of the “Justice and Public Safety” department in Marcy County commented, “Here it’s a big giant mess [laughs]. People [from the Pretrial Department] don’t want to share their data because they’re scared that it will come back at them. They’re careful. They keep saying, ‘we can’t find the data,’ well that’s a bad sign.”

It is worth noting that these two forms of practical resistance – foot-dragging and data obfuscation – appear to be less related to predictive technologies per se than to technologies used as accountability mechanisms. However, resistance was strongest in instances of function creep, whenever a technology initially designed for crime control started to be used to increase managerial control or measure employee performance.

Managerial Control and the Question of Implementation

So far, we emphasized similarities between policing and criminal courts. To conclude this empirical section, we turn to the main difference between the two sites, relating to their hierarchical structures and systems of managerial control. Overall, we found that the implementation of predictive algorithms was more strictly implemented in policing compared to the criminal courts we analyzed. We found that the police department we studied relied on a more clearly enforced hierarchical structure, whereas the criminal courts could better be compared to a series of islands functioning independently of each other. Of course, this is not to minimize the role of discretion in police departments, nor to say that courts are not hierarchical (Kohler-Hausman 2018).

Yet many legal professionals manage to maintain a large amount of autonomy and discretion in deciding how to organize their work. For instance, in Marcy County, judges were elected. As a result, many of them felt that their priorities were towards their electors, not the central administration of the court or the commissioners. Similarly, defense attorneys often invoked their clients as their primary responsibility, an argument they mobilized to refuse to comply with directives or technological initiatives that they felt would damage their clients’ welfare in court. As a defense attorney told us, “It’s like Renaissance Italy, where you had all these different cities having autonomy, competing with each other, getting into alliances and wars. It’s the same here.” Thus, there was very little the technology office could do to convince judges, prosecutors, and defense attorneys to use risk-assessment tools or analytic systems more regularly than they did.

In contrast, the LAPD had a more hierarchical structure and tighter procedures for managerial control. The officers using predictive algorithms were more beholden to their supervisors, largely because they were situated in lower positions in the organizational hierarchy than analogous professionals using predictive tools in courts. Whereas judges could exercise discretion about whether and when to use risk scores, officers were forced to use – or at least to offer the appearance of using, as we have seen – algorithms for risk-based deployment. At first blush, this might seem counter to existing research that suggests police discretion increases towards the bottom of the hierarchy (Chan 2001; Ericson and Haggerty 1997). However, because of the “function creep” between crime prevention and managerial surveillance, the proliferation of predictive technologies and sensors served to increase hierarchical oversight in policing by creating fine-grained data that supervisors could use to triangulate the actual behaviors of their officers with discursive accounts of their whereabouts. It is precisely this shift of accountability mechanisms into performance metrics that officers were frustrated about.

These different organizational structures help us understand why, overall, predictive technologies were used on a more regular basis at the LAPD than in Marcy County, even if some police officers also developed strategies of resistance to avoid using them. Yet this may not be the only causal factor at stake. Indeed, an important limitation of our research design is our focus on single organizational sites, which makes it hard to generalize to “policing” and “criminal courts” more broadly. It is worth noting that there are many police departments (especially in smaller, less urban areas) that do not rely on predictive technologies in their daily operations. Conversely, many criminal courts are likely to rely more on predictive technologies than in the one that we observed.

DISCUSSION: DISCRETION AND INEQUALITY IN THE AGE OF BIG DATA

The previous section emphasized several similarities between police departments and criminal courts. In both cases, the adoption of predictive instruments was justified by presenting the tools as more objective and efficient than “gut feelings.” In both cases, we found processes of professional resistance, including foot-dragging and data obfuscation, fueled by similar fears of surveillance and deskilling. Yet despite these similarities, important differences emerged between the two sites with respect to centralization and managerial authority: overall, algorithms were used less in the criminal court than in the police department under consideration. In this final section, we discuss the relevance of these findings with respect to social and racial inequality in the age of big data.

To date, most of the debate regarding technologies of crime prediction has focused on the internal bias of algorithmic instruments. Scholars have shown that, because algorithms learn by being fed historical data, inequalities from the past will be reflected into the future (Angwin et al. 2016; Brayne 2017; Harcourt 2006; Mayson 2019). Our analysis supports these findings. In addition to embedded bias, we find that predictive technologies can come with performative effects, in the sense that they not only predict events but also contribute to their future occurrence. For instance, in the Chronic Offender System, officers are instructed to focus their attention on the highest point-value individuals. This can easily lead to a feedback loop, in which if individuals have a high point value, they are subjected to heightened surveillance, and, therefore, are more likely to be stopped, thus further increasing their point value while obscuring the role of enforcement in shaping crime statistics and appearing to be objective – or, in the words of one captain, “just math.”

Yet we argue that another pathway through which predictive technologies may increase inequalities relates to their social contexts of reception, and, more specifically, to the shifting role of discretion. There is a long tradition of research on the decoupling or “loose coupling” (Meyer and Rowan 1977) between abstract legal or bureaucratic principles and the daily practices of police officers and legal professionals. Scholars have emphasized the significant amount of authority and discretion that police officers and legal professionals exercise in their daily work. Police officers and legal professionals do not follow a policy of “full enforcement” where they strictly enforce all criminal statutes at all times against all offenders (Bittner 1990; Goldstein 1963; Reiss 1973; Stuart 2016; Van Maanen 1978). Rather, they exercise discretion, constantly using their judgement—informed by a variety of social factors—to decide what to do, whom to police, how to enforce, and what to record (Black 1980).

Similarly, on the criminal justice side, “gap studies” have analyzed the complex relationship between “law on the books” and “law in action” (Bourdieu 1987; Gould and Barclay 2012). Previous studies of misdemeanor courts have shown how managerial, practical, and social considerations regularly encroach over legal rationality in the daily administration of understaffed and overwhelmed court systems (Christin 2008; Feeley 1979; Kohler-Hausman 2018). Both in the policing and criminal court cases, existing research showed how discretionary power tends to amplify social and racial inequality in a context of temporal and budgetary constraints (Alexander 2010; Barocas and Selbst 2016; Beckett et al. 2005; Brayne 2017; Kohler-Hausman 2018; Stuart 2016).

Interestingly, removing discretionary power in order to diminish bias is precisely what predictive algorithms aim to accomplish – or, at least, this is what proponents claim to want to accomplish when they justify adopting big data technologies. By providing more accurate and objective information to police officers and legal professionals, advocates argue that predictive algorithms can make law enforcement and sentencing more efficient and fair l (Milgram 2012). Our research shows that predictive algorithms in policing and courts also come with “function creep,” or the gradual widening of the use of technologies beyond their original purpose, with a particular bias towards managerial surveillance. Thus, the digital data collected on the conduct of officers and legal professionals more tightly couples managerial control and officer decision-making, thus further constraining discretionary processes.

Yet the reality of how predictive algorithms are used on the ground casts doubt on the idea that technology can fundamentally erase discretion in the criminal justice process. Indeed, our analysis shows how, far from eliminating discretionary power in its various forms, the adoption of predictive algorithms in fact displaces discretion to less visible parts of the organization. Thus, both police officers and legal professionals manipulate the data at their disposal to regain the autonomy that they feel is being threatened by the adoption of these technologies. New actors also come into play, including data analysts, data entry specialists, and technology teams, who create novel forms of discretionary power within the institutions.

Such shifts in discretion following the adoption of predictive technologies can, in turn, lead to new increases in discriminatory behaviors. Here, the historical example of sentencing guidelines, which were intended to address earlier concerns about discretion, serves as a cautionary tale (Espeland and Vannebo 2007; Lynch 2017). Progressive advocates thought that existing disparities in sentencing revealed overt discrimination and a punitive mindset among judges. They supported the Sentencing Reform Act and the creation of the sentencing guidelines in 1984. Yet it soon turned out that instead of eliminating discretion, the sentencing guidelines led to a displacement of discretion. The guidelines kept changing to take into account new categories of offenses; judges struggled to follow and implement these changes. Prosecutors, however, were not constrained by the guidelines and saw a significant increase in their relative decision-making power: they were the ones deciding on the charges that would then constrain the decision of the judges, since it would determine the “Offense Level” column in the Sentencing Tables (Espeland and Vannebo 2007; Joh 2016). This increase in prosecutorial discretion – and the exponential increase in long prison sentences that followed – was a central cause of the turn to mass incarceration that took place over the next thirty years (Alexander 2010; Pfaff 2017).

As we saw, predictive algorithms are different from the sentencing guidelines in several key ways: they forecast the risk of future events, rather than standardize decision-making based on historical averages; they draw on more fine-grained data and sophisticated models; they are used throughout the policing and criminal justice process, instead of only in sentencing. Yet we find that they come with similar kinds of effects – which we analyze as strategies of resistance – in terms of moving the locus of discretion in law enforcement and criminal justice.

This opens up pressing questions regarding the interplay of technology, discretion, and inequality in the age of big data. Will these transfers of discretionary power in reaction to the introduction of predictive technologies come with similar or different effects on inequality and incarceration compared to the sentencing guidelines? Who are the new categories of actors currently gaining discretionary power and what are their incentives? How will discretion evolve, given the growing sophistication of predictive instruments and tightened managerial control taking place in policing and criminal courts? In addition to the study of algorithmic bias, future research should investigate the role of contexts of reception, and examine how the recent shifts in discretion will affect inequality in the criminal justice system.

CONCLUSION

This article examined the reception of predictive algorithms in the criminal justice sector. Drawing on in-depth ethnographic fieldwork conducted within a police department and a criminal court, we compared the uses of algorithms at different stages of the criminal justice procedure. We documented many similarities between policing and courts. In both cases, the implementation of predictive algorithms is justified using similar arguments – most importantly that they are more objective and efficient than “gut feelings.” Yet the deployment of algorithmic techniques faces criticism from both law enforcement and legal professionals, who feel that their autonomy and experiential knowledge are threatened. We documented similar processes of professional resistance – from foot-dragging to data obfuscation – among police officers and legal professionals. That said, there are also significant differences between the two cases. Specifically, the modalities of managerial control in the police department and criminal court under consideration are associated with different types of implementation and resistance to algorithmic technologies.

These findings shed new light on the impact of predictive technologies on the daily enforcement of the law in the United States. In addition to current research on the social and racial biases embedded within algorithmic technologies, we find that the implementation of predictive algorithms comes with unintended consequences in terms of discretionary power. This helps us to understand when and why there is a gap between the intended and actual effects of predictive instruments. At a time when algorithms and analytics are multiplying, often with the hope of “reforming” major institutions such as the criminal justice system, our analysis reveals the enduring role of discretion, power, and occupational cultures in shaping the social and political impact of technological change.