A Practical Framework for Considering the Use of Predictive Risk Modeling in Child Welfare

Abstract

Predictive risk modeling (PRM) is a new approach to data analysis that can be used to help identify risks of abuse and maltreatment among children. Several child welfare agencies have considered, piloted, or implemented PRM for this purpose. We discuss and analyze the application of PRM to child protection programs, elaborating on the various misgivings that arise from the application of predictive modeling to human behavior, and we present a framework to guide the application of PRM in child welfare systems. Our framework considers three core questions: (1) Is PRM more accurate than current practice? (2) Is PRM ethically equivalent or superior to current practice? and (3) Are necessary evaluative and implementation procedures established prior to, during, and following introduction of the PRM?

The past four decades have witnessed the dawn of the information age, and with its arrival has come an array of data tools useful to researchers. Advances in computer storage have allowed a vast amount of data to be held essentially for free and processed with breathtaking speed. Advances in a broad array of analytic approaches (e.g., propensity score matching, various regression techniques, machine learning applications) have allowed data to be used to more accurately answer a range of critical practice and policy questions.

Complex algorithms simply could not be executed on large datasets in a reasonable timeframe on generally available computer platforms 30 years ago. Some authors of this article recall spending tens of thousands of dollars to procure storage for datasets that took months of processing time to analyze. Far larger datasets are now transferred routinely across the internet in seconds or minutes, and far more complex analyses are routinely conducted in minutes or, at worst, hours. Put simply, sophisticated analytic methods that use big datasets, like predictive risk modeling (PRM), are newly available for application in fields like child welfare. All new technologies come with risk, however. It is important to fully understand both the potential benefits and the potential risks that adoption of PRM might bring. The obvious question arises: Should we embrace such technologies?

Our Approach to Evaluating Predictive Risk Models in Child Welfare

There is a natural and quite healthy human tendency to be wary of the new. Premature adoption of new technologies or practices can have catastrophic results. One clear example can be found in the heroic actions of Dr. Frances Oldham Kelsey in forestalling the adoption of Thalidomide in the United States. Her caution, based on what she found to be inadequate prior trials, along with other concerns, spared our country the devastating level of impact experienced by various European countries (Rice 2019). Of course, there have also been historical instances where effective new treatments or interventions have been unnecessarily delayed. There is a balance that must be found.

This article is concerned with the application of one particular new analytic tool—PRM—to child welfare programming and policy. PRM is a kind of analysis that uses existing data and machine learning to predict the likelihood of outcomes among people. The prediction of human behavior is both innately complex and has ethical implications. In our view, there are three core conceptual questions that must be answered to evaluate the use case for PRM: (1) Compared to currently available tools and current practice, can PRM approaches more accurately predict outcomes of interest to child welfare policy and practice, both overall and for subpopulations (e.g., racial/ethnic groups, class, age, geography/setting)? (2) Compared to existing options, is PRM ethically inferior, equivalent, or superior? and (3) Are the key factors related to a successful implementation of a PRM in place? Surrounding these are a range of other subsidiary or related practical questions, particularly related to implementation, such as how appropriate use of a PRM is dependent on factors such as workforce training, agency policies, and ongoing empirical monitoring. Our framework for evaluating PRMs builds upon the “validity/equity/reliability/usefulness” model (Coohey et al. 2013; Hughes 2018; Russell 2015), with which it broadly aligns. Their validity and equity components correspond to concerns with accuracy both in general and for subgroups, while their reliability and usefulness components refer to implementation issues. Our focus builds on these factors by adding a focus on ethical issues.

We argue that these questions cannot be addressed in a general or abstract sense but can only be answered with reference to individual use cases. We define a use case as that situation, purpose, and actual use to which the PRM is applied. Absent a clearly operationalized use case, we cannot make judgments about the utility of any tool. A related point that we would like to stress is that a PRM is a tool and must be evaluated as such. For example, you could confidently say, “A hammer is better than a wrench at driving a three-inch nail through a half inch plank of oak”; but it is nonsensical to say, “A hammer is a better tool than a wrench.” Moreover, some issues relating to reliability and validity will be site-specific. For example, larger jurisdictions have a distinct advantage in obtaining sufficiently large training samples. In addition, predictors that are very rare in smaller jurisdictions may not be stable enough to use.

As the title of this article implies, we come at our subject with a clear practical and empirical bias—we favor arguments that are subject to empirical validation over arguments that are purely theoretical or untestable. For example, while we certainly consider what theoretically might occur when PRM is employed, we believe that policy decisions should be based on what can be shown empirically to actually occur when PRM is employed in particular use cases. Implicit in these arguments is the assumption that the ethical and predictive value of any tool is always relative to other available alternatives. PRM cannot be “ethically sound” or “ethically troubling” in any particular use case except compared to other best available practices.

Finally, we acknowledge the difficulty of decision-making in the child protective services (CPS) context. Child welfare hotline screening is important but is often based on scant information. Inclusion or exclusion of particular data elements can have profound, and often counterintuitive, consequences. For example, “ban the box” policies were an attempt to increase the hiring of minority males by forbidding decision-makers (employers) from asking about criminal history. These policies actually reduced hiring of minority males (Doleac and Hansen 2020; Agan and Starr 2016). For us, this highlights the potential chasm between theory or intent and measurable empirical outcome. Throughout this article, we emphasize the need for empirical testing, and we feel this need is especially strong in low-information contexts where unpredictable events may likely occur.

What Are Predictive Analytics, Machine Learning, and Predictive Risk Modeling?

Predictive analytics is a broad term referencing the use of preexisting data to develop models to predict a future outcome. Machine learning refers to those approaches in which computers learn (in part) on their own. Machines can do this through exposure to existing datasets, which can be used to “train” the computer to create an algorithm that predicts the desired outcome as well as possible. This can only be done when both the predictive variables and the outcome variable are available. In simple terms, if you know a number of things about a situation (e.g., the information contained in an initial child welfare referral), and you also know about a later outcome (e.g., having a subsequent rereport or not) for those same cases, you can use a machine learning approach to try to create a predictive model. PRM falls under both the domain of predictive analytics and machine learning. It is a broad term for a set of tools that identify which variables can best predict the outcome that the model is trained on and create a predictive model that results in a risk score for each case. This risk score can be scaled as desired (e.g., 1–100 or 1–20). Some common PRM methods include regression and random forest models, which have the same purpose but have different internal processes. PRM has been considered for use in many child welfare systems, and for a variety of different purposes. It has also been formally employed as standard practice to aid risk assessment in some places, such as Allegheny County, Pennsylvania.

What risk assessments are used currently?

Actuarial risk assessment tools dominate current common practice in risk and safety assessment in child welfare. The history of these tools’ adoption can be traced back 50 years. At that time, child welfare decisions were commonly made using unaided worker judgment, often with a supervisor clearing the decision. Those in child welfare quickly realized that there should be a formal way of determining risk that could support worker decision-making (Pecora, Chahine, and Graham 2013). The system consulted experts and developed “consensus-based” instruments that were theoretically predictive of maltreatment. These instruments, in turn, gave way to “actuarial” instruments, which were quite different in derivation. While consensus-based tools included items believed to be predictive of maltreatment, actuarial tools, as the name implies, included items empirically demonstrated to be predictive of maltreatment. Over the past 20 years, research has found that the use of actuarial tools outperforms both the use of consensus-based tools and unaided worker judgment (Grove and Meehl 1996; D’Andrade, Austin, and Benton 2008; Baird and Wagner 2000). Most empirical research in this area uses system outcomes, such as rereferral as outcome measures. Risk assessment tools, including PRM, can be implemented in many ways. When an individual or family receives a very high-risk score, child welfare generally takes a given course of action (e.g., a referral is accepted), but overrides in the system also exist. For example, a decision not to intervene when a very high-risk score is generated might require supervisory approval. The agency’s risk assessment protocols go far beyond selecting an appropriate tool. Policies and worker training must be designed to optimize the utility of the selected instrument. Workers must be aware of the specific role that risk assessments are intended to play, and the centrality of their own clinical judgment.

One cannot address the advisability of adopting PRM without an understanding of what technology the PRM seeks to supplant. According to Coohey and colleagues (2013, 151), “Actuarial risk assessment is a statistical procedure for estimating the probability that a critical event, such as child maltreatment, will occur in the future.” More specifically, these tools assign probability weights to risk factors statistically known to be associated with child maltreatment to determine the risk level, based on predetermined cut points, of maltreatment reoccurring. These validated tools assign a level of risk to a family, and the tools are used to complement child protective services workers’ clinical judgement throughout critical decision-making points. There are several types of actuarial risk assessments that the child welfare system uses, each with varying levels of support and validation.

One of the most commonly used actuarial systems is structured decision making (SDM). Initiated in 1998, the goal of SDM is to assist CPS workers in making consistent and valid assessments regarding child risk and safety throughout various critical points in the child welfare process including initial screening (National Council on Crime and Delinquency [NCCD] 2019). These critical points include the initiation (screening in or out) and disposition of an investigation, case planning, ongoing case evaluation, reunification, and case closure (NCCD 2019). In one study, 27 percent of families identified as having high or very high risk of recurrence had a rereport within six months (Children’s Research Center 2008). Families identified as having low or moderate risk had less than 6 percent rereport rates.

How is PRM different?

PRM models are capable of considering a much broader array of variables than risk assessment instruments used in the field, including those that may be available from data sources other than the individual reporting the concern about a child, or that which is observed by an investigative caseworker. The use of predictive analytics in child welfare may address some of the challenges and limitations of actuarial risk assessment tools (Cuccaro-Alamin et al. 2017). Generally, the application of PRM to CPS has shown promising results when CPS has identified an appropriate use case. A common measurement of predictive accuracy for PRM is the receiver operating characteristic (ROC) curve. This curve plots the sensitivity (true positive rate) and specificity (false positive rate) for each risk score vis-à-vis a specified subsequent outcome (Coohey et al. 2013). The area under the curve (AUC) indicates the probability that the classification model will assign a higher probability of an outcome to families that truly have higher risk for that outcome. AUC values range from 0 to 1, with AUC values of .70 or higher being desirable. Several studies have evaluated how well these models perform. For example, in one early study by Vaithianathan and colleagues (2013), the prediction of maltreatment risk by PRM model had an AUC of .76.

Illustrative Case Studies

Fields outside of child welfare have historically used predictive analytics (Church and Fairchild 2017). These include, among others, health (Duncan 2011) and criminal justice (Hannah-Moffat 2019). To provide some concrete orientation to the issues addressed in this article, we present two very brief case studies of the use of PRM in a child welfare context. These divergent cases illustrate how important the appropriate “use case,” as well as development, testing, and implementation contexts are.

The Illinois experience with Eckerd’s Rapid Safety Feedback Program

We chose the first case study because it both appears to be an example of a predictive analytic approach that failed to perform and directly illustrates many of the key questions and issues we raise in this article. Although Illinois’ deployment of Eckerd’s Rapid Safety Feedback Program (ERSF) is commonly discussed in both academic and nonacademic sources, we, like Gillingham (2019), believe that all available sources on this matter trace back to a single newspaper article in the Chicago Tribune (Jackson and Marx 2017). The Tribune article asserts that George Sheldon, then-director of the Illinois Department of Children and Family Services (DCFS), set up and used an internal grant mechanism, rather than an open bidding mechanism, to hire Eckerd, and that the grant was given to prior associates of Sheldon. Sheldon hired Eckerd to implement the ERSF, which used existing data from Illinois DCFS to predict if children reported to the DCFS hotline would be “killed or severely injured” (Jackson and Marx 2017).

As it turned out, the ERSF predictions were catastrophically flawed, with more than four thousand children being flagged as having a 90 percent probability of death or injury and almost four hundred children being classified as having a 100 percent chance of serious injury or death. Setting aside the obvious fact that predicting anything at such high levels of certainty is remarkable, the raw predicted numbers were obviously far too high. For example, during 2015, Illinois reported seventy-seven child fatalities (U.S. Department of Health and Human Services [DHHS] 2017), a rate not inconsistent with national averages. In addition to this very high range of false positives, the system had a large number of false negatives, failing to accurately predict maltreatment-related fatalities that did, in fact, occur. The Tribune describes how the program that the DCFS director discontinued and quotes the current director of DCFS, Beverly Walker, who said, “We are not doing the predictive analytics because it didn’t seem to be predicting much.”

This case study is remarkable in that, to the degree the Tribune’s reporting is accurate, this use of predictive analytics violated most or all of the key principles necessary for a practical and ethical use of such a method. In particular, it appears that DCFS did not adequately pretest the program, else it is difficult to understand how so many children could have been classified at such a high level of risk. The algorithm was apparently proprietary and therefore not transparent, so the application of predictive analytics in this use case is intrinsically suspect. Further, the use case itself is highly suspect, as there are no known analytic methods of any kind that can effectively tackle the “haystack problem” (see our discussion of the critiques of PRM).

Allegheny County’s Family Screening Tool

Perhaps the most thoroughly evaluated PRM in use in child welfare is Allegheny County’s Allegheny Family Screening Tool (AFST). Hotline workers use the AFST to help them decide if a case should be screened in or screened out, a decision-making point where there is often limited information with a very brief window of time to make a decision. The AFST is a PRM that uses a large number of data elements derived from the internal CPS data as well as from other state systems, including jail records, juvenile probation records, behavioral health records, and birth records. The AFST uses data relating to both children and adults in a family. An early version of the AFST used public benefit records, but the current version does not use these data (Vaithianathan et al. 2019). Variable inclusion decisions are often both empirical and political, with many localities making decisions to exclude “hot button” variables, such as the race of the family, particularly when they do not markedly improve instrument accuracy. The AFST generates a score between 1 and 20.

Several elements of the AFST development and adoption process are noteworthy. First, the actors involved made concerted and continuing efforts to be transparent and consciously attend to ethical concerns, both in the form of direct outreach and through a large array of publications, even including a periodically updated online Frequently Asked Questions (Allegheny County DHS 2019a, 2019b). This tool was developed over several years and involved an open solicitation of proposals, community outreach, and feedback, a long series of publications describing the nature and use of the instrument (e.g., Vaithianathan et al. 2019), a process evaluation (Hornby Zeller Associates 2018), and a review of the ethical considerations attendant to the use of the instrument (Dare and Gambrill 2017). The discussion over the advisability of the instrument flowed into the public square, with commentary both supportive of and opposed to being easily accessible in various high-profile publications, such as the New York Times and Wired (e.g., Hurley 2019; Miller 2018; Giammarise 2019; Eubanks 2018). It is also notable that workers were not chained to the recommendations of the AFST—their decisions could vary from what the machine suggested (Vaithianathan et al. 2019).

Second, established scientists with a prior history of developing predictive risk models in child welfare developed the AFST. Early publications that described the AFST detailed the methods and algorithms used and included demonstrations of the predictive utility of the tool compared to prior known screener accuracy and also, importantly, against non-CPS outcomes (see “feedback loop” discussion). The AFST was thoroughly pretested by internal and external team members (Vaithianathan et al. 2019) to determine the degree to which the risk score predicted unwanted outcomes (e.g., rereport) both in general and for particular population subgroups. The developers conducted the analyses before DHS deployed the AFST. They were able to model what a predictive model would have determined about historical cases and assess those risk determinations through checking historical outcomes and comparing them to what decisions were actually made, historically. This pretesting or “virtual test drive” of a system is one of the key advantages of predictive analytics—you can use history to validate different algorithms.

Testing what an instrument recommends against prior action is not, however, the same as human beings testing the instrument. To do this, DHS commissioned an independent quasi-experimental (pre/post) review (Goldhaber-Fiebert and Prince 2019). The review found that use of the AFST modestly improved sensitivity (reduced false positives), had a very small degradation in specificity (slightly increased false negatives), did not result in higher workload (the number of screen-ins did not go up), reduced racial disparities, and improved consistency among hotline staff. Both the independent evaluators and Allegheny County found the implementation of the AFST to improve practice, and the county is moving forward with the algorithm and is attempting to further improve both the algorithm and how it uses it.

Highlighting the issues of sensitivity (capturing true positives) and specificity (capturing true negatives) is useful. We argue that the human cost of low specificity (failing to see a low-risk situation correctly) increases with deeper CPS involvement. Wrongly terminating parental rights is worse than wrongly removing a child, which is worse than wrongly substantiating a case, which is worse than unnecessarily screening a case in. On the other hand, the cost of low specificity (failing to see real risk) can be lethal to a child at any point in the process. This is why decision thresholds are lowest at the screen-in level and become more restrictive as CPS involvement continues. Ideally, information, which accrues during engagement with CPS, enhances specificity. Procedures that set thresholds for all risk assessment tools should take this into account.

The Public and Academic Discourse: Time to Embrace Change or Hit the Panic Button?

The public and academic discussions around PRM are contentious. Fortunately, the terms of the debate are fairly well established, with key concerns being divisible into general categories, including issues of accuracy, ethics, and implementation. We outline major arguments both for and against PRM and consider them alongside empirical data and ethical principles.

In our view, almost any discussion about child maltreatment practice or policy can benefit from paradigmatic clarity about the role of CPS. CPS could be framed as akin to a criminal justice or emergency medical system, responding to events after they occur and providing services to ensure immediate safety. This has been colorfully and distressingly illustrated with the “ambulance at the bottom of the cliff” metaphor (Keddell 2019). An alternate paradigm is that CPS is intended to also have a preventative role through provision of services—a public health model. How CPS is viewed fundamentally impacts the choice of how we evaluate success (operationalizing outcome metrics). Under the first paradigm, immediate outcomes make sense; but under the second paradigm, longer-term outcomes such as rereporting and foster care subsequent to future reporting also make sense. Reviewing the mission statements of most state CPS programs suggests that the second paradigm more accurately captures the intent of most states, and it is one we support. It is heartening to note that the outcome variables chosen in many evaluations of PRM (e.g., Allegheny County) include longer-term measures consistent with the evaluation of a preventative role of CPS. In this section, therefore, we conceptualize CPS as having both an emergency response and a preventative role.

Arguments for how PRM may support ethical, effective practice and social justice

Proponents of PRM claim that the modeling offers an accurate way to assess risk and that accurate risk assessment is ethically valuable. While PRM could be used for other child welfare functions besides risk assessment, including targeting service delivery and evaluating internal agency processes (Hughes 2018) or even case-finding (Putnam-Hornstein and Vaithianathan 2018), the heart of the current debate is focused on the use of PRM at the hotline screening level. Accuracy is the sine qua non of risk assessment—without accuracy, assessment serves no purpose. The core assertion that proponents of PRM must make is, therefore, that accuracy is improved.

The instrument itself must be more accurate. That is, the risk assessment scores that the PRM generates must be more accurate than the risk assessment scores that the tools currently in use generate. In addition, this accuracy must extend to subgroups of the population. These two requirements conform to the “accuracy” and “equity” elements of the accuracy/equity/utility/reliability model. Concerns about accuracy within subpopulations stem largely from long-standing concerns among scientists (e.g., Dettlaff and Rycraft 2008) as well as nonscientists (e.g., Wexler 2017) that public systems may discriminate against families by class or ethnic/racial characteristics. Any system that, for example, markedly increases the rate of false positives among screened-in African Americans would be ethically unacceptable. It is important to note that this concern is not unique to screening but has been expounded across all decision points in the child welfare system, from screening to service delivery to substantiation to foster care entry.

Which brings us to ethics. Accuracy is every bit as much an ethical as a technical issue. There is a positive ethical value in using better tools, particularly when the use of these tools impacts people’s lives. In the same way that it is obviously ethically preferable for a surgeon to use a sharp scalpel or a radiologist to use the most detailed available image, it is ethically preferable for child welfare systems to employ a more accurate risk assessment tool. More accurate risk assessment can have benefits accompanying increased sensitivity, such as improving the agency’s likelihood of responding to high-risk situations, as well as benefits attendant to increased specificity, such as reducing concerns regarding possible trauma and stigma resulting from unnecessary contact.

New Zealand, Allegheny County, and California (Dare 2013; Dare and Gambrill 2017; Drake and Jonson-Reid 2018) commissioned separate ethical analyses in contemplating of the use of PRM in child welfare. All three reports found that, in sum, the ethical advantages of employing PRM exceeded the ethical problems raised, and that there could, in fact, be ethical issues in not adopting the most accurate tool available.

Other advantages of using PRM to augment human decision-making include speed and breadth of data that can be reviewed. PRM can be automatically and instantly generated at the level of a hotline call, partly using information residing in the system before the call is even made. In terms of breadth of available data, a PRM will consider any information from any case or other data element that you want. It is subject to some of the same limitations as human review (e.g., reliance on incomplete or missing data) but is not subject to others (e.g., failing to notice a data element or not being able to take the time to adequately review a file).

An additional advantage of PRM over actuarial tools is that it can be consistently generated. Actuarial instruments, even those following a SDM approach, must take input from human beings, who are fallible and can be inconsistent in their behaviors. This is one reason why so much of the literature on the use of actuarial instruments focuses on adequate training of the people who will administer them (Cuccaro-Alamin, Vaithianathan, and Putnam-Hornstein 2017). PRM is also vulnerable to wrong data inputs, but to the degree that it draws on scores of fields, rather than a few, the impact of a single incorrect data element is minimized.

Arguments for how PRM may threaten ethical, effective practice and impede social justice

Concerns regarding the use of PRM in a child welfare context also fit into the “accuracy/ethics/implementation” structure. Various concerns exist regarding system accuracy, including the familiar and potent “GIGO” concern (Glaberson 2019). “GIGO” is an old computer science term meaning “garbage in, garbage out.” Given that PRM relies on preexisting data in computerized systems, critics have raised concerns that any predictions based on such data will be inherently inaccurate. “Bias in, bias out” (BIBO) is an associated problem, one that the criminal justice PRM literature frequently discusses (Howcroft and Rubery 2019). It is also present in the use of PRM in child welfare (Glaberson 2019). From a technical perspective, we might conceptualize GIGO as encompassing both random and systematic error. BIBO asserts that biased data cause systematic error. In particular, data biased against class or, more commonly, racial minorities will cause more false positives (lower specificity) among those populations. For example, if Black adults are disproportionately likely to be arrested simply because of their race, and if arrest is an element in a predictive model of maltreatment risk, then that model could advance this systematic bias into its risk score.

Glaberson (2019) is representative of those who are concerned that PRM may not recognize and account for historical changes—the so-called Zombie Prediction problem. Yet another issue can be found in potential “feedback loops” (Keddell 2019). Feedback loops may occur when within-system indicators of risk, particularly previously accepted CPS reports, are used as an element in a predictive model. In such cases, people may be screened in partly because they were screened in before, theoretically causing a self-perpetuating loop in the system.

Accountability is another frequent issue. For example, a high-risk score for a given individual in random forest models will depend on the outcomes of a vast number of randomly generated decision trees. The way in which they come together to yield a given score in any given case is certainly mathematically knowable but cannot be easily explained to a person, as it involves a massive number of pathways. Accountability problems are multiplied when the public has no ability to examine the algorithms that the model used. For this reason, proprietary algorithms are particularly ethically troubling (Church and Fairchild 2017).

Finally, some have concerns that computer algorithms in child welfare may tend to degrade or dehumanize the quality of clinical social work (Gillingham and Graham 2016). This is not an attack on PRM per se but, instead, a set of concerns about the de-professionalization of workers as they are theoretically reduced to feeding information into and taking orders from machines. We are warned of a “modern times”–like dystopia where workers are mere cogs in machines (Gillingham 2016). Such a concern is present whenever new empirical technologies or approaches become available, such as in discussions of the utility of evidence in practice (e.g., Guyatt et al. 1992). Oak (2016) raises the science fiction film Minority Report as an exemplar of how predictive methods can lead otherwise reasonable people to engage in unjust practices through an uncritical application of risk assessments.

While some scholars have energetically forwarded practical and technical arguments against PRM, we can find the primary objections to PRM in the realm of ethics. Concerns exist about the loss of privacy, specifically the failure to obtain consent for the use of personal data. In addition, there are concerns that a PRM with low specificity could be stigmatizing. In this case, stigma is defined as a personal or social disgrace associated with an unnecessary (false positive) child welfare screen-in or subsequent determination.

Many of the issues regarding the use of predictive analytics in child welfare are not new and have been subject to substantial consideration in other disciplines. For example, in the criminal justice literature, PRM has been used in endeavors such as “predictive policing” (Meijer and Wessels 2019) and in sentencing (Robinson 2017). Many of the same key issues are encountered—the centrality of the use case, the ethical necessity of transparency, concerns over the quality and applicability of the data that the model uses, and the need to prove increased accuracy, both overall and especially for vulnerable subgroups. Many of the key recommendations are also the same—improve data quality, use careful and public analysis of models and outcomes, and avoid the proprietary firms that refuse to make their algorithms public.

Unmuddying the Waters: Evaluating Arguments for and against PRMs in Screening

In writing this article, we were struck by how common it is for there to be a very high level of overlap between those issues that detractors of PRM highlight and those issues that proponents of PRM cite as being necessary to overcome prior to the successful implementation of the technology. In other words, both sides agree on the core issues but disagree on if they can be addressed successfully. We argue that this is an issue that is amenable to the application of evidence with regard to particular use cases.

Many criticisms of PRM are not really specific to PRM

Many of the criticisms of PRM are criticisms of risk assessment in general and not criticisms of PRM in particular. As such, they are not reasonable grounds to oppose the introduction of PRM, unless the advocates of such arguments are willing to apply similar criticisms to all risk assessment tools. These concerns are not trivial or unimportant. Many are central and critical. However, they do not constitute arguments against the use of PRMs per se, which is the focus of this article. The following passage from Church and Fairchild (2017, 73) is illustrative:

Child welfare’s embrace of predictive analytics has brought three challenges to surface. First, there is very little known about how or why a tool is making a prediction or recommendation. In other words, we have very little algorithmic detail on any of the tools. Second, these algorithms focus on predicting rare events, such as identifying high risk cases early in a case, typically at or shortly after intake. Finally, each algorithm’s output is a single numerical risk score, which no doubt oversimplifies most matters.

None of these issues is specific to PRM. The second issue that the passage describes—the difficulty of predicting rare events (i.e., needle in the haystack problem)—is undoubtedly one of the most serious challenges in social science (Lanier et al. 2020), but it exists irrespective of the method used to make the prediction. This is a characteristic of the use case (e.g., trying to predict fatalities vs. trying to predict rereport) not the method. Similarly, why oppose PRM on the basis that it generates a single risk score when many currently used risk assessment tools share this characteristic?

In addressing the first issue in the quotation, we could easily argue that we do not know how or why an actuarial risk assessment tool is making a certain recommendation. We can certainly explain, “Well, it is because we checked this box,” but that is not the same as understanding how or why that item is predictive. Actuarial items are definitionally selected because of correlations with a given outcome, not because we theoretically understand how or why they are related. Deriving questions from how and why was, in fact, the first formal approach that risk assessment used, and those consensus-based tools did not work well. For example, single-parent status, maternal age at birth, and poverty are all highly predictive of maltreatment as individual items. We do not know, however, how or why these items are predictive. Why is single-parent status predictive of maltreatment? Is it due to greater stress burden? Is it due to frequently accompanying poverty? Is single-parent status, itself, particularly at a young age, a proxy for other underlying personal, family, or community issues? We have yet to generate a strong body of explanatory knowledge about the mechanisms for the association between various risk and protective factors and maltreatment risk. This is a fundamental problem for the development of interventions, but it is less of a problem if the aim of the tool is merely to accurately identify families most in need of further assessment.

The GIGO and BIBO problems also largely fall into this category. To the degree that current risk assessment tools or workers focus on similar data points that are subject to either random or systematic error, they share these issues with PRM applications. For example, there is little difference between a worker entering potentially erroneous information about a child’s age into a risk assessment tool and a PRM automatically extracting the same erroneous information and using it.

This same issue applies to the feedback loop problem. Prior child welfare reports are commonly among the most predictive elements in current risk assessment tools. In this way, current tools have the capacity to promulgate this issue, although an argument can be made that the breadth of variables that PRM uses allows for more possibilities for feedback loops to manifest. On the other hand, a risk assessment tool largely driven by prior reports is of little use for assessing risk for families at the time of their first CPS contact. A PRM with access to a broader range of preexisting risk factors may help to overcome this problem. Despite this, we think the feedback loop problem is concerning, and we discuss it further here.

The zombie problem—that of instruments losing predictive utility due to changing conditions in the world (the classic threat to validity usually termed “history”)—is clearly worse for standardized risk assessment tools than for PRM. While local PRM can and should be tuned on an ongoing basis, standardized risk assessment tools are updated far less frequently.

Remaining criticisms of PRM and means to address them

The nonspecific criticisms of PRM may well be valid in whole or in part depending on the context and certainly demand ongoing attention as PRM is implemented. They do not, however, constitute reasons for electing to stay with current procedures over PRM. Many specific concerns surrounding the implementation of PRM remain, however, which we discuss next, along with an account of how these issues have been addressed by those advocating for or adopting PRM.

Accuracy and implementation remain central issues.

PRM must be evaluated rigorously and continuously for accuracy, including sensitivity and specificity, both in general and with subgroups (e.g., Goldhaber-Fiebert and Prince 2019). We strongly oppose any initial or ongoing use of PRM (or any other risk assessment) that does not do this. We further divide the ability to evaluate accuracy into at least two stages.

Prior to implementation, a PRM should be trained and evaluated on historical data, a kind of “virtual test drive.” For example, the AFST was evaluated by external experts prior to implementation (Vaithianathan et al. 2019) to determine if the risk assessment scores generated using historical predictors and outcomes were more accurate than categorizations generated by actual practice. This analysis was extended to specific subgroups and also extended to non–child welfare system outcomes (e.g., hospital-recorded injury). Verification that the predictive model was not biased against subgroups (e.g., did not increase false positives) was necessary to satisfy the requirement for the ethical standard related to it being an equitable instrument. Verification against non–child welfare outcomes is a useful shield against the possibility of feedback loops. This is because, while feedback loops are theoretically capable of influencing outcomes within the child welfare system (e.g., future screening in or substantiation or placement), it is harder to argue that they might influence outcomes beyond the child welfare system, such as hospital-recorded injury, hospital-recorded abusive injury, or suicide, all of which the AFST score predicted (Vaithianathan et al. 2019). Using these two approaches—rigorous subgroup analysis and use of external outcome measures—can effectively address key ethical concerns about bias and feedback loops.

Following implementation, it is necessary for the implementing agency to evaluate system outcomes while the PRM is in use. This is very different from the preimplementation test drive. While the goal of preimplementation modeling is to see if the generated risk score is predictive of a given outcome (e.g., rereport), the purpose of postimplementation testing is to see if system accuracy improves once the PRM is actually in use. In this way, the second test verifies both instrument accuracy and the implementation of that instrument. Randomized controlled trials in such evaluations are ideal to minimize the chances of spurious causality (e.g., historical or unrelated program effects), but the near necessity of implementing such system changes universally will often necessitate quasi-experimental designs, such as the AFST used.

Moreover, even when a model seems to “work,” it may not work for everyone. For instance, a diagnostic test/model that has been validated in a high-prevalence group will have different predictive values when applied to groups with a lower prevalence. Thus, evaluation should be ongoing and should address overall accuracy and subgroup accuracy, and should use internal and external outcomes as measures. One unresolved weakness of this approach is that, as time goes on, it will be less and less possible to compare current PRM benchmarks to an increasingly distant pre-PRM condition. This is, again, not specific to PRM but is true of any similar evaluation of an assessment tool.

Beyond outcome evaluation per se, the implementation of any new tool, practice, or policy must be carefully planned, monitored, and evaluated. Again, this is not unique to PRM. Training is a key ethical and practical consideration for the implementation of any tool. Again, the issues we raised could apply equally well to any risk assessment tool and are not specific to PRM. Drake and Jonson-Reid (2018), drawing on Dare and Gambrill (2017), identified the following key areas for training. First, workers must understand the intended use of the tool and how it fits into their work and overall agency procedures. Perhaps most critically, workers must clearly understand that any risk assessment tool (including a PRM) is just that—a tool to assist human decision-making, not to replace it. With any risk assessment tool, it is likely that agencies will establish guidelines, and workers must be trained in these. For example, such guidelines might require that any hotline risk score over a specific (very high) threshold requires an investigative response unless supervisory override is given.

Ethical concerns.

Key ethical concerns (beyond those we have already discussed) include concerns with loss of privacy, lack of consent for utilizing personal data, the use of data for reasons other than for which they were obtained, and increased stigma. Addressing the last first, we find that this concern is completely subsumed under the accuracy criterion. In the case of the use of PRM for hotline screening, stigma is only a concern in false positive cases, that is, unnecessary investigations. The question of ethical harm from increased stigma, therefore, is the same as the question of accuracy, particularly specificity (assuming the overall screen-in rate does not rise). To the degree that a PRM enhances specificity, it reduces ethical concerns regarding stigma. The child welfare system should not employ a PRM that degrades specificity in the first place.

We now turn to issues of privacy, lack of consent, and use of data for reasons other than for originally collected. It is critical that each separate use case be analyzed individually. To take the simplest possible case, a PRM that the State of California considered as a screening tool (Drake and Jonson-Reid 2018) used only data that the child welfare agency already held and currently used. In a case such as this, there is no expansion of privacy concerns, and the concerns are moot. To expand on this, we argue that, in cases where the child welfare agency has already been granted access to data sources, the use of those data sources poses no new ethical concerns. In a different use case, and if pressed, one could frame a question such as, “Is it ethical to use data you are already using but more comprehensively?” Is it ethical to use records that could have been accessed prior to the PRM but often were not (e.g., arrest data that formerly required time to obtain)? In such a case, one might justify the expanded use of a dataset they already have permission to use, but this is something of a fine distinction given that they already had access.

Concerns regarding consent, loss of privacy, and use of data for purposes other than originally collected are similar and can be dealt with concurrently rather than sequentially. We argue that, again, the use of a PRM is no different from the use of any other method. Consent is most prominent in terms of individual decisions (such as consenting to medical care) and in consenting to be a research subject. Under the Common Rule, which governs human subjects policy in the United States (DHHS 2020), within-agency evaluations are exempted from the requirement to obtain informed consent. For example, if a hospital reviews its own patient records in a study meant to optimize its own triage procedures, that review is not subject to oversight. In such a case, the data are being used for a purpose other than that for which they were collected. For these reasons, we believe that agencies using their own data are largely exempt from these concerns, as long as they use them for the same general purposes (e.g., case decisions) and with the same degree of confidentiality. This situation can easily expand, however. In the case of the AFST, data were used outside the child welfare system. In such cases, ethical access to those data must be justified by the same means that using such data for any other agency purpose should be justified. Again, a PRM’s use of such data is not unique or different or a new frontier in any way. Traditional justifications, safeguards, and oversight should be employed by the implementing agency, ideally in conjunction with stakeholders and outside experts, as they would in any other case.

We also stress an overarching principle that we believe applies to all aspects of any public policy: transparency. In our view, maximizing transparency is key both for ethical reasons and for improving the system in question. Several steps can be taken to support transparency. In our view, all algorithms used must be public. Proprietary agencies that refuse to share their models should never be used. Not only is there a high ethical cost in lost transparency (Church and Fairchild 2017), but secret algorithms cannot be part of the process of knowledge building, which is a bedrock principle of science. In this way, proprietary algorithms are damaging both to the entity contracting for their use and to the advancement of science and the field in general.

Transparency is not a passive concept. In our view, employing any tool requires community engagement in the form of active outreach to stakeholders. This is especially true in the case of big data, which is new and is understandably frightening to many. This should not be a “We’re doing this to check off a box” enterprise, but a sincere attempt to engage stakeholders in design and evaluation from the earliest stages and on a continuing basis.

Transparency is also fostered by documentation of all stages of the consideration, specification, ethical justification, pretesting, process, and implementation analyses. As an example, Allegheny County has produced comprehensive documents covering all these issues, which are available on a public website. Such efforts are, in our view, not only possible but necessary. While there are fears that a screening PRM could become “a silver bullet inside a black box buried deep in a haystack” (Church and Fairchild 2017, 17), this certainly need not be the case.

Concerns regarding de-professionalization as a result of automation date back at least a century and are not new to the use of PRM. We fully agree that caseworkers are professionals and must not be transformed into mindless screen readers and button pushers. Workers must be trained in the use of any new tool and this concern is vibrant in the literature on actuarial risk assessment tools. A risk score that an SDM generates and a risk score that a PRM generates are similar. All ethical reviews of PRM that we are aware of place a fundamental emphasis on the necessity to train workers that PRM is a fallible tool, and not a celestial mandate.

Specifying a Proposed Framework for Implementing and Evaluating PRM in Child Welfare

We suggest a framework for adopting PRM and evaluating its utility in a child welfare context (see Figure 1). While it may seem strange to conflate implementation and assessment, understanding PRM in a child welfare context must simultaneously consider how the program is conceptualized, evaluated, and executed. Many of the elements in this model are not attributable to the authors, as the key elements and process are closely parallel to those that entities (e.g., Allegheny County) have adopted in designing, implementing, and evaluating their models. We do not specify the minutiae of each phase of the model, as we have already discussed our views on the mechanisms for ensuring transparency, equity, and ethical acceptability. Rather, this section is primarily to bring the previous parts of this article together in a simple visual format. The proposed model is not the only viable approach, but it is consistent with and illustrative of the points we have raised.

FIGURE 1.

A Framework for Implementing, Evaluating, and Assessing a PRM-Based Hotline Screening Process

The model in Figure 1 emphasizes the centrality of the use case. No program can be contemplated or assessed without a clear, operationalized description of what it is to be used for. Implementing agencies should build and refine their PRM in parallel with an (ideally) external review of the ethical issues attendant to the use case. They should then do preliminary model testing prior to actual implementation. Community engagement and establishing transparency should begin as the use case is being specified. Historical data can then be used to evaluate and refine the accuracy of the PRM, with promising models being advanced to implementation, again with ongoing evaluation. The lower box is included in the model as a reminder that the three dimensions of accuracy, ethical acceptability, and implementation are ongoing considerations for all stages of the process. There is never a point at which concerns about accuracy, ethics, or implementation are “put to bed”—they remain concerns from initial conceptualization through ongoing evaluation.

Conclusion

We see no insurmountable obstacles to the use of PRM in child welfare practice. We also do not wish to minimize the difficulty or necessity of assuring accuracy, ethical acceptability, and proper implementation. The advisability of adopting PRM as a tool is dependent on the specified use case and the demonstrated empirical performance of the tool, particularly its accuracy. Overall accuracy must be higher than current practice. The model must not systematically disadvantage subgroups. The model should be tested against external outcomes to assess the threat of feedback loops. Ethical concerns are central. While most ethical concerns regarding PRM will be similar to ethical concerns about existing models, implementing agencies must review and implement specific steps to address potential issues such as overreliance on the instrument. Community engagement and transparency are absolutely essential. Implementation of any PRM system must depend on demonstrated superiority to current practice and must address ethical and practical concerns. Processes and tools exist currently to allow for the safe and effective use of this technology, at least for the screening use case. As with all tools that powerfully impact people’s lives, ongoing evaluation, adjustment, and improvement of the tool are absolutely necessary.

Biographies

Brett Drake is a professor at the Brown School at Washington University in St. Louis. His research interests include applying big data to understanding child maltreatment, particularly “front-end” services and issues of class and race. He has coauthored an ethical review of potential predictive risk modeling uses in California.

Melissa Jonson-Reid is the Ralph and Muriel Pumphrey Professor of Social Work Research and director of the PhD program in social work at the Brown School at Washington University in St. Louis. Her research emphasizes improving outcomes for children in public child welfare using a systems perspective.

María Gandarilla Ocampo is a social work doctoral student at Washington University in St. Louis. Her research interests include child maltreatment, child protection systems, and the impact of mandated reporting policies on families and child welfare system outcomes.

Maria Morrison is a social work doctoral student at Washington University in St Louis. She has worked for over a decade for the Equal Justice Initiative. Her research focuses on cumulative traumatic stress among incarcerated and formerly incarcerated men in the context of current and historical racial injustice.

Darejan (Daji) Dvalishvili is completing her PhD in social work from the Brown School at Washington University in St. Louis. She has been working with UNICEF and other international and local nonprofit organizations focusing on child welfare. Her research interests are child maltreatment, gender-based violence, poverty, and economic strengthening interventions.