Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 May 13.
Published in final edited form as: Child Abuse Negl. 2008 Mar;32(3):377–391. doi: 10.1016/j.chiabu.2007.06.006

Caseworker assessments of risk for recurrent maltreatment: Association with case-specific risk factors and re-reports

Shannon Dorsey 1,*, Sarah A Mustillo 1, Elizabeth M Z Farmer 1, Eric Elbogen 1
PMCID: PMC3094152  NIHMSID: NIHMS46949  PMID: 18377987

Abstract

Objective

This article focuses on caseworkers’ assessments of risk of maltreatment recurrence among families in contact with social services. Specifically, the article has two primary goals: 1) to examine the association between caseworkers’ risk assessments and demographic, child, parent and family-level risk factors; and 2) to examine agreement between caseworkers’ risk assessments and any subsequent report, or reports, of maltreatment.

Method

Data are from the baseline, 12-month, and 18-month assessments of the National Survey for Child and Adolescent Well-Being (NSCAW), a nationally-representative sample of youth and families who were the subjects of allegations of maltreatment investigated by child welfare agencies. The sample consisted of a subset of NSCAW participants: cases with a report of child physical abuse or neglect who were not placed in out-of-home care (N = 2,139).

Results

Analyses indicated that parent-level risk factors and a prior report of maltreatment were most strongly associated with caseworkers’ assessments of risk for both physical abuse and neglect cases. A smaller set of factors, which varied by the type of maltreatment, were associated with a subsequent report of maltreatment. Despite some overlap in correlates of risk assessment and subsequent reports, analyses indicated that agreement between caseworkers’ assessments of risk and re-reports was low.

Conclusions

Findings suggest that although caseworkers’ assessments were associated with a limited set of risk factors from the literature, few of these factors also were associated with a recurrent report of maltreatment. Correspondence between caseworkers’ assessments of risk and a subsequent report of maltreatment was low, suggesting that considerable work may be needed to improve accuracy and identification of cases most at risk.

Practice implications

This study provides information to assist caseworkers, administrators, and policymakers in thinking critically about risk assessment policies and procedures. Although caseworkers’ assessments of risk were associated with some of the empirical predictors of recurrent maltreatment, their assessments were only slightly better than guessing. Agreement between caseworkers’ risk assessments and actual subsequent reports was better for low-risk cases, but primarily because the majority of cases did not have a subsequent report during the study period. Clearly, considerable improvement in risk assessment is needed so that at-risk families can be better identified and the limited services available can be directed toward those most in need.

Introduction

Child welfare agencies received more than three million reports of child maltreatment in 2003 (U.S. DHHS, 2005). Among those reports that are substantiated, approximately half do not receive services beyond the initial investigation (U.S. DHHS, 2005). Given the substantial volume of reports, caseworkers’ ability to determine which cases are at greatest risk of subsequent maltreatment deserves attention. Research in this area, however, has demonstrated that even among those considered to be “experts” in the field of child welfare, decisions about risk vary significantly from worker to worker (Rossi, Schuerman, & Budde, 1996). Additional research is needed on how caseworkers make decisions about risk, particularly in the current climate in which dissemination of formal risk assessment measures presents a viable option for enhanced accuracy. These measures have received significant attention, have evidence for improving assessment, but at the same time, have drawn criticism (Gambrill & Shlonsky, 2000). Therefore, it is necessary both to examine how caseworkers utilize the existing knowledge about predictors of maltreatment recurrence and to examine the degree to which these factors relate to accuracy or inaccuracy of risk classification.

Despite inconsistencies in caseworkers’ decision-making, the literature is relatively consistent with regard to factors associated with maltreatment recurrence. Factors reviewed here include those at the family, parent, and child-level. In the area of family-level factors, larger numbers of children in the household (DePanfilis & Zuravin, 1999, 2002; Johnson & L'Esperance, 1984), domestic violence (Baird, Wagner, & Neuenfeldt, 1993), and low family income (Levy, Markovic, Chaudhry, Ahart, & Torres, 1995; Rittner, 2002) have been associated with the recurrence of child physical abuse and neglect. Parent-level factors associated with recurrence include single parent status (Sedlack & Broadhurst, 1996 [Third National Incidence study of Child Abuse and Neglect]), parent's own history of abuse (Gaudin & Dubowitz, 1997), substance abuse (Baird, 1988; English, Marshall, Coghlan, Brummel, & Orne, 1999), high levels of stress (Baird et al., 1993; DePanfilis & Zuravin, 2001), low levels of social support (Baird et al., 1993; DePanfilis & Zuravin, 1999) and mental health problems (Rittner, 2002; Sedlack & Broadhurst, 1996). There also is some evidence that parental ethnicity may be related to recurrence, with Asian/Pacific Islanders having the lowest rates of recurrence (Fluke, Yuan, & Edwards, 1999). Child-related factors include younger age (English et al., 1999; Fryer & Miyoshi, 1994), physical health problems, and/or vulnerability (DePanfilis & Zuravin, 1999, 2002). In addition to family, parent, and child-level factors, the type of maltreatment and having a prior report, or reports, of maltreatment have been associated with recurrence. Of all the factors, a prior report of maltreatment may have the most empirical support in predicting subsequent maltreatment (e.g., Baird, Wagner, & Neuenfeldt, 1993; Fuller, Wells, & Cotton, 2001). Among types of maltreatment, child neglect appears to recur more frequently than other types of maltreatment (e.g., Fluke, Yuan, & Edwards, 1999; Levy et al., 1995; Marks & McDonald, 1989). In addition, although studies often combine maltreatment types, research has indicated that different forms of maltreatment may have some distinct predictors (Marks & McDonald, 1989). For example, parents’ own history of maltreatment, low social support, and high family stress have been more strongly linked with recurrence of neglect (Depanfilis, 1996; Gaudin & Dubowitz, 1997).

Given this accumulated knowledge, over the past 20 years researchers have attempted to utilize these findings systematically to improve the accuracy and objectivity of caseworkers’ risk assessment (Children's Research Center, 1999; Johnson & L'Esperance, 1984). In large part, these efforts were guided by research on decision-making in other fields in which increased accuracy was demonstrated with the utilization of actuarial assessment tools. These risk assessment measures, which formulaically combine empirically supported risk factors, improve accuracy both by increasing reliance on factors from the empirical literature and by decreasing reliance on clinical judgment and intuition (Baird & Wagner, 2000; Grove & Meehl, 1996). Ideally, with the assistance of such tools, caseworkers can more quickly and efficiently assess and triage the growing number of reports to CPS, leaving more time and energy to be spent on identifying, acquiring, and/or providing appropriate intervention services to prevent future maltreatment.

Although actuarial models hold promise for improving assessment of risk, there have been a number of problems with their implementation. First, dissemination and adoption has been somewhat limited. Although the majority of CPS agencies in the United States have instituted some form of risk assessment system, few of them are actuarial (Baird & Wagner, 2000; Camasso & Jagannathon, 2000; Children's Research Center, 1999; Grove & Meehl, 1996). Second, research on utilization of actuarial model in CPS and in other fields (e.g., criminal justice) suggests that even when available, practitioners may not implement the models as intended or may override them entirely (Neil & Meehl, 1986; DePanfilis & Zuravin, 2001). As a result, although some states and agencies have experienced success with actuarial models, on the whole, a significant level of skepticism continues to exist (Munro, 1999; Wald & Woolverton, 1990) and “clinical prediction continues to thrive” (Gambrill & Schlonsky, 2000).

Given these difficulties with formal risk assessment models, it is important to assess the degree to which caseworkers utilize knowledge from the empirical literature on recurrence of child maltreatment, regardless of child welfare agencies’ chosen decision-making tool, particularly as caseworkers may not rely on it entirely. Gaining insight into caseworkers’ utilization of research findings can inform ongoing attempts to improve child welfare's identification of children and families at greatest risk of subsequent maltreatment. Although a sizeable body of research exists on different aspects of caseworker decision-making and risk assessment, the majority of these studies involve small samples and/or are from a single programmatic source or are geographically restricted, thus limiting the ability to generalize the findings. Data for the current study are from the National Survey for Child and Adolescent Well-Being (NSCAW; NSCAW Research Group, 2002), a nationally-representative sample of youth and families who were the subjects of maltreatment allegations investigated by child welfare agencies. The national scope and inclusion of a large representative sample address most of the methodological issues in prior research (e.g., geographically restricted samples, small samples). The current study focuses on assessment of risk for a subsample of children and adolescents involved with child welfare: those who remained in their homes following a report of maltreatment. The focus is on this group for three reasons. First, empirical studies of caseworkers’ decision-making for cases in which children remain with their families have been relatively limited. The majority of research attention has focused on factors associated with need for out-of-home placement (DePanfilis & Scannapieco, 1994). Second, although cases resulting in out-of-home placement often constitute the most serious incidences of maltreatment encountered by CPS (DePanfilis & Zuravin, 2001), case dispositions in which children remain with their families comprise the majority of reports to CPS (ACF; U.S. DHHS, 2005). Third, these latter cases represent those for whom ongoing or repeated maltreatment is a more probable occurrence, as children who remain in the home often also remain in the familial or environmental context in which the maltreatment originally occurred (Rittner, 2002). Therefore, from an intervention perspective, these cases are of particular interest in that improved assessment of risk will assist CPS in reaching their dual goals of keeping children safe and keeping families together.

The objective of the current longitudinal study was threefold. First, the association between caseworkers’ knowledge of risk factors and their assessments of risk of maltreatment recurrence was examined. The risk factors included were drawn from the literature outlined previously and included family, parent, and child-level factors, as well as type of maltreatment and prior report(s) of maltreatment. Second, “accuracy” of the caseworkers’ decision-making was examined. Accuracy was defined by the agreement between classification of risk for future maltreatment and any subsequent reports of maltreatment among those families categorized by caseworkers as low- and high-risk. Subsequent reports of maltreatment were examined at two follow-up interviews occurring approximately 12 and 18 months after the report that resulted in inclusion in the NSCAW sample (hereafter referred to as the index report). Finally, factors associated with “mistakes” in classification of risk were examined among families inaccurately classified by caseworkers as low-risk. This final purpose provided an opportunity to determine if any particular risk factor(s) were being overlooked or should have been given more attention during the decision-making process. According to Gambrill and Shlonsky (2001), identifying where decision-making biases and errors occur provides an opportunity to improve upon the “hit rate” of accurate identification of families and children most at risk.

In the present study, empirical factors associated with mistakes in classification were examined only among families inaccurately classified as low-risk. Examination of mistakes was restricted to this group for two reasons. First, cases classified as high-risk without reports of re-abuse are right-censored. That is, cases could have new reports of maltreatment following to the study's 18-month follow-up period, which would indicate that the cases were not inaccurately classified as high-risk. Second, caseworkers were asked about predictions of subsequent maltreatment reports if the family did not receive services. Although research suggests that rates of maltreatment recurrence are not necessarily lower among those families receiving services (Fryer & Miyoshi, 1994; Levy et al., 1995; Wood, 1997), the true rate of maltreatment recurrence, in the absence of intervention, cannot be determined as families in the high-risk group may have received services (Gambrill & Schlonsky, 2000).

Method

The National Survey for Child and Adolescent Well-Being (NSCAW) was designed by the Department of Health and Human Services to examine the relationships among child and family well-being, family characteristics, experience with the child welfare system, community environment and other factors (Dowd et al., 2001). Institutional review board (IRB) approval for the NSCAW study was obtained from Children's Hospital-San Diego. Informed consent was obtained from the caregiver and assent was obtained from the child. Approval for secondary analyses was obtained from the Duke University School of Medicine's IRB. The NSCAW main sample cohort included 5,501 children, aged birth to 14 at the time of sampling, who had contact with the child welfare system due to investigations/assessments for child abuse or neglect within a 15-month period beginning in October 1999. Two exclusionary criteria were applied in the current study due to the focus on families in which children remained in the home. First, families in which children were not residing in the home of their primary caregiver at the times of data collection were excluded (n = 2,351). Second, only children whose primary type of maltreatment was due to child physical abuse or neglect were included. Therefore, a total of 1,733 children were excluded whose primary type of abuse was sexual (n = 597), moral, legal, emotional, educational or other (n = 664), or where the type of abuse was unknown (n = 472). Given the overlap between out-of-home placement and type of abuse, after applying the above exclusionary criteria, the final sample includes 2,139 children.

Study design

The NSCAW design has been detailed extensively elsewhere and will be summarized here (Dowd et al., 2001; NSCAW Research Group, 2002). The sample was selected using a two-stage, stratified sample design. Children who had been alleged to have experienced maltreatment and had a completed CPS investigation were selected from 92 primary sampling units (PSUs) in 97 counties nationwide, with oversampling of infants, youth with reports of sexual abuse, and youth who were receiving ongoing child welfare services. Current analyses include data from the baseline interview (Wave 1), a follow-up interview that took place approximately 12 months later (Wave 2), and a second follow-up interview that took place approximately 18 months post-baseline (Wave 3).

Weights

The NSCAW sample was weighted to account for differential selection probabilities. The probability weights were constructed in stages corresponding to the stages of the sample-design, with adjustments made for nonresponse and undercoverage (e.g., in some sites, unsubstantiated cases were excluded due to privacy and confidentiality laws; in others, cases were omitted if the children were not receiving services at baseline). All analyses were weighted in order to account for the sampling design.

Measures

Demographic characteristics

Both parent and family demographic characteristics were examined. Demographic variables included: child's age and gender; primary caregiver's age, ethnicity, and marital status; poverty status; and household size. Poverty status was created using a variable for household income and Census Bureau poverty figures for the year in which the interview was completed.

Risk factors

During the baseline interview, the interviewer asked the child welfare caseworker to identify risk factors that existed at the time of the investigation and played a role in their decision about the case. At the beginning of the risk assessment section, the interviewer provided the following introduction: “The next questions are about the primary caregiver's strengths and impairments.” The interviewer then proceeded to ask specifically about 16 risk factors (e.g., “At the time of the investigation, was there active drug use by the caregiver?;” “At the time of the investigation, did the caregiver have poor parenting skills, such as failure to supervise or monitor children routinely or harsh discipline?”). Seven of the 16 risk factors were examined in this study: a) history of own childhood abuse or neglect, b) drug or alcohol use, c) serious mental health problems, d) poor parenting skills, e) presence of domestic violence, f) high stress, and g) low social support.

Maltreatment

: All variables pertaining to maltreatment were taken from the interview with the caseworker. At baseline, child welfare caseworkers identified the types of alleged maltreatment using a modified Maltreatment Classification Scale (Manly, Cicchetti, & Barnett, 1994). The caseworker was asked to indicate the most severe type of maltreatment. The most severe type was used to categorize cases by type of maltreatment. Categories for child physical abuse and both types of neglect—failure to provide and failure to supervise/abandonment—are included in these analyses.

Additionally, variables were included for prior report of maltreatment, caseworkers’ assessments of risk, and any report of maltreatment subsequent to the index report. For prior report of maltreatment, caseworkers were asked during the baseline interview if any prior maltreatment reports had been made to their agency. This variable represents an aggregate variable that includes maltreatment reports of any type. For the likelihood of maltreatment recurrence, caseworkers were asked at the baseline interview to predict the likelihood of maltreatment recurrence (i.e., very low, low, high, and very high) in the next 24 months if no services were provided to the family. Based on significance tests (e.g., ANOVA, Chi Square analyses, etc.) suggesting that the two “low” responses and the two “high” responses did not differ significantly from one another on independent variables in this study, response choices were collapsed into two categories: low and high risk for the likelihood of maltreatment recurrence. Finally, the caseworkers were asked at both follow-up interviews (wave 2 and wave 3) about any new reports of maltreatment subsequent to the index report. As with prior report, this variable also is an aggregate of reports of any type of maltreatment. Caseworkers’ responses of a report at either follow-up interview were combined to create a single variable that reflects a new report of maltreatment after the index report.

Missing data

Missing data on most variables were minimal; however, 9-12% of data were missing for most of the parental risk factors. Rates of missing data did not differ significantly by any other variable in the analysis. One variable, parental history of childhood abuse or neglect, was missing on 16.9% of cases and was imputed using a stochastic imputation procedure prior to conducting logistic analyses (Little & Rubin 1987). All models were re-run using listwise deletion, and there were no appreciable difference in the results.

Data analysis

Due to the complex survey design involving stratification, PSUs, and probability weighting, all analyses were run using Stata 9.0 survey commands (StataCorp., 2005). The sampling design was appropriately accounted for and all percentages and means presented are weighted. Bivariate analyses were based on a Pearson χ2 statistic converted to an F-statistic with noninteger degrees of freedom using a second-order Rao and Scott (1981, 1984) correction (StataCorp., 2005). Multivariable analyses were conducted using logistic regression for survey data.

Three sets of analyses were conducted. First, descriptive and demographic characteristics were examined for the entire sample and for different combinations of caseworker predicted likelihood of risk of maltreatment recurrence (low vs. high) and subsequent reports of maltreatment (yes vs. no). Second, using multiple logistic regression, risk factors associated with caseworkers’ assessments of risk and subsequent reports were examined. In addition, factors that predicted subsequent reports of maltreatment among the group classified as low risk were examined in an effort to identify any factors that caseworkers might have overlooked during the risk assessment process. Finally, caseworker accuracy of risk assessment was examined using two sets of indicators: (1) sensitivity and specificity and (2) positive and negative predictive values. Recommended standards for psychosocial measures suggest 70-80% sensitivity and specificity to detect individuals with and without the symptom/disorder (American Academy of Pediatrics, 2001; Glascoe, 2000). The accuracy measures should be interpreted with caution; however, as they may underestimate effects due to right-censoring.

Results

Descriptive and demographic characteristics

Table 1 presents descriptive and demographic statistics on the child, parent, and family-level factors for the total NSCAW sample included in this analysis (column 1) and for the four groups created by categorizing the sample by caseworkers’ assessment of risk and any subsequent report(s) of maltreatment (columns 2-5). For the entire sample, the child's average age was 6.8 and the ethnicity of half of the sample was White. Caseworkers reported that approximately one-fifth of parents had a personal history of childhood abuse, one-third were estimated to have poor parenting skills, and about one-half had prior reports of maltreatment. Caseworkers assessed approximately one-third of the cases as having a high likelihood of maltreatment recurrence without receipt of services. By the third wave of data collection (i.e., 18 months), one-fifth of the cases in the entire sample had received subsequent reports of maltreatment. The percentage of cases classified as low risk with a recurrent report of maltreatment was 16.1 (see Table 1, column 3). The percentage of cases classified as high risk with a subsequent report or maltreatment was 28.1 (see Table 1, column 5).

Table 1.

Descriptive Statistics and Characteristics by Risk Status (Low, High) and a Subsequent Report, Waves 1-3 of NSCAW Subsample* (unweighted N = 2,139)

Subsequent report of maltreatment (yes, no)
Variable Total Low-no Low-yes High-no High-yes
weighted % or Mean (s.e.)
Total 83.9 16.1 71.9 28.1
Child gender - male 53.9 54.0 64.5 50.9 47.4
Child age 6.8 (0.18) 6.8 (0.22) 6.5 (0.43) 7.0 (0.28) 7.1 (0.40)
Parent race/ethnicity
    White 50.7 50.4 40.1 55 53.8
    Black 25.2 25.9 27.5 25.2 19.9
    Hispanic 16.2 16.7 19.6 13.7 17.1
    Other 7.9 7.0 12.9 6.1 9.2
Parent's age
    <35 68.3 68.1 68 66.1 72.5
    35-44 25.2 23.9 25.5 30.3 23.8
    45+ 6.5 8 6.5 3.6 3.7
Parent married 29.2 31.1 14.7 26.6 38.2
Number in household 4.5 (0.06) 4.4 4.4 4.7 4.6
Poverty 52 46.8 53.4 56.8 65.0
Parent risk factors
    History of abuse 20.2 11.2 23.8 31.9 45.4
    Drug or alcohol abuse 8.9 3.9 1.7 20.2 18.7
    Mental health problems 13.1 6.2 15.0 26.9 20.0
    Poor parenting 27.4 14.5 8.4 56.5 49.7
    Domestic violence 9.2 7.5 4.5 12.9 15.7
    High family stress 49.7 37.5 47.5 72.0 71.2
    Low social support 27.4 17.4 44.0 24.3 51.1
Prior report of maltreatment 48.9 33.6 56.1 68.2 85.7
Most serious type of maltreatment
    Physical 37.9 42.0 35.7 35.6 29.1
    Neglect - failure to provide 26 21.3 29.0 31.9 31.7
    Neglect - failure to supervise, abandonment 36.1 36.7 35.2 32.5 39.2
High likelihood of maltreatment recurrence 31.5
Subsequent report of maltreatment 20.2
Open in a new tab

Note: The parental risk factors, number in household, and prior report of abuse differ significantly by category at p<.01

*

NSCAW subsample includes children who remained in the home after the index report, and for whom the primary type of maltreatment was physical abuse, failure to provide, failure to supervise, or abandonment

Column two of Table 1 depicts families “correctly” classified as at low risk of a subsequent report of maltreatment; column three represents cases that were “incorrectly” classified as at low risk; column four represents the families the caseworker “incorrectly” classified as at high risk; and column five represents families “correctly” classified as high risk. There were no significant differences across the four groups for child-level demographics, parent-level demographics, poverty, or type of abuse. Significant differences existed across the four groups for number in household, all of the parental risk factors, and prior report of maltreatment (p < .01).

Some patterns emerged when looking across the four groups in columns two through four. Caregiver's own history of childhood maltreatment and prior reports of maltreatment were increasingly prevalent across the low-no, low-yes, high-no, and high-yes groups. Parental drug and alcohol abuse were significantly more prevalent among cases assessed as high-risk (i.e., the high-no and high-yes categories) and significantly less prevalent in families rated as low-risk of maltreatment recurrence (i.e., low-no and low-yes categories). Parental mental health problems, overall, were more prevalent in the high-risk categories. With regard to high family stress, caseworkers were more likely to rate those with high family stress as being more at risk of maltreatment recurrence; however, cases assessed as high-risk with a subsequent report did not differ on stress from those without a new report. As would be expected, poor parenting was over-represented among families rated as high-risk, compared to both those rated as low-risk and the total sample. In terms of social support, families with low social support were far more likely to be in the subsequent report categories (low-yes and high-yes), regardless of caseworker determination. Finally, in the area of domestic violence, caseworkers appeared to rate those with a history of domestic violence as being at higher risk of maltreatment recurrence.

Factors associated with caseworker assessment of risk of maltreatment recurrence

Table 2 shows the sociodemographic variables and risk factors associated with three outcomes: 1) Caseworkers’ assessments of high risk of maltreatment recurrence (columns 1 - 4); 2) Subsequent reports of maltreatment after the index report (columns 5 - 8); and 3) Subsequent reports of maltreatment among cases assessed as low risk (columns 9 - 12). The first eight columns of Table 2 include the total sample (unweighted N = 2,139); the last four columns include only those cases assessed by caseworkers as low risk, with a new report of maltreatment subsequent to the index report (16.1% of the weighted sample). For each outcome, the sample was examined separately by type of maltreatment. Child demographics were not significantly associated with caseworker assessments of risk for either type of maltreatment, controlling for other variables. Similarly, parent-level demographics, with the exception of parent's ethnicity and marital status for physical abuse cases, were not significantly associated with caseworkers’ assessments of risk.

Table 2.

Multiple Logistic Regression: Factors Related to Caseworker Assessments of Risk and Susequent Reports of Maltreatment

Assessed as high likelihood of maltreatment recurrence Subsequent report of maltreatment after the index report Subsequent report of maltreatment among those assessed as at low risk
Physical Abuse Neglect Physical Abuse Neglect Physical Abuse Neglect
OR SE OR SE OR SE OR SE OR SE OR SE
Child's gender 0.66 0.25 1.27 0.45 1.02 0.43 1.04 0.33 1.18 0.73 0.89 0.34
Child's age 0.96 0.05 1.03 0.04 0.99 0.06 0.99 0.03 1.01 0.08 0.96 0.04
Parent's race/ethnicity
        Black 1.01 0.52 0.53 0.17 0.79 0.49 1.26 0.41 1.13 0.91 1.15 0.44
        Hispanic 0.91 0.42 1.47 0.96 1.68 0.91 2.07 0.94 1.15 0.87 1.96 1.06
        Other 0.21 0.14* 1.33 0.62 1.88 1.52 2.36 1.41 2.57 2.46 1.99 1.26
Parent's age
        35-44 0.97 0.46 0.60 0.22 1.50 0.82 0.72 0.27 1.33 1.17 0.81 0.39
        45+ 0.49 0.50 0.33 0.19 2.07 1.74 0.44 0.24 1.77 1.69 0.44 0.32
Parent married 2.29 0.95* 1.48 0.54 0.74 0.35 0.93 0.34 0.65 0.40 0.84 0.34
Poverty 0.53 0.21 1.21 0.38 1.23 0.62 0.95 0.30 1.59 1.22 1.06 0.37
Number in household 0.86 0.10 0.96 0.09 1.13 0.12 0.92 0.08 1.16 0.19 0.94 0.10
Parent risk factors
        History of abuse 1.23 0.57 1.59 0.59 3.36 1.55** 1.17 0.41 2.60 2.11 1.06 0.47
        Drug or alcohol abuse 2.04 1.30 3.67 1.32** 1.13 0.66 0.75 0.26 1.29 1.31 0.70 0.30
    Mental health problems 6.10 4.84* 1.02 0.43 0.79 0.45 1.45 0.72 0.88 1.08 1.86 1.10
        Poor parenting 4.02 1.83** 7.29 2.35** 0.69 0.34 0.79 0.29 1.10 0.85 0.48 0.24
        Domestic violence 1.61 0.74 1.38 0.50 1.22 0.84 0.63 0.23 0.57 0.73 0.33 0.18*
        High family stress 5.07 2.28** 1.85 0.62 1.23 0.56 1.12 0.37 1.03 0.66 1.40 0.51
        Low social support 1.27 0.57 3.10 0.91** 1.20 0.54 1.93 0.60* 0.60 0.50 2.44 0.95*
Prior report of maltreatment 3.73 1.30** 7.30 2.31** 2.44 1.07* 2.56 0.79** 3.49 1.99* 1.90 0.67
Open in a new tab
*

p≤.05

**

p≤.01

Factors associated with caseworker's classification of risk predominantly were at the parent and family level. For physical abuse cases, parental mental health problems (OR = 6.10, p < .05), high family stress (OR = 5.07, p < .01), poor parenting (OR = 4.02, p < .01), and prior report of maltreatment (OR = 3.73, p < .01) were associated with high risk assessments. For neglect cases, parental drug or alcohol abuse (OR = 3.67, p < .01), low social support (OR = 3.10, p < .01), poor parenting (OR = 7.29 p < .01), and prior report of maltreatment (OR = 7.30, p < .01) were associated with high risk assessments.

Factors associated with a subsequent report of maltreatment

The fifth through the eighth columns of Table 2 depict analyses examining the sociodemographic and family risk factors associated with a subsequent report, or reports, of maltreatment, after the index report. Few factors were associated with a subsequent report. For cases involving physical abuse, the caseworkers’ report of parents’ having their own history of maltreatment predicted a subsequent report (OR = 3.36, p < .01), as did having a prior report of maltreatment (OR = 2.44, p < .05), controlling for other variables. Significant factors predicting a subsequent report in neglect cases included low social support (OR = 1.93, p < .05) and, as with physical abuse, having a prior report of maltreatment (OR = 2.56, p < .01).

Factors associated with a subsequent report among cases classified as low risk

The final four columns (columns 9 – 12) in Table 2 display results of analyses examining the factors associated with a subsequent report of maltreatment among families classified by caseworkers as low-risk (16.1% of the total weighted sample). For both physical abuse and neglect, neither parent nor child sociodemographic variables were significantly associated with a subsequent report of maltreatment, controlling for other variables. Only one parental risk factor, having a prior report of maltreatment (OR = 3.49, p < .05), was associated with a subsequent report of maltreatment among low-risk physical abuse cases. For low-risk neglect cases, low social support (OR = 2.44, p < .05) was associated with having a subsequent report. Domestic violence, for low-risk neglect cases, was associated with not having a subsequent report (OR = .33, p < .05).

Agreement between caseworkers’ assessments of risk and recurrent reports of maltreatment

Table 3 displays the actual and predicted outcomes for subsequent reports of maltreatment (unweighted ns). Sensitivity (identifying cases with subsequent reports) was calculated as 44% and specificity (identifying cases with no subsequent reports) as 72%. The positive predictive value, the probability that a case assessed as high risk will have a recurrent report, was calculated as 28.1% and the negative predictive value, the probability that a case assessed as high risk will not have a recurrent report, was calculated as 83.9%.

Table 3.

Classification of caseworker predicted risk of maltreatment recurrence by any subsequent reports

Subsequent Report
Yes No
Caseworker-Predicted Risk High 9.1% 23.2%
Low 10.9% 56.7%
Open in a new tab

To test the possibility that receipt of services might prevent maltreatment recurrence and therefore may account for some of the disagreement between caseworker assessment of risk and re-reports, separate models of predictors of subsequent reports were analyzed that included services to the parent as a covariate and as a moderator. Parents’ receipt of services did not significantly affect the results and therefore the results are not shown (analyses are available from the first author upon request).

Discussion

Caseworkers’ assessments of risk for subsequent maltreatment predominantly were associated with parent-level risk factors, which varied across physical abuse and neglect cases. These factors overlapped minimally with a smaller set of factors that predicted actual subsequent maltreatment reports. Caseworkers’ assessments of risk were more accurate for low risk than for high risk cases.

Association between caseworkers’ risk assessment and empirical risk factors

Caseworkers’ assessments of risk were based predominantly on a constellation of parent-level risk factors that, to some degree, overlapped across physical abuse and neglect cases. Parental risk factors associated with high-risk classification for both maltreatment types included caseworker perception of poor parenting skills and caseworkers’ knowledge of a prior report of maltreatment. These findings suggest that caseworkers may have appropriately prioritized the importance of past reports, one of the risk factors with the most empirical support (e.g., Baird et al., 1993; Sedlack & Broadhurst, 1996). Parent-level risk factors unique to maltreatment type included mental illness and high family stress for physical abuse cases and drug or alcohol abuse for neglect cases. These findings, in which the constellation of factors associated with high-risk classification varied by maltreatment type, reflect previous research indicating that specific types of maltreatment have some distinct predictors (e.g., Marks & McDonald, 1989).

In addition to parental risk factors, two parent-level demographic characteristics—ethnicity and marital status—were associated with classification of risk. Specifically, caseworkers were less likely to assess cases with parental ethnicity listed as ‘other’ (i.e., Asian American, Pacific Islanders) as high risk, which mirrors findings in the literature in which Asian Americans and Pacific Islanders have had lower rates of maltreatment recurrence (e.g., Fluke et al., 1999). Caseworkers were more likely to classify cases with a married caregiver as high risk. This finding is divergent from prior research findings, in which married caregiver status often has been associated with lower, rather than higher, rates of maltreatment recurrence (Sedlack & Broadhurst, 1996). Previous research also has cited a child's younger age, lower family income, and larger households as risk factors for maltreatment recurrence; however, neither child nor family-level demographics were related to caseworkers’ assessments of risk (Fluke et al., 1999; Johnson & L'Esperance, 1984; Sedlack & Broadhurst, 1996). Surprisingly, domestic violence was not related to the caseworkers’ ratings of risk of either physical abuse or neglect cases. Potentially, caseworkers gave more weight to parent-level risk factors that they viewed as more proximally related to parent and child interactions (e.g., poor parenting skills, parental mental health problems).

Factors associated with an actual subsequent report of maltreatment

A smaller number of parent-level risk factors and having had a prior report of maltreatment predicted a subsequent report of maltreatment. Parent-level factors included parents’ own history of maltreatment for physical abuse cases and low social support for neglect cases. Comparing these predictors to those associated with caseworkers’ assessments of risk, only having a prior report of maltreatment predicted both. None of the child, parent, or family-level demographic characteristics examined predicted an actual subsequent report.

Agreement between caseworkers’ classification of risk and a subsequent report

The accuracy of caseworkers’ assessments of risk appears to be low in the area of identifying high-risk cases. Sensitivity was calculated as 44% and the positive predictive value was calculated as 28%. These values are outside the recommended acceptable range (70-80%) for valid psychological measures designed to identify individuals with a particular symptom or disorder (American Academy of Pediatrics, 2001; Glascoe, 2000). The findings seem to suggest that when unsure, caseworkers may have erred on the side of overestimating risk. Both specificity and the negative predictive value are at more acceptable levels (71% & 83.9%. respectively), but these values may capitalize on the low base rate of subsequent reports of maltreatment at the time of follow-up (20% for the entire sample; see Table 1, last row).

Despite better accuracy for cases assessed as low risk, 16% of these cases had a subsequent report of child maltreatment. Analyses indicated that two of the factors associated with a subsequent report of maltreatment within this subgroup (i.e., cases assessed as low risk) also were associated with caseworkers’ classifications of high risk for the entire sample. These factors included prior report of maltreatment for physical abuse cases and low social support for neglect cases. Potentially, these factors required greater attention during the risk assessment process. Examination of the bivariate relationships (presented in Table 1, columns 2-5) further underlines this point. Looking within the cases classified as low risk, prior reports of maltreatment and low social support were significantly more prevalent among those with a subsequent report of maltreatment. Given that determining whether there are any prior reports of maltreatment likely is included in almost any formal or informal measure of risk, this factor may be particularly important in influencing caseworkers’ assessments of risk. Therefore, cases with prior reports that were assessed as low risk likely had mitigating factors that influenced the caseworker's assessment. Although it would be inappropriate to recommend that caseworkers unilaterally classify all cases with these risk factors (i.e., prior report, low social support) as high risk, it appears that the importance of these factors, and their association with likelihood of a subsequent maltreatment report even among cases initially viewed as low-risk, may need to be highlighted.

Finally, among cases classified as low risk, the presence of domestic violence was unexpectedly associated with a lower likelihood of an actual subsequent report of maltreatment for neglect cases classified as at low-risk. This finding, however, must be considered within the context of other analyses which indicate that overall (see bivariate relationships in Table 1, columns 2-5), caseworkers were more likely to classify cases involving domestic violence as high risk.

Limitations and strengths

Although utilization of a nationally representative sample addressed some of the methodological issues impacting previous research on caseworker decision-making, a number of limitations exist. First, in order to avoid having findings affected by variations in state and agency policies on substantiation of child maltreatment (Drake, 1996), the NSCAW sample includes all cases with reports of maltreatment, and not just those with substantiated reports. Examining all reports to child protective services may be viewed as a limitation; however, using only substantiated reports also has its own limitations, as the absence of a report or substantiation does not necessarily equal the absence of maltreatment. Second, information regarding prior history of maltreatment and subsequent reports of maltreatment were obtained from caseworkers who may only have been able to give an account of reports made to their own agencies. Therefore, the number of cases with prior reports, as well as the number of cases with a subsequent report, may have been underestimated. Third, as already mentioned, risk factors were based solely on caseworker's reports due to the study's focus on caseworker's knowledge of risk factors and the association with risk assessment. Finally, although it is beyond the scope of this study, it is possible that caseworkers’ assessments of risk for subsequent maltreatment may be more accurate indicators of types of risk, such as children's likelihood of experiencing poor developmental or psychosocial outcomes. Future research should examine the caseworker's ability to assess child developmental or psychosocial risk, as the child's involvement with child welfare may provide a window of opportunity for children to be identified and, regardless of the child's risk of future maltreatment, receive needed services.

The current study has a number of strengths and implications for caseworker decision-making. First, it appears that caseworkers’ assessments of risk are associated with a number of the parent-level risk factors from the literature and that the risk factor with the most empirical support, prior reports of abuse, was associated with both classification of risk and subsequent reports regardless of maltreatment type. However, the overall findings suggest a complex picture of risk assessment in which there are few patterns of risk factors (other than prior reports) that consistently are associated with caseworker classification of risk and subsequent reports.

Furthermore, agreement between caseworkers’ assessments of risk and subsequent reports of maltreatment is relatively low, and caseworkers appear likely to overestimate risk. Although overestimating risk may be understandable given the safety and legal ramifications of underestimating risk, this study concentrates on those cases for which a range of child, parent, and family services are often offered or mandated in an effort to prevent any recurrent maltreatment. Therefore, from an intervention standpoint, correct identification and classification of those most at risk enables limited resources to be utilized more effectively. Given the significance of mistakes in classification—child safety when cases are inappropriately classified as low-risk, and cost expenditures on services for families who are not truly high-risk—increasing caseworker accuracy of classification is critical. These findings suggest that one pathway to improving accuracy of assessment may be to improve caseworkers’ utilization of findings from the research literature, most likely through training and adoption of actuarial risk assessment measures.

Acknowledgments

Work on this project was funded by the NIMH, Caring for Children in Child Welfare project (CCCW) (MH59672) and the Child and Adolescnet Intervention Research Network (CAIRN: R24 MH067377). Data come from the National Survey of Child and Adolescent Well Being (NSCWA), which was developed under contract with the Administration on Children and Families, US Department of Health and Human Services.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. American Academy of Pediatrics, Committee on Children with Disabilities Developmental surveillance and screening of infants and young children. Pediatrics. 2001;108:192–196. [Google Scholar]
  2. Baird S. Development of risk assessment indices for the Alaska Department of Health and Social Services. In: Tatara T, editor. Validation research in CPS risk assessment: Three recent studies. American Public Welfare Association; Washington, DC: 1988. Occasional Monograph Series No. 2. [Google Scholar]
  3. Baird S, Wagner D. The relative validity of actuarial- and consensus-based risk assessment systems. Children and Youth Services Review. 2000;22:839–871. [Google Scholar]
  4. Baird S, Wagner D, Neuenfeldt D. Actual risk assessment and case management in child protective services. In: Tatara T, editor. Sixth National Roundtable on CPS Risk Assessment Summary of Highlights. American Public Welfare Association; Washington, DC: 1993. pp. 152–168. [Google Scholar]
  5. Camasso M, Jagannathan R. Modeling the reliability and predictive validity of risk assessment in child welfare. Children and Youth Services Review. 2000;22:873–896. [Google Scholar]
  6. Children's Research Center . A new approach to child protective services; structured decision-making. National Council on Crime and Delinquency, Children's Research Center; Madison, WI: 1999. [Google Scholar]
  7. DePanfilis D. Social isolation of neglectful families: A review of social support assessment and intervention models. Child Maltreatment. 1996;1:37–52. [Google Scholar]
  8. DePanfilis D, Scannapieco M. Assessing the safety of children at risk of maltreatment: Decision-making models. Child Welfare. 1994;73:229–246. [PubMed] [Google Scholar]
  9. DePanfilis D, Zuravin S. Predicting child maltreatment recurrences during treatment. Child Abuse & Neglect. 1999;23:729–743. doi: 10.1016/s0145-2134(99)00046-0. [DOI] [PubMed] [Google Scholar]
  10. DePanfilis D, Zuravin S. Assessing risk to determine the need for services. Children and Youth Services Review. 2001;23:3–20. [Google Scholar]
  11. DePanfilis D, Zuravin S. The effect of services on the recurrence of child maltreatment. Child Abuse & Neglect. 2002;26:187–205. doi: 10.1016/s0145-2134(01)00316-7. [DOI] [PubMed] [Google Scholar]
  12. Drake B. Unraveling “unsubstantiated.”. Child Maltreatment. 1996;1:261–271. [Google Scholar]
  13. Dowd K, Kinsey S, Wheeless S, Thissen R, Richardson J, Suresh R, Mierzwa F, Biemer P, Johnson I, Lytle T. National Survey of Child and Adolescent Well-Being (NSCAW): Wave 1: Data File User's Manual. National Data Archive on Child Abuse and Neglect; Ithaca, NY: 2001. [Google Scholar]
  14. English D, Marshall D, Coghlan L, Brummel S, Orne M. Causes and consequences of the substantiation decision in Washington state child protective services. Children and Youth Services Review. 1999;21:1–23. [Google Scholar]
  15. Fluke J, Yuan Y, Edwards M. Recurrence of maltreatment: An application of the National Child Abuse and Neglect Data System (NCANDS). Child Abuse & Neglect. 1999;23:633–650. doi: 10.1016/s0145-2134(99)00039-3. [DOI] [PubMed] [Google Scholar]
  16. Fryer G, Miyoshi T. A survival analysis of the revictimization of children: The case of Colorado. Child Abuse & Neglect. 1994;18:1063–1071. doi: 10.1016/0145-2134(94)90132-5. [DOI] [PubMed] [Google Scholar]
  17. Fuller T, Wells S, Cotton E. Predictors of maltreatment recurrence at two milestones in the life of a case. Children and Youth Services Review. 2001;23:49–78. [Google Scholar]
  18. Gambrill E, Shlonsky A. Risk assessment in context. Children and Youth Services Review. 2000;22:813–837. [Google Scholar]
  19. Gambrill E, Shlonsky A. The need for comprehensive risk management systems in child welfare. Children and Youth Services Review. 2001;23:79–107. [Google Scholar]
  20. Gaudin JM, Dubowitz H. Family functioning in neglectful families: Recent research. In: Berrick JD, Barth RP, Gilbert N, editors. Child welfare research review: Volume # II. Columbia University Press; New York: 1997. pp. 28–62. [Google Scholar]
  21. Glascoe Evidence-based approach to developmental and behavioral surveillance using parents’ concerns. Child Care Health and Development. 2000;26:137–149. doi: 10.1046/j.1365-2214.2000.00173.x. [DOI] [PubMed] [Google Scholar]
  22. Grove W, Meehl P. Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical controversy. Psychology, Public Policy, and Law. 1996;2:293–323. [Google Scholar]
  23. Johnson W, L'Esperance J. Predicting the recurrence of child abuse. Social Work Research and Abstracts. 1984;29:21–26. [Google Scholar]
  24. Levy H, Markovic J, Chaudhry U, Ahart S, Torres H. Reabuse rates in a sample of children followed for 5 years after discharge from a child abuse inpatient assessment program. Child Abuse & Neglect. 1995;19:1363–1377. doi: 10.1016/0145-2134(95)00095-p. [DOI] [PubMed] [Google Scholar]
  25. Little RJA, Rubin DB. Statistical Analysis with Missing Data. John Wiley and Sons; New York: 1987. [Google Scholar]
  26. Manly J, Cicchetti D, Barnett D. The impact of subtype, frequency, chronicity, and severity of child maltreatment on social competence and behavior problems. Developmental Psychopathology. 1994;6:121–143. [Google Scholar]
  27. Marks J, McDonald T. Predicting recurrence of child maltreatment. National Child Welfare Resource Center for Management and Administration, University of Southern Maine; Portland, ME: 1989. [Google Scholar]
  28. Munro E. Common errors of reasoning in child protection work. Child Abuse and Neglect. 1999;23:745–758. doi: 10.1016/s0145-2134(99)00053-8. [DOI] [PubMed] [Google Scholar]
  29. NSCAW Research Group Methodological lessons from the National Survey of Child and Adolescent Well-Being: The first three years of the USA's first national probability study of children and families investigated for abuse and neglect. Child Youth Services Review. 2002;24:513–541. [Google Scholar]
  30. Rao J, Scott A. The analysis of categorical data from complex sample surveys: Chi-squared tests for goodness of fit and independence in two-way tables. Journal of the American Statistical Association. 1981;76:221–230. [Google Scholar]
  31. Rao J, Scott A. On chi-squared tests for multiway contingency tables with cell proportions estimated from survey data. The Annals of Statistics. 1984;12:46–60. [Google Scholar]
  32. Rittner B. The use of risk assessment instruments in child protective services case planning and closures. Children and Youth Services Review. 2002;24:189–207. [Google Scholar]
  33. Rossi P, Schuerman J, Budde S. Understanding child maltreatment decisions and those who make them. Chapin Hall Center for Children, University of Chicago; 1996. [Google Scholar]
  34. Sedlack AJ, Broadhurst DD. Executive summary of the third incidence study of child abuse and neglect. DHHS, Administration for Children and Families, NCCAN; Washington, DC: 1996. [Google Scholar]
  35. StataCorp . Stata Statistical Software: Release 9. StataCorp LP; College Station, TX: 2005. [Google Scholar]
  36. U.S. Department of Health and Human Services Administration (U.S. DHHS) Administration for Children and Families . Child Maltreatment 2003. U.S. Government Printing Office; Washington, DC: 2005. [Google Scholar]
  37. Wald M, Woolverton M. Risk Assessment: The emperor's new clothes? Child Welfare. 1990;69:483–511. [PubMed] [Google Scholar]
  38. Wood J. Risk predictors for re-abuse and re-neglect in a predominantly Hispanic population. Child Abuse & Neglect. 1997;21:379–389. doi: 10.1016/s0145-2134(96)00178-0. [DOI] [PubMed] [Google Scholar]

RESOURCES