Benefit-Cost Analysis of Promoting First Relationships®: Implications of Victim Benefits Assumptions for Return on Investment

Abstract

Background.

Child abuse and neglect (CAN) cost United States society $136 billion to $428 billion annually. Preventive interventions that reduce CAN may improve people’s lives and generate economic benefits to society, but their magnitude is likely to vary greatly with assumptions about victim costs avoided through intervention.

Objective.

We examined the implications of different assumptions about avoided victim costs in a benefit-cost analysis of Promoting First Relationships® (PFR), a 10-session attachment and strengths-based home visiting intervention.

Participants and Setting.

Participants were 247 child protection-involved but intact families in Washington State randomized to receive PFR (n = 124) or resource and referral (n = 123).

Methods.

We monetized intervention effects on out-of-home placements and implicit effects on CAN and calculated net present values under three scenarios: (1) benefits from avoided system costs, (2) additional benefits from avoided tangible victim costs, and (3) additional benefits from avoided tangible and intangible quality-of-life victim costs. For scenarios 2 and 3, we varied the CAN effect size and estimated the effect size at which PFR was reliably cost beneficial.

Results.

PFR’s societal net benefit ranged from $1 (scenario 1) to $5,514 - $25,562 (scenario 2) and $7,004 - $32,072 (scenario 3) (2014 USD). In scenarios 2 and 3, PFR was reliably cost beneficial at a CAN effect size of approximately −.25.

Conclusions.

PFR is cost beneficial assuming tangible victim costs are avoided by PFR. Research into the long-term health and economic consequences of reducing CAN in at-risk populations would contribute to comprehensive, accurate benefits models.

Children under the age of 3 years represent a disproportionate share of the 676,000 victims of child abuse and neglect (CAN) each year (U.S. Department of Health & Human Services, 2018), accounting for 28% of all victimizations and 70% of all fatalities. Victimization rates among infants are particularly high, at 24.1 per 1,000 children, declining to 11.9 and 11.2 for those ages 1 and 2, respectively, compared to 9.1 victimizations per 1,000 children ages 17 or younger. Child victims of physical abuse, sexual abuse, psychological abuse, and/or neglect have been shown to experience a variety of lifelong adverse sequelae, including short- and long-term physical and mental health problems, delinquency and criminal behavior, reduced health-related quality of life, lower productivity and workforce participation, and reduced economic well-being (Afifi et al., 2016; Cicchetti, Hetzel, Rogosch, Handley, & Toth, 2016; Corso, Edwards, Fang, & Mercy, 2008; Currie & Spatz Widom, 2010; Fang & Corso, 2007; Jung, Herrenkohl, Klika, Lee, & Brown, 2015; Monnat & Chandler, 2015; Norman et al., 2012; Widom, Czaja, Bentley, & Johnson, 2012). Recent estimates of the annual costs to society from child abuse and neglect are high, ranging from $124 billion (2008 USD, or $136 billion in 2015 USD; Fang, Brown, Florence, & Mercy, 2012) to $428 billion (2015 USD; Peterson, Florence, & Klevens, 2018) for substantiated child maltreatment. Interventions that prevent CAN have the potential to transform lives and also yield economic benefits to victims and society (Prinz, 2016; Sanders, Higgins, & Prinz, 2018). This study reports findings from a benefit-cost analysis (BCA) of Promoting First Relationships®, a home-visiting intervention delivered to child protective services (CPS)-involved families of infants and toddlers in a randomized controlled trial (RCT) known as the Supporting Parents Project (SPP). Specifically, the study examines whether long-term avoided costs and increased revenues attributable to PFR exceed the cost of delivering the intervention, after both have been adjusted for inflation and discounted back to intervention baseline.

Promoting First Relationships (PFR)

PFR is an attachment- and strengths-based home-visiting intervention for caregivers of children from birth to age five. The intervention aims to increase caregiver sensitivity and responsiveness to children’s behavioral and emotional needs, ultimately to strengthen the developing child’s attachment security, promote healthy development, and prevent problems like child maltreatment (Kelly, Zuckerman, & Rosenblatt, 2008; Oxford, Spieker, Lohr, & Fleming, 2016; Spieker, Oxford, Kelly, Nelson, & Fleming, 2012). In a series of 10 weekly sessions that last 60 - 75 minutes, trained interventionists work with caregivers of children at increased risk for adverse outcomes, including CAN, to increase their use of nurturing and effective caregiving strategies that improve the likelihood of favorable developmental outcomes. Over multiple sessions with caregivers, interventionists use reflective practice principles and joint observation of videotaped feedback to increase caregiver capacity to understand their child’s nonverbal communication. They also help caregivers reframe their child’s challenging behavior in terms of underlying social and emotional needs and to empathize with and provide comfort when their child is distressed. Caregivers also learn to reflect on their own needs and feelings as parents and understand how these feelings affect their parenting behavior. Because change is predicated on caregivers feeling safe and secure in their relationship with service providers, PFR interventionists work from a strengths-based perspective, recognizing caregiver competencies, and taking care to avoid inducing guilt or shame. Before delivering PFR, providers complete a structured training process through which they learn specific, measurable strategies (Oxford et al., 2018) to support the caregiver-provider relationship and growth in parenting skills.

PFR has been evaluated in several experimental and quasi-experimental trials with diverse populations. Oxford, Spieker, et al. (2016) found evidence that PFR implementation in the SPP, which is described in the Methods section, had beneficial impact on observed parent sensitivity, parent understanding of toddlers’ social-emotional needs and developmental expectations, and child atypical affective communication through 6 months post intervention. In addition, children in the PFR condition showed parasympathetic nervous system regulation during a series of difficult tasks that was more similar to typically developing and non-maltreated children, then did children in the comparison group (Hastings et al., 2019). Children in the PFR arm were also significantly less likely than children in the control arm to be removed from their homes and placed into foster care one year post intervention, the latest time point for which placement data were available. In an earlier trial involving infants and toddlers in foster care, Spieker et al. (2012) also found significant gains through posttest in the PFR condition relative to control condition in observed parent sensitivity, parent understanding of toddler’s social-emotional needs and developmental expectations, and parent ratings of child competence, for caregiver-toddler dyads receiving PFR after a placement change dictated by child welfare. Results in this study were especially strong for recently reunified birth parents (Oxford, Marcenko, Fleming, Lohr, & Spieker, 2016). More recently, Booth-LaForce et al. (2020) replicated these findings in a trial with American Indian families. Two additional RCTs are underway to further assess PFR impacts in families reunified after a foster care placement of a child under the age of five, and with dyads of infants and mothers with perinatal mood disorders.

Benefits and costs of preventing child maltreatment

BCAs of child maltreatment interventions can help guide public and private investment towards effective programs that return more in long-term benefit to society than they cost. These analyses rely on comprehensive information about the resources needed to implement interventions (Crowley et al., 2018; National Academies of Sciences, 2016) and estimates of the lifetime costs of child maltreatment. The latter provide the foundation for benefits estimates, which reflect child maltreatment costs that are avoided and revenues that are increased due to intervention. Two recent studies underscore the significance of intangible, quality-of-life losses for victims in estimates of the total lifetime costs of child maltreatment. These very real losses include pain, suffering, diminished quality of life, and psychological distress experienced by victims of violence. Their monetized value has typically been estimated using jury awards or willingness-to-pay approaches and found to exceed tangible victim costs like medical and mental health care and lost future earnings (Corso, Fang, & Mercy, 2011; McCollister, French, & Fang, 2010; Miller, Fisher, & Cohen, 2001). Using a human capital approach that did not monetize quality-of-life impacts, Fang and colleagues (2012) estimated the lifetime costs of nonfatal victimizations to be $210,012 (2010 USD, or $229,801 in 2015 USD). However, Peterson et al. (2018) showed that nonfatal victimizations were far more costly when intangible, quality-of-life impacts were monetized, $830,928 (2015 USD). The divergence in estimates produced by Fang et al. (2012) and Peterson et al. (2018) suggests that different assumptions about victimization costs are likely to play a key role in determining whether an investment is cost beneficial.

To bring both rigor and consistency to its BCAs, the Washington State Institute for Public Policy (WSIPP, 2018), a policy-neutral organization that conducts legislative and policy research as directed by the Washington State legislature, developed a model and software tool to provide information about which programs are effective in improving public outcomes and what returns could be expected from investing public dollars in them. The model performs BCAs in child maltreatment and 10 other policy areas (e.g., public health/prevention, juvenile justice). Strengths include the use of consistent methods to estimate benefits from improvements in different outcomes (e.g., crime, education, mental health disorder); a Monte Carlo feature that allows the effects of various forms of uncertainty on costs and benefits to be assessed; disaggregation of benefits by beneficiary (e.g., participant, taxpayer) and source (e.g., earnings-related tax revenue, avoided health care costs); and validation checks, including expert panel review and comparisons to estimates produced in non-WSIPP peer-reviewed studies.

WSIPP’s emphasis on using a consistent set of methods in its BCAs has helped level the analytic playing field, but some important conceptual and methodological questions remain in BCAs of child maltreatment programs, two of which will be considered in this paper. The first concerns estimates of intervention impacts on CAN. Although some studies measure CAN via substantiated maltreatment reports (Olds, Henderson, Chamberlin, & Tatelbaum, 1986), others do not (Chaffin et al., 2004). In the SPP, significant impacts were observed on out-of-home placement (OOHP; Oxford, Spieker, et al., 2016). Because OOHP is predicated on maltreatment, reductions suggest concomitant reductions in CAN, which WSIPP acknowledges in its technical documentation (WSIPP, 2018). However, absent direct measurement of CAN, WSIPP adopts a conservative stance in its BCAs, projecting benefits only from avoided system costs (i.e., CPS, child welfare system, courts). Resulting benefits estimates may therefore underestimate the full impact of interventions that reduce OOHPs because other tangible (e.g., medical care, special education) and intangible (e.g., health-related quality of life) CAN-related victim costs may in fact be substantial (Fang et al., 2012; Peterson et al., 2018).

Further illustrating this point, several studies indicate that OOHP itself has emotional consequences, even when warranted because of threats to children’s safety. They suggest that when children are removed from the home, their attachment to their primary caregiver is disrupted (Doyle & Cicchetti, 2017), with consequences for quality of life. Moreover, each new placement increases the probability of a subsequent placement change (Newton, Litrownik, & Landsverk, 2000; Rubin, O’Reilly, Luan, & Localio, 2007). These changes, including family reunification, are traumatic disruptions involving separation from one primary caregiver and the necessity of attaching to another. Repeated losses and multiple placement transitions compound the risk for emotional and behavioral problems, are associated with adverse developmental outcomes, and increase the probability of high mental health service use (Rubin et al., 2004). Although the pain and suffering from disrupted attachment and the pain and suffering from child maltreatment are entangled, a recent study reported that separations from primary caregivers had adverse consequences after accounting for the impact of violence and other traumatic exposures, and that an accumulation of caregiver separations/losses was especially detrimental (Briggs-Gowan et al., 2019). Collectively, these studies indicate that preventing OOHP should improve quality of life, though the magnitude of benefits is uncertain. The compounding effect of multiple OOHPs suggests potentially increasing costs with more OOHPs.

A second methodological issue for CAN benefits estimates is whether interventions targeting families and children with some prior history of CPS involvement can actually avert future CAN costs, because some prior maltreatment experience leading to future CAN costs may already have occurred. In such cases, WSIPP adopts a cautious approach, even when impacts on CAN are observed post intervention. Benefits estimates include reductions in child welfare system costs and tangible medical and mental health costs to victims but exclude intangible quality-of-life costs as they may already have been triggered by the pre-intervention experience of CAN (WSIPP, 2018). This modeling assumption may, however, not capture important benefits, particularly for attachment-oriented interventions like PFR that are implemented with very young children, as was the case in this trial. Because parental warmth or sensitivity is a protective factor throughout the life course (Black & Hoeft, 2015; Farrell et al., 2019), improving the quality of the parent-child relationship through improved parental sensitivity and responsivity is expected to have lasting impact across domains of functioning, and thus improve the child’s quality of life. Avoiding a subsequent placement change is an additional benefit, as each placement change increases the probability of negative behavioral outcomes, even when initially, as is more often the case for younger children, the child did not have behavioral problems at the initial placement (Newton et al., 2000).

The present study

This study reports findings from a BCA of PFR as delivered to CPS-involved families in the SPP. We estimated the net costs of delivering PFR (i.e., over and above control arm costs) and compared them to long-term benefits expected from measured impacts on OOHP and implicit impacts on CAN. Both costs and benefits were estimated from a societal perspective that included participants, taxpayers, and other members of society. For example, when medical and mental health costs are avoided through intervention, participants, taxpayers as public insurers, and private insurers may all benefit to some degree. Given uncertainty around the extent to which victim costs are avoided when OOHP is reduced, particularly in families with some prior child protective services involvement, sensitivity analyses estimating benefits under different assumptions about avoided victim costs were a central focus of this study.

Method

Participants

Participants in the SPP were residents of King, Snohomish, or Skagit counties in Washington State, had an infant or toddler between the ages of 10 and 24 months, and were the subject of a recent, open CPS report alleging maltreatment. Of 384 eligible families, 133 declined to participate, 251 completed a baseline assessment, and 247 were randomized to the intervention and control arms of the study. Figure 1 describes the flow of families throughout the trial, conducted between 2010 and 2014. Parent participants were mostly female (91.1%), White (77.3%), not living with a spouse or partner (53.0%), living with two or more children in the household (67.2%), and not employed outside the home (83.8%). They were 26.7 years old on average at baseline, averaged $21,873 in median income, many received food stamps (78.9%), and none had a college degree. Just over half of target children were male (53.8%), and their mean age at baseline was 16.4 months. Characteristics did not differ across study arms. For 216 of the 247 families (87.4%), the target child was the alleged victim of maltreatment.

Figure 1.

Consort Flow Diagram for the Supporting Parents Project Trial

* For “lost” categories, “not completed” values varied by follow-up, but “withdrew” values were cumulative

Intervention

PFR

Training and ongoing support.

The 10-week, manualized PFR intervention was delivered by two master’s-level professionals in social work and counseling employed by a local community agency. Interventionists completed PFR training and certification over a 5-month period in late 2010. Certification involved attending an initial workshop led by a PFR master trainer and then participating in mentored training, which included viewing 10 hours of videotaped sessions of parent-child dyads receiving the 10 sessions of PFR and conducting PFR with a practice family while receiving weekly consultation and feedback from the PFR trainer. Throughout the trial, weekly reflective practice sessions led by a PFR consultant were held with the two providers to ensure ongoing high-fidelity implementation and to offer emotional support to interventionists working with high-need families and at-risk infants and toddlers.

Intervention.

Families were offered 10 weekly home-based PFR sessions. Dosage was high, with 93% of intervention arm families participating and 86% completing all 10 sessions, delivered on average over 14.3 weeks (SD = 5.4 weeks; range: 5 - 36 weeks).

Control intervention

To provide some support to vulnerable control arm families, each was offered three resource and referral calls; 89% received all three calls over 14.1 weeks on average (SD = 3.2 weeks; range: 7 - 35 weeks). After an initial needs assessment call, a social services provider mailed a packet including a standard resource list and personalized resources and referrals in the areas of financial support, education, household needs, housing, and parent support. The provider subsequently made two check-in calls.

Measures

Intervention costs

To estimate the opportunity cost of offering PFR in relation to the alternative R&R intervention, all resources needed to deliver PFR and R&R were valued, even those like overhead and parent time that were not paid for directly. Data were from a variety of sources.

Personnel.

Personnel consisted of the two interventionists who worked with families in the PFR condition and one interventionist who offered R&R to families in the control arm, a mentor who carried out the mentored training, some agency supervision, and reflective practice consultants who provided regular support and technical assistance to interventionists during the intervention period. Interventionist cost data came from monthly or bimonthly invoices submitted from the social welfare agency and included hours worked, wage rates, total wages, fringe benefits, and unemployment insurance. Agency supervision costs were included in the agency invoices. Mentor training costs and weekly reflective practice consultation costs included salary and fringe benefits paid to two consultants and were from project records.

PFR workshop.

As part of the PFR certification process, the interventionists attended a 3-day group-training workshop offered by a PFR master trainer. The lump-sum workshop cost reflected trainer time, materials and supplies, and rental space. Interventionist training time and associated costs were from agency invoices.

Supplies.

Supplies for the PFR condition consisted of cell phones for each interventionist, PFR intervention materials (e.g., parent books, social and emotional learning card sets), home visiting supplies (e.g., wipes, toys), and recording equipment and related materials used to record PFR sessions so that video feedback could be offered to parents (e.g., Camcorder, charger, DVDs). In the R&R condition, the only supplies were referral packets, envelopes, and stamps to mail supplies to each family. The amount and cost of these supplies came from project records.

Travel costs.

Mileage and parking cost information was included in agency invoices.

Overhead.

Overhead was estimated at 20% of labor costs and was included to cover resources like office space, office supplies, and other consumables used but not paid for directly.

Parent time.

Project records showed how many PFR sessions each family completed. Sessions lasted 1 hour on average. Parents in the R&R arm spent an average of 30 minutes in the initial assessment call and 10 minutes in each of the two follow-up calls.

Monetizable intervention effects

The trial’s monetizable impacts included out-of-home placements (OOHPs) and implicit effects on CAN. OOHP data through 15 months post SPP enrollment were from child welfare system records obtained in 2015, but CAN had to be inferred from OOHP. Given uncertainty in the degree to which CAN victim costs were averted when OOHP was reduced, our analysis monetized PFR impacts at CAN effect sizes ranging from 0 to equivalent to the OOHP effect size; this strategy yielded a range of potential victim benefits from PFR.

Analysis plan

Approach

The BCA was conducted from a societal perspective in which, on the cost side, all intervention-related resources were valued, consistent with best practice guidelines (Levin & McEwan, 2001; National Academies of Sciences, 2016). On the benefits side, PFR’s monetizable effects on OOHP and implicit effects on CAN were estimated based on an intent-to-treat analytic framework. Benefits estimates were comprehensive, including a variety of stakeholder perspectives (i.e., participants, taxpayers, others) and adjusting for deadweight costs of taxation, or the economic loss to society per dollar of taxes incurred or gained when taxes are avoided (National Academies of Sciences, 2016). The time horizon for collecting cost data was 4 years (2010 - 2014), but benefits were estimated over the lifecycle of children targeted by the intervention. The BCA compared PFR’s expected long-term net benefit per participant to the net average cost per participant, after adjusting estimates for inflation using the Implicit Price Deflator for Personal Consumption Expenditures and discounting back to intervention start at a rate of 3% per year. All estimates were reported in 2014 USD. Benefits greater than costs per family after adjustments were made would indicate that PFR was cost beneficial.

BCA

The WSIPP model required three PFR-related inputs: estimates of impact and standard error for each monetizable outcome, net average cost per family (which included a 10% cost range used in the Monte Carlo analysis), and the type of intervention population.

Monetizable PFR impacts.

Odds ratios summarizing PFR impacts on OOHP at 5 months (time 1) and 15 months (time 2) post intervention were translated to standardized mean difference effect sizes using the Cox d transformation, which is calculated as the natural logarithm of the odds ratio and divided by 1.65. Associated standard errors were calculated as the square root of 0.367 x (1/I_s + 1/I_f + 1/C_S + 1/C_f), where I_s, I_f, C_s, and C_f are the number of successes (i.e., no OOHP) and failures (OOHP) in the intervention and control arms (WSIPP, 2018). The use of two distinct measures allowed benefits estimates to incorporate increasing PFR impacts from 5 to 15 months post intervention.

Net PFR costs.

The total cost of implementing PFR or R&R in the SPP was estimated by summing the costs of all resources involved, ΣC_i. For some resources, C_i was available, but for others, C_i was the product of the amount of the resource used, Q_i, and its unit cost, P_i. The average PFR or R&R cost per family was estimated by dividing the total cost by the number of families in that condition, ΣC_i/n. Because it can be helpful to stakeholders to understand how costs are used for key intervention activities, PFR total and average costs were disaggregated into training, direct intervention, and ongoing support by summing costs of resources relevant to each activity. They were also disaggregated by key resource (e.g., labor, supplies). Family burden was evaluated in terms of time spent on the intervention and also implicit cost. For PFR, the average time burden per family was estimated by dividing the total number of home visits delivered over the course of the intervention by the number of families in the condition; since visits averaged one hour, this quotient represented total PFR time per family. For R&R families, family burden was estimated as the time families spent on telephone calls. To calculate implicit costs to families, we used wage data from the March Supplement to the Current Population Survey to assign a wage rate to each participant based on their highest level of education attainment, obtained from project records. The average wage for individuals age 27 was used for PFR parents and age 26 for R&R participants, corresponding to the average age of parents in each condition. We calculated an average wage for each condition and multiplied it by the average family time in that condition to estimate implicit cost per family.

Benefits.

Benefits were estimated using WSIPP’s BCA model and software tool (WSIPP, 2018) and reflected long-term avoided costs or increased revenues expected to occur over the lifetimes of participants because of PFR impacts on OOHP and implicitly on CAN observed up to 15 months post baseline. Benefits estimates were based on WSIPP’s analysis of causal relationships between OOHP and CAN in the present and future prevalence of behaviors, disorders, or events that generate costs or revenues (e.g., current and future CPS involvement through age 17, medical costs). For this analysis, CAN and CWS recidivism rates were based on child maltreatment prevalence in a “treatment population,” following WSIPP’s standard practice, and were higher than those in a general population. This is a conservative assumption in that 31 of the child participants were not identified victims of maltreatment, and only five had a prior foster placement. Table 1 describes system and tangible and intangible victim benefits expected from PFR impacts.

Table 1.

Promoting First Relationships Population and Sources of Benefit in the Benefit-Cost Analysis

	BCA Scenario a
Population and Benefit Source b	1	2	3
Population c	√	√	√
Treatment, because a precondition for trial participation was a CPS allegation
Avoided system costs: CPS, CWS, and court costs through age 17, as reductions in the present decreased risk for subsequent victimizations and placements d	√	√	√
CPS: Referrals, police involvement, juvenile court dependency, case involvement, in-home services
CWS: Protective custody placement, adoption
Court: Juvenile court termination case involvement
Avoided victim costs from the present through 30 years into the future, as victim costs are experienced over long periods of time e
Tangible: Medical and mental health (anxiety, depression, PTSD, disruptive behavior disorder, disordered alcohol, illicit drug use, and smoking, obesity)f		√	√
Intangible: Pain, suffering, psychological distress, diminished quality of life g			√
Improved earnings and human capital through age 65, due to reductions in CAN h,i		√	√
Other long-term avoided costs causally linked to CAN h,i		√	√
Education: Special education placement, grade retention through age 18
Crime
Premature mortality through age 17

^aBCA=Benefit-cost analysis. CPS=Child Protective Services. CWS=Child welfare system. PTSD=post-traumatic stress disorder. OOHP=Out-of-home-placement. CAN=Child abuse and neglect.

^bBenefits in the Washington State Institute for Public Policy (WSIPP) model reflect avoided costs and increased revenues estimated to accrue over the lifecycle of Promoting First Relationships participants because of impacts on out-of-home placements (OOHP) and implicitly on CAN. Refer to WSIPP’s technical documentation for detailed information about computational routines, unit costs, causal relationship magnitudes, and the like (Washington State Institute for Public Policy, 2018).

^cThis is a conservative assumption as 98% of Supporting Parents Project sample did not have a prior OOHP, the strongest indicator of prior CAN, and most CPS allegations are neither substantiated nor investigated.

^dAvoided costs (i.e., benefits) in each year through age 17 are the product of the likelihood of OOHP occurring in that year, expected CPS, CWS, and court costs per case, and the unit change in OOHP indicated by the intervention effect size.

^eAvoided victim costs in each year through 30 years into the future are the product of the likelihood of CAN occurring in that year, expected direct and indirect costs per case, and the unit change in CAN indicated by the implicit intervention effect size.

^fTangible medical costs are from Miller, Fisher, and Cohen (2001). Tangible mental health, substance use, and obesity costs are from WSIPP’s meta-analyses of the causal relationships between each condition and CAN.

^gIntangible quality-of-life costs are also from Miller et al. (2001).

^hImproved earnings and human capital and other long-term avoided costs avoided reflect WSIPP’s meta-analyses of the causal relationships between each of these outcomes and CAN.

ⁱWSIPP conservatively halves the causal relationship between CAN and each of these indirect benefits areas in BCAs involving treatment populations because some of the indirect cost may already have been incurred prior to intervention. We have not altered this assumption in our analyses.

In general, for each year in which benefit occurred, the amount of benefit was the product of the intervention effect size, the magnitude of the causal relationship between the impact and the source of the avoided cost or increased revenue, and the unit cost or revenue. Relationship magnitudes and standard errors were from WSIPP’s meta-analyses of relevant research literature (relationship between CAN and, e.g., depression, special education placement, alcohol use disorder). Unit values for avoided costs and increased revenues relied on information from national or Washington State databases (e.g., Washington State DSHS Children’s Administration data for per child costs of CPS referrals or foster care placements; Administration on Children, Youth, and Families for CPS police involvement costs). Direct and indirect benefits streamed through their logical lifecycle (e.g., earnings through age 65, CPS involvement through age 17) and to a maximum of age 100. Because of discounting, benefits produced far into the future had a relatively small impact on total benefits estimates. Detailed information about computational routines and the magnitude of causal relationships and unit costs can be found in WSIPP’s technical documentation (WSIPP, 2018).

We estimated benefits under three scenarios reflecting different plausible assumptions about OOHP- and CAN-related benefits: (1) benefits only from avoided system costs, (2) benefits from avoided system and tangible victim costs, and (3) benefits from avoided system and tangible and intangible victim costs. Because of uncertainty in the extent to which reductions in OOHPs were associated with CAN benefits, we varied the CAN effect size in scenarios 2 and 3 in 10% increments, from a minimum of 0 (which reduces to scenario 1), to a maximum value equal in magnitude to the OOHP intervention effect.

BCA summary metrics.

For each scenario and for each effect size analyzed in scenarios 2 and 3, we ran 1,000 Monte Carlo simulations, in which estimates of cost, impact, and other parameters driving benefits models were varied based on random draws from the distribution around each parameter’s point estimate. We recorded several expected values from the 1,000 simulations: total benefits, benefits to stakeholders (e.g., participants, taxpayers, others), benefits by source (e.g., healthcare cost savings, earnings gains), and total costs. PFR’s net present value (NPV) was the difference between expected benefits and expected costs. A positive value provided evidence that PFR was cost beneficial and indicated the net welfare gain to society per family from investing in PFR. PFR’s benefit-cost ratio (BCR) was the ratio of expected benefits to expected costs and reflected the expected dollar return per dollar invested. A value of 1 indicated that PFR was cost beneficial. We established 95% credible intervals around expected values by sorting the 1,000 BCA results by NPV and identifying the 25th lowest value as the lower bound and the 25th highest as the upper bound. To estimate investment risk, we also calculated the percentage of Monte Carlo simulations in which PFR was cost beneficial (produced an NPV greater than 0). We plotted PFR costs, benefits, NPV, and investment risk curves from the results. For each scenario, we estimated the CAN effect size at which 75% or more of the simulations produced a positive NPV, that is, the CAN effect size at which PFR was reliably cost beneficial according to WSIPP standards.

Results

PFR impacts

Of the 124 children in the intervention arm, 3 (2.8%) had been placed outside the home 5 months after enrolling in PFR, compared to 7 (5.8%) in the control arm; results did not differ by interventionist. The gap widened over time, with 7 (5.6%) of the intervention arm children being placed outside the home at the 15-month post-enrollment mark, compared to 16 (13.0%) in the control arm. These results translated to odds ratios for OOHP, and implicit effects on CAN, of .486 and .397. Standardized mean difference effect sizes were −.438 (SE = .404) at 5 months and −.560 (SE = .287) at 15 months.

Net PFR costs

Table 2 shows the costs of implementing PFR in the intervention arm and R&R in the control arm. Before considering parent costs, total PFR costs were $225,307, or $1,817 per family on average. The majority of the PFR investment was in direct intervention activities (78.5%). Among key ingredients, labor costs dominated (72.1%). The net investment in PFR averaged $1,759 per family (i.e., $1,817 PFR - $58 R&R).

Table 2.

Promoting First Relationships and Resource and Referral Costs (2014 USD)

	Promoting First Relationships								Resource and Referral			Net PFR Cost per Family a	Share of Cost
	Year					Total Cost	Average Cost per Family	Share of Cost	Total Cost	Average Cost per Family	Share of Cost
	2010	2011	2012	2013	2014
Cost by Major Activity
Training	$13,416	$1,831	$0	$0	$0	$15,248	$123	6.8%				$123	7.0%
Intervention	$1,812	$37,377	$53,832	$59,394	$24,680	$177,095	$1,428	78.6%	$7,153	$58	100.0%	$1,370	77.9%
Support	$0	$9,241	$9,997	$8,954	$4,772	$32,964	$266	14.6%				$266	15.1%
Total	$15,228	$48,449	$63,830	$68,348	$29,452	$225,307	$1,817	100.0%	$7,153	$58	100.0%	$1,759	100.0%
Cost by Key Ingredient
Personnel	$11,458	$35,415	$45,581	$48,113	$21,885	$162,451	$1,310	72.1%	$5,912	$48	82.7%	$1,262	71.8%
Supplies	$1,479	$3,664	$3,119	$1,949	$470	$10,681	$86	4.7%	$59	$0	0.8%	$86	4.9%
Travel	$0	$2,287	$6,013	$8,664	$2,721	$19,684	$159	8.7%	$0	$0	0.0%	$159	9.0%
Overhead	$2,292	$7,083	$9,116	$9,623	$4,377	$32,490	$262	14.4%	$1,182	$10	16.5%	$252	14.3%
Total a	$15,228	$48,449	$63,830	$68,348	$29,452	$225,307	$1,817	100.0%	$7,153	$58	100.0%	$1,759 a	100.0%
Parent Cost b	$0	$2,860	$3,278	$3,173	$1,113	$10,424	$84	4.4%	$866	$7	10.8%	$77	4.2%
Total with Parent Cost b	$15,228	$51,309	$67,107	$71,521	$30,565	$235,731	$1,901		$8,019	$65		$1,836

^aNet Promoting First Relationships (PFR) cost per family without parent costs was used in the benefit-cost analysis per standard practice.

^bParent costs and total PFR costs with parent costs were estimated to indicate the full societal cost of PFR.

PFR net family burden and cost

The intervention included 1,100 home visits, or 8.9 1-hour visits per family. Family burden was thus estimated at 8.9 hours on average. The implicit cost of parent time was $10,424 across all intervention families, or $84 per family, bringing the average PFR cost per family inclusive of parent time to $1,901. R&R burden per family was estimated at 48 minutes and valued at $7 per family, yielding a total cost, including parent time, of $65 per family. The net PFR cost per family, including parent time, was $1,836 on average.

PFR BCA under different assumptions about benefits to victims

Table 3 summarizes BCA findings from scenarios 1, 2, and 3. Effect sizes at 5 and 15 months post intervention are reported in the upper part of the table, followed by PFR net costs and then scenario 2 BCA summary metrics (PFR total benefits, NPV, and BCR). The second half of the table disaggregates benefits by beneficiary and source. A final entry shows the additional benefits expected under scenario 3, when victim quality-of-life gains were incorporated into benefits projections, and also includes NPV and BCR estimates. For scenarios 2 and 3, we report findings for CAN effect sizes that are 20%, 40%, 60%, 80%, and 100% of the OOHP effect size.

Table 3.

Promoting First Relationships Benefit-Cost Analysis Results Under Different Benefits Assumptions

	Scenario 1: System Benefits Only	Scenarios 2 & 3: System and Victim Benefits
PFR Effect Sizes a, b	ES Time 1, Time 2	ES Time 1, Time 2	ES Time 1, Time 2	ES Time 1, Time 2	ES Time 1, Time 2	ES Time 1, Time 2
OOHP	−0.460, −0.560	−0.460, −0.560	−0.460, −0.560	−0.460, −0.560	−0.460, −0.560	−0.460, −0.560
CAN	0.000, 0.000	−0.092, −0.112	−0.184, −0.224	−0.276, −0.336	−0.368, −0.448	−0.460, −0.560
% of OOHP	0%	20%	40%	60%	80%	100%
Costs, all scenarios c	$1,756	$1,759	$1,757	$1,755	$1,760	$1,758
(credible interval) d	($1,617, $1,893)	($1,619, $1,893)	($1,623, $1,895)	($1,621, $1,891)	($1,627, $1,893)	($1,623, $1,893)
Scenarios 1 & 2
Benefits c	$1,756	$7,273	$13,321	$18,692	$23,869	$27,320
(credible interval)	(−$1,529, $5,149)	(−$30,975, $52,219)	(−$20,032, $56,591)	(−$13,034, $74,540)	(−$9,058, $76,954)	(−$6,525, $85,897)
NPV c	$1	$5,514	$11,564	$16,937	$22,108	$25,562
(credible interval)	(−$3,346, $3,388)	(−$32,739, $50,516)	(−$21,685, $54,979)	(−$14,817, $72,842)	(−$10,853, $75,211)	(−$8,374, $84,154)
BCR	$1.00	$4.13	$7.58	$10.65	$13.56	$15.54
Benefits by beneficiary
Participant	$63	$3,343	$6,834	$9,881	$12,814	$14,778
Taxpayer	$1,717	$3,439	$5,355	$7,101	$8,758	$9,846
Other	−$24	$490	$1,132	$1,711	$2,297	$2,695
Benefits by source
OOHP system	$2,343	$2,308	$2,311	$2,332	$2,372	$2,305
CAN system	$296	$246	$495	$735	$959	$1,134
Earnings e	$0	$4,832	$9,885	$14,291	$18,526	$21,377
Health care	$0	$190	$409	$597	$801	$902
Education	$0	$128	$260	$385	$503	$595
Crime	$0	$424	$837	$1,218	$1,603	$1,894
Property loss	$0	$1	$1	$2	$2	$3
Deadweight loss	−$883	−$856	−$876	−$866	−$896	−$890
Scenario 3
Quality-of-life benefits e	n/a	$1,860	$3,287	$4,678	$5,265	$6,149
NPV c		$7,373	$15,204	$21,994	$27,351	$31,750
(credible interval)		(−$33,909, $55,819)	(−$20,385, $76,023)	(−$13,034, $74,540)	(−$10,166, $85,431)	(−$7,003, $94,038)
BCR		$5.19	$9.65	$13.48	$16.57	$19.05

^aPFR=Promoting First Relationships. ES=Effect Size. OOHP=Out-of-home placements. CAN=Child abuse and neglect. NPV=Net present value. BCR=Benefit cost ratio.

^bOOHP standard errors were .404 and .287 for 5 months (time 1) and 15 months (time 2) post PFR, respectively, and were held constant regardless of the CAN ES.

^cCosts, benefits, and NPV are expected values from 1,000 Monte Carlo simulations (2014 USD).

^d95% credible intervals reflect the 25th lowest and 25th highest values from 1,000 Monte Carlo simulations.

^eEarnings are net of investment in higher education. Quality-of-life benefits accrue to participants.

With respect to expected values, PFR was just cost beneficial under scenario 1, which included system cost savings but no victim benefits. Nearly all of the benefit (98%) accrued to taxpayers. Scenario 2 findings indicated substantial additional benefits when tangible victim cost savings were included in benefits models. Approximately $5,500 in additional benefits per target child was projected even at a very modest reduction in CAN (i.e., ES of −.09 at time 1, 20% of OOHP ES). Scenario 2 NPVs ranged from $5,514 to $25,562 per target child, with BCRs ranging from $4.13 to $15.54, exceeding the net cost of delivering PFR by many multiples.

Under scenario 2, proportionately more benefits accrued to target children (average share = 52%) and other stakeholders (average share = 40%) relative to scenario 1. They were from a variety of sources, reflecting the broad consequences of CAN. The largest benefits were from earnings gains, reductions in crime costs, and health care cost savings. These indirect but tangible benefits were large even though the WSIPP model halves the estimated causal relationship with CAN in BCAs with treatment populations, as was the case here.

Scenario 3 findings in the lower portion of Table 3 highlight substantial additional benefits expected from intangible quality-of-life gains. They ranged from $1,860 per target child at a CAN impact of −.09 (time 1) to $6,149 when CAN impact was equivalent to the OOHP impact. These benefits were second in magnitude only to earnings gains and accrued entirely to target children, increasing their share of total benefits to 60% on average.

Figure 2 summarizes total costs, benefits, and NPV for scenarios 1, 2, and 3. Investment risk (the percentage of Monte Carlo trials producing a positive NPV) is also shown for each ES. The figure shows that at a CAN ES of 0 (scenario 1), PFR was cost beneficial 50% of the time. Under scenario 2, investment risk improved steadily with increasing CAN. At CAN values at or above −.25 (just under half the OOHP effect size), PFR was cost beneficial 75% of the time. This means that PFR would be reliably cost beneficial if CAN costs were avoided in about half of the OOHP cases, if half of expected CAN costs were avoided when OOHP was avoided, or some combination of the two. Under scenario 3, NPV was improved compared to scenario 2 due to additional quality-of-life benefits, with minor improvement to investment risk.

Figure 2.

Benefit-Cost Analysis Expected Values at Varying Child Abuse and Neglect Effect Sizes (2014 USD).

Note. Effect Sizes are at last follow-up (15 months post-intervention, time 2 in Table 3). Scenario 1 benefits, net present value, and investment risk estimates correspond to a child abuse and neglect effect size of 0. Investment risk percentages reflect the proportion of Monte Carlo simulations that returned a positive net present value

Discussion

This study’s comprehensive analysis indicated that it cost $1,817 per family on average to deliver Promoting First Relationships in the Supporting Parents Project, $1,757 more than resource and referral offered to control families. As with most preventive interventions, labor and intervention comprised the largest shares of costs. Training and ongoing support together accounted for about 20% of costs and were important conceptually to ensuring high-quality, impactful intervention. The implicit cost of parent time was low, at $84 per family.

Were these investments in PFR cost beneficial? Our analysis suggests they were, though the extent varied with key assumptions driving lifecycle benefits models. Under the most conservative assumption—that reducing out-of-home placements (OOHP) led only to avoided system costs—PFR benefits were slightly but not reliably higher than costs. Adding direct and indirect victim benefits to projection models had a large effect on PFR’s net present value, with benefits per target child exceeding costs by thousands of dollars even at small child abuse and neglect (CAN) effect sizes. Without considering any quality-of-life gains from reducing CAN, PFR was a reliable investment with an OOHP effect size of −.56 and a CAN effect size of half that magnitude. Adding quality-of-life gains increased expected benefits, but because the standard errors around quality-of-life estimates are relatively large, their impact on investment risk was somewhat muted due to uncertainty.

The different viable assumptions driving benefits estimates underscore the need for future longitudinal research on the impacts of interventions for indicated populations. In this study, PFR was delivered to a sample in which 87% of the toddlers were identified victims in reports to CPS before the intervention began, and the rest were siblings of alleged victims. Five toddlers had previously been removed from the home. For those who experienced CAN pre-intervention, there could be lingering CAN impacts and related costs even though additional CAN was avoided. However, this attachment-based intervention, which was delivered to families with very young children, is intended to enhance parent’s ability to meet their children’s needs for social and emotional support. Significant intervention-related gains in parent understanding of toddlers, child atypical affective communication, and especially parent sensitivity (Booth-LaForce et al., 2020; Oxford, Marcenko, et al., 2016; Oxford, Spieker, et al., 2016; Spieker et al., 2012), along with improved child stress physiology (Hastings et al., 2019; Nelson & Spieker, 2013)—all indicative of healthier parent-child relationships and child developmental outcomes—suggest the potential for a reduction in long-term harms that may have been incurred pre-intervention. Removing children from the home also has emotional costs that may include difficulty forming attachments with new caregivers and increased risk for behavior problems, which can lead to long-term costs that are distinct from the costs of CAN (Briggs-Gowan et al., 2019; Spieker, Oxford, & Fleming, 2014). Thus, when OOHP benefits models are based only on avoided system costs, they may underestimate the benefits of intervention. Our analyses showed that even modest assumptions about victim benefits led to substantially increased benefits from PFR. The widening gap between intervention and control arms in OOHP from 5 to 15 months post intervention provides some evidence, albeit early, that these relational gains were sustained.

Limitations

This study’s strengths include use of a comprehensive BCA model and software tool designed to estimate long-term benefits from avoiding OOHP and CAN. Because the BCA model was developed primarily for use in Washington State, it is particularly germane to performing a BCA of PFR as implemented in the SPP. The study’s detailed, prospective cost data and trial in which 93% of intervention arm families completed at least one session, and 87% all 10 sessions, support the validity of this study’s findings. The use of Monte Carlo simulations to estimate the study’s benefit-cost given multiple sources of uncertainty is a further strength.

The WSIPP model builds in certain safeguards to ensure estimates are conservative, but they may lead to additional undervaluing of PFR’s net societal benefit. These include designating the SPP sample as a treatment population because, per WSIPP convention, the allegations of child maltreatment in the SPP sample indicated “child welfare involvement.” Although SPP families must have had a screened-in allegation of child maltreatment to be eligible for the study, generally screened-in cases are not substantiated. In 2014, for example, 44,312 cases were screened into CPS while only 4.3 per 1000 cases led to an investigation and substantiation (Partners for Our Children, 2014). Moreover, 98% of target children in the SPP sample did not have a prior OOHP, the strongest indicator of prior CAN, and therefore were not “child welfare involved.” This is in contrast to other child maltreatment studies like Parent Child Interaction Therapy, in which parents had a substantiated case of child physical abuse and were clearly a treatment sample (Washington State Institute for Public Policy, 2019b). Although the SPP sample was not a true treatment population, the model nonetheless halved the magnitude of the causal relationship between CAN and indirect outcomes like grade retention, special education placement, crime, and employment, lowering expected benefits form PFR. To further ensure conservative estimates, the model also applied a series of “trumping” rules to avoid possible double counting of benefits from a similar source. For example, health care costs are avoided from PTSD, depression, and anxiety; the NPV includes only avoided PTSD health care costs, which were the highest of the three, ignoring avoided depression and anxiety costs.

Other limitations center on what is known about the costs of CAN and the extent to which future harms to victims can be avoided by early preventative interventions delivered to at-risk populations. Attachment-based interventions focused on developing healthy, more responsive and sensitive parenting leading to more secure parent-child relationships hold promise for long-term impact, but in the SPP trial, follow-up was limited to 15 months after enrollment, restricting the time period over which PFR’s effects on parent-child relationships, OOHP, and implicit CAN could be assessed. Methodologically, benefits models would be improved by greater understanding of the emotional costs of OOHP as distinct from the costs of CAN. Currently, the trauma surrounding removal from the home and its associated economic costs are neither fully understood nor included in benefits models separately from the emotional costs of CAN. Recent research suggests that an accumulation of caregiver separations and losses is detrimental (Briggs-Gowan et al., 2019), and the field of infant and early childhood mental health increasingly recognizes the need to maintain and nurture primary parent-child relationships and avoid disruptions whenever possible (Spieker et al., 2014).

Although carefully crafted simulation models like WSIPP’s that project long-term are valuable, future studies could complement simulation findings by tracking short-term costs and service utilization by intervention participants and controls. Priorities could include documenting costs and use of medical, mental health, and child-welfare system services. Administrative data, such as Medicaid claims showing service utilization and related costs, could be used to estimate intervention-related savings, with potential to illuminate short- and longer term avoided costs.

Implications and future directions

Results show that PFR, a relatively brief home-visiting intervention that is far less costly than more intensive home-based interventions (Washington State Institute for Public Policy, 2019a, 2019b), was expected to generate systems-level savings that exceeded intervention investments. Even modest assumptions about the extent to which PFR reduced tangible and intangible quality-of-life costs sharply increased expected long-term benefits, NPV, and the likelihood that investment in PFR would yield a positive NPV. Findings underscore the need to better understand long-term benefits, including reduced pain and suffering and improved quality of life, from child maltreatment interventions offered to populations at elevated risk. Because BC A findings can influence which programs and policies are supported by limited resources, research that better delineates the long-term health and economic consequences of CAN in indicated populations could help guide investment in effective, cost-beneficial interventions.