Preventing Maltreatment with a Community-Based Implementation of Parent-Child Interaction Therapy

Abstract

The purpose of this study was to examine rates of child abuse and neglect reports following a community implementation of Parent-Child Interaction Therapy (PCIT), an evidence-supported intervention for the prevention of maltreatment. Among a group of families receiving PCIT, predictors of reports were examined including family demographics, course of treatment, changes in clinical measures, and caregiver report for prior maltreatment victimization and perpetration. Participants (n=120) included families at-risk for future maltreatment with and without prior maltreatment history. Agency case records were linked with state administrative records of child welfare reports. Time to follow-up ranged from 13–40 months. Bivariate and multivariate survival analyses are used to model risk for a later report. During the follow-up period, 12.5% of families had a report for physical abuse or neglect. Reports of prior victimization as a child and prior perpetration as an adult were strong predictors of a report of perpetration after PCIT. Dosage of PCIT and change in clinical measures did not increase risk for a later report. PCIT can be an effective intervention for preventing maltreatment. Family history of child welfare involvement is a prominent factor in assessing risk for future involvement.

Introduction

During 2010, an estimated 5.9 million children were referred to child welfare agencies for alleged child abuse or neglect (US DHHS, 2011). An estimated 3.4 million children nationally received prevention services “designed to increase the understanding of parents and other caregivers of the developmental stages of childhood and improve their child-rearing competencies” (US DHHS, 2011, p. 88). As the most common type of service provided, over half of families receive some type of parent training following a child welfare investigation (Hurlburt, Barth, Leslie, Landsverk, & McCrae, 2007). Over the past decade, there has been a significant increase in the supply of parent training interventions with empirical support. While the general approach to parent-focused prevention has been consistent, interventions have evolved over recent years from ideology-based to evidence-based with a sharper focus and intensity on developing a specific set of skills (Institute of Medicine [IOM] and National Resource Council [NRC], 2012).

Many evidence-supported interventions now used for the prevention of child maltreatment were not initially developed for this purpose. Specifically, parent-mediated behavioral interventions such as PCIT were originally created to reduce child externalizing behavior problems, with strong evidence of success for this purpose (Brestan & Eyberg, 1998; Eyberg & Boggs, 1998). There is now support that evidence-based parenting programs such as PCIT can fill a need for services to prevent maltreatment and recidivism (Chaffin, 2011; Thomas & Zimmer-Gembeck, 2011; Urquiza & McNeil, 1996). Evidence from trials of PCIT with high-risk child welfare populations that focus on preventing maltreating behavior of the parent have shown improvements in interaction patterns and skill development on par with studies that focus on improving the child’s behavior (Chaffin, et al., 2004).

PCIT divides the intervention into two phases: Child Directed Interaction (CDI) and Parent Directed Interaction (PDI). During CDI parents are taught to engage their child in a play situation with the goal of strengthening the parent-child relationship. During PDI parents are the focus of the interaction and are taught to use specific behavior management techniques as they play with their child (Herschell, Calzada, Eyberg, & McNeil, 2002). Therapists provide live coaching to parents during these interactions with their child. PCIT has been listed as an evidence-based intervention for the prevention of physical and emotional abuse but is contraindicated as the first intervention for perpetrators of sexual abuse (California Evidence-Based Clearinghouse for Child Welfare, nd; Chadwick Center on Children and Families, 2004).

For families at risk for physical and emotional abuse, PCIT helps caregivers establish non-coercive and non-violent relationship patterns with their children (Timmer, Urquiza, Zebell, & McGrath, 2005; Urquiza & McNeil, 1996). In many cases, physical abuse occurs in the context of harsh discipline, which is more likely in children with behavior problems. This is one of the reasons PCIT has a logical application to physically abusive populations. While most of the research on PCIT in maltreating families has focused on physical abuse, neglect subtypes are often included in reporting outcomes as well. Families at risk for neglect have often failed to develop healthy attachment patterns with their children, have become disengaged and unresponsive to their child’s attempts to interact, and have adopted age-inappropriate supervision and caretaking practices for their young children (Belsky, 1993). PCIT helps parents to become more responsive to their children’s developmental needs for attentive and stimulating caregiving relationships. Despite a lack of focus on neglect prevention specifically, PCIT may be beneficial in cases where there is less risk for physical abuse.

The evidence strongly supports the notion that PCIT can improve parenting skills that effectively address child behavior problems (Thomas & Zimmer-Gembeck, 2007). However, what is less clear is whether changes in targeted practice behaviors translate to a long-term decrease in risk for child abuse and neglect. One study found that interaction patterns mediated child abuse recidivism (Chaffin, et al., 2004; Hakman, Chaffin, Funderbunk, & Silovsky, 2009). All parents in the study were referred from child welfare with a confirmed physical abuse report and report recurrence was followed over an average of two years. Among the families receiving standard PCIT, 19% had a re-report compared to 49% of the control community group. These outcomes were mediated by reductions in the negative parenting behaviors that are the target of PCIT. A follow-up examination of observed parenting behaviors from individual sessions revealed that the physically abusive parents had inappropriately negative responses to their child’s behavior at the first session (Hakman, Chaffin, Funderbunk, & Silovsky, 2009). The study found that changes to an appropriate positive parental response can occur within the first few sessions of PCIT and remain stable throughout treatment. These results provided strong initial evidence that a clinical sample consisting of parents at risk for maltreatment recidivism respond to PCIT as predicted. Further, changes in parent-child interaction patterns reduce behavior problems and risk for physical abuse.

While these findings are encouraging, replication of implementation in diverse community settings is needed to strengthen external validity. Efficacy studies compare outcomes for PCIT delivered under “controlled and favorable conditions” to services as usual in the community environment (Chaffin, et al., 2004, p. 509). This design introduces potential confounds from differences in therapist or service setting that may exist between a university-based versus agency-based intervention environment. To determine whether the outcomes from the laboratory setting could be transferred to the field, Chaffin and colleagues (2011) conducted a trial using randomized delivery within the same agency. The sample consisted of parents referred to a nonprofit community-based agency from child welfare for parenting services. This implementation study was able to replicate positive findings of clinical trials, finding a benefit in reducing recidivism for families receiving PCIT combined with a self-motivational orientation compared to a didactic parenting group.

Another randomized-controlled trial examining the prevention of maltreatment using official reports from Australia also reported positive findings (Thomas & Zimmer-Gembeck, 2011). In this study, the control group was offered PCIT after 12 weeks so the comparison of official notifications focused on those who completed PCIT to those who did not. Seventeen percent of the 45 families that completed PCIT were reported to child welfare compared to 43% of those who did not complete. This study accepted families who were referred from child welfare and other professionals, as well as caregivers who self-referred for high parenting stress due to child behavior problems. Among those referred by child welfare, 47% of those who completed PCIT had a later report compared to 73% of those who dropped out.

The Present Study

There is currently little research regarding the implementation of PCIT in community settings such as mental health or family services agencies (Budd, Hella, Bae, Meyerson, & Watkin, 2011; Lyon & Budd, 2010; Pearl, Thieken, Olafson, Boat, Connelly, Barnes, & Putnam, 2012). The current study is a follow-up of a community-based implementation of PCIT (Lanier, Kohl, Benz, Swinger, Moussette, & Drake, 2011). The previous analysis found significant improvements in clinical outcomes including parenting stress and child behavior problems among families who completed PCIT. Thus, the purpose of this study was to determine if improvements in family dynamics found during the course of treatment translate to the prevention of maltreatment. This study examined rates of child maltreatment in a single-group observational design using prospective, longitudinal analysis allowing for the examination of event history to better understand the complex interplay between prior risk, intervention, and later outcomes.

Method

The data used for this analysis came from a combination of agency clinical case records and linked child welfare administrative records. All procedures were approved by Washington University Human Research Protection Office. Over the two year implementation period, 120 families completed at least one session of PCIT. Based on the existing agency design, some PCIT participants were referred by state child protective services (CPS) (14%) while referrals from other agencies, other services within the implementing agency, self-report, and friends provided the remainder of the participants. The program was advertised via brochure as a no-cost voluntary evidence-based coaching service to help parents with children demonstrating behavioral and emotional problems. Families received PCIT either in their home or at the agency based on family preference. The first study (Lanier, Kohl, Benz, Swinger, Moussette, & Drake, 2011) found no difference in families across treatment location at the onset of treatment on demographic measures or baseline assessments. In an analysis of those participants who completed the entire PCIT program (n=37), families receiving agency-based PCIT had a more rapid decline in PSI scores and only those receiving PCIT in the agency had a significant decline in BASIS scores. No differences were found for child behavior or parent functioning. These findings suggest that parenting stress and mental health may improve more quickly for those families who elected to receive PCIT at the agency. However, advantages for in-home PCIT such as the ability to observe child behaviors in the home environment and develop parenting skills in vivo were also considered.

Participants

Caregivers ranged in age from 13 to 66 (M = 36.0, SD = 11.0) and 90% were female. Fifty-one percent were Caucasian, 42% were African-American, and 7% were other races including Asian, Indian, American Indian, and biracial. The median self-reported annual household income was $12,000 (range $0 to $120,000). Caregivers with more than one child in the home selected one target child of an appropriate age for the intervention. The target child’s age at first session ranged from 2 to 13 years (M = 6.6, SD = 2.8) and 36% were female. The target child age for PCIT is 3–6 years. Similar to the Chaffin, et al., 2004 adaptation for maltreatment prevention, this study included older children and the intervention was adapted to include developmentally appropriate activities.

Clinical Measures

The agency collected the following measures as part of their intake procedure for each family that initiated treatment. Clinical measures were collected during the first visit (pre-intervention), between CDI and PDI phases (mid-intervention), and at the completion of PCIT (post-intervention). The GAF score was given by the therapist as part of the psychosocial assessment and all other clinical measures were completed as written self-report assessments by the participant.

Parenting Stress Index-Short Form (PSI-SF; Abidin, 1995)

The PSI-SF was completed by the parent and measured the level of stress experienced by parent-child dyads. The PSI-SF contains 36 items rated on a 1–5 Likert Scale (Strongly Agree, Agree, Not Sure, Disagree, Strongly Disagree) and measured parental distress, and parent-child dysfunctional interaction. Examples of items were “I feel trapped by my responsibilities as a parent” and “My child seems to cry or fuss more often than other children”.

Global Assessment of Functioning (GAF; APA, 2000)

Based in the DSM-IV-TR (APA, 2000), the Axis V GAF score provides a subjective rating of the psychosocial and occupational functioning of the participants by a clinician. The numeric scale ranges from 0 to 100 and a guide to the measure functioning in ten equivalent ten-point increments is provided in the DSM-IV-TR. Examples include “71–80 If symptoms are present, they are transient and expectable reactions to psychosocial stressors; no more than slight impairment in social, occupational, or school functioning, 61–70 Some mild symptoms or some difficulty in social, occupational, or school functioning, but generally functioning pretty well, has some meaningful interpersonal relationships.”

There have been inconsistent findings regarding the reliability and validity of the GAF scale yielding a questionable research base in light of its widespread usage (Aas, 2010). While studies have confirmed the reliability and validity of the GAF in university-based populations (Hilsenroth, et al., 2000) others have found lower inter-rater reliability in clinical contexts (Vatnaland, Vatnaland, Friis, & Opjordsmoen, 2007). There were no attempts to assess the reliability of the GAF scale in the current sample.

Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999)

The ECBI is a widely used scale to measure behavior problems in children containing two subscales: the Intensity Scale and the Problem Scale. The Intensity Scale was comprised of a parent’s report of the frequency that their child engages in 36 behavioral problems on a seven-point Likert scale (1 Never, 2–3 Seldom, 4 Sometimes, 5–6 Often, 7 Always). Examples include “Sasses adults,” “Yells or screams,” and “Is easily distracted.” The values across the 36 behaviors were added together to calculate an intensity score ranging from 36 to 252. The parent was then asked if each behavior is “a problem for you” and indicated yes or no (0 or 1) for each of the 36 behaviors to measure whether or not the given behavior was currently a problem. These are also added to create a problem score ranging from zero to 36 to indicate the number of behaviors that are problems regardless of frequency. Children rated above 132 on the Intensity Scale or 15 on the Problem Scale are in the “clinical range” for behavior problems. In this analysis, the raw scores were converted to standardized scores.

Behavioral and Symptom Identification Scale (BASIS-32; Eisen, Dill, & Grob, 1994)

The BASIS-32 scale was developed to assess client functioning and mental health outcomes and was used in this study assess caregiver functioning. It was originally used for inpatient public mental health programs and ultimately adapted to self-administered form. Respondents rate perceived difficulties in five areas of symptoms and behavior problems using a 5-point Likert scale ranging from zero (no difficulty) to four (extreme difficulty). The five subscales measure the following: relation to self/others, depression/anxiety, daily living skills, impulse/addictive behavior, and psychosis.

Child welfare contact

Agency data were matched with state child welfare administration data on the report history for the primary caregiver. If available, data were matched using a standard state identification number on file (from income maintenance, youth services, or child protective services), otherwise we matched on first name (4 letters), last name (4 letters), and date of birth including year. The match produced victim and perpetrator records of child abuse and neglect reports, worker findings, and case conclusions. The resulting list included information for the caregiver participant regarding all child welfare contact history as either a victim or perpetrator reported in the state.

Data Analysis

Child Welfare Service History

Data analysis was conducted using SAS 9.2 (SAS Institute, Inc., Cary, NC). The main outcome of interest was the event of a maltreatment report that occurred following PCIT. Enrollment in the program for this study occurred from February 2008 to May 2010. Agency data were matched with state child welfare administrative records in June 2011. Therefore, the time to follow-up ranged from 13–40 months. While it is impossible to ensure that secondary administrative records are complete accounts of service history, this administrative data set has been used previously to follow families (citation omitted) and prior researchers were able to confirm fewer than five percent of the identified families to have moved out of the region over a 7 year period. Given the confluence of empirical research suggesting that the “substantiation” label is not a useful predictor of risk for later harm or recidivism, primary comparisons were made between those families with a later report and those without a report regardless of case disposition (IOM and NRC, 2012; Kohl, Jonson-Reid, & Drake, 2009). However, the substantiation rate will be reported for comparison with other studies and as a marker of burden on child welfare and legal systems.

Factors Predicting a Later Report

Because families were at risk for a report for different periods of time following PCIT, survival analysis was used to analyze the rate of report occurrence over time. Exploratory univariate survival analyses were used to compare estimates of the survivor function (probability of not having a report at a specific time) across demographic variables, treatment factors, clinical measures, and child welfare history. For categorical variables, Kaplan-Meier survival curves were created and the log-rank test of equality across strata was used to test significance. For continuous variables the Wald chi-square test from a Cox regression model with a single predictor was examined. Multivariate Cox proportional hazards regression modeling, a form of multivariate survival analysis, was used to model risk for report controlling for covariates.

There is some disagreement in the literature regarding the best way to model pre-post change in clinical measures predicting survival for a distal outcome. We chose to use the difference score of these measures as a predictor as well as the baseline score as a covariate in the final survival model (Singer & Willet, 2003). To assess for potential issues of temporal sequencing, preliminary analysis indicated that no events (maltreatment reports) occurred between the first and last session when clinical predictor variables were collected. We report the hazard ratio (HR) from the Cox model for key predictors of interest. The HR is similar to an odds ratio and expresses the ratio of risk per unit time.

Imputation of Missing Data

Clinical measures not collected as a result of attrition present an analytic challenge in this study due to large amounts of missing observational data threatening internal validity (Kristman, Manno, & Cote, 2005). Previously, researchers would often delete cases with missing values or use other crude ad hoc imputation strategies that are more likely to yield biased results, result in a loss of statistical power, and have less theoretical support (Enders, 2010; Groenwald, Donders, Roes, Harrell, & Moons, 2012). Given that the probability of missing data in this study is related to other observed covariates, they are assumed to be missing at random (MAR) and more advanced techniques can be employed (Little & Rubin, 1987). In addition to understanding the mechanism, the pattern of missing data is also important in determining the approach to dealing with missing data. Missing longitudinal data with a monotone, non-arbitrary pattern allows for the use of a more flexible and theoretically-valid method that applies a sequential approach to imputation (Li, Yu, & Rubin, 2012). While there is no established cutoff for the proportion of data that can be missing, simulation studies have demonstrated multiple imputation techniques to be acceptable with 40% (Kristman, Manno, & Cote, 2005), 50% (Scheffer, 2002), and 60% missing data (Rubin & Schenker, 1986). Ensuring the appropriate strategy is applied given the mechanism of missingness is of greater importance than the amount of missing data.

A multistage imputation method was used to replace missing observational data in this analysis (Durrant & Skinner, 2006; Little, 1988). First, missing demographic (<1% missing) and baseline clinical measures (23% missing) were imputed using the Markov Chain Monte Carlo (MCMC) approach to establish a monotone missing data pattern using SAS PROC MI to generate five data sets. Next, predictive mean matching method was used to impute mid (62% missing) and post (67% missing) time points (Yuan, 2000). This method ensures that imputed values are more reflective of true observed values (Horton and Lipsitz, 2001). Imputation models included the outcome variable, select auxiliary variables, and all variables used in the subsequent analysis including significant predictors of missingness. The imputed data sets were then analyzed separately and results combined with Rubin’s rules using SAS PROC MIANALYZE (Rubin, 1987).

Clustering

Participants were clustered by assignment to therapist (n=9) and naturally by geography within zip codes (n=42). Intraclass correlations (ICC) were calculated for the main outcome of interest to estimate the variance associated with the therapist or zip code cluster in order to examine the need for multilevel random-effects modeling. Conceptually, clustering by therapist is important to explore due to differences based on different levels of training, clinical skill, or implementation of evidence-supported interventions (McDiarmid Nelson, Shanley, Funderbunk, & Bard, 2011). The ICC for null-models predicting a later report when considering variance at the therapist and the zip code level were equal to zero suggesting no need to adjust for clustering.

Results

Child Welfare Service History

Agency data were linked with state child welfare administration records on the report history for the primary caregiver. There were 15 total families (12.5%) who received at least one report after the first PCIT session where the parent was named as the alleged perpetrator. Three of the 15 reports were coded as both physical abuse and neglect, seven were neglect only, and five were physical abuse only. There were no reports of sexual abuse. For those reported, the average time to first report following the first session was 17 months and ranged from 3 to 27 months (from last session M = 13.9, SD = 7.9 months). Eleven families had one report and four had 2–4 reports. Of these reports, one family had two substantiated reports and another had one substantiated report yielding a substantiated report rate of 1.6% for the sample. Among the 27 families who had a prior perpetration report, a recidivism prevention subgroup, 10 (37%) had a report following PCIT.

Factors Predicting Later Report

Demographic variables

Table 1 provides a comparison of families with a report and those with no later report on key predictor variables and provides results from survival analyses comparing the equality of survivor functions predicting risk for report at a given time. Risk for report did not differ significantly by race, gender, or age of the parent or the child. Income was a significant predictor of risk (Wald χ² = 5.39, p < .05). Families with a later report had a lower self-reported annual income (median = $5,000) compared to those with no report (median = $14,000).

Table 1.

Comparison of Families With and Without a Later CPS Report by Family Demographics, CPS History, and Course of Treatment in PCIT Services

	Report (n=15)	No Report (n=105)	Significance
Race (White)	6 (40%)	55 (52%)	NS
Race (African-American)	8 (53%)	43 (41%)	NS
Parent gender (female)	15 (100%)	93 (89%)	NS
Child gender (female)	7 (47%)	36 (35%)	NS
Parent age	33.3 (11.7)	36.3 (10.9)	NS
Child age	6.4 (3.4)	6.6 (2.8)	NS
Income (log $1,000)	1.5 (1.5)	2.4 (1.3)	*
Parent history of maltreatment victim	5 (33%)	5 (4.8%)	***
Parent prior report as perpetrator	10 (66%)	17 (16%)	***
Prior substantiated report	4 (27%)	5 (5%)	***
Total prior reports	1.6 (2.1)	0.4 (1.2)	***
CPS referral to PCIT	3 (20%)	14 (13%)	NS
Home treatment	6 (40%)	47 (45%)	NS
Number of PCIT sessions	11.1 (6.8)	9.9 (6.8)	NS
CDI dropout	7 (47%)	58 (55%)	NS
PDI dropout	3 (20%)	15 (14%)	NS
Completed PCIT	5 (33%)	32 (31%)	NS

Note. Significance test results of univariate survival analysis reported for log-rank test of equality across strata for categorical variables and Cox proportional hazard regression for individual continuous predictors

^*p < .05,

^**p < .01,

^***p < .001,

NS = not significant at p < .05

Course of treatment

Risk for report was equal for families who chose to receive treatment at home versus the standard office-based PCIT (log-rank χ² = 1.14, p = .29). No difference in risk was found in the course or dosage of treatment. Risk for report was not related to number of sessions (Wald χ² = 0.84, p = .84), as families with a report received a similar number of sessions (M = 10.5, SD = 6.8) as non-reported (M = 9.8, SD = 6.9). The risk for report was the same when comparing those who dropped out during CDI (log-rank χ² = 0.14, p =.71), during PDI (log-rank χ² = 0.18, p = .67), and completed PCIT (log-rank χ² = .01, p = .95).

Clinical measures

Table 2 provides results for the baseline clinical measures and the change in scores from baseline to the end of PCIT for reported compared to non-reported families as well as the results of the survival analysis examining risk for report. There was no statistically significant difference in risk for report when examining baseline or change scores for PSI, ECBI-Problem, ECBI-Intensity, or BASIS-32. Parents with a later report had a lower baseline GAF (M = 58.3, SD = 7.4) compared to those without a later report (M = 64.4, SD = 9.3) and had a significantly different risk for report (Wald χ² = 6.05, p < .05). Change in GAF score was a significant predictor of risk (Wald χ² = 4.46, p < .05) as those with a later report had greater change in their GAF score (M = 11.6, SD = 7.3) compared to those without a later report (M = 7.6, SD = 6.8). One practically significant trend should be noted. Although not statistically significant (Wald χ² = 2.02, p = .15), the average baseline PSI score for reported families (M = 108.5, SD = 22.0) may represent a higher level of parenting stress compared to families without a report (M = 98.4, SD = 23.0) and confer a greater risk for report.

Table 2.

Comparison of Baseline and Pre-Post Change in Imputed Clinical Measures

	Report (n=15)	No Report (n=105)	Significance
Parenting Stress Index (PSI)
Pre	108.5 (22.0)	98.41 (23.0)	NS
Change	−19.5 (27.8)	−15.3 (21.0)	NS
Global Assessment of Functioning (GAF)
Pre	58.3 (7.4)	64.4 (9.3)	**
Change	11.6 (7.3)	7.6 (6.8)	*
Eyberg Problem Inventory
Pre	63.8 (10.0)	63.4 (12.3)	NS
Change	−6.4 (13.3)	−8.7 (11.8)	NS
Eyberg Intensity Inventory
Pre	63.5 (8.2)	62.3 (12.2)	NS
Change	−8.2 (13.3)	−3.7 (14.4)	NS
BASIS-32
Pre	1.2 (0.4)	1.0 (0.7)	NS
Change	0.7 (1.3)	−0.7 (0.9)	NS

Note. Significance test results of univariate survival analysis reported for Cox proportional hazard regression for individual continuous predictors

^*p < .05,

^**p < .01,

NS = not significant at p < .05

Child welfare history

Figure 1 shows the bivariate survival curves comparing caregivers with and without a history of child victimization and curves comparing those with and without a prior report for maltreatment perpetration. Families with a report of maltreatment following PCIT had quite different histories of child welfare contact. Analyses comparing hazard functions indicated that probability of a report was increased by a record of victimization as a child (log-rank χ² = 28.6, p < .001), a record of prior perpetration as an adult (log-rank χ² = 25.0, p < .001), a prior substantiated report (log-rank χ² = 15.3, p < .001), and a higher total number of prior reports (Wald χ² = 12.7, p < .001). There was not a significant increase in risk (log-rank χ² = .31, p = .57) for families referred to the agency for services for PCIT by child protective services.

Figure 1.

Bivariate survival curves examining risk for a maltreatment report following PCIT examining the effect of caregiver victimization as a child and prior perpetration prior. Vertical lines mark one and two years following PCIT. Scales of the y-axes differ.

Given the trend in prior child welfare system contact predicting future reports, we examined bivariate differences in caregivers based on prior contact status using chi-square, t-tests, and Wilcoxon rank sums tests. Caregivers who had reports of victimization as children had significantly lower income (median = $0; Z = −3.45, p < .001) compared to those without a child report (median = $14,000). In fact, six of the 10 caregivers with child victim reports self-reported zero annual income, with a maximum of $11,000. There were no significant differences in clinical measures or family demographics. Caregivers with a child victimization report attended significantly fewer PCIT sessions (Z = −1.9, p < .05) and none completed treatment. A similar trend was found for the 27 caregivers with a report for prior perpetration (eight of these were also child victims). There were no differences in baseline demographic or clinical measures except for lower reported income (Z = −2.79, p < .01). However, this group was equally likely to complete PCIT and received an equal number of sessions.

Multivariate survival analysis

Cox regression modeling was used as the multivariate approach to estimate risk for a report at a given time including baseline demographic and clinical values as covariates. Since report as a child victim and report as an adult perpetrator prior to PCIT were strongly associated (p < .001, Fisher’s Exact Test), these variables were entered into separate models to reduce multicollinearity. Results reflected the findings at the bivariate level. Victimization as a child (hazard ratio [HR] = 38.8, p < .001) and report as a prior perpetrator of maltreatment greatly increased the estimated hazard of a future report (HR = 16.9, p < .001). Increased baseline parenting stress measured by PSI (HR = 1.07, p < .01) and lower GAF (HR = .89, p < .01) remained significant in multivariate models.

Discussion

Among families served by a community agency implementing PCIT with the goal of reducing the risk for later maltreatment, 15% had a later report to CPS after an average follow-up of 28 months. Among families who had a prior report, a recidivism subgroup, 37% had a later report. While the rate of substantiation has limited utility as a marker for future risk, it does give an indication of burden on the child welfare and legal systems. In this study only two families (1.6% of the sample) had a later substantiated report following treatment. The lifetime economic burden for just one year of confirmed cases of maltreatment is estimated to be $124 billion, or over $200,000 per child survivor (Fang, Brown, Florence, & Mercy, 2012). If this intervention has been successful in preventing maltreatment, the benefits extend beyond the family suggesting a sensible social investment.

Given the lack of a randomized comparison group, it is not possible to determine whether or not these rates are higher or lower than what would be expected had these families not received PCIT. A study of recidivism among families in the same geographic area as the current study found that more than half of cases returned to the attention of child protective services within 4.5 years, with lower tract income associated with even higher levels of recidivism (Drake, Jonson-Reid, Way, & Chung, 2003). This recidivism rate is also similar to the 47% rate found in the Chaffin, et al., 2011 study of child welfare-involved parents receiving PCIT without the motivational module. Given the high-risk nature of the sample and the amount of time followed, the rates of maltreatment are relatively low and can perhaps serve as a signal of clinical success although empirical causality cannot be inferred.

However, there was not a relationship between dosage of PCIT (measured in terms of number of sessions or program completion) or changes in clinical outcomes and risk of report. Families who dropped out early were equally likely to have a later report as those who completed the entire course of PCIT. One interpretation could be that dosage of PCIT was determined by the family based on their perceived level of need or pre-existing parenting skill. Families who were able to quickly understand the concepts or felt they got what they needed from the program may have self-selected out of the program earlier while those who perceived more need over a longer period of time stayed for the prescribed duration. PCIT is designed to allow parents to progress at their own pace, achieve mastery in a set of skills, then move on to the next phase. For example, a parent might not achieve PDI mastery per the PCIT criteria although they could have developed these skills then dropped out before clinician assessment could document their progress.

Parents with a later report were assessed by the clinician to be lower functioning at baseline but also had greater improvement in functioning over the course of PCIT. Essentially, therapists were able to identify which parents were more likely to have a future report with a standard but subjective assessment of functioning at baseline. However, by the end of treatment they assessed reported and non-reported parents to have a similar average GAF. This might reflect some bias on the part of the therapist that over-valued parent-child interactions in their assessment of global functioning after weeks of observing progress in this area. Or, the gains in the functioning that occurred during treatment regressed to baseline levels over the course of the follow-up period. Self-reported parenting stress was also higher at baseline among the subgroup of families who would have a later report. While they did have an overall drop in parenting stress, the drop was equivalent to those families without a later report. To be effective, perhaps parenting stress would have to decrease by a magnitude that would make high-stress caregivers “catch-up” with those reporting lower stress by the end of the intervention. Given that maltreatment is determined by a complex array of multilevel factors, it may also be the case that the measured changes in clinical outcomes were too small to account for a change in risk for later maltreatment report.

These findings are not consistent with what would have been predicted by previous studies finding a mediating relationship between change over the course of treatment and later report (Chaffin, et al., 2004). In this study, all families on average responded to the intervention as predicted. There was a decrease in parenting stress and improvements in child behavior and parent functioning regardless of later report. This finding could be related to low validity and reliability of clinical measures collected in the field, low power to detect small effects due to a relatively small sample size, high attrition yielding biased estimates in the imputation strategy, unmeasured effects of the intervention “wearing off” over time, or a lack of a true relationship between changes in measured constructs and the outcome of interest.

In order to put these findings in context, comparison to other studies is useful. One key modification of the PCIT studies conducted by Chaffin and colleagues (2009, 2011) was the inclusion of a short, self-motivational orientation intervention condition. The addition of this brief motivational component prior to PCIT was justified by the nature of the intervention (active, not passive), context of service delivery (coercive child welfare system), and the population of interest (caregivers with low self-motivation). The addition of this self-motivation component improved retention and subsequent reduction of child welfare reports. At follow-up, 29% of families receiving the combined PCIT and motivation component had a later report, compared to 47% of PCIT alone, and 41% for services as usual. After adjusting for bias due to differences in risk exposure in the groups, neither PCIT by itself nor the motivational component alone provided a decreased risk for physical abuse compared to services as usual. A package implementation of PCIT with a motivational component may be critical to the translation and application of parenting skills to the home environment in order to prevent physical abuse over a long period of time.

An important contribution of the current study was examining whether or not treatment location was associated with later child welfare reports. The choice of receiving PCIT in the home or at the agency did not have an impact on risk for future child welfare reports. Given that these families are comparable on all baseline measures, this finding suggests that the protective effects of PCIT in preventing maltreatment are not diminished by offering the intervention in the home setting. Several benefits to home-based treatment may exist such as the ability to teach the skills in the environment that they are intended to be used, potentially making the transition to application smoother. More research is needed to examine the advantages and disadvantages of offering an option for home-based services for different families as well as different phases of treatment.

Caregiver History in Predicting Later Outcomes

Consistent with the previous literature, we found a strong effect of prior child welfare history on future maltreatment. Hazard ratios suggest that caregivers with a report of victimization as a child were almost 40 times more likely and those who perpetrated maltreatment prior to PCIT almost 20 times more likely to have a report following PCIT. Effect sizes of this magnitude reflect the strength of this relationship. This finding may also help explain the lack of a “treatment effect” from PCIT in predicting a later report. Nine of the ten caregivers that had an official record of victimization as a child had a later report for perpetration of abuse or neglect of their own child. This unanticipated finding adds to the literature examining intergenerational transmission of parenting behaviors. Of the 27 caregivers with a report of maltreatment perpetration prior to PCIT, 10 had a report after treatment. Among the 93 caregivers who did not have a previous report for perpetration only five had a later report.

This finding suggests that to have an equivalent effect, prevention strategies for decreasing risk for recidivism must be different than universal or selective prevention interventions. Also, these findings demonstrate that the level of risk for families requiring tertiary services is much higher than those families who have not come to the attention of the child welfare system. The finding that families with a report of child victimization as well as parent perpetration had much lower self-reported incomes supports this notion. Perhaps effects from the same intervention delivered the same way should be expected to accrue at a slower rate for families facing higher risk from extreme levels of poverty.

While the conclusions are less clear, there is evidence to support the notion that maltreatment has an intergenerational exchange (Berlin, Appleyard, & Dodge, 2011; Ertem, Leventhal, & Dobbs, 2000; Widom, 1989). Individuals who were abused or neglected as children are more likely to continue the pattern of negative parenting and thus have a greater chance of being reported for child abuse or neglect if they become caregivers (Newcomb & Locke, 2001). This perspective suggests a transmission of parenting style and disciplinary practices from parent to child as well as the continuation of a high-risk family environment. The strong effect of child victimization in this study, regardless of exposure to an evidence-supported intervention, suggests a need for screening for not just child maltreatment perpetration history but also victimization. Also, additional components or services that focus on helping the caregiver first attempt to resolve prior trauma might increase the effectiveness of parenting interventions.

Strengths and Limitations

The primary limitation of this study is the uncontrolled observational design lacking a randomized comparison group. While this certainly limits the ability to make causal claims regarding treatment effects, this study focuses more on external validity by examining the effectiveness of PCIT delivered in a real-world setting (Jensen, Hoagwood, & Trickett, 1999). A key strength of this study is the combination of families receiving the same preventive service with different child welfare contact histories. Some families had a prior report of maltreatment and thus PCIT was applied with the goal of preventing recidivism. Other families were receiving services based on a professional referral or self-referral and had never had a report of maltreatment. Despite the level of risk, the families received PCIT services in the same way. Often in research studies, families are comprised of one level of prevention risk based on sampling criteria. Given the structure of public preventive child welfare and mental health services delivered in many communities, this implementation scheme likely reflects actual delivery of services. Another aspect of the study that has mixed strengths and limitations is the use of administrative records to measure child abuse and neglect. The additional inclusion of a self-report measure of maltreating behavior would have been beneficial to triangulate with administrative records. However, administrative records are not as susceptible to reporter and recall bias.

Implications for Policy and Practice

These findings have theoretical implications for the future development and delivery of services for families at different levels of risk. When applying an intervention to the prevention of child maltreatment, the theory of change of the model should be built to deliver sufficient protective effect to buffer the existing level of risk the caregiver is facing. PCIT is an operant parent training model informed primarily by social learning theory using coaches and live training to strengthen the relationship between the parent and the child and teach effective parenting skills. When first introducing PCIT as an intervention for physically-abusive parents, Urquiza & McNeil (1996) identified several considerations of the model and population that would limit the effectiveness. If parents already have strong parent-child relationships or have the ability to demonstrate non-coercive parenting skills, the transmission of new knowledge or the opportunity to develop a strong relationship with the child might not be useful. Other issues such as parental substance abuse, cognitive ability, motivation, and cultural differences might also hinder the effectiveness of PCIT.

One critical factor facing the implementation of PCIT with this population is that it was not designed to be a comprehensive intervention to adequately address all of the issues faced by maltreating families (Urquiza & McNeil, 1993). The breadth and depth of stressors faced by families was part of the justification to add additional components in the Chaffin PCIT trials as it was “not clear that a parenting intervention alone is sufficient to reduce future abusive behavior” (Chaffin, et al., 2004, p. 501). Although our current study included therapist assessment of child welfare service history, all families in our study received PCIT as a stand-alone program. In cases of child welfare involvement, there was communication with the case worker regarding the progress of the caregiver. Future implementations of PCIT in community-based agencies might consider developing strategies to include additional components to complement parent training and/or develop connections with other comprehensive therapeutic services. However, a critical finding from Chaffin, et al., 2004 was that efficacy was not enhanced by adding concurrent services and may have even diluted or distracted the potential effects of PCIT. Successful integration of evidence-supported interventions such as PCIT in the current system of care will require more research to find the right balance and ordering of services to maximize efficiency for the system and the consumer.

This study provides further justification for the use of PCIT in a population of families at risk for physical abuse or neglect. Agencies and practitioners delivering PCIT in a community setting, accepting clients from a variety of report sources, would benefit greatly from gaining a full history of child welfare involvement. In many cases this information will be exchanged formally if the report originates from a child welfare agency. In cases of referrals from schools or other organizations or from self-referrals, clinicians must be sure to ascertain a full history of child welfare involvement including the caregiver’s own history of victimization. While this is surely an uncomfortable discussion with questionable reliability, the clinical usefulness in developing a treatment plan and linkages with other services is essential.

To improve the effectiveness of PCIT implemented as a maltreatment prevention programs, this study may also suggest the need to augment the manualized intervention to better meet the needs of each family. Prevention strategies such as PCIT deployed in community settings should be complemented with services that reflect the powerful array of risks facing many families. This approach is more consistent with a public health model of prevention and has been incorporated into multilevel program models such as Triple P (Sanders, 2008). Multidimensional approaches to maltreatment prevention are also more consistent with theory. The addition of a motivational intervention component would likely improve attrition and uptake of parenting strategies. In this study, risk factors such as poverty and childhood victimization were strongly predictive of maltreatment after PCIT. Risk may have been diminished by participation in PCIT but not to the degree to prevent a future report. Agencies providing a stand-alone parenting program such as PCIT may be extremely effective at teaching new parenting skills or helping the caregiver build a stronger relationship with their child, but may not be providing the resource or relationship that is needed to prevent the event of abuse or neglect in the future.