Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Jun 24.
Published in final edited form as: Behav Modif. 2012 Nov 26;37(3):356–377. doi: 10.1177/0145445512465307

Barriers to Success in Parent Training for Young Children With Developmental Delay: The Role of Cumulative Risk

Daniel M Bagner 1, Paulo A Graziano 1
PMCID: PMC4479170  NIHMSID: NIHMS700184  PMID: 23188886

Abstract

The purpose of this study was to examine the effect of cumulative risk on dropout and treatment outcome in parent training. Participants were 44 families of young children (mean age of 49.59 months) who presented with elevated externalizing behavior problems and developmental delay or borderline developmental delay. All families were offered to receive Parent–Child Interaction Therapy (PCIT), an evidence-based, behavioral parent-training intervention, at a hospital-based outpatient clinic. Cumulative risk was calculated as a sum of risk variables, including socioeconomic disadvantage (poverty, low maternal education), family structure (single-parent household), and maternal risk characteristics (minority status, lower intelligence, and parental distress). Families with higher cumulative risk scores, especially those with three or more risks, were more likely to drop out of treatment and display diminished treatment response in child behavior and parenting skills compared with families with lower cumulative risk scores. However, only two individual risk factors (i.e., minority status and family structure) predicted dropout, and one individual risk factor (i.e., maternal education) predicted outcome. These findings suggest that it can be useful to conceptualize risk factors as having a cumulative, in addition to individual, influence on parent-training interventions for children with developmental delay and have significant implications for clinical practice. It is important for clinicians to regularly assess for risk factors, and future research should examine ways in which clinicians can improve retention and outcome of parent training in the presence of multiple risk factors.

Keywords: externalizing behavior problems, developmental delay, parenting, risk factors, treatment outcome

In early childhood, externalizing or disruptive behavior problems are highly prevalent and represent the most common referral of young children to mental health clinics (Keenan & Wakschlag, 2000). In addition, these problems are impairing and persist over time (Briggs-Gowan, Carter, Bosson-Heenan, Guyer, & Horwitz, 2006; Keenan et al., 2010), particularly among children with developmental delay (Baker et al., 2003). Behavioral parent training is an evidence-based treatment for young children with externalizing behavior problems (Eyberg, Nelson, & Boggs, 2008), and recent evidence provides support for the use of parent training with children with developmental delay and behavior problems (Bagner & Eyberg, 2007; McIntyre & Abbeduto, 2008; Roberts, Mazzucchelli, Studman, & Sanders, 2006). In addition to the overall effectiveness of parent-training interventions, several common treatment components (e.g., increasing positive parent–child interactions, promoting consistency, and use of time out) are associated with large effect sizes (Kaminski, Valle, Filene, & Boyle, 2008). Despite the substantial evidence base, however, parent-training interventions may not be equally effective for all families (Lochman, 2000).

One reason for unsuccessful parent training is that as many as 50% of families do not complete treatment (Prinz & Miller, 1994), and families that drop out prematurely have been shown to have poorer long-term outcomes than treatment completers (Boggs et al., 2004). Although limited, some recent research has investigated predictors of dropout from parent-training interventions, although findings have differed across studies. For example, some research suggests demographic risk variables, such as low socioeconomic status (SES) and minority status, predict dropout (Fernandez & Eyberg, 2009; Lavigne et al., 2010), whereas others found parent psychological distress (e.g., parenting stress), but not demographic variables, were associated with dropout (Werba, Eyberg, Boggs, & Algina, 2006). A meta-analysis of predictors of parent training yielded small or insubstantial effect sizes for individual predictors of dropout examined (Reyno & McGrath, 2006), suggesting further research in this area is needed.

For families that complete treatment, there are also conflicted findings on the effect of risk on parent-training outcome. Some research using latent growth curve modeling suggests more disadvantaged and stressed families have better outcomes following the Incredible Years (IY; Beauchaine, Webster-Stratton, & Reid, 2005) and Family Check-Up (FCU; Gardner et al., 2009) interventions. However, early research on the IY program suggests that demographic risk variables, including low SES and single-parent status, as well as stressful life events, are associated with worse child behavior outcomes following treatment (Webster-Stratton, 1985, 1990). More recent meta-analyses yielded similar findings in that parent-training interventions are less successful for more disadvantaged and stressed families (Lundahl, Risser, & Lovejoy, 2006; Reyno & McGrath, 2006).

Collectively, there is some discrepancy as to which risk factors are associated with dropout and outcome in parent-training interventions, and most of the previous research examined the effect of each risk factor independently. The examination of each risk factor independently can be problematic, as most risk factors occur together and not in isolation. For example, children from ethnic minority groups are more likely to live in single-family homes and below the poverty line (Platt, 2010). Similarly, parenting stress is higher among single parents (Anderson, 2008). Given the significant overlap between these risk factors, collinearity may be an issue when examining individual risk factors together. In addition, the magnitude of the effect of each risk factor alone is relatively small (Amato & Keith, 1991; Reid & Crisafulli, 1990; Reyno & McGrath, 2006) but can be substantially stronger when examined as part of a cumulative risk index (Forehand, Biggar, & Kotchick, 1998). Furthermore, the use of a cumulative risk index can help identify the threshold at which these factors become detrimental to success in parent training, which can have important clinical implications. For example, the use of a cumulative risk index when screening families in clinical settings may be helpful in identifying at-risk families (i.e., those that may be more likely to dropout or not respond well to standard treatment). Such information may then be used to enhance the delivery of standard parent training and/or include adjunctive treatments. Therefore, there are strong theoretical, in addition to methodological, reasons to examine the additive effect of these risk factors as a cumulative risk index.

In the developmental literature, evidence supports the use of cumulative risk indices in that more risk factors lead to worse child (Appleyard, Egeland, van Dulmen, & Sroufe, 2005; Trentacosta et al., 2008) and adolescent (Prinstein, Boergers, & Spirito, 2001) behavioral outcomes. For example, Forehand et al (1998) demonstrated that the number of risk factors, regardless of type, predicted short- and long-term adolescent adjustment difficulties, and that there was a steep escalation in adjustment difficulties when the number of risks increased from three to four. Consistent with earlier work (Rutter, 1979), these findings suggest that there may be a “trigger point” at which the accumulation of risk factors has a detrimental effect, which may help identify families that are most likely to not complete or not benefit from parent training.

In addition, research has demonstrated that a cumulative effect of adverse childhood experiences (e.g., abuse domestic violence) is associated with the leading causes of death (e.g., cancer, heart disease; Felitti et al., 1998) and depressive disorders (Chapman et al., 2004) in adulthood, suggesting problems persist and may lead to difficulties participating in treatment.

Despite the strong rationale, little research has examined the effect of cumulative risk on dropout and outcome from parent-training interventions. To our knowledge, only Gardner et al. (2009) used a cumulative risk index, which was not associated with outcome following the FCU and was not examined as a predictor of dropout. It is important to note that families enrolled in this study were recruited at the Women, Infant, and Children (WIC) program, a national program for predominately low-income families, resulting in decreased variability in the cumulative risk scores (e.g., 80% of families were below the poverty line).

No study to date has examined the effect of cumulative risk on behavioral parent training among families with a child with developmental delay. Externalizing behavior problems are considerably more prevalent among children with developmental delay (Dekker, Koot, van der Ende, & Verhulst, 2002; Emerson, Robertson, & Wood, 2005), and parents of children with developmental delay also display higher rates of parenting stress than parents of children without developmental delays (Baker, Blacher, Crnic, & Edelbrock, 2002; Baker et al., 2003). In addition, children with developmental delay are more likely than their nondelayed peers to experience a variety of other risk factors, including low maternal education (Sonnander & Claesson, 1999), poverty (Msall, Bier, LaGasse, Tremont, & Lester, 1998), and single-parent status (Koskentausta, Iivanainen, & Almqvist, 2007). Therefore, families of children with developmental delay represent an ideal population to examine the role of cumulative risk on parent training. Given these elevated risk factors among families of children with developmental delay, it is particularly important to identify the threshold at which the number of risk factors interferes with treatment completion and success. The feasibility of examining cumulative risk with this population is also evident in prior research demonstrating the utility of using a cumulative risk index to predict various developmental outcomes (Burchinal, Roberts, Hooper, & Zeisel, 2000; Sameroff, Seifer, Barocas, Zax, & Greenspan, 1987).

The goal of the current study was to examine the effect of cumulative risk on dropout and outcome in parent training for families with a child with externalizing behavior problems and developmental delay or borderline developmental delay. We included children with borderline developmental delay given the instability of cognitive functioning in early childhood, particularly among more disadvantaged families (Pianta & Egeland, 1994). Based on previous research, we expected that a higher cumulative risk score would predict higher rates of dropout and a dampened treatment response. To our knowledge, the only study examining the effect of cumulative risk on parent training relied on parental report to examine treatment response (Gardner et al., 2009). Our study extends the literature by examining the effect of cumulative risk on parent report of child behavior and observational measures of parenting skill acquisition, which is the primary target of many behavioral parent-training interventions. We also reported the effect of each individual risk factor to compare the utility of the cumulative risk score with using individual risk factors. For example, it may be the case that the cumulative risk score predicts dropout or an outcome variable despite few or no individual predictors of risk.

Method

Participants

Participants were 44 families who had participated in one of two treatment studies for their young child (30 families in Study A and 14 families in Study B), and the main outcome results for these studies are reported elsewhere (Bagner & Eyberg, 2007; Bagner, Sheinkopf, Vohr, & Lester, 2010). The inclusion and exclusion criteria for the two studies were very similar. The mother had to rate their child above the clinically significant range on a measure of child externalizing behavior problems and obtain a standard score of 75 or higher on a cognitive screening measure herself (see below for more detail). Exclusion criteria for the child included major sensory impairments (e.g., deafness, blindness), autism spectrum disorders, and significant motor impairments (e.g., cerebral palsy). For Study A, the child (ages 3 to 6 years) was required to have a developmental delay (IQ score < 75 on a measure of cognitive functioning). For Study B, the child (ages 18 months to 5 years) was considered at risk for a developmental delay due to premature birth, but only those children with scores ≤80 were included in the current study. Overall, the children were mostly boys (73%), with a mean age of 49.59 months (SD = 12.98).

Screening Measures

Maternal cognition

The Wonderlic Personnel Test (Dodrill, 1981) and the Wechsler Abbreviated Scale of Intelligence (Wechsler, 1999) are short and reliable measures of intelligence that were used to exclude mothers with cognitive impairment (i.e., IQ estimate < 75) in Study A and Study B, respectively. The maternal IQ estimate was also used as an index of maternal risk.

Child cognition

The Wechsler Preschool and Primary Scale of Intelligence–Third Edition (WPPSI; Wechsler, 2002) was administered to assess cognitive ability in children aged 3 years and older, whereas the Bayley Scales of Infant and Toddler Development–Third Edition (Bayley, 2006) was administered to assess cognitive ability in children younger than 3 years. Both tools are widely used and reliable measures of cognitive functioning in young children and were used as a criterion of developmental delay or borderline developmental delay (i.e., WPPSI Full Scale IQ [FSIQ] or Bayley Cognitive subtest ≤ 80) for the current study.

Outcome Measures

Child behavior problems

The Child Behavior Checklist (CBCL) for 1½- to 5-year olds (Achenbach & Rescorla, 2001) is a 99-item parent-rating scale and was used to measure the frequency of children’s behavioral problems. The CBCL has excellent 8-day test-retest reliability (r = .68 to .92), interrater reliability (mean mother–father r = .61), and success in discriminating between referred and nonreferred children (Achenbach & Rescorla, 2001). For both studies, children were required to score in the at-risk range for either the aggression subscale or the externalizing problems scale. The externalizing problems scale was used as the main outcome measure for the current study, and internal consistency estimates were .82 and .81 in Study A and Study B, respectively.

Parent–child interactions

The Dyadic Parent-Child Interaction Coding System–Third Edition (DPICS-III; Eyberg, Nelson, Duke, & Boggs, 2004) is a measure of the quality of parent–child interactions with documented reliability and validity (Eyberg et al., 2004), in which parent and child behaviors are coded by recording the frequency of each occurrence in real time using a video coding system. For this study, we chose the categories most relevant to treatment outcome. We created a composite category of do skills (behavior descriptions, reflections, and praises) and don’t skills (questions, commands, and negative talk) reflecting behavioral skills parents are taught during treatment to use and not use during a 5-min child-led play to promote positive parent–child interactions. Undergraduate research assistants were trained to 80% agreement with a criterion tape and were uninformed whether families had received treatment. Half of the observations were coded a second time for reliability. Kappas for the current sample ranged from .63 to .89 (M = .83) for Study A and .41 to .81 (M = .60) for Study B.

Risk Measures

Parenting stress

The Parenting Stress Index–Short Form (PSI-SF; Abidin, 1995) is a 36-item parent self-report instrument containing three scales (Parental Distress, Parent–Child Dysfunctional Interaction, and Difficult Child) with high 6-month test-retest reliability coefficients. The Parental Distress scale was used as a measure of maternal distress, and internal consistency estimates were .73 and .89 in Study A and Study B, respectively.

Cumulative risk index

Consistent with the literature on cumulative risk (Appleyard et al., 2005), we transformed six risk variables into dichotomous variables with a score of 1 indicating the presence of risk and 0 indicating no risk. As displayed in Table 1, three primary areas of risk encompassed these variables and include socioeconomic disadvantage (poverty and maternal education), family structure (single-parent household), and maternal risk characteristics (minority status, lower intelligence, and maternal distress). We chose the primary areas and specific variables based on previous research (described above) indicating that parents of children with developmental delay are more likely to have socioeconomic disadvantage (Msall et al., 1998; Sonnander & Claesson, 1999), difference in family structure (Koskentausta et al., 2007), and maternal risk characteristics (Baker et al., 2002; Baker et al., 2003; Sameroff et al., 1987). Economic disadvantage and family structure variables were coded based on previously established research (Copeland, Shanahan, Costello, & Angold, 2009). For other continuous variables, such as maternal IQ and maternal distress, conservative cutoffs were used to determine the presence of risk (see Table 1 for more detail). Cumulative risk was calculated for each participant by summing the six dichotomized variables (possible range in scores from 0 to 6).

Table 1.

Definition and Cumulative Prevalence of Risk Factors.

Risk factors % total % in 1-risk % in 2-risks % in ≥3-risks
Socioeconomic disadvantage
  1. Poverty: Meets federal guidelines based on income and family size 34.1 7.7 55.6 60.0
  2. Maternal education: Completed a high school education or less 34.1 23.1 22.2 66.7
Family structure
  3. Single-parent household: Only mother living in the home with the child 38.6 15.4 22.2 86.7
Maternal risk characteristics
  4. Minority status: Mother from Black, Hispanic, or biracial background 27.3 15.4 11.1 60.0
  5. Lower intelligence: Mother IQ below average (<25th percentile) 38.6 15.4 22.2 86.7
  6. Maternal distress: Clinically elevated distress (>85th percentile) 38.6 30.8 66.7 46.7
Open in a new tab

Procedure

Families in both studies were recruited at pediatric outpatient clinics (e.g., pediatric offices) and preschools and had contacted the study staff expressing interest in receiving treatment for their child. Both Studies A and B were approved by the affiliated Hospital Institutional Review Board and included a randomized, controlled trial to determine the efficacy of Parent–Child Interaction Therapy (PCIT) compared with a waitlist control (WL) comparison group. For the current study, the “pretreatment” scores were from the assessment immediately preceding the start of treatment (i.e., following a 4-month wait period for the WL group), and the “posttreatment” scores were from the assessment immediately after treatment completion. A total of 22 families (50%) dropped out, including the 4 families that dropped out after the initial visit and the 3 families that dropped out during the waitlist period, yielding different sample sizes for some measures. However, the timing of dropout (i.e., the number of treatment sessions completed before dropping out) was not shown to impact any results.

Intervention description

PCIT is a manualized behavioral parent-training intervention with extensive research demonstrating its efficacy and long-term maintenance in treating young children with externalizing behavior problems (Eyberg et al., 2008). Treatment progresses through two distinct phases. During the Child-Directed Interaction, the parents learn to follow their child’s lead in play and use differential attention. During the Parent-Directed Interaction, the parents learn to use effective commands and timeout for noncompliance and other disruptive behaviors (e.g., aggression). The therapist and cotherapist (who were advanced graduate students, interns, and/or postdoctoral fellows) coach each parent in vivo through a one-way mirror (using a wireless headset) in their use of the skills with their child. Sessions were conducted once a week for approximately 1 h in length (M = 12.79 sessions, SD = 2.13). All therapy sessions were videotaped, and 50% were randomly selected and coded for treatment integrity by an undergraduate research assistant uninvolved in coding behavioral observations. Accuracy, defined as the percent with which the therapist adhered to key elements of each session detailed in the treatment manual, was 97% for Study A and 94% for Study B.

Data Analysis

Preliminary analyses examined any potential demographic differences between the two treatment studies and whether any significant associations existed between the type of study and WL assignment and outcome measures. The cumulative effect of risk on treatment dropout was examined via logistic regression, and regression analyses were subsequently conducted to evaluate the association between the cumulative risk index and treatment outcome variables (i.e., externalizing behavior problems and parenting skills). All analyses were repeated by entering the individual risk factors into the regression equations. Families that dropped out were defined as those families that stopped attending sessions after the first visit and no longer returned our calls to continue treatment. Categorical analyses via MANCOVA were also conducted to isolate which risk factor groups were associated with worse treatment outcome among families that completed treatment. Given the low frequency of families having greater than three risk factors, we combined the three-, four-, and five-risk factor groups. The sample sizes for each risk group were as follows: 0-risks (n = 7), 1-risk (n = 13), 2-risks (n = 9), and ≥3-risks (n = 15). We also provide the frequency of each risk factor within each of these groups (see Table 1). All analyses were conducted using SPSS 18.0.

Results

Descriptive statistics for the study variables, which were all normally distributed, are presented in Table 2. All available data were used for each analysis. Preliminary analyses indicated no significant associations between the study or WL assignment and risk factors, rate of treatment dropout, or outcome measures. However, when comparing the two studies, children in Study A were significantly older, t(42) = 3.92, p < .001, and had lower IQ scores, t(42) = −3.69, p < .01, than children in Study B. Therefore, all subsequent analyses controlled for child age and IQ.

Table 2.

Descriptive Statistics.

306 M SD Minimum Maximum n
Demographic measures
  Maternal education (% high school education or less) 34.1 — — — 44
  Monthly family income in $ 2,890 2,285 220 9,729 44
  Mother IQ 97.48 13.50 75 125 44
  Child IQ 63.20 12.10 42 80 44
  Parenting distress 30.79 7.51 17 45 42
  Cumulative risk score 2.05 1.58 0 5 44
Pretreatment measures
  CBCL externalizing raw score 33.07 7.95 13 47 41
  Parenting do skills 4.87 4.18 0 15 38
  Parenting don’t skills 36.50 17.79 7 67 38
Posttreatment measures
  CBCL externalizing raw score 15.96 8.43 6 39 24
  Parenting do skills 20.83 11.81 0 45 24
  Parenting don’t skills 9.00 8.36 1 30 24
Open in a new tab

Note: CBCL = Child Behavior Checklist.

Cumulative Risk and Treatment Dropout

Logistic regressions were conducted to determine whether cumulative risk was associated with treatment dropout (binary outcome). Results indicated that cumulative risk was significantly associated with dropout (odds ratio = 1.72, 95% confidence interval [CI] = [1.10, 2.70], p = .018). This finding suggests that for each additional risk factor, the likelihood of dropout increased by 72%. Further examination comparing the risk groups indicate that relative to the 0-risk group, the ≥3-risks group had a significantly greater risk of dropping out of treatment (odds ratio = 10.00, 95% CI = [1.26, 79.34], p = .029). This indicates that families with three or more risk factors were 10 times more likely to drop out of treatment compared with families with no risk factors. Analyses on individual risk factors indicated that dropout was predicted by maternal minority status (odds ratio = 15.55, 95% CI = [2.91, 83.00], p = .001) and family structure (odds ratio = 3.62, 95% CI = [1.05, 12.50], p = .042).

Cumulative Risk and Treatment Outcome

Regression analyses were conducted to determine whether the cumulative risk index was significantly associated with treatment outcome on the CBCL externalizing raw scores, parenting do skills, and parenting don’t skills. Each analysis controlled for child age and IQ, as well as pretreatment scores to reflect a change score. As displayed in Table 3, the cumulative risk index was significantly associated with posttreatment CBCL externalizing raw scores, parenting do skills, and parenting don’t skills. Specifically, children from families with higher cumulative risk scores had higher posttreatment CBCL externalizing scores, and mothers from families with higher cumulative risk scores displayed fewer do skills and more don’t skills when interacting with their child during a child-directed play. Analyses on individual risk factors indicated that maternal minority status significantly predicted posttreatment CBCL externalizing scores (β = 7.70, p = .041), whereas mother education significantly predicted posttreatment do skills (β = −12.35, p = .005) and approached significance for don’t skills (β = 6.17, p = .090).

Table 3.

Regression Analyses Examining Predictors of Treatment Outcomes.

β R2 R2 change F
Posttreatment CBCL externalizing raw score
  Step 1 — .32 .32 3.07
    Child age −0.12
    Child IQ 0.06
    Pretreatment CBCL externalizing raw score 0.70
  Step 2 — .64 .33 8.45***
    Cumulative risk index 3.31**
Posttreatment parenting do skills
  Step 1 — .29 .29 2.74
    Child age 0.19
    Child IQ −0.01
Pretreatment parenting do skills 1.31*
  Step 2 — .47 .18 4.16*
    Cumulative risk index −3.40*
Posttreatment parenting don’t skills
    Step 1 — .23 .23 2.02
    Child age −0.04
    Child IQ 0.25
    Pretreatment parenting don’t skills 0.15
  Step 2 .45 .22 3.96*
    Cumulative risk index 2.73*
Open in a new tab

Note: CBCL = Child Behavior Checklist.

*

p < .05.

**

p < .01.

***

p < .001.

Finally, a MANCOVA was conducted to determine whether a particular risk group was likely to have worse treatment outcomes, with child age and IQ along with pretreatment outcome scores as covariates. Risk group was the between-subject variable, and posttreatment CBCL externalizing score, parenting do skills, and parenting don’t skills were entered as the dependent variables. The MANCOVA was significant, F(9, 35) = 2.65, p = .019 (partial η2 = .41), with follow-up ANCOVAs significant for posttreatment CBCL externalizing scores, F(3, 15) = 5.16, p = .012 (partial η2 = .51), and parenting do skills, F(3, 15) = 3.61, p = .038 (partial η2 = .42), and approached significance for parenting don’t skills, F(3, 15) = 2.65, p = .087 (partial η2 =. 35).

As shown in Table 4, follow-up contrast tests revealed that families in the ≥3-risks group reported significantly higher posttreatment CBCL externalizing scores, lower use of do skills, and higher use of don’t skills compared with families in the 0-risk and 2-risks groups. Families in the ≥3-risks group also reported significantly higher posttreatment CBCL externalizing scores compared with families in the 1-risk group. No other significant differences were found in posttreatment CBCL externalizing scores, do skills, or don’t skills among families in the 0-risk, 1-risk, or 2-risk groups.

Table 4.

Summary of Results for Risk Groups.

0-risks (n = 7) 1-risk (n = 13) 2-risks (n = 9) ≥3-risks (n = 15)
Percent dropout 29% 39% 33% 80%
Odds of dropping out of treatment — 1.56 [0.22, 11.37] 1.25 [0.15, 10.70] 10.00* [1.26, 79.34]
Posttreatment outcomes
  CBCL externalizing raw score 10.21 (2.94)a 15.55 (2.16)a 13.95 (2.94)a 24.77 (2.67)b
  Parenting do skills 28.93 (4.61)a 19.42 (3.80) 25.08 (4.60)a 9.91 (4.19)b
  Parenting don’t skills 5.06 (3.65)a 9.38 (2.68) 5.15 (3.64)a 16.96 (3.31)b
Open in a new tab

Note: CBCL = Child Behavior Checklist. Values enclosed in parentheses represent standard errors. All analyses controlled for child age and IQ as well as pretreatment scores. Estimated means in the same row that do not share superscript differ at *p < .05.

Discussion

This study was the first to examine the effect of cumulative risk on dropout from parent training. Consistent with the limited research on the individual risk factors that predict dropout (Fernandez & Eyberg, 2009; Lavigne et al., 2010; Werba et al., 2006), families with higher cumulative risk were more likely to drop out of treatment. Each additional risk factor almost doubled the chances of dropout, and families with three or more risk factors were particularly at risk, displaying a 10-fold increase in dropout rates compared with families with no risk factors. In fact, only 3 (20%) of the 15 families with three or more risk factors completed treatment. This disproportionate effect of the number of risk factors on negative outcome has also been observed in the developmental literature. For example, Forehand and colleagues (1998) found that an increase from three to four risk factors considerably increased the likelihood of adolescent adjustment difficulties. The current study further contributes to the literature by demonstrating the substantial impact cumulative risk can have on the likelihood of families completing parent-training interventions, especially for families of children with developmental delay.

Among families that completed PCIT, the cumulative risk index was also associated with poorer treatment outcome. Mothers from families with more risk factors reported less change in child externalizing behavior problems and displayed fewer improvements in parenting skills taught during treatment compared with mothers from families with less risk factors. Similar to the findings with dropout, families with three or more risk factors were at an increased risk and displayed significantly higher child externalizing behavior problems and less effective behavioral parenting skills during a child-directed play than families with none, one, or two risk factors. The treatment outcome findings were consistent with some recent meta-analyses on individual risk factors (Lundahl et al., 2006; Reyno & McGrath, 2006) but differed from two studies on the IY (Beauchaine et al., 2005) and FCU (Gardner et al., 2009) interventions, which both found that some individual risk factors were associated with improved treatment outcome.

The conflicting findings may be due to some differences between the studies. For the IY intervention study, Beauchaine and colleagues (2005) explored risk factors individually, rather than as a cumulative index. The risk factors were also examined as moderators of a variety of treatment combinations (i.e., parent training, child training, and/or teacher training), making it difficult to separate out the effect of the risk factors on parent training alone. In addition, some of the predictors examined were not measured in the current study (e.g., parental substance use), and maternal age was the only variable consistent between the two studies that yielded an opposite effect on outcome. The IY is a group intervention, whereas PCIT is implemented individually with families, so it is possible that a group format is more beneficial for younger mothers who may be seeking social support from other mothers. In addition, PCIT is unique in comparison with other behavioral parent-training interventions in its level of parental commitment required and the focus on having parents meet specific goals in their parenting skills before completing treatment. Therefore, the results of the current study may be specific to PCIT, and future research should examine the effect of cumulative risk on other parent-training interventions.

With regard to the study on the FCU, Gardner and colleagues (2009) did not find that a cumulative risk index predicted outcome. As previously mentioned, the lack of significant findings may be due to the nature of the intervention program, which targets predominately low-income families. This type of high-risk group would inherently have less variability in cumulative risk scores, particularly given that the inclusion criteria for the study required the presence of at least two risk factors. The higher variability of cumulative risk scores in the current study allowed for a better examination of how cumulative risk can impact treatment outcome. Interestingly, Gardner et al. found that low maternal education had a positive effect on treatment outcome, whereas we found that maternal education had a negative effect on outcome. These differences in findings may be due to the fact that findings in the Gardner et al. study were based on parent report of child behavior, and our findings were based on observation of parenting skills. Similar to previous research (Tymchuk & Andron, 1992), our results suggest that mothers with lower cognitive functioning may have more difficulty learning new parenting skills and may benefit from more specialized interventions.

In addition, the Gardner et al. (2009) study did not examine the effect of individual risk factors or the cumulative risk score on dropout, which is a clinically important issue in the parent-training literature. Comparisons between these studies on the risk factors predicting dropout would have been especially meaningful given the considerable difference in treatment intensity. As described above, PCIT is a somewhat intensive treatment requiring parents to participate in 1-hr weekly sessions over the course of 3 to 4 months. In contrast, FCU is a brief, three-session intervention based on motivational interviewing that includes optional follow-up sessions for interested families. It is important to note that the association between risk factors and treatment dropout/outcome may be moderated by treatment length. Hence, we cannot assume that risk factors experienced in the course of a brief treatment such as FCU will yield a similar effect on dropout such as those found in our study.

The only individual risk factors that predicted dropout in our study were maternal minority status and family structure, such that single mothers from ethnic minority backgrounds were more likely to drop out. These findings are somewhat consistent with the Gardner et al. (2009) outcome finding that two-parent families were more responsive to the FCU intervention. Nevertheless, a focus on the effect of individual risk factors in the current study suggested the other four risk factors did not impact the likelihood of dropout. However, families with at least three risk factors were most likely to dropout, highlighting the important role the other risk factors also played in predicting dropout when examined together with all the risk factors. Therefore, our findings highlight that a cumulative risk index can be used to effectively identify families who may have difficulty completing and benefiting from parent training, particularly those families of children with developmental delay.

The current study has some limitations that are important to consider. First, the sample included children with developmental delay or borderline developmental delay, so the findings about cumulative risk may not generalize to typically developing children with externalizing behavior problems. In addition, as stated above, the results may be specific to PCIT, which may be more intensive than other behavioral parent-training interventions. Second, the study did not include a measure of parental psychopathology. We included parental distress as an index of parent functioning, but parental mental health problems (e.g., depression, substance abuse) are other risk factors that may also predict dropout and/or outcome in parent training. On a related note, only data from the primary caregiver, which was the mother in all cases, were included. Future research should investigate whether the effect of cumulative risk on parent training is the same with fathers and other caregivers. Third, although we included minority status as a risk factor in the cumulative risk index, we did not collect information on the acculturation of families in the current study. It is possible that the level of acculturation moderates the relation between minority status and dropout and outcome, and future research would benefit from exploring this important issue. Fourth, the present study did not include a long-term follow-up, which may also explain some of the differences in findings from the 1- and 2-year follow-up periods in the IY and FCU studies, respectively. Finally, although our sample size was moderate, the substantial number of families that dropped out of treatment reduced our sample size for the treatment outcome analyses.

Despite these limitations, the present study provides insight into how cumulative risk can affect a family’s success in parent training and has important research and clinical implications. We acknowledge the importance of studying individual risk factors. However, analyses on individual risk factors in the current study demonstrated that only one to two risk factors, at most, were individually predictive of either dropout or outcome. Given the conflicting research on individual risk factors and the findings from the current study, the cumulative risk score can be a useful way for clinicians to identify families at risk of dropping out or not achieving the most optimal outcome following behavioral parent training. Clinicians should assess for the number of risk factors as a proxy for predicting which families may be at most risk for dropping out prematurely or not attaining the best outcome from parent training. Based on our results, clinicians should be particularly concerned about families with three or more of the risk factors examined in this study.

Although many of the risk factors examined in the current study are not malleable, further research is needed to examine how best to intervene in the presence of multiple risk factors to prevent subsequent dropout or dampened treatment response. For example, it may be beneficial for parents with multiple risk factors to receive individual therapy along with parent training, such as adjunctive parent treatment components described in a comprehensive review of enhancements to parent training (Chronis, Chacko, Fabiano, Wymbs, & Pelham, 2004). Parents of children with developmental delay also report higher levels of stress than parents of typically developing children (Baker et al., 2002), so additional support for those parents would be particularly relevant. In fact, a meta-analysis demonstrated that interventions for parents of children with developmental disabilities that combined parent well-being components with behavioral parent training were more effective than either component alone (Singer, Ethridge, & Aldana, 2007), suggesting further research explore the enhanced effect of addressing parental wellbeing, particularly for families with multiple risk factors. In addition, nondirective clinical approaches such as motivational interviewing has been shown to reduce dropout among physically abusive parents (who also had many of the risk factors examined in the current study) receiving PCIT (Chaffin, Funderburk, Bard, Valle, & Gurwitch, 2011; Chaffin et al., 2009). These additional parent supports, although not necessarily directly changing the risk factors themselves, may, in combination with PCIT, improve rates of parent training completion and yield more optimal treatment outcome among parents of children with developmental delay.

Acknowledgments

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The current research was funded grants from the National Institute of Mental Health (F31 MH068947 and K23 MH085659) and the National Institute for Child Health and Human Development (F32 HD056748).

Biographies

Daniel M. Bagner, PhD, ABPP, is an assistant professor in the Department of Psychology at Florida International University. He conducts research examining interventions for young children and their families and has specific interests in the prevention and treatment of behavior problems in at-risk infant and child populations.

Paulo A. Graziano, PhD, is a postdoctoral fellow in the Center for Children and Families at Florida International University. He conducts research on the development of children’s self-regulation skills as well as how such neurocognitive processes can be applied to the understanding and treatment of both disruptive behavior disorders and pediatric obesity.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

  1. Abidin RR. Parenting Stress Index: Professional manual. 3rd ed. Odessa, FL: Psychological Assessment Resources; 1995. [Google Scholar]
  2. Achenbach TM, Rescorla LA. Manual for the ASEBA preschool forms & profiles. Burlington: University of Vermont, Research Center for Children, Youth, & Families; 2001. [Google Scholar]
  3. Amato PR, Keith B. Parental divorce and the well-being of children: A meta-analysis. Psychological Bulletin. 1991;110:26–46. doi: 10.1037/0033-2909.110.1.26. [DOI] [PubMed] [Google Scholar]
  4. Anderson LS. Predictors of parenting stress in a diverse sample of parents of early adolescents in high-risk communities. Nursing Research. 2008;57:340–350. doi: 10.1097/01.NNR.0000313502.92227.87. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Appleyard K, Egeland B, van Dulmen MH, Sroufe LA. When more is not better: The role of cumulative risk in child behavior outcomes. Journal of Child Psychology and Psychiatry. 2005;46:235–245. doi: 10.1111/j.1469-7610.2004.00351.x. [DOI] [PubMed] [Google Scholar]
  6. Bagner DM, Eyberg SM. Parent-child interaction therapy for disruptive behavior in children with mental retardation: A randomized controlled trial. Journal of Clinical Child & Adolescent Psychology. 2007;36:418–429. doi: 10.1080/15374410701448448. [DOI] [PubMed] [Google Scholar]
  7. Bagner DM, Sheinkopf SJ, Vohr BR, Lester BM. Parenting intervention for externalizing behavior problems in children born premature: An initial examination. Journal of Developmental & Behavioral Pediatrics. 2010;31:209–216. doi: 10.1097/DBP.0b013e3181d5a294. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Baker BL, Blacher J, Crnic KA, Edelbrock C. Behavior problems and parenting stress in families of three-year-old children with and without developmental delays. American Journal of Mental Retardation. 2002;107:433–444. doi: 10.1352/0895-8017(2002)107<0433:BPAPSI>2.0.CO;2. doi:10.1352/0895-8017(2002)107<0433:BPAPSI>2.0.CO;2. [DOI] [PubMed] [Google Scholar]
  9. Baker BL, McIntyre LL, Blacher K, Crnic K, Edelbrock C, Low C. Pre-school children with and without developmental delay: Behaviour problems and parenting stress over time. Journal of Intellectual Disability Research. 2003;47:217–230. doi: 10.1046/j.1365-2788.2003.00484.x. [DOI] [PubMed] [Google Scholar]
  10. Bayley N. Manual for the Bayley Scales of Infant Development. 3rd ed. San Antonio, TX: The Psychological Corporation; 2006. [Google Scholar]
  11. Beauchaine TP, Webster-Stratton C, Reid MJ. Mediators, moderators, and predictors of 1-year outcomes among children treated for early-onset conduct problems: A latent growth curve analysis. Journal of Consulting and Clinical Psychology. 2005;73:371–388. doi: 10.1037/0022-006X.73.3.371. [DOI] [PubMed] [Google Scholar]
  12. Boggs SR, Eyberg SM, Edwards DL, Rayfield A, Jacobs J, Bagner D, Hood KK. Outcomes of parent-child interaction therapy: A comparison of treatment completers and study dropouts one to three years later. Child & Family Behavior Therapy. 2004;26:1–22. [Google Scholar]
  13. Briggs-Gowan MJ, Carter AS, Bosson-Heenan J, Guyer AE, Horwitz SM. Are infant-toddler social-emotional and behavioral problems transient? Journal of the American Academy of Child & Adolescent Psychiatry. 2006;45:849–858. doi: 10.1097/01.chi.0000220849.48650.59. [DOI] [PubMed] [Google Scholar]
  14. Burchinal MR, Roberts JE, Hooper S, Zeisel SA. Cumulative risk and early cognitive development: A comparison of statistical risk models. Developmental Psychology. 2000;36:793–807. doi: 10.1037//0012-1649.36.6.793. [DOI] [PubMed] [Google Scholar]
  15. Chaffin M, Funderburk B, Bard D, Valle LA, Gurwitch R. A combined motivation and parent-child interaction therapy package reduces child welfare recidivism in a randomized dismantling field trial. Journal of Consulting and Clinical Psychology. 2011;79:84–95. doi: 10.1037/a0021227. [DOI] [PubMed] [Google Scholar]
  16. Chaffin M, Valle LA, Funderburk B, Gurwitch R, Silovsky J, Bard D, Kees M. A motivational intervention can improve retention in PCIT for low-motivation child welfare clients. Child maltreatment. 2009;14:356–368. doi: 10.1177/1077559509332263. [DOI] [PubMed] [Google Scholar]
  17. Chapman DP, Whitfield CL, Felitti VJ, Dube SR, Edwards VJ, Anda RF. Adverse childhood experiences and the risk of depressive disorders in adulthood. Journal of Affective Disorders. 2004;82:217–225. doi: 10.1016/j.jad.2003.12.013. [DOI] [PubMed] [Google Scholar]
  18. Copeland W, Shanahan L, Costello EJ, Angold A. Configurations of common childhood psychosocial risk factors. Journal of Child psychology and Psychiatry. 2009;50:451–459. doi: 10.1111/j.1469-7610.2008.02005.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Chronis AM, Chacko A, Fabiano GA, Wymbs BT, Pelham WE. Enhancements to the behavioral parent training paradigm for families of children with ADHD: Review and future directions. Clinical Child and Family Psychology Review. 2004;7:1–27. doi: 10.1023/b:ccfp.0000020190.60808.a4. [DOI] [PubMed] [Google Scholar]
  20. Dekker MC, Koot HM, van der Ende J, Verhulst FC. Emotional and behavioral problems in children and adolescents with and without intellectual disability. Journal of Child Psychology and Psychiatry. 2002;43:1087–1098. doi: 10.1111/1469-7610.00235. [DOI] [PubMed] [Google Scholar]
  21. Dodrill CB. An economical method for the evaluation of general intelligence in adults. Journal of Consulting and Clinical Psychology. 1981;49:668–673. doi: 10.1037//0022-006x.49.5.668. [DOI] [PubMed] [Google Scholar]
  22. Emerson E, Robertson J, Wood J. Emotional and behavioural needs of children and adolescents with intellectual disabilities in an urban conurbation. Journal of Intellectual Disability Research. 2005;49:16–24. doi: 10.1111/j.1365-2788.2005.00658.x. [DOI] [PubMed] [Google Scholar]
  23. Eyberg SM, Nelson MM, Boggs SR. Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child & Adolescent Psychology. 2008;37:1–23. doi: 10.1080/15374410701820117. [DOI] [PubMed] [Google Scholar]
  24. Eyberg SM, Nelson MM, Duke M, Boggs SR. Manual for the dyadic parent-child interaction coding system. (3rd ed.) 2004 Available from www.pcit.org. [Google Scholar]
  25. Felitti VJ, Anda RF, Nordenberg D, Williamson DF, Spitz AM, Edwards V, Marks JS. Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults: The adverse childhood experiences (ACE) study. American Journal of Preventive Medicine. 1998;14:245–258. doi: 10.1016/s0749-3797(98)00017-8. [DOI] [PubMed] [Google Scholar]
  26. Fernandez MA, Eyberg SM. Predicting treatment and follow-up attrition in parent-child interaction therapy. Journal of Abnormal Child Psychology. 2009;37:431–441. doi: 10.1007/s10802-008-9281-1. [DOI] [PubMed] [Google Scholar]
  27. Forehand R, Biggar H, Kotchick BA. Cumulative risk across family stressors: Short- and long-term effects for adolescents. Journal of Abnormal Child Psychology. 1998;26:119–128. doi: 10.1023/a:1022669805492. [DOI] [PubMed] [Google Scholar]
  28. Gardner F, Connell A, Trentacosta CJ, Shaw DS, Dishion TJ, Wilson MN. Moderators of outcome in a brief family-centered intervention for preventing early problem behavior. Journal of Consulting and Clinical Psychology. 2009;77:543–553. doi: 10.1037/a0015622. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kaminski JW, Valle LA, Filene JH, Boyle CL. A meta-analytic review of components associated with parent training program effectiveness. Journal of Abnormal Child Psychology. 2008;36:567–589. doi: 10.1007/s10802-007-9201-9. [DOI] [PubMed] [Google Scholar]
  30. Keenan K, Boeldt D, Chen D, Coyne C, Donald R, Duax J, Humphries M. Predictive validity of DSM-IV oppositional defiant and conduct disorders in clinically referred preschoolers. Journal of Child Psychology and Psychiatry. 2010 doi: 10.1111/j.1469-7610.2010.02290.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Keenan K, Wakschlag LS. More than the terrible twos: The nature and severity of behavior problems in clinic-referred preschool children. Journal of Abnormal Child Psychology. 2000;28:33–46. doi: 10.1023/a:1005118000977. [DOI] [PubMed] [Google Scholar]
  32. Koskentausta T, Iivanainen M, Almqvist F. Risk factors for psychiatric disturbance in children with intellectual disability. Journal of Intellectual Disability Research. 2007;51:43–53. doi: 10.1111/j.1365-2788.2006.00871.x. [DOI] [PubMed] [Google Scholar]
  33. Lavigne JV, Lebailly SA, Gouze KR, Binns HJ, Keller J, Pate L. Predictors and correlates of completing behavioral parent training for the treatment of oppositional defiant disorder in pediatric primary care. Behavior Therapy. 2010;41:198–211. doi: 10.1016/j.beth.2009.02.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Lochman JE. Parent and family skills training in targeted prevention programs for at-risk youth. Journal of Primary Prevention. 2000;21:253–265. [Google Scholar]
  35. Lundahl B, Risser HJ, Lovejoy MC. A meta-analysis of parent training: Moderators and follow-up effects. Clinical Psychology Review. 2006;26:86–104. doi: 10.1016/j.cpr.2005.07.004. [DOI] [PubMed] [Google Scholar]
  36. McIntyre LL, Abbeduto L. Parent training for young children with developmental disabilities: Randomized controlled trial. American Journal on Mental Retardation. 2008;113:356–368. doi: 10.1352/2008.113:356-368. doi:10.1352/0895-8017(2000)105<0269:RTO IEI>2.0.CO;2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Msall ME, Bier JA, LaGasse L, Tremont M, Lester B. The vulnerable preschool child: The impact of biomedical and social risks on neurodevelopmental function. Seminars in Pediatric Neurology. 1998;5:52–61. doi: 10.1016/s1071-9091(98)80019-3. [DOI] [PubMed] [Google Scholar]
  38. Pianta RC, Egeland B. Predictors of instability in children’s mental test performance at 24, 48, and 96 months. Intelligence. 1994;18:145–163. [Google Scholar]
  39. Platt L. Ten year transitions in children’s experience of living in a workless household: Variations by ethnic group. Population Trends. 2010;139:70–90. doi: 10.1057/pt.2010.6. [DOI] [PubMed] [Google Scholar]
  40. Prinstein MJ, Boergers J, Spirito A. Adolescents’ and their friends’ health-risk behavior: Factors that alter or add to peer influence. Journal of Pediatric Psychology. 2001;26:287–298. doi: 10.1093/jpepsy/26.5.287. [DOI] [PubMed] [Google Scholar]
  41. Prinz RJ, Miller GE. Family-based treatment for childhood antisocial behavior: Experimental influences on dropout and engagement. Journal of Consulting and Clinical Psychology. 1994;62:645–650. doi: 10.1037//0022-006x.62.3.645. [DOI] [PubMed] [Google Scholar]
  42. Reid WJ, Crisafulli A. Marital discord and child behavior problems: A meta-analysis. Journal of Abnormal Child Psychology. 1990;18:105–117. doi: 10.1007/BF00919459. [DOI] [PubMed] [Google Scholar]
  43. Reyno SM, McGrath PJ. Predictors of parent training efficacy for child externalizing behavior problems—A meta-analytic review. Journal of Child Psychology and Psychiatry. 2006;47:99–111. doi: 10.1111/j.1469-7610.2005.01544.x. [DOI] [PubMed] [Google Scholar]
  44. Roberts C, Mazzucchelli T, Studman L, Sanders M. Behavioral family intervention for children with developmental disabilities and behavioral problems. Journal of Clinical Child & Adolescent Psychology. 2006;35:180–193. doi: 10.1207/s15374424jccp3502_2. [DOI] [PubMed] [Google Scholar]
  45. Rutter M. Protective factors in children’s response to stress and disadvantage. In: Kent WM, Rolf JE, editors. Primary prevention of psychopathology. Vol. 3. Hanover, NH: University Press of New England; 1979. pp. 49–74. [Google Scholar]
  46. Sameroff AJ, Seifer R, Barocas R, Zax M, Greenspan S. Intelligence quotient scores of 4-year-old children: Social-environmental risk factors. Pediatrics. 1987;79:343–350. Retrieved from http://pediatrics.aappublications.org/content/79/3/343. [PubMed] [Google Scholar]
  47. Singer GHS, Ethridge BL, Aldana SI. Primary and secondary effects of parenting and stress management interventions for parents of children with developmental disabilities: A meta-analysis. Mental Retardation and Developmental Disabilities Research Reviews. 2007;13:357–369. doi: 10.1002/mrdd.20175. [DOI] [PubMed] [Google Scholar]
  48. Sonnander K, Claesson M. Predictors of developmental delay at 18 months and later school achievement problems. Developmental Medicine & Child Neurology. 1999;41:195–202. doi: 10.1017/s0012162299000389. [DOI] [PubMed] [Google Scholar]
  49. Trentacosta CJ, Hyde LW, Shaw DS, Dishion TJ, Gardner F, Wilson M. The relations among cumulative risk, parenting, and behavior problems during early childhood. Journal of Child Psychology and Psychiatry. 2008;49:1211–1219. doi: 10.1111/j.1469-7610.2008.01941.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Tymchuk AJ, Andron L. Project parenting: Child interactional training with mothers who are mentally handicapped. Journal of Applied Research in Intellectual Disabilities. 1992;5:4–32. [Google Scholar]
  51. Webster-Stratton C. Predictors of treatment outcome in parent training for conduct disordered children. Behavior Therapy. 1985;16:223–243. [Google Scholar]
  52. Webster-Stratton C. Predictors of treatment outcome in parent training for families with conduct problem children. Behavior Therapy. 1990;21:319–337. [Google Scholar]
  53. Wechsler D. Wechsler Abbreviated Scale of Intelligence. San Antonio, TX: The Psychological Corporation; 1999. [Google Scholar]
  54. Wechsler D. Wechsler Preschool and Primary Scale of Intelligence. 3rd ed. San Antonio, TX: The Psychological Corporation; 2002. [Google Scholar]
  55. Werba BE, Eyberg SM, Boggs SR, Algina J. Predicting outcome in parent-child interaction therapy: Success and attrition. Behavior Modification. 2006;30:618–646. doi: 10.1177/0145445504272977. [DOI] [PubMed] [Google Scholar]

RESOURCES