Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Sep 1.
Published in final edited form as: J Emot Behav Disord. 2017 Jul 10;26(3):182–192. doi: 10.1177/1063426617717721

Multiple family group service delivery model for children with disruptive behavior disorders: Impact on caregiver stress and depressive symptoms

Geetha Gopalan 1,✉, Lindsay A Bornheimer 2, Mary C Acri 3, Andrew Winters 4, Kyle H O’Brien 5, Anil Chacko 6, Mary M McKay 7
PMCID: PMC6261513  NIHMSID: NIHMS903393  PMID: 30505141

Abstract

Disproportionately high rates of caregiver stress and depression are found among poverty-impacted communities, with high levels of caregiver stress and depression putting youth at heightened risk for the onset and perpetuation of disruptive behavior disorders. The purpose of this study was to examine the effects of a behavioral parent training program called the 4Rs and 2Ss for Strengthening Families Program (4R2S) on caregiver stress and depressive symptoms among 320 youth aged seven to 11 and their families assigned to either the 4R2S or services as usual (SAU) condition. Among caregivers with clinically significant scores at baseline, 4R2S participants manifested significantly reduced scores on the stress and depressive symptom scores to SAU participants at 6-month follow-up. Findings suggest that 4R2S may reduce caregiver stress and depressive symptoms among those caregivers initially manifesting clinically significant levels of stress or depressive symptoms.

Introduction

The combination of stressors associated with poverty (e.g., community violence, housing instability, limited economic and social supports) frequently result in a common “package” of circumstances within impacted communities, where families manifest elevated rates of parental stress and depression, as well as child behavioral difficulties (Bannon & McKay, 2007; Ertel, Rich-Edwards, & Koenen, 2011; Evans, 2004; Ghandour, Kogan, Blumberg, Jones, & Perrin, 2012; Kazdin & Whitley, 2003; Siefert, Finlayson, Williams, Delva, & Ismail, 2007; Yoshikawa, Aber, & Beardslee, 2012). Economic hardship often contributes to parental stress and depression (Conger et al., 2010), leading to interruptions to the parent-child bonding process, as well as increases in inconsistent, harsh parenting practices (Coiro, Riley, Broitman, & Miranda, 2012; Deater-Deckard, 1998; Hall, 2012). Such parenting practices, in concert with poverty-related stressors, increases children’s vulnerability to the onset and perpetuation of Conduct Disorder (CD) and Oppositional Defiant Disorder (American Psychiatric Association, 2013; ODD), which are part of a larger category of Disruptive Behavior Disorders (DBD) (Forgatch & Patterson, 2010). Characterized by serious and chronic oppositionality, impulsivity, aggression, and delinquent acts (Kilgore, Snyder, & Lentz, 2000), these disorders are a costly public health issue (Scott, Knapp, Henderson, & Maughan, 2001) as they are associated with multiple maladaptive future outcomes for children (e.g., child maltreatment, co-morbid mental health, substance use/abuse, delinquency; Beauchaine & Hinshaw, 2013). Moreover, children’s behavioral difficulties can, in turn, further exacerbate parental stress and depressive symptoms (Marchand, Hock, & Widaman, 2002; Munson, McMahon, & Spieker, 2001; Spieker, Larson, Lewis, Keller, & Gilchrist, 1999).

Unfortunately, untreated parent stress and depression can also hinder treatment for DBDs. Behavioral parent training (BPT) is one of the most effective types of treatment for DBDs (Eyberg et al., 2008; Evans, Owens & Bunford, 2014), with the common goal of decreasing problematic youth behaviors and promoting positive behaviors through enhancing caregivers’ parenting skills (Chacko et al., 2015). However, parental stress and depression negatively impacts caregivers’ ability to implement effective parenting skills for managing child behavior (Chronis, Chacko, Fabiano, Wymbs, & Pelham, 2004). Specifically, caregivers struggling with stress and depression are more likely to discipline children harshly through coercive parenting and physical discipline (Deater-Deckard & Dodge, 1997; Deater-Deckard, Dodge, Bates, & Pettit, 1998). Stress and depression also increase the likelihood that caregivers’ will be less responsive to children’s needs, instead manifesting increased irritability in their interactions with children. As a result, parent-child interactions become increasingly conflict-ridden (Chronis et al., 2004; Kazdin & Whitley, 2003; Lovejoy, Graczyk, O’Hare, & Neuman, 2000). Consequently, children with DBD’s are unlikely to improve if caregiver stress and depression concurrently impede the use of corrective parenting practices. This suggests that addressing both child- and caregiver-level concerns may maximize treatment success for BPTs.

At the same time, there is some debate regarding ideal strategies to address both parent and child needs in treatment. Stress and depression have been shown to abate when parental emotional health is targeted by integrating enhancements to typical BPT such as problem solving sessions (Chacko et al., 2009; Kazdin et al., 1992; Kazdin & Whitley, 2003). However, such enhancements may potentially increase treatment costs and length, due to the addition of separate sessions and/or concurrently-run groups. Alternatively, other studies suggest that BPT without enhancements reduces parental stress and depression by decreasing child behavior problems, which subsequently lead to increased parental sense of efficacy and competence in their parenting skills (DeGarmo, Patterson, & Forgatch, 2004; Pisterman et al., 1992). This suggests that BPT without enhancements may be a more cost-effective, first-line of intervention to address both child behavioral difficulties and parental distress (Barth, 2009).

To date, however, there is uneven evidence that BPT without enhancements for youth with behavioral difficulties can also lead to significant reductions in parental stress and depression. While some studies have found significant reductions in parental stress and depression when comparing treatment to control groups (DeGarmo et al., 2004; Hutchings et al., 2007; Larsson et al., 2009; Pisterman et al., 1992; Stattin, Enebrink, Ozdemir, & Giannotta, 2015) others have found no differences (Braet al., 2009; Charles et al., 2013; Gardner, Burton, & Klimes, 2006; Sigmarsdóttir, Thorlacius, Guđmundsdóttir, & DeGarmo, 2015; Taylor, Schmidt, Peplar, & Hodgins, 1988).

Importantly, the existing literature on effectiveness of BPTs for children and parents living in poverty is limited by the fact that few studies have been conducted predominantly among low-income families, delivered within publically-funded routine clinical practice (e.g., treatment provided by existing practitioners rather than research staff), and offered to youth referred to treatment by their caregivers rather than those recruited for a particular study (Michelson, Davenport, Dretzke, Barlow & Day, 2013; Weisz et al., 2013b). Moreover, in routine community-based services, children typically receive multiple services concurrently (e.g., individual therapy and medication monitoring; Hutchings et al., 2007; Larsson et al., 2009; Pisterman et al., 1992), underscoring the importance of understanding the incremental benefit of an intervention within the context of possibly receiving additional services. Research that fails to address these key factors will have limited generalizability when these interventions are translated into routine, under-resourced service settings with poverty-impacted clientele. Moreover, studies often comparing evidence-based interventions to usual care among clinic-referred youth with serious, complex diagnoses typical of “real world practice” often have indicated non-significant findings (Weisz et al., 2013a). Our review of the BPT treatment literature for child behavioral difficulties that also examined parental stress and depression as outcomes yielded only a handful of studies that reflected the challenges and realities of poverty-impacted families accessing routine, publically available services (e.g., treatment provided by existing practitioners in publically funded settings, treatment provided to predominantly low-income sample of referred youth and families, multiple services provided concurrently). We have discussed them in more detail below.

Efforts to integrate the Incredible Years parent training in the U.K. equivalent of Head Start, called Sure Start, among socioeconomically disadvantaged families in Wales, found significant reductions in child behavioral difficulties and parental stress, although mixed findings on parental depression when compared to families in the wait-list control groups (Charles et al., 2013; Hutchings et al., 2007). Larsson et al. (2009) examined the effectiveness of parent training in community-based child psychiatric clinics in Norway, reporting significant reductions in child conduct problems and parental stress for those in the experimental group versus wait-list control. However, a number of other studies have found no significant benefits of BPT on measures of parental stress and depression even when concluding that BPT reduced child behavioral difficulties (Gardner et al., 2006; Sigmarsdottir et al., 2009; Taylor et al., 1998).

Notably, only two of these studies reviewed above compared BPT to services as usual (i.e., Sigmarsdottir et al., 2009, Taylor et al., 1998). Comparing BPT to a non-active comparison condition like waitlist control is more likely to yield larger treatment effect sizes favoring the active treatment, as waitlist control conditions are meant to have no beneficial therapeutic effect and, thus by design, are weaker than the active treatment being tested. Conversely, usual care, which is designed to have a therapeutic benefit, is a stronger standard to test BPTs. Additionally, efforts to improve mental health services must ultimately focus on the incremental benefit of novel innovative approaches compared to existing models rather than just to wait-list control or alternative, relatively less-utilized interventions/models. Given the substantial costs of implementing many evidence-based interventions within existing clinical practice (due to associated manuals, measures, training, supervision, etc.), the question of whether a novel approach actually provides added value to usual care becomes even more salient (Weisz et al., 2013b), particularly in resource-poor communities. To our knowledge, the incremental value of BPT over and above the benefit of services as usual has yet to be tested regarding parental stress and depression.

The current study extends the existing literature base by examining the incremental benefit of an evidence-based multiple family group BPT program, called the 4Rs and 2Ss Strengthening Families Program (4R2S; http://nrepp.samhsa.gov/ProgramProfile.aspx?id=41), to existing services in outpatient, community-based child mental health clinics. Importantly, this paper reports on findings from an effectiveness study conducted with families predominantly impacted by poverty and within the context of routine community-based clinical settings (Chacko et al., 2015; Gopalan et al. 2015; McKay et al., 2011). Using a common elements approach (Chorpita & Daleiden, 2009; Garland, Hawley, Brookman-Frazee, & Hurlburt, 2008), 4R2S integrates essential practices of BPT and family therapy as well as factors known to affect service utilization, translated into six core intervention components known as the 4Rs and 2Ss: Roles, Responsibilities, Relationships, Respectful communication, Social support, and Stress. Specifically, 4R2S targets key family processes commonly addressed across evidence-based BPTs which have been successful in treating DBDs (e.g., consistent discipline; use of rewards/consequences to shape behavior; and enhanced quality of family organization, relationships, and communication), through a service delivery model that also combines group and family therapy strategies. As stress and lack of social support have been known to inhibit treatment engagement (McKay & Bannon, 2004), 4R2S further helps families identify and cope with sources of stress, as well as enhance their social support network (see Methods section for more details).

The incremental value of 4R2S (4R2S + SAU) was recently evaluated in a large clinical effectiveness trial in comparison to services as usual (SAU) within community-based, publically-funded child mental health clinics for families with school-aged children (ages 7–11). Compared to participants in the SAU condition, findings of previous studies indicated 4R2S participants manifested significantly reduced levels of child behavior difficulties and increased social skills immediately post-treatment (Chacko et al., 2015). Six-months post-treatment, 4R2S continued to demonstrate benefits over SAU in terms of reduced child behavioral difficulties, as well as diminished functional impact on children’s relationships with peers and overall severity of child behavior difficulties (Gopalan et al., 2015).

While 4R2S has been evaluated for child behavioral difficulties and functional outcomes, as well as the relationship between caregiver stress and depression to 4R2S attendance (Jackson, 2014), the incremental value 4R2S has not been evaluated regarding its impact on caregiver stress and depressive symptoms relative to services as usual. Given the impact of caregiver stress and depressive symptoms on effective parenting, child functioning and BPT treatment success, examining these outcomes is particularly important. Based on the prior research suggesting that BPT without enhancements may lead to lower parental stress and depression (e.g., DeGarmo et al., 2004), we hypothesize that caregivers in the 4R2S condition will report lower rates of parental stress and depressive symptoms at post-test and six-months’ follow-up compared to participants in the SAU condition.

Importantly, we also explore whether the 4R2S incremental treatment effect differs based on whether caregivers enter the study with clinically significant levels of stress and depressive symptoms. Prior research has demonstrated that caregivers with initially high levels of stress and depressive symptoms are more likely (compared to those caregivers with normal levels) to demonstrate reductions in these outcomes resulting from BPT (e.g., DeGarmo et al., 2004; Jackson, 2014). As a result, we anticipate that individuals with clinically significant levels of stress and depressive symptoms will demonstrate incremental benefits from 4R2S, whereas those caregivers eliciting non-clinically significant levels of the same outcomes will reveal little to no effect of the 4R2S intervention. Similar to the main effects analyses, we examine the incremental benefit of 4R2S over SAU at post-test and six-months follow-up, separated by clinically significant status.

Methods

Procedures

Institutional Review Board approval was obtained for this study. Recruitment occurred between October, 2006 and October, 2010 in 13 community-based outpatient mental health clinics serving families who were socioeconomically disadvantaged. Each site included in the study provided an array of mental health services to youth and families from neighborhoods in the New York City metropolitan area. Most often, families served at participating clinics identified as being of Latino or African American descent. Clinic staff identified children between seven and 11 years of age with behavior difficulties at intake and informed their adult caregivers about the study. Interested caregivers were then referred to research staff. The child’s legal guardian provided written informed consent and youth provided verbal assent. Families could not be consented where the legal guardian was not available (i.e., children residing in foster care) or significant cognitive or psychiatric impairment (as determined by research staff upon face-to-face meeting) which prohibited understanding of the consent process or intervention components.

Eligibility criteria included English- or Spanish-speaking families with youth who were between the ages of seven and 11 years old, and willing to be a part of treatment at one of the participating clinics. Among those families who met this initial eligibility criteria, caregivers completed the screening measure, the Disruptive Behavior Disorders Rating Scale (Pelham, Gnagy, Greenslade, & Milich, 1992), to determine if the child met criteria for a diagnosis of Oppositional Defiant Disorder (ODD) or Conduct Disorder (CD) as specified in the Diagnostic and Statistical Manual of Mental Disorders (DSM-V; American Psychiatric Association, 2013). Only families where youth met diagnostic criteria for ODD or CD were eligible to participate in the study.

Once determined eligible, participants were assigned to the 4R2S experimental condition (4R2S + SAU; n=225), or services as usual (SAU) comparison condition (n=95). Project coordinators utilized a 2:1 allocation ratio, block comparison design to ensure that there were a sufficient number of families to begin the 4R2S group at each participating clinic. Specifically, the first 6–8 families where the youth screened in as eligible were assigned by research staff to 4R2S condition. Subsequently the next 3–4 eligible youth and their families were allocated to the SAU treatment condition. As indicated by Dumville, Torgerson and Hewitt (2006), threats to statistical validity are a concern only when the allocation ratio is 3:1 or greater. Because treatment conditions can be populated quickly and efficiently, block comparison designs like the one utilized in this study are frequently employed for health services trials with similar logistical constraints (e.g., Goodwin, et al., 2001). To reduce bias, decisions regarding treatment condition assignment were not managed by field staff, rather by project coordinators who were removed from direct family contact. Additionally, research staff and investigators were blind to youth and family profiles during allocation.

Participants

Of the 416 initially screened, 320 youth and their families were enrolled in the study. Prior studies examining the same data (Chacko et al., 2015; Gopalan et al., 2015) have reported 91 children did not meet diagnosis for CD or ODD, three children were considered not appropriate for treatment due to their pervasive developmental delay issues, and two caregivers declined to participate. The majority of caregiver participants identified as Latino (54%) or Black/African American (31%), were the primary caregivers of boys (68%), and reported a family income of less than $30,000 (79%). Two-thirds of families (n=215) identified as having a single-parent household, and 70% (n=223) received publicly funded health insurance. At baseline, caregivers in 4R2S were significantly younger (M=35.72, SD=8.39) than caregivers in SAU (M=38.18, SD=9.30, t= −2.29, df=308, p=0.02). There were no differential rates of attrition by treatment condition between participants at baseline, mid-test, and post-test assessment. Six-month post intervention data were collected from 221 (69%) families. Of the total families at six-month follow up, a significantly greater number of families who received SAU (79%, n=75) were included compared with families in the 4R2S condition (65%, n=146; χ2=6.18, df=1, p=0.01). A significant association between response status and ethnicity (χ2=14.30, df =5, p=0.01) at six-month follow up was found, indicating that African American caregivers were the most represented (79%), followed by Hispanic caregivers (68%). All families who identified as Native American responded (100%), while Caucasian caregivers responded the least (50%). It should be noted, however, that there were no differences on baseline demographic variables between treatment conditions among those participants who responded at the six-month follow-up assessment.

4Rs’ and 2Ss’ Strengthening Families Program (4R2S)

4R2S is a manualized, time-limited (16 weeks, 90–120 minutes per session) mental health service that targets school-age (seven to 11 years), children meeting diagnostic criteria for ODD or CD and their families, including adult caregivers, and siblings over six years of age. 4R2S typically involves six to eight families, and at least two generations of a family are present in each session. The multiple family group service delivery model allows for an in-vivo focus on changing family interactions by keeping children and parents together. Moreover, as multiple families can be seen at one time, this service delivery model addresses the chronic shortages in qualified child mental health providers within poverty-impacted settings, which often results in long waiting lists for families to access treatment (Asen, 2002). Additionally, 4R2S sessions are co-facilitated by teams of clinicians and family advocates (parents with the “lived” experience of caring for a child with emotional/behavioral difficulties and navigating the child mental health service system). Family advocates’ have been shown to deepen caregivers’ engagement in 4R2S (Gopalan et al., 2014) as feedback from peers with similar shared “lived” experiences is often perceived as more credible than suggestions offered by professionals (McKay et al. 1995). In combination, the 4R2S model uniquely allows families the opportunity to validate each other strengths, normalize family struggles, and empower each other through mutual aid, thus augmenting family members’ commitment to treatment and maximizing change (McKay et al., 1995). Session content is delivered via a mixture of didactic and experiential activities. At each session, families are provided with childcare for children five years old and younger, a meal, as well as transportation expenses. As indicated earlier, participants in the 4R2S condition were not restricted from obtaining other services within the participating site (e.g., medication management). Less than half of study participants (47%) received additional services, including individual therapy, medical management, school based mental health, case management and crisis management services. See Chacko et al (2015) for a description of services received, facilitator training, fidelity, and attendance.

Services-As-Usual (SAU)

In this comparison condition, youth and their families received any service typically provided by the participating site. This included medication management, case management, as well as individual, family, or group therapy (see Chacko et al, 2015, Gopalan et al., 2015 for a full description of SAU services).

Measures

Depressive symptoms were measured using the Center for Epidemiologic Studies Depression Scale (CESD; Radloff, 1977). The CESD is a free and publically available screening tool consisting of 20 items that assess the frequency of depressive symptoms occurring in the past week (0 = rarely/none of the time, 1 = some or a little of the time, 2 = occasionally or a moderate amount of time, 3 = most or all of the time). CESD scores are summed and range from 0–60, with a score of 16 and above considered clinically significant (Zich, Attkisson, & Greenfield, 1990). The current study had good internal consistency with Cronbach’s α at baseline of (0.89), mid-test (0.90), posttest (0.90), and six-month follow up (0.91). For the current study, 63% (n=200) of caregivers demonstrated clinically significant CESD scores (≥ 16) at baseline.

Parenting stress in the parent-child system was assessed using the Parenting Stress Index-short form (PSI; Abidin, 1995). Caregivers completed a 36 item questionnaire (e.g., “I often have the feeling that I cannot handle things very well”) with response options ranging from one (strongly disagree) to five (strongly agree). Possible scores on the PSI range from 36 to 180 with higher scores indicating increased levels of parent stress. Cronbach’s α’s are reported at baseline, mid-test, post-test, and six-month follow up, respectively: 0.91, 0.92, 0.94, 0.94. For the current study, 69% (n=221) of caregivers demonstrated clinically significant scores (≥ 90) on the PSI (Schaeffer. Alexander, Bethke, & Kretz, 2005).

Data Analysis

We used an intent to treat (ITT) analysis strategy to compare the treatment effect at post-test and six-month follow-up. All analyses were conducted on Stata 14 (StataCorp, 2015) and utilized mixed effects regression modeling recommended for analysis of longitudinal data with missing data (Gueorguieva & Krystal, 2004). Rather than deleting whole cases where there may be missing data on a single assessment point, mixed effects regression includes cases in analyses as long as there is at least one non-missing data point across all assessment periods. Furthermore, this method takes into consideration correlation between assessment periods within each case. All analyses allowed intercepts to vary randomly, and utilized 3 levels of nesting: assessments (mid, post, six-month follow-up) were nested within (1) individual participant IDs, which were nested within (2) family ID (22 families had more than one child in the 4R2S condition), which were then nested within (3) 4R2S group ID. Preliminary reliability analyses examined the level of clustering by clinic site for each outcome variable using reliability analysis. As all intraclass correlation coefficients (ICC) were non-significant at all time points, with Cronbach alpha’s all close to zero, we concluded that there was no evidence of clustering by clinic site. As result, we determined that it was not necessary to account for this level of nesting in analyses.

Main effects analyses

Main effects analysis included a dichotomous variable identifying the treatment condition (1 = 4R2S, 0 = SAU), dummy variables indicating the mid-test, post-test, and six-month follow up assessment periods (baseline as a reference), as well as Condition by assessment period dummy variable interactions (e.g., Condition X six-month follow-up). As indicated earlier, preliminary analyses indicated baseline differences in caregiver age by treatment condition. To determine if caregiver age impacted the treatment effect on outcome variables, we conducted preliminary sensitivity analyses including as covariates caregiver age and its interaction by all other model parameters. No systematic bias was detected, as no interaction coefficients of caregiver age by treatment condition were statistically significant. Consequently, we did not include caregiver age as a covariate for the final main effects analyses. After running multivariate models, linear contrasts were conducted to test for significant differences between 4R2S vs. SAU at each assessment point (between group differences). As recommended by Kraemer, Wilson, Fairburn, & Agras (2002), differences in treatment effects are best determined by combining the value of all parameters in the multivariate equation and subsequently comparing treatment effect sizes, as interpreting individual interaction parameters (e.g., Condition X Mid-test) is considered insufficient. Finally, Cohen’s d effect sizes were calculated by dividing each linear contrast estimate with the pooled sample baseline standard deviation for that outcome variable.

Moderation analyses

As indicated in the introduction section, examining the 4R2S treatment effect with all caregivers would include those who did not have any clinically significant levels of stress and depression at baseline, thus likely resulting in minimal change over time on such outcomes. Consequently, moderation analyses examined the 4R2S treatment effect separated by caregiver clinically significant status of stress and depression at baseline. Multivariate regression models examining the 4R2S treatment effect by clinically significant status included all covariates from the main effects models as well as a dummy variable indicating clinically significant (CS) status (1 = clinically significant, 0 = not clinically significant), and all interactions between the clinically significant dummy variable, treatment condition, and assessment point. As a result, multivariate models for each outcome included 2-way (Condition X Post-test, CS X Condition) and 3-way interactions (CS X Condition X Post).

Preliminary analyses examined differences in the baseline levels of the outcome and demographic variables by treatment condition separately for clinically significant and non-clinically significant cases, as significant differences between 4R2S and SAU treatment condition within each CS status could bias final mixed effects regression results. Among non-clinically significant participants for the PSI and the CESD, caregivers in the 4R2S condition were significantly younger than SAU caregivers (PSI: t = −2.04, df =96, p=.04; CESD: t = −2.15, df = 116, p = 0.03). Additionally, non-clinically significant SAU participants for the CESD were more likely to have a child identified as “Other” (e.g., not White/Caucasian, Black/African American, Hispanic/Latino, Native American, Asian/Pacific islander) race compared to 4R2S participants (χ2=6.69, df=1, p=0.01).

Consequently, we conducted an additional set of sensitivity analyses utilizing the same procedure documented for the main effects models. Caregiver age and the child race “other” categories were added as covariates along with their interactions with other model parameters for the relevant outcome variables (i.e., PSI – Caregiver age; CESD – Caregiver Age, Child Race-Other). No systematic bias was noted in these analyses for PSI or CESD. Consequently, multivariate models examining moderation effects of clinically significant status did not include caregiver age or child race as covariates. As recommended by Kraemer, et al. (2002), computation of multivariate regression models were followed by linear contrasts to test for significant between group differences at each assessment point (4R2S vs. SAU at post-test, six-month follow-up). Linear contrasts for between group analyses were conducted separately for clinically significant and non-clinically significant subgroups. Cohen’s d were calculated to determine treatment effect sizes for all contrasts.

Results

Table 1 presents means and SDs for all outcomes at each assessment point and by treatment condition. Multivariate regression parameters for main effects models are reported in Table 2. Despite the presence of significant interactions parameters between treatment condition and multiple assessment points (e.g., Condition X Post-Test) for the PSI and CESD in Table 2, the linear contrasts reported in Table 3 indicates that there were no significant between group effects found for the PSI or CESD at any assessment time point, with small treatment effect sizes reported throughout. Post-hoc analyses of trends over time (not shown) indicate that 4R2S participants’ scores on all outcomes decreased from baseline to six-month follow-up, while SAU participants’ scores either stayed constant or increased over time.

Table 1.

Means and SD of outcomes by treatment and assessment point

Outcome 4Rs and 2Ss group (4R2S) n=225
Means (SD)		 Services as usual (SAU) n=95
Means (SD)
Baseline(n=220) Mid-test(n=174) Post-test(n=177) Six-Month F/up(n=146) Baseline(n=95) Mid-test(n=70) Post-test(n=83) Six-Month F/up(n=75)
PSI 100.05 (22.98) 93.52 (24.37) 88.88(24.40) 86.72 (25.32) 95.41 (21.42) 94.67 (20.05) 94.46 (21.55) 93.04 (22.64)
CESD 19.84 (11.15) 18.52 (11.81) 17.95 (10.83) 16.82 (11.47) 18.09 (11.59) 16.51 (10.90) 19.20 (12.16) 19.62 (12.51)
Open in a new tab

PSI = Parenting Stress Index full scale; CESD = Center for Epidemiological Studies Depression Scale

Table 2.

Multivariate regression main effects model parameters for outcome variables

PSI CESD
b SE b SE
Intercept 95.26 2.52*** 18.13 1.24***
Condition 5.23 3.05 2.03 1.50
Mid-test dummy −0.08 2.33 −0.73 1.13
Post-test dummy −0.70 2.16 0.85 1.05
Six-month follow-up dummy −1.74 2.20 1.43 1.09
Condition x Mid −5.80 2.77* −0.50 1.35
Condition x Post −8.53 2.64** −2.73 1.27*
Condition x Six-month follow-up −10.52 2.72*** −4.34 1.34**
Open in a new tab

PSI = Parenting Stress Index full scale; CESD = Center for Epidemiological Studies Depression Scale

*

p<0.05

**

p<0.01

***

p<0.001

Table 3.

Linear contrasts for between groups main effects

Outcome variable n Assessment Multiple Family Group (4R2S) - Services as usual (SAU)
Contrast estimate (b) SE Z p Effect size (Cohen’s d)
PSI 223 Mid-test −0.57 3.31 −0.17 0.863 0.03
236 Post-test −3.30 3.19 −1.03 0.301 0.15
208 Six mo f/u −5.29 3.27 −1.62 0.105 0.23
CESD 224 Mid-test 1.53 1.61 0.95 0.342 0.14
245 Post-test −0.70 1.55 −0.45 0.651 0.06
207 Six mo f/u −2.31 1.60 −1.45 0.148 0.20
Open in a new tab

PSI = Parenting Stress Index full scale; CESD = Center for Epidemiological Studies Depression Scale

*

p<0.05

Table 4 presents the multivariate regression parameters for moderation effects models for all outcomes. Significant interaction parameters for treatment condition, CS indicator, and assessment point were found for the CESD outcome, suggesting the potential moderating effect of baseline clinically significant status on depressive symptoms at post-test. However, Table 5 presents linear contrast estimates for between group moderation effects, with no significant between group differences on the CESD at post-test. At the same time, Table 5 indicates that among participants who scored in the clinically significant range on each outcome variable, 4R2S participants report significantly lower scores at six-month follow-up on the PSI and CESD compared to SAU participants, with medium effect sizes for both outcomes. No significant between group differences were found for the non-clinically significant participants on either outcome.

Table 4.

Multivariate regression moderation effects model parameters

PSI CESD
b SE b SE
Intercept 75.04 3.19*** 7.95 2.21***
Condition 0.47 4.03 2.01 2.45
Mid-test dummy 7.26 3.64 2.35 1.68
Post-test dummy 5.74 3.40 2.60 1.58
six-month follow-up dummy 2.11 3.44 3.40 1.58*
Condition x Mid −3.39 4.43 0.52 2.01
Condition x Post −8.26 4.23 0.82 1.94
Condition x Six-month follow-up −3.70 4.36 −2.44 1.98
CS dummy 32.86 4.04*** 18.58 1.96***
Condition x CS 3.98 5.02 −1.86 2.35
Mid-test dummy x CS −12.05 4.63** −6.08 2.24**
Post-test dummy x CS −10.78 4.33* −3.26 2.09
six-month follow-up dummy x Outcome −6.39 4.39 −3.32 2.16
Condition x Mid-test dummy x CS −2.11 5.59 −0.67 2.67
Condition x Post-test dummy x CS 0.57 5.35 −5.47 2.54*
Condition x Six-month follow-up dummy x CS −10.69 5.51 −3.07 2.66
Open in a new tab

PSI = Parenting Stress Index full scale; CESD = Center for Epidemiological Studies Depression Scale

*

p<0.05,

**

p<0.01,

***

p<0.001

Table 5.

Linear contrasts for between groups moderation effects

Outcome variable n Assessment Clinically Significant Non-Clinically Significant
Contrast estimate (b) SE Z p Effect size (Cohen’s d) Contrast estimate (b) SE Z p Effect size (Cohen’s d)
PSI 223 Mid-test −1.05 3.50 −0.30 0.765 0.05 −2.92 4.53 −0.64 0.520 0.13
236 Post-test −3.24 3.38 −0.96 0.338 0.14 −7.79 4.34 −1.79 0.073 0.35
208 Six mo f/u −9.94 3.47 −2.86 0.004** 0.44 −3.23 4.46 −0.72 0.470 0.14
CESD 224 Mid-test −0.00 2.45 −0.00 0.999 0.00 2.53 2.63 0.96 0.335 0.22
245 Post-test −4.50 2.37 −1.90 0.058 0.40 2.83 2.58 1.10 0.272 0.25
207 Six mo f/u −5.36 2.46 −2.18 0.029* 0.47 −0.43 2.61 −0.16 0.870 0.04
Open in a new tab

PSI = Parenting Stress Index full scale; CESD = Center for Epidemiological Studies Depression Scale

*

p<0.05

**

p<0.01

***

p <0.001

Discussion

Given the high prevalence of DBDs amongst children living in poverty (Bannon & McKay, 2007; Ghandour, Kogan, Blumberg, Jones, & Perrin, 2012), and uneven findings regarding the impact of behavioral parent training (BPT) alone upon parent-level outcomes for families living in poverty-impacted communities (Charles et al., 2013; Gardner et al., 2006; Hutchings et al., 2007; Larsson et al., 2009; Sigmarsdottir et al., 2009; Taylor et al., 1998), this study examined the incremental effects of the 4Rs and 2Ss Strengthening Families Program (4R2S) to decrease caregiver stress and depressive symptoms within a resource-poor, high-risk population seeking services in routine clinical settings. Specifically, caregivers in the 4R2S treatment condition were hypothesized to have significantly improved levels of parental stress and depressive symptoms over time, and in comparison to participants in the services as usual (SAU) condition. Importantly, we also explored whether the incremental treatment benefit of 4R2S differed based upon whether or not caregivers entered the study with clinically significant levels of stress and depressive symptoms. We hypothesized that individuals with clinically significant levels of stress and depressive symptoms, in particular, would demonstrate significant 4R2S incremental treatment benefits compared to SAU.

Although main effect analyses manifested no significant differences between 4R2S and SAU participants’ PSI or CESD scores, moderation effects analyses indicated beneficial between group effects of 4R2S (vs. SAU) at six-month follow-up for those caregivers who scored in the clinically significant range at baseline on the PSI and CESD. This is consistent with our a priori hypothesis that only those caregivers who demonstrated clinically significant stress and depressive symptoms would demonstrate significant 4R2S incremental treatment benefits.

Among those previously reviewed studies with poverty-relevant contextual conditions, moderate to large treatment main effects sizes (d = 0.48 to 1.07) in favor of BPT were reported for parental stress and depression outcomes. However, none of these studies utilized services as usual as control condition, evaluated the incremental benefit of BPT over SAU, nor examined the treatment effect among those caregivers with clinically significant levels of stress and depressive symptoms. In our review, we were able to find one study that conducted analyses among participants with clinically significant levels of depressive symptoms (Stattin et al., 2015), which found significant improvements for participants involved in three of the four BPT interventions (Comet, Cope, and Connect) in comparison to the wait list control group, with effect sizes ranging from 0.23 to 0.30. Importantly, our study found larger effect sizes for participants with clinically significant levels of stress and depressive symptoms (d = 0.44 – 0.47), in comparison to the more conservative control condition (services as usual), and more accurately reflects the realities of poverty-impacted families utilizing publically-available community-based services.

Collectively, these findings align with other studies that suggest that interventions, largely derived from BPT, can have a significant impact on both child DBDs (e.g., Eyberg et al., 2008) and, perhaps equally important, parent-level outcomes (Chronis et al., 2013; Miller & Prinz, 1990; Rajwan, Chacko, Wymbs, & Wymbs, 2014). Importantly, the current study suggests that relatively large group intervention formats, such as 4R2S, can provide benefits for families in key areas that are putatively related to the persistence of DBDs. Unlike other group-based formats, 4R2S focuses on multiple family members working together within the group, allowing for greater efficiency in service delivery relative to group-based intervention that are parent- or child-only groups. The data suggests that the 4R2S format, while more complex, may lead to significant improvements for those caregivers who have clinically significant levels of depressive symptoms and parental stress.

The clinical significance of the data should also be put in context. 4R2S is one among a handful of BPT-based interventions focused on school-age children and has undergone an effectiveness trial in resource poor communities primarily with families of color. Importantly, participants in our study manifested numerous risk factors, owing to high rates of public health insurance as a proxy of poverty (70%), single-parent status (67%), as well as clinically significant levels of depressive symptoms (63%), and parental stress (69%). Based on our review, few BPT effectiveness trials examining parental stress and depression as outcomes demonstrated relevant methodological features for high-risk, poverty-impacted families accessing routine, publically-available social services. Consequently, the results of the current study are among the few that highlight the potential incremental benefits of BPT for treating DBDs among school-age youth, and potentially caregiver stress and depression, in resource-poor communities with poverty-impacted populations.

In light of these findings, the study has several limitations suggesting the need for replication. Although attention was placed on reducing selection bias, the assignment to treatment condition was not randomized. In addition, 4R2S participants responded less at the six-month follow up assessment compared to SAU participants. This attrition raises concerns that those in the sample at six-month follow up may not be representative of the full sample at baseline. However, there were no significant baseline demographic differences between treatment conditions among those who did complete the six-month follow-up assessment. Moreover, because the study sample is predominantly populated by ethnic minority participants, the response pattern at the six-month follow-up assessment suggests that those participants remaining in the sample were representative of the target population. As a result, we concluded that the threat to sample representativeness due to attrition at the six-month follow-up assessment was minimal. All families in the current study were self-referred to child mental health treatment, and may be substantially different from those families who may not have overcome the stigma of seeking mental health treatment for their children. Finally, due to the exclusion criteria for this study, findings for this study may not generalize to younger and older children, as well as those youths with sub-threshold levels of DBD symptoms.

Taken together, the results of this study suggest that 4R2S may have the potential to impact a substantial number of poverty-impacted, school-age children and families given the influence of caregiver stress and depression on child externalizing behaviors, as well as the risk that parental stress and depressive symptoms may undermine therapeutic child outcomes (Beauchaine, Webster-Stratton, & Reid, 2005; Chronis et al., 2004; Kazdin & Whitley, 2003). Future research should examine the difference in child behavior outcomes by clinically significant status of caregiver stress and depression at baseline, as well as how reductions in stress and depressive symptoms through the course of 4R2S ultimately impact behavior among school-age children.

These findings suggest that clinicians working with children should be cognizant of how interventions for children may impact the emotional health of their caregivers. Ideally, practice-level recommendations include screening caregivers throughout treatment for their children for common mental health problems such as depression (Chacko et al., 2015; Rajwan et al., 2014), particularly in clinics where poverty-impacted families frequent. There is a fairly robust literature that shows screening programs, particularly in settings where caregivers are likely to attend, facilitates detection of untreated mental health problems (Acri et al., 2013; Smith, Cavaleri, Poschman, Howell & Yonkers, 2006). When putative parent-level issues (e.g., stress and depressive symptoms) are not sufficiently addressed through BPTs, enhancements with demonstrated effectiveness (e.g., Chacko et al., 2009), as well referring caregivers to receive their own mental health treatment may be warranted (Rajwan et al., 2014). Ideally, co-locating child and adult mental health services within a comprehensive care model addressing the needs of the entire family has the greatest potential for coordinating and streamlining care, decreasing logistical barriers to help-seeking, and ultimately improving child and family outcomes.

Acknowledgments

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this study was obtained through R01 MH072649 (principal investigator: McKay). Salary support for this study also came from National Institute of Mental Health (NIMH; F32MH090614; Gopalan). Dr. Gopalan is also an investigator with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis, through an award from the NIMH (R25MH080916-01A2) and the Department of Veterans Affairs, Health Services Research and Development Service, Quality Enhancement Research Initiative (QUERI).

Footnotes

The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIMH or the National Institutes of Health.

Contributor Information

Geetha Gopalan, School of Social Work, University of Maryland, 525 W. Redwood Street, Baltimore, MD 21201, 410-706-3616.

Lindsay A. Bornheimer, McSilver Institute for Poverty Policy & Research, Silver School of Social Work, New York University, 41 East 11th Street, New York, NY 10003, 212-998-5937.

Mary C. Acri, McSilver Institute for Poverty Policy & Research, New York University Medical Center, 41 East 11th Street, New York, NY 10003, 212-998-5937.

Andrew Winters, School of Social Work, University of Maryland, 525 W. Redwood Street, Baltimore, MD 21201, 410-706-3616.

Kyle H. O’Brien, McSilver Institute for Poverty Policy & Research, Silver School of Social Work, New York, University, 41 East 11th Street, New York, NY 10003, 212-998-5937.

Anil Chacko, Department of Applied Psychology, Steinhardt School of Culture, Education and Human, Development, New York University, 246 Greene Street, New York, NY 10003, 212-998-5749.

Mary M. McKay, Brown School of Social Work, Washington University in St Louis, (314) 935-6693.

References

  1. Abidin RR. Parenting Stress Index, Third Edition: Professional Manual. Odessa, FL: Psychological Assessment Resources, Inc; 1995. [Google Scholar]
  2. Acri MC, Frank S, Olin SS, Burton G, Ball JL, Weaver J, Hoagwood KE. Examining the feasibility and acceptability of a screening and outreach model developed for a peer workforce. Journal of Child & Family Studies. 2013:1–10. doi: 10.1007/s10826-013-9841-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5. Arlington, VA: American Psychiatric Publishing; 2013. [Google Scholar]
  4. Asen E. Multiple family therapy: An overview. Journal of Family Therapy. 2002;24(1):3–16. [Google Scholar]
  5. Bannon WMJ, McKay MM. Addressing urban African American youth externalizing and social problem behavioral difficulties in a family oriented prevention project. Social Work in Mental Health. 2007;5(1–2):221–240. [Google Scholar]
  6. Barth RP. Preventing child abuse and neglect with parent training: Evidence and opportunities. The Future of Children. 2009;19(2):95–118. doi: 10.1353/foc.0.0031. [DOI] [PubMed] [Google Scholar]
  7. Beauchaine TP, Hinshaw SP, editors. Child and adolescent psychopathology. 2. Hoboken, NJ: John Wiley & Sons; 2013. [Google Scholar]
  8. Beauchaine TP, Webster-Stratton C, Reid MJ. Mediators, moderators, and predictors of 1-year outcomes among children treated for early-onset conduct problems: A latent growth curve analysis. Journal of Consulting and Clinical Psychology. 2005;73(3):371–388. doi: 10.1037/0022-006X.73.3.371. [DOI] [PubMed] [Google Scholar]
  9. Braet C, Meerschaert T, Merlevede E, Bosmans G, Van Leeuwen K, De Mey W. Prevention of antisocial behaviour: Evaluation of an early intervention programme. European Journal of Developmental Psychology. 2009;6(2):223–240. [Google Scholar]
  10. Chacko A, Wymbs BT, Wymbs FA, Pelham WE, Swanger-Gagne MS, Girio E, … O’Connor B. Enhancing traditional behavioral parent training for single mothers of children with ADHD. Journal of Clinical Child & Adolescent Psychology. 2009;38(2):206–218. doi: 10.1080/15374410802698388. [DOI] [PubMed] [Google Scholar]
  11. Chacko A, Gopalan G, Franco L, Dean-Assael K, Jackson J, Marcus S, … McKay M. Multiple family group service model for children with disruptive behavior disorders: Child outcomes at post-treatment. Journal of Emotional and Behavioral Disorders. 2015:1–11. doi: 10.1177/1063426614532690. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Charles JM, Bywater TJ, Edwards RT, Hutchings J, Zou L. Parental depression and child conduct problems: Evaluation of parental service use and associated costs after attending the incredible years basic parenting programme. BMC Health Services Research. 2013;13:523. doi: 10.1186/1472-6963-13-523. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Chorpita BF, Daleiden EL. Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology. 2009;77(3):566–579. doi: 10.1037/a0014565. [DOI] [PubMed] [Google Scholar]
  14. Chronis AM, Chacko A, Fabiano GA, Wymbs BT, Pelham WE. Enhancements to the behavioral parent training paradigm for families of children with ADHD: Review and future directions. Clinical Child and Family Psychology Review. 2004;7(1):1–27. doi: 10.1023/b:ccfp.0000020190.60808.a4. [DOI] [PubMed] [Google Scholar]
  15. Chronis-Tuscano A, Clarke TL, O’Brien KA, Raggi VL, Diaz Y, Mintz AD, … Lewinsohn P. Development and preliminary evaluation of an integrated treatment targeting parenting and depressive symptoms in mothers of children with attention-deficit/hyperactivity disorder. Journal of Consulting and Clinical Psychology. 2013;81(5):918–925. doi: 10.1037/a0032112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Coiro MJ, Riley A, Broitman M, Miranda J. Effects on children of treating their mothers’ depression: Results of a 12-month follow-up. Psychiatric Services. 2012;63(4):357–363. doi: 10.1176/appi.ps.201100126. [DOI] [PubMed] [Google Scholar]
  17. Conger RD, Conger KJ, Martin MJ. Socioeconomic status, family processes, and individual development. Journal of Marriage and Family. 2010;(3):685–704. doi: 10.1111/j.1741-3737.2010.00725.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Deater-Deckard K, Dodge KA. Externalizing behavior problems and discipline revisited: Nonlinear effects and variation by culture, context, and gender. Psychological Inquiry. 1997;8:161–165. [Google Scholar]
  19. Deater-Deckard K. Parenting stress and child adjustment: Some old hypotheses and new questions. Clinical Psychology: Science and Practice. 1998;5(3):314–332. [Google Scholar]
  20. DeGarmo DS, Patterson GR, Forgatch MS. How do outcomes in a specified parent training intervention maintain or wane over time? Prevention Science. 2004;5(2):73–89. doi: 10.1023/b:prev.0000023078.30191.e0. [DOI] [PubMed] [Google Scholar]
  21. Dumville JC, Torgerson DJ, Hewitt CE. Research methods: Reporting attrition in randomised control trials. British Medical Journal. 2006;332(7547):969–971. doi: 10.1136/bmj.332.7547.969. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Ertel KA, Rich-Edwards JW, Koenen KC. Maternal depression in the United States: Nationally representative rates and risks. Journal of Women’s Health. 2011;20(11):1609–1617. doi: 10.1089/jwh.2010.2657. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Evans GW, Boxhill L, Pinkava M. Poverty and maternal responsiveness: The role of maternal stress and social resources. International Journal of Behavioral Development. 2008;32(3):232–237. [Google Scholar]
  24. Evans SW, Owens JS, Bunford N. Evidence-based psychosocial treatments for children and adolescents with attention-deficit/hyperactivity disorder. Journal of Clinical Child & Adolescent Psychology. 2014;43(4):527–551. doi: 10.1080/15374416.2013.850700. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Eyberg SM, Nelson MM, Boggs SR. Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology. 2008;37(1):215–237. doi: 10.1080/15374410701820117. [DOI] [PubMed] [Google Scholar]
  26. Forgatch MS, Patterson GR. Parent management training - Oregon model: An intervention for antisocial behaviors in children and adolescents. In: Weisz JR, Kazdin AE, editors. Evidenced-based psychotherapies for children and adolescents. 2. New York: Guilford; 2010. pp. 159–178. [Google Scholar]
  27. Gardner F, Burton J, Klimes I. Randomised controlled trial of a parenting intervention in the voluntary sector for reducing child conduct problems: Outcomes and mechanisms of change. Journal of Child Psychology & Psychiatry. 2006;47(11):1123–1132. doi: 10.1111/j.1469-7610.2006.01668.x. [DOI] [PubMed] [Google Scholar]
  28. Garland AF, Hawley KM, Brookman-Frazee L, Hurlburt MS. Identifying common elements of evidence-based psychosocial treatments for children’s disruptive behavior problems. Journal of the American Academy of Child and Adolescent Psychiatry. 2008;47(5):505–514. doi: 10.1097/CHI.0b013e31816765c2. [DOI] [PubMed] [Google Scholar]
  29. Ghandour RM, Kogan MD, Blumberg SJ, Jones JR, Perrin JM. Mental health conditions among school-aged children: Geographic and sociodemographic patterns in prevalence and treatment. Journal of Developmental and Behavioral Pediatrics. 2012;33(1):42–54. doi: 10.1097/DBP.0b013e31823e18fd. [DOI] [PubMed] [Google Scholar]
  30. Gopalan G, Chacko A, Franco LM, Dean-Assael KM, Rotko L, Marcus SM, … McKay MM. Multiple family group service delivery model for youth with disruptive behaviors: Child outcomes at Six-Month follow-up. Journal of Child and Family Studies. 2015;24(9):2721–2733. doi: 10.1007/s10826-014-0074-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Gopalan G, Acri M, Lalayants M, Hooley C, Einbinder E. Child welfare involved caregiver perceptions of family support in child mental health. Journal of Family Strengths. 2014;14(1) Article 5. [PMC free article] [PubMed] [Google Scholar]
  32. Goodwin PJ, Leszcz M, Ennis M, Koopmans J, Vincent L, Guther H, Hunter J. The effect of group psychosocial support on survival in metastatic breast cancer. New England Journal of Medicine. 2001;345:1719–1726. doi: 10.1056/NEJMoa011871. [DOI] [PubMed] [Google Scholar]
  33. Gueorguieva R, Krystal JH. Move over ANOVA: Progress in analyzing repeated-measures data and its reflection in papers published in the Archives of General Psychiatry. Archives of General Psychiatry. 2004;61(3):310–317. doi: 10.1001/archpsyc.61.3.310. [DOI] [PubMed] [Google Scholar]
  34. Hall P. Current considerations of the effects of untreated maternal perinatal depression and the National Perinatal Depression Initiative. Journal of Developmental Origins of Health and Disease. 2012;3(04):293–295. doi: 10.1017/S2040174412000013. [DOI] [PubMed] [Google Scholar]
  35. Hutchings J, Gardner F, Bywater T, Daley D, Whitaker C, Jones K, … Scott Parenting intervention in Sure Start services for children at risk of developing conduct disorder: Pragmatic randomised controlled trial. BMJ: British Medical Journal. 2007;(7595):678–681. doi: 10.1136/bmj.39126.620799.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Jackson JM. Multi-family groups for multi-stressed families: Initial outcomes and future implications. Research on Social Work Practice. 2015;25(5):537–548. [Google Scholar]
  37. Kazdin AE, Siegel TC, Bass D. Cognitive problem-solving skills training and parent management training in the treatment of antisocial behavior in children. Journal of Consulting and Clinical Psychology. 1992;60(5):733. doi: 10.1037//0022-006x.60.5.733. [DOI] [PubMed] [Google Scholar]
  38. Kazdin AE, Whitley MK. Treatment of parental stress to enhance therapeutic change among children referred for aggressive and antisocial behavior. Journal of Consulting and Clinical Psychology. 2003;71(3):504–515. doi: 10.1037/0022-006x.71.3.504. [DOI] [PubMed] [Google Scholar]
  39. Kilgore K, Snyder J, Lentz C. The contribution of parental discipline, parental monitoring, and school risk to early-onset conduct problems in African American boys and girls. Developmental Psychology. 2000;36(6):835–845. doi: 10.1037//0012-1649.36.6.835. [DOI] [PubMed] [Google Scholar]
  40. Kraemer HC, Wilson GT, Fairburn CG, Agras WS. Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry. 2002;59(10):877–883. doi: 10.1001/archpsyc.59.10.877. [DOI] [PubMed] [Google Scholar]
  41. Larsson B, Fossum S, Clifford G, Drugli MB, Handegård BH, Mørch W. Treatment of oppositional defiant and conduct problems in young Norwegian children: Results of a randomized controlled trial. European Child & Adolescent Psychiatry. 2009;18(1):42–52. doi: 10.1007/s00787-008-0702-z. [DOI] [PubMed] [Google Scholar]
  42. Lovejoy MC, Graczyk PA, O’Hare E, Neuman G. Maternal depression and parenting behavior. A meta-analytic review. Clinical Psychology Review. 2000;20:561–592. doi: 10.1016/s0272-7358(98)00100-7. [DOI] [PubMed] [Google Scholar]
  43. Marchand JF, Hock E, Widaman K. Mutual relations between mothers’ depressive symptoms and hostile-controlling behavior and young children’s externalizing behavior and internalizing behavior problems. Parenting. 2002;2:335–353. [Google Scholar]
  44. McKay MM, Bannon WMJ. Engaging families in child mental health services. Child and Adolescent Psychiatric Clinics of North America. 2004;13(4):905–921. doi: 10.1016/j.chc.2004.04.001. [DOI] [PubMed] [Google Scholar]
  45. McKay MM, Gopalan G, Franco L, Dean-Assael K, Chacko A, Jackson JM, Fuss A. A collaboratively designed child mental health service model: Multiple family groups for urban children with conduct difficulties. Research on Social Work Practice. 2011;21(6):664–674. doi: 10.1177/1049731511406740. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. McKay MM, Gonzales JJ, Stone S, Ryland D, Kohner K. Multiple family therapy groups: A responsive intervention model for inner city families. Social Work with Groups. 1995;18(4):41–56. [Google Scholar]
  47. Michelson D, Davenport C, Dretzke J, Barlow J, Day C. Do evidence-based interventions work when tested in the “real world?” A systematic review and meta-analysis of parent management training for the treatment of child disruptive behavior. Clinical Child and Family Psychology Review. 2013;16(1):18–34. doi: 10.1007/s10567-013-0128-0. [DOI] [PubMed] [Google Scholar]
  48. Miller GE, Prinz RJ. Enhancement of social learning family interventions for child conduct disorder. Psychological Bulletin. 1990;108:291–307. doi: 10.1037/0033-2909.108.2.291. [DOI] [PubMed] [Google Scholar]
  49. Munson JA, McMahon RJ, Spieker S. Structure and variability in the developmental trajectory of children’s externalizing behavior problems: Impact of infant attachment, maternal depressive symptomatology, and child sex. Development and Psychopathology. 2001;13:277–296. doi: 10.1017/s095457940100205x. [DOI] [PubMed] [Google Scholar]
  50. Pelham WE, Gnagy EM, Greenslade KE, Milich R. Teacher ratings of DSM-III-R symptoms for the disruptive behavior disorders. Journal of the American Academy of Child & Adolescent Psychiatry. 1992;31(2):210–218. doi: 10.1097/00004583-199203000-00006. [DOI] [PubMed] [Google Scholar]
  51. Pisterman S, Firestone P, McGrath P, Goodman JT, Webster I, Mallory R, Coffin B. The effects of parent training on parenting stress and sense of competence. Canadian Journal of Behavioural Science/Revue Canadienne Des Sciences Du Comportement. 1992;24(1):41–58. [Google Scholar]
  52. Radloff LS. The CES-D scale a self-report depression scale for research in the general population. Applied Psychological Measurement. 1977;1(3):385–401. [Google Scholar]
  53. Rajwan E, Chacko A, Wymbs BT, Wymbs FA. Evaluating clinically significant change in mother and child functioning: Comparison of traditional and enhanced behavioral parent training. Journal of Abnormal Child Psychology. 2014;42(8):1407–1412. doi: 10.1007/s10802-014-9877-6. [DOI] [PubMed] [Google Scholar]
  54. Scott S, Knapp M, Henderson J, Maughan B. Financial cost of social exclusion: Follow up study of antisocial children into adulthood. BMJ: British Medical Journal. 2001;323(7306):191–194. doi: 10.1136/bmj.323.7306.191. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Schaeffer CM, Alexander PC, Bethke K, Kretz LS. Predictors of child abuse potential among military parents: Comparing mothers and fathers. Journal of Family Violence. 2005;20(2):123–129. [Google Scholar]
  56. Siefert K, Finlayson TL, Williams DR, Delva J, Ismail AI. Modifiable risk and protective factors for depressive symptoms in low-income African American mothers. American Journal of Orthopsychiatry. 2007;77(1):113–123. doi: 10.1037/0002-9432.77.1.113. [DOI] [PubMed] [Google Scholar]
  57. Sigmarsdóttir M, Thorlacius Ö, Guđmundsdóttir EV, DeGarmo DS. Treatment effectiveness of PMTO for children’s behavior problems in Iceland: Child outcomes in a nationwide randomized controlled trial. Family Process. 2015;54(3):498–517. doi: 10.1111/famp.12109. [DOI] [PubMed] [Google Scholar]
  58. Smith MV, Cavaleri MA, Howell H, Poschman K, Yonkers KA. Screening for and detection of depression, panic disorder and PTSD in public-sector obstetrical clinics. Psychiatric Services. 2004;55(4):407–414. doi: 10.1176/appi.ps.55.4.407. [DOI] [PubMed] [Google Scholar]
  59. Spieker SJ, Larson NC, Lewis SM, Keller TE, Gilchrist L. Developmental trajectories of disruptive behavior problems in preschool children of adolescent mothers. Child Development. 1999;70:443–458. doi: 10.1111/1467-8624.00032. [DOI] [PubMed] [Google Scholar]
  60. StataCorp. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP; 2015. [Google Scholar]
  61. Stattin H, Enebrink P, Özdemir M, Giannotta F. A national evaluation of parenting programs in Sweden: The short-term effects using an RCT effectiveness design. Journal of Consulting and Clinical Psychology. 2015;83(6):1069–1084. doi: 10.1037/a0039328. [DOI] [PubMed] [Google Scholar]
  62. Taylor TK, Schmidt F, Pepler D, Hodgins C. Original research: A comparison of eclectic treatment with Webster-Stratton’s parents and children series in a children’s mental health center: A randomized controlled trial. Behavior Therapy. 1998;29:221–240. [Google Scholar]
  63. Weisz JR, Kuppens S, Eckshtain D, Ugueto AM, Hawley KM, Jensen-Doss A. Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. Jama Psychiatry. 2013a;70(7):750–761. doi: 10.1001/jamapsychiatry.2013.1176. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Weisz JR, Ugueto AM, Cheron DM, Herren J. Evidence-based youth psychotherapy in the mental health ecosystem. Journal of Clinical Child & Adolescent Psychology. 2013b;42(2):274–286. doi: 10.1080/15374416.2013.764824. [DOI] [PubMed] [Google Scholar]
  65. Yoshikawa H, Aber JL, Beardslee WR. The effects of poverty on the mental, emotional, and behavioral health of children and youth implications for prevention. American Psychologist. 2012;67(4):272–284. doi: 10.1037/a0028015. [DOI] [PubMed] [Google Scholar]
  66. Zich JM, Attkisson CC, Greenfield TK. Screening for depression in primary care clinics: The CESD-D and the BDI. International Journal of Psychiatry in Medicine. 1990;20(3):259–277. doi: 10.2190/LYKR-7VHP-YJEM-MKM2. [DOI] [PubMed] [Google Scholar]

RESOURCES