Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Sep 8.
Published in final edited form as: J Emot Behav Disord. 2014 May 2;23(3):131–143. doi: 10.1177/1063426614532948

Factors contributing to reduced caregiver strain in a publicly-funded child mental health system

Erin C Accurso a,b, Ann F Garland c,d,e, Rachel Haine-Schlagel e,f, Lauren Brookman-Frazee d,e, Mary J Baker-Ericzén e,g
PMCID: PMC4562689  NIHMSID: NIHMS594290  PMID: 26361434

Abstract

This study examined caregiver strain in families who initiated mental health services for their child. Predictors of strain and the bidirectional relation between strain and child symptoms were examined. Participants included 218 children ages 4–13 with disruptive behavior problems and their caregivers, plus 96 psychotherapists, recruited from six publicly-funded clinics. Child disruptive behavior severity and caregiver strain were assessed at baseline, four, and eight months. Multilevel models were used to examine predictors of reduced caregiver strain, and autoregressive cross-lagged models were used to examine the bidirectional relations between change in caregiver strain and behavior problems over time. There were small to medium decreases in caregiver strain over the eight months after the initiation of mental health services, but few factors predicted change other than initial behavior problem severity. While more severe initial child symptoms predicted greater reductions in caregiver strain, greater child symptom severity sustained at four months predicted lesser improvements in caregiver strain. Simultaneously, greater caregiver strain predicted less improvement in child symptom severity, suggesting that child symptom severity and caregiver strain impact each other over time. These results suggest that attending to both child and caregiver factors may be important in maintaining improvements after initiating usual care.

Introduction

Many caregivers experience significant stress and burden as a result of caring for their children with mental health problems, with 6–11% of caregivers in community samples reporting clinically elevated levels of strain (Angold et al., 1998). Caregivers of children with disruptive behavior problems experience particularly high levels of caregiver strain given the extra demands of caring for children with behavioral problems (Tsai, Yeh, & Slymen, 2013). Caregiver strain includes both observable negative life occurrences or objective strain (e.g., financial strain, interruptions at work, spending less time with other family members/friends) and negative emotions or subjective impacts that are both internalized (e.g., sadness, worry, guilt) and externalized (e.g., anger, resentment, embarrassment) (Brannan, Heflinger, & Bickman, 1997).

A growing body of evidence suggests that caregiver strain influences receipt of child mental health services beyond what can be explained by child symptom severity and functioning (Brannan & Heflinger, 2006). While greater caregiver strain is associated with greater initial use of child mental health services (Brannan & Heflinger, 2006; Bussing, Mason, Leon, & Sinha, 2003; Cook et al, 2004; Garland, Aarons, Hawley, & Hough, 2003; Shin & Brown, 2009), it is also associated with long gaps in treatment (Brannan et al., 2003), poor service coordination (Yatchmenoff, Koren, Friesen, Gordon, & Kinney, 1998), and premature treatment drop-out in community mental health settings (Pellerin, Costa, Weems, & Dalton, 2010). Additionally, higher levels of caregiver strain predict use of more costly services, including psychiatric hospitalization (Bickman, Foster, & Lambert, 1996), residential or inpatient care (Brannan et al., 2003; Cook et al., 2004), and medication use (Cook et al., 2004). Subjective internalized strain in particular appears to be associated with receiving a higher level of care and more inconsistent care (Brannan et al., 2003). Given that higher caregiver strain has been associated with inefficiencies in care and higher overall mental health service costs, it is a relevant factor in efforts to improve the quality and effectiveness of community-based care for children with behavior problems.

Results of efficacy studies indicate that parent training (independent of child involvement in treatment) is associated with reductions in caregiver strain (Anastopoulos, Shelton, DuPaul, & Guevremont, 1993; Feldman & Werner, 2002; Moretti & Obsuth, 2009). In addition, there is some evidence to suggest that evidence-based treatments for children with behavioral problems are more effective when perceived caregiver stress is reduced (Kazdin & Whitley, 2003). The reduction of caregiver strain may be a key factor in maintaining child outcomes over time. Indeed, psychiatrically hospitalized children whose caregivers reported high strain initially but low strain after discharge showed the largest and most sustained decreases in externalizing symptoms (Blader, 2006). However, the impact of usual care psychotherapy (i.e., routine psychotherapy provided in community-based, non-research settings) on caregiver strain is unknown. Furthermore, the impact of caregiver strain on child mental health outcomes in usual outpatient care is not well understood.

Previous research has demonstrated that a number of factors contribute to strain, including child’s severity of emotional and behavioral symptoms (Brannan & Heflinger, 2001; Bussing et al., 2003; Sales, Greeno, Shear, & Anderson, 2004), service system characteristics (Brannan & Heflinger, 2006), and caregiver characteristics, such as educational level, family income, and race/ethnicity (Kang, Brannan & Heflinger, 2005; McDonald, Gregorie, Poertner, & Early, 1997; Shin & Brown, 2009). However, little is known about the extent to which this association may be bidirectional. In addition, little is known about which factors (child, caregiver, and service system) may predict greatest change in strain associated with treatment. Finally, caregiver strain in usual care is particularly relevant to recent calls to reform mental health care for children, with greater emphasis on addressing caregivers’ needs in treatment (see Hoagwood & Burns, 2014). Within this reformed family-centered model of care, efforts are made for caregivers to be better informed and empowered to take an active role in their child’s treatment, therefore helping to sustain their child’s treatment gains. By improving our understanding of caregiver strain in usual care, family support services could be tailored more effectively.

This study examines caregiver strain within a community sample of families involved in publicly-funded out-patient treatment for their child with a disruptive behavior problem. The first purpose of this study is to describe the patterns of change in caregiver strain following initiation of usual care child mental health services. Second, this study seeks to identify predictors of change in caregiver strain from the following areas: child, caregiver, system characteristics, and service use. Finally, the direct relation between child symptom severity and caregiver strain is examined over time, with the hypothesis that it is bidirectional (i.e., higher child symptom severity increases caregiver strain, and higher strain can exacerbate child symptoms).

Methods

Data for this study were collected as part of the Practice and Research: Advancing Collaboration (PRAC) study—an observational study that comprehensively examined usual care for children with disruptive behavior problems served in community-based out-patient clinics from 2004 to 2007. The main aims of the original study were to describe the type of care provided to children who presented to usual care and to examine the impact of care on child and family outcomes. Participants were followed for 16 months following service entry. Please see Garland et al. (2010) for a detailed description of the practice settings, client recruitment, and therapist characteristics.

Participants

Participating Clinics

The six participating clinics represent the largest contractors for publicly-funded, clinic-based out-patient mental health care for children in one of the largest counties in the U.S. and serve ethnically and diagnostically diverse children and their families. No intervention to influence service delivery took place during the study period.

Therapist Participants

All therapists from participating clinics were assigned random numbers and recruited sequentially until a sample was gathered that reflected the county’s therapist distribution by discipline, training, and licensure status. In the three years following initial recruitment, all new therapists working at least half-time were recruited. Most recruited therapists (n=131, 80%) agreed to participate, but only 96 had a patient enrolled in the study who consented and subsequently participated in the study. These 96 therapists were primarily female (n=81, 84%) and Caucasian (n=64, 67%) with an average age of 32.4 years (SD = 9.1) and an average of 2.9 (SD = 3.6; range [0,25]) years of therapy experience.

Child and Parent Participants

A total of 218 children (84% of eligible participants) were included in the current study. Inclusion criteria for child participants included: (a) presenting problems included a disruptive behavior problem (aggression, defiance, delinquency, oppositional behavior by parent report), (b) age 4–13 years, (c) primary language for child and parent was English or Spanish, and (d) child was entering a new episode of psychotherapy (defined as no therapy for the previous three months) with a participating therapist. Characteristics of the 218 participating children and their caregivers are provided in Table 1. The average child age was 9 years, the majority of participants were male (68%), and fewer than half were non-Hispanic Caucasian. Although all children presented with disruptive behavior problems, their clinician-assigned primary diagnoses varied, with ADHD being the most common (n=82; 38%). Caregivers (herein referred to as parents) of these children were primarily female (n=206, 94.5%) mothers (n=178, 81.7%) with an average age of 39.7 years (SD = 10.1). Approximately two thirds had at least a high school diploma (n=142, 67.0%); the other third was split between having less than a high school education (n=39, 18.4%) and having at least a college degree (n= 31, 14.6%). Average annual household income was $30,739 (SD = 34,454). Sixteen percent (n=35) spoke Spanish as a primary language.

Table 1.

Sample descriptives for study participants.

Variable n Mean (SD) or % Range
Caregiver Strain Questionnaire
 Objective 217 2.4 (1.0) [1, 5]
 Subjective Externalizing 217 2.3 (0.8) [1, 5]
 Subjective Internalizing 217 3.6 (1.0) [1, 5]
Demographic Characteristics
 Child age 218 9.0 (2.7) [4, 14]
 Child gender (male) 148 67.9%
 Child race/ethnicity
  Non-Hispanic Caucasian 103 47.2%
  Latino/Hispanic 65 29.8%
  African American 22 10.1%
  Other/Mixed 28 12.8%
Child Clinical Factors at Entry
 Primary diagnosis
  Disruptive Behavior Disorder 47 21.8%
  Attention Deficit Hyperactivity Disorder 82 38.0%
  Mood Disorder 49 22.7%
  Anxiety Disorder 19 8.8%
  Autism Spectrum Disorder/Other 19 8.8%
 Diagnostic comorbidity (>1 diagnosis) 93 43.1%
 Eyberg Child Behavior Inventory 218 147.3 (36.9) [59, 237]
Parent/Family Clinical Factors
 Brief Symptom Inventory 215 56.8 (11.4) [33, 80]
 Family Relationship Inventory 217 9.1 (4.7) [−5, 17]
 Life Events Questionnaire 217 3.0 (1.7) [0, 8]
Treatment Characteristics
 Psychotherapy Services
  Number of visits 215 20.6 (15.9) [0, 70]
  Frequency of visits 215 0.6 (0.2) [0.0, 1.2]
 Medication Use
  Antipsychotics 201 11.0%
  Antidepressants 201 8.3%
  ADHD medication 201 30.3%
  Mood stabilizers/antiepileptics 201 7.3%
  Other 201 4.6%
 Primary referral source
  Parent 113 53.8%
  School staff 48 22.9%
  Other 49 23.3%
Open in a new tab

Procedures

Data used in the current study were collected from (1) baseline in-person interviews with children (age 9 and over) and parents to assess demographic, clinical and family data; (2) telephone follow-up interviews with family participants four and eight months after baseline to assess for caregiver strain, child behavior problems, life events, satisfaction, and medication use (only four month data are used for satisfaction and medication use); (3) abstraction from administrative billing records reliably indicating the total number of treatment visits and funding source (i.e., school-based funding versus Medi-Cal/Medicaid) during the 16-month study period; and (4) clinician-reported primary child psychiatric diagnosis and comorbidity. Child use of psychoactive medication(s) was classified into five medication classes (stimulants plus atomoxetine, antipsychotics, antidepressants, mood stabilizers/antiepileptics, and other), consistent with previous studies (Leslie et al. 2007; Raghavan et al., 2005). Parent report of service use, and specifically medication use, has demonstrated to be both valid and reliable (Bussing et al., 2003). Effect sizes for change in caregiver strain were calculated using Cohen’s (1988) d statistic based on the average standard deviation from both means and correcting for dependence between means (Morris & DeShon, 2002). See Table 1 for sample descriptives on all study measures.

Measures

Eyberg Child Behavior Inventory (ECBI; Eyberg & Ross, 1978)

Parent-reported disruptive behavior problems were assessed with the intensity scale of the ECBI. The ECBI is a well-established 36-item parent-report measure that specifically assesses frequency and severity of disruptive behavior problems in children ages 2 to 16. The measure has strong psychometric characteristics that include strong reliability, convergent validity, internal consistency, and discriminative power (Boggs, Eyberg, & Reynolds, 1990; Eyberg & Ross, 1978; Rich & Eyberg, 2001; Robinson, Eyberg, & Ross, 1980).

Caregiver strain outcomes

Caregiver strain was measured by the Caregiver Strain Questionnaire (CGSQ; Brannan & Heflinger, 1997). The CGSQ is a 21-item self-report measure assessing three subscales of strain that were used as the outcomes in the current study: 1) objective strain, observable negative consequences of caring for the child (e.g., financial burden, disruption of family activities), 2) subjective externalized strain, negative feelings directed outward (e.g., anger, frustration) and 3) subjective internalized strain, negative feelings directed inward (e.g., sadness, guilt). CGSQ subscales demonstrate adequate internal consistency and adequate reliability and validity (Brannan & Heflinger, 1997; Taylor-Richardson, Heflinger, & Brown, 2006). Higher scores indicate greater strain.

Brief Symptom Inventory (BSI; Derogatis & Spencer, 1982)

The BSI is a 53-item measure that is a widely accepted screening tool of general psychopathology with excellent reliability and good convergent validity (Derogatis & Spencer, 1982). The global severity index of the BSI was used to assess parent psychopathology.

Family Relationship Index (FRI; Holahan & Moos, 1983)

The FRI is a 27-item true-false measure that assesses the quality of family relationships, which was completed by parents to assess family functioning. This measure has demonstrated good internal consistency and construct validity (Garland, Haine, & Lewczyk Boxmeyer, 2007; Hoge, Andrews, Faulkner, & Robinson, 1989; Holahan & Moos, 1983).

Family Empowerment Scale (FES; Koren, DeChillo, & Friesen, 1992)

The FES is designed to assess perceived family empowerment in the context of child mental health services. For this study, only the Family subscale (12 items) was utilized to assess empowerment within the family context. These items assess the parent’s perceived ability to handle child/family problems and to access information to better help his/her child with mental health problems. The FES has demonstrated good internal consistency and test-retest reliability (Koren et al., 1992; Singh et al., 1995), as well as evidence of convergent (Resendez, Quist, & Matshazi, 2000) and discriminant validity (Koren et al., 1992).

Life Events Questionnaire (LEQ)

The LEQ was developed for this study and informed by existing scales that measure life events that can be stressful for families. The LEQ assesses whether the participating parent experienced significant changes or stresses (yes/no) related to nine general areas (e.g., finances, employment, medical, family death, legal issues). The total score reflects the count of the areas in which a stressor has occurred. Scores at four and eight month assessments were added together to indicate the total count of stressful events that occurred in the first eight months of treatment.

Treatment expectations and client satisfaction (perceived effectiveness)

Parent-reported treatment expectations were assessed at baseline, and child and parent perceptions of the effectiveness of services (i.e., satisfaction) were assessed at four months.

Parent Expectations about Counseling (PEC)

This six-item measure was created for the current study and was administered at baseline to assess parents’ expectations about the usefulness of treatment (e.g., “I expect counseling to help my child”), how much they will like treatment (e.g., “I expect I will like counseling”), and expected involvement in treatment [e.g., “I expect to be very involved in my child’s counseling (e.g., attending many of the sessions)”]. Items were rated on a scale from one to four, and higher scores indicated more favorable expectations about treatment. Cronbach’s alpha for the PEC is .84 (Garland, Haine-Schlagel, Accurso, Baker-Ericzén, & Brookman-Frazee, 2012).

Multidimensional Adolescent Satisfaction Scale, Perceived Effectiveness subscale (MASS; Garland, Aarons, Saltzman, & Kruse, 2000; Garland, Saltzman, & Aarons, 2000)

The MASS is a 21-item self-report instrument that assesses consumer satisfaction with psychotherapy, which was administered to children ages 9 and older and parents at four months post service entry. For the current study, the Perceived Effectiveness subscale was used, which measures perceived effectiveness of services. This instrument has good internal consistency, strong test-retest reliability, and convergent, divergent, and predictive validity with similar treatment samples (Garland, Aarons, et al., 2000; Garland, Saltzman, et al., 2000).

Analysis Plan

Aim 1: Identify predictors of change in caregiver strain

To address the first goal of the study—to assess for significant predictors of change in caregiver strain—slopes-as-outcome models were used to examine significant predictors of slope for the three components of caregiver strain. Each type of caregiver strain (i.e., objective, subjective externalized, and subjective internalized strain) was examined separately because they function independently of one another and are differentially associated with other variables. The parameter estimate of interest in these models was the predictor by timeframe interaction, which assesses whether a predictor is significantly associated with the slope, or rate of change, in caregiver strain. Prior to running analyses, intraclass correlations (ICCs) were calculated for each of the three components of caregiver strain to assess the percent of variability in each outcome that is attributable to the therapist level. The observed ICCs indicated that it was necessary to account for the therapist level in subsequent analyses. SuperMix Version 1.1 (Hedeker, Gibbons, du Toit, & Patterson, 2008) was used for all primary analyses to account for the nested data structure (children/parents within therapist).

The primary analyses for this aim were conducted in three steps. The first step was to run bivariate associations between predictor variables and each outcome variable. For each predictor, both a random intercept and a random effects model were run to determine whether random effects significantly impacted the fixed effects results. The second step was to place variables associated with each outcome slope at p < .10 into separate multivariable regression models (one for each caregiver strain outcome) to assess for robust predictors. This p-value was chosen in order to screen for variables potentially associated with caregiver strain. The third step, as suggested by Raudenbusch and Bryk (2002) and Snijders and Bosker (1999), was to remove all nonsignificant predictors (p ≥ .05) in the multivariable model for statistical efficiency within the multilevel modeling context and run a final parsimonious multivariable model. This multi-step process is a useful methodology for selecting variables for multivariable analyses when sample size is limited and there is not enough power to enter all variables of interest into a model, and has been used in previous studies of community care (Garland et al., 2007; Leslie et al., 2000; McCabe, 2002). For the significant predictors that remained in the model, values one standard deviation above and below the mean for each numerical predictor were entered one-by-one (i.e., keeping all other variables constant) into the multivariable model to determine the direction of each effect. Given the results of Aim 1, the second aim sought to further examine the relation between child symptoms and caregiver strain.

Aim 2: Examine the direct relationship between change in child symptom severity and change in caregiver strain over time to determine the nature and direction of the association

Given the evidence suggesting that reduction in caregiver strain may be associated with better behavioral child outcomes (Kazdin & Whitley, 2003), the second goal of the study is to examine the direct relationship between child symptom severity and caregiver strain over time. Autoregressive cross-lagged models within the multilevel modeling software package EQS 6.1 (Bentler, 2006) were used to test the hypothesis that change in caregiver strain across time would predict change in child symptom severity. These models allow for analyzing change over time within a structural equation modeling (SEM) framework. SEM allows for examination of relationships among multiple dependent variables simultaneously while accounting for the hierarchical structure of these data, since there are repeated measures (level-1) nested within individuals (level-2). Furthermore, such models readily incorporate “missingness” in analyses with an assumption that data are missing at random. Both levels of this nested data structure are examined to determine whether either level violates the independence assumption, and analyses incorporated both levels. Models examined child symptom severity and caregiver strain and outcomes across baseline, 4 months, and 8 months.

Results

Across all caregivers, average total caregiver strain decreased approximately one half point across 8 months, representing a 12.5% change on the 5-point scale. Moderate decreases (d = .54) were observed in objective caregiver strain from baseline (M=2.39, SD=0.99) to eight months (M=1.92, SD=0.81), as well as subjective internalizing caregiver strain (d = .74) from baseline (M=3.56, SD=0.98) to eight months (M=2.81, SD=1.07). Change in subjective externalizing caregiver strain from baseline (M=2.26, SD=0.83) to eight months (M=1.96, SD=0.81) was small (d = .37).

Intraclass correlations (ICCs) were calculated for all three outcomes to assess the percent of variability in each outcome that is attributable to differences in therapists and parents. For objective caregiver strain, a negligible amount of variance was accounted for at the therapist level (ICC = .021). However, parent level factors accounted for approximately 50% (ICC = .529) of the variance in objective strain, confirming the need to include this level in multivariable analyses. For subjective externalized caregiver strain, approximately 8% of the variance in this outcome was accounted for at the therapist level (ICC = .083) and 50% was accounted for at the parent level (ICC = .506). For subjective internalized strain, therapist level factors accounted for approximately 8% of the variance in this outcome (ICC = .083) while parent level factors accounted for approximately 35% of the variance (ICC = .358). Therefore, both levels were included in subsequent analyses for both externalized and internalized strain.

Aim 1: Identify predictors of change in caregiver strain

Bivariate associations between predictor variables and each outcome variable were first examined (see Table 2a). Predictors for each outcome that were marginal/significant were then entered into an initial multivariable model. The final multivariable model (see Table 2b) included all predictors that were marginal/significant in this initial multivariable model. The following variables were not significant bivariate predictors of any of the three subtypes of caregiver strain: child gender, parent level of education, diagnostic comorbidity, quality of family relationships, family empowerment, treatment expectations, treatment satisfaction, number of psychotherapy visits, psychotherapy service intensity, use of antipsychotics and/or antidepressants, and funding source (Medi-Cal v. school-based).

Table 2.

Predictors of change in caregiver strain.

Table 2a: Bivariate Model Predictors
Objective Externalizing Internalizing
B SE p-value B SE p-value B SE p-value
Child Demographic Characteristics
 Child age 0.01 0.01 .63 < 0.01 0.01 .81 0.02 0.01 .079
 Child gender 0.04 0.07 .52 −0.01 0.10 .88 −0.07 0.08 .42
 Child race/ethnicity
  Caucasian+ v. Latino/Hispanic −0.07 0.08 .35 −0.05 0.08 .51 −0.07 0.09 .42
  Caucasian v. African American −0.06 .11 .55 0.14 0.11 .21 0.06 0.13 .63
  Caucasian v. Other/Mixed −0.17 0.10 .083 −0.18 0.11 .093 −0.05 0.11 .64
Child Clinical Factors
 Primary diagnosis
  DBD v. ADHD 0.19 0.08 .021 0.09 0.08 .25 0.05 0.10 .62
  DBD v. Mood 0.26 0.09 .006 0.02 0.09 .83 0.05 0.11 .64
  DBD v. Anxiety 0.04 0.12 .76 −0.06 0.12 .61 −0.21 0.15 .15
  DBD v. ASD/Other 0.29 0.12 .017 −0.02 0.12 .89 0.07 0.15 .62
 Diagnostic comorbidity 0.02 0.06 .75 −0.05 0.06 .46 0.01 0.08 .90
 Eyberg Child Behavior Inventory < −0.01 < 0.01 < 0.001 < −0.01 < 0.01 .005 < −0.01 < 0.01 .035
Caregiver/Family Clinical Factors
 Brief Symptom Inventory −0.01 < 0.01 .017 < −0.01 < 0.01 .18 < −0.01 < 0.01 .71
 Family Relationship Inventory 0.01 0.01 .20 0.01 0.01 .33 < −0.01 0.01 .84
 Life Events Questionnaire* −0.08 0.06 .15 −0.12 0.05 .018 0.02 0.07 .72
Treatment Characteristics
 Psychotherapy Services
  Number of visits < −0.01 < 0.01 .97 < −0.01 < 0.01 .40 < −0.01 < 0.01 .45
  Frequency of visits −0.19 0.16 .24 0.05 0.14 .71 −0.07 0.17 .70
 Medication Use
  Antipsychotics −0.06 0.19 .75 −0.08 0.19 .66 −0.08 0.24 .74
  Antidepressants −0.14 0.24 .55 < 0.01 0.21 .98 −0.24 0.28 .39
  ADHD medication −0.25 0.13 .055 0.05 0.13 .72 −0.05 0.17 .76
  Mood stabilizers/antiepileptics −0.48 0.23 .032 −0.28 0.22 .20 −0.20 0.29 .49
 Primary referral source
  Self v. School staff 0.07 0.09 .40 0.08 0.08 .31 0.18 0.09 .054
  Self v. Other 0.10 0.08 .23 −0.03 0.08 .65 0.16 0.09 .095
Table 2b: Multivariable Models
Objective Subjective Externalized Subjective Internalized
B SE p-value B SE p-value B SE p-value
Child symptom severity −0.51 0.29 < .00001 < −0.01 < −0.01 .005 < −0.01 < 0.01 .099
Mood stabilizer use −0.80 0.22 .0002 -- -- -- -- -- --
Self v. school referral source -- -- -- -- -- -- 0.17 0.07 .076
 Final overall model −0.33 0.07 < .00001 −0.15 0.03 < .00001 −0.45 0.05 < .00001
Open in a new tab

Note: Results of marginal and significant predictors are italicized. All categorical variables with three groups or more were dummy coded, with the following reference groups: + non-Hispanic Caucasian (race/ethnicity), some high school or less (parent education), Disruptive Behavior Disorder (primary diagnosis), and self-referral (referral source). For the two binary variables, male gender and Medi-Cal funding were coded as 0. An asterisk (*) indicates a square-root transformation.

Objective Caregiver Strain

In the final multivariable model predicting objective caregiver strain, child symptom severity and mood stabilizer/antiepileptic use were the two significant predictors. This final overall model predicted a decrease of .33 points in objective caregiver strain per time point (SE = 0.07, p < .00001) for parents with children who had average symptom severity at entry and were taking a mood stabilizer/antiepileptic by four months into the study period. Child symptom severity was significantly associated with steeper slope, such that higher child symptom severity at entry into services was associated with a steeper decrease in objective caregiver strain over time (B = −0.51, SE = 0.29, p < .00001). In addition, caregivers of children who were using mood stabilizers/antiepileptic reported a faster rate of decline in objective caregiver strain (B = −0.80, SE = 0.22, p = .0002). Race, diagnosis, parent psychopathology, and child stimulant were no longer significant in the initial multivariable model and therefore excluded from the final model.

Subjective Externalized Caregiver Strain

Only child symptom severity significantly predicted subjective externalized strain in the final multivariable model, with an overall decrease of .15 points per time point (SE = 0.03, p < .00001) for caregivers of children with average symptom severity at baseline. Higher initial child symptom severity predicted greater albeit minimal decreases in externalized strain than low symptom severity (B < −0.01, SE < 0.01, p = .005). Race and ongoing negative life events did not remain significant predictors in the initial multivariable model and were therefore excluded.

Subjective Internalized Caregiver Strain

When entered into the initial multivariable model, child symptom severity and school referral were marginally significant. When entered into the final multivariable model, the overall model was significant, predicting a .45 point decrease in subjective internalized caregiver strain per time point (SE = .05, p < .00001) for caregivers of children with average symptom severity using stimulant medication who were self-referred. Higher initial child symptom severity predicted greater but small decreases in strain than low symptom severity (B < −.01, SE < .01, p = .017). School referral for services marginally predicted less steep decreases in strain (B = .17, SE = .07, p = .076). Child age and stimulant use were no longer significant predictors of subjective internalized strain in the initial multivariable model and therefore excluded.

Aim 2: Examine the direct relationship between change in child symptom severity and change in caregiver strain over time to determine the nature and direction of the association

Objective Caregiver Strain

Simultaneous correlations between caregiver strain and child symptom severity were moderate at each time point and ranged from .45 to .53 (p’s < .001). In the autoregressive model (see Figure 2), greater objective caregiver strain at four months (accounting for both baseline caregiver strain and symptom severity) predicted higher child symptom severity at eight months (accounting for earlier measurements of symptom severity; β = .18, SE < .01, p < .05). No other paths between child symptom severity and objective caregiver strain were statistically significant.

Figure 2.

Figure 2

Open in a new tab

Objective caregiver strain autoregressive model.

Note: The Caregiver Strain Questionnaire (CGSQ) was used to assess objective caregiver strain; the Eyberg Child Behavior Inventory (ECBI) was used to assess child disruptive behavior problems. For each examined path, the first number represents the standardized regression coefficient (i.e., β) while the second number represents the standard error around the coefficient.

p < .05*

p < .001**

Subjective Externalized Caregiver Strain

Simultaneous correlations between caregiver strain and child symptom severity were moderate at each time point and ranged from .31 to .39 (p’s < .001). In the autoregressive model (see Figure 3), greater baseline externalized caregiver strain predicted higher child symptom severity at four months (controlling for baseline symptom severity; β = .12, SE = 2.54, p < .05). In turn, higher child symptom severity at four months predicted greater externalized caregiver strain at eight months (accounting for earlier measurements of caregiver strain; β = .13, SE < .01, p < .05). No other paths between child symptom severity and externalized caregiver strain were statistically significant.

Figure 3.

Figure 3

Open in a new tab

Subjective externalized caregiver strain autoregressive model.

Note: The Caregiver Strain Questionnaire (CGSQ) was used to assess subjective externalized caregiver strain; the Eyberg Child Behavior Inventory (ECBI) was used to assess child disruptive behavior problems. For each examined path, the first number represents the standardized regression coefficient (i.e., β) while the second number represents the standard error around the coefficient.

p < .05*

p < .001**

Subjective Internalized Caregiver Strain

Simultaneous correlations between caregiver strain and child symptom severity were moderate at each time point and ranged from .41 to .51 (p’s < .001). In the autoregressive model (see Figure 4), similar to externalized strain, higher child symptom severity at four months predicted greater internalized caregiver strain at eight months (β = .27, SE < .01, p < .001). No other paths between child symptom severity and internalized caregiver strain were statistically significant.

Figure 4.

Figure 4

Open in a new tab

Subjective internalized caregiver strain autoregressive model.

Note: The Caregiver Strain Questionnaire (CGSQ) was used to assess subjective internalized caregiver strain; the Eyberg Child Behavior Inventory (ECBI) was used to assess child disruptive behavior problems. For each examined path, the first number represents the standardized regression coefficient (i.e., β) while the second number represents the standard error around the coefficient.

p < .001**

Discussion

Overall, there were small to medium decreases in caregiver strain over the eight-month study period. Caregivers reported less objective and subjective strain eight months after initiating mental health services, with the greatest change occurring in the first four months. However, subjective internalized caregiver strain remained in a moderate range eight months after initiating treatment, even though much larger reductions in caregiver strain have been found in other usual care settings with time-limited therapy (i.e., 10 sessions) (Moretti & Obsuth, 2009). Despite examining a range of demographic and clinical factors, few robust associations were found. Therefore, it is possible that decreases in strain may largely be the result of initiating mental health services rather than actual receipt of services. Indeed, the number of psychotherapy visits and service intensity were not associated with reductions in caregiver strain. Nevertheless, the results suggest that targeting child disruptive behavior problems may impact reductions in both externalized and internalized subjective strain, while targeting objective caregiver strain may reduce severity of disruptive behavior problems.

Across all three types of strain, initial disruptive behavior severity was the most robust predictor of reduced strain. It is possible that this effect reflects greater “relief” on the part of caregivers with more symptomatic children upon entering treatment. However, this effect was only clinically significant for objective strain, as reductions in subjective strain associated with symptom severity were minimal, suggesting that it may require greater therapist efforts (or greater child symptom reduction) to change caregivers’ subjective experience of strain compared to change in actual objective events associated with strain. While more severe initial disruptive behaviors predicted a steeper decline in caregiver strain, greater disruptive behavior severity sustained at four months (accounting for initial symptoms) predicted less improvement in subjective externalized and internalized strain at eight months. In other words, caregiver strain is greater at eight months for caregivers of children whose symptom severity remains high at four months.

Autoregressive models also revealed the reverse—greater caregiver strain impacted child disruptive behavior severity. After accounting for initial objective caregiver strain, greater objective caregiver strain at four months predicted less improvement in child behavior at eight months. Similarly, greater initial subjective externalized caregiver strain predicted less improvement in child behavior at four months. These results reinforce a possible causal relation between disruptive behavior problems and caregiver strain. Furthermore, they suggest there is a true bidirectional relationship between caregiver strain and disruptive behavior problems, such that reducing behavior problems may enhance positive changes in subjective strain, at the same time as reducing objective strain may lead to reductions in behavior problems.

These findings suggest the equal importance of addressing child symptom improvement and caregiver strain reduction in treatment as changes in one impacts the other. Indeed, caregivers who receive education and support services report reduced caregiver strain improved emotional functioning in their children (Kutash, Garraza, Ferron, Duchnowski, Walrath, & Green, 2013). Further, among parents who report initial high caregiver strain, peer support was associated with decreased strain and increased treatment efficacy (Kutash, Duchnowski, Green, & Ferron, 2011). There is also evidence to suggest that treatment for children with behavioral problems is more effective when caregiver strain is reduced (Kazdin & Whitley, 2003). Therapists might intervene directly to reduce caregiver strain through a variety of techniques, including teaching skills, reframing child problems, increasing caregiver social support, and providing families with additional resources, support, and empathy. Some of these techniques are included as part of parent training, which has been associated with reductions in caregiver strain (Anastopoulos et al., 1993; Feldman & Werner, 2002; Moretti & Obsuth, 2009), but research suggests that adjunct services provided specifically to caregivers may be needed for improved outcomes, particularly among caregivers with high initial strain (Kutash et al., 2013; Kutash et al., 2011). Indirectly, caregiver strain might also be reduced by improving child behavior, improving child coping skills, and improving child-caregiver interactions.

Use of certain psychotropic medications was also associated with steeper decreases in caregiver strain (objective), even after accounting for initial severity of disruptive behavior problems. Caregivers of children taking a mood stabilizer/antiepileptic reported greater decreases in objective strain than caregivers whose children were not using these medications, regardless of symptom severity at entry into services. Again, it may be that this reduction reflects an alleviation of strain upon receiving services (in this case medication management) for their child. Alternatively, decreases in child problems as a direct result of mood stabilizer/antiepileptic use may be responsible for decreases in caregiver strain. Finally, school referral (as compared to self-referral) was associated with a less steep decline in subjective internalized strain. In other words, caregivers referred to child services by school staff did not experience as great of a decrease in strain as those who were self-referred, suggesting that caregiver “relief” is greater for those who self-initiate entry to treatment. It is possible that treatment of these children may have focused more on behaviors in the school context compared to the home context, or caregivers of these children may have been less active in treatment than when they self-referred.

The role of caregiver strain in influencing mental health service utilization is well established. However, it is less commonly studied as an indicator of treatment outcome, despite its potential impact on future child symptom severity and reentry into mental health services. This study was comprehensive in its examination of predictors of caregiver strain, with longitudinal data on children, families, and the services provided to them. The sample of children with disruptive behavior problems who presented for usual care services provides a unique context in which to examine caregiver strain. The multifaceted data allowed for exploration of a variety of predictors consistent with the literature, and the analytic approach was appropriate in dealing with multilevel data and participant attrition. However, these findings may not generalize to other clinical populations (i.e., presenting problems other than disruptive behavior) and/or other geographic areas. In addition, many but not all factors related to caregiver strain were available; for example, social support may buffer against strain (McCabe, Yeh, Lau, Garland, & Hough, 2003) but was not measured in this study. Furthermore, only caregiver report was obtained for disruptive behavior problem severity and caregiver strain, such that caregivers who perceived greater themselves to have greater strain may have also reported more severe child behavior problems. Finally, it is not possible to know the extent to which caregiver strain would have decreased naturally as a result of time.

Despite some limitations, this study adds to current evidence suggesting that caregiver strain plays an important role in predicting child symptomatology; as such, it is an important outcome of child treatment. Therapists providing mental health services to families are challenged with the task of attending to multiple treatment goals. Striking a balance between intervention at the child functioning (e.g., symptom severity) and caregiver functioning level (e.g., caregiver distress and strain as a result of the child’s problems) is important. Indeed, high caregiver strain is associated with less effective service utilization (Brannan et al., 2003; Pellerin et al., 2010; Yatchmenoff et al., 1998), which may directly impact child outcomes. Furthermore, child symptomatology and caregiver strain reciprocally impact each other, highlighting the importance of addressing both as treatment targets.

Due to the naturalistic nature of this study, it is not possible to know how strain would have been affected for caregivers whose children were not receiving care or were receiving different types of treatment. However, the lack of association of reduction in caregiver strain with number of service visits and service intensity is informative and suggests that that usual care services may not be optimally effective in reducing caregiver strain. This highlights a potential need for specific parent-focused intervention strategies to mediate the interaction between caregiver strain and child symptoms in order to better serve the mental health needs of the entire family. Future research might also better disentangle the most indicated sequence of treatment—whether to focus initially on child symptoms, caregiver strain, or both simultaneously. Ultimately, this understanding may help to improve effective utilization of mental health services and lead to better child and family outcomes.

Figure 1.

Figure 1

Open in a new tab

Change in caregiver strain across time.

Acknowledgments

This research was supported in part by National Institute of Mental Health Grants R01MH66070 (AFG), F31MH083399 (ECA), and K01MH69665 (MBE). We thank Scott Roesch for his assistance with data analysis. In addition, we would like to acknowledge all participating therapists and families.

References

  1. Anastopoulos AD, Shelton T, DuPaul GJ, Guevremont DC. Parent training for Attention Deficit Hyperactivity Disorder: Its impact on parent functioning. Journal of Abnormal Child Psychology. 1993;21:581–596. doi: 10.1007/BF00916320. [DOI] [PubMed] [Google Scholar]
  2. Angold A, Messer SC, Stangl D, Farmer EMZ, Costello EJ, Burns BJ. Perceived parental burden and service use for child and adolescent psychiatric disorders. American Journal of Public Health. 1998;88:75–80. doi: 10.2105/ajph.88.1.75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bentler PM. EQS 6 structural equations program manual. Encino, CA: Multivariate Software, Inc; 2006. [Google Scholar]
  4. Blader JC. Pharmacotherapy and postdischarge outcomes of child inpatients admitted for aggressive behavior. Journal of Clinical Psychopharmacology. 2006;26:419–425. doi: 10.1097/01.jcp.0000227356.31203.8a. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Boggs SR, Eyberg S, Reynolds LA. Concurrent validity of the Eyberg Child Behavior Inventory. Journal of Clinical Child Psychology. 1990;19:75–78. [Google Scholar]
  6. Brannan AM, Heflinger CA. Distinguishing caregiver strain from psychological distress: modeling the relationships among child, family, and caregiver variables. Journal of Child and Family Studies. 2001;10:405–418. [Google Scholar]
  7. Brannan AM, Heflinger CA. Caregiver, child, family, and service system contributors to caregiver strain in two child mental health service systems. The Journal of Behavioral Health Services & Research. 2006;33:408–422. doi: 10.1007/s11414-006-9035-1. [DOI] [PubMed] [Google Scholar]
  8. Brannan AM, Heflinger CA, Foster EM. The role of caregiver strain and other family variables in determining children’s use of mental health services. Journal of Emotional and Behavioral Disorders. 2003;11:77–91. [Google Scholar]
  9. Brannan AM, Heflinger CA, Bickman L. The Caregiver Strain Questionnaire: Measuring the impact on the family of living with a child with serious emotional problems. Journal of Emotional and Behavioral Disorders. 1997;5:212–222. [Google Scholar]
  10. Bussing R, Mason DM, Leon CE, Sinha K. Agreement between CASA parent reports and provider records of children’s ADHD services. Journal of Behavioral Health Services & Research. 2003;30:462–469. doi: 10.1007/BF02287433. [DOI] [PubMed] [Google Scholar]
  11. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2. Lawrence Erlbaum Associates; 1988. [Google Scholar]
  12. Cook JA, Heflinger CA, Hoven CW, Kelleher KJ, Mulkern V, Paulson RI, Kim J. Multi-site Study of Medicaid-funded Managed Care Versus Fee-for-Service Plans’ Effects on Mental Health Service Utilization of Children With Severe Emotional Disturbance. Journal of Behavioral Health Services & Research. 2004;31:384–402. doi: 10.1007/BF02287691. [DOI] [PubMed] [Google Scholar]
  13. Derogatis LR, Spencer PM. Administration and procedures: BSI Manual I. Baltimore: Clinical Psychometric Research; 1982. [Google Scholar]
  14. Eyberg SM, Ross AW. Assessment of child behavior problems: the validation of a new inventory. Journal of Clinical Child Psychology. 1978;7:113–116. [Google Scholar]
  15. Feldman MA, Werner SE. Collateral effects of behavioral parent training on families of children with developmental disabilities and behavior disorders. Behavioral Interventions. 2002;17:75–83. [Google Scholar]
  16. Garland AF, Aarons GA, Hawley KM, Hough RL. Relationship of youth satisfaction with mental health services and changes in symptoms and functioning. Psychiatric Services. 2003;54:1544–1546. doi: 10.1176/appi.ps.54.11.1544. [DOI] [PubMed] [Google Scholar]
  17. Garland AF, Aarons GA, Saltzman MD, Kruse MI. Correlates of adolescents’ satisfaction with mental health services. Mental Health Services Research. 2000;2:127–139. doi: 10.1023/a:1010137725958. [DOI] [PubMed] [Google Scholar]
  18. Garland AF, Brookman-Frazee L, Hurlburt MS, Accurso EC, Zoffness R, Haine-Schlagel R, Ganger W. Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services. 2010;61:788–795. doi: 10.1176/appi.ps.61.8.788. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Garland AF, Haine RA, Lewczyk Boxmeyer C. Determinates of youth and parent satisfaction in usual care psychotherapy. Evaluation and program planning. 2007;30:45–54. doi: 10.1016/j.evalprogplan.2006.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Garland AF, Haine-Schlagel R, Accurso EC, Baker-Ericzén MJ, Brookman-Frazee L. Exploring the effect of therapists’ treatment practices on client attendance in community-based care for children. Psychological Services. 2012;9:74–88. doi: 10.1037/a0027098. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Garland AF, Saltzman MD, Aarons GA. Adolescent satisfaction with mental health services: development of a multidimensional scale. Evaluation and Program Planning. 2000;23:165–175. [Google Scholar]
  22. Hedeker D, Gibbons RD, Du Toit SHC, Patterson D. SuperMix: A Program for Mixed-Effects Regression Models. Chicago: Scientific Software International; 2008. [Google Scholar]
  23. Hoagwood KE, Burns B. Vectoring for true north: Building a research base on family support. Journal Administration and Policy in Mental Health and Mental Health Services Research. 2014;41:1–6. doi: 10.1007/s10488-013-0516-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hoge RD, Andrews DA, Faulkner P, Robinson D. The Family Relationship Index: validity data. Journal of Clinical Psychology. 1989;45:897–903. doi: 10.1002/1097-4679(198911)45:6<897::aid-jclp2270450611>3.0.co;2-t. [DOI] [PubMed] [Google Scholar]
  25. Holahan CJ, Moos RH. The quality of social support: Measures of family and work relationships. British Journal of Clinical Psychology. 1983;22:157–162. [Google Scholar]
  26. Kang E, Brannan AM, Heflinger CA. Racial differences in responses to the Caregiver Strain Questionnaire. Journal of Child and Family Studies. 2005;14:43–56. [Google Scholar]
  27. Kazdin AE, Whitley MK. Treatment of parental stress to enhance therapeutic change among children referred for aggressive and antisocial behavior. Journal of Consulting and Clinical Psychology. 2003;71:504–515. doi: 10.1037/0022-006x.71.3.504. [DOI] [PubMed] [Google Scholar]
  28. Koren PE, DeChillo N, Friesen BJ. Measuring empowerment in families whose children have emotional disabilities: A brief questionnaire. Rehabilitation Psychology. 1992;37:305–321. [Google Scholar]
  29. Kutash K, Duchnowski AJ, Green AL, Ferron J. Supporting parents who have youth with emotional disturbances through a parent-to-parent support program: A proof of concept study using random assignment. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38:412–427. doi: 10.1007/s10488-010-0329-5. [DOI] [PubMed] [Google Scholar]
  30. Kutash K, Garraza LG, Ferron JM, Duchnowski AJ, Walrath C, Green AL. The relationship between family education and support services and parent and child outcomes over time. Journal of Emotional and Behavioral Disorders. 2013;21(4):264–276. [Google Scholar]
  31. Leslie LK, Aarons GA, Haine RA, Hough RL. Caregiver depression and medication use by youths with ADHD who receive services in the public sector. Psychiatric Services. 2007;58:131–134. doi: 10.1176/ps.2007.58.1.131. [DOI] [PubMed] [Google Scholar]
  32. Leslie LK, Landsverk J, Ezzet-Lofstrom R, Tschann JM, Slymen D, Garland A. Children in foster care: Factors influencing outpatient mental health service use. Child Abuse and Neglect. 2000;24:465–476. doi: 10.1016/s0145-2134(00)00116-2. [DOI] [PubMed] [Google Scholar]
  33. McCabe KM, Lansing AE, Garland A, Hough R. Gender differences in psychopathology, functional impairment, and familial risk factors among adjudicated delinquents. Journal of the American Academy of Child & Adolescent Psychiatry. 2002;41(7):860–867. doi: 10.1097/00004583-200207000-00020. [DOI] [PubMed] [Google Scholar]
  34. McCabe KM, Yeh M, Lau A, Garland A, Hough R. Racial/ethnic differences in caregiver strain and perceived social support among parents of youth with emotional and behavioral problems. Mental Health Services Research. 2003;5:137–147. doi: 10.1023/a:1024439317884. [DOI] [PubMed] [Google Scholar]
  35. McDonald TP, Gregoire TK, Poertner J, Early TJ. Building a model of family caregiving for children with emotional disorders. Journal of Emotional and Behavioral Disorders. 1997;5:138–148. [Google Scholar]
  36. Moretti MM, Obsuth I. Effectiveness of an attachment-focused manualized intervention for parents of teens at risk for aggressive behaviour: the Connect Program. Journal of Adolescence. 2009;32:1347–1357. doi: 10.1016/j.adolescence.2009.07.013. [DOI] [PubMed] [Google Scholar]
  37. Morris SB, DeShon RP. Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods. 2002;7:105–125. doi: 10.1037/1082-989x.7.1.105. [DOI] [PubMed] [Google Scholar]
  38. Pellerin KA, Costa NM, Weems CF, Dalton RF. An examination of treatment completers and non-completers at a child and adolescent community mental health clinic. Community Mental Health Journal. 2010;46:273–281. doi: 10.1007/s10597-009-9285-5. [DOI] [PubMed] [Google Scholar]
  39. Raghavan R, Zima BT, Andersen RM, Leibowitz AA, Schuster MA, Landsverk J. Psychotropic medication use in a national probability sample of children in the child welfare system. Journal of Child and Adolescent Psychopharmacology. 2005;15(1):97–106. doi: 10.1089/cap.2005.15.97. [DOI] [PubMed] [Google Scholar]
  40. Raudenbush SW, Bryk AS. Hierarchical linear models: Applications and data analysis methods. 2. Newbury Park, CA: Sage; 2002. [Google Scholar]
  41. Resendez MG, Quist RM, Matshazi DG. A longitudinal analysis of family empowerment and client outcomes. Journal of Child and Family Studies. 2000;9:449–460. [Google Scholar]
  42. Rich BA, Eyberg SM. Accuracy of assessment: The discriminative and predictive power of the Eyberg Child Behavior Inventory. Ambulatory Child Health. 2001;7:249–257. [Google Scholar]
  43. Robinson EA, Eyberg SM, Ross AW. The standardization of an inventory of child conduct problem behaviors. Journal of Clinical Child Psychology. 1980;9:22–29. [Google Scholar]
  44. Sales E, Greeno C, Shear MK, Anderson C. Maternal caregiving strain as a mediator in the relationship between child and mother mental health problems. Social Work Research. 2004;28(4):211–223. [Google Scholar]
  45. Shin SH, Brown TA. Racial and ethnic disparities in caregiver strain and the use of child mental health services: a structural equation model. Psychiatric Services. 2009;60:1039–1045. doi: 10.1176/ps.2009.60.8.1039. [DOI] [PubMed] [Google Scholar]
  46. Singh NN, Curtis WJ, Ellis CR, Nicholson MW, Villani TM, Wechsler HA. Psychometric analysis of the family empowerment scale. Journal of Emotional and Behavioral Disorders. 1995;3:85–91. [Google Scholar]
  47. Snijders T, Bosker R. Multilevel analysis: an introduction to basic and advanced multilevel modeling. London, UK: Sage; 1999. [Google Scholar]
  48. Soni A. Statistical Brief # 242. Agency for Healthcare Research and Quality; Rockville, MD: Apr, 2009. The Five Most Costly Children’s Conditions, 2006: Estimates for the U.S. Civilian Noninstitutionalized Children, Ages 0–17. http://www.meps.ahrq.gov/mepsweb/data_files/publications/st242/stat242.pdf. [PubMed] [Google Scholar]
  49. Taylor-Richardson KD, Heflinger CA, Brown TN. Experience of strain among types of caregivers responsible for children with serious emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders. 2006;14:157–168. [Google Scholar]
  50. Tsai KH, Yeh M, Slymen D. Strain in Caring for Youths Meeting Diagnosis for Disruptive Behavior Disorders. Journal of Emotional and Behavioral Disorders. 2013 doi: 10.1177/1063426613503498. Advance online publication. [DOI] [Google Scholar]
  51. Yatchmenoff DK, Koren PE, Friesen BJ, Gordon LJ, Kinney RF. Enrichment and stress in families caring for a child with a serious emotional disorder. Journal of Child and Family Studies. 1998;7:129–145. [Google Scholar]

RESOURCES