Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 May 2.
Published in final edited form as: J Clin Child Adolesc Psychol. 2018 May 2;47(SUP1):S530–S541. doi: 10.1080/15374416.2018.1458313

The Family Life Impairment Scale: Factor Structure and Clinical Utility with Young Children

Nicholas D Mian 1, Timothy W Soto 2, Margaret J Briggs-Gowan 3, Alice S Carter 4
PMCID: PMC6214795  NIHMSID: NIHMS968954  PMID: 29718718

Abstract

Although it is well-established that young children experience significant psychopathology, diagnostic decisions continue to be challenging, in part due to the way impairment is understood, defined, and measured. Most existing clinical tools assess impairment in an individualized manner, whereas for many young children, impairment is more accurately conceptualized as a family-oriented, multidimensional construct, impacting various parental and family activities. Two studies were completed using the Family Life Impairment Scale (FLIS), a multidimensional parent-report measure of family and associated impairment designed for young children. In Study 1, factor analysis was used in a large (n = 945) representative sample (23–48 months of age). FLIS associations with measures of parent and child well-being were explored to investigate convergent validity. Study 2 was completed in a sample (n = 174) of young children (18–33 months of age) diagnosed with autism spectrum disorders to explore factorial consistency in a clinical sample. Study 1 yielded evidence of a four-factor solution, including parent impairment (affecting parental well-being), family impairment (affecting family activities and routines), childcare impairment (affecting challenges with childcare), and positive growth (parental learning and growth associated with the child’s problem). Evidence of convergent validity was also found, as factors were differentially associated with established measures of child symptoms and parent stress. Factor structure was supported in the clinical sample. Results support both the factorial structure and clinical utility of the FLIS for use across clinical and nonclinical populations of young children.

Substantial evidence documents that young children (5 years of age and younger) demonstrate significant behavioral, developmental, and mental health challenges that warrant clinical attention (Carter, Briggs-Gowan, & Davis, 2004; DelCarmen-Wiggins & Carter, 2004; Egger & Angold, 2006; Luby, 2006; Zeanah, 2009). The overall rate of impairing emotional and behavioral disorders in preschool and kindergarten-age children is estimated at 12% to 20%, rates similar to those found for older children and adults (Carter et al., 2010; Egger & Angold, 2006). Despite emphasis on the importance of early detection (American Academy of Pediatrics, 2006), many early emerging mental health problems go undetected and, consequently, unaddressed by service systems (Lavigne, LeBailly, Hopkins, Gouze, & Binns, 2009; Mian, 2014; Pavuluri, Luk, & McGee, 1996). This problem is partly due to ongoing challenges with assessment and diagnostic systems that fail to accurately reflect the rapid developmental changes, limitations of language, and interdependence of the child with his or her caregivers that characterize early childhood (Egger & Emde, 2011; Wakschlag et al., 2007). More specifically, the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association, 2013) does not accurately reflect the types of impairment found in young children, which are typically manifested in disruptions in family routines or family functioning, often due to accommodations that families make in response to their child’s emotional and behavioral problems. Despite evidence that family impairment is an important construct for both diagnostic purposes and developmental psychopathology research, there is a lack of adequate measurement tools for family impairment in this age group. In this investigation, we investigate the psychometric properties of the Family Life Impairment Scale (FLIS), a novel tool for assessing how much a child’s problem impairs family functioning.

Effects of Early Childhood Psychopathology on Parents and Family Functioning

Psychiatric conditions in young children have been shown to affect various domains of functioning, including financial (Lavelle et al., 2014), social, practical challenges (Redmond & Richardson, 2003), parental well-being (Epstein, Saltzman-Benaiah, O’Hare, Goll, & Tuck, 2008), and overall quality of family life (McStay, Trembath, & Dissanayake, 2014). These effects can resonate for decades for children with intellectual disabilities (Jokinen & Brown, 2005). Early childhood mental health problems, including developmental problems and irritability, also affect the broader family system (Llewellyn, McConnell, Thompson, & Whybrow, 2005; Wakschlag et al., 2015). Although the stress associated with receiving a diagnosis of autism spectrum disorder (ASD; Osborne, McHugh, Saunders, & Reed, 2008) and raising a child on the autism spectrum is well documented (Davis & Carter, 2008; Gray, 2002; Rao & Beidel, 2009), behavior problems (Eisenhower, Baker, & Blacher, 2009; Petitclerc et al., 2015) and anxiety (Langley, Bergman, McCracken, & Piacentini, 2004) are also associated with family impairment. In fact, in a sample of parents with children diagnosed with ASD, child externalizing behavior was the strongest predictor of parent stress and negative effects on overall quality of life (McStay et al., 2014).

Impairment as a Diagnostic Criterion in Young Children

Distinguishing clinically significant problems from normative manifestations can be especially challenging with young children because of rapid developmental changes, language limitations, interdependence between child and caregivers, and cognitive limitations (e.g., metacognition; Egger & Emde, 2011). Another long-standing challenge is differentiating between early childhood psychopathology and temperamental variations (Rapee & Coplan, 2010). Moreover, many behaviors that are clinical symptoms in later childhood, such as temper tantrums, are normative in early childhood. Thus, finding the threshold between normative and non-normative symptom presentation can be challenging, particularly because both quantity and quality of behavioral manifestations must be taken into consideration (Wakschlag et al., 2012).

Studying impairment represents one important way to distinguish between clinical and nonclinical variations. Theoretically, a measure of family impairment should be more strongly associated with measures of symptom severity (i.e., a dimensional proxy for diagnosis) than to measures of temperamental variations. In response to the limitations of DSM criteria, the Research Diagnostic Criteria: Infancy and Preschool workgroup (Task Force on Research Diagnostic Criteria: Infancy and Preschool, 2003) was developed to facilitate further research on the reliability, validity, and clinical utility of developmentally sensitive criteria for DSM-IV (American Psychiatric Association, 2000) disorders. Egger and Emde (2011) characterized this as a “top-down” approach, as it has been effective in adjusting existing criteria to age-appropriate changes. The Research Diagnostic Criteria workgroup (Task Force on Research Diagnostic Criteria: Infancy and Preschool, 2003) emphasized the importance of maintaining a criterion for “disability/impairment” for each disorder; with young children, this is necessary because the symptoms themselves (i.e., temper tantrums, fears, emotional lability) overlap considerably with typical development in this age range (Wakschlag et al., 2012). A second approach, which represents more of a “bottom-up” approach, is the development of the Diagnostic Classification: 0–5 (DC: 0–5; Zero to Three, 2016). DC: 0–5 represents a diagnostic system specifically generated for young children rather than a downward extension of criteria for older children or adults. Notably, consistent with our framework, the DC: 0–5 recognizes that impairment in young children must be understood within the context of both the individual child and/or the family.

In this review, we differentiate between impairment at the individual level and at the family level. Family impairment is defined here as the degree to which a child’s problem interferes with or negatively impacts family routines, the relationships of family members, and the well-being of all members of the family system. In the DC: 0–5 manual (Zero to Three, 2016), the impairment criterion for assigning a diagnosis can be met by the presence of family impairment. Assessing individual impairment in young children is complicated by parental accommodations in response to child symptoms. For example, if parents reduce a child’s exposure to challenging situations to minimize distress, there may be minimal impairment in the child’s daily activities (Task Force on Research Diagnostic Criteria: Infancy and Preschool, 2003). However, the level of impairment in the parent or family may be significant; parents may be unable to participate in adult activities or attend the types of social events the family would like to. In addition, the corollary effect of this accommodation may be a reduction in the child’s exposure to age-appropriate settings and experiences that are crucial for their growth and attainment of key developmental competencies. Therefore, the development of a measure that captures impairment at the family level may be necessary to systematically measure the type of impairment most appropriate in young children.

The Need for a New Measure for Family Impairment

A number of measures developed for older children focus on the child’s functioning at the individual level. The Brief Impairment Scale (BIS; Bird et al., 2005), a parent-report measure designed for school-age children and adolescents, assesses individual, global clinical impairment across three domains (interpersonal relations, school/work, and self-care/self-fulfillment). The BIS was designed to improve on previous measures of global impairment, such as the Columbia Impairment Scale (Bird, Shaffer, Fisher, & Gould, 1993), which has shown problems with respect to multidimensional construct validity (Singer, Eack, & Greeno, 2011) and use in samples with developmental disabilities (Zielinski, Wood, Renno, Whitham, & Sterling, 2014). However, a measure of family impairment is notably absent from the BIS. Other measures, such as the Parenting Stress Index (PSI; Abidin, 2009), are often used to assesses parent stress as it relates to childhood psychopathology. However, the focus of the PSI is on parent distress, and it does not measure broader dimensions of family impairment, such as limitations in participation in routine family activities. The PSI is also unable to assess the degree to which the parent’s distress specifically results from the child’s difficulties. The Impact on Family Scale (revised) is a 15-item parent-report measure that assesses the impact of chronic illness or disability on the family but is not specific to mental health or behavioral problems (Stein & Jessop, 2003; Williams, Piamjariyakul, Williams, Bruggeman, & Cabanela, 2006). Again, the Impact on Family Scale measures family impairment in a unidimensional manner.

The Preschool Age Psychiatric Assessment (PAPA) is a comprehensive parent-report structured diagnostic interview for younger children assessing impairment at the individual level (Egger & Angold, 2004). The PAPA assesses individual impairment across many life areas (daycare/school, peer relationships, sibling and parent relationships, etc.). In the PAPA, the presence of impairment (the PAPA uses the term “incapacity”) is established for each of these areas. Once impairment in an area is established, each symptom area is reviewed to determine the extent to which the impairment is attributed to that specific symptom domain. The incapacity score is typically used as part of the diagnostic algorithm and lacks the detail needed to evaluate the level of impairment as a construct in its own right. Another disadvantage of the PAPA is its time-intensive administration in interview format.

For young children, understanding both individual and family impairment is of utmost importance, especially in light of research demonstrating that parents are much more likely to recognize problems and seek treatment for their children when the problem interferes with their own lives (Costello, Pescosolido, Angold, & Burns, 1998). Parents of young children are more likely to be worried about their child’s behavior when family routines are disrupted, and this worry has been shown to predict service-seeking behavior (Ellingson, Briggs-Gowan, Carter, & Horwitz, 2004). Although family impairment as a construct is still not fully understood, it is clear that there are many different ways that a child’s development or symptoms might affect a family’s routines or well-being. Hence, it is likely that family impairment represents a multidimensional construct. In addition, some research has found the development of positive characteristics in families with a child with psychopathology and/or developmental disability, including parental competence (Dempsey, Keen, Pennell, O’Reilly, & Neilands, 2009), family enrichment (Yatchmenoff, Koren, Friesen, Gordon, & Kinney, 1998), and positive perceptions of parents’ expertise (Redmond & Richardson, 2003). Jokinen and Brown (2005) found that parents of adult children with intellectual disabilities attributed family cohesion and increased parental patience to raising a child with disabilities. We conceptualize this construct as positive growth, stemming from a family’s efforts in response to a young child’s mental health challenges.

The purpose of the present investigation is to examine the factor structure and psychometric properties of a new parent questionnaire —the FLIS—that measures the degree to which a child’s developmental challenges or difficult behaviors have led to impairment in various domains of family functioning. As an exploratory investigation, the FLIS was designed to include a scale that reflects parents’ perceptions of positive growth as a result of their child’s challenges. Although preceding the current study, the FLIS has already been used in several empirical studies based predominantly on its internal consistency and face validity (Carter et al., 2010; Petitclerc et al., 2015; Wakschlag et al., 2015). The aims of the present investigation are presented in two studies. The goals of this investigation are to (a) present a description of this new measure (Study 1), (b) assess the multidimensional factor structure of the FLIS (Study 1), (c) investigate convergent validity of the FLIS using established measures (Study 1), and (d) provide evidence of factor consistency and clinical utility of the FLIS within a clinical sample of children diagnosed with ASD (Study 2).

STUDY 1

In Study 1, we investigated the factor structure of the FLIS, including model fit, individual factor loadings, and correlations between factors. We also investigated the convergent validity of the FLIS with established measures. We hypothesized the following:

  • H1: The FLIS will conform to a multidimensional factor structure, representing different types of impairment within the family.

  • H2: Factors related to impairment will be positively associated with child symptoms, parent depression, and parent stress.

  • H3: Items on the FLIS that measure positive growth will also be positively associated with child symptoms (we investigated the relationship between positive growth and competence in an exploratory manner, as there is not adequate research in this area to make a directional hypothesis).

  • H4: A measure of temperament (i.e., inhibition to novelty) will demonstrate comparatively weaker associations with family impairment compared to an anxiety symptom scale.

  • H5: FLIS scores will generate an additive prediction of parental worry about the child, controlling for child symptoms.

Participants

The current work represents analysis of data collected within a longitudinal study of an age- and sex-stratified random population sample, initially ascertained from birth records provided by the state of Connecticut for children born from July 1995 to September 1997 at Yale New Haven Hospital (Carter, Briggs-Gowan, Jones, & Little, 2003). Based on birth record data, children at risk for developmental delays because of prematurity, low birth weight, low APGAR scores, birth complications, or long newborn hospital stay or those with an already-sampled sibling were excluded (n = 971; 12%). After applying exclusions, 7,433 families were eligible and, based on power analysis and feasibility, 1,788 were randomly sampled. Families were excluded if they had moved out of the state, the child was not in the custody of a biological parent, or if neither parent spoke English (n = 183). In addition, children were excluded if they could not be located despite intensive search efforts (n = 112) and in two cases, due to severe parent illness. Of 1,491 eligible families, 86% (n = 1,280) participated in Year 1. These participants were similar in sociodemographic characteristics to families living in the New Haven–Meriden Standard Metropolitan Statistical Area of the 1990 Census (Carter et al., 2003).

In the present study, data from Year 2 were examined. The Year 2 sample (n = 1,219) included 49 families that were initially selected in Year 1 but participated for the first time in Year 2, reflecting 91.4% retention and an 81.8% overall response rate. Seventeen families were deemed ineligible due to developmental delays and genetic disorders (e.g., autism spectrum, Fragile X). The sample used in this report was 945 after excluding children with nonmaternal respondents (n = 114) and children older than 48 months (n = 51). Children included in present analyses ranged from 23 to 48 months (M = 35.75, SD = 7.02). The sample was approximately equally distributed in terms of child sex (47.7% boys). Parents identified their children as being White (73%), African American/Black (15%), Hispanic (6.3%), Asian (2.7%), and other (3%). Maternal education varied, including 4.8% who had not completed high school, 14.3% with a high school degree or equivalent, 34.9% with some education beyond high school, and 46% with at least a college degree. Approximately 16.9% of households were single-parent homes. Poverty estimates, based on household income and family composition, indicated that 12.2% of families were living below the federal poverty line and 15.4% of families were living in borderline poverty (with incomes above 100% but below 185% of the poverty line).

Procedures

Parents were invited by mail to complete a survey about their child and family life when their child was between 11.5 and 23 months of age. Those who completed this survey were asked to complete a second survey, used in this report, the following year. Most parents completed the surveys independently; less than 1% were interviewed. Several measures were taken to encourage participation (e.g., sending a children’s book with the questionnaire booklet, telephoning, visiting their homes). Parents received $25 for each questionnaire they completed. All procedures were approved by two university Institutional Human Subjects Review Boards. Informed consent was obtained from all participants.

Measures

Family Impairment

The FLIS is a parent-report measure developed to assess the level of impairment experienced within a family due to a child’s developmental, behavioral, or emotional difficulties. It was designed to be used across a broad range of diagnostic categories. Items were drawn from clinical experience and broadly reflect impairment related to different areas of family life, such as participating in family activities, childcare, and parental well-being (see Table 1 for a list of items). Finally, in response to observation of parents’ sharing positive changes related to their child’s problems, this measure was also designed to capture how parents or families were affected in positive ways by a child’s developmental challenges.

TABLE 1.

Standardized Factor Loadings from Confirmatory Factor Analysis for FLIS Items

FLIS Item Family Imp. Childcare Imp. Positive Growth Parent Imp.
1. We rarely take him/her to visit family or friends .77
2. We rarely take him/her out to family restaurants .65
3. We rarely take him/her grocery shopping (or on other errands) .68
4. We rarely take him/her to meet new people or to new places .77
5. We rarely take him/her on car, bus, or train trips longer than 15 minutes .57
6. We rarely leave him/her with relatives whom s/he knows well .66
7. We rarely leave him/her with a babysittera .50
8. We rarely make changes to his/her daily schedule .46
9. It has been difficult to find appropriate childcare for him/hera .36
10. I have become more spiritual or religious .41
11. My relationships with family or friends have improved .65
12. I feel that I have grown or matured .88
13. I have learned to speak out for myself or for my child .85
14. My relationship with my spouse/partner has improvedb .66
15. I am unable to see or talk to my relatives as much as I would like .55
16. I am unable to give my other children enough attentiona .45
17. I am usually exhausted all day .57
18. My spouse/partner and I have more disagreementsb .56
19. I am unable to spend enough quiet or intimate timeb with my partner .68
Open in a new tab

Note: All loadings are significant at p < .001. All items begin with the sentence stem “Because of my child’s behavior, personality or special needs ….” FLIS = Family Life Impairment Scale; Imp. = Impairment.

a

Items with “not applicable” option.

b

Items parents are told not to answer unless they have a spouse or partner.

The FLIS contains 19 items beginning with the sentence stem, “Because of my child’s behavior, personality or special needs …” with the remaining being item specific. Parents respond using a 3-point scale: 0 (not true), 1 (somewhat true), and 2 (very true). The last three items pertain specifically to relational issues, which parents complete only if they have a spouse or partner. For certain items (identified in Table 1), parents are able to select not applicable, such as for items about childcare when parents elect to stay home by choice (these responses are treated as missing). Summary scores are calculated by taking the average of the individual items and multiplying by the number of items answered in each scale/subscale.

Child Temperament and Symptoms

The Infant–Toddler Social and Emotional Assessment (ITSEA) is a 166-item parent-report questionnaire that measures social-emotional/behavioral problems and competencies in children 11 to 48 months old (Carter & Briggs-Gowan, 2006; Carter, Briggs-Gowan, Jones, & Little, 2003). Items are rated on a 3-point scale from 0 (not true/rarely) to 2 (very true/often). This study included the Internalizing, Externalizing, Dysregulation, and Competence domain scales. The Competence domain assesses child competencies rather than symptoms or disabilities. To explore the ability of the FLIS to differentiate between temperamental and symptom-oriented scales, the Inhibition to Novelty (e.g., is quiet or less active in new situations), and General Anxiety (e.g., seems nervous, tense, or fearful) scales were used. The domain scales and subscales demonstrated acceptable reliability and validity with respect to internal consistency, test–retest reliability, interrater reliability, and both predictive and concurrent validity across three assessment years (Carter & Briggs-Gowan, 2006).

Parent Depression

The Center for Epidemiologic Studies Depression Inventory is a widely used, 20-item, self-report measure of adult depression symptoms (Radloff, 1977). This measure has high internal consistency, with coefficient alphas from .84 to .90 (α = .75 in our sample) and good test–retest reliability (ranging from .51 to .67) for 2- and 4-week intervals.

Parent Stress

Parent stress was measured with the 12-item Parent Distress subscale (α = .73) of the PSI–Short Form, a widely used measure (Abidin, 2009).

Parental Worry About Child Development

Parental worry about child development was assessed with four items, each beginning with the same item stem: “In general, how worried are you about your child’s [language development, emotional development, social development, behavior]?” Items are rated on a 5-point scale from 1 (not at all worried) to 5 (extremely worried). The four items were summed to create a composite score for parental worry about child development (α = .81).

Results

Exploratory Factor Analysis

The sample was randomly divided in half so that exploratory factor analysis (EFA) could be performed on one half of the sample (n = 483). Mplus Version 5.0 was used for latent variable analysis with the maximum likelihood estimator. All 19 original items were included in analysis. EFA with an oblique rotation resulted in four factors with eigenvalues greater than 1.0. A scree plot also suggested a four-factor solution. Inspection of individual items revealed that factors corresponded to three domains of impairment, representing impairment in family activities, impairment in childcare, and impairment in parent well-being. The fourth factor represented positive growth (how a parent was affected positively in response to parenting challenges).

Confirmatory Factor Analysis

Confirmatory factor analysis (CFA) was performed on the other half of the sample (n = 462). Results suggested that the four-factor model based on EFA (depicted in Figure 1) was a good fit for the data (df = 143; comparative fit index [CFI] = 0.94, Tucker–Lewis index [TLI] = 0.93, root mean square error of approximation [RMSEA] = 0.05, standardized root mean square residual [SRMR] = 0.05; for explanations and interpretations, see Hu & Bentler, 1999). Standardized factor loadings were between 0.36 and 0.88, and all loadings were statistically significant (p < .001). Standardized factor loadings are depicted in Table 1. All factors were significantly intercorrelated (see Table 2), with correlations ranging from .19 to .71. The lowest correlations were between the Positive Growth factor and the three impairment factors. This model was then compared to two other possible models: a single-factor solution and a two-factor solution, which represented one “Impairment” factor (encompassing family, parent, and childcare-related impairment) and the Positive Growth factor. The Bayesian Information Criterion (BIC) was used to compare model fit across models, with smaller BIC indicating a preferred model; a difference of more than 10 indicates “very strong” probability that the models are different (Raftery, 1995). The four-factor solution (BIC = 8821.99) was comparatively better than the single-factor solution (BIC = 9661.39) and the two-factor solution representing Impairment and Positive Growth (BIC = 8992.14). A one-factor solution with only the 14 items assessing impairment (i.e., excluding Positive Growth items) approached but failed to reach acceptable levels (df = 74; CFI = 0.90, TLI = 0.97, RMSEA = 0.07, SRMR = 0.63). A final model was tested in which items loaded on the same four original factors but with the three Impairment factors loading onto a higher order Total Impairment factor (see Figure 1). This model also demonstrated a good fit for the data (df = 145; CFI = 0.93, TLI = .92, RMSEA = 0.05, SRMR = 0.05), and had comparable fit for the four-factor model (BIC = 8829.74; difference < 10). Factor loadings for the total impairment factor were significant (p < .001) for Family (0.73), Childcare (0.91), and Parent (0.74) Impairment factors.

FIGURE 1.

FIGURE 1

Open in a new tab

Four factor confirmatory factor analysis model and four-factor confirmatory factor analysis model with higher-order factor. Note: All loadings and parameters (standardized estimates shown) are statistically significant at p < .001.

TABLE 2.

Intercorrelations Among Family Life Impairment Scale Factors from Confirmatory Factor Analysis for Both Samples

1 2 3 4
Study 1: Community Sample
  1. Family Impairment — .71 .19 .52
  2. Childcare Impairment — .30 .63
  3. Positive Growth — .43
  4. Parent Impairment —
Study 2: ASD Sample
  20. Family Impairment — .66 −.10 .46
  21. Childcare Impairment — −.10 .38
  22. Positive Growth — −.10
  23. Parent Impairment —
Open in a new tab

Note: All factor loadings are significant at p < .001. ASD = autism spectrum disorder.

Internal Consistency

The Cronbach’s alpha statistic was calculated for each FLIS subscale as a measure of internal consistency. Alphas for the four subscales were 0.84 (Family Impairment), 0.64 (Childcare Impairment), 0.83 (Positive Growth), and 0.73 (Parent Impairment). The alpha was also tested for the 14-item Total Impairment summary scale (α = 0.83).

Associations between FLIS and Demographic Variables

Boys had slightly higher levels of total impairment compared to girls on the FLIS, F(1) = 4.42, p < .05. When the three Impairment subscales were tested in the same model with child sex, only Parent Impairment approached significance, F (1) = 3.14, p = .08, with boys having higher levels than girls. Positive Growth was not associated with child sex. Significant differences in total impairment were found between socioeconomic groups (nonpoverty, borderline poverty, and poverty, F (2) = 38.58, p < .001; post hoc tests revealed that higher Impairment scores were associated with each higher level of poverty. This same pattern was apparent for the Positive Growth scale, F(2) = 45.04, p < .001, which was positively associated with higher levels of poverty. Child age was not associated with either the FLIS Total Impairment or any of the four subscales.

Associations between the FLIS and Other Variables

Relationships between the FLIS factors and other measures are presented in Table 3. FLIS factors demonstrated strong relationships with domain scales and symptom scales of the ITSEA, as well as parent depression and parent stress, providing evidence of convergent validity. Differences between FLIS factors also emerged with respect to several correlates. Regarding child symptoms, Parent Impairment yielded the strongest associations with externalizing behaviors and dysregulation. Not surprisingly, Parent Impairment was also most strongly associated with both parent depression and parent distress. Regarding child symptoms, Family Impairment was most strongly associated with externalizing behaviors and dysregulation, whereas Childcare Impairment was strongly associated with internalizing symptoms. The Positive Growth factor showed modest, albeit significant, associations with a range of measures. As expected, the Inhibition to Novelty scale, which is designed to assess temperamental variation rather than psychopathology, demonstrated only modest associations with all subscales of the FLIS, and each of these was smaller (p < .05) in comparison to those of the General Anxiety subscale. The ITSEA Competence scale demonstrated a lack of association with Positive Growth. The Competence scale was negatively associated with the other three FLIS factors and the Total Impairment factor.

TABLE 3.

Associations between Family Life Impairment Scale Factors and Other Measures

Variable Family Impairment
β (SE)		 Childcare Impairment
β (SE)		 Positive Growth
β (SE)		 Parent Impairment
β (SE)		 Total Impairment
β (SE)
Internalizing .25 (.03)*** .38 (.04)*** .21 (.03)*** .33 (.04)*** .38 (.03)***
Externalizing .35 (.03)*** .33 (.04)*** .22 (.03)*** .58 (.03)*** .49 (.03)***
Dysregulation .30 (.03)*** .31 (.04)*** .17 (.03)*** .46 (.03)*** .41 (.03)***
Parent Distress .30 (.03)*** .29 (.04)*** .15 (.03)*** .60 (.03)*** .46 (.04)***
Parent Depression .25 (.03)*** .24 (.04)*** .19 (.03)*** .43 (.03)*** .34 (.04)***
Inhibition to Novelty .05 (.04) .12 (.04)** .06 (.04) .07 (.04) .09 (.04)*
General Anxiety .22 (.03)*** .38 (.04)*** .31 (.03)*** .31(.04)*** .36 (.03)***
Competence −.24 (.03)*** −.21 (.04)*** .03 (.04) −.24 (.04)*** −.27 (.04)***
Open in a new tab

Note: Total Impairment does not include the Positive Growth factor. All scales are derived from the ITSEA, except Parent Depression (Center for Epidemiologic Studies Depression Inventory) and Parent Distress (from the Parenting Stress Index–Short Form).

*

p < .05.

**

p < .01.

***

p < .001.

Hierarchical regression was used to investigate associations between FLIS scores and parental worry about child development, controlling for symptoms. Child sex, child age, and poverty status (dichotomized) were entered in the first step as covariates, and child symptoms were entered into the second step. A separate model was run for each symptom domain (internalizing, externalizing, and dysregulation). The FLIS Total Impairment score was entered into the third step. For all three models, both symptoms and FLIS scores were significant predictors in the final step (see Table 4). All models accounted for a significant portion of the variance in parental worry (total R2 = .22, .20, and .23 for models including internalizing, externalizing, and dysregulation, respectively).

TABLE 4.

Hierarchical Regression Models Predicting Parental Worry about Child Development (Final Step Shown)

Model Predictor Step Entered B SE β R2 Change
1 Internalizing 2 3.20 .34 .30***
FLIS Total Imp. 3 0.18 .02 .24*** .06
2 Externalizing 2 2.37 .31 .26***
FLIS Total Imp. 3 0.18 .03 .24*** .05
3 Dysregulation 2 3.20 .34 .30***
FLIS Total Imp. 3 0.18 .02 .24*** .05
Open in a new tab

Note: Child age, child sex, and poverty status entered into Step 1 as covariates for all models. Imp. = Impairment.

***

p < .001.

STUDY 2

The purpose of Study 2 was to test the factor structure of the FLIS in a clinical sample of children diagnosed with ASD. Previous research has found differences in factor structure when testing measures in samples of parents with children with ASD (Zaidman-Zait et al., 2011). Because the FLIS was designed to be relevant and applicable to any developmental or psychiatric condition in early childhood (including clinical as well as nonclinical samples), we hypothesized that the same four-factor structure would be supported in this clinical sample.

Participants

Participants included 174 toddlers with ASD (M age = 28 months, SD = 3.9, range = 18–33 months) and their mothers who were enrolled in a longitudinal study of developmental trajectories of young children (for additional details, see Ben-Sasson, Soto, Martínez-Pedraza, & Carter, 2013). Data for this study are from the first wave of data collection. Most of the sample (78%, n = 136) were boys, consistent with other ASD samples (Robinson, Lichtenstein, Anckarsäter, Happé, & Ronald, 2013). Participants were mothers of toddlers, primarily the biological parent (99%) with an average age of 36.17 years (range = 19–58, SD = 5.03) at the time of participation. Eighty-four percent of mothers self-identified as non-Hispanic/Latino White. Most mothers reported middle- to upper-class household incomes (88%) and had at least 2 years of college (79%). The majority of mothers were married or cohabiting (91%). Inclusion criteria were (a) child age between 18 to 33 months and (b) a diagnosis of autistic disorder or a pervasive developmental disorder based on meeting research criteria on the Autism Diagnostic Interview–Revised (Rutter, Le Couteur, & Lord, 2003), the Autism Diagnostic Observational Schedule–Generic (Lord, Rutter, DiLavore, & Risi, 2002), and the clinical impression of an experienced psychologist. Children with a physical disability, known genetic disorder, or neurological disorder were excluded. Only one child per family was included in these analyses.

Procedures

Participating families were recruited through early intervention providers, physicians specializing in the diagnosis of ASD, local conferences, and events for families of children with ASD. Families completed two visits in each assessment year: a child visit, which took place in a laboratory setting, and a parent interview, which was completed in either an office setting or the parents’ home. Parents were also asked to complete a questionnaire booklet, which included the FLIS. Families were followed annually with the same child assessments, interviews, and questionnaires. Parents were compensated $50 for participating in the larger study.

Results

Cronbach’s alphas for the four subscales were 0.85 (Family Impairment), 0.62 (Childcare Impairment), 0.68 (Positive Growth), and 0.74 (Parent Impairment). The alpha for the Total Impairment summary scale was 0.85. Mplus 5.0 was used for CFA. Overall model fit was good (df = 145; CFI = 0.94, TLI = 0.93, RMSEA = 0.04, SRMR = 0.06) for the four-factor structure presented in Study 1. Standardized factor loadings for individual items ranged from 0.36 and 0.9, and all loadings were statistically significant (p < .001). Standardized factor correlations produced in the CFA are depicted in Table 2. The model with the higher order Total Impairment factor, as described in Study 1, was also tested. This higher order model also exhibited a good fit for the data (df = 145; CFI = 0.94, TLI = 0.94, RMSEA = 0.05, SRMR = 0.06). Factor loadings for the Total Impairment factor were significant (p < .001) for Family (0.89), Childcare (0.74), and Parent (0.52) Impairment factors. Unlike results from Study 1, the Positive Growth factor was negatively correlated (r = −.10) with the Total Impairment factor, as well as each of the Impairment scales: Family Impairment (r = −.09), Childcare Impairment (r = −.07), and Parent Impairment (r = −.08).

DISCUSSION

Researchers and clinicians are aware that young children’s mental health problems can be impairing to family participation in expected routines and activities of family life but have limited standardized tools available to evaluate this important construct. The FLIS is the first questionnaire that was specifically designed to assess clinical impairment by focusing on impairment within the family, making it developmentally appropriate for young children. Results provide strong evidence for the validity of the four-factor structure of this multidimensional measure of family impairment; three domains of impairment and one domain of positive growth were validated in both a representative, normative sample, as well as sample of families with a toddler with ASD. Convergent validity was also supported using known, previously validated measures of parent distress and child symptoms. This study indicates that the FLIS is a valid measure of parental perceptions of the effects of a child’s disability and behaviors in the family. The FLIS may be a useful clinical tool to assess the impact of the child in the family and to identify specific targets for intervention.

Study 1 investigated the factor structure of the FLIS in a normative, representative sample of toddlers. As hypothesized, the FLIS conformed to a multidimensional structure, reflecting childcare, parent, and family impairment. This finding lends support to the long-held assertion that mental health conditions in young children have profound effects on the child’s family (Carter et al., 2004). Furthermore, results suggest that different conditions have different profiles of impairment. For example, internalizing symptoms were most associated with childcare-related difficulties, which is not surprising considering that, for example, being left with a babysitter can be a problem for a child with separation anxiety (separating from parents), social anxiety (needing to interact with an unfamiliar adult), or generalized anxiety (worrying about what might happen when the parent is not home). In contrast, externalizing symptoms were most impairing to parent well-being. Again, this is consistent with previous studies showing a positive association between parent stress and behavior problems in young children (Petitclerc et al., 2015; Wakschlag et al., 2015; Zaidman-Zait et al., 2014). As hypothesized, the FLIS showed weaker associations with inhibited temperament (only a modest association with childcare impairment), compared to stronger associations with anxiety symptoms. This suggests that the FLIS may provide useful information to help disentangling symptoms from temperamental constructs such as inhibition (Mian, 2014; Rapee & Coplan, 2010), although more study will be necessary to further investigate this notion.

The design of the FLIS and study results suggest that it is appropriate for both research and clinical applications. It is noteworthy that the data provided comparable fit for a model with a single, higher-order factor reflecting Total Impairment. From a theoretical perspective, we believe that this higher order model is the preferred model because it reflects both the multidimensional nature of family impairment while demonstrating that the three impairment factors “hang together” in a sound manner. In clinical settings or community-based research contexts, it may be most appropriate to use the score for Total Impairment, rather than the subscales, to simplify scoring procedures and interpretation. Due to the brevity of the FLIS, it could also be appropriate for screening practices, as identifying families with higher levels of family impairment could be of use in studies intended to identify young children at risk of more severe psychopathology or families that would benefit from family-based treatments.

Perhaps one of the most intriguing findings is that the positive growth factor, reflecting growth associated with having a child with a developmental or behavioral problem, also emerged as a separate factor that was confirmed in both samples. This scale lends preliminary support to the idea that parents often benefit (through growth or maturity) in the process of facing child mental health problems. This factor offers researchers a way to measure a phenomenon that has thus far been largely ignored. The idea of studying—perhaps with the hope of clinically harnessing—positive parental growth ascertained through negotiating a child’s mental health challenges is highly consistent with the Positive Psychology movement’s emphasis on resilience and positive well-being (Reivich, Gillham, Chaplin, & Seligman, 2005). If, through clinical intervention, parents can identify with ways in which they have grown or matured as a result of taking steps to help a troubled child, this could bolster parenting self-efficacy, an important indicator of effective parenting (Jones & Prinz, 2005). However, it is worth nothing that while the Positive Growth factor was positively correlated with Total Impairment in the community sample, it was negatively correlated in the ASD sample. This may indicate that the relationship between growth and impairment is nonlinear, wherein there is a positive association when the level of impairment is low-to-moderate but the association is negative when impairment levels are high (as may be the case with children with ASD). In addition, the association may be negative when parents are in the early stages of adapting to a child’s diagnosis; indeed, the mean age of the ASD sample was slightly lower than the community sample. It will be important to examine these relations over time to more fully understand the patterns observed in this cross-sectional study.

In addition to demonstrating convergent validity with scales that would be expected to predict family impairment, the FLIS also showed a negative association with child competency, suggesting that delays in the acquisition of social and emotional competencies are associated with family impairment. Although the FLIS was developed to specifically assess family impairment related to psychopathology, this finding suggests that it is sensitive enough to detect challenges that are less severe. Notably, competence (or lack thereof) was not associated with positive growth, suggesting that such growth appears to be specific to a parent’s process of confronting a developmental or behavioral problem, perhaps because of the impairment associated with it (and the associated need for formal treatment, social support, or advocacy).

Incorporation of family impairment (rather than a sole focus on individual impairment) into diagnostic formulations for young children is of paramount importance. Consistent with this approach, the recent DC: 0–5 Diagnostic Classification of Mental Health and Developmental Disorders of Infancy and Early Childhood (Zero to Three, 2016) has resulted in updated diagnostic criteria for young children that specifically reflects family impairment. In DC: 0–5, diagnostic criteria include “Symptoms of the disorder, or caregiver accommodations in response to the symptoms, significantly impact the child and/ or family functioning” (Zero to Three, 2016, p. 17). Consistent with the conceptual framework of the FLIS, this criterion goes on to further define this impact in several ways, including limiting “the family’s participation in everyday activities” (Zero to Three, 2016, p. 17). Inclusion here marks family impairment as an essential construct for evaluation, necessitating the availability of valid measures to test it.

Broader application of the FLIS with young children could lead to an improved understanding of family impairment as a treatment target or as a construct indirectly ameliorated through treatment, which could provide evidence of effectiveness in clinical trials. The National Institute of Mental Health has called on clinical researchers to “develop objective surrogate measures of outcome and clinical change that extend beyond symptoms, to assess if target mechanisms underlying general health and quality of life have been modified by treatments” (The National Institute of Mental Health, 2015, p. 39). In young children, reducing family impairment is arguably the most important domain to assess (beyond symptoms) for evaluating clinical effectiveness. Evidence of reducing family impairment would lend considerable support for the power of psychological interventions.

Despite robust evidence to support the factor structure and validity of the FLIS, results should be considered within certain limitations. First, both studies relied exclusively on parent reports for child and parent symptoms and competencies. However, the ASD sample in Study 2 included children who had received a clinical diagnosis of ASD, providing some evidence that findings regarding factor structure extend beyond parent-reported symptoms. Second, although the use of a clinical sample of children diagnosed with ASD is a strength of this report, we did not investigate the factor structure or validity of the FLIS in other clinical samples. Future studies focused on internalizing and externalizing disorders would be helpful. Finally, careful research to identify and validate clinical cut-scores will be important for clinical and research applications in the future.

The FLIS demonstrated that family impairment is a multidimensional construct that is related to both child symptoms and parent well-being. Furthermore, its brevity and ease of interpretation make it compatible with both research and clinical applications. Further employment of the FLIS in studies of developmental psychopathology, as well as intervention research, provides an opportunity to expand our knowledge of this important construct.

Acknowledgments

FUNDING

This work was supported by the National Center for Research Resources (M01 RR0053) and the National Institute of Mental Health (R01MH55278, U54MH066398).

Contributor Information

Nicholas D. Mian, Department of Life Sciences, University of New Hampshire

Timothy W. Soto, Clinical Psychology Department, William James College

Margaret J. Briggs-Gowan, Department of Psychiatry, University of Connecticut Health Center

Alice S. Carter, Department of Psychology, University of Massachusetts Boston

References

  1. Abidin RR. Parenting stress index-short form (PSI-SF) New York, NY: Multi-Health Systems; 2009. [Google Scholar]
  2. American Academy of Pediatrics. Council on children with disabiltiies, section on developmental behavioral pediatrics, bright futures steering committee and medical home initiatives for children with special needs project advisory committee: Identifying infants and young children with developmental disorders in the medical home: An algorithm for developmental surveillance and screening. Pediatrics. 2006;118:405–420. doi: 10.1542/peds.2006-1231. [DOI] [PubMed] [Google Scholar]
  3. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Fifth. Washington, DC: Author; 2013. [Google Scholar]
  4. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4. Washington DC: American Psychiatric Association; 2000. Text revision. [Google Scholar]
  5. Ben-Sasson A, Soto TW, Martínez-Pedraza F, Carter AS. Early sensory over-responsivity in toddlers with autism spectrum disorders as a predictor of family impairment and parenting stress. Journal of Child Psychology and Psychiatry. 2013;54(8):846–853. doi: 10.1111/jcpp.2013.54.issue-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bird HR, Canino GJ, Davies M, Ramírez R, Chávez L, Duarte C, Shen S. The Brief Impairment Scale (BIS): A multidimensional scale of functional impairment for children and adolescents. Journal of the American Academy of Child & Adolescent Psychiatry. 2005;44(7):699–707. doi: 10.1097/01.chi.0000163281.41383.94. [DOI] [PubMed] [Google Scholar]
  7. Bird HR, Shaffer D, Fisher P, Gould MS. The Columbia Impairment Scale (CIS): Pilot findings on a measure of global impairment for children and adolescents. International Journal of Methods in Psychiatric Research. 1993;3(3):167–176. [Google Scholar]
  8. Carter AS, Briggs-Gowan MJ. ITSEA infant-toddler social and emotional assessment. San Antonio, TX: Psychological Corporation Harcourt Press; 2006. [Google Scholar]
  9. Carter AS, Briggs-Gowan MJ, Davis NO. Assessment of young children’s social-emotional development and psychopathology: Recent advances and recommendations for practice. Journal of Child Psychology and Psychiatry. 2004;45(1):109–134. doi: 10.1046/j.0021-9630.2003.00316.x. [DOI] [PubMed] [Google Scholar]
  10. Carter AS, Briggs-Gowan MJ, Jones SM, Little TD. The Infant-Toddler Social and Emotional Assessment (ITSEA): Factor structure, reliability, and validity. Journal of Abnormal Child Psychology. 2003;31(5):495–514. doi: 10.1023/A:1025449031360. [DOI] [PubMed] [Google Scholar]
  11. Carter AS, Wagmiller RJ, Gray SAO, McCarthy KJ, Horwitz SM, Briggs-Gowan MJ. Prevalence of DSM-IV disorder in a representative, healthy birth cohort at school entry: Sociodemographic risks and social adaptation. Journal of the American Academy of Child & Adolescent Psychiatry. 2010;49(7):686–698. doi: 10.1016/j.jaac.2010.03.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Costello EJ, Pescosolido BA, Angold A, Burns A. A family network-based model of access to child mental health services. Research in Community Mental Health. 1998;9:165–190. [Google Scholar]
  13. Davis NO, Carter AS. Parenting stress in mothers and fathers of toddlers with autism spectrum disorders: Associations with child characteristics. Journal of Autism and Developmental Disorders. 2008;38(7):1278–1291. doi: 10.1007/s10803-007-0512-z. [DOI] [PubMed] [Google Scholar]
  14. DelCarmen-Wiggins R, Carter A. Handbook of infant, toddler, and preschool mental health assessment. New York, NY US: Oxford University Press; 2004. [Google Scholar]
  15. Dempsey I, Keen D, Pennell D, O’Reilly J, Neilands J. Parent stress, parenting competence and family-centered support to young children with an intellectual or developmental disability. Research in Developmental Disabilities. 2009;30(3):558–566. doi: 10.1016/j.ridd.2008.08.005. [DOI] [PubMed] [Google Scholar]
  16. Egger HL, Angold A. Common emotional and behavioral disorders in preschool children: Presentation, nosology, and epidemiology. Journal of Child Psychology and Psychiatry. 2006;47(3):313–337. doi: 10.1111/j.1469-7610.2006.01618.x. [DOI] [PubMed] [Google Scholar]
  17. Egger HL, Angold A. The Preschool Age Psychiatric Assessment (PAPA): A structured parent interview for diagnosing psychiatric disorders in preschool children. In: DelCarmen-Wiggins R, Carter A, editors. Handbook of infant, toddler, and preschool mental health assessment. New York, NY US: Oxford University Press; 2004. pp. 223–243. [Google Scholar]
  18. Egger HL, Emde RN. Developmentally sensitive diagnostic criteria for mental health disorders in early childhood: The diagnostic and statistical manual of mental disorders—IV, the research diagnostic criteria—preschool age, and the diagnostic classification of mental health and developmental disorders of infancy and early childhood—revised. American Psychologist. 2011;66(2):95–106. doi: 10.1037/a0021026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Eisenhower AS, Baker BL, Blacher J. Children’s delayed development and behavior problems: Impact on mothers’ perceived physical health across early childhood. Social Science & Medicine. 2009;68(1):89–99. doi: 10.1016/j.socscimed.2008.09.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Ellingson KD, Briggs-Gowan MJ, Carter AS, Horwitz SM. Parent identification of early emerging child behavior problems: Predictors of sharing parental concern with health providers. Archives Of Pediatrics & Adolescent Medicine. 2004;158(8):766–772. doi: 10.1001/archpedi.158.8.766. [DOI] [PubMed] [Google Scholar]
  21. Epstein T, Saltzman-Benaiah J, O’Hare A, Goll JC, Tuck S. Associated features of Asperger syndrome and their relationship to parenting stress. Child: Care, Health and Development. 2008;34(4):503–511. doi: 10.1111/j.1365-2214.2008.00834.x. [DOI] [PubMed] [Google Scholar]
  22. Gray DE. Ten years on: A longitudinal study of families of children with autism. Journal of Intellectual and Developmental Disability. 2002;27(3):215–222. doi: 10.1080/1366825021000008639. [DOI] [Google Scholar]
  23. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6(1):1–55. doi: 10.1080/10705519909540118. [DOI] [Google Scholar]
  24. Jokinen NS, Brown RI. Family quality of life from the perspective of older parents. Journal of Intellectual Disability Research. 2005;49(10):789–793. doi: 10.1111/jir.2005.49.issue-10. [DOI] [PubMed] [Google Scholar]
  25. Jones TL, Prinz RJ. Potential roles of parental self-efficacy in parent and child adjustment: A review. Clinical Psychology Review. 2005;25(3):341–363. doi: 10.1016/j.cpr.2004.12.004. [DOI] [PubMed] [Google Scholar]
  26. Langley AK, Bergman RL, McCracken J, Piacentini JC. Impairment in childhood anxiety disorders: Preliminary examination of the child anxiety impact scale-parent version. Journal of Child and Adolescent Psychopharmacology. 2004;14(1):105–114. doi: 10.1089/04454604773840544. [DOI] [PubMed] [Google Scholar]
  27. Lavelle TA, Weinstein MC, Newhouse JP, Munir K, Kuhlthau KA, Prosser LA. Economic burden of childhood autism spectrum disorders. Pediatrics. 2014;133(3):e520–e529. doi: 10.1542/peds.2013-0763. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Lavigne JV, LeBailly SA, Hopkins J, Gouze KR, Binns HJ. The prevalence of ADHD, ODD, depression, and anxiety in a community sample of 4-year-olds. Journal of Clinical Child and Adolescent Psychology. 2009;38(3):315–328. doi: 10.1080/15374410902851382. [DOI] [PubMed] [Google Scholar]
  29. Llewellyn G, McConnell D, Thompson K, Whybrow S. Out-of-home placement of school-age children with disabilities and high support needs. Journal of Applied Research in Intellectual Disabilities. 2005;18(1):1–6. doi: 10.1111/j.1468-3148.2004.00201.x. [DOI] [Google Scholar]
  30. Lord C, Rutter M, DiLavore PC, Risi S. Autism Diagnostic Observation Schedule-Generic (ADOS-G) manual. Los Angeles, CA: Western Psychological Services; 2002. [Google Scholar]
  31. Luby JL. Handbook of preschool mental health: Development, disorders, and treatment. New York, NY US: Guilford Press; 2006. [Google Scholar]
  32. McStay RL, Trembath D, Dissanayake C. Stress and family quality of life in parents of children with autism spectrum disorder: Parent gender and the double ABCX model. Journal of Autism and Developmental Disorders. 2014;44(12):3101–3118. doi: 10.1007/s10803-014-2178-7. [DOI] [PubMed] [Google Scholar]
  33. Mian ND. Little children with big worries: Addressing the needs of young, anxious children and the problem of parent engagement. Clinical Child and Family Psychology Review. 2014;17(1):85–96. doi: 10.1007/s10567-013-0152-0. [DOI] [PubMed] [Google Scholar]
  34. Osborne LA, McHugh L, Saunders J, Reed P. A possible contra-indication for early diagnosis of autistic spectrum conditions: Impact on parenting stress. Research in Autism Spectrum Disorders. 2008;2(4):707–715. doi: 10.1016/j.rasd.2008.02.005. [DOI] [Google Scholar]
  35. Pavuluri MN, Luk S-L, McGee R. Help-seeking for behavior problems by parents of preschool children: A community study. Journal of the American Academy of Child & Adolescent Psychiatry. 1996;35(2):215–222. doi: 10.1097/00004583-199602000-00015. [DOI] [PubMed] [Google Scholar]
  36. Petitclerc A, Briggs-Gowan MJ, Estabrook R, Burns JL, Anderson EL, McCarthy KJ, Wakschlag LS. Contextual variation in young children’s observed disruptive behavior on the DB-DOS: Implications for early identification. Journal of Child Psychology and Psychiatry. 2015;56(9):1008–1016. doi: 10.1111/jcpp.12430. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Radloff LS. The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement. 1977;1(3):385–401. doi: 10.1177/014662167700100306. [DOI] [Google Scholar]
  38. Raftery AE. Bayesian model selection in social research. Sociological Methodology. 1995;25:111–163. doi: 10.2307/271063. [DOI] [Google Scholar]
  39. Rao PA, Beidel DC. The impact of children with high-functioning autism on parental stress, sibling adjustment, and family functioning. Behavior Modification. 2009;33(4):437–451. doi: 10.1177/0145445509336427. [DOI] [PubMed] [Google Scholar]
  40. Rapee RM, Coplan RJ. Conceptual relation between anxiety disorder and fearful temperament. In: Gazelle H, Rubin K, editors. Social anxiety in childhood: Bridging developmental and clinical perspectives. New directions for child and adolescent development. San Francisco, CA: Jossey-Bass; 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Redmond B, Richardson V. Just getting on with it: exploring the service needs of mothers who care for young children with severe/profound and life-threatening intellectual disability. Journal of Applied Research in Intellectual Disabilities. 2003;16(3):205–218. doi: 10.1046/j.1468-3148.2003.00165.x. [DOI] [Google Scholar]
  42. Reivich K, Gillham JE, Chaplin TM, Seligman MEP. From helplessness to optimism: The role of resilience in treating and preventing depression in youth. Handbook of resilience in children. New York, NY: Kluwer Academic/Plenum Publishers; 2005. pp. 223–237. [Google Scholar]
  43. Robinson EB, Lichtenstein P, Anckarsäter H, Happé F, Ronald A. Examining and interpreting the female protective effect against autistic behavior. PNAS Proceedings of the National Academy of Sciences of the United States of America. 2013;110(13):5258–5262. doi: 10.1073/pnas.1211070110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Rutter M, Le Couteur A, Lord C. Autism diagnostic interview-revised manual. Los Angeles, CA: Western Psychological Services; 2003. [Google Scholar]
  45. Singer JB, Eack SM, Greeno CM. The Columbia Impairment Scale: Factor analysis using a community mental health sample. Research on Social Work Practice. 2011;21(4):458–468. doi: 10.1177/1049731510394464. [DOI] [Google Scholar]
  46. Stein REK, Jessop DJ. The impact on family scale revisited: Further psychometric data. Journal of Developmental and Behavioral Pediatrics. 2003;24(1):9–16. doi: 10.1097/00004703-200302000-00004. [DOI] [PubMed] [Google Scholar]
  47. Task Force on Research Diagnostic Criteria: Infancy and Preschool. Research diagnostic criteria for infants and preschool children: The process and empirical support. Journal of the American Academy of Child & Adolescent Psychiatry. 2003;42(12):1504–1512. doi: 10.1097/00004583-200312000-00018. [DOI] [PubMed] [Google Scholar]
  48. The National Institute of Mental Health. NIMH strategic plan for research. 2015 Retrieved from http://www.nimh.nih.gov/about/strategic-planning-reports/index.shtml.
  49. Wakschlag LS, Briggs-Gowan MJ, Carter AS, Hill C, Danis B, Keenan K, Leventhal BL. A developmental framework for distinguishing disruptive behavior from normative misbehavior in preschool children. Journal of Child Psychology and Psychiatry. 2007;48(10):976–987. doi: 10.1111/j.1469-7610.2007.01786.x. [DOI] [PubMed] [Google Scholar]
  50. Wakschlag LS, Choi SW, Carter AS, Hullsiek H, Burns J, McCarthy K, Briggs-Gowan MJ. Defining the developmental parameters of temper loss in early childhood: Implications for developmental psychopathology. Journal of Child Psychology and Psychiatry. 2012;53(11):1099–1108. doi: 10.1111/j.1469-7610.2012.02595.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Wakschlag LS, Estabrook R, Petitclerc A, Henry D, Burns JL, Perlman SB, Briggs-Gowan ML. Clinical implications of a dimensional approach: The normal: Abnormal spectrum of early irritability. Journal of the American Academy of Child & Adolescent Psychiatry. 2015;54(8):626–634. doi: 10.1016/j.jaac.2015.05.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Williams AR, Piamjariyakul U, Williams PD, Bruggeman SK, Cabanela RL. Validity of the revised Impact on Family (IOF) scale. The Journal Of Pediatrics. 2006;149(2):257–261. doi: 10.1016/j.jpeds.2006.04.003. [DOI] [PubMed] [Google Scholar]
  53. Yatchmenoff DK, Koren PE, Friesen BJ, Gordon LJ, Kinney RF. Enrichment and stress in families caring for a child with a serious emotional disorder. Journal of Child and Family Studies. 1998;7(2):129–145. doi: 10.1023/A:1022935014502. [DOI] [Google Scholar]
  54. Zaidman-Zait A, Mirenda P, Duku E, Szatmari P, Georgiades S, Volden J, Thompson A. Examination of bidirectional relationships between parent stress and two types of problem behavior in children with autism spectrum disorder. Journal of Autism and Developmental Disorders. 2014;44(8):1908–1917. doi: 10.1007/s10803-014-2064-3. [DOI] [PubMed] [Google Scholar]
  55. Zaidman-Zait A, Mirenda P, Zumbo BD, Georgiades S, Szatmari P, Bryson S, Thompson A. Factor analysis of the parenting stress index-short form form with parents of young children with autism spectrum disorders. Autism Research. 2011;4(5):336–346. doi: 10.1002/aur.213. [DOI] [PubMed] [Google Scholar]
  56. Zeanah CH., Jr . Handbook of infant mental health. 3. New York, NY US: Guilford Press; 2009. [Google Scholar]
  57. Zero to Three. DC:0–5: Diagnostic classification of mental health and developmental disorders of infancy and early childhood. Washington, DC: Author; 2016. [Google Scholar]
  58. Zielinski K, Wood JJ, Renno P, Whitham S, Sterling L. Examining the validity of the Columbia impairment scale for assessing level of functioning in youth with autism spectrum disorder. Journal of Child and Adolescent Psychopharmacology. 2014;24(9):509–512. doi: 10.1089/cap.2014.0095. [DOI] [PubMed] [Google Scholar]

RESOURCES