Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Nov 6.
Published in final edited form as: Am J Orthopsychiatry. 2009 Oct;79(4):532–540. doi: 10.1037/a0017848

Children With Co-Occurring Anxiety and Externalizing Disorders: Family Risks and Implications for Competence

Joan P Yoo 1, Pamela J Brown 2, Suniya S Luthar 3
PMCID: PMC3490217  NIHMSID: NIHMS410756  PMID: 20099944

Abstract

This study used data from 340 mother–child dyads to examine characteristics of children with co-occurring diagnoses of anxiety and externalizing disorders and compared them with children with a sole diagnosis or no diagnosis. Comparisons were made using 4 child-diagnostic groups: anxiety-only, externalizing-only, co-occurrence, and no-problem groups. Most mothers were characterized by low income and histories of psychiatric diagnoses during the child’s lifetime. Analyses using multinomial logistic regressions found the incidence of co-occurring childhood disorders to be significantly linked with maternal affective/anxiety disorders during the child’s lifetime. In exploring implications for developmental competence, we found the co-occurrence group to have the lowest level of adaptive functioning among the 4 groups, faring significantly worse than the no-problem group on both academic achievement and intelligence as assessed by standardized tests. Findings underscore the importance of considering co-occurring behavior problems as a distinct phenomenon when examining children’s developmental outcomes.

Keywords: co-occurring psychiatric disorders, anxiety disorders, externalizing disorders

Comorbidity refers to the coexistence of two or more distinct disorders in the same individual at the same point in time (Achenbach, 1991; Caron & Rutter, 1991). Previous research suggests that children with comorbid conditions are more likely to have early onset of symptoms, increased levels of symptomatology, worse trajectories of disorders, and poorer treatment outcomes when compared with children who have a single disorder (Loeber & Keenan, 1994; Newman, Moffitt, Caspi, & Silva, 1998; Nottelman & Jensen, 1995; Wright, Zakriski, & Drinkwater, 1999). Researchers have emphasized that studies on comorbidity have major implications for research and treatment in areas such as diagnosing psychopathology, identifying an etiology and its consequences, and evaluating interventions and treatments (Caron & Rutter, 1991; Nottelman & Jensen, 1995; Verhulst & van der Ende, 1993).

The importance of establishing a clear distinction between the terms comorbidity and co-occurrence, which are often used interchangeably, has been emphasized by researchers studying issues of comorbidity (Keiley, Lofthouse, Bates, Dodge, & Pettit, 2003). Comorbidity refers to the verified coexistence of two or more psychiatric disorders that have been diagnosed with rigorous multimethod diagnostic procedures based on the criteria from psychiatric references such as Diagnostic and Statistical Manual of Mental Disorders (4th. ed.; DSM–IV; American Psychiatric Association, 1994; Lilienfeld, Waldman, & Israel, 1994). However, in situations where such verification of diagnoses is difficult (e.g., identifying psychiatric problems using standardized survey instruments), it has been suggested that the term co-occurrence is more appropriate than comorbidity (Hinden, Compas, Howell, & Achenbach, 1997; Keiley et al., 2003; Lilienfeld et al., 1994). Therefore, given the methodological constraints of this study, we have used the term co-occurrence rather than comorbidity to refer to cases when two or more broadband psychiatric disorders have been identified.

Co-Occurrence of Anxiety and Externalizing Problems

Anxiety disorders are one of the most common psychiatric disorders experienced by children and adolescents (Brown & Barlow, 1992), which have also been found to have high rates of comorbidity (Angold, Costello, & Erkanli, 1999). Angold and colleagues (1999) reported that the degree of association, reported in median odds ratios, between anxiety and externalizing problems (i.e., ADHD–anxiety, conduct disorder (CD)/oppositional-defiant disorder (ODD)–anxiety) is approximately 3.0. This finding and findings from other studies indicate that comorbidity between anxiety and externalizing problems is not just a statistical artifact but an actual phenomenon (Franco, Saavedra, & Silverman, 2007).

Although numerous studies have separately examined the risk and protective factors associated with anxiety and externalizing behavioral problems (Costello, Egger, & Angold, 2005; Deater-Deckard, Dodge, Bates, & Pettit, 1998), substantially fewer studies have examined these factors with comorbid psychiatric disorders (Keiley et al., 2003). For example, it has been well established that boys are more likely to exhibit externalizing problems (Achenbach, 1991; Crick & Zahn-Waxler, 2003). On the other hand, it has been reported that girls are more likely to report anxiety disorders than boys (see Costello et al., 2005, for review), although there is some evidence that these gender differences in anxiety disorders may be due to confounding variables (Lewinsohn, Gotlib, Lewinsohn, Seelfey, & Allen, 1998). However, gender differences in heterotypic comorbidity, particularly between anxiety and externalizing problems, have yet to be thoroughly investigated. Similarly, the effects of otherwise well-documented risk factors such as low socioeconomic status (see Bradley & Corwyn, 2002; Duncan & Brooks-Gunn, 1997, for reviews) and maternal psychiatric disorders, including depression and substance abuse (Beidel & Turner, 1997; Luthar & Sexton, 2007; Pilowsky, Wickramaratne, Nomura, & Weissman, 2006), have been insufficiently explored in relation to co-occurring disorders in children.

Implications of Co-Occurring Disorders for Developmental Levels and Competence

Several studies have examined the affect of anxiety or externalizing problems on children’s developmental outcomes (Fergusson & Horwood, 1995; Henricsson & Rydell, 2006; Nelson, Benner, Lane, & Smith, 2004; Rapport, Denney, Chung, & Hustace, 2001). Although some findings were inconsistent, generally, children’s externalizing problems were found to be associated with under-achievement, relatively low intelligence, and poor social competence (Hinshaw, 1992; Nelson, Benner, Neill, & Stage, 2006). Other researchers have suggested that manifestation of externalizing behaviors may be normative or even adaptive, such as in urban youth who routinely experience violence in their communities (Richters & Cicchetti, 1993). However, the implications of pure and co-occurring externalizing problems for children’s developmental outcomes have not been sufficiently examined.

Furthermore, studies have shown that children experiencing internalizing problems, including those with anxiety disorders, are more likely to exhibit compromised academic achievement and cognitive abilities (Ollendick, Weist, Borden, & Greene, 1992; Rapport et al., 2001). A few studies that examined the consequences of comorbidity of anxiety disorders and externalizing problems reported that children with both anxiety and externalizing problems have worse outcomes than children with only anxiety or externalizing problems (Kusche, Cook, & Greenberg, 1993), and that the consequences of comorbidity were greater for adolescents with anxiety disorders than adolescents experiencing comorbidity without anxiety disorders (Lewinsohn, Rohde, & Seeley, 1995).

On the basis of the existing literature, we pursued two central goals in our examination of inner-city children and adolescents with anxiety, externalizing, and co-occurring problems. First, we sought to understand the child demographic and maternal characteristics of the four diagnostic groups (i.e., no problem, anxiety only, externalizing only, and co-occurrence). Second, we sought to determine how the overall level of children’s development differed across the four diagnostic groups by using three indices (academic achievement, intelligence, and adaptive skills). Children’s level of academic achievement, measured by a standardized test, was used as an indicator of adaptation on the basis of the long-term implications of schooling on an individual’s well-being. Children’s intelligence levels, also measured by a standardized test, were included as indicators of adaptive functioning on the basis of long-standing arguments that intelligence is a central component of an individual’s overall capacity to effectively navigate his or her environment (Sternberg, 1997; Wechsler, 1997; Zigler & Phillips, 1960). Finally, mothers’ ratings of their children’s adaptive behaviors were included in the analysis to assess the extent to which children exhibited positive and desirable behaviors in their everyday lives.

Method

Sample

Analyses for this study were conducted using an existing data set collected between 1997 and 2001. These data were obtained from the first wave of a longitudinal study on maternal substance abuse, psychiatric disorders, and poverty (Luthar & Sexton, 2007). The sample consisted of 360 mother–child dyads from a low-income urban community in the Northeast. Approximately three quarters of the sampled mothers met diagnostic criteria for at least one psychiatric disorder during their children’s lifetimes (details follow). Because this study sought to investigate the characteristics of children with no problems, anxiety only, externalizing only, and co-occurring anxiety and externalizing disorders, 18 mother–child dyads were excluded because the child met the criteria for affective disorders and two mother–child dyads were excluded because of missing information. As a result, the final analytic sample was 340 mother–child dyads.

Demographic characteristics of the sample are presented in Table 1. Approximately, 46% of the children were boys, and 70% were ethnic minorities. The mean age of the children was 12.01 years. We established four categories on the basis of the children’s diagnoses: 49% of the children did not have any diagnoses (no-problem group), 31% had only anxiety (anxiety-only group), 10% had only externalizing (externalizing-only group), and 11% had both anxiety and externalizing disorders (co-occurrence group). On average, mothers had attained 12.53 years of education. Approximately 47% of the mothers had affective and anxiety disorders, 59% had antisocial personality disorder, and 45% had alcohol or substance abuse problem during the child’s life. In addition, 23% of the mothers did not meet diagnostic criteria for any psychiatric diagnosis.

Table 1.

Descriptive Characteristics of Sample Children and Their Mothers (N = 340)

Variable Statistic
Demographic characteristics
 Mean (SD) child age (years) 12.01 (2.77)
 Male, % (n) 46.33 (158)
 Ethnic minority status, % (n) 70.09 (239)
 Mean (SD) mother’s education (years) 12.53 (2.50)
Maternal characteristics, % (n)
 Diagnosed with affective/anxiety disorder 47.35(161)
 Diagnosed with antisocial personality disorder 58.82 (200)
 Alcohol and drug abuse/dependence 45.29 (154)
 No diagnosis 23.46 (80)
Groupings of child behavioral problem, % (n)
 No behavioral problems 48.68 (165)
 Anxiety problems only 30.50 (104)
 Externalizing behavioral problems only 9.68 (33)
 Co-occurrence of anxiety and externalizing problems 11.14(38)
Child outcomes
 Mean (SD) child intelligence (K-BIT) 92.89(14.84)
Mean (SD) child academic achievement 45.43 (9.56)
Mean (SD) child adaptive skills (BASC–PRS) 46.78(11.72)
Open in a new tab

Note. K-BIT = Kaufman Brief Intelligence Test; WRAT-3 = Wide Range Achievement Test 3; BASC–PRS = Behavioral Assessment System for Children and Adolescents—Parent Rating Scale.

Measures

The presence of child psychiatric disorders was assessed using the interviewer-administered computerized version of the Diagnostic Interview Schedule, Child–Parent version (Erdman et al., 1992) and Child version (Robins, Cottler, Bucholz, & Compton, 1995). Child’s lifetime occurrence of DSM–IV symptoms including those associated with internalizing (e.g., anxiety and affective disorders) and externalizing disorders (e.g., conduct, oppositional-defiant, and attention deficit and hyperactivity disorders) were assessed. Anxiety or externalizing disorders were considered present if the child’s or mother’s report of the child’s symptoms met the lifetime criteria for one or more psychiatric disorders.

The adult version of the computerized Diagnostic Interview Schedule was also used to document the presence of lifetime maternal psychiatric disorders (Robins et al., 1995). Disorders were considered to be present if they occurred during the lifetime of the child involved in the study. This study included assessing the presence of three distinct types of broadly defined disorders: affective/anxiety disorder, antisocial personality disorder, and substance use disorder.

Child intelligence

Child intelligence was measured using the Kaufman Brief Intelligence Test (K-BIT; Kaufman & Kaufman, 1990). K-BIT is a widely used standardized measure of intelligence that consists of Vocabulary and Matrices subtests (Kaufman & Kaufman, 1990). Composite total IQ and subtest scores were summed and converted into age-appropriate T scores as per the manual. The psychometric properties of K-BIT are well established (Kaufman & Kaufman, 1990; Miller, 1995; Young, 1995). The reliability coefficients for the composite total IQ ranged from .88 to .98 (Kaufman & Kaufman, 1990); it was .96 in this study.

Child academic achievement

Assessments of child academic achievement were conducted using the Wide Range Achievement Test 3 (WRAT-3; Wilkinson, 1993). WRAT-3 is a standardized measure that has been widely used to assess individual’s level of achievement. WRAT-3 consists of three subtests (i.e., Arithmetic, Reading, and Spelling). T scores of the composite Achievement scale and the three subscales were used in the analysis. Evidence to support content, construct, and concurrent validity has been provided (Smith & Smith, 1998; Wilkinson, 1993). In addition, reliability coefficients for the composite achievement score have been reported to range from .92 to .95 (Wilkinson, 1993); it was .97 in this sample.

Adaptive skills

Children’s levels of adaptive skills were assessed using the Adaptive Skills subscale of the Behavioral Assessment System for Children and Adolescents—Parent Rating Scale (BASC–PRS; Reynolds & Kamphaus, 1992). This instrument includes three subscales (i.e., Adaptability, Leadership, and Social Skills) for the Adaptive Skills scale in the child version and two subscales (i.e., Leadership and Social Skills) for the adolescent version. Each subscale was summed and converted into age-appropriate T scores. The T scores of each subscale were summed and converted into composite adaptive skills T scores. The scores from the composite adaptive skills and subscales (i.e., Leadership and Social Skills) were used in the analysis. The reliability and validity of the BASC–PRS are well established (Flanagan, 1995; Gladman & Lancaster, 2003; Kamphaus & Frick, 1996). For this sample, Cronbach’s alpha for the composite adaptive skills scale was .85.

Procedure

The sample of mother–child dyads was obtained using a convenience sampling strategy. The sample was recruited via targeted announcements posted in outpatient treatment facilities (e.g., drug abuse treatment programs and mental health clinics) and community settings (e.g., social service offices and primary care clinics) in low-income neighborhoods. Mothers who contacted the researcher expressing interest in study participation were interviewed to ensure that both the mother and her child met the eligibility criteria; mothers were excluded if they were on antipsychotic medications or had a history of psychosis or bipolar affective disorder. In addition, the mothers’ screening included ensuring that the mother was the biological parent and legal guardian of the study child and currently lived with that child. Child participants were indirectly recruited through mothers who expressed interest in participation. After establishing eligibility, the interviewers obtained written informed consent from the mothers and oral assent from the children for study participation.

Mothers and children participated in separate in-person interviews that were 60 to 90 min in length. All interviews were conducted by highly trained interviewers with at least a bachelor’s degree in psychology or social work. Each of the mothers and each of the children received $40 as compensation for their time in completing the interview; in addition, mothers received an additional $20 if their child completed the child interview. The research protocol was approved by Yale University School of Medicine’s Institutional Review Board.

Results

As shown in Table 1, more than half of the children in the study sample experienced anxiety, externalizing problems, or co-occurrence of these disorders. The prevalence of specific narrowband diagnoses within the four diagnoses groups is presented in Table 2. The percentage of children experiencing specific disorders was not exclusive because children with co-occurring homotypic disorders were considered to be in the same diagnosis group. Approximately 65% of the children in the anxiety-only group and 53% of the children in the co-occurrence group experienced specific phobias,1 which was the most common anxiety disorder in this sample. An interesting finding was that higher rates of prevalence were found for panic disorder (8%) in the co-occurrence group compared with those in the anxiety-only group (4%). However, major discrepancies were found in prevalence rates of narrowband externalizing disorders experienced by the externalizing-only and co-occurrence groups. Conduct disorder was the most prevalent disorder experienced in the externalizing-only group (59%), whereas oppositional-defiant disorder (58%) was the most prevalent narrowband externalizing disorder in the co-occurrence group.

Table 2.

Types of DSM–IV Disorders Associated With Each Behavioral Problem Group (N = 340)

Variable Anxiety problems only (n = 104) Externalizing problems only (n = 31) Co-occurrence (n = 38)
Disorder, % (n)
 Social phobia 20.19 (29) n/a 21.05 (8)
 Separation anxiety 27.88 (29) n/a 26.32 (10)
 Specific phobia 65.38 (68) n/a 52.63 (20)
 Panic disorder 3.85 (4) n/a 7.89 (3)
 Agoraphobia 29.81 (31) n/a 31.58 (12)
 Generalized anxiety disorder 5.77 (6) n/a 5.26 (2)
 Posttraumatic stress disorder 4.81 (5) n/a 2.63 (1)
 Attention deficit/hyperactivity disorder n/a 43.75 (14) 52.63 (20)
 Oppositional-defiant disorder n/a 31.25 (10) 57.89 (22)
 Conduct disorder n/a 59.38 (19) 28.95 (11)
Disorders diagnosed, % (n)
 1 59.62 (62) 59.38 (19)
 2 28.85 (30) 28.13 (9) 42.11 (16)
 3 6.73 (7) 12.50 (4) 39.47 (15)
 4 3.85 (4) n/a 7.89 (3)
 5 0.96 (1) n/a 10.53 (4)
Average number of disorders diagnosed 1.58 1.53 2.87
Open in a new tab

Note. There were 165 children who did not have any problems. The frequencies are in parentheses, n/a = not applicable, by definition excluded from the group.

The average number of diagnoses for each of the child diagnostic groups is also presented in Table 2. The Kruskal–Wallis non-parametric test showed that the co-occurrence group had a significantly greater number of diagnoses than the no-problem, anxiety-only, and externalizing-only groups (χ2 = 314.83, p < .001), suggesting that children in the co-occurrence group were experiencing greater number of psychiatric problems across the broadband spectrum than children in other groups.

Factors Associated With Co-Occurrence of Behavioral Problems

We used a multinomial logistic regression technique to examine risk factors associated with each of the four child diagnostic groups (no problem, anxiety only, externalizing only, and co-occurrence); results are presented in Table 3. Because the co-occurrence group was assigned as the reference group, the effects of predictor variables reflect the odds of a child being in each of the other groups relative to the odds of being in the co-occurring diagnoses group.

Table 3.

Multinomial Logistic Regression Results for Children’s Diagnostic Group (N = 340)

Variable No problem (n = 165)
Anxiety problems only (n = 104)
Externalizing behavioral problems only (n = 31)
B SE OR B SE OR B SE OR
Intercept 3.80*** 1.38 4.55*** 1.47 −0.19 1.87
Child is minority 0.04 0.40 1.04 0.33 0.43 1.38 −0.62 0.52 0.54
Child gender (1 = male) 10.19 0.38 1.21 −0.48 0.40 0.62 0.43 0.51 1.54
Child age 0.04 0.07 1.04 −0.06 0.07 0.94 0.20** 0.09 1.22
Maternal education −0.14* 0.07 0.87 −0.14* 0.08 0.87 −0.12 0.09 0.89
Maternal affective/anxiety disorder −1.15*** 0.42 0.32 −1.02** 0.44 0.36 −1.61*** 0.55 0.20
Maternal antisocial personality disorder −0.71 0.46 0.49 −1.19** 0.48 0.30 −0.17 0.62 0.85
Maternal drug/alcohol use −0.00 0.41 1.00 0.79* 0.44 2.19 0.34 0.56 1.41
Open in a new tab

Note. Reference group is the co-occurrence group (n = 38).

*

p<.10.

**

p<.05.

***

p<.01.

Results of the regression analysis showed that ethnic minority and male status were not associated with group assignments. Older children were more likely to be in the externalizing-only group compared with the co-occurrence group (OR = 1.22, p < .05). Findings on maternal psychopathology showed that children whose mothers had affective/anxiety disorders were much more likely to be in the co-occurrence group than any of the other diagnostic groups. For example, the odds ratios for being in the no-problem, anxiety-only, and externalizing-only groups, relative to the co-occurrence group, were 0.32, 0.36, and 0.20, respectively (p < .05). In addition, children whose mothers met diagnostic criteria for antisocial personality disorder were less likely to be in the anxiety-only group compared with the co-occurrence group (OR = 0.30, p < .05). Although marginally significant, children whose mothers had substance abuse disorders were much more likely to be in the anxiety-only group compared with the co-occurrence group (OR = 2.19, p = .07).

Differences in Adaptive Functioning

To examine the implications of having anxiety, externalizing, or co-occurring disorders on children’s adaptive developmental levels, we conducted a multivariate analysis of covariance with three outcome variables (i.e., academic achievement, intelligence, and adaptive skills) controlling for child’s age, gender, ethnicity, and maternal education. Before conducting the analyses, we tested the outcome variables for normality and homogeneity of variance and covariance. The adaptive skills variable was root-square transformed to meet these assumptions. The results are presented in Table 4.

Table 4.

Group Effects on Children’s Achievement, Intelligence, and Adaptive Skills Controlling for Child Age, Gender, Minority Status, and Maternal Education

Scale Group 1 (n = 165)
Group 2 (n = 104)
Group 3 (n = 31)
Group 4 (n = 38)
F(3, 329)
M SD M SD M SD M SD
Pillai’s Trace MANOVA, F(9, 987) = 2.62***
Achievement 46.86 9.44 44.54 9.28 45.00 8.91 42.96 9.81 4.21*** 1 > 4**
Intelligence 94.57 14.43 91.12 14.29 96.06 12.15 88.11 18.66 5.39*** 1 > 4*** 3 > 4**
Adaptive skills (root square) 6.86 0.92 6.80 0.88 6.65 0.74 6.53 0.75 1.93 ns
Open in a new tab

Note. Group 1 = No-problem group; Group 2 = Anxiety-only group; Group 3 = Externalizing-only group; Group 4 = Co-occurrence group.

*

p < .10.

**

p < .05.

***

p < .01.

The multivariate test showed significant overall group differences. The estimated value for Pillai’s Trace, which is known to be the most robust multivariate test, was 0.07, F(9, 987) = 2.62, p < .01. Follow-up analyses of variance (ANOVAs) revealed significant differences among the groups for achievement, F(3, 329) = 4.21, p < .01, and intelligence, F(3, 329) = 5.39, p < .05. Subsequent post hoc comparisons of means, with adjustment for covariates, showed that children with co-occurring diagnoses had significantly lower scores on achievement and intelligence compared with the no-problem group.

In a final set of analyses, we used ANOVA to examine how children in the four diagnostic groups differed in relation to the subscales of academic achievement, intelligence, and adaptive skills. Similar processes were used to test whether the outcome variables met the required assumptions for these statistical analyses. All subscales were found to meet the requirements. As shown in Table 5, we found significant differences among the groups for all subscales of achievement, which included the Reading, F(3, 329) = 2.83, p < .05, Spelling, F(3, 329) = 2.84, p < .05, and Arithmetic, F(3, 329) = 5.04, p < .01, subscales. For intelligence, mean scores for both the Vocabulary and Matrices subscales showed statistically significant differences across groups, F(3, 329) = 3.60, p < .05; F(3, 329) = 5.13, p < .01, respectively. No significant group differences were found for the Adaptive Skills subscales.

Table 5.

Univariate Analysis Results by Different Diagnostic Groups for Achievement, Intelligence, and Adaptive Skills Subscales

Scale Group 1 (n = 165)
Group 2 (n = 104)
Group 3 (n = 31)
Group 4 (n = 38)
F(3, 329)
M SD M SD M SD M SD
Achievement
 Reading 47.81 10.61 45.88 10.65 45.26 10.76 44.03 12.23 2.83** ns
 Spelling 47.92 11.09 45.62 10.20 45.29 10.13 44.68 10.65 2.84** ns
 Arithmetic 44.84 9.59 42.12 9.97 44.45 9.025 40.16 10.36 5.04*** 1 > 4***
Intelligence
 Vocabulary 91.87 15.12 87.46 15.44 92.94 12.70 87.84 18.78 3.60** 1 > 2**
 Matrices 98.28 15.06 96.40 14.03 100.16 12.39 90.60 17.76 5.13*** 1 > 4***
3 > 4***
Adaptive Skills
 Social Skills 47.92 12.86 48.26 12.05 44.45 10.44 44.05 10.58 1.552 ns
 Leadership 48.70 11.50 48.02 11.28 47.29 9.45 45.35 9.87 1.123 ns
Open in a new tab

Note. The analysis also controls for child age, gender, minority status, and maternal education. Group 1 = No-problem group; Group 2 = Anxiety-only group; Group 3 = Externalizing-only group; Group 4 = Co-occurrence group; ns = not statistically significant at .10 level.

*

p <.10.

**

p <.05.

***

p <.01.

Subsequent post hoc comparisons of means adjusting for covariates showed that, when compared with the no-problem group, children in the co-occurrence group had lower scores on the Arithmetic subscale for academic achievement and Matrices subscale for intelligence at a statistically significant level. In addition, children in the co-occurrence group had lower scores than the externalizing-only group on the Matrices subscale for intelligence at a statistically significant level. On the other hand, children in the anxiety-only group had lower scores on the Vocabulary subscale for intelligence than the no-problem group at a statistically significant level.

Discussion

In this study of 340 inner-city youth, we sought to examine characteristics of four groups based on children’s lifetime psychiatric diagnoses: co-occurring anxiety and externalizing disorders, anxiety only, externalizing only, and no diagnoses. We found that among the study sample, the co-occurrence group experienced worse outcomes compared with children in the other three groups. When compared with the no-diagnosis group, children from the co-occurrence group had significantly lower scores on academic achievement and intelligence. Of the various maternal and child characteristics examined as potential vulnerability factors, maternal depressive and anxiety disorders were significantly associated with children’s co-occurring anxiety and externalizing problems. Major findings are discussed in turn.

Co-Occurring Anxiety and Externalizing Disorders: Distinguishing Features

On the basis of the analytic results, children with co-occurring disorders were found to have lower levels of achievement and intelligence than those in other diagnostic groups. Subsequent analyses of subscales revealed that these children in the co-occurrence group had significantly lower scores on Arithmetic (academic achievement) and Matrices (intelligence) subscales than those in the no-diagnosis group and significantly lower scores on Matrices than those in the externalizing-only group. Both the Matrices and Arithmetic subscales assess children’s nonverbal reasoning and flexibility in solving novel problems (Kaufman & Kaufman, 1990; Wilkinson, 1993). Although further analyses are needed, our findings suggest that children with co-occurring anxiety and externalizing problems may have difficulties with reasoning and problem solving, which are critical domains for coping with challenges and stressors in everyday life.

In considering the association among maternal characteristics and children’s diagnostic groups, we found that children in the co-occurrence group were more likely to have mothers with histories of depressive symptoms or anxiety disorders during the children’s lifetime. Therefore, we conducted a sensitivity analysis to examine whether the association of mothers’ depressive and anxiety disorders and children’s co-occurring problems remained when we used information from only the child’s report of psychiatric disorders. The significant association between the two variables in the multinomial logistic regression was no longer statistically significant (results not shown). However, both multivariate and univariate analyses on children’s developmental competence yielded similar results (results not shown). The insignificant finding in the multinomial logistic regression sensitivity test provides some evidence that mothers with affective disorders are more likely to overreport their children’s behavior and emotional problems (see De Los Reyes & Kazdin, 2005, for a review). However, the insignificant finding can also be attributed to the significant reduction in the co-occurrence group size when data were limited to child report only (i.e., n = 38 for mother and child report and n = 10 for child report only). Considering that most variables found to be significant in the multinomial logistic regression analytic results (see Table 3) were also found to be insignificant in the sensitivity analysis, it may be possible that this reduced group size and the resulting lack of statistical power in the sensitivity test are responsible for the insignificant association of mother’s affective disorder and child’s report of child’s co-occurring psychiatric disorders. Further studies that examine this association using data obtained from clinical diagnoses rather than self-reports are needed.

“Pure” Child Externalizing or Anxiety Pathology

Children in the externalizing-only group were, on average, older than children in the co-occurrence group. This age trend may account for the fact that a larger proportion of children with conduct disorders were found in the “pure,” that is, the sole diagnosis, externalizing-only group, whereas a larger proportion of children with oppositional-defiant disorders was found in the co-occurrence group. In addition, developmental competence of children in the externalizing-only group was not significantly different from that of children in the no-diagnosis group. Considering that the overall intelligence score for children in the externalizing-only group was higher than that of children in the co-occurrence group, it is possible that the level of intelligence, particularly as measured by the Matrices subscale, for children in the externalizing-only group may be comparable with that of children in the no-diagnosis group. However, further investigation is needed to better understand the link between intelligence, occurrence of externalizing and anxiety disorders, and future achievements.

When the anxiety-only group was compared with the co-occurrence group, we found that children in the anxiety-only group were less likely to have mothers with antisocial personality disorder. In addition, they were more likely to have mothers with substance abuse problems than the co-occurrence group at a statistically marginal level of significance. Significant differences in the prevalence of mothers with antisocial personality disorder among the anxiety-only and co-occurrence groups suggest that children whose mothers have antisocial personality disorder may be more vulnerable to externalizing problems than children whose mothers do not have antisocial personality disorder. This finding is consistent with previous studies that reported children of parents with antisocial personality disorder are more vulnerable to conduct disturbances via genetic predispositions as well as parenting behavior pathways (Jaffee et al., 2005; Johnson, Cohen, Kasen, Smailes, & Brook, 2001). Although marginally significant, our findings on maternal substance abuse are somewhat perplexing. Similar to that of antisocial personality disorder, maternal substance abuse can also be considered as an externalizing problem (Substance Abuse & Mental Health Services Administration, 2004). Consequently, we expected our analysis to yield similar findings for maternal substance use to those of maternal antisocial personality disorder. However, we found that children of mothers with histories of substance abuse were more likely to be in the anxiety-only group than the co-occurrence group. Further research is needed to illuminate these contradictory patterns.

In terms of overall developmental levels, the anxiety-only group did not show statistically significant differences on overall scores of achievement, intelligence, and adaptive skills when compared with the no-problem or the co-occurrence groups. This finding suggests that although these groups were not substantially worse off than children with no diagnoses, children with a sole diagnosis for anxiety disorders were also not substantially better off than those in the co-occurrence group. Failure to detect significant differences among these groups may partially be due to the uneven group sizes. Thus, future studies are warranted to compare the developmental levels of these diagnostic groups using larger samples that are more evenly distributed among groups.

Clinical Implications

The study findings suggest that children with co-occurring anxiety and externalizing disorders experience greater disadvantage not only in academic achievement but also in their ability to effectively navigate their environment (i.e., intelligence) than other children. In addition, a moderating effect of anxiety on the severity of externalizing disorders has been reported (Russo & Beidel, 1994). These findings suggest that children with co-occurring anxiety and externalizing disorder are at a greater risk for worse outcomes in managing their disorders, as well as greater risk for school failure and developmental adaptation. Although further investigation is needed to better understand the causes and sequelae of co-occurring anxiety and externalizing disorders, findings from this study suggest that practitioners should carefully assess occurrence of both anxiety and externalizing disorders when providing services and should consider tailoring interventions for children with co-occurring disorders as well as for those with a sole diagnosis of anxiety or externalizing disorders.

Study Limitations

A major limitation of this study was posed by the study design. It is likely that the use of a convenience sampling strategy to recruit mothers with psychopathology or substance abuse resulted in a biased sample. For example, participants were recruited through targeted announcements posted in outpatient treatment facilities and community settings in low-income neighborhoods. Researchers on comorbidity have raised concerns that samples of children receiving clinical services are more likely than community samples to be subjected to referral and Berkson’s (1946) biases,2 which could not only inflate the prevalence of co-occurring disorders in these studies but also potentially bias findings on risk factors and ramifications (Angold et al., 1999). Given the limitations of the study design, we were not able to control of potential biases that may have been caused by posting notices in the clinic. In addition, other potential sampling biases including mother’s level of literacy skills and daily functioning may have also potentially biased our sample selection, which may also limit the interpretation and generalizability of our study findings.

Another major limitation of this study was posed by the cross-sectional nature of the data. Although we identified some of the characteristics associated with the different diagnostic groups, the causal ordering of these associations remains unclear. Thus, children with poor problem-solving skills could be more prone to co-occurring psychiatric disorders. However, it is also plausible that children with multiple, qualitatively diverse psychiatric difficulties will falter in their cognitive functioning and school achievement; however, these multiple challenges may not account for the findings of greater differences on the nonverbal, reasoning tests than the verbal-based tests.

In addition, the uneven group sizes in children’s diagnostic groups posed several challenges to the study. First, given the small cell sizes in the externalizing-only and co-occurrence groups, we were restricted in the number of variables that we could include in our analyses. As a result, the possibility of omitted-variable biases exists for both analyses.

Second, we needed to incorporate both the mother reports and child reports of children’s lifetime psychiatric disorders to obtain a sufficient number of children for each cell. However, it is possible that mothers with affective and anxiety disorders may be more likely to negatively report their child’s psychopathology, which may have skewed our findings regarding the association between mother’s affective and anxiety disorders and children’s diagnostic groups. Although we have conducted supplementary analyses to better understand this issue, we were not able to satisfactorily examine whether and in what ways our study findings were affected by the response biases of mothers with affective and anxiety disorders.

Third, disproportionate sizes in heterotypic diagnostic groups as well as subcategories within anxiety and externalizing disorders may potentially limit the interpretation of our findings. For example, whether the limited findings in the analyses among children in the externalizing-only group compared with the no-problem or co-occurrence group were due to small group size or due to actual group characteristics remains unclear. Thus, studies that examine this issue with more comparable diagnostic group sizes are needed to better understand the characteristics of children in the externalizing-only group.

Given the inconsistencies found in the literature on the types of anxiety disorders included in comorbidity studies (Costello et al., 2005) and the high homotypic comorbity rates between specific phobia and other anxiety disorders found in our sample, we included all types of anxiety disorders in the analyses. However, a large proportion of children who were experiencing anxiety, either in the anxiety-only or co-occurrence groups, were reported to have lifetime experiences of a specific phobia. When supplementary analyses were conducted to determine whether these findings remained significant when children with a specific phobia were excluded from the analyses, findings for both multivariate and univariate analyses were reduced to the level of marginal significance or nonsignificance, even though the estimated marginal means for each of the diagnostic groups followed similar trends (results not shown). This finding suggests that the children diagnosed with a specific phobia, both with or without other anxiety disorders, may have driven some of the significant findings in the present study. It is also possible that increases in the standard error that may have resulted from reduced group sizes could partially explain this lack of findings.

In addition, it is unclear from the analytic findings whether the differences observed in adaptive outcomes between groups are based on the greater number of co-occurring psychiatric problems or whether they are based on co-occurrence of anxiety and externalizing disorders. The limited cell sizes in the co-occurrence group made it difficult to fully explore this issue. However, we found similar results when we conducted a supplementary analysis in which we limited our analysis to children with two diagnoses (results not shown).

Disproportionate cell sizes of some predictor variables also may have caused some difficulty in adequately interpreting the findings. For example, 70% of the sample children had racial or ethnic minority status. Consequently, it is not clear whether the lack of association between minority status and group assignment was due to disproportionate cell sizes or actual lack of association. To further explore this issue, we conducted exploratory analyses in which we selected random subsamples of children from the minority group to match the number of children in the nonminority group; we unable to find any significant associations between minority status and group classifications (results not shown). However, future studies with matching samples for each psychiatric group may better address this issue.

We used the categorical method (rather than the covariation method) for identifying co-occurring externalizing and internalizing disorders, which did not account for children at near-clinical levels of psychiatric disorders. Thus, it is not clear how the results might change when children with near-clinical levels of psychiatric disorders are taken into consideration.

Conclusion

In conclusion, this study suggests that when compared with youth without psychiatric diagnoses, inner-city youth with co-occurring anxiety and externalizing disorders are more likely to have compromised academic achievement and intelligence, particularly in domains reflecting nonverbal reasoning and problem solving. These inner-city youths with co-occurring problems may have substantial differences in the pathways that lead to co-occurring anxiety and externalizing disorders in children and in the consequences of living with such co-occurring conditions. Thus, it is important that this co-occurrence be considered a phenomenon that is distinct from a sole diagnosis when practitioners are examining the causes and sequelae of childhood and adolescent psychiatric disorders.

Acknowledgments

This research was supported in part by Grants RO1-DA10726 and T32 MH019734 from the National Institutes of Health.

Footnotes

1

Among children experiencing specific phobias, 48% of the children in the anxiety-only group and 35% in the co-occurrence group were also experiencing other types of anxiety disorders, suggesting a high prevalence rate of co-occurring homotypic disorders (results not shown).

2

Berkson (1946) showed that clinical samples are more likely to have greater proportions of children with comorbid disorders than the general population because the probability of a child with comorbid disorders being referred to a clinic is associated with the combined referral probabilities of each disorder.

Contributor Information

Joan P. Yoo, School of Social Work, University of North Carolina.

Pamela J. Brown, School of Medicine, Yale University.

Suniya S. Luthar, Department of Counseling and Clinical Psychology, Teachers College, Columbia University.

References

  1. Achenbach TM. “Comorbidity” in child and adolescent psychiatry: Categorical and quantitative perspectives. Journal of Child and Adolescent Psychopharmacology. 1991;1:271–278. [Google Scholar]
  2. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4. Washington, DC: Author; 1994. [Google Scholar]
  3. Angold A, Costello EJ, Erkanli A. Comorbidity. Journal of Child Psychology and Psychiatry. 1999;40:57–87. [PubMed] [Google Scholar]
  4. Beidel DC, Turner SM. At risk for anxiety: I. Psychopathology in the offspring of anxious parents. Journal of the American Academy of Child & Adolescent Psychiatry. 1997;36:918–924. doi: 10.1097/00004583-199707000-00013. [DOI] [PubMed] [Google Scholar]
  5. Berkson J. Limitations of the application of fourfold table analysis to hospital data. Biometrics Bulletin. 1946;2:47–53. [PubMed] [Google Scholar]
  6. Bradley RH, Corwyn RF. Socioeconomic status and child development. Annual Review of Psychology. 2002;53:371–399. doi: 10.1146/annurev.psych.53.100901.135233. [DOI] [PubMed] [Google Scholar]
  7. Brown TA, Barlow DH. Comorbidity among anxiety disorders: Implications for treatment and DSM–IV. Journal of Consulting and Clinical Psychology. 1992;60:835–844. doi: 10.1037//0022-006x.60.6.835. [DOI] [PubMed] [Google Scholar]
  8. Caron C, Rutter M. Comorbidity in child psychopathology: Concepts, issues, and research strategies. Journal of Child Psychology and Psychiatry. 1991;32:1063–1080. doi: 10.1111/j.1469-7610.1991.tb00350.x. [DOI] [PubMed] [Google Scholar]
  9. Costello EJ, Egger HL, Angold A. The developmental epidemiology of anxiety disorders: Phenomenology, prevalence, and comorbidity. Child and Adolescent Psychiatric Clinics of North America. 2005;14:631–648. doi: 10.1016/j.chc.2005.06.003. [DOI] [PubMed] [Google Scholar]
  10. Crick NR, Zahn-Waxler C. The development of psychopathology in females and males: Current progress and future challenges. Development and Psychopathology. 2003;15:719–742. [PubMed] [Google Scholar]
  11. Deater-Deckard K, Dodge KA, Bates JE, Pettit GS. Multiple risk factors in the development of externalizing behavior problems: Group and individual differences. Development and Psychopathology. 1998;10:469–493. doi: 10.1017/s0954579498001709. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. De Los Reyes A, Kazdin AE. Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin. 2005;131:483–509. doi: 10.1037/0033-2909.131.4.483. [DOI] [PubMed] [Google Scholar]
  13. Duncan GJ, Brooks-Gunn J. Consequences of growing up poor. New York: Russell Sage Foundation; 1997. [Google Scholar]
  14. Erdman HP, Klein MH, Greist JH, Skare SS, Husted JJ, Robins LN, Miller JP. A comparison of two computer-administered versions of the NIMH Diagnostic Interview Schedule. Journal of Psychiatric Research. 1992;26:85–95. doi: 10.1016/0022-3956(92)90019-k. [DOI] [PubMed] [Google Scholar]
  15. Fergusson DM, Horwood LJ. Early disruptive behavior, IQ, and later school achievement and delinquent behavior. Journal of Abnormal Child Psychology. 1995;23:183–199. doi: 10.1007/BF01447088. [DOI] [PubMed] [Google Scholar]
  16. Flanagan R. A review of the Behavior Assessment System for Children (BASC): Assessment consistent with the requirements of the Individuals With Disabilities Education Act (IDEA) Journal of School Psychology. 1995;33:177–186. [Google Scholar]
  17. Franco X, Saavedra LM, Silverman WK. External validation of comorbid patterns of anxiety disorders in children and adolescents. Journal of Anxiety Disorders. 2007;21:717–729. doi: 10.1016/j.janxdis.2006.10.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Gladman M, Lancaster S. A review of the Behaviour Assessment System for Children. School Psychology International. 2003;24:276–291. [Google Scholar]
  19. Henricsson L, Rydell AM. Children with behaviour problems: The influence of social competence and social relations on problem stability, school achievement and peer acceptance across the first six years of school. Infant and Child Development. 2006;15:347–366. [Google Scholar]
  20. Hinden BR, Compas BE, Howell DC, Achenbach TM. Covariation of the anxious–depressed syndrome during adolescence: Separating fact from artifact. Journal of Consulting and Clinical Psychology. 1997;65:6–14. doi: 10.1037//0022-006x.65.1.6. [DOI] [PubMed] [Google Scholar]
  21. Hinshaw SP. Externalizing behavior problems and academic underachievement in childhood and adolescence: Causal relationships and underlying mechanisms. Psychological Bulletin. 1992;111:127–155. doi: 10.1037/0033-2909.111.1.127. [DOI] [PubMed] [Google Scholar]
  22. Jaffee SR, Caspi A, Moffitt TE, Dodge KA, Rutter M, Taylor A, Tully LA. Nature X Nurture: Genetic vulnerabilities interact with physical maltreatment to promote conduct problems. Development and Psychopathology. 2005;17:67–85. doi: 10.1017/s0954579405050042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Johnson JG, Cohen P, Kasen S, Smailes E, Brook JS. Association of maladaptive parental behavior with psychiatric disorder among parents and their offspring. Archives of General Psychiatry. 2001;58:453–460. doi: 10.1001/archpsyc.58.5.453. [DOI] [PubMed] [Google Scholar]
  24. Kamphaus RW, Frick PJ. Clinical assessment of child and adolescent personality and behavior. 1. Needham Heights, MA: Allyn & Bacon; 1996. [Google Scholar]
  25. Kaufman AS, Kaufman NL. Kaufman Brief Intelligence Test. Circle Pines, MN: American Guidance Service; 1990. [Google Scholar]
  26. Keiley MK, Lofthouse H, Bates JE, Dodge KA, Pettit GS. Differential risks of covarying and pure components in mother and teacher reports of externalizing and internalizing behavior across ages 5 to 14. Journal of Abnormal Child Psychology. 2003;31:267–283. doi: 10.1023/a:1023277413027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kusche CA, Cook ET, Greenberg MT. Neuropsychological and cognitive functioning in children with anxiety, externalizing, and comorbid psychopathology. Journal of Clinical Child Psychology. 1993;22:172–195. [Google Scholar]
  28. Lewinsohn PM, Gotlib IH, Lewinsohn M, Seeley JR, Allen NB. Gender differences in anxiety disorders and anxiety symptoms in adolescents. Journal of Abnormal Psychology. 1998;107:109–117. doi: 10.1037//0021-843x.107.1.109. [DOI] [PubMed] [Google Scholar]
  29. Lewinsohn PM, Rohde P, Seeley JR. Adolescent psychopathology: III. The clinical consequences of comorbidity. Journal of the American Academy of Child & Adolescent Psychiatry. 1995;34:510–519. doi: 10.1097/00004583-199504000-00018. [DOI] [PubMed] [Google Scholar]
  30. Lilienfeld SO, Waldman ID, Israel AC. A critical examination of the use of the term and concept of comorbidity in psychopathology research. Clinical Psychology: Science and Practice. 1994;1:71–83. [Google Scholar]
  31. Loeber R, Keenan K. Interaction between conduct disorder and its comorbid conditions: Effects of age and gender. Clinical Psychology Review. 1994;14:497–523. [Google Scholar]
  32. Luthar SS, Sexton CC. Maternal drug abuse versus maternal depression: Vulnerability and resilience among school-aged and adolescent offspring. Development and Psychopathology. 2007;19:205–255. doi: 10.1017/S0954579407070113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Miller MD. Review of the Kaufman Brief Intelligence Test. In: Conoley JC, Impara JC, editors. The twelfth mental measurements yearbook. Lincoln: University of Nebraska Press; 1995. pp. 533–534. [Google Scholar]
  34. Nelson JR, Benner GJ, Lane K, Smith BW. Academic achievement of K–12 students with emotional and behavioral disorders. Exceptional Children. 2004;71:59–73. [Google Scholar]
  35. Nelson JR, Benner GJ, Neill S, Stage SA. Interrelationships among language skills, externalizing behavior, and academic fluency and their impact on the academic skills of students with ED. Journal of Emotional and Behavioral Disorders. 2006;14:209–216. [Google Scholar]
  36. Newman DL, Moffitt TE, Caspi A, Silva PA. Comorbid mental disorders: Implications for treatment and sample selection. Journal of Abnormal Psychology. 1998;107:305–311. doi: 10.1037//0021-843x.107.2.305. [DOI] [PubMed] [Google Scholar]
  37. Nottelmann ED, Jensen PS. Comorbidity of disorders in children and adolescents: Developmental perspectives. In: Ollendick TH, Prinz RJ, editors. Advances in clinical child psychology. Vol. 17. New York: Plenum Press; 1995. pp. 109–155. [Google Scholar]
  38. Ollendick TH, Weist MD, Borden MC, Greene RW. Sociometric status and academic, behavioral, and psychological adjustment: A five-year longitudinal study. Journal of Consulting and Clinical Psychology. 1992;60:80–87. doi: 10.1037//0022-006x.60.1.80. [DOI] [PubMed] [Google Scholar]
  39. Pilowsky DJ, Wickramaratne P, Nomura Y, Weissman M. Family discord, parental depression, and psychopathology in offspring: 20-year follow-up. Journal of the American Academy of Child & Adolescent Psychiatry. 2006;45:452–460. doi: 10.1097/01.chi.0000198592.23078.8d. [DOI] [PubMed] [Google Scholar]
  40. Rapport MD, Denney CB, Chung KM, Hustace K. Internalizing behavior problems and scholastic achievement in children: Cognitive and behavioral pathways as mediators of outcome. Journal of Clinical Child Psychology. 2001;30:536–551. doi: 10.1207/S15374424JCCP3004_10. [DOI] [PubMed] [Google Scholar]
  41. Reynolds CR, Kamphaus RW. Behavioral Assessment System for Children (BASC) Circle Pines, MN: American Guidance Service; 1992. [Google Scholar]
  42. Richters JE, Cicchetti D. Mark Twain meets DSM–III–R: Conduct disorder, development, and the concept of harmful dysfunction. Development and Psychopathology. 1993;5:5–29. [Google Scholar]
  43. Robins LN, Cottler L, Bucholz K, Compton W. Diagnostic Interview Schedule for DSM–IV. St. Louis, MO: Washington University Press; 1995. [Google Scholar]
  44. Russo MF, Beidel DC. Comorbidity of childhood anxiety and externalizing disorders: Prevalence, associated characteristics, and validation issues. Clinical Psychology Review. 1994;14:199–221. [Google Scholar]
  45. Smith TD, Smith BL. Relationship between the Wide Range Achievement Test 3 and the Wechsler Individual Achievement Test. Psychological Reports. 1998;83:963–967. doi: 10.2466/pr0.1998.83.3.963. [DOI] [PubMed] [Google Scholar]
  46. Sternberg RJ. The concept of intelligence and its role in lifelong learning and success. American Psychologist. 1997;52:1030–1037. doi: 10.1037//0003-066x.52.10.1134. [DOI] [PubMed] [Google Scholar]
  47. Substance Abuse and Mental Health Services Administration. Results from the 2003 National Household Survey on Drug Use and Health: National findings. Rockville, MD: U.S. Department of Health and Human Services; 2004. [Google Scholar]
  48. Verhulst FC, van der Ende J. “Comorbidity” in an epidemiological sample: A longitudinal perspective. Journal of Child Psychology and Psychiatry. 1993;34:767–783. doi: 10.1111/j.1469-7610.1993.tb01070.x. [DOI] [PubMed] [Google Scholar]
  49. Wechsler D. Wechsler Adult Intelligence Scale. 3. San Antonio, TX: Psychological Corporation; 1997. Technical manual. [Google Scholar]
  50. Wilkinson GA. WRAT3: Wide Range Achievement Test. Wilmington, DE: Wide Range; 1993. [Google Scholar]
  51. Wright JC, Zakriski AL, Drinkwater M. Developmental psychopathology and the reciprocal patterning of behavior and environment: Distinctive situational and behavioral signatures of internalizing, externalizing, and mixed-syndrome children. Journal of Consulting and Clinical Psychology. 1999;67:95–107. doi: 10.1037//0022-006x.67.1.95. [DOI] [PubMed] [Google Scholar]
  52. Young JW. Review of the Kaufman Brief Intelligence Test. In: Conoley JC, Impara JC, editors. The twelfth mental measurements yearbook. Lincoln: University of Nebraska Press; 1995. pp. 534–536. [Google Scholar]
  53. Zigler E, Phillips L. Social effectiveness and symptomatic behaviors. Journal of Abnormal and Social Psychology. 1960;6:231–238. doi: 10.1037/h0041431. [DOI] [PubMed] [Google Scholar]

RESOURCES