Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Feb 10.
Published in final edited form as: J Abnorm Child Psychol. 2011 Jan;39(1):109–123. doi: 10.1007/s10802-010-9444-8

Structure and Etiology of Co-occurring Internalizing and Externalizing Disorders in Adolescents

Victoria E Cosgrove, Soo H Rhee, Heather L Gelhorn, Debra Boeldt, Robin C Corley, Marissa A Ehringer, Susan E Young, John K Hewitt
PMCID: PMC3036783  NIHMSID: NIHMS243833  PMID: 20683651

Abstract

Several studies suggest that a two-factor model positing internalizing and externalizing factors explains the interrelationships among psychiatric disorders. However, it is unclear whether the covariation between internalizing and externalizing disorders is due to common genetic or environmental influences. We examined whether a model positing two latent factors, internalizing and externalizing, explained the interrelationships among six psychiatric disorders (major depressive disorder, generalized anxiety disorder, separation anxiety disorder, attention-deficit/hyperactivity disorder, oppositional defiant disorder, and conduct disorder) in adolescents, and whether there are common genetic and environmental influences on internalizing and externalizing latent factors. Multivariate behavior genetic analyses of data from 1162 twin pairs and 426 siblings ascertained from the general population via the Colorado Center for Antisocial Drug Dependence (CADD) were conducted. We found support for a model positing two latent factors (internalizing and externalizing). These factors were moderately heritable and influenced by significant common genetic and nonshared environmental influences. These findings suggest that co-occurrence of internalizing and externalizing psychopathology in adolescents results from both genetic and environmental influences.

Keywords: adolescence, psychopathology, comorbidity, genetics, environment

Psychiatric comorbidity, or co-occurrence of disorders, is commonly observed in general population samples, and alternative theoretical models explaining the causes of comorbidity have been proposed (Angold, Costello, & Erkanli, 1999). Many behavioral genetic studies have examined the etiology of psychiatric comorbidity in adolescents and adults, as increased knowledge regarding the etiology of comorbidity informs the understanding of the etiology, course, and treatment of psychiatric disorders.

An Internalizing-Externalizing Model for Co-Occurring Disorders

An internalizing-externalizing model has received considerable attention as a potential theoretical framework for understanding co-occurring psychiatric disorders (Krueger, 1999). Internalization is the propensity to express distress inwards; common internalizing disorders include mood disorders (e.g., major depressive disorder, dysthymia) and anxiety disorders (e.g., generalized anxiety disorder, separation anxiety disorder, phobias, obsessive-compulsive disorder). In contrast, externalization describes the propensity to express distress outwards; commonly recognized externalizing disorders include attention-deficit/hyperactivity disorder, oppositional defiant disorder, conduct disorder, antisocial personality disorder, and substance use disorders.

Factor analytic studies of psychiatric disorders have found support for two factors, internalizing and externalizing, underlying common psychopathology in adults (e.g., Krueger, Caspi, Moffitt, & Silva, 1998, Krueger, 1999; Krueger, McGue, & Iacono, 2001). Support for an internalizing-externalizing model for psychopathology has also been observed in adolescent samples. Hewitt et al. (1997) conducted factor analyses on major depressive disorder (MDD), separation anxiety disorder (SAD), overanxious disorder (OAD), oppositional defiant disorder (ODD), conduct disorder (CD), and attention-deficit/hyperactivity disorder (ADHD) in a sample of 8- to 16-year-olds and found evidence supporting the distinction between internalizing and externalizing disorders. Although ADHD symptoms were relatively independent of other domains, there were moderate to high correlations among separation anxiety disorder, overanxious disorder, and major depressive disorder as well as a high correlation between oppositional defiant disorder and conduct disorder.

Although the above-mentioned evidence suggests that internalizing and externalizing psychopathology are distinctive, there is also evidence of co-occurrence between internalizing and externalizing disorders. For example, moderate correlations were observed between internalizing and externalizing latent factors in the factor-analytic studies with adults cited above (e.g., r = .45 in Krueger et al., 1998; r = .51 in Krueger, 1999), and modest correlations between internalizing and externalizing disorders were observed in Hewitt et al.’s (1997) study of adolescents. Another report (Burcusa, Iacono, & McGue, 2003) examined the prevalence of comorbidity between major depression and other DSM-defined, psychiatric disorders in 624 17-year-old twin pairs. Depression co-occurred with other internalizing disorders as well as externalizing disorders with similar frequencies. Finally, a meta-analysis (Angold et al., 1999) examined diagnostic co-occurrence in children and adolescents. While the highest levels of comorbidity were observed between ADHD and CD/ODD and between anxiety and depression, statistically significant comorbidity was also observed between internalizing (anxiety and depression) and externalizing (ADHD and CD/ODD) disorders.

Alternative theoretical models have suggested further division within internalizing disorders. Multiple reports have found evidence for two subfactors within internalizing disorders, Anxious-Misery (e.g., major depressive episode, dysthymia, generalized anxiety disorder) and Fear (e.g., social phobia, simple phobia, agoraphobia, and panic disorder) (Krueger, 1999; Kendler, Prescott, Myers, & Neale, 2003). The tripartite model (Clark &Watson, 1991) proposes that while anxiety and depression share elevated negative affect, depressed individuals exhibit low positive affect and anxious individuals display high physiological hyperarousal. One report (Cole, Truglio, & Peeke, 1997) found evidence for the tripartite model in adolescents (i.e., 6th graders).

Etiology of Internalizing, Externalizing, and Co-occurring Disorders

Behavioral genetic studies estimate the magnitude of genetic and environmental influences on the variance of psychiatric disorders in the population by taking advantage of the fact that monozygotic (MZ) twin pairs share 100% of their genes while dizygotic (DZ) twin pairs and ordinary sibling pairs share 50% of their segregating genes. Multivariate behavioral genetic studies are conducted to examine the etiology of co-occurring psychopathology. We review evidence for the magnitude of genetic and environmental influences on psychopathology in three domains: internalizing disorders, externalizing disorders, and co-occurring internalizing and externalizing disorders.

Internalizing Disorders

A comprehensive review of bivariate behavior genetic studies by Middeldorp, Cath, van Dyck, and Boomsma (2005) compiled the results of 23 twin studies and 12 family studies examining the comorbidity between anxiety and depression. They concluded that overlap between genetic influences on anxiety and depression was greater than overlap between nonshared environmental influences, with little evidence of common shared environmental influences. The correlation between genetic influences on MDD and generalized anxiety disorder (GAD) was high and ranged from .86 to 1.00. Silberg and Bulik (2005) also reported that there is a common genetic factor influencing separation anxiety, overanxious disorder, depression, and eating disorders.

Externalizing Disorders

There have been several studies examining the etiology of comorbidity among externalizing disorders. Young et al. (2000) found evidence of a highly heritable (h2 = .84) behavioral disinhibition common factor that explained the covariance among novelty seeking, substance use, conduct disorder, and ADHD, suggesting that vulnerability to these externalizing disorders may result from a general inability to regulate impulses. Twin studies have supported evidence for genetic influences on the co-occurrence between ADHD and CD (e.g., Nadder et al., 1998; Nadder et al., 2002; Silberg et al., 1996; Thapar et al., 2001), and the co-occurrence among CD, ADHD, and ODD (Waldman et al., 2001; Dick, Viken, Kaprio, Pulkkinen, and Rose, 2005). At least one study found strong evidence of shared environmental influences on the co-occurrrence among the externalizing disorders (Burt, Krueger, McGue, and Iacono, 2001).

Co-occurring Internalizing and Externalizing Disorders

As noted above, there is significant comorbidity between internalizing and externalizing disorders (Harrington, Fudge, Rutter, Pickles, & Hill, 1991; Puig-Antich, 1982; Zoccolillo, 1992). Results of behavior genetic studies examining the etiology of co-occurrence between internalizing and externalizing disorders are inconsistent. Kendler and colleagues (2003) examined the etiology of co-occurrence among multiple, DSM-III and -IV psychiatric disorders (MDD, GAD, phobia, alcohol dependence, drug abuse and dependence, antisocial behavior, and CD) in a genetically informative sample of adults. They concluded that two genetic risk factors predisposed both men and women to two broad groups of internalizing and externalizing disorders. A latent internalizing genetic risk factor loaded on MDD, GAD, and phobia, and a latent externalizing genetic risk factor loaded on alcohol dependence, drug abuse and dependence, adult antisocial behavior, and CD. Interestingly, the internalizing genetic factor did not have significant loadings on any of the externalizing disorders, and the externalizing genetic factor did not have significant loadings on any of the internalizing disorders. Although the distinction between genetic influences on internalizing and externalizing disorders was clear, distinctions between shared and nonshared environmental influences on internalizing vs. externalizing disorders were not found.

Several reports find evidence for genetic and environmental factors influencing the development of co-occurring psychopathology in adolescents. Gjone and Stevenson (1997) examined the etiology of internalizing and externalizing behavior, assessed by the parent-rated Child Behavior Checklist (CBCL; Achenbach, 1991), in children and adolescents. They concluded that there are both common genetic and shared environmental influences on internalizing and externalizing behavior, but that the magnitude of common shared environmental influences was larger for both younger (age 5-9 years) and older (age 12-15 years) children. They also found little evidence of nonshared environmental influences common to internalizing and externalizing behavior.

O’Connor and colleagues (O’Connor, McGuire, Reiss, Hetherington, & Plomin, 1998) examined the covariation between depression and antisocial behavior in adolescents using a composite of parent- and self-report questionnaires and observers’ reports. They concluded that there were significant genetic, shared environmental, and nonshared environmental influences (h2= .45, c2=.30, e2=.25) on the covariation between depression and antisocial behavior. Recently, Subbarao et al. (2008) examined the covariance between DSM-IV MDD and CD in the same sample of adolescents examined in the present study, and found statistically significant genetic and nonshared environmental influences on the co-occurrence between MDD and CD.

The Current Study

In the present study, we examined the co-occurrence among three psychiatric disorders within the internalizing spectrum (MDD, GAD, SAD) and three psychiatric disorders within the externalizing spectrum (ADHD, CD, and ODD) in adolescents. These six psychiatric disorders were chosen because of availability and adequate prevalence in the Colorado Center for Antisocial Drug Dependence (CADD). Data for posttraumatic stress disorder (PTSD) and eating disorders were also available in CADD; however, low prevalence rates and lack of whole life data precluded their inclusion in our analyses.

The present study is a follow-up to Ehringer et al.’s (2006) study, which examined the magnitude of genetic and environmental influences on these six psychiatric disorders separately, and Subbarao et al.’s (2008) study, which examined the covariance between MDD and CD in the same sample. Goals of the present study were threefold. First, we examined the factor structure of adolescent psychiatric disorders. We hypothesize that our data will be well-represented by a two-factor model positing internalizing and externalizing factors. Second, we examined the etiology of these common internalizing and externalizing factors. Given previous reports, we believe that internalizing and externalizing factors are substantially influenced by genetic and nonshared environmental influences. Finally, we examined the magnitude of genetic and nonshared environmental influences on the covariation between the internalizing and externalizing factors in adolescents.

The present study is the first to examine whether an a priori internalizing-externalizing model underlies co-occurrence among multiple DSM-IV psychiatric disorders (and compare this model to alternative hypotheses) in a multivariate behavior genetic study of adolescents. This study makes a unique contribution to the existing literature for several reasons. Its focus is on adolescence, a period of development distinctive from early childhood and adulthood. Understanding patterns of co-occurring disorders during specific developmental periods such as adolescence informs clinical understanding of illness progression. Kendler and colleagues (2003) identified distinctive genetic influences on internalizing and externalizing factors in adults; however, the influence of genes and the environment on psychopathology may differ in adolescents. The expression of psychopathology across development often changes in intensity and nature (Hudziak et al., 2007). For example, Moffitt (1993) noted that antisocial behavior increases almost ten-fold during adolescence, and proposed the distinction between life-course-persistent antisocial behavior, which is more likely to be genetically influenced, and adolescence-limited antisocial behavior, which is hypothesized to be mimicked from antisocial models. Also, Hicks et al. (2007) examined the development of externalizing disorders from late adolescence to early adulthood, and reported an increasing heritability for men but a trend toward decreasing heritability for women. Furthermore, a developmental twin study examining anxiety and depression from childhood to early adulthood found evidence of new genetic risk factors in early adolescence, late adolescence, and early adulthood (Kendler, Gardner, & Lichtenstein, 2008). Given these findings, it is important to consider the possibility that the etiology of the co-occurrence of psychiatric disorders may be unique during adolescence.

Additionally, research examining the etiology of adolescent psychopathology may inform the development of prevention efforts. Increased understanding of the etiology of adult psychopathology has led to the development of effective, evidence-based pharmacological and psychological interventions designed to treat already-developed symptoms. Comprehensive understanding of co-occurring adolescent psychopathology may inform prevention enterprises designed to stave off initial onset of symptoms.

Finally, previous studies have utilized behavioral phenotypes such as CBCL factor scores rather than DSM-IV psychiatric symptoms and diagnoses. Examining DSM-defined psychopathology in adolescents is important because the DSM is the standard for identification and treatment of mental illness across clinical settings and age groups. This standardization allows for comparisons between adults and adolescents, possibly exposing unique experiences of internalizing and externalizing psychopathology during adolescence.

Method

Participants

The Center for Antisocial Drug Dependence (CADD) is an ongoing, multi-component, collaborative study underway at the Institute for Behavioral Genetics (IBG) at the University of Colorado. In the present study, 2750 individual adolescents from the Center for Antisocial Drug Dependence (CADD) were assessed. The participants were 1162 twin pairs (570 MZ pairs, 370 same sex DZ pairs, and 222 opposite sex DZ pairs) and 426 non-twin siblings of twins (one sibling nearest in age to the twins per family) from two community-based samples: the Colorado Longitudinal Twin Sample (LTS) and Community Twin Sample (CTS). Detailed information regarding recruitment and sample description of both samples can be found in Rhea et al. (2006). In brief, LTS and CTS are two samples recruited into the CADD from the Colorado Twin Registry (CTR), a population-based registry housed at IBG at the University of Colorado. The CTR is comprised of twins born in Colorado between 1968 and 2004 recruited with help from the Division of Vital Statistics at the Colorado Department of Health. The LTS is a sample of twins whose development has been studied since birth, and the CTS is a sample of twins recruited for the first time by the CADD. Males (48%) and females (52%) were approximately equally represented among the sample of adolescents aged 12 to 18 years (mean 14.84 ± 2.08). The ethnic composition of the sample consisted of 85% White, 2% African-American, 8% Hispanic, 3% Asian, 2% Native American, and 1% unknown, as reported by the subjects on a questionnaire.

Procedure

Zygosity of same-sex twin pairs was measured in two ways in order to ensure accuracy (Hannelius et al., 2007). First, a nine-item assessment of physical characteristics was completed by interviewers and compared to the concordance of the twin pairs’ genotype at 11 highly informative short tandem repeat polymorphisms. Twin pairs with similar physical characteristics and concordant genotypic markers were classified as MZ pairs. Those with disparate physical features and genotypes were classified as DZ pairs. Among these, only nine had discordant 9-item assessments and DNA data. These discordances were re-examined and resolved.

In the present study, three internalizing spectrum disorders, MDD, GAD, and SAD, and three externalizing spectrum disorders, ADHD, CD, and ODD, were assessed via adolescent interviews using the Diagnostic Interview Schedule for Children – IV (DISC-IV; Shaffer, Fisher, Lucas, Dulcan, & Schwab-Stone, 2000). The DISC has shown moderate to good validity (Schwab-Stone, Shaffer, Dulcan et al., 1996) and reliability (Shaffer, Fisher, Dulcan et al., 1996; Shaffer, Fisher, Lucas et al.., 2000) across multiple diagnoses. The DISC-IV is a structured psychiatric interview assessing DSM-IV (American Psychiatric Association, 1994) symptoms and diagnoses for Axis I disorders and includes questions about symptoms of psychiatric disorders over the past year as well as over the lifetime. Ehringer et al. (2006) noted that the prevalence rates for ADHD, CD, ODD, GAD, SAD, and MDD in the present sample are comparable to those reported in the Methods for the Epidemiology of Child and Adolescent Mental Disorders Study (Schaffer et al., 1996). Computer algorithms were developed to determine the presence or absence of symptoms and diagnoses for each disorder in accordance with instructions from the instrument’s authors. Individuals were categorized as having no symptoms, one or more symptoms, or a diagnosis of a disorder during the lifetime.

Analyses

The twin-sibling design allows us to decompose phenotypic variance into genetic and environmental components. With this design, genetic influence (A) is estimated by comparing the extent to which MZ twins are more similar than DZ twins or regular non-twin siblings. Shared environmental influences (C) comprise the excess of observed twin resemblance after accounting for genetic influences, and represent non-genetic influences that contribute to similarity among relatives. Non-shared environmental influences (E) are environmental influences that lead to differences between sibling pairs, and also includes measurement error (Plomin, DeFries, & McClearn, 1990). Including non-twin siblings in the analyses is beneficial in two ways. First, adding non-twin siblings increases power to detect genetic and shared environmental influences (Posthuma & Boomsma, 2000). Second, the inclusion of non-twin siblings enables us to assess the magnitude of twin-specific shared environmental influences (T), or environmental influences specific to twin pairs that lead them to be more similar than non-twin siblings (e.g., environmental influences that siblings of the same age are more likely to share, such as having more friends in common).

A thorough examination of psychopathology during adolescence should account for the fact that the prevalence of many disorders varies by age and sex (see Table 1). Thresholds accounting for age and sex trends were estimated for each disorder for every participant. Then, these age- and sex-specific thresholds were used as definition variables in all analyses; they allow us to account for the fact that not all respondents have passed through the age of risk and that prevalence levels vary by sex.

Table 1.

Percentage of individuals with no symptoms, one or more symptoms, and diagnosis by age and sex.

Age, Gender, and Sample Size
12 13 14 15 16 17 18
M F M F M F M F M F M F M F
Disorder 285 311 123 113 213 217 168 197 170 201 193 186 163 210
MDD No Symptoms 96.8 95.2 95.1 90.3 95.3 91.7 90.5 78.2 92.4 73.6 83.9 79.0 82.8 71.0
≥ 1 Symptoms 2.5 4.5 4.1 4.4 2.8 4.6 6.0 15.2 4.7 12.9 8.8 8.6 8.6 12.4
Diagnosis 0.7 0.3 0.8 5.3 1.9 3.7 3.6 6.6 2.9 13.4 7.3 12.4 8.6 16.7
SAD No Symptoms 81.8 65.3 75.6 65.5 80.8 69.6 82.7 57.4 80.0 62.7 79.8 60.8 76.7 58.6
≥ 1 Symptoms 18.2 31.5 22.8 33.6 17.4 29.5 16.1 38.1 17.6 32.8 19.2 34.9 21.5 36.7
Diagnosis 0.0 3.2 1.6 0.9 1.9 0.9 1.2 4.6 2.4 4.5 1.0 4.3 1.8 4.8
GAD No Symptoms 72.6 60.8 65.0 63.7 64.3 52.1 61.9 51.3 57.1 49.8 60.6 42.5 63.2 47.6
≥ 1 Symptoms 26.3 38.6 34.1 35.4 34.7 46.1 35.7 42.6 41.8 43.8 37.3 52.2 36.2 42.9
Diagnosis 1.1 0.6 0.8 0.9 0.9 1.8 2.4 6.1 1.2 6.5 2.1 5.4 0.6 9.5
ADHD No Symptoms 62.5 67.2 53.7 63.7 60.6 61.3 54.8 58.9 56.5 54.7 44.6 50.0 46.6 48.1
≥ 1 Symptoms 35.4 32.3 41.5 32.7 35.2 36.9 41.1 36.5 36.5 41.8 49.2 44.6 47.9 47.6
Diagnosis 2.1 0.6 4.9 3.5 4.2 1.8 4.2 4.6 7.1 3.5 6.2 5.4 5.5 4.3
CD No Symptoms 69.1 74.6 49.6 65.5 45.1 63.6 36.3 53.3 34.7 51.7 28.5 48.9 25.8 44.3
≥ 1 Symptoms 26.3 24.8 35.8 31.9 44.6 33.6 43.5 42.6 51.2 38.8 44.6 45.2 47.9 43.8
Diagnosis 4.6 0.6 14.6 2.7 10.3 2.8 20.2 4.1 14.1 9.5 26.9 5.9 26.4 11.9
ODD No Symptoms 88.8 90.7 83.7 83.2 82.6 84.8 79.2 82.7 82.4 79.1 79.3 75.3 80.4 71.9
≥ 1 Symptoms 10.9 8.7 13.8 15.9 14.1 12.4 17.9 11.7 14.1 14.9 13.5 18.8 16.6 17.1
Diagnosis 0.4 0.6 2.4 0.9 3.3 2.8 3.0 5.6 3.5 6.0 7.3 5.9 3.1 11.0
Open in a new tab

Raw data were analyzed using the statistical package Mx (Neale, Boker, Xie, & Maes, 2002). Data were analyzed assuming that a normal continuous liability distribution underlies the ordinal variables (i.e., 0 = no symptoms, 1 = one or more symptoms, and 2 = diagnosis), given that DSM-IV symptom counts for psychiatric disorders are highly skewed. This is an optimum approach, because it retains the statistical advantages conferred by the normality assumptions for the underlying liability, retains an explicit mapping between the underlying liability and observed behavior, and correctly recovers the underlying correlations and parameter estimates (Derks et al., 2004; Stallings et al., 2001).

Preliminary Gender and Age Analyses

The present research follows up that of Ehringer et al. (2006), who conducted separate univariate analyses to examine the magnitude of genetic and environmental influences on GAD, SAD, MDD, ADHD, CD, and ODD in the identical CADD sample of 1,162 twin pairs and 426 non-twin siblings of twins. They found evidence for genetic and non-shared environmental influences for all six disorders as well as evidence for shared environmental influences on MDD and GAD. Our analyses build on their univariate models by considering multivariate models examining the interrelationships among the six psychiatric disorders in twins and siblings.

Ehringer et al. (2006) found little evidence of sex differences in the magnitude of genetic and environmental influences in their study. We also conducted preliminary analyses comparing the fit of the model where parameters were free to vary across male and female twin pairs, and the fit of the model where parameters were constrained to be equal across male and female twin pairs. For the Cholesky model (AIC = −8449.55 vs. −8386.90; BIC = −31417.43 vs. −31248.00; sample size adjusted BIC = −13595.71 vs. −13516.79), the bivariate common pathway model (AIC = −8460.18 vs. −8425.80; BIC = −31500.28 vs. −31422.52; sample size adjusted BIC = −13627.74 vs. −13589.68), and the single common pathway model (AIC = −8437.37 vs. −8412.79; BIC = −31500.99 vs. −31440.24; sample size adjusted BIC = −13620.51 vs. −13591.52), the model constraining the parameters across the two sexes had lower model fit indices and better fit than the model where parameters were free to vary across the two sexes. Therefore, data for males and females was combined in the present study.

Similarly, for the Cholesky model (AIC = −10176.55 vs. −10055.70; BIC = −40098.84 vs. −39750.35; sample size adjusted BIC = −18101.17 vs. −17933.73) , the bivariate common pathway model (AIC = −10182.74 vs. −10113.73; BIC = −40182.82 vs. −40021.94; sample size adjusted BIC = −18134.33 vs. −18052.85), and the univariate common pathway model (AIC = −10156.70 vs. −10104.09; BIC = −40182.44 vs. −40055.04; sample size adjusted BIC = −18126.01 vs. −18062.14), a model constraining parameters across three age groups (i.e., young - age 12 to 14, middle - age 15 to 16, old - age 17 to 18) had lower model fit indices and better fit than a model where parameters were free to vary across the three age groups. Therefore, data for individuals of all ages were combined in the present study. All analyses were conducted while using age- and sex-specific thresholds to address age and sex differences in the prevalence of internalizing and externalizing disorders. Also, the present study examined only lifetime data, given very similar patterns of results for past year and lifetime data in Ehringer et al. (2006).

Correlations

Polychoric phenotypic correlations were estimated for the six internalizing and externalizing disorders (GAD, SAD, MDD, ADHD, CD, ODD). Also, within-trait cross-sibling correlations and cross-trait cross-sibling correlations were estimated for MZ and DZ twin pairs and for twin-sibling trios (i.e., twin1-twin2-sibling).

Multivariate Models

A full biometrical model includes additive genetic effects (A), nonadditive genetic effects (D), shared environmental influences (C), nonshared environmental influences (E), and special twin-specific environmental influences (T). C comprises environmental influences shared by all siblings, whereas T comprises environmental influences shared only by twin pairs. C and D cannot be estimated simultaneously, as estimation of both C and D rely on the same information (i.e., the difference between the MZ and DZ correlations).

We conducted preliminary analyses utilizing Cholesky models in order to establish appropriate starting points for our analyses. In these Cholesky models, the number of genetic, shared environmental, and nonshared environmental factors are equal to the number of disorders. The first set of factors influences all six disorders, the second set of factors influences only the last five disorders, the third set of factors influences only the last four disorders, and so on. The Cholesky model was the most saturated and unconstrained model tested here.

Five alternative multivariate Cholesky models were tested: the ACTE model (AIC = −11606.33; BIC = −47333.57, sample size adjusted BIC = −21252.69), the ADTE model (AIC = −11606.73; BIC = −47333.77, sample size adjusted BIC = −21252.89), the ACE model (AIC = −11643.10; BIC = −47405.06, sample size adjusted BIC = −21290.82), the ADE model (AIC = −11643.16; BIC = −47405.09, sample size adjusted BIC = −21290.86), and the AE model (AIC = −11685.07; BIC = −47479.15, sample size adjusted BIC = −21331.57). Twin environmental influences (T) could be dropped from both the ACTE model (Δχ2 (21) = 5.24, p = .99) and the ADTE model (Δχ2 (21) = 5.57, p = .99). Similarly, shared environmental influences could be dropped from the ACTE model (Δχ2 (42) = 5.26, p = .99) or the ACE model (Δχ2 (21) = 0.02, p = .99), and nonadditive genetic influences could be dropped from the ADTE (Δχ2 (42) = 5.66, p = .99) or ADE (Δχ2 (21) = 0.09, p = .99) models. Overall, the best fitting, most parsimonious model was the AE model.

Ehringer et al.’s (2006) univariate results suggested that non-additive genetic effects and special twin-specific environmental influences do not contribute significantly to the variance of the internalizing and externalizing disorders examined. Shared environmental influences contributed to the variance of MDD and GAD. Based on these results, and the results of our multivariate Cholesky models, our analyses started with models including additive genetic (A), shared environmental (C), and non-shared environmental (E) effects, but excluding non-additive genetic (D) effects and the twin environment (T).

Three alternative multivariate models were tested (see Figure 1). The first model, the Cholesky model (see Figure 1A), described above, is our most saturated and unconstrained model. (In Figure 1A, the AE model, rather than the ACE model, is shown for the sake of simplicity.) The second model, the bivariate common factor model (Figure 1B), suggests that a single latent internalizing factor explains the covariation among MDD, GAD, and SAD and that a single latent externalizing factor explains the covariation among ADHD, ODD, and CD. Furthermore, the bivariate common factor model proposes an overlap between the genetic and environmental influences on the latent internalizing and externalizing factors, and disorder-specific genetic and environmental influences. The third model, the single common factor model (Figure 1C), is the most constrained model. It suggests that a single latent psychopathology factor explains the covariation among all six psychiatric disorders, and that there are disorder-specific genetic and environmental influences. The fit of the three alternative models were compared using the Akaike Information Criterion (AIC; i.e., −2 log likelihood minus 2df), the Bayesian Information Criterion (BIC), and the sample size adjusted BIC, which takes into account both model fit and parsimony. Lower values of AIC, BIC, and sample size adjusted BIC indicate better fit.

Figure I.

Figure I

Open in a new tab

Three alternative multivariate models. A = additive genetic influences; E = nonshared environmental influences; MDD = major depressive disorder; SAD = separate anxiety disorder; GAD = generalized anxiety disorder; ADHD = attention-deficit/hyperactivity disorder; CD = conduct disorder; ODD = oppositional defiant disorder.

Results

Correlations

Table 2 presents the phenotypic correlations among the six internalizing and externalizing disorders. The correlations among internalizing disorders range from .29 to .38, the correlations among externalizing disorders range from .33 to .49, and the correlations between internalizing and externalizing disorders range from .16 to .45. On average, the correlations between internalizing disorders and externalizing disorders are lower (.28) than the correlations among internalizing disorders (.33) or the correlations among the externalizing disorders (.41), supporting the distinction between internalizing and externalizing disorders. A model allowing the correlations between internalizing disorders and externalizing disorders, the correlations among internalizing disorders, and the correlations among the externalizing disorders to differ (χ2 (12) = 270.81, p < .01; CFI = .92; RMSEA = .09) and a model constraining these correlations to be equal (χ2 (14) = 443.50, p < .01; CFI = .86; RMSEA = .11) were tested; the difference between these models was significant (χ2 (2) = 172.69, p < .01). Overall, internalizing disorders’ correlations with other internalizing disorders were higher than those with externalizing disorders, whereas externalizing disorders’ correlations with other externalizing disorders were higher than those with internalizing disorders. An exception was phenotypic correlations between MDD and other disorders; MDD’s correlations with ADHD (r = .34) and ODD (r = .45) were higher than those with GAD (r = .33) and SAD (r = .29).

Table 2.

Phenotypic correlations among disorders

MDD GAD SAD CD ADHD ODD
MDD 1.00
GAD 0.33 1.00
SAD 0.29 0.38 1.00
CD 0.27 0.16 0.24 1.00
ADHD 0.34 0.29 0.27 0.33 1.00
ODD 0.45 0.23 0.25 0.40 0.49 1.00
Open in a new tab

Note. MDD = major depressive disorder; GAD = generalized anxiety disorder; SAD = separation anxiety disorder; CD = conduct disorder; ADHD = attention-deficit/hyperactivity disorder; ODD = oppositional defiant disorder.

Results of an exploratory factor analysis (conducted in Mplus, Muthén & Muthén, 1998-2004) suggest that a two-factor model (χ2 (4) = 30.83, p < .01; CFI = .99; RMSEA = .05) fits better than a one-factor model (χ2 (9) = 311.88, p < .01; CFI = .90; RMSEA = .11). Factor loadings for MDD, GAD, and SAD are .29, .71, and .50 on the internalizing factor and .41, −.01, and .12 on the externalizing factor. Factor loadings for CD, ADHD, and ODD are .08, .20, and −.01 on the internalizing factor and .46, .51, and .84 on the externalizing factor. GAD and SAD have higher loadings on the internalizing factor, and CD, ADHD, and ODD have higher factor loadings on the externalizing factor. Although MDD’s loading on the externalizing factor is higher than its loading on the internalizing factor, both loadings were statistically significant. Also, our goal was to test an a priori, two-factor model supported by the literature, which suggests that MDD is an internalizing disorder (e.g., Kendler, Prescott, Myers, & Neale, 2003; Krueger, Caspi, Moffitt, & Silva, 1998, Krueger, 1999; Krueger, McGue, & Iacono, 2001). Therefore, we chose to have MDD load on the internalizing factor rather than the externalizing factor in subsequent analyses (i.e., the bivariate common pathway model described below).

The within-trait, cross-sibling correlations for MDD, GAD, SAD, CD, ADHD, and ODD in the MZs, DZs, and siblings are presented in Table 3. With the exception of GAD, the MZ correlations are larger than the DZ and sibling correlations, suggesting genetic influences. There is not a clear pattern of DZ correlations being larger than sibling correlations; this result is consistent with Ehringer et al.’s finding that twin-specific environmental influences were not statistically significant for any of the disorders. The cross-trait, cross-sibling correlations (e.g. the correlation between twin 1’s MDD and twin 2’s SAD) are also presented in Table 3. Again, in general, the MZ correlations are larger than the DZ and sibling correlations, suggesting genetic influences on the covariance between childhood psychiatric disorders.

Table 3.

Within-trait, cross-sib correlations and cross-trait, cross-sib correlations

MZs (570 Pairs) DZs (592 Pairs) Twin-Sibs (426 Trios)
MDD 0.32 0.09 0.29
GAD 0.21 0.21 0.11
SAD 0.39 0.14 0.18
CD 0.57 0.32 0.20
ADHD 0.35 0.16 0.12
ODD 0.38 0.14 0.06
MDD-SAD 0.25 −0.01 0.08
MDD-GAD 0.21 −0.04 0.09
MDD-ADHD 0.20 0.13 0.12
MDD-CD 0.19 0.05 0.11
MDD-ODD 0.20 0.17 0.07
SAD-GAD 0.16 0.13 0.08
SAD-ADHD 0.19 0.11 0.01
SAD-CD 0.14 0.06 0.04
SAD-ODD 0.23 0.04 0.04
GAD-ADHD 0.13 0.11 0.07
GAD-CD 0.10 0.07 0.03
GAD-ODD 0.13 0.09 0.07
ADHD-CD 0.26 0.15 0.09
ADHD-ODD 0.26 0.19 0.12
CD-ODD 0.27 0.14 0.04
Open in a new tab

Note. MZs = monozygotic twin pairs; DZs = dizygotic twin pairs; MDD = major depressive disorder; GAD = generalized anxiety disorder; SAD = separation anxiety disorder; CD = conduct disorder; ADHD = attention-deficit/hyperactivity disorder; ODD = oppositional defiant disorder.

Multivariate Models

The Cholesky model (Figure 1A) had a −2 log likelihood of 21242.92 with 16443 degrees of freedom (AIC = −11643.08; BIC = −47405.05; sample size adjusted BIC = −21290.82), the bivariate common factor model (Figure 1B) had a −2 log likelihood of 21295.87 with 16475 degrees of freedom (AIC = −11654.13; BIC = −47491.50; sample size adjusted BIC = −21326.45), and the single common factor (Figure 1C) had a −2 log likelihood of 21349.43 with 16480 degrees of freedom (AIC = −11610.57; BIC = −47482.36, sample size adjusted BIC = −21309.37). The bivariate common factor model had a lower AIC, BIC, and sample size adjusted BIC than the Cholesky model, indicating that it does not fit worse than the Cholesky model. In contrast, the single common factor model had a higher AIC (but lower BIC and sample size adjusted BIC) than the Cholesky model.

The results of the full bivariate common factor model are presented in Figure 2A. The parameter estimates that could be dropped from the model without causing a significant decrement in fit are in regular, non-bolded font. Figure 2B presents the results of the final bivariate common factor model, where the statistically non-significant parameter estimates were dropped. The final bivariate common factor model fit slightly better (AIC = −11658.51; BIC = −47526.57, sample size adjusted BIC = −21340.87) than the full bivariate common factor model AIC = −11654.13; BIC = −47491.50; sample size adjusted BIC = −21326.45).

Figure II.

Figure II

Open in a new tab

Results of the full bivariate common factor model (Figure 2A) and results of the final bivariate common factor model (Figure 2B). A = additive genetic influences; C = shared environmental influences; E = nonshared environmental influences; rA = correlation between genetic influences on internalizing and externalizing disorders; rC = correlation between shared environmental influences on internalizing and externalizing disorders; rE = correlation between nonshared environmental influences on internalizing and externalizing disorders; MDD = major depressive disorder; SAD = separate anxiety disorder; GAD = generalized anxiety disorder; ADHD = attention-deficit/hyperactivity disorder; CD = conduct disorder; ODD = oppositional defiant disorder.

The internalizing disorders latent factor explained 41% of the variance of MDD, 32% of the variance of SAD, and 30% of the variance of GAD. The externalizing disorders latent factor explained 44% of the variance of ADHD, 26% of the variance of CD, and 52% of the variance of ODD. There were moderate genetic influences (h2 = .60) and nonshared environmental influences (e2 = .40) on the latent internalizing disorders factor, and moderate genetic influences (h2 = .65) and nonshared environmental influences (e2 = .35) on the latent externalizing disorders factor. There was a significant correlation between the genetic influences on internalizing and externalizing disorders (rA = .72) and a significant correlation between the genetic influences on the nonshared environmental influences (rE = .74). However, neither the rA (Δχ2 (1) = 7.10, p < .01) nor the rE (Δχ2 (1) = 4.00, p = .05) could be constrained to 1.0. The correlation between the two latent factors was .72, and the percentage of covariance between internalizing and externalizing disorders explained by common genetic and common nonshared environmental influences was 62% and 38%, respectively. There were moderate disorder-specific nonshared environmental influences on each disorder, but little evidence of disorder-specific familial influences, with the exception of MDD (disorder-specific c2 = .11) and CD (disorder-specific h2 = .38).

Given that some of the correlations between MDD and externalizing disorders were higher than those between MDD and the other internalizing disorders, we repeated the analyses omitting MDD (i.e., examining GAD and SAD as manifestations of the latent internalizing factor and ADHD, CD, and ODD as manifestations of the latent externalizing factor). The pattern of results was similarly to those from analyses including MDD. The bivariate common factor model (AIC = −8363.82, BIC = −38899.32, sample size adjusted BIC = −17096.96) had a lower AIC, BIC, and sample size adjusted BIC than the Cholesky model (AIC = −8348.41, BIC = −38846.09, sample size adjusted BIC = −17072.32), indicating that it does not fit worse than the Cholesky model, whereas the single common factor model had a higher AIC (but lower BIC and sample size adjusted BIC; AIC = −8306.68, BIC = −38883.40, sample size adjusted BIC = −17073.10) than the Cholesky model. The overall correlation between the internalizing and externalizing latent factors was .61, the rG was .66, the rC was .00, and the rE was .55.

Discussion

We examined whether a two-factor model positing internalizing and externalizing factors represented the structure and co-occurrence of six psychiatric disorders (MDD, SAD, GAD, ODD, ADHD, and CD) in adolescents. Additionally, we examined the etiology (i.e., the magnitude of genetic and environmental influences) of the latent internalizing and externalizing factors and the covariation between the internalizing and externalizing factors. Phenotypic correlations suggested significant covariation among all six psychiatric disorders. Correlations among internalizing disorders and among externalizing disorders were higher on average than correlations between the two sets of disorders, and the results of factor analyses confirmed that there are two latent factors (internalizing and externalizing) underlying these common psychiatric disorders. In general, within-trait/cross-sibling correlations and cross-trait/cross-sibling correlations were higher in the MZ twin pairs than in DZ twin pairs or sibling pairs, suggesting genetic influences on the variations in individual disorders as well as on covariation between disorders.

The latent internalizing and externalizing factors explained 30% to 41% of the variance of the individual internalizing disorders and 26% to 52% of the variance of the individual externalizing disorders, respectively. These results are consistent with prior reports that found evidence for a two-factor model positing internalizing and externalizing factors (Achenbach, 1991; Krueger, Caspi, Moffitt, & Silva, 1998; Krueger, McGue, & Iacono, 2001).

The etiology of the common internalizing and externalizing factors was best explained by additive genetics and non-shared environment (h2 = .60 and e2 = .40 for the internalizing factor and h2 = .65, e2 = .35 for the externalizing factor). The common internalizing and externalizing factors have higher heritabilities than each of the six individual disorders and are not influenced by shared environmental influences. In contrast, Ehringer et al.’s (2006) univariate study of the same sample reported heritabilities of .36, .31, .56, and .30 for SAD, ADHD, CD, and ODD, respectively and was unable to distinguish between genetic and shared environmental influences on GAD and MDD because of inadequate power.

Most of the familial influences on the six psychiatric disorders examined were those on the latent internalizing and externalizing factors (see Figure 3). There was little evidence of disorder-specific familial influences, with the exception of shared environmental influences on MDD (disorder-specific c2 = .11) and additive genetic influences on CD (disorder-specific h2 = .38). Disorder-specific influences were limited to nonshared environmental influences (i.e., environmental influences leading to differences in sibling pairs or measurement error) for SAD, GAD, ADHD, and ODD.

The correlation between latent internalizing and externalizing factors (r = .72) was due to both common genetic (62%) and nonshared environmental (38%) influences. Although our results provide strong support for two separate internalizing and externalizing factors, common genetic and environmental factors confer risk across internalizing and externalizing factors. Since this result suggests the presence of a broad child psychopathology factor, we tested a broad childhood psychopathology model with a single latent psychopathology factor. This model did not represent psychopathology in our sample as well as the model including two correlated but separate internalizing and externalizing factors.

Of studies examining comorbidity between internalizing and externalizing disorders, our results most resemble those of O’Connor and colleagues (1998) who found strong evidence for a common genetic liability underlying comorbidity between depression and antisocial behavior in adolescents. However, our results are inconsistent with those of Gjone and Stevenson (1997), who concluded that covariation between internalizing and externalizing behavior in childhood mainly results from common shared environmental influences.

As noted above, there are several methodological differences (e.g., the construct examined, the assessment method, and the age of the participants; see Introduction) that may have led to discrepancies in the results of these studies. First, it is possible that the use of parent report in Gjone and Stevenson (1997) may have increased the estimate of common shared environmental influences. Parental ratings may be subject to rater biases (i.e., overestimating or underestimating scores consistently), and if such biases inflate the correlations for both MZ and DZ twin pairs, the evidence of shared environmental influences will be inflated (Hewitt et al., 1992). Second, the statistically significant common nonshared environmental influences found in the present study may be inflated given the use of self-report interviews in the present study. Rater biases or measurement error specific to individuals may have increased the correlation between nonshared environmental influences on internalizing and externalizing disorders. Third, some studies have noted problems with the reliability and validity of the DISC (McMahon & Frick, 2005; Silverman & Ollendick, 2005), which was used in the present study. Specifically, the test-retest reliability kappa coefficient for MDD (.92) is higher than that for SAD (.46) and GAD (.58), suggesting that the DISC may be a better assessment of depressive than anxiety-related symptomatology (Shaffer et al., 2000). If diagnoses made with the DISC in the present study are unreliable, the estimates of the comorbidity between internalizing and externalizing also may be unreliable. Furthermore, symptoms for diagnoses assessed earlier in a DISC interview may be more valid than those assessed later because endorsement of symptoms tends to diminish over the course of an interview (Jensen et al., 1999). The degree to which methodological differences explain the discrepancies in the results is unclear, as systematic studies examining whether these methodological differences lead to differing conclusions regarding the etiology of the covariance between internalizing and externalizing disorders have not been conducted.

Our findings also differ from those of Kendler et al. (2003), and there are notable methodological differences between our study and Kendler et al.’s. Their internalizing factor included fear disorders and their externalizing factor included substance use disorders, whereas in the present study, the internalizing factor did not include fear disorders and the externalizing factor included ADHD and ODD. Kendler et al. examined an adult sample, whereas the present study examined an adolescent sample. Third, their multivariate model was quite different, with orthogonal internalizing and externalizing genetic factors.

Genetic contributions to developmental psychopathology are well recognized in contemporary research. Our results suggest that internalizing and externalizing disorders may in part share genetic influences. In other words, adolescents genetically predisposed to internalizing disorders may also be predisposed to externalizing psychopathology. Based on similar assumptions, recent reports have examined multiple genes as candidates for both internalizing and externalizing disorders (e.g., serotonin transporter gene – Caspi et al., 2003; Sakai et al., 2006, serotonin 2A receptor gene – Eley et al., 2004; Mik et al., 2007; dopamine D4 receptor gene – Lopez Leon et al., 2005; Rowe et al., 2001). While definitive roles for specific genes in the development of internalizing, externalizing, or co-occurring disorders have not yet been established, these candidate genes may confer risk for both classes of psychopathology.

Nonshared environment influenced latent internalizing and externalizing psychopathology factors as well as their covariation significantly. Nonshared environment includes measurement error and all influences that result in differences between individuals growing up in the same family. Possible sources of common non-shared environmental influences affecting development include family composition, sibling interactions, and peer influences (e.g., Plomin et al., 2001).

Environmental conditions that are not shared between siblings appear to contribute more to the development of psychopathology than shared environment. Our findings of minimal shared environmental influences on either latent internalizing or externalizing factors are consistent with many behavioral genetics findings of psychopathology (Turkheimer et al., 2000) and are not surprising. Shared environment seems to be particularly influential in high-risk environments such as disadvantaged neighborhoods or unstable family life (Burt, 2009). However, ours is a community-based sample where prevalence rates for internalizing and externalizing psychopathology are relatively low and a majority of children were likely being raised in safe, stable environments.

On average, we observed higher correlations between MDD and externalizing disorders than between MDD and the other internalizing disorders (although MDD was also significantly correlated with the other internalizing disorders). In contrast, GAD and SAD were more highly correlated with the other internalizing disorders than with the externalizing disorders. This result is somewhat consistent with results from Hewitt et al. (1997), who reported higher factor correlations between MDD and ADHD, CD, and ODD (.34, .25, and .37, respectively) than between SAD or overanxious disorder and externalizing disorders (.07 to .22) in a sample of 8- to 16-year-olds. Similarly, Angold et al.’s (1999) meta-analysis concluded that the odds ratio for the comorbidity between depression and CD/ODD (OR = 6.6) and between depression and ADHD (OR = 5.5) was higher than that between anxiety and externalizing disorders (OR = 3.0 to 3.1). It is possible that the higher than expected correlation between MDD and externalizing disorders is spurious or due to measurement error. Another hypothesis for the higher comorbidity between MDD and externalizing disorders is that in children and adolescents, irritability is a symptom common to both major depression as well as externalizing psychopathologies. Ideally, we would have tested a model examining whether MDD is still highly correlated with externalizing psychopathology after symptoms of “irritability” are removed. However, we could not test this hypothesis, as we did not have information regarding all symptoms for all CADD participants since the DISC-IV was stopped after diagnosis was established (i.e., enough symptoms were present).

Limitations

The results of the present study should be interpreted while considering the following limitations. The present study examined only three internalizing disorders, MDD, GAD, and SAD, and did not examine any of the fear and panic disorders. Therefore, we were not able to test the validity of models making finer distinctions among the internalizing disorders, such as the distinction between the Anxious-Misery and Fear subfactors (Kendler et al., 2003; Krueger, 1999). Additional behavior genetic studies examining a larger number of internalizing disorders in children and adolescents are needed.

The data analyzed were collected from the general population, where the prevalence of psychiatric disorders is low (especially for internalizing disorders in males; see Ehringer et al, 2006). Caution should be used when interpreting prevalence rates (see Table 1) for CD and ADHD diagnoses in our sample since they do not reflect DSM-IV criteria with respect to age of onset of symptoms or symptom clustering. The data were based on self-report interviews, which may have led to underestimates of heritability (Eaves et al., 1997). For example, the heritability of .31 for ADHD in the present sample is lower than those reported by researchers examining parent reports (although there is evidence that rating bias may be present in parent reports of ADHD symptoms [Simonoff et al., 1998]).

As noted above, the DISC may have imperfect reliability and validity (McMahan & Frick, 2005; Silverman & Ollendick, 2005), leading to incorrect estimates of comorbidity between internalizing and externalizing disorders. However, it is important to note that significant covariances were observed among internalizing and externalizing disorders, and that these covariances were stronger within than between internalizing and externalizing disorders. Furthermore, there were significant genetic influences on the latent internalizing and externalizing factors (which would not be possible with an unreliable, invalid measure). These findings suggest that the DISC was reliable and valid in the assessment of these disorders.

Conclusion and Future Directions

This is the first study to examine alternative behavioral genetic models of co-occurrence among DSM-IV psychiatric disorders in a genetically informative sample of adolescents. We found support for a model positing two latent internalizing and externalizing factors explaining interrelationships among six psychiatric disorders in adolescents. Latent internalizing and externalizing disorder factors were moderately heritable and were influenced by common genetic and nonshared environmental factors.

Krueger (1999) suggested that comorbidity in psychiatric disorders results from common, underlying core psychopathological processes and argued for “focusing research on these core processes themselves, rather than on their varied manifestations as separate disorders” (p. 921). The present study’s results (i.e., the higher heritabilities for the latent internalizing and externalizing factors than for individual disorders and the relative lack of familial disorder-specific influences) support Krueger’s statements. Personality and temperament may underlie some of these core processes. Varying levels of constructs such as negative emotionality, neuroticism, effortful control, and low fear response (e.g., Krueger et al., 2001; Lahey & Waldman, 2003; Lilienfeld, 2003; Nigg, 2006; Muris & Ollendick, 2005) may confer particular risk for development of psychopathology in adolescents. For example, Lahey and Waldman suggest that negative emotionality is a risk factor common to internalizing and externalizing disorders, whereas daring is a risk factor specific to externalizing disorders. Similarly, Wolff and Ollendick’s (2006) developmental model of comorbid conduct problems and depression suggest a set of common risk factors such as negative emotionality and unique risk factors such as undercontrol of emotions for conduct problems and negative self-concept for depression.

These findings have significant implications for ongoing research examining developmental risk factors contributing to adolescent liability for internalizing, externalizing, and co-occurring psychopathology. First, research efforts must identify specific genes or combinations of genes that place adolescents at risk for the development of one or more psychiatric disorders. Simultaneous research efforts must continue to identify temperament and personality dimensions that predispose particular individuals to psychopathology. Finally, environmental risk factors common and specific to internalizing and externalizing disorders must be identified. Greater understanding of genetic and environmental influences that increase the risk of onset of psychopathology in adolescents may inform the development of effective intervention and prevention efforts (Jaffee & Price, 2007; Knapp & Mastergeorge, 2009).

Acknowledgments

This work was supported by National Institutes of Health Grants MH43899, MH16880, HD10333, DA11015, DA13956, AA015336, and HD50346. Earlier versions of this paper were presented at the meeting of the Behavior Genetics Association on July 2, 2005 in Los Angeles, CA and the meeting of the World Congress of Psychiatric Genetics on October 17, 2005 in Boston, MA. We thank the participants who contributed their time to this project and the research assistants at the Institute for Behavioral Genetics for careful work in data collection, coding, and management.

References

  1. Achenbach TM. Manual for the Child Behavior Checklist/4-18 and 1991 Profile. University of Vermont, Department of Psychiatry; Burlington: 1991. [Google Scholar]
  2. Angold A, Costello EJ, Erkanli A. Comorbidity. Journal of Child Psychology and Psychiatry. 1999;40:57–87. [PubMed] [Google Scholar]
  3. American Psychiatric Association . Diagnostic and statistical manual of mental disorders. 4th ed Author; Washington, D.C.: 1994. [Google Scholar]
  4. Burcusa SL, Iacono WG, McGue M. Adolescent twins discordant for major depressive disorder: shared familial liability to externalizing and other internalizing disorders. Journal of Child Psychology and Psychiatry. 44(7):997–1005. doi: 10.1111/1469-7610.00184. [DOI] [PubMed] [Google Scholar]
  5. Burt SA. Rethinking environmental contributions to child and adolescent psychopathology: a meta-analysis of shared environmental influences. Psychological Bulletin. 2009;135:608–637. doi: 10.1037/a0015702. [DOI] [PubMed] [Google Scholar]
  6. Burt SA, Krueger RF, McGue M, Iacono WG. Sources of covariation among attention-deficit/hyperactivity disorder, oppositional defiant disorder, and conduct disorder: the importance of shared environment. Journal of Abnormal Psychology. 2001;110:516–525. doi: 10.1037/0021-843X.110.4.516. [DOI] [PubMed] [Google Scholar]
  7. Caspi A, Sugden K, Moffitt TE, Taylor A, Craig IW, Harrington H, et al. Influence of life stress on depression: moderation by a polymorphism in the 5-HTT gene. Science. 2003;301:386–389. doi: 10.1126/science.1083968. [DOI] [PubMed] [Google Scholar]
  8. Clark LA, Watson D. Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. Journal of Abnormal Psychology. 1991;100:316–336. doi: 10.1037//0021-843x.100.3.316. [DOI] [PubMed] [Google Scholar]
  9. Cole DA, Truglio R, Peeke L. Relation between symptoms of anxiety and depression in children: a multitrait-multimethod-multigroup assessment. Journal of Consulting and Clinical Psychology. 1997;65:110–119. doi: 10.1037//0022-006x.65.1.110. [DOI] [PubMed] [Google Scholar]
  10. Dick DM, Viken RJ, Kaprio J, Pulkkinen L, Rose RJ. Understanding the covariation among childhood externalizing symptoms: genetic and environmental influences on conduct disorder, attention deficit hyperactivity disorder, and oppositional defiant disorder symptoms. Journal of Abnormal Child Psychology. 2005;33:219–229. doi: 10.1007/s10802-005-1829-8. [DOI] [PubMed] [Google Scholar]
  11. Derks EM, Dolan CV, Boomsma DI. Effects of censoring on parameter estimates and power in genetic modeling. Twin Research. 2004;7:659–669. doi: 10.1375/1369052042663832. [DOI] [PubMed] [Google Scholar]
  12. Eaves LJ, Silberg JL, Meyer JM, Maes HH, Simonoff E, Pickles A, Rutter M, Neale MC, Reynolds CA, Erikson MT, Heath AC, Loeber R, Truett KR, Hewitt JK. Genetics and developmental psychopathology: 2. The main effects of genes and environment on behavioral problems in the Virginia Twin Study of Adolescent Behavioral Development. Journal of Child Psychology and Psychiatry. 1997;38:965–980. doi: 10.1111/j.1469-7610.1997.tb01614.x. [DOI] [PubMed] [Google Scholar]
  13. Ehringer MA, Rhee SH, Young S, Corley R, Hewitt JK. Genetic and environmental contributions to common psychopathologies of childhood and adolescence: a study of twins and their siblings. Journal of Abnormal Child Psychology. 2006;34:1–17. doi: 10.1007/s10802-005-9000-0. [DOI] [PubMed] [Google Scholar]
  14. Eley TC, Sugden K, Corsico A, Gregory AM, Sham P, McGuffin P, et al. Gene-environment interaction analysis of serotonin system markers with adolescent depression. Molecular Psychiatry. 2004;9:908–915. doi: 10.1038/sj.mp.4001546. [DOI] [PubMed] [Google Scholar]
  15. Gelhorn HL, Stallings MC, Young SE, Corley RP, Rhee SH, Hewitt JK. Genetic and environmental influences on conduct disorder: symptom, domain, and full-scale analyses. Journal of Child Psychology and Psychiatry. 2005;46:580–591. doi: 10.1111/j.1469-7610.2004.00373.x. [DOI] [PubMed] [Google Scholar]
  16. Gjone H, Stevenson J. The association between internalizing and externalizing behavior in childhood and early adolescence: genetic of environmental common influences? Journal of Abnormal Child Psychology. 1997;25:277–286. doi: 10.1023/a:1025708318528. [DOI] [PubMed] [Google Scholar]
  17. Guttmann-Steinmetz S, Crowell JA. Attachment and externalizing disorders: A developmental psychopathology perspective. Journal of the American Academy of Child and Adolescent Psychiatry. 2006;45:440–451. doi: 10.1097/01.chi.0000196422.42599.63. [DOI] [PubMed] [Google Scholar]
  18. Hannelius U, Gherman L, Makela VV, Lindstedt A, Zucchelli M, Lagerberg C, Tybring G, Kere J, Lindgren CM. Large-scale zygosity testing using single nucleotide polymorphisms. Twin Research and Human Genetics. 2007;10(4):604–625. doi: 10.1375/twin.10.4.604. [DOI] [PubMed] [Google Scholar]
  19. Harrington R, Fudge H, Rutter M, Pickles A, Hill J. Adult outcomes of childhood and adolescent depression: II. Links with antisocial disorders. Journal of the American Academy of Child and Adolescent Psychiatry. 1991;30:434–439. doi: 10.1097/00004583-199105000-00013. [DOI] [PubMed] [Google Scholar]
  20. Hewitt JK, Silberg JL, Neale MC, Eaves LJ, Erickson M. The analysis of parental ratings of children’s behavior using LISREL. Behavior Genetics. 1992;22:293–317. doi: 10.1007/BF01066663. [DOI] [PubMed] [Google Scholar]
  21. Hewitt JK, Silberg JL, Rutter M, Simonoff E, Meyer JM, Maes H, et al. Genetics and developmental psychopathology: 1. Phenotypic assessment in the Virginia Twin Study of Adolescent Behavioral Development. Journal of Child Psychology and Psychiatry. 1997;38:943–963. doi: 10.1111/j.1469-7610.1997.tb01613.x. [DOI] [PubMed] [Google Scholar]
  22. Hicks BM, Blonigen DM, Kramer MD, Krueger RF, Patrick CJ, Iacono WG, McGue M. Gender differences and developmental change in externalizing disorders from late adolescence to early adulthood: A longitudinal twin study. Journal of Abnormal Psychology. 2007;116:433–447. doi: 10.1037/0021-843X.116.3.433. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hudziak JJ, Achenbach TM, Althoff RR, Pine DS. A dimensional approach to developmental psychopathology. International Journal of Methods in Psychiatric Research. 2007;16(S1)I:S16–S23. doi: 10.1002/mpr.217. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Jaffee SR, Price TS. Gene-environment correlations: a review of the evidence and implications for prevention of mental illness. Molecular Psychiatry. 2007;12:432–442. doi: 10.1038/sj.mp.4001950. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Jensen PS, Watanable HK, Richters JE. Who’s up first? Testing for order effects in structured interviews using a counterbalanced experimental design. Journal of Abnormal Child Psychology. 1999;27(6):439–445. doi: 10.1023/a:1021927909027. [DOI] [PubMed] [Google Scholar]
  26. Kendler KS, Gardner CO, Lichtenstein P. A developmental twin study of symptoms of anxiety and depression: evidence for genetic innovation and attenuation. Psychological Medicine. 2008;38:1567–1575. doi: 10.1017/S003329170800384X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kendler KS, Prescott CA, Myers J, Neale MC. The structure of genetic and environmental risk factors for common psychiatric and substance use disorders in men and women. Archives of General Psychiatry. 2003;60:929–937. doi: 10.1001/archpsyc.60.9.929. [DOI] [PubMed] [Google Scholar]
  28. Knapp P, Mastergeorge AM. Clinical implications of current findings in neurodevelopment. Psychiatr Clin N Am. 2009;32:177–197. doi: 10.1016/j.psc.2008.11.006. [DOI] [PubMed] [Google Scholar]
  29. Krueger RF. The structure of common mental disorders. Archives of General Psychiatry. 1999;56:921–926. doi: 10.1001/archpsyc.56.10.921. [DOI] [PubMed] [Google Scholar]
  30. Krueger RF, Caspi A, Moffitt TE, Silva PA. The structure and stability of common mental disorders (DSM-III-R): a longitudinal-epidemiological study. Journal of Abnormal Psychology. 1998;107:216–227. doi: 10.1037//0021-843x.107.2.216. [DOI] [PubMed] [Google Scholar]
  31. Krueger RF, McGue M, Iacono WG. The higher-order structure of common DSM mental disorders: internalization, externalization, and their connections to personality. Personality and Individual Differences. 2001;30:1245–1259. [Google Scholar]
  32. Lahey BB, Waldman ID. A developmental propensity model of the origins of conduct problems during childhood and adolescence. In: Lahey BB, Moffitt TE, Caspi A, editors. Causes of conduct disorder and juvenile delinquency. The Guilford Press; New York: 2003. pp. 76–117. [Google Scholar]
  33. Lilienfeld SO. Comorbidity between and within childhood externalizing and internalizing disorders: reflections and directions. Journal of Abnormal Child Psychology. 2003;31:285–291. doi: 10.1023/a:1023229529866. [DOI] [PubMed] [Google Scholar]
  34. Leon S. Lopez, Croes EA, Sayed-Tabatabaei FA, Claes S, Van Broeckhoven C, van Duijn CM. The dopamine D4 receptor gene 48-base-pair-repeat polymorphism and mood disorders: a meta-analysis. Biological Psychiatry. 2005;57:999–1003. doi: 10.1016/j.biopsych.2005.01.030. [DOI] [PubMed] [Google Scholar]
  35. McMahon RJ, Frick PJ. Evidence-based assessment of conduct problems in children and adolescents. Journal of Clinical Child and Adolescent Psychology. 2005;34(3):477–505. doi: 10.1207/s15374424jccp3403_6. [DOI] [PubMed] [Google Scholar]
  36. Middeldorp CM, Cath DC, Van Dyck R, Boomsma DI. The co-morbidity of anxiety and depression in the perspective of genetic epidemiology. A review of twin and family studies. Psychological Medicine. 2005;35:611–624. doi: 10.1017/s003329170400412x. [DOI] [PubMed] [Google Scholar]
  37. Mik HM, Ehtesham S, Baldassarra L, De Luca V, Davidge K, Bender D, et al. Serotonin system genes and childhood-onset aggression. Psychiatric Genetics. 2007;17:11. doi: 10.1097/YPG.0b013e3280114103. [DOI] [PubMed] [Google Scholar]
  38. Moffitt TE. Adolescence-limited and life-course-persistent antisocial behavior: A developmental taxonomy. Psychological Review. 1993;100:674–701. [PubMed] [Google Scholar]
  39. Muris P, Ollendick TH. The role of temperament in the etiology of child psychopathology. Clinical and Child Family Psychology Review. 2005;8:271–288. doi: 10.1007/s10567-005-8809-y. [DOI] [PubMed] [Google Scholar]
  40. Muthén LK, Muthén BO. Mplus User’s Guide. Third ed Muthén & Muthén; Los Angeles, CA: 1998-2004. [Google Scholar]
  41. Nadder TS, Rutter M, Silberg JL, Maes HH, Eaves LJ. Genetic effects on the variation and covariation of attention deficit-hyperactivity disorder (ADHD) and oppositional-defiant disorder/conduct disorder (Odd/CD) symptomatologies across informant and occasion of measurement. Psychological Medicine. 2002;32:39–53. doi: 10.1017/s0033291701004792. [DOI] [PubMed] [Google Scholar]
  42. Nadder TS, Silberg JL, Eaves LJ, Maes HH, Meyer JM. Genetic effects on ADHD symptomatology in 7- to 13-year-old twins: results from a telephone survey. Behavior Genetics. 1998;28:83–99. doi: 10.1023/a:1021686906396. [DOI] [PubMed] [Google Scholar]
  43. Neale MC, Boker SM, Xie G, Maes HH. Mx: Statistical modeling. 6th ed Department of Psychiatry, Virginia Commonwealth University; Richmond, VA: 2002. [Google Scholar]
  44. Nigg JT. Temperament and developmental psychopathology. Journal of Clinical Psychology and Psychiatry. 2006;47:395–422. doi: 10.1111/j.1469-7610.2006.01612.x. [DOI] [PubMed] [Google Scholar]
  45. O’Connor TG, McGuire S, Reiss D, Hetherington EM, Plomin R. Co-occurrence of depressive symptoms and antisocial behavior in adolescence: a common genetic liability. Journal of Abnormal Psychology. 1998;107:27–37. doi: 10.1037//0021-843x.107.1.27. [DOI] [PubMed] [Google Scholar]
  46. Plomin R. Behavioral genetics: A primer. 2nd ed W.H. Freeman and Company; New York: 1990. [Google Scholar]
  47. Plomin R, Asbury K, Dip PG, Dunn J. Why are children in the same family so different? Nonshared environment a decade later. Canadian Journal of Psychiatry. 2001;46:225–233. doi: 10.1177/070674370104600302. [DOI] [PubMed] [Google Scholar]
  48. Posthuma D, Boomsma DI. A note on the statistical power in extended twin designs. Behavior Genetics. 2000;30:147–158. doi: 10.1023/a:1001959306025. [DOI] [PubMed] [Google Scholar]
  49. Puig-Antich J. Major depression and conduct disorder in prepuberty. Journal of the American Academy of Child and Adolescent Psychiatry. 1982;21:118–128. doi: 10.1016/s0002-7138(09)60910-9. [DOI] [PubMed] [Google Scholar]
  50. Rhea SA, Gross AA, Haberstick BC, Corley RP. Colorado Twin Registry. Twin Resesearch and Human Genetics. 2006;9:941–949. doi: 10.1375/183242706779462895. [DOI] [PubMed] [Google Scholar]
  51. Rowe DC, Stever C, Chase D, Sherman S, Abramowitz A, Waldman ID. Two dopamine genes related to reports of childhood retrospective inattention and conduct disorder symptoms. Molecular Psychiatry. 2001;6:429–433. doi: 10.1038/sj.mp.4000874. [DOI] [PubMed] [Google Scholar]
  52. Sakai JT, Young SE, Stallings MC, Timberlake D, Smolen A, Stetler GL, et al. Case-control and within-family tests for an association between conduct disorder and 5HTTLPR. American Journal of Medical Genetics B: Neuropsychiatric Genetics. 2006;141B:825–832. doi: 10.1002/ajmg.b.30278. [DOI] [PubMed] [Google Scholar]
  53. Schwab-Stone ME, Shaffer D, Dulcan MK, Jensen PS, Fisher P, Bird HR, Goodman SH, Lahey BB, Lichtman JH, Canino G, Rubio-Stipec M, Rae DS. Criterion validity of the NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC 2.3) Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35(7):878–888. doi: 10.1097/00004583-199607000-00013. [DOI] [PubMed] [Google Scholar]
  54. Shaffer D, Fisher P, Dulcan MK, Davies M, Piacentini J, Schwab-Stone ME, et al. The NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC-2.3): description, acceptability, prevalence rates, and performance in the MECA Study. Methods for the Epidemiology of Child and Adolescent Mental Disorders Study. Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35:865–877. doi: 10.1097/00004583-199607000-00012. [DOI] [PubMed] [Google Scholar]
  55. Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME. NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry. 2000;39:28–38. doi: 10.1097/00004583-200001000-00014. [DOI] [PubMed] [Google Scholar]
  56. Silberg J, Rutter M, Meyer J, Maes H, Hewitt J, Simonoff E, et al. Genetic and environmental influences on the covariation between hyperactivity and conduct disturbance in juvenile twins. Journal of Child Psychology and Psychiatry. 1996;37:803–816. doi: 10.1111/j.1469-7610.1996.tb01476.x. [DOI] [PubMed] [Google Scholar]
  57. Silberg JL, Bulik CM. The developmental association between eating disorders symptoms and symptoms of depression and anxiety in juvenile twin girls. Journal of Child Psychology and Psychiatry. 2005;46:1317–1326. doi: 10.1111/j.1469-7610.2005.01427.x. [DOI] [PubMed] [Google Scholar]
  58. Silverman WK, Ollendick TH. Evidence-based assessment of anxiety and its disorders in children and adolescents. Journal of Clincal Child and Adolescent Psychology. 2005;34(3):380–411. doi: 10.1207/s15374424jccp3403_2. [DOI] [PubMed] [Google Scholar]
  59. Simonoff E, Pickles A, Hervas A, Silberg JL, Rutter M, Eaves L. Genetic influences on childhood hyperactivity: contrast effects imply parental rating bias, not sibling interaction. Psychological Medicine. 1998;28:825–837. doi: 10.1017/s0033291798006886. [DOI] [PubMed] [Google Scholar]
  60. Stallings MC, Hewitt J, Lessem JM, Young SE, Corley R, Mikulich SK, et al. Modeling the familial transmission of alcohol dependence symptom counts in clinical and control family pedigrees [Abstract] Behavior Genetics. 2001;31:470. [Google Scholar]
  61. Subbarao A, Rhee SH, Young SE, Ehringer MA, Corley RP, Hewitt JK. Common genetic and environmental influences on major depressive disorder and conduct disorder. Journal of Abnormal Child Psychology. 2008;36:433–444. doi: 10.1007/s10802-007-9189-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Thapar A, Harrington R, McGuffin P. Examining the comorbidity of ADHD-related behaviours and conduct problems using a twin study design. British Journal of Psychiatry. 2001;179:224–229. doi: 10.1192/bjp.179.3.224. [DOI] [PubMed] [Google Scholar]
  63. Turkheimer E, Waldron M. Nonshared environment: A theoretical, methodological, and quantitative review. Psychological Bulletin. 2000;126:78–108. doi: 10.1037/0033-2909.126.1.78. [DOI] [PubMed] [Google Scholar]
  64. Waldman ID, Rhee SH, Levy F. Causes of overlap among symptoms of ADHD, oppositional defiant disorder, and conduct disorder. In: Levy F, Hay DA, editors. Attention, genes, and ADHD. Brunner-Routledge; New York: 2001. pp. 115–138. [Google Scholar]
  65. Weiss B, Catron T. Specificity of the comorbidity of aggression and depression in children. Journal of Abnormal Child Psychology. 1994;22:389–401. doi: 10.1007/BF02168081. [DOI] [PubMed] [Google Scholar]
  66. Wolff JC, Ollendick TH. The comorbidity of conduct problems and depression in childhood and adolescence. Clinical Child and Family Psychology Review. 2006;9:201–220. doi: 10.1007/s10567-006-0011-3. [DOI] [PubMed] [Google Scholar]
  67. Young SE, Stallings MC, Corley RP, Krauter KS, Hewitt JK. Genetic and environmental influences on behavioral disinhibition. American Journal of Medical Genetics. 2000;96B:684–695. [PubMed] [Google Scholar]
  68. Zoccolillo M. Co-occurrence of conduct disorder and its adult outcomes with depressive and anxiety disorders: a review. Journal of the American Academy of Child and Adolescent Psychiatry. 1992;31:547–556. doi: 10.1097/00004583-199205000-00024. [DOI] [PubMed] [Google Scholar]

RESOURCES