External Validity of a Hierarchical Dimensional Model of Child and Adolescent Psychopathology: Tests using Confirmatory Factor Analyses and Multivariate Behavior Genetic Analyses

Abstract

Several recent studies of the hierarchical phenotypic structure of psychopathology have identified a General psychopathology factor in addition to the more expected specific Externalizing and Internalizing dimensions in both youth and adult samples and some have found relevant unique external correlates of this General factor. We used data from 1,568 twin pairs (599 MZ & 969 DZ) age 9–17 to test hypotheses for the underlying structure of youth psychopathology and the external validity of the higher-order factors. Psychopathology symptoms were assessed via structured interviews of caretakers and youth. We conducted phenotypic analyses of competing structural models using Confirmatory Factor Analysis and used Structural Equation Modeling and multivariate behavior genetic analyses to understand the etiology of the higher-order factors and their external validity. We found that both a General factor and specific Externalizing and Internalizing dimensions are necessary for characterizing youth psychopathology at both the phenotypic and etiologic levels, and that the three higher-order factors differed substantially in the magnitudes of their underlying genetic and environmental influences. Phenotypically, the specific Externalizing and Internalizing dimensions were slightly negatively correlated when a General factor was included, which reflected a significant inverse correlation between the non-shared environmental (but not genetic) influences on Internalizing and Externalizing. We estimated heritability of the general factor of psychopathology for the first time. Its moderate heritability suggests that it is not merely an artifact of measurement error but a valid construct. The General, Externalizing, and Internalizing factors differed in their relations with three external validity criteria: mother’s smoking during pregnancy, parent’s harsh discipline, and the youth’s association with delinquent peers. Multivariate behavior genetic analyses supported the external validity of the three higher-order factors by suggesting that the General, Externalizing, and Internalizing factors were correlated with peer delinquency and parent’s harsh discipline for different etiologic reasons..

General Scientific Summary:

In this study we established the external validity of a hierarchical dimensional model of child and adolescent psychopathology by examining differential associations with various correlates and with genetic and environmental influences.

Introduction

For over 35 years, the dominant conceptualization of childhood psychopathology has been the categorical diagnostic structure embodied in successive versions of the Diagnostic and Statistical Manual (the DSM). Despite its predominance, for the past 50 years dimensional analyses of childhood psychopathology have consistently revealed two higher-order dimensions, Externalizing and Internalizing (Achenbach, 1966). These dimensions have been found across several commonly used assessment instruments, whether symptoms of DSM psychiatric disorders (Doyle, Murphy, and Shevlin, 2016) or non-diagnostic behavior problems were assessed, and across ages ranging from preschoolers (Olino et al., 2014) to adolescents (Lahey et al., 2008) and adults (Krueger, 1999). Evidence for the validity of this dimensional structure has accrued to the point where the DSM-5 has acknowledged a transition to this organizational higher-order structure, albeit while retaining categorical diagnoses within it (American Psychiatric Association, 2014). Typically the Externalizing dimension includes symptom dimensions of Attention Deficit Hyperactivity Disorder (ADHD), Oppositional Defiant Disorder (ODD), and Conduct Disorder (CD), with Antisocial Personality Disorder (ASPD) and Substance Use Disorders (SUDs) also included in late adolescents and adults. The Internalizing dimension typically includes symptom dimensions of Major Depressive Disorder (MDD), dysthymia, Generalized Anxiety Disorder (GAD), Specific Phobia, Social Phobia, Separation Anxiety Disorder (SAD), Agoraphobia, Panic Disorder, Post Traumatic Stress Disorder (PTSD), and Obsesssive Compulsive Disorder (OCD).

Recently, several factor analytic studies of the structure of psychopathology (e.g., Caspi et al., 2014; Laceulle et al., 2015; Lahey et al., 2011, 2012, 2015; Olino et al., 2014; Patalay et al., 2015) have found that the best-fitting model includes a General factor of psychopathology on which every form of psychopathology loads, in addition to the higher-order Externalizing and Internalizing dimensions in samples of children, adolescents, and adults. This bifactor model containing General, Externalizing, and Internalizing higher-order factors – the latter two factors being correlated in some but not all studies – appeared to fit best in all of these studies (e.g., Caspi et al., 2014; Laceulle et al., 2015; Lahey et al., 2011, 2012, 2015; Olino et al., 2014; Patalay et al., 2015). Evidence for this General factor was found across these studies despite differences in the ages of the samples, the specific measures used, and the particular first-order dimensions of psychopathology assessed.

Support for the criterion validity of the General factor comes from evidence of unique relations with external validating criteria such as a more deprived childhood, childhood maltreatment, a family psychiatric history, suicide attempts, psychiatric hospitalization, greater reliance on social-welfare benefits throughout adulthood, violent crime convictions, lower IQ and more cognitive and neuropsychological deficits (Caspi et al., 2014), and a personality or temperament profile characterized by high Neuroticism and low Agreeableness and low Conscientiousness (Caspi et al., 2014; Olino et al., 2014). The General factor prospectively predicted future psychopathology and functioning, in addition to physical or sexual abuse or caregiver neglect (Lahey et al., 2012). Inclusion of the General factor greatly diminished the relations of the external validity measures with the specific Distress, Fears, and Externalizing factors. In another recent study (Lahey et al., 2015) the General factor was associated with various aspects of teacher-rated behavior and academic performance and objective measures of grade retention and placement in special education. Such associations were routinely stronger with the General factor than with the specific Internalizing and Externalizing factors.

Across these studies (e.g., Caspi et al., 2014; Lacuelle et al., 2015; Lahey et al., 2011, 2012, 2015; Olino et al., 2014; Patalay et al., 2015) the Externalizing and Internalizing dimensions were moderately positively correlated ( r’s = .33 to .59) in the absence of the General factor, consistent with findings from many studies in the extant literature. Interestingly, once the General factor was included in the model, however, the specific Externalizing and Internalizing dimensions became weakly to moderately negatively correlated ( r’s = −.05 to −.47). In addition, in the presence of the General factor, the specific Externalizing and Internalizing dimensions showed a more distinct pattern of external correlates than in the model without the General factor. For example, Externalizing was associated with a family history of antisocial and substance-use disorders (Caspi et al., 2014), and a personality or temperament profile characterized by low Agreeableness and low Conscientiousness but High Extraversion, whereas Internalizing was associated with High Neuroticism and Low Extraversion (Caspi et al., 2014; Olino et al., 2014).

Although relations with these external criterion variables help substantiate the validity of the General factor, questions remain about its nature and interpretation, specifically regarding whether it merely reflects some methodological artifact, general severity or impairment, or some more substantive entity. In addition, the nature of the General factor differs across studies, such that it primarily reflects psychotic symptoms in some studies (Caspi et al., 2014) versus distress in other studies (Lahey et al., 2011, 2015; Tackett et l., 2013). Indeed, although we refer to the three higher-order factors in this study as General, Externalizing, and Internalizing, the pattern and magnitude of loadings in our previous studies of this sample (Lahey et al., 2011; Tackett et l., 2013) suggest that the General factor may be construed as a Distress factor, whereas the Internalizing factor may be construed largely as a Fears factor. Nonetheless, to be consistent with both our prior studies and the larger literature we maintain the factor labels of General, Externalizing, and Internalizing, while recognizing that these may be revised pending the findings of further research. In addition, although the reasons for the varying nature of the General factor across studies are unclear from the extant literature, these may include differences across samples in age, the specific disorders assessed in children versus adults, and the inclusion of symptom dimensions of rare severe disorders (e.g., schizophrenia, bipolar disorder, autism spectrum disorders) in addition to more prevalent forms of psychopathology.

Multivariate behavior genetic analyses can illuminate these issues by testing alternative models for the etiology of higher-order dimensions of psychopathology, as well as for the etiology of their relations with the external validity measures, and by characterizing the magnitude of their underlying genetic and environmental influences. For example, if the General factor merely reflects some methodological artifact, one would expect it to be influenced only by non-shared environmental influences if it reflects common measurement error or by shared environmental influences if it reflects common rater or respondent bias. Alternatively, insofar as genetic influences underlie the General factor this supports its substantive nature. Similarly, if there is a negative correlation between the genetic (or environmental) influences on the specific Externalizing and Internalizing factors this would suggest that the genes (or environments) that predispose to Externalizing conditions may actually be protective for Internalizing psychopathology. In addition, to the extent that higher-order factors differ in their sharing of common genetic or environmental influences with the external validity measures this would support their discriminant validity. Finally, the magnitudes of the loadings on the General factor and the specific Externalizing and Internalizing factors may help illuminate their nature, particularly if the loadings of certain first-order dimensions of psychopathology are especially high, as this would suggest that such lower-order dimensions virtually define those higher-order dimensions. Herein we use tests of alternative models in multivariate behavior genetic analyses to elucidate these issues.

Questions Addressed

In this study, we used several methods to examine the external validity of the higher-order psychopathology dimensions and transcend our previous analyses of data from this sample in several important ways. First, we used CFA to contrast a much larger set of alternative phenotypic models for youth psychopathology. We previously conducted phenotypic CFAs that were limited to characterizing only the higher-order Externalizing and Internalizing factors but did not entertain models that included a General factor (Lahey et al., 2008). Subsequently, we used CFA to test alternative models for the structure of the genetic and non-shared environmental influences on the 11 first-order dimensions (Lahey et al., 2011) and only two alternative phenotypic models that included a General factor (Tackett et al., 2013). Herein, we test phenotypic models in which both the specific Externalizing and Internalizing factors and the General factor are dropped to confirm that each is necessary for good model fit. Second, we extend these CFAs to directly model the etiology of the higher-order factors by estimating their genetic, shared, and non-shared environmental influences. This analytic approach is distinct from that used previously (Lahey et al., 2011; Petterson et al., 2016) as it yields estimates of the magnitude of genetic and environmental influences on the higher-order factors and has the advantage of greater statistical power to detect shared environmental influences (Schmitz et al., 1998). Characterizing the etiology of the higher-order dimensions facilitates examining the sharing of that etiology with external validity variables. Third, we broaden our previous research by examining within a latent variable modeling framework differences in the associations of the higher-order factors with external validity variables at the family, twin-pair, and twin-specific levels. Fourth, we use our twin sample and multivariate behavior genetic analyses to examine the etiology of the associations between the higher-order factors and selected external validity variables that vary between twins. These four objectives enable us to extend our previous research by establishing the validity of the higher-order dimensions of youth psychopathology. To our knowledge, this is the first study to characterize the etiology of the higher-order psychopathology dimensions and their relations with external validity measures.

Methods

Participants

The Tennessee Twin Study (TTS) sample was representative of 6–17-year-old twins who were born in Tennessee and still living in one of the state’s five MSAs (Nashville, Memphis, Knoxville, Chattanooga, and Bristol). Current addresses of potentially eligible families were provided by the Tennessee Department of Health. A random sample of identified families was selected stratified on the age of the twins and 35 geographic subareas. Interviews were completed with 2,063 adult caretakers (90.8% biological mothers, 7.5% biological fathers, 0.5% step-mothers, and 1.2% grandmothers), with a response rate for caretakers of 69.8%. When the adult caretaker was interviewed, 97.9% of the twin pairs were also interviewed. Caretakers gave written informed consent and twins who were old enough to be interviewed (≥ 9 years of age) gave oral assent. Zygosity of same-sex twin pairs was determined using 9 standard questions about similarities in the twins’ physical appearance (Peeters, Van Gestel, Vlietinck, Derom, & Derom, 1998). Zygosity of twins in the ambiguous range of questionnaire scores was determined using 12 highly polymorphic markers genotyped from DNA extracted from swabbed cheek cells. Twins were randomly designated as twin 1 or 2 after data collection, and 37% of the twins were monozygotic (MZ), 33% were same-sex dizygotic (DZ), and 30% were opposite-sex DZ. Youth were excluded if the caretaker reported that the youth had been diagnosed as psychotic (twin 1 N = 17; twin 2 N = 11), or autistic (twin 1 n = 8; twin 2 n = 5). The sample ranged in age from 6–18 years old (M = 11.7, SD = 3.3), 49% was male, and 71.4% of the sample was of Non-Hispanic European-American ethnic background, 24.0% was of African-American background, and the remaining 4.6% was of mixed or other ethnic background. Family income for participants in 2001 US dollars ranged from $0 - >$150,000 (M = $58,633, SD = $43,086). Among families with a completed interview of the caregiver, only 1.6% did not have completed interviews of the youth. Only the 9–17 year old twin pairs with diagnostic interviews of both caretakers and youth (N = 1,568 twin pairs) were included in the present analyses given that we analyzed the combined caretaker and youth reports (Lahey et al., 2011).

Measures

Child and Adolescent Psychopathology Scale (CAPS).

The CAPS (Lahey et al., 2004) assesses symptoms of common child and adolescent disorders, including CD, attention-deficit/hyperactivity disorder (ADHD), oppositional defiant disorder (ODD), major depression, and anxiety disorders. CAPS items covering DSM-IV symptoms were based on “stem questions” in Version IV of the Diagnostic Interview Schedule for Children (Shaffer et al., 1996), modified from “yes or no” questions to fit a response scale of: “1. Not at all, 2. Just a little, 3. Pretty much, 4. Very much.” Respondents were asked to rate each item by thinking about how well the item describes the youth’s emotion or behavior, how often it occurred, and how serious it was during the last 12 months. In addition, new items were written to describe relevant non-overlapping emotions and behaviors that are referenced in the delinquency and aggression literatures, and other widely-used rating scales, but only DSM-IV symptoms were used in these analyses.

The CAPS was administered to the adult caretaker of 6–17-year-olds and a youth-report version of the CAPS was administered to 9–17-year-olds. Items were randomized and administered in counterbalanced order, with random halves of the participants administered the items in forward or reverse order to control order effects. Robust correlations with relevant measures of functional impairment indicated strong external validity for both the caretaker and youth versions of the CAPS (Lahey et al., 2004). The test–retest intra-class correlations (ICCs) for unit-weighted mean ratings of the symptoms of each DSM-IV disorder based on caretaker reports were: CD=0.89; ODD=0.80; inattention=0.89; hyperactivity-impulsivity=0.88; MDD=0.82; SAD=0.76; GAD=.080; social phobia=0.65; specific phobia=0.84; agoraphobia=0.77; and OCD=0.73. The ICCs based on youth reports of symptoms were: CD=0.78; MDD=0.69; GAD=0.65; SAD=0.68; social phobia=0.62; specific phobia=0.83; agoraphobia=0.70; and OCD=0.67 (Lahey et al., 2008). Data were combined across respondents by taking the highest score for each item across caretakers and youth and then summing the items into the first-order symptom dimensions (Lahey et al., 2011). The only exceptions were ADHD and ODD for which only caretaker reports were used given that youth are not reliable or valid reporters for these disorders (Lahey et al., 2011).

External Validity Measures

Demographic variables (age, gender, ethnicity, family income) were assessed using questions from the MECA Study (Lahey et al., 1996.) Family history of psychopathology including parent’s self-report of anxiety, depression, substance abuse, and antisocial behavior was assessed using the Family History Interview for Epidemiologic Studies (FHE; Lish, Weissman, Adams, Hoven, & Bird, 1995.) Mothers were additionally asked to report if they drank or smoked during pregnancy and what age they were when the twins included in the study were born. The Revised Conflict Tactic Scale (Straus, Hamby, Boney-McCoy, & Sugarman, 1996) was used to assess the harshness of psychical discipline. Child’s friendships with delinquent peers was assessed via child report using the well-validated measures from the Pittsburgh Youth Study and Developmental Trends Study (Loeber et al., 1995). Youth participants were also administered the Digit Span subtest from the Wechsler Intelligence Scale for Children, Revised (WISC-R; Wechsler, 1974). The Digit Span subtest is designed to test children’s ability to repeat a sequence of numbers in the order they were read and in the reverse order. This yields scores for Digit Span Forward, Digit Span Backward, and overall Digit Span IQ. This subtest is meant to measure auditory rehearsal and temporary storage capacity in working memory.

Children’s birth weights and gestational ages were reported by the caretaker who completed the CAPS. Retrospective maternal reports of birth weight and gestational age ten to fifteen years following childbirth have been previously demonstrated to correlate strongly with birth weight data obtained from medical records (ICC = 0.99 for birth weight, 0.90 for gestational age; Yawn, Suman, & Jacobsen, 1998).

Data Analyses

Confirmatory Factor Analyses of First-Order Symptom Dimensions

All data analyses were performed using Mplus 7 (Muthen & Muthen, 2007–2014). We first conducted tests of 9 alternative CFA models for the underlying structure of the CAPS simultaneously in MZ and DZ twins. Data were analyzed using the “cluster” option and the Yuan-Bentler scaled |² (using the MLR estimator in Mplus 7) to account for twins being nested within twin pairs, as well as for any non-normality in the distribution of the symptom dimensions.

For the evaluation of model fit, we reported the following goodness-of-fit indices: the χ² test statistic and its associated degrees-of-freedom (df), the Tucker-Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (RMSR), RMSEA is an index of the discrepancy between observed and predicted covariances per degree of freedom and yields both an index and a test of close fit. The RMSR indicates the average difference between the observed and predicted variances and covariances, expressed in a correlational metric, and values ≤ .05 indicate a close fit whereas values ≤ .10 suggest an adequate fit to the data (Cole, 1987). Assessment of the adequacy of model fit was based on guidelines suggested in the literature, including: TLI greater than 0.95 for reasonably good fit (Hu & Bentler, 1999) and values between 0.90 – 0.95 for acceptable model fit (Bentler, 1990); RMSEA ≤ .08 for adequate fit and ≤ .05 for close fit (Browne & Cudeck, 1993; MacCallum, Browne & Sugawara, 1996). Although the X² values and df’s are presented for each model tested in the tables because they were used to calculate the RMSEAs and other fit indices, their p values from the test of exact fit were not presented nor used to assess model fit as a model would always be rejected under this test given large sample size even though the difference between the observed and predicted variance-covariance matrix is practically non-significant (Brown, 2006). The fit of a single model was evaluated using the combination of TLI, RMSEA, and RMSR, as each individual fit index has its strengths and limitations and no consensus has been reached regarding the use of a single fit index to evaluate the adequacy of model fit (Loehlin, 2004). While these indices were also used in assessing the relative fit of alternative models, we primarily relied on the Bayesian Information criterion (BIC) to adjudicate among models (Loehlin, 2004; Markon & Krueger, 2006). No absolute guidelines exist for the interpretation of BIC, but the model with the lowest value for BIC is considered the best fitting model (Loehlin, 2004; Markon & Krueger, 2006).

Behavior Genetic Analyses

In biometric model fitting analyses, the total observed variation in one or more traits or symptom dimensions is partitioned into estimates of variance related to four latent constructs reflecting four types of causal influences: additive genetic influences (A), non-additive genetic influences (D), shared environmental influences (C), and nonshared environmental influences (E). These latent constructs are derived from differences in the magnitudes of correlations between identical twins, who share all of their segregating genes (r_g = 1) and fraternal twins, who share half of their segregating genes on average (r_g = .5). Greater correlations between identical than fraternal twins implies genetic influences, whereas identical twin correlations less than 1 implies non-shared environmental influences. Similarly, identical twin correlations that are greater than twice the fraternal twin correlations implies non-additive genetic influences, whereas identical twin correlations that are less than twice the fraternal twin correlations implies shared environmental influences. In addition, the direct influence of one twin’s trait or symptom level on their co-twin’s trait or symptom level was included in several models. This parameter also may reflect rater contrast effects on the trait or symptom dimension of interest. Previous univariate behavior genetic analyses in this sample (Lahey et al., 2011) showed no evidence of non-additive genetic influences (i.e., D) on any of the first-order symptom dimensions, thus our multivariate behavior genetic analyses focused only on estimating A, C, and E.

As is standard in quantitative genetic analyses, all symptom dimensions and twin-specific external validity measures were first residualized on sex, age, age-squared, and the interactions of both age terms with sex (McGue & Bouchard, 1984). The biometric models were fit to raw data using Full Information Maximum Likelihood in Mplus 7 (Muthen & Muthen, 2007–2014). In general, the most parsimonious model with the best fit (i.e., the model that most closely reproduces the observed variances and covariances with the fewest parameters) is the preferred model. The overall fit of the models is expressed as a log-likelihood and - 2 times the log-likelihood has an approximate |² distribution in large samples. In addition, we compared fit among alternative models using the RMSEA, RMSR, the TLI, and the BIC as described above (Loehlin, 2004; Markon & Krueger, 2006).

The Common Pathway Model for Common and Unique Genetic and Environmental Influences

A quantitative genetic model that is well suited for estimating common and unique genetic and environmental influences on a set of traits or disorders is the Common Pathway model (Neale & Cardon, 1992). In these analyses we tested a set of alternative models that included or excluded the genetic, shared environmental, and non-shared environmental influences on the higher-order factors to assess the importance of their contributions to the etiology of the first-order psychopathology dimensions. In contrast to our previous multivariate behavior genetic analyses of the Tennessee Twin Study data (Lahey et al., 2011) in which CFAs were conducted on the genetic and non-shared environmental covariances for the 11 symptom dimensions, here we conducted analyses that combined CFAs on the raw twin data with biometric analyses of the higher-order phenotypic dimensions to directly estimate their genetic and environmental influences, which was not possible using our earlier method.

We used a set of alternative Common Pathway models to test our a priori hypotheses regarding the sharing of genetic and environmental influences among the first-order psychopathology dimensions. For example, one can independently drop additive genetic or shared and non-shared environmental influences on each of the higher-order phenotypic dimensions and assess the fit of such restricted models relative to the full Common Pathway model. These models are ideal for such hypothesis-testing, given the clear partitioning of genetic and environmental influences on higher-order and first-order symptom dimensions into those that are shared in common with the other symptom dimensions and those that are unique to each particular symptom dimension. As used in the behavior genetics literature, the full Common Pathway model contains an additive genetic, a shared environmental (or non-additive genetic), and a non-shared environmental latent causal variable for each of the higher-order psychopathology dimensions, which in turn influence all of the observed phenotypic variables in the model (the first-order symptom dimensions). The genetic and environmental influences on the higher-order psychopathology dimensions, combined with the factor loadings of the first-order symptom dimensions on the higher-order dimensions, represent the genetic and environmental influences that the first-order symptom dimensions (the phenotypic variables) share in common. In addition to these common etiological influences, the full Common Pathway model contains additive genetic, shared environmental (or non-additive genetic), and non-shared environmental latent causal variables which specifically influence each of the observed phenotypic variables in the model. This part of the model represents the unique genetic and environmental influences on the first-order symptom dimensions. Estimates of these various etiological influences are reflected in the path coefficients or “loadings” of each of the phenotypic variables on the genetic, shared environmental (or non-additive genetic), and non-shared environmental latent causal variables or “factors”. These loadings represent the magnitude of a particular etiological influence on each of the phenotypic variables and can be tested for significance by comparing their estimated values to their standard errors, as well as by contrasting the fit of the model in which these paths are included versus the fit of the restricted model from which they are omitted.

Specific etiological hypotheses can be tested using the Common Pathway model by contrasting the fit of reduced models in which one or more parameters of interest are dropped to that of a more comprehensive or full model. For example, we can test whether there are additive genetic, or shared (or nonshared) environmental, influences on the General higher-order factor that influence the phenotypic covariation among the Inattention, Hyperactive-Impulsive, ODD, and CD symptom dimensions that they share with the other symptom dimensions reflecting depression and anxiety disorders. Alternatively, we can test whether there are additive genetic, or shared (or nonshared) environmental, influences on the Externalizing higher-order dimension that only influence the Inattention, Hyperactive-Impulsive, ODD, and CD symptom dimensions. Likewise, we can test whether there are additive genetic, or shared (or nonshared) environmental, influences on the Internalizing higher-order dimension that only influence the depression and anxiety first-order symptom dimensions. Finally, we can test whether all of the additive genetic or shared (or nonshared) environmental influences underlying the 4 externalizing (or 7 internalizing) symptom dimensions have unique effects only on each phenotype independent of the others. More specifically, for example we can test whether the additive genetic influences on CD are completely shared in common with the additive genetic influences on all of the other symptom dimensions, with the additive genetic influences on the three other externalizing dimensions, or whether they are completely unique to CD. Analogously, we can also test whether the shared (or nonshared) environmental influences on CD are completely shared in common with the shared (or nonshared) environmental influences on all of the other symptom dimensions, with the shared (or nonshared) environmental influences on the three other externalizing dimensions, or whether they are completely unique to CD. We used the Common Pathway model to test these alternative hypotheses regarding the common etiology of psychopathology dimensions in an effort to further evaluate their validity.

Relation of the External Validity Variables with the Higher-Order Factors

We next regressed each of the External Validity measures on all the higher-order psychopathology dimensions simultaneously in a set of latent variable models, which allowed such regressions to be conducted on the latent psychopathology dimensions, thus obviating the need to save factor scores. We controlled for subjects’ ethnicity for all variables and for the sex and age covariates described above for the twin-specific variables (I.e., Peer Delinquency, Parent’s Harsh Disciple, Digit Span, and birthweight). For each of the external validity measures we contrasted the fit of two models, one in which the regression coefficients for each of the higher-order psychopathology dimensions were freely estimated and one in which they were fixed to be equal. Comparison of the fit of these models (using BIC) tests the hypothesis of discriminant validity among the higher-order psychopathology dimensions.

Common Etiology of the External Validity Variables with the Higher-Order Factors

Finally, we conducted a set of multivariate behavior genetic analyses - using a modified Cholesky decomposition model (Neale & Cardon, 1992) - to estimate the common genetic, shared, and non-shared environmental influences on the higher-order factors and selected external validity variables. To minimize multiple testing, we had three criteria for selecting the external validity measures for analysis: 1. They had to vary between twins within twin pairs, 2. They had to be moderately associated (beta > .2) with at least one of the higher-order factors, and 3. They had to be differentially associated with the higher-order factors. Two variables, Peer Delinquency and Parent’s Harsh Discipline, met these criteria and were included in the analyses of shared etiology with the higher-order factors.

Results

Confirmatory Factor Analyses (CFAs) of First-Order Psychopathology Dimensions

As shown in Table 1, the full model fit quite well according to all fit indices, although the negative correlation between the specific Externalizing and Internalizing factors (−.14 [−.29, .00 ]) was non-significant. The first restricted model tested (Model 2) dropped the loadings of MDD and GAD on the Internalizing factor, retaining their loadings only on the General factor. This model fit better than the full model according to most indices. In contrast to the non-significant correlation between Externalizing and Internalizing in the full model in this restricted model this correlation was significant (−.13 [−.19, −.07 ]).

Table 1.

Fit of Alternative CFA Models

Model	χ2	df	TLI	RMSEA	SRMR	BIC	rEXT-INT
1. Full CFA Model	422	32	.953	.062	.025	29231	−.14 (−.29, .00)
2. MDD & GAD loading only on the General factor	424	34	.956	.060	.026	29228	−.13 (−.19, −.07)
3. Model 1 + No rEXT-INT	419	33	.955	.061	.027	29231
4. Model 2 + No rEXT-INT	443	35	.955	.061	.031	29250
5. No General Factor	2429	43	.787	.133	.087	31823	.57 (.53, .60)
6. No General Factor + No rEXT-INT	3160	44	.728	.133	.087	31823
7. Only General Factor	4308	44	.628	.176	.123	34497
8. No Internalizing Factor	2280	40	.785	.134	.079	31634
9. No Externalizing Factor	2373	37	.757	.142	.065	31415

Note: The Full CFA Model is shown in the middle of Figure 1; r_EXT-INT is the correlation between the Externalizing and Internalizing factors; MDD is Major Depressive Disorder; GAD is Generalized Anxiety Disorder; best fitting model is in bold.

We next fit a series of more restricted models to see whether we could drop the General factor, either (or both of) the specific Externalizing and Internalizing factor(s), the correlation between the Externalizing and Internalizing factors, and the combination of these restrictions with the first two models tested (the full and first restricted models). As shown for Models 3 and 4 in Table 1, dropping the correlation between the specific Externalizing and Internalizing factors from the Full model and from the first restricted model slightly worsened model fit. In addition, dropping the General factor, or either (or both) of the Externalizing and Internalizing factors (Models 5–9), drastically worsened model fit. Although Model 5, which dropped the General factor did not fit well, it is worth noting that the correlation between the Externalizing and Internalizing factors was .57 [.53, .60], consistent with that seen in previous studies that omitted the General factor. Thus, the best fitting model remained the first restricted model in which MDD and GAD only loaded on the General factor and Internalizing and Externalizing were slightly negatively correlated (Model 2).

Common Pathway Models of Genetic and Environmental Influences

Given the complexity of Common Pathway models there are a large number of potential hypothesis tests that can be conducted at both the etiological factor and factor loading levels. We adopted a targeted approach to the hypotheses that we tested in these etiological models to build on the foundation provided by our CFA results. Specifically, we first fit the full Common Pathway model and then tested a series of restricted models by successively dropping all additive genetic, shared environmental, or non-shared environmental influences on the General higher-order dimension and on either the specific Externalizing or Internalizing dimensions. (Note that we could not drop all of the specific non-shared environmental influences as this would imply no measurement error on the 11 first-order symptom dimensions, which is highly implausible). We also tested the fit of models that dropped the factor loadings of the MDD and GAD first-order symptom dimensions on the Internalizing factor, included a correlation between the Externalizing and Internalizing additive genetic influences and/or non-shared environmental influences, and included a correlation between the unique additive genetic influences and/or non-shared environmental influences on Agoraphobia and Specific Phobia and on ODD and CD. The latter correlations were tested given that phenotypic correlations between Agoraphobia and Specific Phobia, and ODD and CD, were shown to improve model fit in our previous factor analyses (Lahey et al., 2008). The fit statistics for these alternative models are shown in Table 2.

Table 2.

Common Pathway Model Fit Statistics

Model	χ2	df	TLI	RMSEA	SRMR	BIC
Full Common Pathway Model	1131	475	.964	.042	.073	27052
1. No A on General	1163	476	.963	.043	.075	27085
2. No C on General	1140	476	.964	.042	.082	27057
3. No E on General	1748	476	.931	.058	.103	27823
4. No A on Internalizing	1146	476	.964	.042	.073	27065
5. No C on Internalizing	1137	476	.964	.042	.073	27054
6. No A on Externalizing	1199	476	.961	.044	.075	27131
7. No C on Externalizing	1135	476	.964	.042	.072	27050
8. With rAEXT-INT & rEEXT-INT	1126	474	.965	.042	.068	27053
9. With rAEXT-INT	1135	475	.964	.042	.072	27057
10. With rEEXT-INT	1125	475	.965	.042	.068	27046
11. No MDD or GAD on INT	1138	477	.964	.042	.064	27048
12. With rAODD-CD	1063	474	.968	.040	.067	26976
13. With rA/ESPP-AG	990	472	.972	.037	.065	26898
14. With C on EXT	978	471	.972	.038	.065	26117
15. With Equal As & Cs on General, EXT, and INT	1144	475	.963	.043	.087	26294
16. No General Factor	2460	484	.892	.073	.155	27872

Note: The Full Common Pathway Model is shown in Figure 18; A is additive genetic influences; C is shared environmental influences; E is nonshared environmental influences; rA_EXT-INT is the correlation between the additive genetic influences on the Externalizing and Internalizing factors; rE_EXT-INT is the correlation between the nonshared environmental influences on the Externalizing and Internalizing factors; MDD is Major Depressive Disorder; GAD is Generalized Anxiety Disorder; EXT is the Externalizing factor; INT is the Internalizing factor; rA_ODD-CD is the correlation between the additive genetic influences on ODD and CD; rA_SPP-AG is the correlation between the additive genetic influences on Specific Phobia and Agoraphobia; rE_SPP-AG is the correlation between the nonshared environmental influences on Specific Phobia and Agoraphobia; best fitting model is in bold.

As shown in Table 2 and Figure 1, the full Common Pathway model fit the data well but we could not drop either the additive genetic influences, shared environmental influences, or non-shared environmental influences on the General factor without significant worsening of model fit. Similarly, we could not drop either additive genetic influences or shared environmental influences on Internalizing without significant worsening of model fit. In contrast, while we could not drop additive genetic influences, we could drop shared environmental influences on Externalizing without any decrement in model fit. It is worth noting that although they could be dropped (according to the BIC) the magnitude of shared environmental influences was not trivial (.16). There was no improvement in model fit when a correlation was estimated between the additive genetic influences on the Externalizing and Internalizing dimensions, but model fit improved with inclusion of the correlation between the non-shared environmental influences on Externalizing and Internalizing, whereas a model that estimated both correlations fit similarly. In contrast, model fit was slightly worsened by dropping the Internalizing factor loadings for MDD and GAD. Better fit was achieved for models that estimated a correlation between the additive genetic influences (but not the non-shared environmental influences) on ODD and CD and between the additive genetic influences as well as the non-shared environmental influences on Agoraphobia and Specific Phobia, with moderate correlations between the additive genetic and non-shared environmental influences on Agoraphobia and Specific Phobia (r = .42 and .29, p’s < .001, respectively) and a moderate correlation between the additive genetic influences on ODD and CD (r = .40, p < .001). Given the generally low power to detect shared environmental influences, and the increased power to detect them in more complex multivariate behavior genetic analyses (Schmitz et al., 1998), we retested for shared environmental influences on Externalizing by adding them back into the final model, which improved model fit. In addition, given the importance of differential etiology to discriminant validity we tested whether the genetic and shared environmental influences on the General, Externalizing, and Internalizing factors differed or could be equated. The model in which these were equated fit worse, suggesting that the etiology differed across the higher-order factors. Finally, we tested the fit of a reduced model in which the General factor was dropped and examined the magnitude of genetic and environmental influences on the first-order symptom dimensions in the original General-Internalizing-Externalizing model versus the restricted Internalizing-Externalizing model. As we show in Table 2 and Figure 2, the restricted model fit worse and the model that included the General factor accounted for substantially more common genetic variance in the 11 first-order symptom dimensions due to the higher-order factors than the model that included only the Internalizing and Externalizing factors (.26 versus .15 on average).

Figure 1.

Best Fitting Common Pathway Model

Note: Path coefficients for factor loadings of the first-order symptom dimensions on the higher-order factors are standardized regression coefficients, whereas path coefficients for the genetic and environmental influences on the higher-order psychopathology factors and for the residual genetic and environmental influences on the first-order symptom dimensions represent squared standardized regression coefficients (i.e., variance components). Decimal points are omitted to save space. A = Additive genetic influences, C = Shared environmental influences, E = Non-shared environmental influences, HI = Hyperactivity/impulsivity, ODD = Oppositional Defiant Disorder, CD = Conduct Disorder, MDD = Major Depressive Disorder, GAD = General Anxiety Disorder, Agora = Agoraphobia, SAD = Separation Anxiety Disorder, OCD = Obsessive Compulsive Disorder. INT = Internalizing Specific Factor, EXT = Externalizing Specific Factor, GENL = General Factor.

Figure 2.

Genetic and Environmental Influences from Common Pathway Model with and without the General Factor. Note: Gen+Ext+Int is the model that includes the General, Externalizing, and Internalizing factors; Ext+Int is the model that includes the Externalizing and Internalizing factors;A2 are additive genetic influences; C2 are shared environmental influences; E2 are non-shared environmental influences.

Relation of each of the Higher-order Dimensions with External Validity Measures

We next examined the relation of each of the three higher-order dimensions with the external validity measures at the family, twin-pair, and twin-specific levels. In a series of structural equation models, we regressed each of the external validity variables on the General, Externalizing, and Internalizing dimensions simultaneously and tested whether these relations differed or could be equated across the higher-order dimensions by contrasting the fit of models in which these regression coefficients were free to vary or were equated across the three higher-order psychopathology dimensions. As shown in Figures 3a and 3b, the relations of Harsh Discipline (regression coefficients free to vary versus equated across the higher-order dimensions: BIC = 31754 versus 31814) , Peer Delinquency ( BIC = 31754 versus 31850), Father’s Depression ( BIC = 113511 versus 113529), Father’s Substance Abuse ( BIC = 113511 versus 113517), Father’s Antisocial Behavior ( BIC = 113511 versus 113523), Mother’s Antisocial Behavior ( BIC = 113511 versus 113525), Mother’s Anxiety ( BIC = 113511 versus 113514), Mother’s Depression ( BIC = 113511 versus 113558), Mother’s Substance Abuse ( BIC = 113511 versus 113519), and cigarettes smoked during pregnancy ( BIC = 113511 versus 113518) differed across the General, Externalizing, and Internalizing dimensions. Both Harsh Discipline and cigarettes smoked during pregnancy were more strongly related with the Externalizing than the General factor, and with the General factor than the Internalizing factor. Peer delinquency was much more strongly related with the General factor than with Externalizing, and with the Externalizing than the Internalizing factor. In contrast, the parental history of psychopathology variables were more associated with the General and Externalizing factors than with the Internalizing factor.

Figure 3a.

Standardized Regressions of the External Validity Variables on the Three Higher-Order Psychopathology Factors. Parental Psychopathology and Adjustment.

Note: Beta Coefficient is the standardized regression coefficient. Betas are shown with their 95% confidence intervals. Asterisks indicate external validity measures for which the Beta Coefficient significantly differed across the three higher-order factors. FatherAntisocial = Father’s history of antisocial behavior, FatherDepressed = Father’s history of depression, FatherSubAbuse = Father’s history of substance abuse, FatherAnxiety= Father’s history of anxiety, FatherJail = whether or not the father has been to jail, FatherBreakdown = Father’s history of mental breakdowns, MotherAntisocial = Mother’s history of antisocial behavior, MotherDepressed = Mother’s history of depression, MotherSubAbuse = Mother’s history of substance abuse, MotherAnxiety = Mother’s history of anxiety, MotherJail = whether or not the mother has been to jail, MotherBreakdown = Mother’s history of mental breakdowns.

Figure 3b.

Other External Validity Measures.

Note: Beta Coefficient is the standardized regression coefficient. Betas are shown with their 95% confidence intervals. Asterisks indicate external validity measures for which the Beta Coefficient significantly differed across the three higher-order factors. HarshDiscipline = Parent’s harsh discipline, PeerDelinquency= Child’s history of peer delinquency, BirthWeight = Birth Weight, DigitSpanIQ = overall Digit Span score, DigitSpanForward = score on Digit Span Forward subtest, DigitSpanBackward = score on Digit Span Backward subtest, FamilyIncome = Family Income, MomAgeAtBirth = Mother’s age at the child’s birth, CigsSmoked = if the mother smoked cigarettes while pregnant, DrankWhilePregnant = if the mother drank while pregnant, GestAge = Child’s gestational age at birth.

Etiology of the Relations between the Higher-Order Factors and Selected External Validity Variables

We next tested the external validity of the hierarchical dimensional model by conducting a set of multivariate behavior genetic analyses to examine the etiology of the relations between the higher-order factors and the external validity variables (Peer Delinquency and Parent’s Harsh Discipline) that varied across twins within each twin pair and which showed substantial and differential relations with the General, Externalizing, and Internalizing factors. The magnitudes of the genetic and shared environmental influences were greater for Harsh Discipline (h²= .45, c² = .43, e² = .12) than for Peer Delinquency (h²= .28, c² = .27, e² = .45). We partitioned the genetic and environmental influences on the General, Externalizing, and Internalizing factors into those that are shared with Peer Delinquency or Harsh Discipline versus those that are unique to each higher-order factor. The full model for these analyses is shown in Figure 4a. In Table 3 we report the results of tests of the fit of alternative models for common and unique etiology of the higher-order factors with Peer Delinquency and Harsh Discipline. No common genetic or environmental influences between Peer Delinquency and Internalizing were estimated given that there was no phenotypic relation between the two (see Figure 3b). As shown in model 2 in Table 3, once the common shared environmental influences with Peer Delinquency were estimated, no unique shared environmental influences on either the General or Externalizing factors remained, as these could be fixed to 0. In addition, in model 3 both the common genetic influences on the General factor and the common non-shared environmental influences on the General and Externalizing factors could be dropped without any loss of fit. As shown in Figure 4b, only the Externalizing factor shared genetic influences with Peer Delinquency (.07), while both the General and Externalizing factors had common shared environmental influences with Peer Delinquency, which were higher for the General than the Externalizing factor (.33 versus .26). In model 4 we tested whether the shared environmental influences on the General and Externalizing factors differed or could be equated. The model in which these were equated fit worse, suggesting that the etiology differed across the General and Externalizing factors.

Figure 4a.

Full Cholesky Decomposition Model of the Common Etiological Influences on the External Validity Measures and the Higher-Order Factors

Note: A = Additive genetic influences, C = Shared environmental influences, E = Non-shared environmental influences, EV = External Validity Measure, G = General Factor, E = Externalizing Specific Factor, I = Internalizing Specific Factor.

Table 3.

Fit of Models for the Common Etiology of the Higher-Order Factors with Peer Delinquency and Harsh Discipline.

Model	χ2	df	TLI	RMSEA	SRMR	BIC
Peer Delinquency
1.  Full Model	1313	558	.960	.042	.056	28898
2.  No C on EXT or      General	1314	560	.960	.041	.056	28883
3.  No Common A      or E onGeneral	1311	562	.960	.041	.056	28871
4.  Equal Cs on      General & EXT	1325	563	.960	.042	.057	28883
Harsh Discipline
1.  Full Model	1049	555	.974	.034	.047	25698
2.  No C on EXT or      General	1051	557	.974	.034	.047	25681
3.  No Common      Es or As on      INT or EXT	1053	562	.975	.033	.048	25651
4.  Equal Cs on      General, INT, &      EXT	1152	564	.970	.036	.066	25758

Note: A is additive genetic influences; C is shared environmental influences; E is nonshared environmental influences; EXT is the Externalizing factor; INT is the Internalizing factor; best fitting model is in bold.

Figure 4b.

Common Etiological Influences on Peer Delinquency and the Higher-Order Factors

Note: A = Additive genetic influences, C = Shared environmental influences, E = Non-shared environmental influences, PD = Peer Delinquency, G = General Factor, E = Externalizing Specific Factor, I = Internalizing Specific Factor. Path coefficients for the genetic and environmental influences on the higher-order psychopathology factors and on Peer Delinquency are squared standardized regression coefficients (i.e., variance components). Decimal points are omitted to save space.

Similar to the results for Peer Delinquency, as shown in model 2 in Table 3, once the common shared environmental influences with Harsh Discipline were estimated, no unique shared environmental influences on either the General or Externalizing factors remained, as these could be fixed to 0. In addition, in model 3 both the common genetic influences on the Externalizing and Internalizing factors and the common non-shared environmental influences on all 3 factors could be dropped without any loss of fit. As shown in Figure 4c, only the General factor shared a small amount of genetic influences with Harsh Discipline (~.01), whereas all three higher-order dimensions had common shared environmental influences with Harsh Discipline. The magnitude of these was highest for Externalizing, intermediate for General, and lowest for Internalizing (.24, .18, and .06, respectively). In model 4 we tested whether the shared environmental influences on the General, Externalizing, and Internalizing factors differed or could be equated. The model in which these were equated fit worse, suggesting again that the etiology differed across the higher-order factors. Interestingly, all of the shared environmental influences on the General and Externalizing factors were shared in common with those on Peer Delinquency and Harsh Discipline, whereas Internalizing had substantial unique shared environmental influences (.39) after accounting for those shared in common with Harsh Discipline,

Figure 4c.

Common Etiological Influences on Harsh Discipline and the Higher-Order Factors

Note: A = Additive genetic influences, C = Shared environmental influences, E = Non-shared environmental influences, HD = Harsh Discipline, G = General Factor, E = Externalizing Specific Factor, I = Internalizing Specific Factor. Path coefficients for the genetic and environmental influences on the higher-order psychopathology factors and on Harsh Discipline are squared standardized regression coefficients (i.e., variance components). Decimal points are omitted to save space.

Discussion

There are several important findings of this study not previously addressed in prior analyses of the present sample (Lahey et al., 2011; Tackett et al., 2013) or in other studies. First, similar to previous studies (e.g., Caspi et al., 2014; Laceuelle et al., 2015; Lahey et al., 2012, 2015; Olino et al., 2014; Patalay et al., 2015) we found strong evidence for a General psychopathology factor in addition to the more expected specific Externalizing and Internalizing dimensions, and vice versa, as dropping any of the three factors significantly worsened the fit of the model. Relative to previous studies, we tested and ruled out a greater number of alternative models, thus strengthening our findings of the General factor. Second, although the Externalizing and Internalizing dimensions were moderately positively correlated in the absence of the General factor, consistent with a large psychopathology literature, these factors were negatively correlated once the General factor was included in the model. Although this correlation was small phenotypically (−.13) it was much larger (−.32) between the non-shared environmental influences on the Externalizing and Internalizing dimensions and it is noteworthy that there was no such correlation between the corresponding additive genetic influences. These findings replicate and extends previous results from the recent literature on the taxonomy of psychopathology, both in youth and in adults (Caspi et al., 2014; Lacuelle et al., 2015; Lahey et al., 2011, 2012, 2015; Olino et al., 2014; Patalay et al., 2015).

In addition to replicating previous findings on the hierarchical phenotypic structure of psychopathology, using multivariate behavior genetic analyses we extended them in several novel and important ways. First, the fact that the General factor is moderately heritable strongly suggests that it is not merely a methodological artifact of measurement error, but rather has a substantive nature that remains to be elucidated via its correlation with external validity measures. Second, while we found evidence for genetic, shared, and non-shared environmental influences on each of the three higher-order dimensions, the magnitudes of these differed substantially. Heritability was highest for Externalizing, intermediate for the General factor, and lowest for Internalizing, whereas the reverse was true for shared environmental influences. This suggests important discriminant validity among the three higher-order factors. Third, we could not drop the General factor from the Common Pathway Model and its inclusion explained substantially more common genetic variance in the 11 first-order symptom dimensions. This bolsters the external validity of the hierarchical dimensional structure of youth psychopathology - and specifically the inclusion of the General factor - that we advance in this manuscript. Fourth, it is important for models of the causal structure of psychopathology that the negative correlation between the Externalizing and Internalizing dimensions was present for non-shared environmental influences but not for genetic influences. This suggests that the environmental risk factors for Internalizing might actually be protective against Externalizing and vice versa and that they are individual-specific, rather than shared among family members.

Fifth, to further validate the higher-order General, Internalizing, and Externalizing factors we examined their relations with a set of external validity measures at the family level (e.g., parental psychopathology and adjustment), twin-pair level (e.g., gestational age), and twin-specific levels (e.g., birth weight). In general, the General and Externalizing factors showed stronger relations than the Internalizing factor. This was especially true for the parental psychopathology and adjustment measures, for which the General factor tended to be slightly more related to those measures than the Externalizing factor, whereas both factors were much more related than Internalizing. This finding replicates previous results that showed the General factor to be associated with various indices of familial psychopathology (Caspi et al., 2014). The General factor was also associated with Parent’s Harsh Discipline, similar to previous results showing its relation to caregiver maltreatment (Caspi et al., 2014), and physical or sexual abuse or caregiver neglect (Lahey et al., 2012), and was most strongly associated with Peer Delinquency.

The differential associations of Parent’s Harsh Discipline and Peer Delinquency among the higher-order factors, as well as the fact that these variables differed across twins within twin pairs, led us to examine their shared etiology with each of the higher-order factors. These novel analyses yielded some provocative findings. First, the magnitudes of shared environmental influences on the Externalizing and General factors were higher in multivariate behavior genetic analyses of these factors with Peer Delinquency and Harsh Discipline than they were when the higher-order factors were analyzed alone in the Common Pathway model. Second, 100% of the shared environmental influences on the General and Externalizing factors were shared in common with those on Peer Delinquency and Harsh Discipline, whereas this was not true for the Internalizing factor. Third, in addition to common shared environmental influences, Externalizing also shared common genetic influences with Peer Delinquency, similar to previous findings that an adolescent’s delinquency and their friends’ delinquency had common genetic influences (Rowe & Osgood, 1984). Fourth, in contrast to the results for Peer Delinquency, only the General factor shared common genetic influences with Harsh Discipline (although these were quite small in magnitude). While these findings need to be replicated, they make a unique contribution to establishing the external validity of the higher-order psychopathology dimensions.

In sum, our findings complement previous behavior genetic studies of the structure of psychopathology (Lahey et al., 2011; Pettersson et al., 2013, 2015; Tackett et al., 2013) as those studies focused primarily on identifying a General genetic factor – i.e., a general set of genes – that contributes to the etiology of individual disorders and the comorbidity among them to varying degrees. Our focus herein was different, as we instead sought to characterize the etiology and external validity of the General factor and the two specific Externalizing and Internalizing factors in a hierarchical model of the causal structure of psychopathology. Our findings also help to clarify what the General factor is and what it is not. Given its moderate heritability and association with numerous external validity measures one would be hard pressed to suggest that the General factor is merely some sort of methodological artifact. Based on the very high loadings of MDD and GAD, it seems plausible that the General factor reflects Distress or Negative Emotionality (NE) to a considerable extent, at least in our sample. Previous findings based on this sample (Tackett et al., 2013) substantiate this, as the General factor correlated substantially with Negative Emotionality, but not with Prosociality and Daring, both phenotypically and genetically. These findings suggest substantial specificity of the relation of the General factor with NE, rather than with other dispositional traits. One profitable direction of future studies of the structure of psychopathology will be to further explore the nature of the General factor of psychopathology, given that some studies of other samples (e.g., Lahey et al., 2012) have similarly found the General factor to be particularly characterized by high loadings on internalizing symptom dimensions such as MDD, dysthymia, and GAD, whereas others (e.g., Caspi et al., 2014; Laceulle et al., 2015) have found the General factor to be strongly characterized by high loadings on Thought Disorder symptom dimensions such as mania, schizophrenia, and OCD. More generally, future studies should explore the nature and interpretation of the General factor as a function of more systematic and comprehensive inclusion of disparate psychopathology dimensions.

Limitations

This study has a number of limitations that characterize many of the extant recent studies of the structure of psychopathology. First, all such studies have failed to assess all of the relevant first-order dimensions of psychopathology. This problem is compounded by the fact that different studies assessed different subsets of dimensions or diagnoses. This situation is unsurprising, given how daunting a task it would be to include all of the common and rare forms of psychopathology in a single study. Nonetheless, this poses a problem given that it limits the nature of the inferences that we can draw regarding the underlying structure of psychopathology. For example, some studies have suggested that Autism Spectrum Disorders (ASDs) primarily reflect the General factor (Pettersson et al., 2013) whereas other studies have highlighted the overlap of ASDs with ADHD symptoms (Ronald, Simonoff, Kuntsi, Asherson, & Plomin, 2008; Nijmeijer et al., 2009; Polderman et al., 2013) Given that we did not assess ASDs in our study we are unable to make any statements regarding the source of their overlap with the ADHD dimensions, nor are we able to investigate the extent to which the inclusion of ASDs might change the nature, composition, or interpretation of the General factor. The same holds true for other dimensions of rare conditions (e.g., thought disorder, schizophrenia spectrum disorders, and bipolar disorder) that have proved to be important in other such investigations (e.g., Caspi et al., 2014; Laceulle et al., 2015). A second limitation, related to the first, is that few studies of the structure of psychopathology have included participants that span the range of development from early childhood to adulthood. The age of the participants places obvious limits on the disorders that are included, with thought disorder, schizophrenia spectrum disorders, and bipolar disorder more likely included in adult samples and ADHD, ODD, and CD more likely included in childhood samples. This systematic lack of overlap of disorders places limits on what we can learn about the nature of these disorders’ overlap and the covariation among their symptoms, as well as about the complete elaboration of the General factor. Nonetheless, at the phenotypic level, findings regarding the General factor from various studies of preschool children (Olino et al., 2014) through adults (Lahey et al., 2012) have yielded very consistent findings.

A third limitation of our study is that we conducted both the CFAs and multivariate behavior genetic analyses on first-order symptom dimensions, rather than on their constituent symptoms, which would be preferable. Beginning by clustering individual symptoms into first-order symptom dimensions presumes rather than tests the first-order dimensional structure, which is disadvantageous both conceptually and statistically (Franic et al., 2013). It is thus important for future studies of the structure of psychopathology to use very large twin samples that will allow robust analyses to be conducted at the item level. Fortunately, several such twin registries exist for both child and adult twin samples (e.g., large twin registries in Australia, the Netherlands, and Sweden) in which participants have been assessed on a broad spectrum of common and rare psychopathology using the most common child and adult psychopathology assessment instruments. Use of such very large twin registries will be a necessary component of progress in this area of research.