Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Mar 1.
Published in final edited form as: Int J Methods Psychiatr Res. 2017 Oct 8;27(1):10.1002/mpr.1593. doi: 10.1002/mpr.1593

Measuring the Hierarchical General Factor Model of Psychopathology in Young Adults

Benjamin B Lahey 1, David H Zald 2, Scott F Perkins 3, Victoria Villalta-Gil 4, Katherine B Werts 5, Carol A Van Hulle 6, Paul J Rathouz 7, Brooks Applegate 8, Quetzal A Class 9, Holly E Poore 10, Ashley L Watts 11, Irwin D Waldman 12
PMCID: PMC5834349  NIHMSID: NIHMS908507  PMID: 28990308

Abstract

Objectives

There is evidence that models of psychopathology specifying a general factor and specific second-order factors fits better than competing structural models. Nonetheless, additional tests are needed to examine the generality and boundaries of the general factor model.

Methods

In a selected second wave of a cohort study, first-order dimensions of psychopathology symptoms in 499 23–31 year old twins were analyzed.

Results

Using confirmatory factor analysis (CFA), a bifactor model specifying a general factor and specific internalizing and externalizing factors fit better than competing models. Factor loadings in this model were sex invariant despite greater variances in the specific internalizing factor among females and greater variances in the general and specific externalizing factors among males. The bifactor structure was robust to the exclusion of any single first-order dimension of psychopathology. Furthermore, the results were essentially unchanged when all overlapping symptoms that define multiple disorders were excluded from symptom dimensions. Furthermore, the best-fitting bifactor model also emerged in exploratory structural equation modeling (ESEM) with freely estimated cross-loadings.

Conclusions

The general factor of psychopathology was robust across variations in measurement and analysis.

Keywords: general factor of psychopathology, p factor, hierarchical taxonomy of psychopathology

There is evidence of a robust hierarchical structure underlying psychopathology. Specifically, correlations among symptoms define first-order dimensions, and in turn, correlations among first-order dimensions define second-order internalizing (i.e., anxiety and depression) and externalizing (behavior problems and substance misuse) dimensions (Achenbach & Edelbrock, 1978; Krueger & Markon, 2006). Furthermore, we hypothesized that a useful model includes a general factor of psychopathology, which reflects positive correlations among every dimension of psychopathology (Lahey et al., 2012; Lahey et al., 2015). Based on a review of bifactor analyses of phenotypic, genetic, and environmental correlations among dimensions of psychopathology (Lahey et al., 2012; Lahey et al., 2015; Lahey, Van Hulle, Singh, Waldman, & Rathouz, 2011; Tackett et al., 2013; Waldman, Poore, Van Hulle, Rathouz, & Lahey, 2016), we proposed that the general factor quantifies a phenotype influenced by nonspecific causal factors that increase the risk for every dimension of psychopathology. Such bifactor models remove general risk from the specific internalizing and externalizing factors, making the discovery of both nonspecific and dimension-specific etiologies more tractable (Lahey et al., 2017).

Although many studies consistently show that a hierarchical model of psychopathology including a general factor fits well across the life span, there are gaps in knowledge. Nearly all studies of adults have been based on categorical diagnoses based on self-reported symptoms, whereas studies of children were based on dimensional parent ratings (Lahey et al., 2017). Given the limitations of categorical diagnoses (Krueger & Markon, 2011), we evaluated the general factor model of psychopathology dimensions in adults using counts of self-reported symptoms. We did so based on both evidence supporting the dimensionality (Helzer, Kraemer, & Krueger, 2006; Krueger & Piasecki, 2002; Pickles & Angold, 2003) of psychopathology and evidence that symptoms of psychopathology are associated with distress and functional impairment even in persons who do not meet diagnostic criteria (Copeland, Wolke, Shanahan, & Costello, 2015; Fergusson, Horwood, Ridder, & Beautrais, 2005; Roberts, Fisher, Turner, & Tang, 2015).

Given its potential importance to understanding of the nature of psychopathology, it is necessary to test the general factor model of psychopathology to determine if there are boundaries on it. We previously provided evidence that the general factor is not an artifact of measurement error and exhibits criterion validity (Lahey et al., 2017; Lahey et al., 2015; Tackett, Waldman, Van Hulle, & Lahey, 2011). Here we report the results of tests that could invalidate or delimit the general factor hypothesis depending on the findings:

1. Overlapping symptoms

Some symptoms define more than one mental disorder. These overlapping symptoms could inflate correlations among first-order dimensions of psychopathology and generate an artifactual general factor. We evaluate this possibility using first-order dimensions from which all overlapping symptoms are excluded.

2. Assumptions underlying CFA

Previous studies used CFA to extract a general factor of psychopathology and two or more specific second-order factors based on the remaining residual correlations (reviewed by Lahey et al., 2017). Based on the strong assumption that each first-order dimension loads on only one specific factor, CFA fixes the loading of each first-order dimension to zero on the specific second-order factor(s) on which it is not hypothesized to load. If each first-order dimension actually has substantial loadings on other second-order factors, however, restricting the cross-loadings to zero might create an artifactual general factor by forcing shared variance through it. We evaluate this possibility using exploratory structural equation modeling (ESEM), which freely estimates the loadings of each dimension on each factor (Asparouhov & Muthén, 2009). If a bifactor model in ESEM: (1) reveals significant cross-loadings, or (2) does not identify a general factor, or (3) does not fit better than a correlated two-factor model in ESEM, such findings would disconfirm the general factor model.

3. Sensitivity to the dimensions that define higher-order factors

Although the general factor model has been supported in studies measuring a variety of different dimensions of psychopathology (Lahey et al., 2017), it is possible that a particular first-order dimension is necessary for the extraction of the general factor. We consider this possibility by refitting the best-fitting model while omitting each first-order dimension one at a time.

Conversely, little is known about the effects of including some first-order dimensions in analyses. Data are sparse on mania and posttraumatic stress disorder (PTSD), and although there is evidence that attention-deficit/hyperactivity disorder (ADHD) is prevalent (Kessler et al., 2006) and commonly treated in adults (Reimherr, Marchant, Gift, Steans, & Wender, 2015), no previous study of the general factor in adults has included ADHD. Including these dimensions will help determine if the hypothesized general factor model can incorporate these forms of psychopathology. Furthermore, including them will further understanding of these dimensions by revealing their relations to the general factor.

4. Sex differences

It is essential to know if the general factor model applies equally to both sexes. Although a study of adolescents provided evidence of invariance in the general factor model across sex (Carragher et al., 2016) and one study supported the general factor model in girls (Lahey et al., 2015), studies of adults have not addressed sex invariance.

METHOD

Participants and Procedures

Participants were assessed during the second wave of the Tennessee Twin Study (TTS) (Lahey et al., 2008) conducted 11–15 years after wave 1.

Wave 1

The TTS sample is representative of all 6–17-year-old twins born in Tennessee and living in one of the state’s five metropolitan statistical areas (MSAs) in 2000–2001. The Tennessee Department of Health identified birth records in this age range. A random sample was selected, stratified by age and geographic subareas, proportional to the number of families in each subarea. Among 4012 selected households, 89.5% were located and screened, with 2646 of screened families being eligible (both twins co-resided with the adult caretaker at least 50% of the time during the past 6 months and the twins and caretaker spoke English). Interviews were completed with 2063 adult caretakers (90.8% biological mothers), with a 70% response rate. After exclusion of twin pairs in which either had been diagnosed with autism, psychosis, or seizure disorder, the sample consisted of 3,990 twins in 1,995 complete pairs. Caretakers classified 71% of the twins as non-Hispanic white, 24% African American, and 5% as Hispanic and other groups.

In wave 1, adult caretakers and youth were interviewed separately using the reliable and valid Child and Adolescent Psychopathology Scale (CAPS) by interviewers (Lahey et al., 2004). The CAPS queried DSM-IV symptoms of attention-deficit/hyperactivity disorder (ADHD), oppositional defiant disorder (ODD), conduct disorder (CD), major depressive disorder (MDD), and anxiety disorders during the last 12 months (Lahey et al., 2004).

Wave 2

A selected subsample of 405 pairs of 10–17 year old twins in the TTS was asked to participate in wave 2 in four replicates in reverse order of wave 1 age to narrow the age distribution in wave 2. Because wave 2 included neuroimaging at Vanderbilt University, 4.8% of twin pairs were excluded due to living >300 miles from Vanderbilt. Twins were oversampled on wave 1 CAPS psychopathology based on the greater rating of each symptom by parents or youth. High-risk pairs were selected if either twin had ratings on the total number of internalizing, attention-deficit/hyperactivity disorder, or the combination of oppositional defiant and conduct disorder in the top 10% of that age range. In addition, 19–23% of the remainder of each replicate was randomly selected with two constraints: 40% of the randomly selected dizygotic pairs in the remainder of the sample were excluded, and the number selected varied slightly to equate replicate sizes.

Forty of the 405 selected twin pairs could not be located or decline to be screened. Eighteen selected pairs were declared out of scope due to previous participation in the pilot study, mental or physical incapacity, residence outside the U.S., imprisonment, or death, resulting in 347 twin pairs. A total of 114 screened twins were excluded from neuroimaging for safety reasons, large body size, or claustrophobia, but scan-ineligible twins could complete interview assessments in person or by telephone. Of the 694 screened individual twins, 499 were interviewed in wave 2 (64.7%). Interviews were completed for 248 complete twin pairs (49.6% monozygotic; 66.9% high risk) and 3 individuals without their co-twin. Demographic characteristics of the participants are in Table 1.

Table 1.

Demographic characteristics of the sample (N=499)

Demographic variable Percent/Mean (SD)
Sex (% Female) 52.1
Race-ethnic group
  Non-Hispanic white (%) 71.5
  African American (%) 25.2
  Other groups (%) 3.2
Monozygotic twin (%) 49.5
Still in school (%) 26.0
Age
  In wave 1 (Mean, SD) 13.6 (2.5)
  In wave 2 (Mean, SD) 26.0 (1.8)
  Range in wave 2 23–31
Years of education completed (Mean, SD) 14.3 (2.3)
Open in a new tab

In wave 2, psychopathology was assessed using the Young Adult version of the Diagnostic Interview for Children (YA_DISC) (Shaffer, Fisher, Lucas, Dulcan, & Schwab-Stone, 2000; Shaffer, Fisher, Piacentini, & Lucas, 2008), which queried diagnostic criteria for ADHD, MDD, GAD, PTSD, agoraphobia, panic attacks, social phobia, specific phobia, manic episodes, antisocial personality disorder (APD), and maladaptive nicotine, alcohol, and marijuana use symptoms during the last 12 months. The YA-DISC queries nearly all symptoms in all participants. The previous study of adults that used symptom counts (Caspi et al., 2014) was able to generate less comprehensive dimensional measures of psychopathology because they used an interview with skip patterns that resulted in all of symptoms of each disorder only being queried in participants who could meet criteria for the diagnosis. Nonetheless, some skip patterns were used in the YA_DISC: Questions regarding symptoms of PTSD were only administered to participants who reported a lifetime traumatic event that they thought about during the last year. All GAD symptoms were only queried if the participant reported the cardinal symptom of frequent worry for at least 6 consecutive months during the last year. Furthermore, although all participants were asked about all symptoms of depression, contingent questions that could be used to set a threshold for the presence of each symptom based on frequency and duration were asked only for dysphoria and anhedonia. This may have resulted in higher prevalences of depression symptoms, but this was done in the same way for participants who did or did not report dysphoria or anhedonia.

Data Analyses

All analyses were weighted by the inverse of their probability of selection into wave 2, incorporated selection strata, and took clustering within twin pairs into account. Furthermore, because the loss of participants was not negligible, weights also adjusted for nonresponse within each stratum using data on demographic characteristics and wave 1 psychopathology, dispositions, and cognitive ability. This allows valid parameter estimates when weighted back to the full TTS sample (Korn & Graubard, 1999). Analyses were performed using Mplus 7.11 (Muthén & Muthén, 2013). MLR estimation accounted for any non-normality in symptom dimensions and adjusted standard errors to reflect the clustering of twins within pairs. Model fit was evaluated using the Tucker-Lewis index (TLI), comparative fit index (CFI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). The relative fit of alternative models was evaluated using differences in scaled χ2 and the Bayesian Information criterion (BIC) (Loehlin, 2004; Markon & Krueger, 2006).

CFAs of Factor Structure

We first conducted CFA models of the structure of the first-order dimensions of psychopathology to replicate previous findings (Lahey et al., 2017). Based on the previous literature (Krueger & Markon, 2006), the first two models specified two (i.e., internalizing and externalizing) or three (i.e., fears, distress, and externalizing) correlated second-order factors. The fit of these models were compared to that of a bifactor model specifying a general factor and specific internalizing and externalizing factors.

Is the General Factor an Artifact of Overlapping Symptoms?

Table 2 lists 9 symptoms used to define multiple DSM-5 mental disorders. The best-fitting correlated-factor and bifactor models were re-estimated using first-order dimensions from which these 9 overlapping symptoms were eliminated to determine if a bifactor model including a general factor still fit better than the correlated factors model and if the structure was consistent with the hypothesized general factor model.

Table 2.

Overlapping Symptoms by DSM-5 Diagnosis

MDD GAD PTSD Mania Inattention HI Agora-
phobia 		Specific
phobia
fatigue fatigue
insomnia insomnia insomnia
concentration concentration concentration inattention
distractibility distractibility
irritability irritability irritability
anhedonia anhedonia
restlessness restlessness restlessness fidget/seat/driven
pressured speech talkative
bridges/tunnels bridges
Open in a new tab

MDD = major depressive disorder; GAD = generalized anxiety disorder; PTSD = post-traumatic stress disorder; mania = manic episode; HI = hyperactivity-impulsivity

Is the General Factor an Artifact of the Assumptions of CFA?

The results of the best-fitting bifactor CFA model, in which cross-loadings of first-order dimensions on specific externalizing and internalizing factors were set to zero were compared to the results of a bifactor model conducted using ESEM. ESEM allows all cross-loadings to be estimated empirically. Thus, contrasting the fits of the same bifactor model in CFA and ESEM provides a direct test of whether the general factor is an artifact of the restrictive assumptions of CFA.

Sensitivity of the General Factor to the Dimensions Used to Test It

To determine if any of the 14 dimensions of psychopathology used in this study are necessary for the identification of the general factor, each first-order dimension was excluded one at a time from the CFA and the magnitudes and variabilities of general factor loadings for the remaining first-order symptom dimensions were examined. Furthermore, because relatively little is known about the location of dimensions of severe forms of psychopathology in the general factor model (Lahey et al., 2017), a focus was placed on identifying the most appropriate specification for mania in the best-fitting model.

Measurement and Structural Invariance by Sex

We conducted formal tests to test whether the same structural model of psychopathology applies to both sexes. We first compared the fit of the correlated-factors and bifactor CFA models separately in each sex. We next contrasted the fits of the best-fitting CFA model across sexes in a set of three progressively restrictive measurement invariance models. The configural invariance model specifies the same pattern, but allows different magnitudes, of factor loadings in both sexes. The metric invariance model further specifies that the factor loading magnitudes are equal for males and females. If the findings are consistent with metric invariance, one can further test for (a) scalar invariance, which additionally specifies equal intercepts of the first-order symptom dimensions across sex, and (b) structural invariance, which also specifies equal variances of the general, externalizing, and internalizing factors in females and males. If scalar invariance holds, one also can determine if the general, externalizing, and internalizing factor means are the same in females and males (Loehlin, 1998).

RESULTS

CFAs of First-Order Symptom Dimensions

Table 3 shows that a correlated 2-factor CFA model without a general factor (Model 1a) fit the data adequately. Each first-order dimension loaded significantly on the specific internalizing or externalizing factors. Mania loaded significantly on both externalizing (.39, p < .0001) and internalizing factors (.18, p < .05), with the correlation between these factors being r = .43, p < .0001. The correlated 3-factor Model 1b, specifying externalizing, distress (GAD, MDD, and PTSD), and fears (social phobia, specific phobia, and agoraphobia and panic disorder) factors, fit slightly less well than the two-factor model according to BIC, with the best-fitting version (Model 1b) allowing mania to load on externalizing only. Because the fears and distress factors were correlated at r = .94, the correlated 2-factor Model 1a was selected for comparisons with the bifactor model.

Table 3.

Comparisons of the Fits of Alternative Confirmatory Factor Analysis Models of the Correlational Structure of 14 Dimensions of Psychopathology

Model χ2 df CFI TLI RMSEA SRMR BIC rEXT-INT rEXT-FRS rEXT-DIS rFRS-DIS Δχ2 (df)
Correlated factor models without a general factor
1a. Two correlated factors; mania on externalizing* and internalizing
183** 75 .807 .765 .054 .072 21114 .43*
1b. Three correlated factors; mania on externalizing onlya
180** 74 .811 .768 .053 .073 21118 0.36* 0.57** 0.94**
Bifactor models (general factor and specific internalizing and externalizing factors)
2a. Mania loads on general, externalizing, and internalizing
84* 61 .959 .939 .028 .036 20994 −.03 2a vs 1a: 88.73 (13)**
2b. Mania loads on general and externalizing
84* 62 .961 .942 .027 .036 20988 −.04 2a vs 2b: 0.14 (1)
2c. Mania loads on general and internalizing
82* 62 .964 .947 .026 .036 20989 −.07 2a vs 2c: 0.23 (1)
2d. Mania loads on general only
82 63 .966 .951 .024 .036 20983 −.07 2a vs 2d: 0.37 (1)
2e. Mania loads on general only
81 64 .969 .956 .023 .036 20978 fixed to 0 2d vs 2e: 0.26 (1)
Bifactor models (general factor and specific fears, distress, and externalizing factors)
3. Mania on general onlya
82* 61 .963 .945 .026 .036 20990 −.08 −.07 0.87**
Open in a new tab

CFI = confirmatory fit index; TLI = Tucker-Lewis Index; RMSEA = root mean square error approximation; SRMR = standardized room mean square residual ; BIC = Bayesian information criterion; rEXT-INT = Pearson correlation between the latent specific internalizing and externalizing factors; rEXT-FRS = Pearson correlation between the latent specific externalizing and fears factors; rEXT-DIS = Pearson correlation between the latent specific externalizing and distress factors; Δχ2 = Satorra-Bentler difference chi-square test.

*

p < .05;

**

p < .0001

a

Mania did not load on either distress or fears in any 3-factor model.

Four bifactor models were then fitted in CFA that included a general factor and specific internalizing and externalizing factors. As required in bifactor models, correlations of the specific internalizing and externalizing factors with the general factor were set to zero (Brown, 2006). A direct comparison was possible between nested models 1a and 2a, which included a general factor and allowed mania to load on all factors (i.e., general, internalizing, and externalizing). Model 2a fit significantly better than Model 1a and all fit indices favored Model 2a (Table 3). Models 2b–2d were more parsimonious than Model 2a in allowing mania to load on fewer factors. There were no significant differences in χ2 favoring the more general Model 2a over any of Models 2b–2e, and each model fit better than Model 2a in terms of BIC and other indices (Table 3). The best-fitting of these four models was Model 2d, in which mania was specified to load on only the general factor. A version of this model (Model 2e), in which the correlation between the internalizing and externalizing factors was set to zero, had a lower BIC than Model 2d, in which the internalizing-externalizing correlation was small and nonsignificant (r = −.07).

Factor loadings for the best-fitting CFA Model 2e are shown in Figure 1. All first-order symptom dimensions loaded significantly on the general factor, except for specific phobia. All first-order internalizing dimensions loaded significantly on the specific internalizing factor, except OCD. APD and the three dimensions of substance use symptoms loaded significantly on the specific externalizing factor, whereas inattention and hyperactivity-impulsivity had robust loadings on the general factor but nonsignificant loadings on externalizing. The results of Model 3 in Table 2 showed that a bifactor model with a general factor and three specific factors fit less well than the bifactor model with two specific factors (Model 2e). Again, the correlation between fears and distress showed little separation of these factors (r = .87).

Figure 1.

Figure 1

Open in a new tab

Factor loadings for the best-fitting bifactor Model 2e (Table 3) of psychopathology dimensions in young adults. Note: All indicated loadings significant at p < .05.

Is the General Factor an Artifact of Overlapping Symptoms?

The CFA models summarized in Table 3 were repeated, except that the overlapping symptoms listed in Table 2 were not included in any of the symptom dimensions. The results were qualitatively identical with or without the overlapping symptoms. As shown in Supplemental Table S1, the best-fitting correlated 2-factor model had mania loading only on the externalizing factor (Model 3c). The best-fitting bifactor model had mania loading only on the general factor (Model 4e), with the correlation between the specific internalizing and externalizing factors set to 0. The factor loadings for the best-fitting bifactor Model 4e are shown in Figure 2.

Figure 2.

Figure 2

Open in a new tab

Factor loadings for the best-fitting bifactor (Model 4e Supplemental Table S1) of psychopathology dimensions in young adults, but with all overlapping symptoms removed from the measures of the first-order dimensions. Note: All indicated loadings significant at p < .05.

Sensitivity to the Assumptions of CFA

To ensure that the hierarchical general factor model is not an artifact of setting cross-loadings to zero in CFA, we conducted two ESEMs, including a model with bifactor rotation to define a general factor. As in CFA, the fits using ESEM of both a correlated two-factor model (χ2 = 133, df = 65, p < .05, CFI = .878, TLI = .830, BIC = 21076, RMSEA = .046, SRMR = .051) and a bifactor model with a general factor and two correlated specific factors (χ2 = 70, df = 53, p = .055, CFI = .969, TLI = .946, BIC = 21019, RMSEA = .026, SRMR = .034) were acceptable. Nonetheless, the ESEM with bifactor rotation with a general factor and two orthogonal specific factors fit better than the ESEM with two correlated factors, Δχ2 = 58.06, df = 12, p < .0001. The correlation between the two specific factors in the bifactor ESEM was negative (r = −.22), but non-significant. As shown in Figure 3 and Supplemental Table S2, the pattern of factor loadings for the ESEM with bifactor rotation was very similar to that in the best-fitting bifactor Model 2e in Figure 1. Indeed, in the bifactor ESEM none of the cross-loadings of the first-order dimensions that loaded significantly on the specific internalizing factor in CFA had a statistically significant loading on the specific externalizing factor, and vice-versa. The most notable difference was that MDD and PTSD loaded significantly on the general factor in the bifactor ESEM, but did not load significantly on the specific internalizing or externalizing factors. In contrast, in the best-fitting CFA bifactor Model 2e, MDD and PTSD had significant loadings on the general factor and on the specific internalizing factor.

Figure 3.

Figure 3

Open in a new tab

Factor loadings for the best-fitting exploratory structural equation model with bifactor rotation of psychopathology dimensions in young adults. Note: All indicated loadings significant at p < .05, except for APD on the general factor, p = .052.

Sensitivity to the Specific Dimensions that Define the General Factor

To evaluate the extent to which the general factor model might require a particular first-order dimension in the analysis, a series of CFAs based on the best-fitting bifactor Model 2e in Table 3 was conducted in which each of the 14 psychopathology dimension was excluded one at a time. Supplemental Table S3 shows that the fit of each model was similar to one another. In every model in Table S2, a clear general factor was identified on which nearly all first-order dimensions loaded significantly. Moreover, the standardized loadings of each dimension on the general factor were highly correlated across the 14 models (median r = .94, mean r = .90). Similarly, factor loadings on specific internalizing and externalizing factors varied little when each dimension was deleted. The one clear and important exception was that no first-order dimensions loaded significantly on the specific externalizing factor in the bifactor model when APD was excluded, reflecting its central importance in defining the externalizing dimension in adults.

Measurement and Structural Invariance by Sex

Before testing for measurement invariance, we fit the most informative CFA models (1a, 2d, and 2e in Table 3) separately in males (n = 239) and females (n = 260). As shown in Table S4, the fit of the correlated two-factor model was unacceptable for both sexes, with robust correlations between the externalizing and internalizing factors. In contrast, the fit of the bifactor model was acceptable and significantly better than the correlated two-factor model in both males (Δχ2 = 50.49, df = 11, p < .0001) and females (Δχ2 = 30.09, df = 11, p < .0001). Notably, in these smaller single-sex subsamples, none of the loadings on the specific externalizing factor in the bifactor model were significant in Model 2e for females, whereas were all significant in males.

The fit of the metric invariance model was better than the configural model on most fit indices (Table 4), indicating that loadings of the first-order dimensions on the higher-order factors did not vary by sex. In contrast, the scalar invariance model fit worse than both the configural and metric invariance models, suggesting sex differences in the intercepts of the first-order dimensions and ruling out tests of structural invariance of the higher-order factor means. Given metric invariance, we tested the structural invariance hypothesis of equal factor variances for all three higher-order factors, while holding factor loadings constant across sex. The metric invariance model fit better than the equal factor variances model (Table 4). We also could reject the hypothesis of equal factor variances across sex for each of the three factors considered individually. Variances for the general and specific externalizing factors were considerably lower for females (general = .55, SE = .21; externalizing = .24, SE = .16), whereas the variance of the specific internalizing factor was higher for females (2.32, SE = 1.56). Because factor variances were fixed to 1 for males, variances for females are interpreted as a proportion of those for males. Thus the variances of the general and externalizing factors in females were 55% and 24% of that in males, respectively. In stark contrast, the variance of the internalizing factor for females was 2.3 times the variance in males.

Table 4.

Fit Statistics and Comparisons of Alternative Measurement and Structural Invariance Models across Sex

Using Confirmatory Factor Analysis
Alternative Models χ2 df CFI TLI RMSEA SRMR BIC
Configural 210 128 .878 .826 .051 .056 20925
Metric 230 152 .883 .860 .045 .070 20847
Equal factor variances 250 155 .858 .833 .050 .092 20880
Scalar 282 166 .827 .810 .053 .087 20855
Comparisons of Models Using the Satorra-Bentler Difference Chi-square Test
Δχ2 df p =
1. Metric versus configural 26 24 .3285
2. Scalar versus configural 70 38 .0013a
3. Scalar versus metric 50 14 <.0001b
4. Equal factor variances versus metric 10 3 < .05c
Open in a new tab

CFI = confirmatory fit index; TLI = Tucker-Lewis Index; RMSEA = root mean square error approximation; SRMR = standardized room mean square residual ; BIC = Bayesian information criterion; configural=same factor pattern for males and females; metric=equal factor loadings for males and females; scalar=equal factor loadings and intercepts for males and females.

a

indicating worse fit of the scalar than the configural model;

b

indicating worse fit of the scalar than the metric model;

c

indicating worse fit of the equal factor variances than the metric variance model.

DISCUSSION

Using a broad range of first-order dimensions of psychopathology in young adults, we replicated previous findings reviewed by Lahey et al. (2017) that a bifactor model specifying both a general factor and specific internalizing and externalizing factors fits the data better than a correlated two-factor internalizing-externalizing model. Notably, essentially the same results were found using first-order dimensions when all overlapping symptoms that are used to define more than one mental disorder were eliminated. This argues against the possibility that the general factor of psychopathology is an artifact of the use of overlapping symptoms to define more than one mental disorder. Furthermore, very similar results were found using ESEM instead of CFA. This argues that the general factor is not an artifact of failing to allow cross-loadings of first-order dimensions on the specific internalizing and externalizing factors in CFA.

To explore potential boundary conditions for the general factor model, we included a range of first-order dimensions, including ADHD symptom dimensions, which had not previously been included in bifactor models in adults. Furthermore, the best-fitting CFA model (Model 2e in Table 3) was repeated 14 times with one first-order dimension omitted each time. Much remains to be learned, but the findings failed to identify dimensions of psychopathology that do not fit the model or a dimension that is essential to the identifcation of the general factor.

These findings have substantive implications that should be considered in future work. First, every CFA and ESEM model revealed robust loadings of the ADHD inattention and hyperactivity-impulsivity dimensions on the general factor, but the loadings of these dimensions on the externalizing factor were not significant. This differs from findings on children and adolescents, which found significant loadings on both general and externalizing factors (Lahey et al., 2017). This could reflect either developmental differences or a difference in informants. Second, CFA and ESEM models consistently showed that mania loads robustly on the general factor, but weakly on the other factors, if at all. This is confirms previous findings that mania is strongly related to the general factor (Lahey et al., 2017).

In each of the best-fitting bifactor models conducted in CFA and ESEM, the correlation between the specific internalizing and externalizing factors was small and nonsignificant, suggesting that the general factor captures the common variance shared by the internalizing and externalizing factors. This provides support for the testable hypothesis that the general factor quantifies a phenotype that reflects the nonspecific causal influences and mechanisms that are related to varying degrees to most, if not all, forms of psychopathology to varying degrees (Lahey et al., 2017). In contrast, the specific internalizing and externalizing factors—after the covariation due to the general factor is accounted for—may represent essentially orthogonal phenotypes that are influenced by the different causes and mechanisms that are shared by only the internalizing or by only the externalizing first-order dimensions (Lahey et al., 2017). These and other findings provide a foundation for future studies by providing disconfirmable hypotheses regarding a posited hierarchy of etiologies and mechanisms underlying psychopathology.

Loadings of first-order dimensions on second-order factors did not differ in magnitudes by sex, but individual differences (means and variances) in the general and externalizing factors were much greater for males, whereas the reverse was true for the internalizing factor. This argues that the same structural model of psychopathology applies to both sexes.

Limitations

The current findings are limited by the unavailability of an ideal instrument for measuring psychopathology in dimensional terms in adults. Some aspects of the present results also may have been influenced by the moderate sample size. Because we replicated previous findings regarding the relative fits of the correlated-factors and bifactor models, this does not appear to be a serious limitation. Furthermore, model comparisons using chi-square difference tests were almost always consistent with differences in BIC and other fit statistics. Nonetheless, interpretations of findings, particularly nonsignificant factor loadings, should consider the modest sample size. For example, specific phobia did not load significantly on the general factor in either CFA or ESEM in the present analyses. Because specific phobia has been found to have a modest but significant loading on the general factor in several larger studies (reviewed by Lahey et al., 2017), the present nonsignificant loadings may be due to sample size.

The modest sample size also precluded some potentially important analyses. The symptoms reported to be present by YA_DISC scoring algorithms were summed to define first-order dimensions according to DSM5 rules. The sample was too small to conduct factor analyses at the symptom level to evaluate the DSM5 dimensions of symptoms. Specifically, there were cells with 0 observations in the 2 × 2 tables for many pairs of low prevalence symptoms, which prevented the estimation of tetrachoric correlations for factor analysis. Thus, future research with larger samples is needed using instruments that address all symptoms of every major dimension of psychopathology, including personality. The use of dimensional ratings of symptoms rather than the binary presence or absence of symptoms and expansion of the item pool to include symptoms not in DSM5 may be of further value.

Future Directions

Although the present findings support the general factor model of psychopathology, they do not address the essential topic of criterion validity. The general factor model can only be said to be valid and useful if the general and specific factors arise because they have different etiologies and underlying psychobiological mechanisms and are associated with different outcomes over time (Bonifay, Lane, & Reise, 2017; Lahey et al., 2017; Waldman et al., in press). Although supportive evidence has been reported (reviewed by Lahey et al., 2017), far more such tests of the criterion validity of the general factor model are needed.

In addition, the findings of sex differences point to important topics for future research. Although sex differences in the means and variances of internalizing and externalizing psychopathology are well known, the causes of these differences are not understood. In particular, the sex difference in means and variances in the general factor of psychopathology is an important topic for future study.

Supplementary Material

Supp TableS1-4

Acknowledgments

Supported by grants R01-MH098098 from the National Institute of Mental Health and CTSA grants UL1-TR000445 and UL1-TR000430.

Footnotes

No conflicts of interest.

Contributor Information

Benjamin B. Lahey, University of Chicago

David H. Zald, Vanderbilt University

Scott F. Perkins, Vanderbilt University

Victoria Villalta-Gil, Vanderbilt University.

Katherine B. Werts, Vanderbilt University

Carol A. Van Hulle, University of Wisconsin, Madison

Paul J. Rathouz, University of Wisconsin, Madison

Brooks Applegate, Western Michigan University.

Quetzal A. Class, University of Illinois at Chicago

Holly E. Poore, Emory University

Ashley L. Watts, Emory University

Irwin D. Waldman, Emory University

References

  1. Achenbach TM, Edelbrock CS. Classification of child psychopathology: Review and analysis of empirical efforts. Psychological Bulletin. 1978;85:1275–1301. [PubMed] [Google Scholar]
  2. Asparouhov T, Muthén B. Exploratory structural equation modeling. Structural Equation Modeling. 2009;26:397–438. [Google Scholar]
  3. Bonifay W, Lane SP, Reise SP. Three concerns with applying a bifactor model as a structure of psychopathology. Clinical Psychological Science. 2017;5:184–186. [Google Scholar]
  4. Brown TA. Confirmatory factor analysis for applied research. New York: Guilford; 2006. [Google Scholar]
  5. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors. Testing structural equation models. Newbury Park, CA: Sage; 1993. pp. 136–162. [Google Scholar]
  6. Carragher N, Teesson M, Sunderland NC, Newton NC, Krueger RF, Conrod PJ, Slade T. The structure of adolescent psychopathology: A symptom-level analysis. Psychological Medicine. 2016;46:981–994. doi: 10.1017/S0033291715002470. [DOI] [PubMed] [Google Scholar]
  7. Caspi A, Houts RM, Belsky DW, Goldman-Mellor SJ, Harrington H, Israel S, Moffitt TE. The p factor: One general psychopathology factor in the structure of psychiatric disorders? Clinical Psychological Science. 2014;2:119–137. doi: 10.1177/2167702613497473. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Copeland WE, Wolke D, Shanahan L, Costello EJ. Adult functional outcomes of common childhood psychiatric problems: A prospective, longitudinal study. Jama Psychiatry. 2015;72:892–899. doi: 10.1001/jamapsychiatry.2015.0730. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Fergusson DM, Horwood LJ, Ridder EM, Beautrais AL. Subthreshold depression in adolescence and mental health outcomes in adulthood. Archives of General Psychiatry. 2005;62:66–72. doi: 10.1001/archpsyc.62.1.66. [DOI] [PubMed] [Google Scholar]
  10. Helzer JE, Kraemer HC, Krueger RF. The feasibility and need for dimensional psychiatric diagnoses. Psychological Medicine. 2006;36:1671–1680. doi: 10.1017/s003329170600821x. [DOI] [PubMed] [Google Scholar]
  11. Kessler RC, Adler L, Barkley R, Biederman J, Conners CK, Demler O, Zaslavsky AM. The prevalence and correlates of adult ADHD in the United States: Results from the National Comorbidity Survey Replication. American Journal of Psychiatry. 2006;163:716–723. doi: 10.1176/appi.ajp.163.4.716. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Korn EL, Graubard BI. Analysis of health surveys. New York: Wiley; 1999. [Google Scholar]
  13. Krueger RF, Markon KE. Reinterpreting comorbidity: A model-based approach to understanding and classifying psychopathology. Annual Review of Clinical Psychology. 2006;2:111–133. doi: 10.1146/annurev.clinpsy.2.022305.095213. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Krueger RF, Markon KE. A dimensional-spectrum model of psychopathology: Progress and opportunities. Archives of General Psychiatry. 2011;68:10–11. doi: 10.1001/archgenpsychiatry.2010.188. [DOI] [PubMed] [Google Scholar]
  15. Krueger RF, Piasecki TM. Toward a dimensional and psychometrically-informed approach to conceptualizing psychopathology. Behaviour Research and Therapy. 2002;40:485–499. doi: 10.1016/s0005-7967(02)00016-5. [DOI] [PubMed] [Google Scholar]
  16. Lahey BB, Applegate B, Hakes JK, Zald DH, Hariri AR, Rathouz PJ. Is there a general factor of prevalent psychopathology during adulthood? Journal of Abnormal Psychology. 2012;121(4):971–977. doi: 10.1037/a0028355. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Lahey BB, Applegate B, Waldman ID, Loft JD, Hankin BL, Rick J. The structure of child and adolescent psychopathology: Generating new hypotheses. Journal of Abnormal Psychology. 2004;113:358–385. doi: 10.1037/0021-843X.113.3.358. [DOI] [PubMed] [Google Scholar]
  18. Lahey BB, Krueger RF, Rathouz PJ, Waldman ID, Zald DH. A hierarchical causal taxonomy of psychopathology across the life span. Psychological Bulletin. 2017;143:142–186. doi: 10.1037/bul0000069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Lahey BB, Rathouz PJ, Applegate B, Van Hulle C, Garriock HA, Urbano RC, Waldman ID. Testing structural models of DSM-IV symptoms of common forms of child and adolescent psychopathology. Journal of Abnormal Child Psychology. 2008;36:187–206. doi: 10.1007/s10802-007-9169-5. [DOI] [PubMed] [Google Scholar]
  20. Lahey BB, Rathouz PJ, Keenan K, Stepp SD, Loeber R, Hipwell AE. Criterion validity of the general factor of psychopathology in a prospective study of girls. Journal of Child Psychology and Psychiatry. 2015;4:415–422. doi: 10.1111/jcpp.12300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Lahey BB, Van Hulle CA, Singh AL, Waldman ID, Rathouz PJ. Higher-order genetic and environmental structure of prevalent forms of child and adolescent psychopathology. Archives of General Psychiatry. 2011;68:181–189. doi: 10.1001/archgenpsychiatry.2010.192. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Loehlin JC. Latent variable models: An introduction to factor, path, and structural equation analysis. Third. Hillsdale, NJ: Lawrence Erlbaum Associates; 1998. [Google Scholar]
  23. Muthén B, Muthén L. Mplus 7.1. Los Angeles: Muthén & Muthén; 2013. [Google Scholar]
  24. Pickles A, Angold A. Natural categories or fundamental dimensions: On carving nature at the joints and the rearticulation of psychopathology. Development and Psychopathology. 2003;15:529–551. doi: 10.1017/s0954579403000282. [DOI] [PubMed] [Google Scholar]
  25. Reimherr FW, Marchant BK, Gift TE, Steans TA, Wender PH. Types of adult attention-deficit hyperactivity disorder (ADHD): baseline characteristics, initial response, and long-term response to treatment with methylphenidate. Adhd-Attention Deficit and Hyperactivity Disorders. 2015;7:115–128. doi: 10.1007/s12402-015-0176-z. [DOI] [PubMed] [Google Scholar]
  26. Roberts RE, Fisher PW, Turner JB, Tang M. Estimating the burden of psychiatric disorders in adolescence: the impact of subthreshold disorders. Social Psychiatry and Psychiatric Epidemiology. 2015;50:397–406. doi: 10.1007/s00127-014-0972-3. [DOI] [PubMed] [Google Scholar]
  27. Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME. NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): Description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry. 2000;39:28–38. doi: 10.1097/00004583-200001000-00014. [DOI] [PubMed] [Google Scholar]
  28. Shaffer D, Fisher P, Piacentini JC, Lucas C. Young Adult Diagnostic Interview Schedule for Children. New York: Columbia University; 2008. [Google Scholar]
  29. Tackett JL, Lahey BB, Van Hulle CA, Waldman ID, Krueger RF, Rathouz PJ. Common genetic influences on negative emotionality and a general psychopathology factor in childhood and adolescence. Journal of Abnormal Psychology. 2013;122:1142–1153. doi: 10.1037/a0034151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Tackett JL, Waldman ID, Van Hulle CA, Lahey BB. Shared genetic influences on negative emotionality and major depression/conduct disorder comorbidity. Journal of the American Academy of Child and Adolescent Psychiatry. 2011;50:818–827. doi: 10.1016/j.jaac.2011.05.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Waldman ID, Poore H, Van Hulle C, Rathouz PJ, Lahey BB. External validity of a hierarchical dimensional model of child and adolescent psychopathology: Tests using confirmatory factor analyses and multivariate behavior genetic analyses. Journal of Abnormal Psychopathology. 2016 doi: 10.1037/abn0000183. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supp TableS1-4
NIHMS908507-supplement-Supp_TableS1-4.docx (26.5KB, docx)

RESOURCES