Measuring the hierarchical general factor model of psychopathology in young adults

Abstract

There is evidence that models of psychopathology specifying a general factor and specific second‐order factors fit better than competing structural models. Nonetheless, additional tests are needed to examine the generality and boundaries of the general factor model. In a selected second wave of a cohort study, first‐order dimensions of psychopathology symptoms in 499 23‐ to 31‐year‐old twins were analyzed. Using confirmatory factor analysis, a bifactor model specifying a general factor and specific internalizing and externalizing factors fit better than competing models. Factor loadings in this model were sex invariant despite greater variances in the specific internalizing factor among females and greater variances in the general and specific externalizing factors among males. The bifactor structure was robust to the exclusion of any single first‐order dimension of psychopathology. Furthermore, the results were essentially unchanged when all overlapping symptoms that define multiple disorders were excluded from symptom dimensions. Furthermore, the best‐fitting bifactor model also emerged in exploratory structural equation modeling with freely estimated cross‐loadings. The general factor of psychopathology was robust across variations in measurement and analysis.

1. INTRODUCTION

There is evidence of a robust hierarchical structure underlying psychopathology. Specifically, correlations among symptoms define first‐order dimensions, and in turn, correlations among first‐order dimensions define second‐order internalizing (i.e., anxiety and depression) and externalizing (behavior problems and substance misuse) dimensions (Achenbach & Edelbrock, 1978; Krueger & Markon, 2006). Furthermore, we hypothesized that a useful model includes a general factor of psychopathology, which reflects the positive correlations among every dimension of psychopathology (Lahey et al., 2012; Lahey et al., 2015). Based on a review of bifactor analyses of phenotypic, genetic, and environmental correlations among dimensions of psychopathology (Lahey et al., 2012; Lahey et al., 2015; Lahey, Van Hulle, Singh, Waldman, & Rathouz, 2011; Tackett et al., 2013; Waldman, Poore, Van Hulle, Rathouz, & Lahey, 2016), we proposed that the general factor quantifies a phenotype influenced by nonspecific causal factors that increase the risk for every dimension of psychopathology. Such bifactor models remove general risk from the specific internalizing and externalizing factors, making the discovery of both nonspecific and dimension‐specific etiologies more tractable (Lahey, Krueger, Rathouz, Waldman, & Zald, 2017).

Although many studies consistently show that a hierarchical model of psychopathology including a general factor fits well across the life span, there are gaps in knowledge. Nearly all studies of adults have been based on categorical diagnoses based on self‐reported symptoms, whereas studies of children were based on dimensional parent ratings (Lahey et al., 2017). Given the limitations of categorical diagnoses (Krueger & Markon, 2011), we evaluated the general factor model of psychopathology dimensions in adults using counts of self‐reported symptoms. We did so based on both evidence supporting the dimensionality (Helzer, Kraemer, & Krueger, 2006; Krueger & Piasecki, 2002; Pickles & Angold, 2003) of psychopathology and evidence that symptoms of psychopathology are associated with distress and functional impairment even in persons who do not meet diagnostic criteria (Copeland, Wolke, Shanahan, & Costello, 2015; Fergusson, Horwood, Ridder, & Beautrais, 2005; Roberts, Fisher, Turner, & Tang, 2015).

Given its potential importance to understanding of the nature of psychopathology, it is necessary to test the general factor model of psychopathology to determine if there are boundaries on it. We previously provided evidence that the general factor is not an artifact of measurement error and exhibits criterion validity (Lahey et al., 2017; Lahey et al., 2015; Tackett, Waldman, Van Hulle, & Lahey, 2011). Here, we report the results of tests that could invalidate or delimit the general factor hypothesis depending on the findings:

1. Overlapping symptoms. Some symptoms define more than one mental disorder. These overlapping symptoms could inflate correlations among first‐order dimensions of psychopathology and generate an artifactual general factor. We evaluate this possibility using redefined first‐order dimensions from which all overlapping symptoms are excluded.

2. Assumptions underlying confirmatory factor analysis (CFA). Previous studies used CFA to extract a general factor of psychopathology and two or more specific second‐order factors based on the remaining residual correlations (reviewed by Lahey et al., 2017). Based on the strong assumption that each first‐order dimension loads on only one specific factor, CFA fixes the loading of each first‐order dimension to zero on the specific second‐order factor(s) on which it is not hypothesized to load. If each first‐order dimension actually has substantial loadings on other second‐order factors, however, restricting the cross‐loadings to zero might create an artifactual general factor by forcing shared variance through it. We evaluate this possibility using exploratory structural equation modeling (ESEM), which freely estimates the loadings of each dimension on each factor (Asparouhov & Muthén, 2009). If a bifactor model in ESEM (a) reveals significant cross‐loadings, (b) does not identify a general factor, or (c) does not fit better than a correlated two‐factor model in ESEM, such findings would disconfirm the general factor model.

3. Sensitivity to the dimensions that define higher order factors. Although the general factor model has been supported in studies measuring a variety of different dimensions of psychopathology (Lahey et al., 2017), it is possible that a particular first‐order dimension is necessary for the extraction of the general factor. We consider this possibility by refitting the best‐fitting model while omitting each first‐order dimension one at a time.

Conversely, little is known about the effects of including some first‐order dimensions in analyses. Data are sparse on mania and posttraumatic stress disorder (PTSD), and although there is evidence that attention deficit hyperactivity disorder (ADHD) is prevalent (Kessler et al., 2006) and commonly treated in adults (Reimherr, Marchant, Gift, Steans, & Wender, 2015), no previous study of the general factor in adults has included ADHD. Including these dimensions will help determine if the hypothesized general factor model can incorporate these forms of psychopathology. Furthermore, including them will further understanding of these dimensions by revealing their relations to the general factor.

4. Sex differences. It is essential to know if the general factor model applies equally to both sexes. Although a study of adolescents provided evidence of invariance in the general factor model across sex (Carragher et al., 2016) and one study supported the general factor model in girls (Lahey et al., 2015), studies of adults have not addressed sex invariance.

2. METHOD

2.1. Participants and procedures

Participants were assessed during the second wave of the Tennessee Twin Study (TTS) (Lahey et al., 2008) conducted 11–15 years after Wave 1.

2.1.1. Wave 1

The TTS sample is representative of all 6‐ to 17‐year‐old twins born in Tennessee and living in one of the state's five metropolitan statistical areas in 2000–2001. The Tennessee Department of Health identified birth records in this age range. A random sample was selected, stratified by age and geographic subareas, proportional to the number of families in each subarea. Among 4,012 selected households, 89.5% were located and screened, with 2,646 of screened families being eligible (both twins coresided with the adult caretaker at least 50% of the time during the past 6 months and the twins and caretaker spoke English). Interviews were completed with 2,063 adult caretakers (90.8% biological mothers), with a 70% response rate. After exclusion of twin pairs in which either had been diagnosed with autism, psychosis, or seizure disorder, the sample consisted of 3,990 twins in 1,995 complete pairs. Caretakers classified 71% of the twins as non‐Hispanic white, 24% African American, and 5% as Hispanic and other groups.

In Wave 1, adult caretakers and youth were interviewed separately using the reliable and valid Child and Adolescent Psychopathology Scale (CAPS) by interviewers (Lahey et al., 2004). The CAPS queried Diagnostic and Statistical Manual of Mental Disorders (DSM)‐IV symptoms of ADHD, oppositional defiant disorder, conduct disorder, major depressive disorder (MDD), and anxiety disorders during the last 12 months (Lahey et al., 2004).

2.1.2. Wave 2

A selected sub‐sample of 405 pairs of 10‐ to 17‐year‐old twins in the TTS was asked to participate in Wave 2 in four replicates in reverse order of Wave 1 age to narrow the age distribution in Wave 2. Because Wave 2 included neuroimaging at Vanderbilt University, 4.8% of twin pairs were ineligible due to living >300 miles from Vanderbilt. Twins were oversampled on Wave 1 CAPS psychopathology based on the greater rating of each symptom by parents or youth. High‐risk pairs were selected if either twin had ratings on the total number of internalizing, ADHD, or the combination of oppositional defiant and conduct disorder in the top 10% of that age range. In addition, 19–23% of the remainder of each replicate was randomly selected with two constraints: 40% of the randomly selected dizygotic pairs in the remainder of the sample were excluded, and the number selected varied slightly to equate replicate sizes.

Forty of the 405 selected twin pairs could not be located or decline to be screened. Eighteen selected pairs were declared out of scope due to previous participation in the pilot study, mental or physical incapacity, residence outside the United States, imprisonment, or death, resulting in 347 twin pairs. One hundred fourteen screened twins were excluded from neuroimaging for safety reasons, large body size, or claustrophobia, but scan‐ineligible twins could complete interview assessments in person or by telephone. Of the 694 screened individual twins, 499 were interviewed in Wave 2 (64.7%). Interviews were completed for 248 complete twin pairs (49.6% monozygotic; 66.9% high risk) and three individuals without their cotwin. Demographic characteristics of the participants are in Table 1.

Table 1.

Demographic characteristics of the sample (N = 499)

Demographic variable	Percent/mean (SD)
Sex (% female)	52.1
Race‐ethnic group
Non‐Hispanic white (%)	71.5
African American (%)	25.2
Other groups (%)	3.2
Monozygotic twin (%)	49.5
Still in school (%)	26.0
Age
In Wave 1 (mean, SD)	13.6 (2.5)
In Wave 2 (mean, SD)	26.0 (1.8)
Range in Wave 2	23–31
Years of education completed (mean, SD)	14.3 (2.3)

Note. SD = standard deviation.

In Wave 2, psychopathology was assessed using the Young Adult version of the Diagnostic Interview for Children (YA_DISC; Shaffer, Fisher, Lucas, Dulcan, & Schwab‐Stone, 2000; Shaffer, Fisher, Piacentini, & Lucas, 2008), which queried diagnostic criteria for ADHD, MDD, generalized anxiety disorder (GAD), PTSD, agoraphobia, panic attacks, social phobia, specific phobia, manic episodes, antisocial personality disorder (APD), and maladaptive nicotine, alcohol, and marijuana use symptoms during the last 12 months. The YA_DISC queries nearly all symptoms in all participants. The previous study of adults that used symptom counts (Caspi et al., 2014) was able to generate less comprehensive dimensional measures of psychopathology because they used an interview with skip patterns that resulted in all of symptoms of some disorders only being queried in participants who could meet criteria for the diagnosis. Nonetheless, some skip patterns were used in the YA_DISC: Questions regarding symptoms of PTSD were only administered to participants who reported a lifetime traumatic event that they thought about during the last year. All GAD symptoms were only queried if the participant reported the cardinal symptom of frequent worry for at least 6 consecutive months during the last year. Furthermore, although all participants were asked about all symptoms of depression, contingent questions that could be used to set a threshold for the presence of each symptom based on frequency and duration were asked only for dysphoria and anhedonia. This may have resulted in higher prevalences of depression symptoms, but this was done in the same way for participants who did or did not report dysphoria or anhedonia.

2.2. Data analyses

All analyses were weighted by the inverse of their probability of selection into Wave 2, incorporated selection strata, and took clustering within twin pairs into account. Furthermore, because the loss of participants was not negligible, weights also adjusted for nonresponse within each stratum using data on demographic characteristics and Wave 1 psychopathology, dispositions, and cognitive ability. This allows valid parameter estimates when weighted back to the full TTS sample (Korn & Graubard, 1999). Analyses were performed using Mplus 7.11 (Muthén & Muthén, 2013). Maximum‐likelihood estimation using robust standard errors (MLR) accounted for any nonnormality in symptom dimensions and adjusted standard errors to reflect the clustering of twins within pairs. Model fit was evaluated using the Tucker‐Lewis index (TLI), comparative fit index (CFI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). The relative fit of alternative models was evaluated using differences in scaled χ² and the Bayesian information criterion (BIC; Loehlin, 2004; Markon & Krueger, 2006).

2.2.1. CFAs of factor structure

We first conducted CFA models of the structure of the first‐order dimensions of psychopathology to replicate previous findings (Lahey et al., 2017). Based on the previous literature (Krueger & Markon, 2006), the first two models specified two (i.e., internalizing and externalizing) or three (i.e., fears, distress, and externalizing) correlated second‐order factors. The fit of these models were compared to that of a bifactor model specifying a general factor and specific internalizing and externalizing factors.

2.2.2. Is the general factor an artifact of overlapping symptoms?

Table 2 lists nine symptoms used to define multiple DSM‐5 mental disorders. The best‐fitting correlated‐factor and bifactor models were reestimated using redefined first‐order dimensions from which these nine overlapping symptoms were eliminated to determine if a bifactor model including a general factor still fit better than the correlated‐factors model and if the structure was consistent with the hypothesized general factor model.

Table 2.

Overlapping symptoms by DSM‐5 diagnosis

MDD	GAD	PTSD	Mania	Inattention	HI	Agoraphobia	Specific phobia
Fatigue	Fatigue
Insomnia	Insomnia	Insomnia
Concentration	Concentration	Concentration		Concentration
			Distractibility	Distractibility
	Irritability	Irritability	Irritability
Anhedonia		Anhedonia
Restlessness	Restlessness		Restlessness		Fidget/restlessness
			Pressured speech		Talkative
						Bridges/tunnels	Bridges

Note. MDD = major depressive disorder; GAD = generalized anxiety disorder; PTSD = posttraumatic stress disorder; mania = manic episode; HI = hyperactivity‐impulsivity. DSM = Diagnostic and Statistical Manual of Mental Disorders.

2.2.3. Is the general factor an artifact of the assumptions of CFA?

The results of the best‐fitting bifactor CFA model, in which cross‐loadings of first‐order dimensions on specific externalizing and internalizing factors were set to zero, were compared to the results of a bifactor model conducted using ESEM. ESEM allows all cross‐loadings to be estimated empirically. Thus, contrasting the fits of the same bifactor model in CFA and ESEM provides a direct test of whether the general factor is an artifact of the restrictive assumptions of CFA.

2.2.4. Sensitivity of the general factor to the dimensions used to test it

To determine if any of the 14 dimensions of psychopathology used in this study are necessary for the identification of the general factor, each first‐order dimension was excluded one at a time from the CFA, and the magnitudes and variabilities of general factor loadings for the remaining first‐order symptom dimensions were examined. Furthermore, because relatively little is known about the location of dimensions of severe forms of psychopathology in the general factor model (Lahey et al., 2017), a focus was placed on identifying the most appropriate specification for mania in the best‐fitting model.

2.2.5. Measurement and structural invariance by sex

We conducted formal tests to test whether the same structural model of psychopathology applies to both sexes. We first compared the fit of the correlated‐factors and bifactor CFA models separately in each sex. We next contrasted the fits of the best‐fitting CFA model across sexes in a set of three progressively restrictive measurement invariance models. The configural invariance model specifies the same pattern but allows different magnitudes of factor loadings in both sexes. The metric invariance model further specifies that the factor loading magnitudes are equal for males and females. If the findings are consistent with metric invariance, one can further test for (a) scalar invariance, which additionally specifies equal intercepts of the first‐order symptom dimensions across sex, and (b) structural invariance, which also specifies equal variances of the general, externalizing, and internalizing factors in females and males. If scalar invariance holds, one also can determine if the general, externalizing, and internalizing factor means are the same in females and males (Loehlin, 1998).

3. RESULTS

3.1. CFAs of first‐order symptom dimensions

Table 3 shows that a correlated two‐factor CFA model without a general factor (Model 1a) fit the data adequately. Each first‐order dimension loaded significantly on the specific internalizing or externalizing factors. Mania loaded significantly on both externalizing (.39, p < .0001) and internalizing factors (.18, p < .05), with the correlation between these factors being r = .43, p < .0001. The correlated three‐factor Model 1b, specifying externalizing, distress (GAD, MDD, and PTSD) and fears (social phobia, specific phobia, and agoraphobia and panic disorder) factors, fit slightly less well than the two‐factor model according to BIC, with the best‐fitting version (Model 1b) allowing mania to load on externalizing only. Because the fears and distress factors were correlated at r = .94, the correlated two‐factor Model 1a was selected for comparisons with the bifactor model.

Table 3.

Comparisons of the fits of alternative confirmatory factor analysis models of the correlational structure of 14 dimensions of psychopathology

Model	χ2	df	CFI	TLI	RMSEA	SRMR	BIC	rEXT‐INT	rEXT‐FRS	rEXT‐DIS	rFRS‐DIS	Δχ2 (df)
Correlated factor models without a general factor
1a. Two correlated factors; mania on externalizing* and internalizing
	183 **	75	.807	.765	.054	.072	21,114	.43 *
1b. Three correlated factors; mania on externalizing onlya
	180**	74	.811	.768	.053	.073	21,118		0.36*	0.57**	0.94**
Bifactor models (general factor and specific internalizing and externalizing factors)
2a. Mania loads on general, externalizing, and internalizing
	84*	61	.959	.939	.028	.036	20,994	−.03				2a versus 1a: 88.73 (13)**
2b. Mania loads on general and externalizing
	84*	62	.961	.942	.027	.036	20,988	−.04				2a versus 2b: 0.14 (1)
2c. Mania loads on general and internalizing
	82*	62	.964	.947	.026	.036	20,989	−.07				2a versus 2c: 0.23 (1)
2d. Mania loads on general only
	82	63	.966	.951	.024	.036	20,983	−.07				2a versus 2d: 0.37 (1)
2e. Mania loads on general only
	81	64	.969	.956	.023	.036	20,978	Fixed to 0				2d versus 2e: 0.26 (1)
Bifactor models (general factor and specific fears, distress, and externalizing factors)
3. Mania on general onlya
	82 *	61	.963	.945	.026	.036	20,990	−.08	−.07	0.87 **

Note. CFI = confirmatory fit index; TLI = Tucker‐Lewis index; RMSEA = root mean square error approximation; SRMR = standardized root mean square residual; BIC = Bayesian information criterion; r_EXT‐INT = Pearson correlation between the latent specific internalizing and externalizing factors; r_EXT‐FRS = Pearson correlation between the latent specific externalizing and fears factors; r_EXT‐DIS = Pearson correlation between the latent specific externalizing and distress factors; Δχ² = Satorra‐Bentler difference chi‐square test; df = degrees of freedom; best‐fitting models of each type are in bold.

^aMania did not load on either distress or fears in any three‐factor model.

^* p < .05.

^** p < .0001.

Four bifactor models were then fitted in CFA that included a general factor and specific internalizing and externalizing factors. As required in bifactor models, correlations of the specific internalizing and externalizing factors with the general factor were set to zero (Brown, 2006). A direct comparison was possible between nested Models 1a and 2a, which included a general factor and allowed mania to load on all factors (i.e., general, internalizing, and externalizing). Model 2a fit significantly better than Model 1a, and all fit indices favored Model 2a (Table 3). Models 2b–d were more parsimonious than Model 2a in allowing mania to load on fewer factors. There were no significant differences in χ² favoring the more general Model 2a over any of Models 2b–e, and each model fit better than Model 2a in terms of BIC and other indices (Table 3). The best fitting of these four models was Model 2d, in which mania was specified to load on only the general factor. A version of this model (Model 2e), in which the correlation between the internalizing and externalizing factors was set to zero, had a lower BIC than Model 2d, in which the internalizing–externalizing correlation was small and nonsignificant (r = −.07).

Factor loadings for the best‐fitting CFA Model 2e are shown in Figure 1. All first‐order symptom dimensions loaded significantly on the general factor, except for specific phobia. All first‐order internalizing dimensions loaded significantly on the specific internalizing factor, except obsessive‐compulsive disorder. APD and the three dimensions of substance use symptoms loaded significantly on the specific externalizing factor, whereas inattention and hyperactivity‐impulsivity had robust loadings on the general factor but nonsignificant loadings on externalizing. The results of Model 3 in Table 2 showed that a bifactor model with a general factor and three specific factors fit less well than the bifactor model with two specific factors (Model 2e). Again, the correlation between fears and distress showed little separation of these factors (r = .87).

Figure 1.

Factor loadings for the best‐fitting bifactor Model 2e (Table 3) of psychopathology dimensions in young adults. Note: All indicated loadings significant at p < .05. Inatt = inattention; HI = hyperactivity‐impulsivity; MJ = marijuana; NIC = nicotine; ALC = alcohol; APD = antisocial personality disorder; Mania = manic episode; MDD = major depressive disorder; GAD = generalized anxiety disorder; PTSD = posttraumatic stress disorder; SoPh = social phobia; SpPh = specific phobia; Ag/Panic = agoraphobia/panic attacks; OCD = obsessive‐compulsive disorder

3.2. Is the general factor an artifact of overlapping symptoms?

The CFA models summarized in Table 3 were repeated, except that the overlapping symptoms listed in Table 2 were not included in any of the redefined symptom dimensions. The results were qualitatively identical with or without the overlapping symptoms. As shown in Table S1, the best‐fitting correlated two‐factor model had mania loading only on the externalizing factor (Model 3c). The best‐fitting bifactor model had mania loading only on the general factor (Model 4e), with the correlation between the specific internalizing and externalizing factors set to zero. The factor loadings for the best‐fitting bifactor Model 4e are shown in Figure 2.

Figure 2.

Factor loadings for the best‐fitting bifactor (Model 4e Table S1) of psychopathology dimensions in young adults but with all overlapping symptoms removed from the measures of the first‐order dimensions. Note: All indicated loadings significant at p < .05. Inatt = inattention; HI = hyperactivity‐impulsivity; MJ = marijuana; NIC = nicotine; ALC = alcohol; APD = antisocial personality disorder; Mania = manic episode; MDD = major depressive disorder; GAD = generalized anxiety disorder; PTSD = posttraumatic stress disorder; SoPh = social phobia; SpPh = specific phobia; Ag/Panic = agoraphobia/panic attacks; OCD = obsessive‐compulsive disorder

3.3. Sensitivity to the assumptions of CFA

To ensure that the hierarchical general factor model is not an artifact of setting cross‐loadings to zero in CFA, we conducted two ESEMs, including a model with bifactor rotation to define a general factor. As in CFA, the fits using ESEM of both a correlated two‐factor model (χ² = 133, degrees of freedom [df] = 65, p < .05, CFI = .878, TLI = .830, BIC = 21076, RMSEA = .046, SRMR = .051) and a bifactor model with a general factor and two correlated specific factors (χ² = 70, df = 53, p = .055, CFI = .969, TLI = .946, BIC = 21019, RMSEA = .026, SRMR = .034) were acceptable. Nonetheless, the ESEM with bifactor rotation with a general factor and two orthogonal specific factors fit better than the ESEM with two correlated factors, Δχ² = 58.06, df = 12, p < .0001. The correlation between the two specific factors in the bifactor ESEM was negative (r = −.22) but nonsignificant. As shown in Figure 3 and Table S2, the pattern of factor loadings for the ESEM with bifactor rotation was very similar to that in the best‐fitting bifactor Model 2e in Figure 1. Indeed, in the bifactor ESEM, none of the cross‐loadings of the first‐order dimensions that loaded significantly on the specific internalizing factor in CFA had a statistically significant loading on the specific externalizing factor and vice versa. The most notable difference was that MDD and PTSD loaded significantly on the general factor in the bifactor ESEM but did not load significantly on the specific internalizing or externalizing factors. In contrast, in the best‐fitting CFA bifactor Model 2e, MDD and PTSD had significant loadings on the general factor and on the specific internalizing factor.

Figure 3.

Factor loadings for the best‐fitting exploratory structural equation model with bifactor rotation of psychopathology dimensions in young adults. Note: All indicated loadings significant at p < .05, except for APD on the general factor, p = .052. Inatt = inattention; HI = hyperactivity‐impulsivity; MJ = marijuana; NIC = nicotine; ALC = alcohol; APD = antisocial personality disorder; Mania = manic episode; MDD = major depressive disorder; GAD = generalized anxiety disorder; PTSD = posttraumatic stress disorder; SoPh = social phobia; SpPh = specific phobia; Ag/Panic = agoraphobia/panic attacks; OCD = obsessive‐compulsive disorder

3.4. Sensitivity to the specific dimensions that define the general factor

To evaluate the extent to which the general factor model might require a particular first‐order dimension in the analysis, a series of CFAs based on the best‐fitting bifactor Model 2e in Table 3 were conducted in which each of the 14 psychopathology dimension was excluded one at a time. Table S3 shows that the fit of each model was similar to one another. In every model in Table S2, a clear general factor was identified on which nearly all first‐order dimensions loaded significantly. Moreover, the standardized loadings of each dimension on the general factor were highly correlated across the 14 models (median r = .94, mean r = .90). Similarly, factor loadings on specific internalizing and externalizing factors varied little when each dimension was deleted. The one clear and important exception was that no first‐order dimensions loaded significantly on the specific externalizing factor in the bifactor model when APD was excluded, reflecting its central importance in defining the externalizing dimension in adults.

3.5. Measurement and structural invariance by sex

Before testing for measurement invariance, we fit the most informative CFA models (1a, 2d, and 2e in Table 3) separately in males (n = 239) and females (n = 260). As shown in Table S4, the fit of the correlated two‐factor model was unacceptable for both sexes, with robust correlations between the externalizing and internalizing factors. In contrast, the fit of the bifactor model was acceptable and significantly better than the correlated two‐factor model in both males (Δχ² = 50.49, df = 11, p < .0001) and females (Δχ² = 30.09, df = 11, p < .0001). Notably, in these smaller single‐sex sub‐samples, none of the loadings on the specific externalizing factor in the bifactor model were significant in Model 2e for females whereas were all significant in males.

The fit of the metric invariance model was better than the configural model on most fit indices (Table 4), indicating that loadings of the first‐order dimensions on the higher order factors did not vary by sex. In contrast, the scalar invariance model fit worse than both the configural and metric invariance models, suggesting sex differences in the intercepts of the first‐order dimensions and ruling out tests of structural invariance of the higher order factor means. Given metric invariance, we tested the structural invariance hypothesis of equal factor variances for all three higher order factors, while holding factor loadings constant across sex. The metric invariance model fit better than the equal factor variances model (Table 4). We also could reject the hypothesis of equal factor variances across sex for each of the three factors considered individually. Variances for the general and specific externalizing factors were considerably lower for females (general = .55, standard error [SE] = .21; externalizing = .24, SE = .16), whereas the variance of the specific internalizing factor was higher for females (2.32, SE = 1.56). Because factor variances were fixed to 1 for males, variances for females are interpreted as a proportion of those for males. Thus, the variances of the general and externalizing factors in females were 55% and 24% of that in males, respectively. In stark contrast, the variance of the internalizing factor for females was 2.3 times the variance in males.

Table 4.

Fit statistics and comparisons of alternative measurement and structural invariance models across sex using confirmatory factor analysis

Alternative models	χ2	df	CFI	TLI	RMSEA	SRMR	BIC
Configural	210	128	.878	.826	.051	.056	20,925
Metric	230	152	.883	.860	.045	.070	20,847
Equal factor variances	250	155	.858	.833	.050	.092	20,880
Scalar	282	166	.827	.810	.053	.087	20,855
Comparisons of models using the Satorra‐Bentler difference chi‐square test
	Δχ2	df	p
1. Metric versus configural	26	24	.3285
2. Scalar versus configural	70	38	.0013a
3. Scalar versus metric	50	14	<.0001b
4. Equal factor variances versus metric	10	3	<.05c

Note. CFI = confirmatory fit index; TLI = Tucker‐Lewis index; RMSEA = root mean square error approximation; SRMR = standardized root mean square residual; BIC = Bayesian information criterion; configural = same factor pattern for males and females; metric = equal factor loadings for males and females; scalar = equal factor loadings and intercepts for males and females; df = degrees of freedom.

^aIndicating worse fit of the scalar than the configural model.

^bIndicating worse fit of the scalar than the metric model.

^cIndicating worse fit of the equal factor variances than the metric variance model.

4. DISCUSSION

Using a broad range of first‐order dimensions of psychopathology in young adults, we replicated previous findings reviewed by Lahey et al. (2017) that a bifactor model specifying both a general factor and specific internalizing and externalizing factors fits the data better than a correlated two‐factor internalizing–externalizing model. Notably, essentially the same results were found using first‐order dimensions when all overlapping symptoms that are used to define more than one mental disorder were eliminated. This argues against the possibility that the general factor of psychopathology is an artifact of the use of overlapping symptoms to define more than one mental disorder. Furthermore, very similar results were found using ESEM instead of CFA. This argues that the general factor is not an artifact of failing to allow cross‐loadings of first‐order dimensions on the specific internalizing and externalizing factors in CFA.

To explore potential boundary conditions for the general factor model, we included a range of first‐order dimensions, including ADHD symptom dimensions, which had not previously been included in bifactor models in adults. Furthermore, the best‐fitting CFA model (Model 2e in Table 3) was repeated 14 times with one first‐order dimension omitted each time. Much remains to be learned, but the findings failed to identify dimensions of psychopathology that do not fit the model or a dimension that is essential to the identification of the general factor.

These findings have substantive implications that should be considered in future work. First, every CFA and ESEM model revealed robust loadings of the ADHD inattention and hyperactivity‐impulsivity dimensions on the general factor, but the loadings of these dimensions on the externalizing factor were not significant. This differs from findings on children and adolescents, which found significant loadings on both general and externalizing factors (Lahey et al., 2017). This could reflect either developmental differences or the difference in informants. Second, CFA and ESEM models consistently showed that mania loads robustly on the general factor but weakly on the other factors if at all. This confirms previous findings that mania is strongly related to the general factor (Lahey et al., 2017).

In each of the best‐fitting bifactor models conducted in CFA and ESEM, the correlation between the specific internalizing and externalizing factors was small and nonsignificant, suggesting that the general factor captures the common variance shared by the internalizing and externalizing factors. This provides support for the testable hypothesis that the general factor quantifies a phenotype that reflects the nonspecific causal influences and mechanisms that are related to varying degrees to most, if not all, forms of psychopathology to varying degrees (Lahey et al., 2017). In contrast, the specific internalizing and externalizing factors—after the covariation due to the general factor is accounted for—may represent essentially orthogonal phenotypes that are influenced by the different causes and mechanisms that are shared by only the internalizing or by only the externalizing first‐order dimensions (Lahey et al., 2017). These and other findings provide a foundation for future studies by providing disconfirmable hypotheses regarding a posited hierarchy of etiologies and mechanisms underlying psychopathology.

Loadings of first‐order dimensions on second‐order factors did not differ in magnitudes by sex, but individual differences (means and variances) in the general and externalizing factors were much greater for males, whereas the reverse was true for the internalizing factor. This argues that the same structural model of psychopathology applies to both sexes.

4.1. Limitations

The current findings are limited by the unavailability of an ideal instrument for measuring psychopathology in dimensional terms in adults. Some aspects of the present results also may have been influenced by the moderate sample size. Because we replicated previous findings regarding the relative fits of the correlated‐factors and bifactor models, this does not appear to be a serious limitation. Furthermore, model comparisons using chi‐square difference tests were almost always consistent with differences in BIC and other fit statistics. Nonetheless, interpretations of findings, particularly nonsignificant factor loadings, should consider the modest sample size. For example, specific phobia did not load significantly on the general factor in either CFA or ESEM in the present analyses. Because specific phobia has been found to have a modest but significant loading on the general factor in several larger studies (reviewed by Lahey et al., 2017), the present nonsignificant loadings may be due to sample size.

The modest sample size also precluded some potentially important analyses. The symptoms reported to be present by YA_DISC scoring algorithms were summed to define first‐order dimensions according to DSM5 rules. The sample was too small to conduct factor analyses at the symptom level to evaluate the DSM5 dimensions of symptoms. Specifically, there were cells with zero observations in the 2 × 2 tables for many pairs of low prevalence symptoms, which prevented the estimation of tetrachoric correlations for factor analysis. Thus, future research with larger samples is needed using instruments that address all symptoms of every major dimension of psychopathology, including personality. The use of dimensional ratings of symptoms rather than the binary presence or absence of symptoms and expansion of the item pool to include symptoms not in DSM5 may be of further value.

4.2. Future directions

Although the present findings support the general factor model of psychopathology, they do not address the essential topic of criterion validity. The general factor model can only be said to be valid and useful if the general and specific factors arise because they have different etiologies and underlying psychobiological mechanisms and are associated with different outcomes over time (Bonifay, Lane, & Reise, 2017; Lahey et al., 2017; Waldman, Poore, Van Hulle, Rathouz, & Lahey, 2016). Although supportive evidence has been reported (reviewed by Lahey et al., 2017), far more such tests of the criterion validity of the general factor model are needed.

In addition, the findings of sex differences point to important topics for future research. Although sex differences in the means and variances of internalizing and externalizing psychopathology are well known, the causes of these differences are not understood. In particular, the sex difference in means and variances in the general factor of psychopathology is an important topic for future study.

Supporting information

Table S1. Comparisons of the fits of alternative confirmatory factor analysis models of the correlational structure of 14 dimensions of psychopathology with overlapping symptoms omitted.

Table S2. Standardized factor loadings for the exploratory structural equation model (ESEM) with a bifactor rotation and two specific factors illustrated in Figure 3.

Table S3. Fit statistics for re‐estimations of the best‐fitting bifactor model 2e in Table 2 2e (general factor on which every dimension loads with specific internalizing and externalizing factors), excluding one psychopathology dimension at a time (all models had 53 degrees of freedom).

Table S4. Separate confirmatory factor analyses models of the correlational structure of 14 dimensions of psychopathology in males and females based on models 1a, 2d, and 2e.

Lahey BB, Zald DH, Perkins SF, et al. Measuring the hierarchical general factor model of psychopathology in young adults. Int J Methods Psychiatr Res. 2018;27:e1593 10.1002/mpr.1593