The “Big Everything”: Integrating and Investigating Dimensional Models of Psychopathology, Personality, Personality Pathology, and Cognitive Functioning

Abstract

Common factors are increasingly used to model the structure of psychopathology (“p”), personality (General Factor of Personality [GFP]), pathological personality (General Factor of Pathological Personality [GFPP]), and intelligence (“g”). Using 4 waves spanning ages 18–29 in a cohort of college students (baseline n = 489), this study used indicators of psychopathology, personality, pathological personality, and cognitive functioning to compare models that included Cognitive Functioning, p, GFP, GFPP, and a “Big Everything” factor (which included cross-domain measures as indicators). GFP, GFPP, and p exhibited substantial overlap, and the Big Everything factor accounted for considerable variance in psychopathology, personality, and pathological personality indicators. Only a self-report measure of cognitive functioning loaded significantly onto the Big Everything. This study highlights concerns in the pursuit of identifying and reifying common factors based on the modeling of residual variances and limitations of using factor modeling to determine the structure of psychologically relevant phenomena.

At various points in history, delineating between different forms of psychopathology has been considered the sine qua non of psychiatric science. Emil Kraepelin, considered by some to be the founder of modern scientific psychiatry (Eysenck, Arnold, & Meili, 1972), first distinguished manic depression (a construct with features similar to mood disorders such as bipolar disorder in modern classification systems) and dementia praecox (later reclassified as schizophrenia by Bleuler [1908]). By demarcating features that were previously considered to reflect a unitary construct, this so-called Kraepelinian dichotomy (Kraepelin, 1896) was immensely impactful on subsequent psychiatric classification systems (such as later versions of the Diagnostic and Statistical Manual of Mental Disorders [American Psychiatric Association, 2013] and the International Statistical Classification of Diseases and Related Health Problems [World Health Organization, 2004]). Indeed, the shift to explicit diagnostic criteria, which significantly increased the reliability of psychiatric diagnoses, has been referred to as the “neo-Kraepelinian” movement in psychiatry (Klerman, 1978).

More recently, others have argued for a focus on homogenous constructs to enhance both clinical research and practice. For example, Smith, McCarthy, and Zapolski (2009) posited that hierarchical organizations of constructs “are useful descriptively and theoretically, but higher order composites do not refer to definable psychological processes” (p. 272) and suggested that homogenous dimensions provide “… improved validity, utility, and parsimony. In contrast, the use of composite diagnoses can retard scientific progress and hamper clinicians’ efforts to understand and treat dysfunction” (p. 272). More recent evaluations of higher order constructs in psychopathology (i.e., the p factor) suggest substantive interpretation and application of these constructs is difficult (Smith, Atkinson, Davis, Riley, & Oltmanns, 2020), such that the only shared featured that can be gleaned is a general “index of overall impairment that is nonspecific and secondary to the variables that load on the p factor” (Smith et al., 2020, p. 9).

Recently, there has been a rise in the emphasis of “lumping” approaches in modeling the structure of various psychologically relevant constructs. For example, Caspi et al. (2014) introduced the notion of a p factor in the domain of psychopathology to account for the positive manifold typically observed among a wide array of psychiatric disorders (reflecting covariation or diagnostic co-occurrence among disorders; see Kotov et al., 2017). Using bifactor modeling (Holzinger & Swineford, 1939; Schmid & Leiman, 1957) within the rich Dunedin Multidisciplinary Health and Development Study data set, the authors concluded that psychiatric disorders,

were initially explained by three higher-order factors (Internalizing, Externalizing, and Thought Disorder) but explained even better with one General Psychopathology dimension. We have called this dimension the p factor because it conceptually parallels a familiar dimension in psychological science: the g factor of general intelligence. (p. 119)

Since Caspi et al.’s (2014) publication, as well as in some previous work by others (Lahey et al., 2012; Simms, Grös, Watson, & O’Hara, 2008), various research groups have applied bifactor modeling to measures of psychopathology to identify the p factor in multiple data sets (Greene & Eaton, 2017; Kim & Eaton, 2015; Laceulle, Vollebergh, & Ormel, 2015; Lahey et al., 2012; Martel et al., 2017; Patalay et al., 2015; Snyder, Young, & Hankin, 2017).

As noted in van Bork, Epskamp, Rhemtulla, Borsboom, and van der Maas’s (2017) review of the p factor, the shift to ever-increasing common factor structures in the domain of psychopathology mirrors similar scientific movements in other fields. For example, in the domain of personality, researchers have promoted the five-factor model (in addition to other models; see Church, 1994; Markon, Krueger, & Watson, 2005) as well as the General Factor of Personality (GFP, also referred to as the “the Big One”; Musek, 2007). Similarly, personality disorders have been construed as 10 distinct disorders (e.g., avoidant personality disorder, borderline personality disorder) that coalesce into one common factor (referred to here as the General Factor of Personality Pathology [GFPP]; see Hengartner, van der Linden, & Dunkel, 2017; Sharp et al., 2015).

Serving as a forerunner to these subsequent scientific movements, multiple structures of intelligence have been considered across time, with the general factor of intelligence (i.e., the g factor) being the most well-known general factor solution. These research traditions were inspired by Galton’s desire for an “independent estimate of the man’s powers” (Galton, 1890, p. 380), work by others suggesting “no correspondence” among experimental tasks (e.g., “splitters” such as Bagley, Binet, and Seashore; see Spearman, 1904, pp. 206–219), and most notably Spearman’s observation of a positive manifold among intelligence tasks. Indeed, the observation that participants that tended to do well on one task did well on other tasks arguably gave rise to the seminal implementation of factor analysis to identify g (Spearman, 1904).

Recent work indicates that some general factors may strongly correlate across domains. In a cross-sectional sample of 474 participants (who were previously or currently involved in mental health treatment) recruited from the United States via Mechanical Turk, Oltmanns, Smith, Oltmanns, and Widiger (2018) demonstrated that a GFP (based on a five-factor framework), a p factor (based on 16 scales measuring various forms of psychopathology), and a GFPP factor (based on two measures of maladaptive personality; see Oltmanns et al., 2018) overlapped strongly (r = −.70 between GFP and p; rs > |.90| between GFPP and the p and GFP factors). A second cross-sectional examination based on a sample of 1,630 older adults (see Oltmanns et al., 2018) also suggested high overlap between GFP and GFPP (r = −.82). Based on these findings, the authors concluded that these general factors “are likely to entail a common individual differences continuum, which may impact on how these general factors are to be understood” (p. 581). That is, given that correlations among lower order factors can be respecified as factor loadings onto a higher order factor, one way to interpret these findings is that yet another general factor that includes p, GFP, and GFPP as indicators could be estimated.

Although the field has focused considerably on common factors in recent years, the extent to which these models have theoretical utility, reflect substantive, psychologically meaningful structures, or confer additional predictive validity remains open for debate. For example, the GFP has been interpreted as a meaningful factor reflecting a continuum of positive versus negative features of personality (Musek, 2007), as well as a result of a “mistaken understanding of basic psychometrics” (Revelle & Wilt, 2013; see Hopwood, Wright, & Donnellan, 2011; MacCann, Pearce, & Jiang, 2017, for further critiques). There have also been controversies regarding the g factor of intelligence (Jensen, 1999), which has been scrutinized, given this solution’s questionable ability to account for systematic behavioral, neurological, and biological variability (e.g., see van der Maas, Dolan, Grasman, Wicherts, Huizenga, & Raijmakers, 2006). For example, van der Maas et al. (2006) noted that the positive manifold among cognitive tasks “emerges purely by positive beneficial interactions between cognitive processes during development. A single underlying g factor plays no role in the model” (p. 842). Lastly, the nature of the p factor remains unclear. Indeed, in the influential Caspi et al. (2014) article, the authors had a vast range of conclusions, including that several observations suggest that the p factor “is a dimension that unites all disorders and has neurological roots” (p. 131) to acknowledging that the observed pattern of “intercorrelations also could result from dynamic processes during development, rather than from a single unitary cause” (p. 131) to openly questioning whether the p factor is “merely a statistical reductio ad absurdum” (p. 132). Notably, recent commentary (Caspi & Moffitt, 2018) appears to favor substantive implications of the p factor: “For example, evidence that p predicts objective, real-world life outcomes (e.g., suicide) suggests that it may be indexing something substantive, not merely something about how people behave while data are being collected” (p. 835).

Crucially, the “p factor perspective” is being currently used, to some extent, to support the efforts to develop a Hierarchical Taxonomy of Psychopathology (HiTOP; Kotov et al., 2017). More specifically, Kotov et al. noted that the proposed HiTOP spectra (i.e., Somatoform, Internalizing, Thought Disorder, Disinhibited Externalizing, Antagonistic Externalizing, and Detachment; see Figure 2 of Kotov et al.) display a positive manifold, and this pattern is,

consistent with the existence of a general psychopathology factor or p factor (Caspi et al., 2014; Lahey, Van Hulle, Singh, Waldman, & Rathouz, 2011, Lahey et al., 2012). This possibility has been supported by studies that evaluated a bifactor model, which is composed of a general dimension defined by all forms of psychopathology and specific dimensions defined by smaller groups of disorders, (p. 462)

Present Study

Common factor solutions have been proposed in the domains of psychopathology, personality, personality pathology, and intelligence. There is initial evidence that common factors of personality, pathological personality, and psychopathology show significant overlap. Despite the attention these models have attracted (and efforts from various research groups), there are a marked variety of perspectives regarding the utility and validity of these models. To further delineate the implications of different factor structures and modeling approaches, we drew upon longitudinal data from a prospective sample of a cohort of college students (baseline N = 489), first ascertained as first-time freshman (M_age∼18) and followed over the third decade of life (roughly at ages 21, 25, and 29). Correlations among common factors (p, GFP, GFPP, and Cognitive Functioning [CF] factors) based on numerous indicators of psychopathology, personality, pathological personality, and cognitive functioning (or intelligence) were examined in an initial model. This model was then compared with a model that included a “Big Everything” factor (i.e., a factor that included all lower order indicators of p, GFP, GFPP, and CF as indicators) to determine the extent to which this factor accounted for variance in indicators across domains and impacted the relation between the domain-specific common factors and their indicators. Previous work (Oltmanns et al., 2018) suggests that indicators of psychopathology, personality, and pathological personality could load onto a general Big Everything factor. There is also evidence that measures of intelligence relate to the p factor (Caspi et al., 2014), providing a rationale for testing if measures of cognitive functioning would load onto a Big Everything factor in these data. Various bifactor indices were calculated to evaluate factors included within the Big Everything model.

Method

Participants and Procedures

The current data were drawn from a longitudinal study on family history of alcoholism collected at a large, Midwestern university (i.e., the Alcohol, Health, and Behavior data set; see Sher, Walitzer, Wood, & Brent, 1991, for a more detailed description). At baseline, the sample included 489 first-year college students (54% women, M_age = 18.2 years), with 51% of the respondents classified as family history positive of alcoholism (see Sher et al., 1991). Respondents were assessed seven times over a 17-year period at approximately ages 18 (Wave 1), 19 (Wave 2), 20 (Wave 3), 21 (Wave 4), 25 (Wave 5), 29 (Wave 6), and 35 (Wave 7) via both interview and paper-and-pencil questionnaire. For the current study, only data from Waves 1, 4, 5, and 6 were used. This was because the personality, psychopathology, and CF measures of interest were each not assessed at all waves, and we sought to minimize artifacts induced via the timing of assessments, given that the constructs are typically conceptualized as individual differences, though are known to be temporally variable in light of developmental, contextual, and other idiosyncratic processes (see Analytic Approach for additional details). Overall retention was high, with 84% of participants completing measures through Wave 6. This study was approved by the University of Missouri institutional review board.

Measures

Cognitive functioning.

CF was assessed at Wave 1 using several available indicators related to intelligence, including measures from the Wechsler Adult Intelligence Scale–Revised (WAIS–R; Wechsler, 1981). Indicators were WAIS–R Vocabulary (standard score), WAIS–R Similarities (standard score), Block Design (total scale score), the Weschler Memory Quotient (Nixon, 1995; Russell, 1975; Wechsler, 1944), American College Testing English and Math scores, and scores based on the Cognitive Failures Questionnaire (CFQ; a measure of “self-reported failures in perception, memory, and motor function”; Broadbent, Cooper, FitzGerald, & Parkes, 1982, p. 1). All measures were treated as continuous.

Internalizing psychopathology.

Psychopathology was assessed using the Brief Symptoms Inventory (Derogatis & Spencer, 1982); the current analyses used measures from Waves 1 and 4–6. The BSI is a 53-item self-report measure that assesses distress (i.e., “In the past week, how much were you distressed by”) related to various types of mental health symptomology (e.g., an item on the Depression subscale = “feeling lonely”) using a 5-point Likert-like scale (i.e., 0 = not at all, 1 = a little bit, 2 = moderately, 3 = quite a bit, 4 = extremely) to assess for symptoms of common mental health diagnoses. For the current study, the Depression, Obsessive–Compulsive, Anxiety, and Phobic subscales of the BSI were used to provide coverage of constructs central to internalizing psychopathology while limiting model complexity (i.e., including all available BSI measures would have further contributed to model complexity). Given distributional concerns of these measures (i.e., these measures were positively skewed), values from these scales were used to create ordered polytomous variables for each domain of psychopathology at each wave. For the measures of depression, obsessive–compulsive, and anxiety, three-level ordinal variables were created (0 = average score of 0 on a given scale, i.e., reported not at all on all scale items; 1 = average score greater than zero but one or less, 2 = average scores greater than 1). Given frequent averages of zero across waves for the Phobic subscale, this measure was rescored as binary (0 = average score of 0, 1 = average scores greater than 0). This scoring approach resulted in adequate frequencies of symptom endorsement across types of symptomatology across the waves of assessment and facilitated modeling approaches that were compatible with the zero-inflated distributions of these symptom measures.

Externalizing psychopathology.

Past-year alcohol consequences were assessed using 27 dichotomous (yes/no) items (e.g., “awakened the morning after a good bit of drinking and found that you could not remember a part of the evening before?”) adapted from the Michigan Alcoholism Screening Test (Selzer, Vanosdall, & Chapman, 1971) and scales by Blane (1987), Engs (1977), and the Jessors (Jessor & Jessor, 1973, 1981). Sum scores of these consequences at each wave were recoded as ordered polytomous variables (i.e., 0 = no consequences reported in the past year, 1 = between one and three consequences reported in the past year, 2 = between four and six consequences reported in the past year, and 3 = seven or more consequences reported in the past year). Past-year cigarette use was assessed using measures to cigarette frequency and quantity; these measures were used to create ordered polytomous variables of cigarette use at each wave (i.e., 0 = “I don’t smoke now” reported in the past year, 1 = a positive smoking frequency of 3–4 days a week or less or using one to five cigarettes a day or less on smoking days, 2 = a positive smoking frequency of 5–6 days a week or more and using a half-a-pack a day or more on smoking days). Past-year drug consequences were assessed using 18 items (e.g., “Have you ever lost friends [including girlfriends/boyfriends] because of your drug use?”) adapted from the Drug Abuse Screening Test (Skinner, 1982). Sum scores of these consequences at each wave were recoded as ordered polytomous variables (i.e., 0 = no consequences reported in the past year, 1 = between one and two consequences reported in the past year, 2 = three or more consequences reported in the past year).

Personality.

Personality traits were assessed at Waves 4, 5, and 6 using the NEO-Five Factor Inventory (McCrae & John, 1992). The NEO consists of five broad personality domains: neuroticism, extraversion, openness, conscientiousness, and agreeableness. At each wave, these measures were created by summing the items assessing each personality domain. All measures were treated as continuous.

Personality pathology.

All personality pathology features were assessed at Wave 6 using the Structured Interview for Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition,–Personality (Pfohl, Blum, & Zimmerman, 1997), with the exception of antisocial personality disorder, which was assessed with the Diagnostic Interview Schedule-IV. Ordered polytomous variables for each of the personality disorder domains (i.e., paranoid, schizoid, schizotypal, antisocial, borderline, histrionic, narcissistic, avoidant, dependent, obsessive–compulsive) were created based on the following raw scores calculated for each participant for each domain: 0 = not present or limited to rare isolated examples, 1 = subthreshold (some evidence of the trait, but it is not sufficiently pervasive or severe to consider the criterion present), 2 = present (criterion is present at least 50% of the time during the last 5 years), 3 = strongly present (criterion is associated with subjective distress or some impairment in social or occupational functioning or intimate relationships). These values were rescaled as ordered polytomous for each of the 10 PDs (with three levels: 0 = total 0, 1 = total 1, 2 = 2+).

Analytic Approach

All analyses were conducted in MPlus Version 7 (Muthén & Muthén, 1998–2017) using the diagonally weighted least squares mean- and variance-adjusted estimator. All indicators of psychopathology and pathological personality were treated as ordered polytomous, whereas indicators of intelligence and personality were treated as continuous.

We began by examining a model that estimated common factors across the domains of intelligence, pathological personality, psychopathology, and personality (see Figure 1). Model specification varied depending on whether measures were used from a single wave (cognitive functioning, pathological personality) or across multiple waves (psychopathology, personality). For example, a CF factor was estimated with manifest measures at Wave 1 (see Measures) as indicators. Similarly, a GFPP was estimated using the 10 measures of pathological personality at Wave 6 as indicators. For psychopathology, individual traits (i.e., depression, obsessive–compulsive, anxiety, phobic, alcohol, tobacco, drug) were estimated using manifest indicators at Waves 1, 4, 5, and 6 (e.g., trait depression was estimated with measures of depression at the aforementioned waves as indicators). A p factor was then estimated using these traits as indicators. Likewise, measures of personality across Waves 4, 5, and 6 were used to construct personality traits (e.g., three measures of agreeableness served as indicators of the Agreeableness factor), which were then used as indicators of a GFP factor. The four common factors were allowed to correlate, whereas all other factors were orthogonal. Intermediate factors (e.g., Alpha and Beta factors for personality, internalizing and externalizing for psychopathology) were not included, given multiple considerations (i.e., to enhance comparability across common factors of the various domains, a lack of evidence supporting the estimation of the intermediate Beta factor for personality [trait openness and extraversion r = −.07, p = .17], latent variable correlations above unity when separate Cluster A and Cluster C pathological personality traits were estimated, and to facilitate model convergence).

Figure 1.

Correlated traits and Big Everything models. a = the correlated traits model; b = the bifactor Big Everything model; CF = Cognitive Functioning; GFPP = General Factor of Personality Psychopathology; GFP = General Factor of Personality; p = General Factor of Psychopathology; 1 = ACT English; 2 = ACT Mathematics; 3 = working memory; 4 = vocabulary; 5 = similarities; 6 = block design; 7 = Cognitive Failures Questionnaire; 8 = paranoid; 9 = schizoid; 10 = schizotypal; 11 = avoidant; 12 = dependent; 13 = obsessive-compulsive personality; 14 = antisocial; 15 = borderline; 16 = histrionic; 17 = narcissistic; 18 = Agreeableness; 19 = Conscientiousness; 20 = Extraversion; 21 = Neuroticism; 22 = Openness; 23 = Obsessive-Compulsive: Brief Symptoms Inventory; 24 = Anxiety; 25 = Depression; 26 = Phobia; 27 = Alcohol; 28 = Drugs; 29 = Tobacco Smoking.

Building on this model, a cross-domain common factor, the Big Everything, was included (i.e., a factor where the six indicators of intelligence [e.g., Wechsler Memory Quotient], the eight psychopathology traits [e.g., depression], the five factors of personality [e.g., neuroticism], and the 10 domains of personality psychopathology [e.g., borderline] served as indicators). In line with traditional bifactor modeling, all other factors were modeled as orthogonal (see Figure 1).

For this model, several estimates of bifactor model reliability were calculated. To index the amount of variance accounted for by the Big Everything factor relative to the other general factors, we calculated explained common variance (ECV; Stucky & Edelen, 2015). ECV is the proportion of all common variance explained by a given factor. Two types of ECVs can be obtained for factors that do not include all measures as indicators (i.e., all factors in this model besides the Big Everything); the first ECV computes the strength of a specific factor relative to all explained variance of all items, even those not loading on the specific factor of interest, whereas the second ECV computes the strength of a specific factor relative to all explained variance only of the items loading on that specific factor (Dueber, 2017). Omega, a model-based estimate of internal reliability of a factor, was also calculated (McDonald, 1999). Omega Hierarchical (omegaH) was also calculated for the Big Everything factor. This estimate “reflects the percentage of systematic variance in unit-weighted (raw) total scores that can be attributed to the individual differences on the general factor … when omegaH is high (>.80), total scores can be considered essentially unidimensional” (Rodriguez, Reise, & Haviland, 2016, p. 224). This estimate was also calculated for the other common factors, reflecting “the proportion of reliable variance in the subscale composite that is independent of the general factor” (Reise, Bonifay, & Haviland, 2013, p. 225). Relative omega (i.e., omegaH divided by omega) was also calculated. Relative omega represents the percent of reliable variance in the multidimensional composite due to the Big Everything factor when calculated for this general factor. For the other factors (i.e., g, GFP, p, GFPP), relative omega reflects the portion of reliable variance in these factors that is independent of the general factor (see Dueber, 2017). A measure of construct replicability, H, was also calculated (Hancock & Mueller, 2001). H “represent[s] the correlation between a factor and an optimally weighted composite … high H values (>.80) suggest a well-defined latent variable” (Rodriguez et al., 2016, p. 230) and considered to be the most appropriate index for bifactor model factor reliability (see Watts, Poore, & Waldman, 2019).

The fit of each of the models was assessed using the χ² value, the comparative fit index (CFI), and the root mean square error of approximation (RMSEA). For models that involved manifest measures across waves, residual covariances were estimated within each wave (e.g., residual variance in agreeableness at Wave 4 was covaried with residual variance in depression at Wave 4) and constrained to equality across waves (e.g., the residual covariances between agreeableness and depression were constrained to equality at Waves 4–6). In instances where the residual manifest variance was less than 5% of the total variance, correlated residuals were not estimated to minimize Heywood cases involving correlated residual variances.

Results

Correlated Common Factors Model

Factor loadings for the model are shown in Table 1, model fit: χ²(1626, N = 489) = 2934.10, p < .001, RMSEA = .04, CFI = .86 All indicators significantly loaded onto their respective common factors. These loadings were generally positive and appreciable, though the CFQ measure had a small loading on CF (λ = −.18). Consistent with the notion that the GFP reflects well-adjusted personality, or even social desirability, individuals higher in GFP were estimated to have higher scores in conscientiousness, extraversion, and agreeableness, as well as lower scores in neuroticism and openness.

Table 1.

Factor Loadings of the Three Models

	Correlated traits				Bifactor Big Everything longitudinal					Bifactor Big Everything cross-sectional
Indicator	GFPP	GFP	p	CF	Big E	GFPP	GFP	p	CF	Big E	GFPP	GFP	p	CF
Paranoid	0.64				0.45	0.77				0.51	0.51
Schizoid	0.47				0.33	0.38				0.30	0.43
Schizotypal	0.78				0.59	0.50				0.60	0.52
Avoidant	0.59				0.45	0.36				0.49	0.31
Dependent	0.41				0.28	0.41				0.34	0.36
Obsessive-compulsive PD	0.54				0.41	0.40				0.38	0.42
Antisocial	0.72				0.60	0.26				0.55	0.34
Borderline	0.85				0.72	0.24				0.71	0.28
Histrionic	0.53				0.40	0.40				0.39	0.44
Narcissistic	0.67				0.47	0.57				0.46	0.63
Neuroticism		−0.73			0.69		−0.17			0.63		−0.50
Extraversion		0.37			−0.28		0.96			−0.30		0.32
Openness		−0.30			0.28		0.03			0.28		0.31
Agreeableness		0.48			−0.42		−0.26			−0.49		0.03
Conscientiousness		0.54			−0.52		0.12			−0.44		0.09
Depression			0.79		0.79			−0.35		0.62			0.41
Anxiety			0.69		0.70			−0.47		0.49			0.43
Phobia			0.77		0.76			−0.31		0.55			0.45
Obsessive-compulsive BSI			0.74		0.75			−0.41		0.65			0.45
Alcohol			0.59		0.59			0.43		0.56			−0.30
Drugs			0.73		0.71			0.43		0.59			−0.14
Tobacco			0.50		0.49			0.33		0.44			−0.45
ACT English				0.73	0.01				0.76	0.03				0.75
ACT Math				0.68	0.00				0.69	0.02				0.69
WMQ				0.59	0.03				0.59	0.05				0.59
Vocab				0.71	0.03				0.70	0.11				0.70
Similarities				0.43	−0.06				0.42	−0.04				0.42
Block design				0.48	−0.03				0.49	−0.02				0.49
CFQ				−0.18	0.40				−0.14	0.26				−0.16
Factor intercorrelations for the correlated traits model
GFPP–GFP: −0.81; GFPP–p: .71; GFPP–CF: 0.07; GFP–CF: −0.08; GFP–p: −0.87; p–CF: −0.12

Note. GFPP = General Factor of Pathological Personality comprising paranoid, schizoid, schizotypal, avoidant, dependent, obsessive–compulsive PD (personality), antisocial, borderline, histrionic, and narcissistic; GFP = General Factor of Personality comprising agreeableness, conscientiousness, extraversion, neuroticism and openness; p = General Factor of Psychopathology comprising obsessive-compulsive BSI, anxiety, depression, phobia, alcohol, drugs, and tobacco smoking; CF = Cognitive Functioning factor comprising ACT English, ACT Mathematics, WMQ (working memory), vocabulary, similarities, block design, and CFQ; Big E = Big Everything; PD = personality disorder; BSI = Brief Symptoms Inventory; ACT = American College Test; WMQ = Working Memory Questionnaire; Vocab = vocabulary; CFQ = Cognitive Failures Questionnaire. Bifactor Big Everything cross-sectional factor loadings were obtained from Wave 6 for all constructs except cognitive functioning, which was measured at Wave 1. All bolded estimates are significant at p < .05.

Regarding intercorrelations among these factors, the GFPP, GFP, and p were correlated strongly and significantly (ps < .001). GFPP correlated −.81 with GFP and .71 with p. GFP and p correlated at −.87. CF, in contrast, appeared to be largely independent of the other common factors (rs ranged from −.12 to .07), though the correlation between p and CF (−.12) was statistically significant (p = .045; see the online supplemental materials for correlations among indicators).

The Big Everything

Factor loadings from the model that included the Big Everything factor are shown in Table 1. Initial estimation suggested a small, nonsignificant negative residual variance for trait extraversion and thus the residual variance of this trait was constrained to zero, model fit: χ²(1605, N = 489) = 2596.48, p < .001, RMSEA = .04, CFI = .89. All indicators of GFP, p, and GFPP loaded significantly onto the Big Everything factor (see Table 1). The only CF indicator that significantly loaded onto the Big Everything factor was the CFQ (λ = .40), whereas the other CF indicators did not (loadings ranged from −.06 to .03).

The inclusion of the Big Everything factor affected the pattern of factor loadings of several of the previous common factors (see Table 1). Notably, the factors from the correlated common factors model were identified by nonresidual factors, whereas in this model the common factors are based on factors residualized on the Big Everything factor. After accounting for the common variance across domains, the GFP appeared to largely reflect variance in extraversion. The signs of the factor loadings for the p factor (see Table 1) flipped for the internalizing traits of depression, anxiety, phobia, and obsessive–compulsive, such that higher levels in p were now related to lower levels of internalizing conditions and higher levels of externalizing behaviors.

A primary focus of this model was to determine to what extent the Big Everything factor accounted for variance in the various indictors relative to the domain-specific common factors. The ECV for the Big Everything in this model was about 52% (see Table 2), which suggests the Big Everything accounted for more than half of all common variance among the indicators. Indeed, even when only considering indicators that loaded on a specific common factor (e.g., depression, anxiety, phobia, obsessive–compulsive, alcohol, tobacco, and drugs on the p factor), the ECV of the Big Everything Factor was higher compared with all other common factors with the exception of the CF factor. After taking into account the Big Everything factor, the reliability of the GFP (as indexed by omega) was low (∼.34). Further, with the exception of CF, omegaH (an index of reliable systematic variance in these factors after portioning out variance related to the Big Everything factor) was low for all other common factors, particularly the p factor (∼.01); relative omega and H were also particularly low for the p factor.

Table 2.

Explained Common Variance of Big Everything Models

Factor models	ECV (S&E)	ECV (NEW)	Omega	OmegaH	Relative omega	H
Big Everything, longitudinal
Big E	0.52	0.52	0.89	0.66	0.74	0.93
GFPP	0.15	0.47	0.88	0.40	0.46	0.77
GFP	0.08	0.49	0.34	0.33	0.96	0.92
p	0.08	0.25	0.90	0.01	0.01	0.57
CF	0.17	0.93	0.73	0.72	0.99	0.90
Big Everything, cross-sectional
Big E	0.50	0.50	0.86	0.61	0.71	0.89
GFPP	0.17	0.44	0.88	0.39	0.44	0.72
GFP	0.04	0.31	0.05	0.02	0.38	0.36
p	0.09	0.33	0.81	0.04	0.05	0.56
CF	0.20	0.97	0.73	0.72	0.99	0.81

Note. ECV (S&E) = explained common variance computing the strength of a specific factor relative to all explained variance of all items, even those not loading on the specific factor of interest (S&E = Stucky & Edelen, 2015); ECV (New) = explained common variance computing the strength of a specific factor relative to all explained variance only of the items loading on that specific factor; Omega = estimate of internal reliability; OmegaH = omega hierarchical, the variance attributable to individual differences on the general factor; Relative omega = the variance attributed to the general factor; H = construct reliability; Big E = Big Everything; GFPP = General Factor of Pathological Personality comprising paranoid, schizoid, schizotypal, avoidant, dependent, obsessive-compulsive PD (personality), antisocial, borderline, histrionic, and narcissistic; GFP = General Factor of Personality comprising agreeableness, conscientiousness, extraversion, neuroticism and openness; p = General Factor of Psychopathology comprising obsessive-compulsive BSI, anxiety, depression, phobia, alcohol, drugs, and tobacco smoking; CF = Cognitive Functioning factor comprising ACT English, ACT Mathematics, WMQ (working memory), vocabulary, similarities, block design, and CFQ; PD = personality disorder; BSI = Brief Symptoms Inventory; ACT = American College Test; WMQ = Working Memory Questionnaire; CFQ = Cognitive Failures Questionnaire.

Given some domains (personality, psychopathology) were assessed across multiple waves and other domains (cognitive functioning, pathological personality) were only assessed at one time point, we estimated a model that only used data at Wave 6 for all constructs besides CF, which was assessed at Wave 1. We considered this model to determine the impact of only using cross sectional data on the estimates discussed immediately above. For this model, all indictors were manifest variables (e.g., the GFP factor reflected manifest measures of the five factors assessed at Wave 6), which obviated the need to account for correlated residuals for time of assessment.

Summary estimates based on this model are shown in Table 1. Although this model did not fit the data as well compared with the model that incorporated data across waves, χ²(348, N = 489) = 825.65, p < .001, RMSEA = .05, CFI = .83, the estimates from this model yielded similar conclusions compared with the primary models discussed above (the Big Everything factor accounted for roughly half of all explainable variance among the indicators and was higher compared with all common factors except CF; the proportion of reliable variability attributed to the p factor after accounting for the Big Everything factor was quite low). Notably, the reliability estimates for the GFP were lower in the model that only used cross-sectional data (see Table 2).

Discussion

Within the fields of psychopathology, personality, pathological personality, and intelligence, “lumping” approaches have been emphasized regarding the structure of psychologically relevant constructs. Using a prospective data set of high-risk college students, we were able to replicate previous findings (Oltmanns et al., 2018) suggesting that the general factors of psychopathology, personality, and pathological personality exhibited substantial overlap. Proceeding one step further up the ladder of lumping, we also demonstrated that a Big Everything factor accounted for considerable indicator variance across the domains of psychopathology, personality, and pathological personality. With the exception of the self-report CFQ, the Big Everything factor did not account for significant variance among CF indicators. Bifactor indices from models based on both longitudinal and exclusively cross-sectional data indicated little evidence of reliable variance in common factors, once accounting for variance attributed to the Big Everything factor.

In the primary correlated traits models, the GFP, p, and GFPP common factors showed substantial overlap. This result is similar to recent work by Oltmanns et al. (2018), which showed general factors of personality, psychopathology, and pathological personality exhibited substantial overlap within two cross-sectional samples that are seemingly different from the sample of high-risk college students used here (see Oltmanns et al., 2018). These findings are also consistent with evidence from the current work that the Big Everything factor accounted for considerable variance in various domain indicators, with the exception of the performance-based measures. Indeed, the Big Everything factor accounted for roughly half of all explainable common variance among the indictors, even when accounting for variance due to previously identified domain-specific common factors. Individuals higher on this factor tended to have higher levels of maladaptive personality features and tended to engage in more problematic behaviors (e.g., these individuals were generally higher in measures of drug consequences).

However, it is unclear what substantive interpretations can be gleaned from reifying this Big Everything factor. Indeed, in some ways, attempting to interpret what variation in this factor represents is analogous to struggles in interpreting other common factors in various disciplines (see the discussion sections of Caspi et al., 2014, and Oltmanns et al., 2018; also see Smith et al., 2009, 2020, and Watts et al., 2019, for a broader discussion). More specifically, Oltmanns et al. (2018) noted their interpretations were speculative, yet “perhaps no more speculative than what has been previously offered” (p. 587). As noted in the introduction, there has been various interpretations of the general factor literature, with some camps highlighting potential substantive meanings of these factors, whereas others appear to dismiss these solutions as artifactual.

Regardless of the varying potential interpretations of common factors, at the very least these current data suggest that yet another factor that captures covariation among indicators of personality, psychopathology, and pathological personality can be estimated; including this factor does not deteriorate model fit (though we cautiously interpret model fit, given recent research suggesting pitfalls in using fit to adjudicate among factor solutions such as those considered here; see Bonifay & Cai, 2017; Bonifay, Lane, & Reise, 2017; Markon, 2019; Morgan, Hodge, Wells, & Watkins, 2015; Murray & Johnson, 2013; Reise, Kim, Mansolf, & Widaman, 2016; van Bork et al., 2017); and, overall, this factor accounts for considerable variance among most factor indicators with the exception of performance-based measures. To put it another way, it is unclear what limit, if any (besides those related to empirical model identification), there is in using confirmatory factor analysis (e.g., bifactor modeling) to justify “lumping” approaches in the quest to define the structure of psychologically relevant constructs.

These data also highlight the difficulty in interpreting residual variances (e.g., Lynam, Hoyle, & Newman, 2006; Meehl, 1971; Rogers et al., 2018) once increasingly aggregated factors are incorporated in factor solutions involving measures across multiple domains. For example, in Caspi et al. (2014), results from a correlated–factors model suggested a significant positive correlation between externalizing and internalizing factors (r = .328) yet a negative correlation (r = −.471) in the model that included the p factor (see Caspi et al., 2014). The authors concluded, this

suggests that Externalizing and Internalizing are positively correlated in the population because they share a common liability to General Psychopathology, but after controlling for General Psychopathology, individuals who are prone to substance use and antisocial behavior are less prone to depression and anxiety, and vice versa. (p. 126)

This finding is quite similar to the current work that demonstrated factor loadings for internalizing (e.g., depression) and externalizing (e.g., drug involvement) indicators had opposite signs on the p factor when accounting for variance attributed to the Big Everything factor (also see Eid, Geiser, Koch, & Heene, 2017; Sellbom & Tellegen, 2019, for further discussion regarding limitations of residuals variances within bifactor models).

Regardless of interpretation, these findings highlight several issues that are important to consider in work that uses confirmatory factor analysis to delineate among alternative structures of psychologically relevant constructs, including the inherent difficulty in interpreting residualized variances and covariances among more specific factors. As recently noted by Watts et al. (2019) regarding interpreting estimates involving specific factors within bifactor models, “specific factors should be interpreted cautiously, given that their contents might not be consistent with their conceptualization and that researchers should pay close attention to what variance is extracted from a specific factor and allotted to the general factor” (p. 13).

Importantly, the bifactor estimates gleaned from our models calls into question the utility of some of the common factors, particularly the p factor, in light of identifying the Big Everything factor. Notably, Smith et al. (2020) highlighted the importance of considering the application of higher order factors, suggesting that the p factor may be best interpreted as a general marker of distress and that its clinical utility occurs in relation to other lower order homogenous factors (e.g., neuroticism). Applying similar logic to the shift in focus from the constructs of internalizing and externalizing to the p factor, one potential implication of the current findings is that researchers should shift their focus to Big Everything factor models in the presence of cross-domain assessments, given this factor appears to capture even broader liability to manifest higher levels of problematic personality traits (e.g., neuroticism), psychopathology, and pathological personality features and the questionable reliability of factors such as p when the Big Everything factor is considered.

However, in light of more focused tests of the bifactor model (Watts et al., 2019), we believe this conclusion to be problematic. Specifically, Watts et al. (2019) demonstrated bifactor models yielded less reliable specific factors (e.g., distress) when these specific factors were modeled as orthogonal and when including the common p factor into the model. That is, the general tendency for bifactor models to yield less reliable specific factors (when these factors are modeled as orthogonal) relative to a common factor may speak more to the pitfalls of bifactor modeling (the primary analytic tool of choice supporting various common factor models) than to something substantive that could truly inform the structure of psychologically relevant individual differences. As summarized by Watts et al. (2019), “bifactor models of psychopathology may be placing undue burden on the data (Bonifay, 2015; Bonifay & Cai, 2017; Reise et al., 2016), in turn compromising the integrity of either the general or specific factors” (p. 14).

Perhaps the clearest conclusion garnered from this study is that, with the exception of the CFQ, the other CF indicators appeared to be largely unrelated to measures of personality, psychopathology, and pathological personality among this sample of high-risk college students. Caspi et al. (2014) did identify several significant correlations between the p factor and multiple indices of intelligence within the Dunedin data set, though these correlations were (in absolute magnitude) .204 or lower, and Harden et al. (2020) showed a significant genetic association between executive functioning and psychopathology. Given these findings, coupled with extant work has positing that cognitive deficits (e.g., executive functioning, memory, and attention; Smith et al., 2020; Trivedi, 2006) are present in a wide range of psychological conditions spanning internalizing, externalizing, and personality disorder symptomology, we anticipated that CF would evince relations with p, GFP, and GFPP. In the current data, CF only correlated significantly with the p factor, though the magnitude of this relation, r = −.12, p = .045, was much smaller compared with the correlations among the other common factors. Further, when the CFQ indicator was removed, the correlation between p and CF became statistically nonsignificant (p and CF r = −.09, p = .15). It is potentially more difficult to detect statistically significant relations in samples such as ours, given all participants were university students with ostensibly above-average intelligence though previous work using these data have demonstrated meaningful variance in measures of intelligence in this sample (Sher, Martin, Wood, & Rutledge, 1997; Littlefield, Vergés, McCarthy, & Sher, 2011).

Differences in assessment approaches between the CF measures (largely performance-based assessments) and other assessments (measures based on self-report) may also contribute, to some degree, to the lack of evidence that CF is strongly linked to these other domains. Highlighting this observation, the self-reported CFQ was the only indicator to load onto the Big Everything and exhibited a low magnitude loading onto the CF factor. These findings mirror seminal work involving the CFQ (Broadbent et al., 1982), as well as subsequent work (Wagle, Berrios, & Ho, 1999), that indicates this scale significantly correlates with measures of depression and other types of internalizing features but does not show significant overlap with performance-based assessments (e.g., the Signal Detection Memory Test; see Wagle et al., 1999). Oltmanns et al. proffered a speculative interpretation accounting for the apparent higher overlap among common factors, such that these factors may generally reflect individual differences “of impairment or dysfunction within the respective persons’ lives, irrespective of the basis for that dysfunction or impairment” (p. 587). Extending this observation into the current work, perhaps individual differences in perceived functioning are reflected across a range of measures, some related directly to psychopathology, others related to perception, memory, and motor lapses, and yet other assessments related to normal personality variation. When these differences are indexed by common factors, the high overlap among these factors may be capturing the tendency that individuals who report higher levels of psychopathology also report higher levels of traits associated with more negative outcomes (such as neuroticism; Brandes, Herzhoff, Smack, & Tackett, 2019). Minimally, these data indicate that level of functioning based on performance-based measures is largely (linearly) unrelated to the propensity to report various types of psychopathology and personality features in this sample.

There are several limitations of the current study. Although the present study used several indicators of both internalizing and externalizing psychopathology, not all potential indicators were included in the current models. However, this is a common issue within the p factor literature (see Watts et al., 2019, for more details). Given the scope of our primary model and the nature of these data, it was not feasible to incorporate intermediate factors that are considered relevant to some of the domains (e.g., Alpha and Beta for personality; internalizing and externalizing for psychopathology). Further, data were collected using a high-risk sample of college students and, as such, findings may not be generalizable to other populations. Despite these limitations, we were able to replicate previous findings regarding the overlap among common factors that did incorporate these intermediate factors and used data from samples distinct from a high-risk college sample (see Oltmanns et al., 2018). CF measures (assessed at Wave 1) and pathological personality features (assessed at Wave 6) were only assessed at one occasion, and thus there was uneven coverage of items reflecting these domains compared with other domains. However, analyses based on only one wave of data for each domain yielded conclusions largely consistent with those based on the longitudinal data. Despite its name, the Big Everything factor certainly did not include all potential assessments of individual differences that are psychologically relevant. Although several steps were taken to reduce the complexity of this project (e.g., not including available Wave 7 assessments), the models described here involved many indicators and parameter estimates given the sample size. As possible, other research groups with access to larger, prospective data sets that include relative measures of psychopathology, personality, pathological personality, and CF should evaluate models such as those considered currently. Additionally, other research groups may consider comparing models such as those estimated here to alternative modeling approaches to psychologically relevant traits, such as those based on network analysis.

In sum, the psychopathology literature has recently emphasized and scrutinized the utility of a common factor that ostensibly unites all disorders. This “lumping” approach mirrors other efforts in the fields of personality, pathological personality, and intelligence. Among other issues, the current work highlights limitations regarding using factor analytic approaches to clearly determine the structure of psychologically relevant phenomenon. Currently, there is no clear consensus within these fields regarding the utility of these common factors as a way to understand the potential structure of important constructs or to inform theoretical and clinical efforts. In some ways, the disparate opinions found in these scientific fields simply reflect tensions between “splitting” versus “lumping” approaches and the associated pros and cons. Importantly, there are many cogent arguments to support movements such as HiTOP to derive alternative taxonomic systems that (at least in our opinion) do not necessarily hinge on identifying and reifying common factors. Overall, the current findings provide several examples as to why researchers should be cautious when using factor analytic techniques to engender evidence for or against certain assumptions regarding the structure of important phenomenon in the pursuit of further refining psychological science.