The general factor of psychopathology: a comparison with the general factor of intelligence with respect to magnitude and predictive validity

Abstract

In recent years, there has been a surge of interest in the general factor of psychopathology (“p”), which is intended to summarize broad psychiat­ric comorbidity into a single index. In this study, rather than attempting to validate this model using statistical techniques, we compared the magnitude (as indicated by the variance explained in the respective indicators) and the predictive validity of the “p” factor with those of the gen­eral factor of intelligence (“g”). To compare the magnitude, for “g”, we analyzed fifteen Wechsler Adult Intelligence Scale subtests (N=1,200). For “p”, we analyzed fourteen psychiatric diagnoses in Swedish adults (N=909,699), eight self‐ and parent‐rated psychopathology scales in Swedish adolescents (N=2,069), and sixteen parent‐rated psychopathology scales in Swedish children (N=14,589). To compare the predictive validity, we analyzed Swedish male military conscripts (N=414,595, mean age: 18.3 years) with measures on both “g” and “p” (derived from eight psychiatric diagnoses). We then examined their unique associations with three intelligence‐related outcomes (annual income, highest education, and university entrance exam scores), and sixteen adverse outcomes (e.g., suicidal behavior, psychotropic medication prescription, and criminality) retrieved from registers (mean age at follow‐up = 29.2 years). Results indicated that the magnitudes of “g” and “p” were very similar. Controlling for “p”, “g” significantly predicted later education (standardized beta, β=0.38, SE=0.01) and university entrance exam scores (β=0.48, SE=0.01). Controlling for “g”, “p” significantly predicted all adverse outcomes (mean β=0.32; range: 0.15 to 0.47). These findings support the notion that psychopathology indicators can be combined into a single score, similar to how intelligence subtests are combined into a general intelligence score. This “p” score might supplement specific diagnoses when formulating a management plan and predicting prognosis.

Individuals who perform well on one intelligence subtest tend to perform well on all other intelligence subtests1, 2. This empirical observation is the reason why intelligence subtests are combined into a single score, commonly labeled “g” (general intelligence factor). Introduced over a century ago, this factor has offered utility for researchers and clinicians3, 4, 5. For example, it predicts future education about as well as height predicts weight ⁶ .

Similar to the intelligence domain, individuals who suffer from one mental health problem are at increased risk of suffering from virtually all other mental health problems7, 8, 9, 10. For example, in a Danish population study of three million individuals, all psychiatric diagnoses were positively associated ¹¹ . Recently, Lahey et al12, 13 proposed that a general factor of psychopathology could serve as a useful summary of this comorbidity. Caspi et al ¹⁴ replicated this general factor of psychopathology and labeled it “p”, to highlight its similarity to “g”.

Just as “g” predicts future education, studies indicate that “p” predicts future adverse outcomes, highlighting its clinical utility. For example, the cumulative burden of parent‐rated psychiatric problems in childhood predicts adverse outcomes in adolescence and young adulthood over and above specific psychiatric problems15, 16, 17, 18. To date, however, no studies have examined whether general psychopathology severe enough to warrant psychiatric diagnoses in late adolescence predicts register‐based adverse outcomes in young adulthood. Furthermore, no studies have examined whether “p” predicts adverse outcomes over and above “g”, which is important because they are moderately negatively associated14, 19.

The first goal of the present study was to compare the magnitude of the general factors of psychopathology and intelligence. The magnitude of a general factor is determined by the strength of the overlap among its indicators. For example, because the correlations among intelligence subtests are relatively large, the magnitude of “g” is also relatively large ²⁰ . If “p” were to have a similar magnitude as “g”, then it might be useful to combine indicators of psychopathology into a single “p” score, just as intelligence subtests are summed into a single “g” score.

The second goal of this study was to compare the predictive validity of “p” and “g”, after adjusting for their overlap. If “p” were to uniquely predict unfavorable outcomes as well as “g” uniquely predicts favorable outcomes, then “p” might offer the psychiatric domain clinical and research utility. For example, it might supplement primary diagnoses when formulating the management plan and predicting prognosis.

METHODS

Samples

To measure the magnitude of the general intelligence factor, we relied on summary data from six US standardization subsamples published in the fourth edition of the Wechsler Adult Intelligence Scale (WAIS‐IV) ¹ . We combined the six subsamples (20‐25 years old; 25‐30 years old; 30‐35 years old; 35‐45 years old; 45‐55 years old; and 55‐65 years old; each N=200) into a single sample (20‐65 years old; N=1,200).

To measure the magnitude of the general psychopathology factor, we relied on three different samples.

First, we examined that magnitude in adulthood by analyzing all individuals born in Sweden between 1969 and 1979 from the Multi‐Generation Register (N=1,056,041), such that the participants were between 35 and 45 years old at the end of the follow‐up period at December 31, 2013. After excluding subjects who had died or migrated before the end of the study period, the final sample included 909,699 individuals.

Second, we examined that magnitude in adolescence by analyzing 16‐year old individuals from the Swedish Twin Study of Child and Adolescent Development (TCHAD) ²¹ . At age 16, 1,067 (74%) of the parents and 2,369 (82%) of the twins responded. There was both self‐ and parent‐report information on 2,069 individuals.

Third, we examined that magnitude in childhood by analyzing 9‐year old individuals from the Child and Adolescent Twin Study in Sweden (CATSS) (N=14,589) ²² . The response rate was 75%.

To compare the predictive validity of the general factors of intelligence and psychopathology, we examined Swedish male military conscripts born between 1980 and 1992 (N=414,595; mean age: 18.3 years). Over 95% of all Swedish males attended the mandatory conscription evaluation ²³ . We excluded all participants who had died (except from suicide) or migrated.

The study was approved by the Regional Ethical Review Board in Stockholm. Informed consent was acquired from the twin samples. By law, register data do not require informed consent because they are pseudonymized.

Measures

To measure the magnitude of the general factor of intelligence, we analyzed the Pearson correlations among the fifteen WAIS‐IV subtests.

To measure the magnitude of the general factor of psychopathology among the 35‐45 year old population sample, we linked the participants to the National Patient Register, which captures inpatient (1969‐2013) and outpatient (2001‐2013) psychiatric diagnoses according to the ICD‐8 (1969‐1986), ICD‐9 (1987‐1996) or ICD‐10 (1997‐present). This register covers 99% of psychiatric inpatient and 70 to 95% of psychiatric outpatient admissions ²⁴ . We examined whether the individuals had ever been diagnosed with depression, anxiety, obsessive‐compulsive disorder (OCD), post‐traumatic stress disorder (PTSD), eating disorders, alcohol misuse, drug abuse, attention‐deficit hyperactivity‐disorder (ADHD), oppositional defiant/conduct disorder (ODD), autism, tics, bipolar disorder, schizophrenia, and schizoaffective disorder.

To measure the magnitude of the general factor of psychopathology among the 16‐year olds, we relied on parent‐ and self‐ratings from the Child Behavior Checklist (CBCL) ²⁵ . This consists of eight continuous scales measuring anxiety/depression, withdrawn behavior, somatic complaints, social problems, thought problems, attention problems, rule‐breaking behavior, and aggressive behavior. The CBCL is reliable and well‐validated (e.g., the mean test‐retest reliability of parent ratings was 0.90 across eight days, and 0.70 across 24 months) ²⁶ .

To measure the magnitude of the general factor of psychopathology among the 9‐year olds, we relied on parent ratings on the Autism‐Tics, AD/HD and other Comorbidities inventory (A‐TAC) ²⁷ , which consists of continuous scales measuring problems with coordination, sensory issues, inattention, impulsivity, learning, organization, memory, language, sociability, flexibility, tics, compulsions, OCD, oppositional defiance, and conduct problems. Furthermore, we included the parent‐rated Screen for Child Anxiety Related Emotional Disorders (SCARED), which measures anxiety, and the parent‐rated Short Mood and Feelings Questionnaire (SMFQ), which measures depression. These scales have good psychometric properties (e.g., the mean test‐retest reliability of the A‐TAC scales based on clinician interviews across two months was 0.85)27, 28, 29, 30.

To measure the general factor of intelligence at conscription, we included general intelligence scores from the Computer Aided Testing ‐ Swedish Enlistment Battery (CAT‐SEB), administered during the conscription evaluation ²³ . The CAT‐SEB included 12 subtests (mean internal consistency = 0.83; range = 0.70 to 0.93) and took on average 62 min to complete. The general factor score reliability estimate was 0.90 ³¹ .

To measure the general factor of psychopathology at conscription, we linked the military conscripts to the National Patient Register to examine if they had been diagnosed with anxiety, depression, PTSD, bipolar disorder, drug abuse, alcohol misuse, ODD and ADHD prior to conscription (we excluded diagnoses of schizophrenia, schizoaffective disorder, autism, tics, and eating disorders because these did not co‐occur frequently enough to estimate tetrachoric correlations).

We then examined whether the participants had experienced a wide variety of outcomes after conscription. We included three intelligence‐related outcomes. From the Longitudinal Integration Database for Health Insurance and Labor Market Studies Register (LISA; coverage: 1990‐2013), we included the highest annual log of income and the highest obtained education level. We also included the highest score on the Swedish Scholastic Aptitude Test (SweSAT), a voluntary test administered twice a year (end of coverage: 2015) that grants admission to Swedish universities ³² .

We further examined sixteen adverse outcomes. From the National Patient Register, we included diagnoses of acute drug and alcohol intoxication (i.e., overdoses), and diagnoses of certain and uncertain suicide attempts. We combined certain suicide diagnoses with death from suicide (identified in the Death Register). From the Prescribed Drug Register (coverage: 2005‐2013), we included prescriptions of anxiolytic, sedative, antidepressant, stimulant, anti‐alcohol, anti‐opioid, lithium, antiepileptic, and antipsychotic medications (classified according to the Anatomical Therapeutic Chemical, ATC system). From the National Crime Register (coverage: 1973‐2013), we included court convictions of property or violent crimes. From LISA, we included use of social welfare benefits. All adverse outcomes were treated as binary variables.

The mean follow‐up time from date of conscription was 10.9±3.3 years. We included year of birth as a covariate to adjust for unequal follow‐up times and diagnostic secular trends. Table 1 displays prevalence rates and average time‐to‐event by outcome.

Table 1.

Adverse outcome statistics

Outcome	Descriptive	Time‐to‐event (years, mean±SD)
Acute drug intoxication (%)	0.68	6.93±3.65
Acute alcohol intoxication (%)	1.80	5.12±3.67
Suicide attempt, certain (%)	1.10	6.05±3.64
Suicide attempt, uncertain (%)	1.57	5.54±3.50
Prescription of anxiolytics (%)	10.61	9.15±3.80
Prescription of sedatives (%)	9.08	9.46±3.79
Prescription of SSRIs (%)	12.25	9.89±3.82
Prescription of stimulants (%)	1.69	10.53±3.67
Prescription of anti‐alcohol ­medication (%)	0.97	9.28±3.68
Prescription of anti‐opioid ­medication (%)	0.13	11.84±3.17
Prescription of lithium (%)	0.26	10.67±3.56
Prescription of antiepileptics (%)	2.43	10.30±3.70
Prescription of antipsychotics (%)	2.19	9.96±3.78
Property crimes (%)	2.96	4.62±3.67
Violent crimes (%)	4.21	5.31±3.48
Use of social welfare benefits (%)	16.07	2.92±2.55
Highest median annual income in SEK (median absolute ­deviation)	254,400 (87,770)	8.59±3.52
Education level, range 1‐7 (SD)	4.43 (1.12)	9.89±3.28
Highest SweSAT score, range 0.05‐2 (SD)	1.01 (0.44)	3.98±3.04

SSRIs – selective serotonin reuptake inhibitors, SEK – Swedish krona, SweSAT – Swedish Scholastic Aptitude Test

Education level: 1= less than 9 years, 2= 9 years, 3= 1‐2 years of high school, 4= 3 years of high school, 5= 1‐2 years of undergraduate college, 6= 3 or more years of undergraduate college, 7= graduate studies

Statistical analyses

We estimated the magnitude of the general factors in three ways. We relied on a visual examination of the correlation distributions and their means; on the variance accounted for by the first principal component (PC₁); and on the explained common variance (ECV) ²⁰ . The ECV is the ratio of the variance explained by the general factor divided by the variance explained by the full factor model33, 34. It ranges from 0 (none of the modeled variance is attributable to the general factor) to 1 (all of the modeled variance is attributable to the general factor).

To derive the ECV, we conducted exploratory factor analyses (EFAs) of the intelligence and the mental health measures. We relied on exploratory rather than confirmatory factor analysis because we did not have strong hypotheses regarding the loading patterns, and because we expected the data to have a complex structure (i.e., that the cross‐loadings would not equal zero).

We determined how many factors to extract based on the scree plot, which contrasts the eigenvalues against the eigenvectors (for the WAIS‐IV subsamples, we computed the eigenvalues separately in each subsample, and then derived their means) ³⁵ . Eigenvectors that account for less than one unit of variance might be attributable to sample variation ³⁶ . We then rotated the extracted factors to a general and several specific factors using the Direct Schmid‐Leiman rotation ³⁷ . This rotation funnels the variance shared among all indicators into a general factor, and the variance unique to subsets of the indicators into uncorrelated specific factors. Simulations indicate that this rotation performs well ³⁸ .

For the adolescent sample, we only analyzed the overlap between the self‐ and parent‐ratings to minimize potential rater bias. For the six WAIS‐IV standardization subsamples, we combined all correlation matrices into a single histogram; computed the PC₁ separately in each subsample and derived their mean (PC₁); and estimated the ECV from a single EFA with the loadings constrained to equality across the six different age groups.

For the assessment of predictive validity, we examined the dimensionality of the eight mental disorders using the scree plot ³⁵ , and rotated the EFA solution toward one general and several uncorrelated specific factors using the Direct Schmid‐Leiman approach ³⁷ . Subsequently, we used exploratory structural equation modeling to regress each of the outcomes onto the general intelligence scale, the exploratory general and specific factors, and birth year in a multiple regression framework to estimate their unique effects ³⁹ .

We used probit regression for the binary outcomes, and linear regression after outcome standardization (mean = 0; variance = 1) for the continuous outcomes. This allowed for comparing the regression betas on the same scale across the differently distributed outcomes. All analyses were conducted with the Mplus software, and the rotation matrices were derived using the R‐package GPArotation40, 41.

For the general factor magnitude sensitivity analyses, we extracted up to two factors more than that indicated by the scree plot because the ECV index varies by dimensionality. We subscripted the ECV index to display how many extracted factors it was based on (e.g., ECV₃ indicates that it was based on three extracted factors). Furthermore, because there are several ways to identify a general factor ⁴² , we re‐estimated the ECVs using a bifactor rotation ⁴³ . For the six differently aged WAIS‐IV stan­dardization samples, we examined whether the factor loadings could be constrained to equality without a loss in model fit when computing the ECV index.

For the predictive validity sensitivity analyses, we first conducted the analyses without a general factor, using an oblique (correlated) Geomin rotation. Second, we attempted to extract an additional factor above and beyond that indicated by the scree plot, and to use a bifactor rotation. Third, we re‐ran the analyses after excluding all participants who had died (except from suicide) five or more years after conscription.

RESULTS

The scree plots for the four samples are displayed in Figure 1. Figure 2 displays that the distributions of the correlations were similar for the WAIS‐IV subtests and the psychopathology measures. Furthermore, the mean correlations, the PC₁ and the ECV indices were highly similar in the two domains. This indicates that the magnitudes of the general factors of intelligence and psychopathology were largely indistinguishable.

Figure 1.

Scree plots for adult intelligence, and adult, adolescent and child psychopathology

Figure 2.

Histograms of correlations among Wechsler Adult Intelligence Scale (WAIS‐IV) subtests in adulthood, and among psychopathology measures in adulthood, adolescence and childhood. PC₁= variance accounted for by first principal component, ECV= explained common variance index, where the sub‐index indicates factor dimensionality

The scree plot of the eight mental disorders at conscription indicated the presence of two factors (Table 2). All disorders loaded substantially on the general factor (mean loading = .55; range: .44 to .66). The first specific factor captured internalizing problems (depression loading = .64; anxiety loading = .48), and the second specific factor captured externalizing problems (ODD loading = .52; drug abuse loading = .39).

Table 2.

Exploratory factor analysis of psychiatric diagnoses assigned prior to conscription

	Rotation: Direct Schmid‐Leiman
Psychiatric diagnosis	General psychopathology factor	Specific internalizing factor	Specific externalizing factor
Depression	0.66	0.64	0.03
Anxiety	0.53	0.48	0.05
Post‐traumatic stress disorder	0.54	0.42	0.12
Bipolar disorder	0.55	0.33	0.22
Alcohol misuse	0.44	0.07	0.37
Drug abuse	0.54	0.16	0.39
Oppositional‐defiance/conduct disorder	0.62	0.10	0.52
Attention‐deficit/hyperactivity disorder	0.54	0.09	0.45

Loadings equal to or greater than 0.30 are bolded

Root mean square error of approximation = 0.005, 90% CI: 0.004‐0.005, confirmatory fit index = 0.983, Tucker‐Lewis index = 0.963,

χ²=127.771, df=13, p<0.001

We then regressed each outcome onto the general intelligence factor and the general and specific psychopathology factors in a multiple regression framework. Figure 3 displays that the general intelligence factor uniquely and significantly predicted later education (β=0.38, SE=0.01) and SweSAT scores (β=0.48, SE=0.01), but not annual income (β=0.00, SE=0.01). The general factor of psychopathology uniquely and significantly predicted all sixteen adverse outcomes (mean β=0.32; range: 0.15 to 0.47). Individuals scoring one standard deviation above the mean on the general factor of psychopathology had, on average, after transforming the mean probit beta to an odds ratio, a 79% higher risk of suffering the adverse outcomes.

Figure 3.

Unique associations between general psychopathology (p) and general intelligence (g) factors measured at conscription and later outcomes. The standardized betas for income, education level and SweSAT were based on linear regression. SSRIs – selective serotonin reuptake inhibitors, SweSAT – Swedish Scholastic Aptitude Test

As displayed in Figure 4, the specific psychopathology factors primarily predicted related outcomes (e.g., the specific internalizing, but not the specific externalizing, factor predicted prescription of selective serotonin reuptake inhibitors, SSRIs), showing that covariation not accounted for by the general factor of psychopathology also had predictive validity.

Figure 4.

Unique associations between specific internalizing (Int) and externalizing (Ext) factors measured at conscription and later outcomes (all associations adjusted for general psychopathology and intelligence). The standardized betas for income, education level and SweSAT were based on linear regression. SSRIs – selective serotonin reuptake inhibitors, SweSAT – Swedish Scholastic Aptitude Test

For the magnitude comparison, the ECV indices remained similar between intelligence and psychopathology, regardless of dimensionality and general factor rotation. For the WAIS‐IV subsamples, the factor loadings could be constrained to equality across the six subsamples without a loss in model fit (results available upon request).

For the predictive validity analyses, the betas based on the correlated factors model were similar to the specific factors in the original analyses, but obviously did not demonstrate the effect of the general factor. Although we attempted to extract a third exploratory factor from the eight mental disorders at conscription, this factor contained only small loadings (e.g., mean Varimax loading = –0.03; range: –0.07 to 0.10). This suggested over‐extraction, and did not permit proceeding to a Direct Schmid‐Leiman or bifactor rotation. The betas also remained similar when we re‐ran the analyses after excluding all participants who had died (except from suicide) five years or later after conscription (results available upon request).

DISCUSSION

Our findings document that the general factors of intelligence and psychopathology have similar magnitudes, indicating that it might be useful to combine psychopathology indicators into a “p” score, just as WAIS‐IV subtests are combined into a “g” score.

Furthermore, whereas previous research had demonstrated that parent‐rated general psychopathology in childhood predicts adverse outcomes in adolescence and young adulthood15, 16, 17, 18, we additionally demonstrated that a general psychopathology factor based on psychiatric diagnoses predicts register‐based adverse outcomes a decade later in young adulthood, even when holding general intelligence constant. To put the magnitude of these associations in context, the general factor of psychopathology predicted the adverse outcomes about as well as psychotherapy predicts subsequent well‐being, or about as well as sleeping aid medication reduces short‐term insomnia ⁶ .

It remains unclear what the general factor of psychopathology measures. Hypotheses include the personality trait neuroticism, impulsivity, or irrational thinking44, 45, 46. We speculate that the general factor of psychopathology might quantify overall distress and impairment, similar to how the general factor of intelligence quantifies gross abstract reasoning. However, given the lack of consensus about the meaning of the century‐old general factor of intelligence ⁴ , it might be unrealistic to expect a solution to this conundrum anytime soon.

Regardless of interpretation, the general factor of intelligence has offered clinical and research utility over the past century, suggesting that the general factor of psychopathology might do so too47, 48. In terms of clinical utility, the general factor of psychopathology might supplement diagnoses. A continuous or binned (e.g., small, medium and large) general psychopathology score might assist with prognosis; might differentiate among patients with the same primary diagnosis to indicate who might need additional care; and might help individuals who present with a large number of symptoms, but fail to meet diagnostic criteria for a diagnosis, to gain access to care.

In terms of research utility, we echo past arguments that it might be beneficial to isolate the general factor of psychopathology when examining associations with risk factors13, 14. As an analogy, a hypothetical association between processing speed and future level of education might not indicate specificity; however, if such an association were to remain after isolating general intelligence, it would lend stronger support to the unique role of rapid thinking in educational success.

This work should be interpreted in light of certain limitations. First, the predictive results were limited to men only. Although past research has demonstrated that a parent‐rated general psychopathology factor predicted teacher‐rated adverse outcomes years later in a sample of girls, it would be important to examine if these results replicate among females ¹⁷ . Furthermore, some of the diagnoses tend to co‐occur with the outcomes (e.g., depression and prescription of SSRIs), which might have increased the associations. However, the adverse outcomes occurred on the average eight years after conscription, and the general psychopathology factor predicted more independent outcomes (e.g., criminality and use of social welfare benefits) equally well.

Second, it is possible that the associations among mental health indicators are influenced by collider bias ⁴⁹ . Individuals with multiple disorders might be more prone to seek mental health assistance, leading to an overestimation of associations among disorders in national registers. In contrast, individuals with multiple syndromes might be less prone to participate in survey research, leading to an underestimation of associations among symptom scales. Despite these potential ascertainment biases, the general factor magnitude metrics were remarkably similar regardless of sampling method. On a related note, it is possible that the general factor of psychopathology might partly represent a rating bias; however, for the adolescent sample, we analyzed the magnitude only based on the overlap between self‐ and parent‐report data. Furthermore, it seems unlikely that a rating bias would predict such a wide range of adverse outcomes years later.

Third, because it is challenging to predict time‐to‐event data in a structural equation modeling framework, we only predicted the probability of whether the outcomes occurred or not. Survival analyses would additionally have predicted the probability of the events as a function of time.

Fourth, it is important to keep in mind that observed data that appear to consist of a general factor could be generated by processes lacking a general factor50, 51, 52, 53. Because it is difficult to infer the true data generating process, it is probably wise to harbor a healthy level of skepticism toward all nosological models to protect against reification ⁵⁴ . Nevertheless, even if a process without a general factor had generated the observed data patterns in this study, the general factor of psychopathology might still be a convenient summary index of such underlying process.

Finally, although the general factors of intelligence and psychopathology had similar magnitude and predictive validity, there are also substantive differences between the two domains. In contrast to psychopathology, intelligence tests have a logically correct answer, whereas short‐term fluctuations are generally more important in the mental health domain (e.g., a depressive episode might warrant temporary suicide prevention efforts).

In conclusion, whereas current diagnostic systems measure diagnoses relatively well, they place less emphasis on broad symptomatology. It might be useful to combine psychopathology indicators into a single score, similar to how intelligence subtests are combined into a general intelligence score. Such a single score might supplement specific diagnoses when developing treatment plans or predicting prognosis.