Beyond the Factor Indeterminacy Problem Using Genome-Wide Association Data

Abstract

Latent factors, such as general intelligence, depression, and risk-tolerance, are invoked in nearly all social science research where a construct is measured via aggregation of symptoms, question responses, or other measurements. Because latent factors cannot be directly observed, they are inferred by fitting a specific model to empirical patterns of correlations among measured variables. A longstanding critique of latent factor theories is that the correlations used to infer latent factors can be produced by alternative data generating mechanisms that do not include latent factors. This is referred to as the factor indeterminacy problem. Researchers have recently begun to overcome this problem using information on the associations between individual genetic variants and measured variables. We review historical work on the factor indeterminacy problem and describe recent efforts in genomics to rigorously test the validity of latent factors, advancing understanding of behavioral science constructs.

“Factorists have often fallen prey to a temptation for reification-for awarding physical meaning to all strong principal components. Sometimes this is justified…But such a claim can never arise from the mathematics alone…”

– Stephen J. Gould, Mismeasure of Man [280] ¹

“Causal reasons lie behind the positive correlations of most mental tests. But what reasons? We cannot infer the reasons from a strong first principal component any more than we can induce the cause of a single correlation coefficient from its magnitude. We cannot reify g as a “thing” unless we have convincing, independent information beyond the fact of correlation itself.”¹

– Stephen J. Gould, Mismeasure of Man [281] ¹

Latent variables, such as Openness to Experience, Subjective Well-Being, Depression, and General Intelligence, are ubiquitous in the behavioral sciences. Latent variables are theoretical constructs that are not directly observable but are presumed to underly constellations of measured variables (e.g., symptoms, item responses, or test scores) that are moderately or strongly intercorrelated. Factor analysis formally models these observed intercorrelations as arising from a single general factor, or from a small number of general factors. Even when not formally modeled within a factor analytic framework, latent variables are implicitly invoked when a construct has been measured via aggregation of question responses, symptoms, or measurements. In other words, when sum scores are used, a latent variable is typically assumed.

Despite their widespread use in the behavioral sciences, the ontological and epistemological status of latent variables has been a topic of considerable historical and contemporary debate. In this article, we address the central criticism of latent variable models, which is that a wide range of causal models may account for the observed data upon which factor analysis is applied. In other words, given observed correlations alone, it is impossible to distinguish whether the data generating process resulted from the action of the hypothesized latent factor(s) or a different causal mechanism altogether. For example, does a person score similarly on different cognitive tests because their performance on all tests is influenced by a general capacity for reasoning, as implied by the latent factor of general intelligence, g? If people respond consistently to questions such as “How satisfied are you with your life as a whole these days?” and “How would you rate your overall personal well-being?”, is that evidence that people’s responses to survey items reflect their Happiness in life? We refer to this issue as the factor indeterminacy problem^2,3.

As highlighted in the quotes by Gould above, independent information, beyond the correlations among observed variables, is needed to overcome this indeterminacy and confirm or disconfirm the latent variable model. In this paper, we review how researchers have recently used genome-wide data to provide a source of compelling collateral information that surmounts the factor indeterminacy problem because the genome can be decomposed into a set of uncorrelated units. We begin by briefly describing the methods and origins of factor analysis, outlining key critiques of latent variable models, and surveying popular alternative causal models. We then use simulation to illustrate how genome-wide association data may provide a unique source of additional information to formally falsify latent variable models. Finally, we review recent empirical applications of this approach.

Latent Variables and Factor Analysis

In psychology, formal taxonomies of personality (e.g., the Big Five⁴, the HEXACO⁵), psychopathology (e.g., individual disorders such as Depression and Anxiety or superfactors such as Internalizing and Externalizing Psychopathology⁵), and cognitive abilities (e.g., Spearman’s theory of General Intelligence^6,7, Cattell’s theory of Fluid and Crystallized Intelligence⁸, and Carroll’s three stratum model of Intelligence⁹) are all rooted in factor analysis. In economics, political science, and sociology, researchers are less likely to conduct factor analyses or to conceptualize their measurements explicitly within a factor analytic framework. Yet all of these fields rely on measurements, whether they be self-report responses to survey questions or performance on controlled behavioral tasks, that purport to reflect underlying attributes of a person that cannot be directly observed, such as Subjective Well-Being, Risk Tolerance, or Conservatism.

Factor analysis is a statistical tool within the Structural Equation Modeling framework that is used to infer the existence of such unobserved constructs from the correlations between observed variables. For example, in personality psychology, self-reports of enjoying parties, being talkative, and being energetic are often found to be moderately intercorrelated. When applying a factor analytic model to these data, one can impose the assumption that the correlations arise due to a shared cause or disposition, commonly labelled Extraversion¹⁰. Similarly, in psychopathology, the correlation between sleep disturbance and fatigue can be attributed to the common influence of depression on both symptoms¹¹. Indeed, it would be sensible to infer that the Diagnostic and Statistical Manual of Mental Disorders (DSM) treats diagnoses as entities and symptoms as their manifestations^11,12. Some recent frameworks (e.g., HiTOP¹³) have made latent variable models of psychopathology explicit by positing super and sub-factors that collectively influence symptoms and traits¹³.

Factor analysis originated in Spearman’s research on cognitive abilities. He found that correlations between scores on mental tests are nearly always positive and substantial in magnitude, a pattern which he termed the positive manifold ^6,7. Spearman proposed a “two-factor theory,” in which performance on a given cognitive test was caused by two factors: a general underlying factor, g, common to all tests, and a component specific to that test, s. According to this theory, the positive manifold arises because all tests are influenced by the same g, but the correlations within the positive manifold are imperfect because each test is influenced by a different s.

One intuitive representation of the factor model is a line drawn through the major dimension of variation in a scatterplot occupying multidimensional space. (This representation more accurately describes principal components analysis, however we describe it here to give an intuitive representation of how factor analysis describes relationships among variables.) Hunt memorably likened a three-dimensional space containing many individual data points to a hot dog containing many pimentos (Fig. 1; see figure 2a for a diagrammatic representation of the factor model)¹⁴. The length of the hot dog represents the principal axis of variation, and- to extend the metaphor- the line drawn through it is a skewer. Hunt [p. 357]¹⁴ wrote:

Figure 1. Metaphor for factor analysis reproduced from Hunt (1995).

Each pimento embedded in the hot dog represents an individual data point in 3-dimensional space. The white line (or skewer) through the hot dog captures the variation between pimento locations, representing how factor analysis extracts general factors from variation in the data.

Figure 2. The factor model and alternative models.

Spearman’s common factor model (A), Thomson’s bonds model as described by Ceci (1990) (B) and van der Maas’ mutualism model (C) all generate a positive manifold from which a general factor can be extracted, despite originating from different causal architectures. Across panels, y1–5 depicts the measured variables that display a positive manifold, and arrows depict the directional flow of causation.

“To get an intuitive idea of factor analysis, imagine buying a hot dog with pimientos embedded in it. The hot dog is a three-dimensional object, so it takes three dimensions to specify the exact location of each pimiento. However, you can locate a pimiento reasonably accurately by saying where it is along the long axis of the dog. In factor-analytic terms the pimientos are the data from each person, and the three dimensions of the hot dog represent the individual tests. The long axis of the hot dog would be the first factor to be extracted and would capture most of the variation between pimiento locations. If we apply factor analysis to test scores, instead of hot dogs, the first factor accounts for most of the variation between people just as the length of the hot dog accounts for most of the positioning of the pimientos. But instead of saying “length of hot dog,” we say “general intelligence.”

Following Spearman, it was discovered that the length of the hot dog was insufficient to describe the relations among the intelligence test pimentos: more than one general factor is needed to account for the patterning of correlations among cognitive tests scores. But, in some representations of the data, these factors were themselves correlated, such that a general factor could be extracted at a higher order level. Thurstone argued against such a representation and instead rotated the axes of each factor so that all axes were near clusters of the test vectors while remaining uncorrelated (“orthogonal”), resulting in his model of primary mental abilities that did not include a general factor¹⁵. The key insight from Thurstone’s work and Spearman’s subsequent defense¹⁶ was that both models were statistically equivalent, establishing that the position of factor axes, and the resulting number and nature of factors, was predicated on theory¹⁷.

Thurstone’s conflict with Spearman provided a foundation for debate about the nature of latent variables. The seemingly arbitrary placement of factor axes led Gould to ask “What psychological meaning could g claim if it represented but one possible rendering of information subject to radically different, but mathematically equivalent, interpretations?”¹ The construction of latent variables, then, could be driven by theoretical assumptions that could not necessarily be refuted or confirmed by the data.

Debates Surrounding the Ontological and Epistemological Status of Latent Variables

Despite the challenge posed by the factor indeterminacy problem, latent variable models have gained wide popularity in the behavioral sciences, so much so that factor analysis is now used to investigate the validity of nearly all measures of interindividual differences, including measures of personality, cognition, psychopathology, attitudes, preferences, and environmental contexts. Nevertheless, the legitimacy of latent variable models continues to be widely debated and critiqued.

One of the most high-profile critiques of factor analysis, Gould’s influential Mismeasure of Man¹, centered on the argument that factor analysis alone does not provide sufficient information to confirm the “reality” of latent factors. This critique about “realness” collapses two separable claims: one ontological, one epistemological. The ontological claim concerns whether or not the factors are “real,” i.e., that the entity represented by the latent factor exists independently of measurement (and of measurers). Some theorists have argued that the question of factor validity is best conceptualized as a debate about the reality of latent variables¹⁸. For example, Borsboom and colleagues argued that the use of reflective latent variable models (i.e., those that assume that observed factor indices reflect the causal influence of an unobserved latent variable) necessitates a realist interpretation of latent variables¹⁹. In sum, this perspective suggests that the representation of constructs as latent variable models implies a positivist ontological claim that the construct has an objective reality.

We take the perspective that the question about whether latent constructs (or, indeed, whether any constructs) have an objective “reality” is a philosophical question that cannot be definitively resolved with the addition of new empirical data. We therefore focus on the more limited epistemological claim of whether or not the empirical techniques applied in factor analysis provide adequate empirical information to falsify the proposed model. From this perspective, factors are “a working reference frame, located in a convenient manner in the ‘space’ defined by all behaviors of a given type” ²⁰. Cronbach & Meehl suggested that the most valid construct is, regardless of whether it is objectively “real”, “one around which we can build the greatest number of inferences, in the most direct fashion.”²⁰ Although Cronbach & Meehl’s constructivist perspective avoids specifying the ontological status of the construct itself, even constructed factors are still challenged by factor indeterminacy. If the mechanisms underlying variation in a theoretical construct of interest are governed by another type of formal model (e.g., a network model), we may arrive at substantially different inferences, at least at some levels of analysis, than if that variation is erroneously modeled in terms of a latent factor.

Indeed, although factor analysis often closely accounts for patterns of correlations across a variety of empirical contexts, alternative models may describe the correlation matrices equally well without the use of latent factors. Thomson presented sampling theory as one alternative explanation for the correlations described by Spearman’s factor model²¹. Thomson wrote “the alternative theory… is that each test calls upon a sample of the bonds that the mind can form, and that some of these bonds are common to two tests and cause their correlation.”²² Thomson suggested that the appearance of a common factor was due to each higher order function sampling distinct, albeit overlapping, sets of bonds. He conceived of these bonds as “elements of [mental] activity” ²², similar to what others later conceptualized as “micro-level processes” ²³. Perhaps the most recent version of Thompson’s sampling theory is process overlap theory proposed by Kovacs and Conway, in which overlap among specific cognitive processes (e.g. verbal, visuospatial, and executive processes), rather than the action of a general factor, generates a positive manifold²⁴.

As a simplified example of sampling theory, Ceci conceived of a scenario in which three cognitive tests sample from a group of microlevel processes A, B, c, D, e, and f,²³ shown in Fig. 2b. Three of these processes (c, e, and f) are specific to only one of the three tests. Microlevel process A is sampled by tests one and two, process B is sampled by tests two and three, and process D is sampled by test one and three. The shared sampling of processes by two tests causes those two tests to be correlated. Across multiple tests, the sampling overlap produces a positive manifold and the appearance of a general factor, despite the fact that no microlevel process contributes to variation in more than two tests²³.

Another popular alternative model to the general factor model, referred to as mutualism, describes correlations among individual traits as resulting from reciprocal causation between them^25,26. Under the mutualism model positive feedback among cognitive processes over time, rather than a common factor, give rise to the positive manifold of test interrelations (Fig. 2c). The mutualism framework can be considered the precursor of the more general network approach, which has become popular in personality and psychopathology research^10,27. In network representations of psychopathology, disorders arise as a result of clusters of symptoms mutually affecting one another (represented graphically as edges connecting nodes, representing individual symptoms), rather than as a result of a latent common cause ¹¹. As discussed in Bartholomew and colleagues, mutualism and network models can be conceptualized as forms of Thomson’s bond’s model, wherein rather than representing a shared cognitive process, a bond represents (potentially mutual) developmental causation between pairs of individual traits²².

Importantly, there are many other models for which there have not been formal theories proposed that may fit the correlation matrices typically modeled using factor analysis similarly well. Tomarken & Waller emphasized that the close fit of a factor model to an observed correlation matrix is insufficient to conclude that the factor model represents anything close to the true data-generating mechanism²⁸. They capture this point succinctly in their statement “my model fits well and so do 1000 others.”²⁸ In other words, any model fit to data is but one representation of the underlying structure, and there may often be a large number of plausible alternatives including both equivalent models and non-equivalent but well-fitting alternatives. Some of the factors that Tomarken & Waller list as contributing to ambiguity in interpreting well-fitting factor models are the potential for omitted variables, good overall model fit even in the presence of poor local fit, poor sensitivity of fit indices to meaningful misspecification, and reliance on exploratory methods for model specification (“specification searches”) without replication²⁸. As we later discuss, the careful analysis of genomic data may allow us to considerably reduce the set of alternative plausible models and falsify previously unfalsifiable models.

Despite the fact that critiques of the factor model are longstanding, there continues to be little empirical evidence directly favoring one of these models over the others. While our review has focused primarily on debates specific to factor models of cognitive abilities, similar debates have focused on other latent variables, including personality traits¹⁰ and psychopathology¹¹. Ultimately, one model may be more appropriate for some constellations of variables (e.g., personality traits) and another model may be more appropriate for other constellations of variables (e.g., cognitive traits). Moreover, within a given constellation of variables, one model may be appropriate at one level of analysis (e.g., the factor model may be appropriate for explaining the relation between broader manifestations of decontextualized personality facets, like sociability and assertiveness), whereas another model may be appropriate at a different level of analysis (e.g. the bonds model may be appropriate for explaining the relations among specific manifestations of individual personality items, like enjoying parties and having many friends²⁹. Currently, the validity of latent variables is not just unknown, it is widely acknowledged to be unknowable with existing data and methods.

Elementary Processes and Beyond

Shortly after Carroll published his 1993 book, which is widely credited as establishing the modern three-stratum taxonomy of cognitive abilities, of which g occupies the apex, he admitted to being vexed by the simple version of sampling theory described by Ceci^9,23. After reporting on results of a basic simulation to test Ceci’s proposal, Carroll wrote that, if the correlations among psychological tests were the result of overlapping processes, rather than the causal influence of a single factor, the “standard procedures of factor analysis will not ordinarily reveal this.”³⁰ He concluded that “we should be more concerned about the reality of g than we usually have been” and acknowledged that “perhaps g is indeed a cluster of microlevel components that pervades the cognitive domain” ³¹. Carroll called for the identification of uncorrelated processes that could be leveraged to either falsify or support the general factor^30,31. van der Maas and colleagues similarly suggested that if sampling theory was correct, and “if we were able to measure elementary processes independently [and they were indeed uncorrelated], the positive manifold would disappear.”²⁵

Both authors converged on the same insight: One path toward resolving the factor indeterminacy problem was to identify a set of uncorrelated elementary processes relevant to the variables under study. The existence of multiple, uncorrelated causal processes could point to either one or more processes (i.e. a general factor) that is shared across all measured variables, or to a mosaic of processes that are individually shared by subsets of measured variables but none of which is shared by all measured variables. The uncorrelated nature of the processes is crucial to such an exercise: should the processes be correlated, the question remains as to what gives rise to the correlations (i.e. a general factor, or subset-wise sharing at an even simpler level of analysis).

Attempts to identify and measure orthogonal microlevel processes have a long history in cognitive psychology. Carroll first defined elementary cognitive tasks (ECTs) as those that required a “relatively small” number of mental processes to perform while also evincing different success rates depending on individual abilities or strategies³². Sternberg, Detterman, and Kranzler and Jensen all proposed models of intelligence that were fundamentally composed of independent, potentially uncorrelated, processes that in combination gave rise to higher order cognition^33–35. This research tradition aimed to refine elementary cognitive tasks to directly measure the processes that they hypothesized constituted the uncorrelated building blocks of cognition.

Attempts to measure elementary cognitive processes resulted in a set of narrowed but still correlated tasks ^36–38. Kranzler and Jensen noted “the fact that individual differences in conceptually distinct processes (as measured by ECTs) are correlated indicates the presence of some more fundamental level of processes, presumably neurological, which are shared by conceptually distinct information processes.”³⁵ Although Kranzler and Jensen suggested establishing neurological correlates as the next step³⁵, there has been little success in identifying orthogonal elementary processes using brain imaging approaches. Consistent with sampling theory, multiple studies have found cognitive tasks correlate with brain structure across many overlapping brain regions ^39,40. However, these brain regions are themselves strongly correlated^41,42, thus leaving open the question of whether a shared common factor causes those variables to be correlated.

In summary, it remains unanswered whether putative elementary processes are themselves correlated because they share a common factor, or because they share a distributed suite of narrower uncorrelated processes. In the study of cognition, attempts to identify uncorrelated elementary processes have failed. In other areas of the behavioral sciences (e.g. psychopathology, personality), there are even fewer examples of attempts to identify such uncorrelated processes. To date, attempts to reduce behavioral measures into uncorrelated biological components has proven to be just as challenging as attempts to reduce behavioral measures into uncorrelated cognitive components. In the following sections, we describe how genomic data overcome the fundamental limitation shared by brain imaging, elementary cognitive tasks, and other “elementary” approaches. We explain why, given appropriate designs, because different segments of the genome are effectively uncorrelated and because genetic associations capture a potentially wide range of unobserved mechanisms that give rise to interindividual variation in psychological and behavioral phenotypes, associations involving individual genetic variants provide especially unique and valuable information for addressing the factor indeterminacy problem.

The Genome is a Set of Elementary Processes

As described above, any attempt to decompose complex constructs into elementary components that are themselves correlated leaves open the question of whether the correlations arise from a common factor or an alternative data generating mechanism. The genomics revolution provides a radical alternative to the incremental, level-by-level search for elementary processes that, to date, has failed to produce reliable sets of uncorrelated predictors of complex phenotypes. Genetic polymorphisms are at the farthest upstream end of a causal chain of biological and social processes that culminate in interindividual variation in downstream phenotypes. Individual differences in genetic code (DNA) at the molecular level come to be associated with individual differences in phenotypes through a long causal chain including mutually interacting biological and social processes, many of which are not yet well understood or documented (Fig. 3). Indeed, associations between individual polymorphisms and complex phenotypes are exceedingly small⁴³.

Figure 3. Heuristic representation of the causal chain from genetic polymorphisms to complex phenotype under a factor model scenario and overlap scenario.

Genetic polymorphisms in the genome represent the farthest upstream processes in the causal chain that, through a series of cascades in which intermediate processes combine and interact, manifest in phenotypic differences. Complex traits are affected by many individual genetic polymorphisms that give rise to intermediate processes that at intermediate points along the causal chain combine with one another to affect a more limited set of pathways. These pathways in turn give rise to individual differences in the phenotype of interest. A fundamental question in the behavioral sciences is whether phenotypes belonging to a given constellation are correlated because they all draw on a common intermediate process (as in the factor scenario), or because subsets of phenotypes within the cluster draw on overlapping sets of processes without any given process being common to all phenotypes within the constellation (as in the overlap scenario). It can be seen that under both scenarios, many intermediate processes share further upstream processes, which may explain why previous attempts to distill complex phenotypes into uncorrelated sets of building blocks were unsuccessful. Note that only pleiotropic pathways (i.e., those linking a single variant with multiple phenotypes) are represented. Direct pathways linking individual variants to individual phenotypes are also assumed under both scenarios. Protein images from the Mol* viewer⁷⁵, RCSB PDB (rcsb.org) of PDB ID (from left to right) 6VZK ⁷⁶, 1ZBD ⁷⁷, 7VFS ⁷⁸.

Recent surges in the sample sizes of genomic studies have provided sufficient statistical power both to detect individual genetic effects and to gauge the total amount of association signal among tagged variants, including those that by themselves do not pass stringent multiple-comparison corrected significance thresholds. Even more importantly, when ancestral background and assortative mating are appropriately controlled for (ideally, by way of within-family genetic association data ⁴⁴), spatially distal regions of DNA are uncorrelated with one another.

The statistical independence of spatially distal genetic variants, which results from Mendel’s laws of segregation and independent assortment⁴⁵ (Box 1: Why is DNA independent?), means that correlations among heritable phenotypes must arise via downstream junctures and are not themselves caused by a common process further upstream. Genetic variants are unique among explanatory variables in that they exist at the furthest upstream level of complex cascades of mechanisms and are effectively uncorrelated with one another, thus satisfying the key conditions stipulated by researchers (e.g., Ceci, Carroll, van der Maas, as reviewed in the previous section) seeking to identify elementary processes that could be used to resolve the factor indeterminacy problem. Next, we provide some basic background on the properties of DNA of particular relevance to Genome Wide Association Studies (GWAS). We then detail how GWAS data can be used to surmount the factor indeterminacy problem.

Box 1 |. Basic Concepts in Genetics.

What is DNA?

The human genome is made up 23 pairs of chromosomes containing coils of DNA. DNA is itself made up of nucleotides (see Box 2). Nucleotides group into coding regions, which code instructions for the formation of proteins, and noncoding regions, which often include regulatory or other functions.

Why is DNA independent?

Genetic inheritance involves a process known as meiosis, in which two chromosomes, one from each parent, are randomly split and recombined to form two new chromosomes that each contain half of the genetic material from each parent (Laird & Lange, 2011). The recombination process ensures that genetic material is, generally speaking, independently assorted: The probability of inheriting from one’s parent a particular allele within one region of the genome is independent of the probability of inheriting a particular allele on another region of the genome.

How does linkage disequilibrium affect independent assortment?

DNA is inherited from parents in segments. Thus, although segments of DNA are generally independently assorted, variants that are near one another tend to remain correlated. These correlations are referred to as linkage disequilibrium (see Box 2). Groups of spatially proximal genetic variants that are in high linkage disequilibrium with one another are referred to as LD blocks. Variants that are in different LD blocks are effectively uncorrelated with one another (i.e., LD blocks can be considered independent).

How does population stratification affect independent assortment?

Correlations among spatially distal loci are expected to occur in instances in which individuals from distinct ancestral populations are included in the same sample. Including very heterogeneous samples of individuals from different ancestral populations can result in population stratification (see Box 2). For this reason, genetic analyses are typically conducted on samples that are ancestrally homogeneous at a macro (e.g., continental) scale one at a time, and further statistical controls are employed to account for residual population stratification (e.g., regional variation within individuals of European ancestry). Under such conditions, spatially distal genetic variants are expected to be uncorrelated, whereas spatially proximal genetic variants remain in high LD.

Background: Genotype-Phenotype Pathways and Genome-Wide Association Studies

We provide background on relevant key concepts in genetics in Box 1 with a glossary of terms in Box 2. Genomic work in the past decade has underscored that nearly all human traits are heritable and highly polygenic, meaning that traits are influenced by many variants that each have small effects ^46,47. These small effect sizes require very large sample sizes to reach genome wide significance (typically set at p < 5e-8 to account for multiple testing), and many variants may have meaningful effects on the phenotype that are not significant due to underpowered sample sizes or shallow phenotyping ^46,47. Genome wide association studies provide important information about variants that do reach the significance threshold (“hits”), but also provide additional information about the total genetic signal for a trait that might be informed by sub-threshold effects ⁴⁸.

Box 2 |. Glossary of Genetic Terms.

nucleotide	The basic structural unit of DNA and RNA.
polymorphism	A difference in DNA sequence that commonly occurs among individuals or populations, at the level of single nucleotides (SNPs) or sequences of spatially contiguous nucleotides (haplotypes).
allele	The variant of a given polymorphism (i.e. A, C, T, or G) in the genome.
linkage disequilibrium (LD)	Patterns of correlation between spatially proximal genetic loci in the population that result from genetic material being passed to offspring in large segments of many base pairs.
Population stratification	Spurious genetic associations that result from confounding environmentally caused differences in phenotypes with differences in allele frequencies across different populations.
Genome-wide association study (GWAS)	Quantification of associations of genotyped variants (most often SNPs) on a phenotype of interest by estimating a linear regression for the effect of each individual variant on the phenotype. GWAS tend to focus on common variants that are present in 1% or more of the general population (Visscher et al., 2017).
genetic correlation	An aggregate index of the degree of genome-wide genetic sharing between a pair of traits that can be conceptualized as the correlation between genetic propensities towards each of the two traits. Genetic correlations do not simply index whether the causal variants relevant for each of the traits overlap, but also take into account consistency in the direction and relative magnitude of effects.

GWAS do not provide direct insight into the mechanisms underlying an observed link between a genetic variant and variation in a complex phenotype. However, their results can be used to establish genotype-phenotype associations that serve as “bread crumbs” indicating potential avenues for further interrogation and investigation. Under best case scenarios, this will result in a complete understanding of the pathway from gene sequence to phenotype. These pathways may involve the interplay between both biological and experiential processes. For example, a relatively early (and now highly replicated) GWAS discovery was that variants within the nicotinic acetylcholine receptor (CHRNA5) gene are associated with lung cancer. As described by Kendler and colleagues,

“Many of these associations (but perhaps not all) are mediated through smoking quantity… A genetic variant in a nicotinic receptor increases risk for nicotine dependence, which then causes individuals to go out into the environment, purchase cigarettes and repeatedly inhale the smoke into their lungs, thereby increasing their risk for lung cancer… The gene increases risk for nicotine dependence and it is the drug that is ingested as a result of the dependence that causes all (or at least most) of the oncogenesis.”⁴⁹

Importantly, when ancestry is appropriately controlled, experiential pathways such as the sort described above are downstream along the causal chain from gene sequence. (The most rigorous method for controlling for ancestry is to conduct GWAS within-family so as to estimate associations that are not only free of “indirect” genetic effects but also largely free of confounds associated with population stratification and assortative mating.⁵⁰) This is because, while experiences may alter gene expression, they (with rare exceptions, such as high dose radiation exposure) cannot alter gene sequence. More commonly, even in situations in which complex pathways are elucidated, full mechanistic insight rarely occurs. Fig. 3 provides a heuristic depiction of the pathway from genotype to multiple phenotypes under two scenarios: one in which all processes converge through an intermediate “common factor” which then affects the phenotypes, and another in which two phenotypes might share intermediate phenotypes but there is no single process shared among all measured phenotypes. In both scenarios depicted in Fig. 3, polymorphisms (e.g., SNPs) determine the construction and regulation of proteins, which then influence a variety of intermediate processes represented by blue circles with arrows indicating the direction of the effect. Although the blue circles often represent unknown processes, it is still possible to capitalize on empirical associations between the SNPs and phenotypes to distinguish between different forms of genetic sharing.

The Factor Indeterminacy Problem Persists when Analyzing Genetic Correlations

In the next section, we demonstrate why associations involving individual genetic variants provide especially unique and valuable information for addressing the factor indeterminacy problem. First, however, is it important that we make clear that the naive implementation of genetic methods does not by itself do anything to obviate the factor indeterminacy problem. This is perhaps clearest in the example of factor analyzing genetic correlations. Genetic correlations index the degree of genome-wide genetic sharing among phenotypes (Box 2) and are especially informative in that they draw on information from all available variants regardless of significance, and they are not biased by estimation error in the GWAS effect sizes. A high positive genetic correlation between a pair of phenotypes indicates that the effect sizes of genetic variants on each of the phenotypes correspond closely to one another: those variants with effects on the first phenotype tend to be the same variants with effects on the second phenotype, the effects tend to be in the same direction for the two phenotypes, and those with the largest effects on the first phenotype also tend to have the largest effects on the second phenotype. There are a variety of methods to estimate genetic correlations, some of the most popular of which do not require access to individual level data, nor do they require that both phenotypes be measured in the same sample ⁵¹. This has been tremendously advantageous in opening up opportunities to examine genetic sharing among phenotypes that would be difficult (e.g. two disorders with extremely low prevalence rates; or early childhood cognitive function and late onset dementia) or even impossible (e.g. two disorders whose diagnostic criteria are mutually exclusive of one another, e.g. major depression [which does not include manic features] and bipolar disorder [which includes both depressive and manic features]) to measure in in the same sample. The integration of data from disjointed samples also provides a particularly unique means of avoiding spurious correlations that can result from shared method bias (such as a general tendency for some participants to provide acquiescent responses more than others) or when the same individual-level data are used for determination of multiple phenotypes (such as when a single endorsed symptom contributes to symptom counts for multiple disorders). However, integrating data from disjointed samples poses an interpretive challenge: To what populations and constructs can the results be generalized? For instance, when a genetic correlation is computed between one diagnosis made in a child sample and a second diagnosis made in an adult sample, this correlation must be interpreted as representing genetic risk sharing across disorders spanning development (and likely contexts) and not interpreted as representing genetic risk sharing for concurrent trait levels or comorbid disorders. We return to these issues in describing empirical applications later on.

Just as factor models can be fit to phenotypic correlation matrices using structural equation modeling, factor models can be fit to genetic correlation matrices using genomic structural equation modeling (Genomic SEM⁵²). A common factor extracted from a genetic correlation matrix can be interpreted as representing a dimension of genetic variation that is broadly relevant to all the phenotypes that load on it. However, because genetic correlations are aggregate-level measures of genetic sharing across an extremely large number of variants, the same interpretational ambiguities discussed earlier with respect to standard factor models also apply to factor models fit to genetic correlation matrices: just because a factor model fits a genetic correlation matrix well, does not mean that the factor model is correct. Put differently, just because positive genetic correlations are observed across all combinations within a set of phenotypes, does not necessarily mean that there are any variants that affect all phenotypes; one set of variants may have similar effects on each of one pair of phenotypes and an entirely different set of variants may have similar effects on each of a different pair of phenotypes. Thus, in order to leverage genetic data for disambiguating this indeterminacy, we must extend our genomic structural equation modeling analysis to include associations involving individual genetic variants and not just genetic correlations.

Associations with Individual Genetic Variants Provide Leverage to Resolve the Factor Indeterminacy Problem

To demonstrate why evaluating the factor model using individual variant-level genetic data offers a particularly strong means of falsifying the factor model (or supporting it via a particularly rigorous stress test against which it holds up), we return to the scenario described by Ceci by expanding the simple simulation described by Carroll^23,30. We purposefully focus on a relatively simple simulation to highlight the key insights that are gained from employing variant-level information without obscuring these insights with ancillary complexity. We simulated data for nine phenotypes under two generating models: a common factor model (Fig. 4A) and an overlap model (Fig. 4B). Full details are described in the Supplementary Note. Our goal was to use each generating model to produce a 9 × 9 correlation matrix containing 3 clusters (domains) of variables each, in which variables within clusters were highly correlated and variables between clusters were moderately correlated. Such a pattern corresponds closely to commonly observed empirical patterns in the behavioral sciences, such as a broad personality domain (e.g., extraversion) containing facets (e.g., gregariousness, assertiveness, excitement seeking), general intelligence and individual ability domains (e.g., abstract reasoning, processing speed, episodic memory) or a broad clinical disorder (e.g., attention deficit hyperactivity disorder; ADHD) that contains subtypes (e.g., hyperactive, impulsive, inattentive).

Figure 4. Structure of simulations for common factor and overlap generating models.

In the common factor model (A) intermediate phenotype A influences all nine phenotypes, and intermediate phenotypes b, c, and d each affect three phenotypes. In the overlap model (B) each intermediate phenotype (B, A, and C) influences six phenotypes.

We generated data for each phenotype by first generating intermediate phenotypes that were themselves each affected by two components: an individual SNP and by a random normally distributed component (representing other, unspecified, genetic and environmental sources of variation). All SNPs were specified to be uncorrelated. In the common factor simulation, each phenotype was affected by one domain-specific intermediate phenotype (affecting three phenotypes total), a general intermediate phenotype that affected all domain-specific intermediate phenotypes (and thus all nine of the phenotypes), and a phenotype-specific residual. In the overlap simulation, each phenotype was affected by two bi-domain intermediate phenotypes that affected phenotypes belonging in two domains (i.e., six phenotypes total), and a phenotype-specific residual. Thus, in the overlap simulation, no single intermediate phenotype affected all nine of the phenotypes, but each phenotype was linked to phenotypes in all other domains via combinations of bi-domain intermediate phenotypes. In these simulations, the shared intermediate phenotypes can be taken to represent shared biological or cognitive processes, or developmental causation between pairs of intermediate traits.

The correlation matrices obtained for the 9 simulated phenotypes were indistinguishable across the two generating models (Fig. 5, square matrices in panels A and B). In the factor model, positive correlations arise because a single intermediate phenotype underlies every phenotype, causing all phenotypes to be correlated with one another. In the overlap model, positive correlations arise because every pair of phenotypes is influenced by at least one shared intermediate phenotype, even though no single intermediate phenotype influences all phenotypes. These simulations illustrate the factor indeterminacy problem: Although two theories may specify fundamentally different data generating models, different models may not translate to differences in observed data. Indeed, when confirmatory factor analysis was applied to fit a common factor model to the data from both generating models, fit was both excellent and indistinguishable across models (Supplementary Table 1), underscoring the fact that model fit cannot be unequivocally used to arbitrate between causal models. (In the Supplementary Note, we report results of a more complex simulation in which the patterning of overlap is randomly generated on a SNP-by-SNP basis, producing a positive manifold of intercorrelations to which a general factor model also fits exceedingly well, yielding all positive and significant loadings on a general factor.)

Figure 5. Correlation matrices generated from process overlap and common factor models.

Correlation matrices for simulated data generated from (A) a common factor model and (B) a process overlap model over 1000 replications. Larger, darker circles indicate stronger positive correlations. Square matrices show correlations among the phenotypes, and rectangular matrices show correlations between SNPs and the phenotypes. The correlation matrices among the phenotypes are identical, however when SNP effects are included they become starkly distinguishable.

When the effects of individual SNPs are explicitly added to the correlation matrix, stark differences between the overlap and common factor models are evident (Fig. 5, rectangular matrices in panels A and B). Without individual SNPs in the correlation matrix, factor indeterminacy remains. However, with the individual SNP effects incorporated, we observe clear differences: In the common factor simulation (Fig. 5A) we observe associations between SNP A and all nine phenotypes, whereas in the overlap simulation (Fig 5B), we observe that no SNP is associated with more than six phenotypes. This simple visualization illustrates how associations involving individual genetic variants can be leveraged to test between competing otherwise indistinguishable explanatory accounts. Importantly, because the SNPs are not themselves correlated, there is no question about what gives rise to further upstream correlations – such correlations are null and thus not in need of further explanation.

Formally Testing the Factor Model at the Level of Individual Variants

We have intuitively shown how individual genetic variants can be used to distinguish between generating models. Indeed, the relationship between variants and the underlying data generating model can be formally quantified by comparing analytic models of the data (Supplementary Figure 2). In the first model, the SNP associations with the nine individual phenotypes are specified to occur by way of a common factor, such that the expected SNP effects are proportional to each phenotype’s loadings on the factor (a common pathway model). In the second model, the SNP associations with the nine individual phenotypes are specified to occur directly, such that the expected SNP effects are unconstrained by the factor loading pattern (an independent pathways model). When all phenotypes load strongly and positively on the common factor (as will always be the case when their intercorrelation matrix exhibits a positive manifold), the common pathways model produces the restrictive expectation that any SNP that is associated with one phenotype is associated with all nine phenotypes - an expectation that will be violated if the SNP does not indeed act on the phenotypes via the factor. The independent pathways model produces no such expectation. The comparison between the common and independent pathways model produces a SNP-specific test statistic that indexes the extent to which the individual SNP associations with each of the phenotypes departs from those that would be expected on the basis of a model in which the SNP acts on them exclusively via the factor ^{c.f. 53–55}. We term this SNP-specific test statistic Q_SNP, and view it as a heterogeneity statistic, in the sense that it indexes heterogeneity in SNP-phenotype relations beyond that which would be expected under the factor model ⁵⁶. When Q_SNP is low, SNP-phenotype associations conform to the pattern expected under the common pathways model. When Q_SNP is high, the observed pattern of associations is inconsistent with the expectations of the common pathways model. In other words, when Q_SNP is high, the proposal that the genetic variant operates on the factor is falsified.

Importantly, because the heterogeneity statistic (Q_SNP) is calculated separately for each SNP, we expect to observe a distribution of Q_SNP values across variants. A SNP exhibiting high Q_SNP may violate the expectations of the factor model either because the factor model is universally incorrect, or because the factor model is indeed correct but that particular SNP operates by way of a pathway that does not include the common factor (e.g., it is a domain-specific SNP). Given adequate statistical power, the factor model predicts that at least some SNPs will have meaningfully high associations with the factor and low Q_SNP values, whereas alternative models such as the overlap model predict that any SNP that is statistically associated with the factor will exhibit high Q_SNP, indicating that the factor association is spurious. Thus, while the Q_SNP is informative about each specific SNP under investigation, it can be even more informative when its distribution is examined across SNPs. Examining Q_SNP estimates across many SNPs sampled from across the genome enables us collect genome-wide evidence to inform our evaluation of the validity of the factor model as a whole. When power is high, and the factor model is incorrect, Q_SNP can be used to falsify the factor model. For instance, we can examine the number of genome-wide significant “hits” for Q_SNP (i.e., the number of SNPs that violate the expectations of the factor model beyond a predetermined, typically genome-wide, significance threshold), whether these Q_SNP hits are for the same SNPs that exhibit genome-wide significant associations with the factor (indicating that such associations are likely spurious), and the relative number of Q_SNP hits to factor hits. Moreover, we can examine the full distribution of Q_SNP statistics, including those that do not surpass the significance threshold, to gauge the overall amount of Q_SNP signal in the data.

Examining Patterns of SNP Associations across Generating Models

To demonstrate how Q_SNP behaves under different data generating models, we extended our simulations to include a total of 1000 iterations for each generating model, generating 5000 SNPs total (The methods for the simulation are described in the Supplementary Note). We begin by describing the statistically significant associations. The number of SNPs that have statistically significant associations with the factor (referred to as “hits” on the factor) compared to those that have a significant heterogeneity (hits for Q_SNP) differ between the simulated factor and overlap generating models. In the factor condition, 71 SNPs are significant hits for only the factor and 6 SNPs are significant hits for Q_SNP. In the overlap condition, only 1 SNP is significantly associated with the factor and 13 SNPs are significant hits for Q_SNP. In the factor condition, SNPs significant for Q_SNP are domain-specific SNPs, while significant hits for Q_SNP in the overlap condition are overlap SNPs.

In order to compare the distribution of test statistics for association of the SNP on the factor to Q_SNP, we express both as a χ²(1) statistic. Note that χ² is a test statistic where higher values are more significant. Thus, high association χ² represents a SNP that is more significantly associated with a common factor, and high Q_SNP χ² represents a SNP that is more significantly heterogenous, acting through domain or phenotype-specific pathways.

The differences between genome-wide distributions of Q_SNP χ²(1) and association χ²(1) for common factor SNPs (affecting all 9 phenotypes), overlap SNPs (affecting 6 phenotypes from two domains), and domain-specific SNPs (affecting 3 phenotypes from one domain) are shown in the SNP-specific distribution of Q_SNP χ²(1) and association χ²(1) statistics (Fig. 6). The distribution of Q_SNP χ²(1) statistics for overlap and domain-specific SNPs is highly inflated in that it is displaced substantially above 1 (the expected mean under the null), indicating that the common pathways model provides a poor representation of these SNPs’ associations with the 9 phenotypes (Fig. 6B). In contrast, the distribution of Q_SNP χ²(1) for common factor SNPs clusters tightly around 1, indicating that the common pathways model represents the SNP effects on the 9 phenotypes well. In other words, as expected, SNPs acting through the common factor result in lower Q_SNP χ²(1) values than SNP acting on processes specific to a subset of phenotypes and not through the common factor. The distribution of association χ²(1) statistics for the factor SNPs is highly inflated, indicating high levels of association with the factor, and the low values of Q_SNP χ²(1) for these factors indicates that the common pathways model is plausible. In contrast, although the distribution of the association χ²(1) statistics for the overlap SNPs is also somewhat inflated, the high displacement of the Q_SNP χ²(1) for these SNPs indicates that their association signal is attributable to model misspecification. That the distribution of association χ²(1) statistics for the domain-specific (group) SNPs clusters close to 1 and the distribution of Q_SNP χ²(1) is inflated indicates that these SNPs do not act on the factor. Patterns of Q_SNP χ²(1) and association χ²(1) at the level of individual SNPs therefore provide considerable leverage for distinguishing a common factor model from otherwise similarly well-fitting alternative models.

Figure 6. Histograms of the distribution of the heterogeneity statistic and the association statistic across simulation conditions.

The heterogeneity statistic, QSNP, represents the extent to which the expectations of the factor model are violated at the level of the individual SNP. When QSNP is low, the SNP effects conform to the pattern expected under the factor model, whereas when QSNP is high, the observed pattern of associations is inconsistent with the expectations of the factor model. The association statistic represents the strength of the estimated association between the SNP and the factor, assuming the factor model is correct. Higher values indicate stronger associations. Panel B displays histograms of QSNP χ²(1) values per iteration generated by factor, overlap, and domain-specific SNPs. It can be seen that the entire distribution of QSNP χ²(1) is displaced substantially to the right for overlap and domain-specific SNPs compared to factor SNPs. Panel A displays the histograms of mean Association χ²(1) for factor, overlap, and domain-specific SNPs. It can be seen that more pleiotropic SNPs display greater association signal, as the factor SNPs are displaced farther to the right than the overlap SNPs, and both are displaced farther right than domain-specific SNPs. Dashed lines show the means of the distributions.

Recent Empirical Applications

We have described scenarios in which a common factor model may fit the data well, even when the data were generated under a model in which no causal mechanism is common across the set of variables loading on the factor. Further, we have demonstrated a framework by which the measurement of individual genetic variants can be leveraged to falsify the factor model under such scenarios, overcoming the limitations of past data to provide a solution to the factor indeterminacy problem. Here we describe several recent empirical applications of this framework. These empirical applications differ from one another in their levels of individual variant-level empirical support for the validity of the factor models, indicating that the approach described does not “have a finger on the scales” and instead can produce both evidence for and against latent factor models depending on the data. We take care, however, to interpret the results in light of the empirical data being modeled and point to outstanding inferential questions.

General Psychopathology (p).

The p factor has been proposed as a general factor explaining comorbidity and genetic sharing among psychiatric disorders ^57,58. Factor analyses at both phenotypic and genetic levels of analysis have indicated that a p factor can consistently be extracted from a diverse set of diagnoses or symptom counts, is stable over time, and is highly heritable, suggesting that diverse forms of psychopathology may have a common genetic component ^59–62. Interpretation of these results varies considerably across scientists, with some suggesting that p may not be a cohesive construct ^61,63,64.

Among the first empirical applications of Genomic SEM, Grotzinger and colleagues fit a genetic p factor model to five psychiatric disorders (Schizophrenia, Bipolar Disorder, Major Depressive Disorder, Post Traumatic Stress Disorder, and Anxiety Disorders)⁵⁶. All disorders loaded positively and significantly on the genetic p factor, with standardized loadings ranging from .29 for Post-Traumatic Stress Disorder to .86 for Schizophrenia, and the model fit was acceptable. When individual variants were incorporated into the model, 128 independent loci were associated with the p factor at genome-wide significant levels whereas only independent locus 1 was significant for Q_SNP, indicating strong overall support for the p factor at a molecular level of analysis.

In a substantial update to the 2019 analysis, constituting an expansion of both the sample sizes and the range of disorders under investigation, Grotzinger and colleagues examined the genetic structure of 11 different psychiatric disorders and tested whether SNPs associated with these disorders acted via the p factor, group factors, or disorder-specific mechanisms⁶⁵. The additional disorders included several categories of disorders not well represented in the 2019 analysis: compulsive disorders (e.g. Tourette Syndrome, Obsessive Compulsive Disorder, Anorexia), neurodevelopmental disorders (e.g. Autism, Attention Deficit Hyperactivity Disorder), and problematic alcohol use. Although there were positive genetic correlations across all 11 disorders, and a p factor could be fit to the data with positive and significant loadings and good fit, the p factor was not well supported by an analysis that included individual genetic variants. In a hierarchical factor model that included domain-specific factors (representing Neurodevelopmental Disorders, Internalizing Disorders, Psychotic Disorders, and Compulsive Disorders) in addition to a higher order p factor, only 2 independent genetic loci achieved genome-wide significance for a general factor, whereas 69 loci exhibited genome-wide significant Q_SNP. The mean association χ²(1) for p was 1.795 and the associated mean Q_SNP χ²(1) was 1.667. Relatively similar association χ²(1) and Q_SNP χ²(1) for p despite 2 genome-wide significant association hits and 69 genome-wide significant Q_SNP hits suggests the action of many variants with highly pleiotropic effects across subsets of disorders, but few if any that act exhaustively across the entire set of disorders according to a pattern that would be expected under a p factor model. Indeed, in the same data, narrower domain factors representing subsets of disorders exhibited patterns more consistent with their validity. For instance, narrower internalizing and psychotic disorders factors exhibited a higher number of genome-wide significant association SNPs (44 and 108, respectively) and far fewer genome-wide significant Q SNPs (3 and 6, respectively). The results obtained from this study are consistent with an account of genetic sharing in which correlations among factors representing different disorder domains arise from shared pathways among different subsets of domains, rather than a single pathway shared across all domains⁶⁵.

There are a number of explanations for differences in conclusions of the 2019 and 2022 analyses (i.e., support for p in the earlier analysis but not the later analysis). One possibility is that a p factor may be a good account of genetic sharing underlying the set of major psychiatric adult disorders examined in the 2019 paper, but may not be so broad as to be a sufficient account of genetic sharing among the broader set of disorders (including neurodevelopmental, substance use, and compulsive disorders in addition to major adult psychiatric disorders) examined in the 2022 analysis. A second possibility is that the expanded sample size and range of disorders integrated in the 2022 analysis may have spanned a wider range of birth cohorts, ages, and social contexts, such that any variants associated with the p factor would have not only needed to have been shared across all disorders, but also across all ages and contexts. Under this latter scenario, the genetic architecture of the p factor is expected to shift across ages, time, and/or contexts. However, because the integration of data across potentially heterogeneous samples isolates genetic signal that is shared across ages and contexts, one might have expected such an approach to be best suited for identifying genetic variants shared across disorders. Indeed, Cai and colleagues found that aggregating more heterogeneous phenotypes for depression resulted in greater estimates of genetic sharing with other psychiatric disorders⁶⁶. At the same time, the analysis of data from disjointed samples provides a particularly rigorous means of avoiding spurious genetic sharing that result from shared method bias (such as interindividual differences in the general tendency to provide acquiescent responses to symptom queries). Nevertheless, it is important that continuing applications of these methods exercise care in considering potential differences in recruitment, participation, ascertainment, and measurement across datasets.

General Intelligence (g).

Spearman discovered the positive manifold of cognitive ability interrelations and developed factor analysis to instantiate his “two factor” theory that diverse cognitive variables are positively correlated because they are all influenced by a common factor, which he referred to as general intelligence (g), in addition to being influenced by variable-specific factor (s)⁶. This positive manifold has robustly replicated over countless studies throughout the past century, so much so that Deary described it as “arguably the most replicated result in all psychology.”⁶⁷ Although a model containing a g factor (in addition to domain-specific and test-specific factors) continues to be the most commonly accepted explanatory account for the positive manifold^9,68, several alternative models have been proposed that do not posit a g factor ^24,25,21.

De la Fuente and colleagues examined the genetic factor structure of seven diverse cognitive tasks, including memory, matrix reasoning, trail making, and verbal-numeric reasoning (VNR)⁶⁹. They found that a genetic g factor accounted for 58% of the genetic variance in the seven cognitive tasks. They reported 30 genome-wide significant hits for the genetic g (3 of which were determined to be false positive associations with g as indicated by Q_SNP) factor and 24 genome-wide significant Q_SNP hits ⁶⁹. The mean association χ²(1) for genetic g was 1.471 and the mean Q_SNP χ²(1) was 1.337. Just like the p factor, the g factor also displayed a positive manifold of correlations and an overall balance of association χ²(1) to Q_SNP, but the g factor had similar numbers of genome-wide association hits (that were not Q_SNP hits) and Q_SNP hits. A later functional genomic analysis of the same data by Grotzinger and colleagues found that the number of genes and gene categories evincing patterns of associations with the seven cognitive traits consistent with their operation via the g factor exceeded the number of genes and gene categories evincing patterns of associations inconsistent with their operation via g⁷⁰. These findings are consistent with a factor model in which many genetic variants act on individual cognitive tasks via a g factor, and others act on the tasks via more specific pathways. An example of a variant that exhibited patterns of associations with the individual cognitive tasks that was inconsistent with its operation via the g factor was a genome-wide significant Q_SNP hit located within the APOE region. The APOE region is known to be associated with risk for cognitive aging and dementia ⁷¹. Follow-up analyses indicated that the pattern of associations between this variant and the individual cognitive tasks was consistent with the expectations of the factor model for all tasks except the VNR task, which is the only task within the set that exhibits mean increases through late middle age (the remaining 6 tasks exhibit progressive mean declines across the 40–75 year age range under study). The content of the VNR tasks suggests that it is unique among the tasks studied that draws strongly on crystallized knowledge, a domain of cognitive function that is well known to be robust to aging-related cognitive declines until later adulthood. De la Fuente and colleagues inferred that that this variant is related more specifically to the aging of fluid abilities, rather than more broadly associated with general intelligence⁶⁹. Such non-g pathways are expected under a factor model in which both general and specific factors account for variation in the individual cognitive tests.

Externalizing Psychopathology.

Externalizing describes intercorrelated psychiatric traits related to self-regulation, such as risk-taking behavior, substance use disorders, and antisocial behaviors⁷². Linnér and colleagues examined the genetic factor structure of seven externalizing phenotypes and modeled an externalizing common factor among them⁷³. The overall association χ²(1) and Q_SNP χ²(1) were 2.98 and 1.956, respectively. They reported 579 loci significant for the externalizing factor, few of which showed heterogenous associations, supporting a unitary dimension of genetic liability across all seven indicators rather than associations specific to certain phenotypes. They additionally reported 160 loci significant for Q_SNP, the majority of which were found outside of the loci significant for the externalizing factor. These results are consistent with a factor model in which many genetic variants act on a general externalizing factor, and others act on individual externalizing behaviors via specific pathways. One strong trait-specific association with a highly significant Q_SNP was a SNP located in the alcohol dehydrogenase 1B (ADH1B) gene, which is well-characterized in the biology of alcohol metabolism ⁷⁴, and therefore expected to operate on problematic alcohol use though a specific pathway, rather than through a general externalizing factor. This observation illustrates the inferential strength of variant-level heterogeneity analyses for identifying specific pathways of risk, even when the factor model holds more generally.

Discussion

In standard empirical settings, latent variable models are indistinguishable from alternative models that predict intercorrelations across measures without proposing general mechanisms or pathways. We have described a theoretical and empirical framework by which genomic data can be leveraged for validating or disconfirming latent variable models in exactly these scenarios. By incorporating measured genetic variants into a factor model, researchers can formally test whether the patterns of association between each individual genetic variant and the indicators of that factor are consistent with the expectations of a model in which the effects act through the factor. The factor model can be considered plausible when many independent variants measured across the genome exhibit such a pattern. Alternatively, when few genetic variants exhibit patterns of associations with individual indicators that is consistent with the expectations of the factor model, and many independent genetic variants exhibit patterns of associations inconsistent with the expectations of the factor model, it can be inferred that the factor model is not valid, or at least not useful at a disaggregated level of analysis. In such latter circumstances, correlations among indicators of a proposed factor are likely to be best described as arising from more specific pathways shared by subsets of those indicators.

Importantly, the approach that we have described here is of particular value for testing the validity of the factor model itself, but, without considerable modification, it is not specifically equipped for testing between the many alternatives to the factor model. The independent pathways model used to generate the Q_SNP heterogeneity statistic is unrestrictive and thus able to stand-in for a variety of alternatives to the factor model, but does not meaningfully differentiate between them. Heterogeneity tagged by Q_SNP may arise, for example, from reciprocal causation during development (a network model) or pairs of shared mechanisms across subsets of phenotypes (an overlap model).

The approach that we have described is also equipped to distinguish the factor model from scenarios in which intercorrelations arise from a heterogeneous set of general pathways. For example, one set of SNPs may increase one’s predisposition to each phenotype in the factor model by a uniform amount, and another set of SNPs may increase one’s predisposition to half the phenotypes by a small amount and the other half by a large amount. Such scenarios intuitively resemble the factor model in that there are intermediate pathways or processes that influence all phenotypes in the system. However, when a general factor is fit to data generated under this scenario, the specific pattern of factor loadings obtained will represent the amalgamation of heterogeneous patterns associated with the individual pathways, thus typically failing to represent any specific pathway. By including individual genetic variants in an analytic model in the way that we have described here, one would be able to discern between circumstances in which a single general shared pathway vs. multiple heterogeneous, albeit general, pathways operate. Thus, the approach described here provides a stringent test of the factor model even relative to conceptually close models.

By including individual variants in our modeling, we gain substantial leverage on otherwise intractable questions. Nevertheless, by relying on genetic sources of information to identify our models, we are unable to provide direct tests of environmental structure that is uncorrelated with genetics. If the latent factors that we seek to test are thought to represent phenotypic entities, then their effects would be expected to generalize across genetic and environmental components. This is the same core assumption used in Mendelian randomization (MR) studies, in which genetic variants are used as instruments so as to infer causal effects between pairs of variables ⁴⁵.

The ability to identify plausible causal models is crucial for the development of effective interventions in the behavioral sciences. As we have discussed, patterns of phenotypic correlation structures are not sufficient for the validation of theoretical models. We have focused this paper specifically on the unique properties of individual genetic variances for providing leverage on the factor indeterminacy problem. Not only do genetic variants represent the furthest “upstream” level of analysis, but they are uniquely constituted of uncorrelated components of phenotypically-relevant variation. We have highlighted properties of individual genetic variants that make them particularly advantageous for answering questions with respect to the validity of latent factors. When the inferential goal is not to evaluate this question of validity, but to evaluate pragmatic questions regarding whether factors are inferentially useful in terms of describing epidemiological risk or therapeutic efficacy, it may be sensible to apply the approaches described here at further downstream levels of analysis. For instance, in addition to testing associations involving individual genetic variants, Grotzinger and colleagues tested whether associations between individual psychiatric disorders and external biobehavioral traits are well-described as operating through broad factors underlying the psychiatric disorders⁶⁵. The answer to whether a factor is appropriate for describing associations with biobehavioral correlates and whether it is appropriate for describing associations with individual genetic variants need not be the same. However, because they are unique in being naturally randomized due to Mendel’s laws of segregation and independent assortment, individual genetic variants allow for more rigorous tests of factor model validity than measures from more aggregate levels of analysis allow for. Thus, the careful integration of genomic data provides a particularly rigorous and tractable empirical means of distinguishing between theoretical models with potentially differing implications for understanding risk and developing treatments and interventions to improve life outcomes.