Generative prediction of causal gene sets responsible for complex traits

Significance

Researchers have long sought to bridge the gap between phenotypes and the genotypes that cause them. This gap remains open because current methods focus on associating phenotypes to a combinatorially explosive number of genotypic possibilities, resulting in a loss of statistical power. We overcome this limitation by employing transcriptomic data from complex, polygenic, human diseases combined with measured transcriptomic responses to gene perturbations in cell lines. The former data allow us to perform generative modeling and dimensional reduction to map transcriptome to phenotype, while the latter incorporate causal information regarding how gene regulation shapes phenotype. We predict sets of genes that explain the emergence of complex traits, which suggest possible multitarget disease treatments.

Abstract

The relationship between genotype and phenotype remains an outstanding question for organism-level traits because these traits are generally complex. The challenge arises from complex traits being determined by a combination of multiple genes (or loci), which leads to an explosion of possible genotype–phenotype mappings. The primary techniques to resolve these mappings are genome/transcriptome-wide association studies, which are limited by their lack of causal inference and statistical power. Here, we develop an approach that combines transcriptional data endowed with causal information and a generative machine learning model designed to strengthen statistical power. Our implementation of the approach—dubbed transcriptome-wide conditional variational autoencoder (TWAVE)—includes a variational autoencoder trained on human transcriptional data, which is incorporated into an optimization framework. Given a trait phenotype, TWAVE generates expression profiles, which we dimensionally reduce by identifying independently varying generalized pathways (eigengenes). We then conduct constrained optimization to find causal gene sets that are the gene perturbations whose measured transcriptomic responses best explain trait phenotype differences. By considering several complex traits, we show that the approach identifies causal genes that cannot be detected by the primary existing techniques. Moreover, the approach identifies complex diseases caused by distinct sets of genes, meaning that the disease is polygenic and exhibits distinct subtypes driven by different genotype–phenotype mappings. We suggest that the approach will enable the design of tailored experiments to identify multigenic targets to address complex diseases.

Complex traits are polygenic, orchestrated by networks of interacting genes that work together to produce phenotypic variation (1–3). An outstanding question in the study of such traits is the identification of the specific combinations of gene variants that give rise to the different phenotypic expressions (4–10). Association studies have been performed to search for genetic loci significant to a complex trait phenotype by conducting hypothesis tests on individual genetic loci, from which independent mutations/genes are statistically associated with the phenotype in question (11–15). We innovate on these techniques by developing a framework to jointly predict sets of genes while accounting for collective behavior not captured by statistical tests on individual genes.

Association studies, such as genome/transcriptome-wide association studies (GWAS/TWAS), have been broadly adopted in over 5,700 studies and 3,300 traits as of 2021 (12). A common critique of these studies is that they have low statistical power due to the combinatorial explosion in the number of gene sets that must be tested (11, 13, 16). Post-GWAS/TWAS analyses such as fine-mapping attempt to address this limitation by considering the correlation structure of the genetic data (17–21). However, they rely on an initial association study to select what variants to fine-map, potentially leaving behind genes that would be significant collectively but have low independent effect size. In the framework presented here, we develop and apply an approach that considers all genes simultaneously, regardless of their individual effect size. The framework combines generative machine learning, dimensionality reduction, and constrained optimization (Fig. 1).

Fig. 1.

Schematic of the proposed approach. Synthetic transcriptomic profiles are generated from a machine learning model that learns from RNA-Seq data on complex traits. The generated data are projected onto eigengenes, which are linear combinations of genes that vary independently, retain important gene correlations, and differentiate between complex trait phenotypes. From here, gene perturbations whose transcriptional responses bridge the gap between trait phenotypes are found by constrained optimization.

A key aspect of our approach is the use of increasingly available trait-labeled transcriptomic data from bulk and single-cell RNA-Seq experiments, which contends with the biological networks that influence complex traits (22, 23). To better extract patterns from our transcriptional data, we develop the transcriptome-wide conditional variational autoencoder (TWAVE), a generative deep learning model that generates denoised transcriptional profiles for the relevant phenotypes. To understand how regulatory changes affect expression levels, we also incorporate complementary data on transcriptional response to gene perturbations (knockdowns and overexpressions) (24). These transcriptional data are dimensionally reduced while maintaining causal information, which we achieve using the concept of eigengenes, to facilitate the optimization over gene perturbations (25). Together, these data sources allow us to explore how the regulatory network drives phenotypic changes without prior knowledge of network structure.

We focus on the human disease traits in Table 1. Throughout, we take care to distinguish between traits (e.g., eye color) and their phenotype variants (e.g., blue, brown, green), and we consider traits that have a baseline and variant phenotype. Moreover, we interpret the states associated with each variant as defined by distinct attractors of the gene regulatory network (26–28). The problem of identifying the genes that cause a trait phenotype can thus be mapped to an optimization over combinations of transcriptional perturbations that steer transcriptomic states from baseline to variant attractors and vice-versa. The resulting framework reveals groups of gene perturbations that most influence phenotypic variation, pinpointing the molecular underpinnings that determine complex traits.

Table 1.

Gene Expression Omnibus and DepMap datasets

Complex trait	Tissue	Seq. type(s)	$N_{\mathrm{baseline}}$	$N_{\mathrm{variant}}$	GEO series
Allergic asthma	Peripheral blood mononuclear cells	Bulk	277	166	GSE96783
Inflammatory bowel disease	Gastrointestinal tissue	Bulk	461	2029	GSE193677
Food allergy	CD4+ T cells	Bulk	71	63	GSE114065
Cancer metastasis	Metastasis: Breast $\to$ lung	Single cell	1170	1274	GSE202695
Macular degeneration	Macular retina and retinal pigment epithelium	Bulk & single cell	433	104	GSE135092
Type 1 diabetes	CD4+ T cells	Single cell	557	2502	GSE182870
Non–small cell lung cancer	Blood platelets	Bulk	376	400	GSE89843
Simple trait	Tissue	Seq. type	$N_{\mathrm{baseline}}$	$N_{\mathrm{variant}}$	GEO series
MODY3	Differentiated embryonic stem cells	Single cell	113	158	GSE129653
Complex trait	Tissues	Seq. type	$N_{\mathrm{baseline}}$	$N_{\mathrm{variant}}$	DepMap series
Pancancer metastasis	86 primary cancers, 34 lineages	Single cell	838	447	24Q2

The columns represent (from Left to Right) the traits considered, the tissues of origin, the type of sequencing, the number of samples of the baseline and variant phenotypes, and the GEO or DepMap series accession numbers.

Results

Generating Complex Trait Transcriptomes.

Identifying sets of differentially expressed genes is complicated by the fact that transcriptional measurements include a large number of genes and a comparatively small number of samples. It is precisely this feature that makes it challenging to distinguish real biological differences from random variance when using statistical tests that treat genes as independent variables. Our method recognizes that genes operate in concert rather than independently to orchestrate cell function, which transforms the problem into learning an effective representation of these relationships from data. Fig. 2 presents TWAVE, which solves this problem by looking at the data as a whole, using a neural network encoder to embed high-dimensional gene expression profiles onto a low-dimensional latent space ($Z$), where data points can be classified and new representative points can be generated. Points in the latent space, including newly generated ones, are decoded back up to the full gene expression space as in Fig. 2A. The model (consisting of the encoder, decoder, and latent space classifier) is trained with a combination of three loss terms. The first term accounts for how accurately the autoencoder can reconstruct the original data. The second term is the Kullback–Leibler (KL) divergence loss, which regularizes $Z$ so that each dimension contributes roughly equally to the overall variance in the latent space. The third term is a classification loss to mold the structure of the latent space so that different phenotypes of the trait (baseline and variant for example) can be distinguished by their transcriptional states, as detailed in Materials and Methods.

Fig. 2.

TWAVE construction and validation, presented for the inflammatory bowel disease trait. (A) TWAVE architecture, where gene expression profiles are projected onto a low-dimensional latent space ($Z$) and subsequently reconstructed with a neural network decoder. (B) First two principal components (PC) of the latent space $Z$, showing a clear separation of complex trait phenotypes (baseline and variant). A linear interpolation between the means of the two populations in the latent space (blue-to-red stars) falls along the first principal component. (C) Comparison between original (blue) and TWAVE-reconstructed (red) distributions of gene expression for four different genes, conveying strong agreement. (D) Receiver operating characteristic for significant gene associations in the reconstructed versus original data, confirming that TWAVE retains significant associations found in the original data.

Fig. 2B shows that transcriptional measurement associated with the baseline and variant populations (blue and red clusters) segregate to coordinates along the first principal component, which is a natural outcome of the latent space learned by TWAVE. Here, we use the representative case of inflammatory bowel disease to illustrate the construction of TWAVE, but we observe similar performance among the other traits we considered in Table 1. The figure also shows that a linear interpolation between the two clusters in $Z$-space lies along the first principal component, which accounts for the largest fraction of variation in the data. Fig. 2C demonstrates a close agreement between the original gene expression distributions and the reconstructions from TWAVE. To test whether our autoencoder retains associations between genes and the complex trait of interest, we compare the differentially expressed genes identified by $t$ tests on both the original and reconstructed expression profiles using the area under the receiver operating characteristic curve (AUROC). Fig. 2D shows this curve, which is constructed by 1) arranging both sets of genes from smallest to largest in terms of their $p$-values, 2) varying the threshold at which genes are statistically significant, and 3) counting the fraction of significant genes in the original data that are selected by TWAVE (true positive rate) as a function of the fraction of genes selected by TWAVE that are not significant in the original data (false positive rate). For all traits, we find that the AUROC approaches 1, indicating that the sets of differentially expressed genes identified by TWAVE and within the original data are nearly identical. Full technical details concerning the construction of TWAVE are provided in Materials and Methods.

For each of the complex trait datasets described in Table 1, we use TWAVE to estimate the distributions of the transcriptional data in the latent space arising from the baseline and variant phenotypes, while retaining the fundamental features that distinguish the two populations. We then draw points from these distributions in the latent space, decoding them as depicted in Fig. 2A. By choosing an equal number of each phenotype (baseline and variant), estimates of the distributions from the data are equally precise for each of these trait phenotypes. It is instructive to compare against the method of “extreme pseudosampling,” in which the latent space of a variational autoencoder (VAE) is sampled randomly (29, 30). Our method is different from extreme pseudosampling in two crucial ways. First, TWAVE employs a conditional VAE (i.e., it includes a latent space explicitly trained to classify between baseline and variant). Second, TWAVE draws from a probability distribution in the latent space associated with the trait phenotype label instead of drawing randomly from any state in the latent space. Overall, TWAVE allows us to make the most from limited transcriptional data, drawing new representative samples in a way that would be unfeasible without generative modeling.

Causal Dimensions of Complex Trait Variation.

We select the eigengenes that are most determinative of the trait phenotypes (i.e., are causal). Conceptually, eigengenes are weighted combinations of genes that vary in concert within the eigengene, but any given eigengene can vary independently of the others. Mathematically, they are eigenvectors of the TWAVE-generated gene expression matrix $Y$ and form an orthogonal basis ${\mathit{e}}_i$ in gene expression space (25). This basis corresponds to the columns of the unitary matrix $U=[{\mathit{e}}_1,...,{\mathit{e}}_i,...,{\mathit{e}}_l]$ in the singular value decomposition $Y^T=U\mathrm{\Sigma }{V}^T$, where $\mathrm{\Sigma }$ is a diagonal matrix of singular values in descending order and $V^T$ contains the left eigenvectors of $Y^T$. For each dataset, we perform singular value decompositions of the $m\times n$ matrix $Y$, where $m$ is the number of sample expression profiles in the TWAVE dataset, $n$ is the number of genes in each sample, and $l=min(m,n)$ is the rank of $Y$.

We proceed to determine which eigengenes are most likely to capture differences between the baseline and variant trait phenotype by adapting Bayesian fine-mapping (17) to eigengenes. This procedure seeks a small set of $r$ eigengenes that can accurately distinguish between the phenotypes according to the posterior inclusion probability, which quantifies how well a proposed set of eigengenes explains the data (i.e., how causal the set is). The fine-mapping procedure starts by projecting the data onto the $d=200$ eigengenes with the largest singular values $X=Y{U}_{n\times d}$. This choice of $d$ ensures that the set of eigengenes from which the causal set is selected accounts for the large majority of the variance, as shown in SI Appendix, Fig. S1. A logistic regression model over eigengenes is then fit using the expression profiles and associated trait phenotype labels, achieving high accuracy, F1-score, and recall, for all datasets analyzed. We also show that a maximum likelihood estimator for the posterior distribution of causal eigengenes can be formed from: 1) the odds ratios $\zeta =log[\rho /(1-\rho )]$ from the logistic regression, where $\rho$ is the probability of a data point belonging to the variant class; and 2) the projected expression matrix $X$. To reduce from $d$ to $r$ eigengenes, we perform Markov chain Monte Carlo (MCMC) sampling to maximize the likelihood of causal eigengenes given the regression data. For each of these $d$ eigengenes, we evaluate the posterior inclusion probability that each eigengene is causal $p({\mathit{e}}_i\mathrm{causal}|\mathrm{data}=\{X,\zeta \})$ and build sets from the top $r=50$ causal eigengenes ($r=10$ for allergic asthma) with the largest posterior inclusion probability. We show $p({\mathit{e}}_i|X,\zeta )$ for the first $d$ principal components of $Y$ in Fig. 3A, ordered from noncausal ($p=0$) to causal ($p=1$).

Fig. 3.

Dimension reduction by selecting the most causal eigengenes for the inflammatory bowel disease trait. (A) Posterior inclusion probability for eigengenes to be causal arranged in descending order of their singular values. (B) Classification accuracy from logistic regression on data projected onto an increasing subset of the top $d$ eigengenes arranged in descending order of the posterior probability of being causal (blue), in descending order of the singular values (gray), and randomly (red).

As a validation test for the selected eigengenes, we perform the logistic regressions on $X$ summarized in Fig. 3B. The regression accuracy for including data projected onto the first $i$ eigengenes sorted in order of causality $p({\mathit{e}}_i|X,\zeta )$ quickly climbs to above $0.9$ within $r$ top eigenvectors. This validates our choice of keeping $r$ eigengenes in our dimensionality reduction. On the other hand, including eigengenes in principal component order (i.e., ordered by the fraction of variance that aligns along each eigenvector), yields significantly poorer classification results. Arranging the eigengenes randomly can actually produce a better result than arranging them in order of the singular values when keeping less than $60$ eigengenes, but principal component ordering outperforms the random one as the dimension of this reduced space is increased.

Complex Trait Transitions via Eigengene Perturbations.

Constrained optimization.

To implicate genes responsible for transitioning between the baseline and variant phenotypes, we explore extensive data on transcriptional responses to gene perturbations. Specifically, we define a perturbation response matrix, $B$, whose rows represent eigengenes in the original dataset and whose columns are average transcriptional responses to a transcriptional perturbation (one response profile for each column). This matrix consists of 10% overexpressions and 90% knockdowns (most of the latter are implemented through RNA interference), as indicated in Dataset S1. With this matrix in hand, we investigate which combinations of perturbations can cause the baseline transcriptional profile to match the variant, and vice-versa. Formally, this question is answered by solving the following constrained optimization problem:

$${\mathit{u}}^{\ast }=\underset{\mathit{u}|0\le u_{\alpha }\le 1}{\mathrm{argmin}}\{D(\mathit{u})+\lambda \sum _{\alpha }{u}_{\alpha }\},$$ [1]

where

$$D(\mathit{u})=||{\mathit{x}}_{\mathrm{variant}}-{\mathit{x}}_{\mathrm{baseline}}-B\mathit{u}||,$$ [2]

and $||·||$ denotes the Euclidean distance. The choice of Euclidean distance reflects our assumptions 1) that there is a single phenotype for each transcriptional state and 2) that differences in the expression of each eigengene are equally likely to contribute to phenotypic differences. We discuss alternative choices of the distance metric in SI Appendix. Here, perturbations add with weight $\mathit{u}$ in causal eigengene space to transition from the state ${\mathit{x}}_{\mathrm{baseline}}$ to the state ${\mathit{x}}_{\mathrm{variant}}$ (Fig. 4A). Before considering transitions between individual states in the baseline and variant clusters, we consider transitions between average baseline and variant states.

Fig. 4.

Attributing complex trait phenotypes to sets of genes. (A) Identification of gene sets that differ between baseline and variant trait phenotypes, where blue and red dots represent individual states. A state in the baseline cluster (blue background) transitions to a state in the variant cluster (red background) upon targeted transcriptional perturbations. (B) Coefficient of determination for controlled transitions between phenotypes of complex traits, where $R^2$ close to 1 indicates that the final transcriptional state approaches that of the target phenotype.

Since we expect the baseline and variant phenotypes to be stable with respect to the fluctuations inherent to transcription, the closer we approach states known to belong to a given phenotype transcriptionally, the more likely that state is to exhibit that phenotype. We quantify this likelihood with the coefficient of determination

$$R^2=1-D({\mathit{u}}^{\ast })/D(\mathbf{0}),$$ [3]

where $R^2\le 1$ and $R^2$ close to $1$ indicates high efficacy. In particular, $R^2>0.5$ indicates that the distance between the two states has been at least halved. The relationship between distance and cell behavior becomes less precise in the full high-dimensional expression space because there are many points that are a given distance away from a target point. Thus, reducing the number of dimensions of our problem by working with a select set of eigengenes is crucial, which is implemented through our choice to express matrix $B$ in the space of eigengenes.

We use the Python function minimize from SciPy (which implements the L-BFGS-B method) to solve the constrained optimization problem in Eq. 1 for all of our RNA-Seq datasets. The optimal ${\mathit{u}}^{\ast }$ yields $R^2\approx 1$ for all complex traits we consider, namely allergic asthma, inflammatory bowel disease, food allergy, cancer metastasis (where baseline and variant refer to primary and metastatic tumors), age-related macular degeneration, type 1 diabetes, and non–small cell lung cancer (Fig. 4B). This is further confirmed by examining the selected perturbations for individual trait phenotypes and noting that they generally point toward the phenotype expression in question.

Table 2 shows the genes with the top 12 perturbation weights in ${\mathit{u}}^{\ast }$ for allergic asthma along with a brief annotation of their function. Many of these top-selected genes have been implicated in allergic asthma, lung and airway function, and inflammation and immunity. Mutations in BMPR2, for example, have been shown to cause asthma-like symptoms and pulmonary hypertension in response to mild antigens in the airway (34, 35). In addition, TCF7 promotes T-cell differentiation to Th2 or memory T cells (36), consistent with allergic asthma being the result of immune system dysregulation. Other identified genes, such as TARDBP, TENT4B, and HNRNPL, have not been previously implicated in allergic asthma. However, these genes are associated with RNA metabolism and modifications including alternative splicing and poly-A tail alteration, which in turn are related to immune response (46). In particular, TARDBP (TDP-43) has been shown to regulate alternative splicing and alternative polyadenylation in CD8+ T cells, and specific RNA splicing and polyadenylation events depend on the presence of TARDBP during CD8+ T-cell costimulation (31). TENT4B is involved in mRNA stabilization, influencing B-cell proliferation and the cellular response to viral infections (32), whereas HNRNPL participates in the regulation of inflammatory responses, particularly through its interaction with long noncoding RNAs and its role in regulating TNF-$\alpha$ transcription (44, 45). We performed the optimization between the average baseline and variant states for all 6 other complex traits shown in Fig. 4B, and refer the reader to SI Appendix, Tables S1–S6 for the top selected genes for each trait.

Table 2.

Allergic asthma transcriptional perturbations (GSE96783)

Gene	Annotation
TARDBP−	RNA metabolism and regulation (31)
TENT4B−	Posttranscriptional modifications (32)
KRR1−	Ribosome biogenesis (33)
BMPR2−	Inflammatory signaling (34, 35)
TCF7+	Growth/migration in airway (36)
APOBEC3G+	Innate immunity, antiviral (37)
INTS12−	Lung function via protein synthesis pathways (38)
NEAT1−	Inflammation in asthma (39)
MTHFD1−	Folate and methionine metabolism (40, 41)
PRMT5−	Allergic airway inflammation (42)
FASTKD1−	Mitochondrial function, apoptosis (43)
HNRNPL−	RNA splicing and expression regulation (44, 45)

Optimization for individual baseline–variant states.

We perform optimizations across individual baseline and variant pairs to account for the fact that the measured transcriptional signatures of a given phenotype vary heterogeneously across cell samples and individuals (Fig. 4A). An analogous approach in TWAS would require subsampling an already small number of measurements, leading to a large uncertainty in the variance and an inability to detect baseline–variant differences (47). By recasting the hypothesis test as an optimization problem, we avoid this issue using information on how the regulatory network responds to perturbations. This reformulation allows us to investigate how the gene perturbations responsible for a trait may change across individual baseline–variant pairs. We break down our strategy into two steps: 1) find the optimal set of perturbations for a large number of baseline–variant state pairs and 2) compare the observed co-occurrence of perturbation pairs with a null model. The null model is designed to preserve both the frequency with which each perturbation is selected and the number of perturbations needed for each pair of states.

The transition between any ${\mathit{x}}_i$ and ${\mathit{x}}_j$ can be induced by applying the perturbation

$${\mathit{u}}_{\mathit{ij}}=\underset{\mathit{u}|0\le u_{\alpha }\le 1}{\mathrm{argmin}}\{||{\mathit{x}}_i-{\mathit{x}}_j-B\mathit{u}||+\lambda \sum _{\alpha }{u}_{\alpha }\},$$ [4]

where the variant state is ${\mathit{x}}_i$ and the baseline state is ${\mathit{x}}_j$. We solve this optimization problem over a range of different $\lambda$ values, taking the largest $\lambda$ (the sparsest solution) such that $R^2=1-D({\mathit{u}}_{\mathit{ij}})/D(\mathbf{0})>0.99$. This is repeated for $N=2,500$ randomly selected pairs in the forward (baseline-to-variant) and reverse directions. We then construct a bipartite network represented by the (adjacency) matrix

$$A={[...,{\mathit{u}}_{\mathit{ij}},...]}^T,$$ [5]

where the columns of $A$ are perturbed genes and the rows are different accepted $i,j$ pairs. We take the dot-product between the columns $\mu$ and $\nu$ of $A$ to get the frequency $f_{\mu \nu }$ at which the corresponding perturbed genes co-occur in the same ${\mathit{u}}_{\mathit{ij}}$:

$$f_{\mu \nu }=\frac{1}{N}{\displaystyle \sum _{\alpha =1}^N}{A}_{\alpha \mu }{A}_{\alpha \nu }.$$ [6]

To identify statistically significant pairs $\mu$, $\nu$, we compare $f_{\mu \nu }$ to the frequency at which $\mu$ and $\nu$ occur together in a null model of a random maximum entropy graph whose row and column sums are fixed to that of $A$ (48, 49). The expected co-occurrence frequency in the null model is

$$⟨f_{\mu \nu }⟩=\frac{1}{N}{\displaystyle \sum _{\alpha =1}^N}{p}_{\alpha \mu }{p}_{\alpha \nu },$$ [7]

where $p_{\mathit{ij}}$ is the probability of an edge existing between nodes $i$ and $j$ of the maximum entropy graph (Materials and Methods). In addition to $⟨f_{\mu \nu }⟩$, the SE ${\sigma }_{\mu \nu }$ can be approximated via error propagation using the fact that the probability of edge occurrences are Bernoulli random variables. From these ensemble statistics a $z$-value can be constructed as $z_{\mu \nu }=(f_{\mu \nu }-⟨f_{\mu \nu }⟩)/{\sigma }_{\mu \nu }$.

Now that we have established a null model for the co-occurrence of perturbations in causing/reversing the variant behavior, we can examine the network formed by the statistically significant co-occurrences that deviate from the maximum entropy model. To determine where co-occurrences begin to deviate from the null model, we identify the set of significant pairs with high $z_{\mu \nu }$ above a threshold defined by inspection of the quantile–quantile plot (SI Appendix, Fig. S2).

Application to Allergic Asthma.

In the case of allergic asthma, pairs with $z_{\mu \nu }>20$ were kept for analysis. The corresponding network is depicted in Fig. 5A for the baseline-to-variant transition (i.e., the onset of asthma). Each node represents a perturbed gene within a significantly co-occurring pair and is color-coded according to the nature of the perturbation (knockdown or overexpression). Edges represent significant co-occurrences and are color-coded by the expression correlations across responses between the genes they connect. We find that genes with many connections in this network representation, such as ADAR, PAN3, and MAPK1 have been implicated in allergic asthma before (50–52). We also optimized over gene perturbations in the reverse direction, as shown in Fig. 5B. Among the genes featured in this network, we again find several linked to allergic inflammation: SUZ12 inhibition is associated with the reduction of allergic inflammation through is role in the protein complex PRC2, JAK2 inhibitors have been proposed to alleviate asthma because of JAK2’s role in the JAK-STAT signaling pathway, and knockdown of MYC has been shown to repress ILC2 (type 2 innate lymphoid cell, a type of immune cell) activity, which in turn reduced airway inflammation and immune hyperresponsiveness (53–55).

Fig. 5.

Genes perturbed in transitions between baseline and variant clusters for allergic asthma. (A and B) Gene perturbation co-occurrence networks for forward (A) and reverse (B) transitions. Edges appear between a pair of perturbed genes if they frequently co-occur in successful transitions (i.e., they both occur with high frequency in ${\mathit{u}}_{\mathit{ij}}$ compared to a maximum entropy graph null model). The edges are colored by gene–gene correlation in the perturbation response dataset. The nodes are sized proportionally to the number of edges and color-coded according to whether the perturbation is a knockdown (blue) or overexpression (red). (C) Histograms of the number of gene perturbations required to induce the forward transition (red) and the reverse transition (blue).

Note that many of the genes in the forward and reverse co-occurrence graphs in Fig. 5 A and B are distinct. In a dynamical network, reversing a perturbation does not generally restore the state of the system, a phenomenon that is accounted for by the bounds placed on $\mathit{u}$ in Eqs. 1 and 4. These bounds may prevent the same genes from being selected in the forward and reverse directions, which within our approximation reflects the fact that the responses to knockdowns and overexpressions are not exactly antialigned. In our case, this phenomenon gives rise to the observation that the genes causing a given trait phenotype are not necessarily the ones that mitigate it. Moreover, as shown in Fig. 5C, the number of perturbations required to make the transition in the forward and reverse directions are also different. Remarkably, it takes a combination of fewer single-gene perturbations to induce a transition in the reverse direction (i.e., from the variant to the baseline state) than in the forward one. This may be because it takes more perturbations to go to a particular variant state than to a generic baseline state (“all roads lead to Rome” but not necessarily the inverse). A second possibility is that there is an overrepresentation of certain genes in the library of gene perturbations (Materials and Methods).

Importantly, both genes involved in the forward direction and genes involved in the reverse direction can be associated with the trait because they can promote or reverse a change from the baseline to variant phenotype. For example, the involvement of MYC in the reverse direction may be due to its known function in promoting plasticity between transcriptional states and amplifying gene expression overall when overexpressed (56, 57). Moreover, pluripotent stem cells with MYC knocked-down have been shown to decrease allergic reactions in mice by inhibiting T-helper cell immune reaction (58). Aberrant translation of the CEBPA gene, which is implicated in the forward direction, has also been associated with causing bronchial smooth muscle cells—a tissue that plays a key role in asthma—to proliferate faster (59). Notably, there are genes featured in both the forward and reverse co-occurrence networks that play roles in both promoting and attenuating allergic asthma. For example, FOXO1 overexpression in mice has been shown to promote allergic asthma through macrophage polarization, Th9 (T-helper 9 cell) differentiation, and regulation of IRF4 expression, though inhibition of FOXO1 led to attenuation of immune response and asthmatic inflammation through regulation of IRF4 (60, 61). Likewise, the role of JUNB depends on the state of other transcription factors. Though JUNB significantly influences Th2 (T-helper 2 cell) differentiation and the production of Th2 cytokines, promoting allergic inflammation, it also plays a role in maintaining homeostasis. Specifically, knockdown of JUNB limits excessive inflammation by modulating regulatory T-cell differentiation (62).

We also find that, except for BMPR2 and TENT4B, the genes appearing in the co-occurrence graphs from the optimization between individual baseline–variant states are distinct from those identified by the average variant-baseline optimization (Table 2). The apparent contrast between optimizing over average transcriptional states and individual pairs highlights the fact that there can be multiple paths (defined by different perturbation sets) through which the disease progresses and is mediated and that these paths are not necessarily the ones connecting the average transcriptional states. Consequently, the distinct co-occurring genes in Fig. 5 could potentially relate to different mechanisms playing a role in allergic asthma. We emphasize that such a co-occurrence structure cannot be inferred from studying population averages alone, as typically done in GWAS/TWAS (47).

It is instructive to consider the transcription factors that regulate the genes in our perturbation response library (upstream transcription factors). Not all upstream transcription factors have transcriptional responses measured in our library, so we use the Enrichr gene set enrichment database to find them. We focus on the case where the upstream factors simultaneously regulate both genes in a co-occurring pair so that a single transcription factor could explain their combined influence on the trait phenotype. For instance, GATA2, TET2, and TWIST1 are enriched for more than one gene co-occurrence pair and are known to influence allergic asthma (63–65). The most parsimonious explanation for the enrichment of these transcription factors is that they exert their influence on the phenotype (at least in part) through the genes that appear in our perturbation response dataset. The enrichment of these additional factors shows that we may be able to infer trait-associated genes outside our dataset.

Learning Across Different Contexts with TWAVE.

Thus far, we have considered 7 complex traits, each based on data from a unique tissue type and previously examined by differential expression. Next, we generalize to contexts where 1) the data (and trait phenotype) in question are associated with multiple disparate tissues and 2) the trait phenotype is caused by a mutation that affects the function of the protein instead of its transcriptional expression.

We consider a phenotype that manifests itself through multiple tissues by studying the trait of cancer metastasis in the cancer dependency map (DepMap) dataset (66). Because these pancancer data come from many different cell types, there are confounding variables that render a simple differential expression analysis unable to detect any differentially expressed genes common to the process of metastasis across all tissues. Indeed, we found no statistically significant associations by performing such an analysis. However, using TWAVE, we are able to disentangle the effects of the confounding variables associated with different disease contexts (e.g., cell type, tissue origin, tumor location, and systematic effects) to uncover common biological mechanisms driving cancer metastasis (SI Appendix, Fig. S3). In this case, we again find that our co-occurring perturbation networks contain many genes previously found to promote or mitigate cancer metastasis, including NF1 knockdown, SOX5 overexpression, CBFB overexpression, TOX4 knockdown, PROX1 overexpression, and EHF knockdown (67–72). An Enrichr search for the co-occurring genes reveals out-of-sample upstream transcription factors that are known to affect metastasis as well, such as STAT3 and CTCF (73, 74). Although both of these genes appear to be essential for growth (75, 76), which limits opportunities to perturb them in cell-line experiments, one can detect their influence on trait phenotype variation through the pairs of genes in the co-occurrence graph that they regulate.

To examine the scenario in which a causal mutation affects a gene’s protein function but not its transcriptional expression, we consider maturity-onset diabetes of the young type 3 (MODY3). MODY3 is known to be a largely monogenic trait caused by mutations to the transcription factor HNF1A that impact beta cell function and diabetes in general. Since HNF1A is one of the genes perturbed in our perturbation response matrix $B$, MODY3 provides an excellent example where the solution is known. Documented mutations of HNF1A alter its protein function, which in turn alters the expression of other genes as opposed to its own. In fact, HNF1A overexpression appears in 30.2% of baseline–variant pair optimizations. This is compared to the top overall perturbation, NEAT1 knockdown, appearing 57.4% of the time. However, HNF1A, as opposed to NEAT1, also appears in the forward co-occurrence network (SI Appendix, Fig. S4). Of the top 13 gene perturbations, three of them—MED1 knockdown, HNF1A overexpression, and GATA2 overexpression—were also in the forward co-occurrence network and have been implicated in diabetic function.

This narrows down the large list of possible genes to a number that could be tested in low-throughput lab experiments. For instance, MED1 knockout mice show a heightened sensitivity to insulin and an improved glucose tolerance (77). All of the other genes in the co-occurrence network exhibit transcriptional responses that are highly positively correlated with that of HNF1A, meaning that their corresponding column vectors in $B$ all point in the same direction. This pattern is markedly different from those observed in the complex traits above, in which we also find transcriptional responses that are negatively correlated and uncorrelated. Among the perturbations correlated with HNF1A overexpression is ALOX5 overexpression, which also impacts beta cell function in diabetes via increased insulin resistance (78, 79).

Finally, we directly compare the genes identified by our method, differential expression, and TWAS in the case of inflammatory bowel disease in SI Appendix, Fig. S5. We find that the only 8% of the differentially expressed genes identified in the dataset (80) overlap with the TWAS genes, which is reflective of the challenges inherent to reconciling results produced by different approaches. Applying our method, we find that 36% of the genes participating in over 54% of the solutions to Eq. 4 overlap with TWAS. We emphasize that this improvement by our method occurs because it accounts for the downstream impacts of the gene perturbations through the $B$ matrix, which naturally filters out spurious differentially expressed genes.

Discussion

The approach presented here leverages existing transcriptomic data to address the challenges that complex traits pose to traditional mutation-association screening methods (13). We implement this by identifying generalized cellular pathways (eigengenes) relevant to a complex trait and by calculating optimal sets of gene perturbations whose transcriptional responses change the combined state of these pathways from one phenotype to another. Our approach accounts for limited data, heterogeneity within phenotypes, confounding biological variation, and combinatorial explosion in gene sets in ways that traditional methods cannot (13). In particular, limited data are addressed by our development of the generative model TWAVE; inference of heterogeneous pathways is illustrated in the example of allergic asthma; common drivers of cancer progression across biological subtypes are found in the DepMap example; and finally, a combinatorial explosion is avoided by casting the identification of causal genes as an optimization problem. The approach can also implicate candidate genes through their known downstream effects on the gene regulatory network obtained from experiments.

An overarching goal of our approach is to narrow the scope of candidate gene combinations to a number amenable to targeted low-throughput experiments. As in previous successful applications of Boolean networks (81–86) and principal component-based techniques that uncover low-dimensional structure in gene regulatory networks (87), we aim to uncover causal influences. The main advantage of our approach is that it can generate predictions solely from publicly available data without explicit network reconstruction or specific knowledge concerning the gene functions and interactions, making it broadly applicable.

It is constructive to reflect on the key assumptions underlying our approach. First, we assume that cellular traits are well reflected by gene expression, which is validated by the fact that we and others (24, 88–92) can accurately classify gene expression profiles by their phenotypic labels. While transcriptional data do not directly account for posttranscriptional/translational regulation (93), they do account for downstream impacts on gene expression. Nevertheless, it is straightforward to incorporate multiomic (94–97) data to directly account for mechanisms beyond transcription. Second, we assume that transcriptional responses combine additively, which has been demonstrated to be a good approximation to control cell behavior (24). Recent work has applied VAEs to the forward problem of estimating nonadditive transcriptional responses to combinatorial perturbations (98), raising the possibility of going beyond the additive assumption in the future. Integration of this technique into our method to solve the inverse problem of mapping causes to trait phenotypes, as considered here, would require targeted experiments to train the VAE to recognize nonadditivity. Finally, we assume that our library of transcriptional responses is sufficiently large and diverse to comprehensively capture the impact of genes. Notwithstanding, we demonstrate that enrichment analysis (99, 100) can implicate upstream transcription factors that are not included in our library.

The success of our approach has several far-reaching implications. First, it suggests that cell line perturbations in vitro are informative of the gene behavior in situ (101). Second, it shows that the genes needed to drive a phenotypic change can be distinct from those that reverse the change, a hallmark of complex systems with nonlinear dynamics. This is a consequence of our optimization model acknowledging the fundamentally different network impacts of reversing a knockdown versus overexpressing a given gene, which is consistent with persistent responses to transient perturbations observed in gene regulatory networks (102–104). Third, the success of TWAVE suggests that gene expression can be represented in a low-dimensional space (87), which might be a general feature across many complex network systems (105). Ultimately, our approach provides a tool to investigate genotype–phenotype relationships in complex traits, which is applicable across a range of organisms and traits. In humans, our approach also lays the groundwork for the design of next-generation multitarget strategies for the treatment of complex diseases.

Materials and Methods

TWAVE Architecture.

We employ a conditional variational autoencoder in PyTorch, which is tailored for the analysis of genomic data and leverages class labels (baseline or variant) to impart enhanced interpretability and classification precision. The architectural blueprint consists of three neural networks: an encoder, a decoder, and a classifier.

The encoder comprises two fully connected hidden layers, each embedding 256 and 128 units, respectively, with Rectified Linear Unit (ReLU) activation functions. The input layer takes a single gene expression profile, length aligned with the number of genes, and is concatenated with pertinent class labels. This design not only captures the intricate gene expression patterns but also incorporates class-specific information for a more nuanced latent representation. The decoder component reconstructs the input RNA-Seq data through a series of ReLU-activated layers, culminating in a sigmoid activation function. This reconstruction process aims for the faithful reconstruction of the input profile from the encoded latent space. Simultaneously, the classifier, featuring a linear layer, facilitates class predictions grounded in the extracted latent representation.

TWAVE Training and Sampling.

During the training phase, a set of loss functions drives the optimization process. The combination of reconstruction loss and Kullback–Leibler divergence loss is deployed, ensuring a balance between accurate data reproduction and the regularization of the latent space. The training regimen spans $500-{10}^4$ epochs, depending on the dataset, with minibatches consisting of 50 to 200 samples. We use an Adam optimizer, with a learning rate set at 0.0001.

For sampling, we generate synthetic profiles within the latent space of TWAVE. Latent vectors are obtained by sampling from clusters corresponding to distinct class labels. The latent vectors corresponding to different classes are then clustered, and marginal distributions of baseline and variant profiles in the latent space are extracted using kernel density estimation (KDE). The bandwidth parameter for our KDE is set to 0.2 to control the smoothness of the estimated density. We then sample new latent space points from these two clusters using our density estimator, and these points are decoded to the full gene expression space with the decoder, producing synthetic profiles for both class labels.

Bayesian Inference of Causal Eigengenes.

We perform Bayesian inference of causal eigengenes from the gene expression data projected onto the top $d=200$ eigengenes, $X=Y{U}_{n\times d}$, where we observe that these eigengenes account for over the overwhelming majority of the variance in all traits. In SI Appendix, Fig. S1, we show that the remaining percent variance after $d=200$ eigengenes is less than 1% for all traits except (lung) cancer metastasis and type 1 diabetes, where the remaining eigengenes account for about 22% and 23.5% of the total variance, respectively. Our choice $d$ trims the number of eigengenes to a tractable one that allows for relatively fast Monte-Carlo optimization.

Our inference procedure adapts the fine-mapping of causal variants in GWAS (17) to eigengenes. The first step of the fine-mapping is a logistic regression using class labels. We fit the log-odds ratios $\zeta$ to the data $X$ with effect sizes $\beta$:

$$\zeta =log\frac{\rho }{1-\rho }=X\beta +ϵ,ϵ\sim \mathcal{N}(0,{\sigma }^2),$$ [8]

where $ϵ$ is a noise vector, the vector $\rho$ denotes the probabilities that each (binary) label is 1, and the $log$ function is taken element-wise. In this scheme, we seek to infer an optimal $d$-dimensional vector of causal effects $\gamma =({\gamma }_i)$, ${\gamma }_i\in \{0,1\}$, which take on values of zero or one depending on whether eigengene $i$ is causal or not.

It can be shown that the likelihood of $\zeta ,X$ given $\gamma$ is

$$p(\zeta ,X|\gamma )=\mathcal{N}(\xi |0,R+R{\mathrm{\Sigma }}_{\gamma }R),$$ [9]

where $\xi =X^T\zeta /d$ is the $z$-value, $R=X^{T}X/\sqrt{d{\sigma }^2}$ is the eigengene correlation matrix, ${\mathrm{\Sigma }}_{\gamma }=d{s}^2\mathrm{diag}(\gamma )$, and $s$ is a hyperparameter. We set an initial $s=0.05$ as in FINEMAP (17). Taking a uniform prior on the number of causal effects $k$,

$$q_k={(1/d)}^k{(1-1/d)}^{d-k},$$ [10]

we can then express the posterior distribution of causal effects given our data as

$$p(\gamma |\zeta ,X)=q_{k}p(\zeta ,X|\gamma ).$$ [11]

We use MCMC to optimize this distribution over $s$ and $\gamma$, though other techniques such as FINEMAP or the sum of single effects (SuSiE) model could be employed as well (19). Posterior inclusion probabilities are calculated as an average over MCMC samples of the posterior distribution

$${\mathrm{PIP}}_i=p({\gamma }_i|\zeta ,X)=\frac{1}{N}{\displaystyle \sum _{j=1}^N}{\gamma }_i^{(j)},$$ [12]

where $N$ is the number of samples. The Monte Carlo steps consist of flipping a causal effect (eigengene) on or off at random and we attempt ${10}^5$ steps with a burn-in period of ${10}^3$ steps, according to a Metropolis acceptance criterion.

Maximum Entropy Null Graph Model.

As a null model for our gene concurrence graph, we construct a maximum entropy graph constrained by the row and column sums of our matrix $A$. The null model $G$ maximizes the entropy $S(G)=-\sum _{G}P(G)lnP(G)$, where $P(G)$ is the canonical distribution $P(G)\propto exp[-\sum _i{\beta }_i{k}_i-\sum _i{\gamma }_i{\kappa }_i]$. Here, $k_i=\sum _j{A}_{\mathit{ij}}$ is the row sum and ${\kappa }_i=\sum _j{A}_{\mathit{ji}}$ is the column sum of node $i$, whereas ${\beta }_i$ and ${\gamma }_i$ are the respective Lagrange multipliers that enforce $k_i$ and ${\kappa }_i$ to be fixed as all other degrees of freedom equilibrate. The row and column sums sequences must follow the maximum entropy conditions

$$k_i=\sum _{j\ne i}\frac{e^{{\beta }_i+{\gamma }_j}}{1+e^{{\beta }_i+{\gamma }_j}},{\kappa }_j=\sum _{i\ne j}\frac{e^{{\beta }_j+{\gamma }_i}}{1+e^{{\beta }_j+{\gamma }_i}}.$$ [13]

These conditions can be solved for the Lagrange multipliers iteratively as

$${\beta }_i^{(\ell +1)}=log{k}_i-log\sum _{j\ne i}r({\beta }_i^{(\ell )},{\gamma }_j^{(\ell )})$$ [14]

and

$${\gamma }_j^{(\ell +1)}=log{\kappa }_j-log\sum _{i\ne j}r({\beta }_j^{(\ell )},{\gamma }_i^{(\ell )}),$$ [15]

where $\ell$ denotes the iteration and $r(x,y)=1/(e^{-y}+e^x)$. From here, the link probabilities in Eq. 7 can be computed as

$$p_{\mathit{ij}}=\frac{1}{1+e^{-({\beta }_i+{\gamma }_j)}}.$$ [16]

Complex Disease Data Curation.

We obtained our seven datasets from GEO (Table 1). The expression matrices, originally in raw counts, are curated keeping those genes and samples that meet the following criterion: average counts in a gene $>$5 and total counts in a sample $>$10⁵. The data were normalized to the number of transcripts per million ($N_{\text{TPM}}$) using reference transcript lengths mapped from the ENSEMBL gene database, and the final expression data were saved as ${log}_{10}(N_{\text{TPM}}+{10}^{-10})+10$. Labels, whether they are baseline or variant, were one-hot encoded.

Transcriptional Response Library.

The transcriptional response data was curated as described in ref. 24. The list of GEO series accession numbers and associated gene perturbations are listed in Dataset S1. The inclusion of a knockdown in the library does not imply the inclusion of its overexpression and vice versa.

Supplementary Material

Appendix 01 (PDF)

Dataset S01 (XLSX)

Data, Materials, and Software Availability

Raw gene expression counts data are available through GEO. Relevant accession numbers are included in Table 1 and Dataset S1. The software and processed data for employing the method are available from the GitHub repository (106). Source data for training TWAVE are stored on Dryad (107).