Data integration for inferring context-specific gene regulatory networks

Abstract

Transcriptional regulatory networks control context-specific gene expression patterns and play important roles in normal and disease processes. Advances in genomics are rapidly increasing our ability to measure different components of the regulation machinery at the single-cell and bulk population level. An important challenge is to combine different types of regulatory genomic measurements to construct a more complete picture of gene regulatory networks across different disease, environmental, and developmental contexts. In this review, we focus on recent computational methods that integrate regulatory genomic data sets to infer context specificity and dynamics in regulatory networks.

Introduction

Transcriptional regulatory networks specify the connections between regulatory proteins and target genes and control the context-specific expression of genes. These networks play important roles in diverse disease, developmental, environmental, and evolutionary contexts [1,2]. The structure of regulatory networks involves many components including but not limited to cis-regulatory elements (CREs), transcription factors, and chromatin remodelers [3] that interact through processes such as three-dimensional organization of the genome [2,4], chromatin modifications [5], and genome accessibility [6]. Advances in next-generation sequencing tools have greatly expanded our ability to measure different aspects of the regulatory machinery. For instance, transcription factor (TF) binding and histone modifications (ChIP-seq [7]), transcription (RNA-seq [8]), chromatin accessibility (DNase-seq, ATAC-seq [6]), and genome-wide three-dimensional organization (chromosome conformation capture [9]) have been measured across individuals [10], diseases [11], developmental stages [12], and environments [13] in both bulk populations and single cells [14].

Using these diverse readouts of the cellular state and regulatory landscape (Figure 1A), there are two major approaches to model gene regulatory networks [15]. The first approach predicts regulatory proteins, for example, TFs, chromatin remodelers, and signaling proteins that determine a gene’s expression level (Figure 1B). The second approach identifies CREs affecting gene expression levels (Figure 1C), that is, TF binding sites in or near the promoter, or distal enhancers interacting with the promoter through chromatin looping [2]. In this review, we discuss advances from the last two years in these two types of computational approaches addressing three main challenges: inferring genome-scale context-specific transcriptional regulatory networks to predict regulatory proteins of individual or groups of genes, predicting target genes of CREs, and integrating single-cell multiomics data sets for gene regulatory network inference.

Figure 1. Data types used in integrative approaches for two main types of regulatory networks:

(a) Input data for inferring regulatory networks can be categorized into context-specific and static data sets. Context-specific gene regulatory events are measured across different layers of gene regulation using various genomic technologies. DNase-seq or ATAC-seq measure open regions of the chromatin and can be leveraged to identify regulatory elements. Chromosome conformation capture (3C) techniques such as Hi-C, HiChIP, and ChIA-PET allow global mapping of 3D genome organization. ChIP-seq and protein binding assays are used for detecting transcription factor (TF) binding to DNA. Transcriptomic assays measure expression levels of regulators and target genes under different conditions. Static databases often used in integrative inference of regulatory networks are DNA-binding motif collections such as JASPAR and CIS-BP, predefined signaling networks, and protein–protein interactions (PPIs). Databases of functional gene annotations and pathways are used for evaluating gene regulatory networks but can also be used as another source of data for integration. (b) Regulatory networks that capture the relationship between regulatory proteins, such as transcription factors and signaling proteins, and target genes. Gene expression measured with RNA-seq or microarrays is the primary data type for inferring these networks. Integrative inference approaches integrate other data types with gene expression such as sequence motifs, filtered by available accessibility (DNAse-seq, ATAC-seq) measurements, TF ChIP-seq, and gene perturbation data. (c) Cis-regulatory networks represent the noncoding regulatory sequences such as enhancers that regulate genes. These networks can be inferred either by learning a gene-centric model that predicts gene expression based on the chromatin state and/or accessibility of candidate enhancers across multiple conditions or a global model that learns predictive rules of enhancer–promoter interactions based on chromatin marks, architectural proteins, RNA-seq, and chromosome conformation (3C) data.

Integrating data to infer genome-scale context-specific transcriptional regulatory networks

A context-specific regulatory network is defined as a network that captures interactions between regulatory proteins and target genes in a particular context, such as a developmental stage, tissue, disease, or environment (Figure 1B and 2A). While there are a large number of mathematical formalisms to represent regulatory networks [16], we focus on statistical models, which include information-theoretic methods [17], dependency networks [18,19], Bayesian networks [20], and Gaussian graphical models [21]. Given a particular modeling formalism, a simple approach to infer a context-specific regulatory network is to use sequence-specific motifs on gene promoters to first define a comprehensive set of possible regulatory edges and then prune out edges with low statistical support using context-specific expression as evidence. Statistical support is assessed by Pearson’s correlation or by regressing candidate regulators to the expression of a target gene. The motif-derived edges can be filtered by requiring the presence of context-specific ATAC-seq or DNase-seq peak overlapping the motif instance [22-26]. This approach identified the regulatory network for Huntington disease [22] and early brain development [26] and predicted key regulators in neurodegenerative diseases which were experimentally validated, for example, SMAD3 [22] and POU3F2 [26]. Although straightforward to implement, this approach can only add edges between TFs and targets with known motifs and may miss regulators with unknown specificity, and accessible motifs are not sufficient to comprehensively capture true regulatory interactions.

Figure 2. Classes of problems in gene regulation that integrate diverse data types and computational approaches to address these problems.

For each class of problems, the input data, outputs, and the algorithmic components used are shown. (a) Inference of genome-scale regulatory networks in a specific condition or across multiple conditions. These methods learn the structure of the network by predicting relationships between TFs and target genes and integrate gene expression with static data from public databases and context-specific data from ChIP-seq and ATAC-seq experiments. Methods use a variety of approaches to infer networks, including Gaussian graphical models (GGMs), dependency networks, mutual information (MI). Context-specific networks are learned by joint inference of networks across multiple conditions. Multi-task learning is a popular framework to enable information sharing across different contexts. (b) Predicting target genes of regulatory sequence elements. These methods use regulatory genomic data sets such as ChIP-seq, ATAC-seq, and 3D genome organization assays to identify targets of cis-regulatory elements. Regression and classification approaches have been developed to either predict counts or discriminate between an interacting and noninteracting pair. (c) Integration of different types of single-cell omic measurements to identify cell type–specific gene regulatory networks. These methods leverage single-cell RNA-seq (scRNA-seq) and scATAC-seq and occasionally bulk data sets. Methods typically use clustering and dimensionality reduction, for example, via matrix factorization, to define data set–specific clusters followed by linking the clusters. Some methods focus on genic regions, while others can incorporate auxiliary data sets such as HiChIP to link to genes. GGM: Gaussian graphical model; SEM: structural equation model; TF: transcription factor.

Incorporating prior to constrain network structure and estimate regulator activity

To enable more comprehensive network inference, sequence-specific motif-derived edges can be incorporated as prior knowledge to encourage the network inference algorithm to include edges from the prior. Prior-based regulatory network inference can use either a structure prior approach [18,20] or parameter prior approach [24]. With the structure-based prior, one puts a prior probability distribution on the graph structure, such that an edge with motif support is more likely to be included in the regulatory network model. For example, Chasman et al. [23] used DNase-seq filtered motifs as a structure prior for a network inference algorithm [18] and context-specific gene expression data of hindbrain and spinal cord development to infer a regulatory network in neuroepithelial stem cells. The parameter prior framework has been used within a regularized regression setting [24], which learns a sparse network by penalizing edge addition while optimizing prediction of a target’s expression level. The prior can be used to reduce the penalty for addition of an edge in the network [24].

Newer methods have used the prior network to additionally estimate hidden TF activity (TFA) to overcome the assumption that mRNA levels must correlate with the regulator’s protein activity on a gene’s promoter [24,27]. TFA estimation takes in an initial, noisy regulatory network and expression and models the expression as a product of the hidden TFAs and a refined network structure [27]. The TFA can be estimated independently followed by network inference, for example, mLASSO-StARS [24], or in a single iterative algorithm, for example, NetRex [27]. mLASSO-StARS [24] learns a dependency network by solving a set of independent LASSO regression problems, one per gene. Both mRNA and TFA are used as potential predictors of a target gene’s expression. NetRex [27] jointly estimates both the TFA and the network to explain the expression data by using regularization to penalize the mismatch in the number of edges of the inferred and prior networks. NetRex updates the prior network until the rewired network and the estimated TFA optimally explains the expression data. Both NetRex and mLASSO-StARS need to specify the hyperparameters to obtain a final network structure. mLASSO-StARS uses StARS to select the hyperparameters by controlling the average instability of edges. NetRex uses a grid search over hyperparameters and obtains the final result from the consensus of multiple nearly optimal settings. NetRex has additional constraints that can better capture coregulatory relationships in gene modules but can make network inference computationally expensive. These approaches were shown to improve the quality of inferred networks and were useful for identifying key regulators and targets in mammalian [23,24] and insect [27] systems.

Network inference across multiple contexts

Often data from multiple contexts are available and a key question is to define regulatory networks for each context and identify similarities and differences. Although one approach would be to infer networks for each context independently [28], examining multiple contexts simultaneously could improve the quality of the inferred networks [21,29], make comparisons easier [30,31], and help in scenarios with low sample sizes [32]. To enable simultaneous network inference across multiple conditions, several approaches have used multi-task learning (MTL) (Figure 2A) [33]. In MTL, related tasks are jointly solved while sharing the information across tasks. In one class of methods, a regulatory network is modeled as a Gaussian graphical model (GGM), which represents the genome-wide expression levels of genes as a multivariate Gaussian and the network structure by a precision matrix. One GGM approach, JRmGRN [21], models each condition’s network as sum of two sparse precision matrices, one for condition-specific components and the second for shared component across all conditions. JRmGRN additionally regularized the shared network to favor more hub genes and was more effective at identifying shared hubs and context-specific network components than existing approaches. While JRmGRN models each condition with one precision matrix, another GGM-based approach, NETI2 [31], additionally models the intrasample heterogeneity, which is common in tumor samples because they represent a mix of cancerous and noncancerous cells. NETI2 infers cancer subtype–specific networks and a network for noncancerous cells shared across the subtypes. The proportion of cancerous and noncancerous cells for each sample is known; however, the expression of each cell type is not known and is estimated using an expectation–maximization algorithm. On TCGA data [34] for different breast cancer subtypes, NETI2 captured subtype-specific gene hubs that exhibited reduced connections in the noncancerous network.

Regulatory networks across multiple conditions have also been modeled using dependency networks [35] and structural equation models (SEMs) [29]. Compared with GGMs, these models can capture directed edges and are therefore more suitable for representing regulatory networks. In addition, dependency networks are computationally faster and often outperform GGMs [19]. One approach, FSSEM [29], uses SEMs to infer a regulatory network for two conditions from gene expression and genetic variation, available in expression quantitative trait locus data sets to better identify the effect of TFs on target genes. SEMs are a class of statistical models that represent observed measurements as a linear combination of other observed or latent variables. To enable sharing, FSSEM uses a penalized framework to minimize the difference in the network between the two conditions. Inferelator-AMuSR [35] is a dependency network-based approach that uses MTL with a block-sparse penalty to encourage similarity between networks while learning overall sparse networks. The regression weight of a regulator across conditions is considered a block, and the sparsity criteria select a small number of conserved blocks, while allowing some regulators to be condition specific. Both FSSEM and Inferelator-AMuSR inferred more accurate networks than when inferring networks independently, suggesting a benefit of joint inference across multiple conditions.

Often the number of samples for a given condition may be too few to predict the regulators of individual genes. To overcome the small sample size, it is better to group genes based on expression levels and model regulatory relationships at the group level at the expense of losing gene-level regulatory information [32]. PSIONIC [32] is an example of this framework that learns patient-specific TF regression weights in multiple cancers using an approach called grouping and overlapping in MTL [36]. In PSIONIC, ATAC-seq–filtered TF motifs from enhancers and promoters associated with a gene are used to predict gene expression. Instead of learning a regression vector for each patient profile independently, PSIONIC’s grouping and overlapping in MTL approach decomposes the matrix of regression vectors into lower dimensional latent regulatory programs and tumor-specific weights representing the contribution of each regulatory program to the expression of each sample. PSIONIC was significantly better at predicting expression in test samples than a single task model, suggesting that sharing information across tumors while learning regulatory programs is advantageous.

To summarize, current methods for genome-scale regulatory network inference have used prior knowledge to constrain the network structure and estimate hidden regulator activity [24,37]. Across multiple contexts, MTL-based approaches have improved network inference, especially for the shared components [21,29]. The type of network inference algorithm depends upon the number of gene expression samples and the availability of sequence-specific motifs and matched ATAC-seq measurements for each context of interest.

Data integration for predicting CREs controlling expression of target genes.

Identification of CREs that regulate a gene’s context-specific expression, proximally or distally, is a grand challenge in gene regulation [3-5]. A number of approaches exist (refer the study by Xu et al. [38] for a comprehensive review); here, we review recent methods that integrate multiple types of regulatory genomic data to predict such interactions. These methods can be grouped into (Figure 1C) (a) gene-centric approaches, which predict the regulatory elements by learning a model for each gene [39-41], and (b) global approaches, which learn a single model for all gene–element pairs [42,43].

Gene-centric methods to predict cis-regulatory interactions

Gene-centric methods leverage gene expression and a small number of regulatory signals from multiple conditions to explain the expression level of a gene as a function of the regulatory activity of CREs. Current methods have used regularized regression [40,44] or mutual information [41] to link elements to genes (Figure 2B, [38]). These methods also vary in the size of the neighborhood around a gene to search for regulatory elements, the specific regulatory signal used to correlate or predict expression, and the number of cell lines examined. In particular, FOCS and Vijayabaskar et al. [40,44] used regression to select CREs for a gene within the 100 Kb and 500 Kb region, respectively, around the transcription start site (TSS). Instead, MICMIC [41] used conditional mutual information to predict which regulatory elements, defined by CpG methylation levels, within the 300 Kb window of a gene’s TSS explain the expression of the gene. MICMIC is suitable for large-scale studies that have matched RNA and DNA methylation measurements, for example, in cancer [45]. The activity-by-contact (ABC) model [39] predicted enhancer–gene connections assuming that an en-hancer’s quantitative effect on a gene’s expression de-pends on its strength (activity), weighted by the contact count of the enhancer. The activity is estimated from H3K27ac and DNaseI accessibility for the element and count from a Hi-C or related experiment. The ABC score is the relative effect of an element on a gene’s expression and accurately captured the measurements of CRISPRi-FlowFish experiments, which perturb en-hancers and quantify the effects on gene expression. The ABC model is conceptually the simplest model and does not have any parameters; however, it might need more careful calibration compared with regression-based or information-theoretic approaches.

Global methods to infer cis-regulatory interactions

Global methods in contrast learn a single predictive model of interaction presence or strength by leveraging cell type–specific measurements such as gene expression, histone modifications, and TF binding. These approaches vary depending upon the input data types and whether they use a classification or regression framework (Figure 2B). Recent classification-based approaches have relied on sequence as the input features to predict interactions (e.g, SPEID [46]) and accessible sequences (e.g., DeepTACT [47]), while others have used sequence conservation and chromatin marks (e.g, EPIP [48]). A variety of classification algorithms have been used including AdaBoost (EPIP [48]) and deep neural networks (DeepTACT [47], SPEID [46]). Methods that rely on sequence are broadly applicable across many contexts but might be limited in capturing cell line–specific information compared with methods that incorporate additional data sets such as accessibility and chromatin marks.

Classification approaches rely on a training set of true enhancer–promoter interactions and could be susceptible to the definition of the positive label set. Recent approaches have attempted to predict the contact count directly using regression instead of discriminating between interactions and noninteractions. For example, HiC-Reg [42] trains a random forest regression model to predict Hi-C contact counts by integrating published high-resolution Hi-C data sets with epigenomic marks and architectural proteins. Another approach, 3DPredictor, predicted Hi-C contact counts with gradient boosting and features derived from RNA-seq and CTCF ChIP-seq data [43]. While HiC-Reg predicts counts for all pairs of genomic loci, including pairs with nonpromoter regions, 3DPredictor focuses on enhancer–promoter pairs. Regression-based approaches have the advantage that they do not need positive labels of interactions, and their outputs can be used to identify topologically associating domains.

To summarize, the advantage of gene-centric approaches is that they are specific to each gene and can model multiple enhancer elements per gene. However, most of these approaches learn a single model for all contexts and require sufficient samples for model learning. Global approaches can predict interactions in a single biological context but typically model only one enhancer per gene and require multiple measurements for the same context. Selection of a specific tool for predicting CREs of a gene should be guided by (a) the number of available samples with gene expression and regulatory signals, (b) availability of Hi-C or similar measurements, and (c) need to identify interaction pairs or examine higher-order organizational units.

Methods to learn regulatory networks from single-cell omics data types

Single-cell techniques are rapidly increasing our ability to measure multiple types of omic profiles, such as transcriptomes [49], epigenomes [50], methylomes [51], and 3D genome organization [52] for individual cells. Several methods have been developed [53] and benchmarked for inferring gene regulatory networks from scRNA-seq data [54]. More recently, the availability of scRNA-seq and scATAC-seq data from the same underlying subpopulations (but not necessarily from the same cells) offers an exceptional opportunity to infer cell type–specific gene regulatory networks (Figure 2C).

A key challenge with integrating scRNA-seq and scATAC-seq data is the lack of correspondence between cells as they come from different populations. To address this issue, one strategy is to cluster or project the scRNA-seq and scATAC-seq data separately and then integrate the clusters. Bravo González et al. [55] leveraged the spatial information of cell types in the Drosophila eye-antennal disc tissue to integrate scRNA-seq and scATAC-seq data. Briefly, cell types are defined using the scRNA-seq and scATAC-seq measurements independently, followed by assigning each cell a pseudotime based on their position on the distal–proximal (antennal cells) or anterior–posterior (eye cells) axis. Each cell was also mapped onto a spatial map of virtual cells based on the cell type annotations and their relative position in pseudotime. Once mapped, transcriptomic and epigenomic profiles are available for the same virtual cell, which was used to derive links between enhancers and genes using random forest regression to predict expression based on accessibility at candidate regions in a gene-centric manner.

In contrast, SOMatic [56] integrated scATAC-seq and scRNA-seq data with self-organizing maps (SOMs), which are based on neural networks and used for dimensionality reduction and visualization of high-dimensional data on a 2D map. SOMs were first trained on scATAC-seq and scRNA-seq data separately; then, SOM units, each representing a cluster of genes or regulatory regions, were grouped into metaclusters so that SOM units nearby on the 2D map remain close in the cluster. A linking function was used to link genes and genomic regions into Linked SOM metaclusters of regulatory regions around the TSS that also overlap with ATAC-seq peaks. Linked SOM metaclusters with enriched motifs were used to construct a regulatory network. Compared with Gonzalez-Blas et al., SOMatic does not require additional spatial information to integrate the scRNA-seq and scATAC-seq data. However, it is dependent on the ability to reliably link a scATAC-seq cluster with a scRNA-seq cluster. SOMatic also does not predict enhancer–promoter interactions.

While SOMatic and González-Blas et al. both first define clusters and then link them, DC3 [57] uses an alternative approach by jointly factoring the scRNA-seq and scATAC-seq data by integrating single-cell or bulk HiChIP data available for the same condition. The HiChIP data enable DC3 to incorporate long-range interactions between enhancers and genes. DC3 uses a joint non-negative matrix factorization (NMF) approach where the scRNA-seq and scATAC-seq data sets are each represented by an NMF decomposition term but the NMF factors are coupled to maximize the concordance with HiChIP. Importantly, DC3 is flexible to incorporate single-cell or bulk RNA-seq, ATAC-seq, and HiChIP. The output of DC3 provides the gene expression, accessibility, and long-range interaction profile for each subpopulation, which can be used to identify important regulators based on differential expression, accessibility, and regulatory networks based on the enhancer–gene interaction profile.

To summarize, recent integrative methods for regulatory network inference from single-cell data sets use dimensionality reduction or clustering to define cell populations independently [55,56] or jointly [57] from scRNA-seq and scATAC-seq data. Methods that independently define clusters use additional postprocessing to link the clusters but do not require additional long-range data sets (e.g., HiChIP or Hi-C). On the other hand, methods such as DC3 align cell type–specific gene expression patterns to accessibility patterns by incorporating long-range interaction profiles. Regulatory networks are defined by identification of CREs associated with scATAC-seq clusters linked to scRNA-seq clusters.

Conclusion

In this review, we covered recent approaches to integrate regulatory genomic data sets to predict sequence or protein regulators of context-specific gene expression from bulk and single-cell data sets (Supplementary Table 1). Despite the availability of several promising methods, network inference remains challenging and we envision development in several areas going forward. One area is to develop methods that can combine both cis and trans regulatory information to define context-specific regulatory networks, while accounting for both distal and proximal interactions of CREs. In addition to being more comprehensive, such integrated regulatory networks would help determine the role of noncoding variation in gene expression modules and pathways in complex traits [58]. Another emerging area is to integrate single-cell and bulk data sets to gain insights into cell type–specific regulatory networks [57] and to extract cell type–specific signatures from complex patient samples [59], where single-cell assays might be difficult or expensive. Such approaches could significantly advance our understanding of cell type–specific regulatory networks derived from disease samples. A third challenge is to develop systematic benchmarks for evaluation of regulatory networks in mammalian systems. This would require a concerted effort from experimental and computational researchers to test model-driven predictions in a high-throughput manner. Recent technological advances such as Perturb-Seq [60] and Perturb-ATAC [61] offer powerful tools to perturb and monitor transcriptional state and enable high-throughput validation of network predictions. Such benchmarks would be important for the development and application of regulatory network-based methods for examining normal and disease processes.