GRANet: a graph residual attention network for gene regulatory network inference

Abstract

The reconstruction of gene regulatory networks (GRNs) is crucial for uncovering regulatory relationships between genes and understanding the mechanisms of gene expression within cells. With advancements in single-cell RNA sequencing (scRNA-seq) technology, researchers have sought to infer GRNs at the single-cell level. However, existing methods primarily construct global models encompassing entire gene networks. While these approaches aim to capture genome-wide interactions, they frequently suffer from decreased accuracy due to challenges such as network scale, noise interference, and data sparsity. This study proposes GRANet (Graph Residual Attention Network), a novel deep learning framework for inferring GRNs. GRANet leverages residual attention mechanisms to adaptively learn complex gene regulatory relationships while integrating multi-dimensional biological features for a more comprehensive inference process. We evaluated GRANet across multiple datasets, benchmarking its performance against state-of-the-art methods. The experimental results demonstrate that GRANet consistently outperforms existing methods in GRN inference tasks. In addition, in our case study on EGR1, CBFB, and ELF1, GRANet achieved high prediction accuracy, effectively identifying both known and novel regulatory interactions. These findings highlight GRANet’s potential to advance research in gene regulation and disease mechanisms.

Introduction

Gene regulatory networks (GRNs) characterize the regulatory relationships between transcription factors (TFs) and their target genes, playing a crucial role in exploring gene interactions, cellular behaviors, and disease mechanisms. The study of GRNs provides insights into cell fate determination, disease development, and transcriptional regulatory mechanisms [1]. The rapid development of single-cell RNA sequencing (scRNA-seq) technology has revolutionized the study of cellular heterogeneity, enabling high-resolution inference of cell-specific GRNs [2].

With advancements in high-throughput sequencing technologies, single-cell transcriptomic data offer unprecedented opportunities for GRN reconstruction [3]. Various machine learning and deep learning approaches have been employed to infer GRNs at the single-cell level. For instance, SCODE [4] models the dynamic changes in gene expression using linear ordinary differential equations (ODEs). However, the assumption of linear regulatory relationships oversimplifies the inherently complex and nonlinear nature of GRNs. GENIE3 [5], a random forest–based algorithm, formulates GRN inference as a regression problem, learning the impact of regulatory genes on their targets. While effective, GENIE3 is computationally intensive, making it less scalable for large datasets.

SINCERITIES [6] focuses on temporal single-cell gene expression data, capturing regulatory relationships by analyzing changes in gene expression distributions across time points. It employs regularized linear regression (ridge regression) and partial correlation analysis to infer activation or inhibition interactions.

With the rise of deep learning [8], methods such as GNE [7] and CNNC [9] have outperformed traditional approaches by leveraging neural networks. GNE utilizes multi-layer perceptrons (MLPs) to quantify gene pair associations, facilitating the identification of active regulatory pathways under various conditions. CNNC employs a supervised learning framework, transforming gene pair relationships into histograms, which are then processed using convolutional neural networks (CNNs) to uncover regulatory relationships.

More advanced deep learning models, such as dynDeepDRIM [10] and DeepSEM [11], further enhance GRN inference. dynDeepDRIM utilizes high-dimensional CNNs to reconstruct GRNs from time-stamped scRNA-seq data, incorporating contextual gene interactions into its predictions. DeepSEM combines variational autoencoders (VAEs) with structural equation modeling (SEM) to capture nonlinear relationships and latent mechanisms, making it particularly robust against missing data.

Graph-based models, such as NSRGRN [12] and GENELink [13], integrate topological structures and gene embeddings to enhance GRN inference. NSRGRN introduces a network structure refinement algorithm that improves the inference network by considering both local and global topologies. GENELink utilizes graph attention networks to embed single-cell gene expression data, projecting TF-gene pairs into a low-dimensional space for causal inference. Similarly, matrix factorization-based models like GMFGRN [14] exploit latent feature spaces to uncover complex regulatory patterns beyond traditional approaches.

Other notable methods further expand GRN inference capabilities. DeepRIG [15] constructs weighted gene co-expression networks and employs graph autoencoders (GAEs) to model global regulatory dynamics. LogBTF [16] integrates logistic regression with Boolean threshold functions to reconstruct GRNs from time-series expression data. PSGRN [17] employs pseudo-Siamese networks and DenseNet frameworks to learn spatiotemporal features of gene interactions, while Gene regulatory network inference based on causal discovery integrating with graph neural network (GRINCD) [18] combines causal asymmetry learning with graph representation learning to capture both linear and nonlinear regulatory relationships.

These diverse approaches demonstrate the evolving methodologies in GRN inference, showcasing the integration of traditional computational models with cutting-edge deep learning techniques to address the challenges of single-cell and large-scale datasets.

GATCL [19], an extension of GENELink, enhances graph attention networks (GATs) by replacing the matrix multiplication between the feature matrix and weight matrix with convolutional layers, significantly improving computational efficiency. Additionally, GATCL integrates multi-head graph attention with self-attention mechanisms, allowing for a more comprehensive consideration of multiple data features. This enables the model to adaptively assign different weights to the regulatory neighbors of each gene during optimization, dynamically selecting the most influential genes for inference. By reducing redundant information, this mechanism enhances model accuracy. However, GATCL’s limited preprocessing capabilities constrain its ability to handle noise and uncertainty in large-scale datasets. High training errors observed in multiple experiments indicate potential areas for improving its feature extraction layers.

While existing GRN inference methods demonstrate distinct strengths, they often fail to extract comprehensive information from gene expression data. In recent years, graph neural networks (GNNs), particularly GATs, have gained prominence in analyzing network-structured data due to their ability to dynamically assign weights to neighboring nodes through attention mechanisms [20]. Compared to traditional approaches, GATs are more effective in capturing the intricate regulatory relationships among genes, especially in the high-dimensional feature space of single-cell RNA-seq data.

We present the following key contributions in this study:

(i) We propose GRANet, a supervised deep learning framework based on multi-head GATs for constructing GRNs. GRANet addresses key limitations of existing methods, including inadequate gene modeling, limited integration of multi-source data, and suboptimal deep network expression capabilities. Our model demonstrates robust performance across multiple publicly available datasets.

(ii) We introduce a novel data extraction pipeline tailored for scRNA-seq data, generating three distinct gene feature representations from gene expression matrices: smoothed, discretized, and standardized feature sets. These representations enhance the integration of diverse gene expression features, improving GRN inference.

(iii) To address overfitting challenges in graph-based models applied to small-scale scRNA-seq data, we introduce an internal residual mechanism to stabilize GAT training. Additionally, we leverage multi-head attention mechanisms to model complex regulatory relationships among genes dynamically.

Materials and methods

Datasets

We evaluated the performance of GRANet in GRN inference using seven widely-used single-cell RNA sequencing (scRNA-seq) datasets, which are considered standard benchmarks for deep learning methods [21–23]. The datasets include: (i) human embryonic stem cells (hESCs); (ii) human mature liver cells (hHEPs); (iii) mouse dendritic cells (mDCs); (iv) mouse embryonic stem cells (mESCs); (v) mouse hematopoietic stem cells with erythroid lineage (mHSC-Es); (vi) mouse hematopoietic stem cells with granulocyte–monocyte lineage (mHSC-GMs); and (vii) mouse hematopoietic stem cells with lymphoid lineage (mHSC-Ls). These datasets contain real functional interaction networks sourced from the STRING database [24], nonspecific ChIP-seq experiments [25–27], and cell-type-specific ChIP-seq experiments [28–30]. The detailed results are presented in Table 1.

Table 1.

Comprehensive details of the eight datasets. The size of training sets about each ground-truth network with TFs and the 500 (or 1000) most-varying genes.

Dataset	Species	STRING	Nonspecific	Cell-type-specific
hESC	Human	4257 (5149)	3441 (4617)	4545 (7084)
hHEP	Human	7523 (9003)	4129 (5351)	9939 (15558)
mDC	Mouse	4815 (5898)	3067 (3918)	756 (1193)
mESC	Mouse	7762 (8479)	6893 (8030)	29,613 (42795)
mHSC-E	Mouse	1371 (1826)	1425 (1960)	11,557 (21975)
mHSC-GM	Mouse	748 (1311)	743 (1358)	7364 (14135)
mHSC-L	Mouse	137 (154)	279 (317)	4398 (5180)

Each scRNA-seq dataset was processed using the method proposed by Pratapa et al. [23], with interactions inferred from TFs. During this analysis, we selected the top 500 and 1000 genes showing the most significant expression level changes and including all TFs with corrected P-values below .01 [37]. Finally, we divide the dataset into three subsets: training set, validation set, and test set. Further details are provided in Supplementary Text S1. The seven scRNA-seq datasets are available from the Gene Expression Omnibus under the following accession numbers: GSE81252 (hHEP), GSE75748 (hESC), GSE98664 (mESC), GSE48968 (mDC), and GSE81682 (mHSC). Additionally, all datasets with their true networks for four different types of interactions can be accessed at https://doi.org/10.5281/zenodo.3378975.

Framework of GRANet

GRANet, similar to other GNN models [31–34], predicts relationships between nodes based on their distinct features. In the task of GRN inference, nodes are categorized as either TFs or non-TFs, with edges representing regulatory interactions. As illustrated in Fig. 1, GRANet is composed of three main components: the multi-feature set extraction module, the graph attention fusion module, and the embedding prediction module.

Figure 1.

The framework of GRANet. (A) Multiple feature sets are extracted from single-cell RNA sequencing (scRNA-seq) data to capture comprehensive gene expression information. Through normalization, moving average smoothing, and discretization, data quality is improved, thereby enhancing model performance and providing robust support for subsequent analyses. (B) The processed multi-feature data are then fed into the prediction model. RGAT leverages a multi-head attention mechanism to capture diverse relationships between nodes, incorporating residual connections to enhance model stability. Subsequently, RCGAT integrates convolutional layers to extract local spatial features, further refining the node representations. (C) The final node embeddings are passed into independent fully connected modules, which separately process the features of transcription factors and target genes. Regulatory scores between nodes are computed using dot products of their respective embeddings.

The multi-feature set extraction module processes the original gene expression matrix through several transformations, including standardization, smoothing, and discretization, to modify its distribution. These transformations reduce sparsity and noise in the matrix, thereby enhancing the quality of input features.

The graph attention fusion module integrates two enhanced GATs and a CNN layer. The first GAT captures local interactions between TF nodes and their immediate neighbors, generating preliminary node representations. The second GAT refines these representations by incorporating global interactions, considering broader neighborhoods, and resulting in higher-level node embeddings. This layer-wise feature extraction and information aggregation enable the model to capture the complex relationships and structural dynamics within gene regulatory networks, improving its performance.

Finally, the node embeddings are passed into the MLP layer (embedding prediction module) to further refine the representations for both TFs and non-TFs. The relationship score between two nodes is computed using the dot product of their respective embeddings.

Multi-feature set extraction module

In this study, we first reconstruct the gene expression matrix using a Variational Autoencoder (VAE) [35] to capture the underlying structure and reduce noise in the single-cell RNA sequencing (scRNA-seq) data. The VAE is a generative model that learns a low-dimensional latent representation of the high-dimensional gene expression data by optimizing a variational lower bound on the data likelihood. Specifically, the VAE consists of an encoder network, which maps the input gene expression data to a latent space, and a decoder network, which reconstructs the data from the latent representation [38]. The latent space is constrained to follow a prior distribution, encouraging the model to learn a smooth and continuous representation of the data. The VAE is trained separately on the scRNA-seq data using a loss function that combines a reconstruction loss (to ensure accurate data reconstruction) and a Kullback–Leibler (KL) divergence term (to regularize the latent space), as shown in the following equations:

(1)

(2)

where is the reconstructed gene expression data, is the mean of the latent space, is the log variance, and is the dimensionality of the latent space. This reconstruction step enhances data quality by denoising and imputing missing values, providing a more robust foundation for subsequent feature extraction.

Following reconstruction, multiple feature sets are extracted from the processed scRNA-seq data to capture he underlying information in gene expression more comprehensively. These multi-dimensional feature extractions enable a more accurate characterization of gene expression patterns across different spatial locations, providing rich informational support for subsequent model training and prediction.

To eliminate batch effects and scale differences, we first performed normalization on the reconstructed gene expression data. This normalization process transforms the data to have zero mean and unit variance, thereby enhancing comparability and improving model stability.

To reduce noise in the reconstructed gene expression data, we applied a moving average smoothing method. For each gene’s expression value , a rolling window of size 3 is used to compute the average of expression values within the window:

(3)

where is the window size. This process helps to reduce fluctuations in the data, thereby enhancing the stability and predictive performance of the model.

After applying the above smoothing step, we performed discretization on the reconstructed gene expression data by mapping extreme values (outliers) into a reasonable range of bins, thereby minimizing their impact on the model. The discretization steps are as follows:

First, we apply a log transformation to the nonzero reconstructed expression values to reduce skewness and make the data more normally distributed. Let the transformed values be denoted as . Next, we compute the mean , minimum , standard deviation , and maximum of these log-transformed values. The discretization bounds are defined as:

(4)

(5)

The bucket width is determined as:

(6)

where is the number of discrete bins used during the discretization. Once the discretization boundaries and bucket width are defined, each expression value is mapped to a corresponding bin:

(7)

Finally, any expression values exceeding the upper bound are capped at , and values below the lower bound are set to 0. This discretization process transforms continuous gene expression values into discrete bins, simplifying data processing and model training while preserving key expression information.

Graph attention fusion module

In the proposed fusion prediction module, the first layer incorporates a multi-head graph attention mechanism with internal residual connections (RGAT), as shown in Fig. 2. Unlike traditional graph attention mechanisms [32], where all attention heads share the same input feature matrix, our model assigns a distinct input to each attention head, allowing each one to capture different aspects of the data.

Figure 2.

The architecture of the RGAT. This figure illustrates the graph attention mechanism with an internal residual connection. The input node features and their neighboring features are aggregated through attention-weighted summation to form the multi-head attention representation . Simultaneously, the original input node features pass through a multilayer perceptron (MLP) to generate an adaptive residual connection. Finally, the residual connection is added to the multi-head attention results to obtain the final node representation . This module effectively captures complex graph structural relationships and enhances the model’s expressive power.

The input consists of three feature sets derived from the multi-feature extraction component, along with the corresponding adjacency matrices. For each attention head, the input feature matrix is linearly transformed to produce a new node representation:

(8)

where is the linear transformation matrix for the k-th attention head, is the input dimension, and is the output dimension. The resulting matrix represents the transformed node features produced by the k-th attention head and serves as the input for subsequent attention-based aggregation. After obtaining the node representations for each head, a masking mechanism is applied to compute the attention weights between node and its neighbor :

(9)

where is the parameter vector used for computing the attention weights, denotes the concatenation of features, and [36] is the activation function. This step captures the relationship strength between nodes. These attention weights are then normalized using the softmax function to ensure they sum to 1:

(10)

where represents the set of neighbors for node , and is the normalized attention weight for node relative to node . The normalized attention weights are then used to aggregate features from neighboring nodes to update the feature representation for node . The outputs from all attention heads are concatenated (or averaged) to form the final node feature representation:

(11)

where represents the number of attention heads. The averaging operation enhances the model’s ability to capture diverse graph structural features.

To prevent vanishing gradients and accelerate convergence, residual connections are introduced. Specifically, the final feature representation for node is a weighted combination of the original features and the output from the multi-head attention:

(12)

where and are learnable parameters. This dynamic adjustment of residual coefficients during training improves the model’s performance by better integrating information across attention heads.

In the second layer, we introduce a module that combines convolutional layers with the graph residual attention mechanism, referred to as the Residual Convolutional Graph Attention Network (RCGAT). This module enhances the GRN inference by integrating spatial feature extraction with graph structure modeling. To improve the model’s ability to capture latent features, we prepend a convolutional layer to the RCGAT.

The output from the first RGAT layer is processed through a convolutional layer as follows:

(13)

After the output from the first RGAT layer undergoes convolutional embedding, the convolutional layer replaces the matrix multiplication typically used in GAT with a convolutional interface. This allows the model to extract local spatial features from the node representations. The aggregation of neighboring node features is then performed similarly to the RGAT layer:

(14)

where represents the attention weights for each node pair, computed as in the first RGAT layer. Finally, the multi-head attention results are fused through averaging, and a residual connection is applied with the original input features:

(15)

Embedding prediction module

After feature extraction by RCGAT, we refine the regulatory relationships between TFs and target genes by designing two independent fully connected feature extraction modules. These modules apply specific nonlinear transformations to the embedded feature matrix , allowing the model to learn dedicated representations for the TFs and target genes. The transformation process is as follows:

(16)

(17)

where are the weight matrices for the linear layers, and and are the bias terms.

After obtaining the feature representations for transcription factors and target genes, we compute the regulatory score between each TF–target gene pair using the dot product operation to quantify their potential association strength:

(18)

Finally, the GRANet model is optimized using a modified pointwise binary cross-entropy (BCE) loss function:

(19)

where is the weight for positive samples, improving the model’s handling of imbalanced datasets.

Results

Performance comparison with state-of-the-art methods

To comprehensively evaluate the performance of GRANet, we compared it with several state-of-the-art gene regulatory network inference methods across seven datasets (hESC, hHEP, mDC, mESC, mHSC-E, mHSC-GM, mHSC-L), two scales (TFs + 500 and TFs + 1000), and three ground-truth networks (STRING, nonspecific ChIP-seq, and cell-type-specific ChIP-seq). The comparison included a diverse range of methods, spanning statistical approaches, machine learning models, and advanced deep learning frameworks, such as GATCL, GENELink, GNE, GRINCD, SCENIC, DeepRIG, GENIE3, Pearson Correlation Coefficient (PCC), and DeepSEM. These methods represent a broad spectrum of techniques widely used for gene regulatory network inference tasks.

For all experiments, we employed standardized evaluation metrics—AUROC and AUPRC—to assess the models’ performance in classifying positive and negative samples. Each metric was averaged over 50 independent runs to ensure statistical robustness [41, 42].

Extensive experiments were conducted on datasets derived from STRING, Non-Specific, and Specific networks to validate GRANet’s effectiveness in gene regulatory network inference rigorously. In terms of AUROC, GRANet consistently outperformed existing baseline models, including GATCL and GENELink, as illustrated in Fig. 3. Specifically, GRANet achieved average performance improvements of 1.62%, 2.24%, 3.62%, 23.97%, 45.47%, 26.76%, 31.53%, 32.23%, and 34.77% over GATCL, GENELink, DeepRIG, GNE, GRINCD, SCENIC, GENIE3, PCC, and DeepSEM, respectively.

Figure 3.

Average AUROC scores of 10 models across seven ground-truth networks on three kinds of datasets. This figure illustrates how GRANet and nine competing approaches perform in discriminating true regulatory interactions across three different ground-truth networks: (A) the STRING dataset, (B) the nonspecific ChIP-seq dataset, and (C) the cell-type-specific ChIP-seq dataset.

Similarly, the evaluation of GRANet using the AUPRC metric is presented in Fig. 4. GRANet achieved the best performance in 88% (37 out of 42) of the scRNA-seq datasets, demonstrating its robustness across diverse conditions. On average, GRANet outperformed the baseline methods by 10.66%, 25.62%, 70.48%, 88.29%, 68.21%, 102.55%, 124.14%, 123.57%, and 98.01% over GATCL, GENELink, DeepRIG, GNE, GRINCD, SCENIC, GENIE3, PCC, and DeepSEM, respectively.

Figure 4.

Average AUPRC scores of 10 models across seven ground-truth networks on three kinds of datasets. This figure highlights the precision-recall trade-offs for GRANet compared to nine alternative methods on various ground-truth networks: (A) the STRING dataset, (B) the nonspecific ChIP-seq dataset, and (C) the cell-type-specific ChIP-seq dataset.

These results further highlight GRANet’s superior ability in distinguishing positive and negative samples, especially in scenarios where positive samples are scarce and imbalanced. Its consistent performance across various datasets highlights GRANet’s potential as a reliable tool for GRN inference.

Moreover, the experimental results demonstrate that GNN-based models consistently outperform traditional machine learning models and other neural network architectures across key performance metrics. This underscores the rising importance of GNNs in the field of GRN inference.

Compared to conventional machine learning methods, GNNs provide a more effective framework for analyzing the complex, non-Euclidean relationships among genes. Specifically, GNNs and their variants, such as GATs, graph convolutional networks, and GRNs, excel at capturing both the local topological structures of gene regulatory networks and long-range relational features through multi-layered embedding mechanisms. This dual capability makes them particularly well suited for uncovering intricate regulatory patterns within biological networks.

Ablation study

To further validate the impact of different modules on the overall performance of GRANet, we conducted a series of ablation experiments by progressively removing or replacing key components of the model and analyzing their effects on performance [39]. Specifically, we constructed four variants of GRANet (as shown in Table 2): (i) GRANet w/o M: GRANet without the multi-feature set extraction module, using only the raw gene expression matrix as input. (ii) GRANet w/o R: GRANet without the residual connections in the graph attention mechanism. (iii) GRANet w/o C: GRANet without the convolution operations in the second RCGAT layer, replaced by standard matrix multiplication. (iv) GRANet w/o V: GRANet without the VAE encoder, which removes the variational autoencoder module used for gene expression matrix reconstruction. Each variant was trained using the same hyperparameters as GRANet and evaluated its performance on the same seven benchmark scRNA-seq datasets (TFs + 500).

Table 2.

Variants of GRANet and their modifications.

Model	Multi-feature set	Residual connections	Convolution	VAE
GRANet	√	√	√	√
GRANet w/o M	×	√	√	√
GRANet w/o R	√	×	√	√
GRANet w/o C	√	√	×	√
GRANet w/o V	√	√	√	×

The check mark indicates that this module is included in the model, while the cross mark indicates that this module is not included in the model.

As shown in Fig. 5, removing any of the key components significantly degraded the model’s performance. Notably, the removal of the multi-feature set extraction module led to the most substantial performance drop, with the AUROC decreasing by 6.82% and the AUPRC decreasing by 6.45%. Removing the residual connections in GAT resulted in a 4.37% decrease in AUROC and a 4.12% decrease in AUPRC, while removing the VAE encoder caused a 5.23% decrease in AUROC and a 4.98% decrease in AUPRC. These results demonstrate that the multi-feature set extraction module, internal residual connections in GAT, embedded convolution operations in RCGAT, and the VAE encoder each contribute significantly to the performance improvement of GRANet.

Figure 5.

Ablation experiments. (A) Distribution of AUROC values for the four variant models and GRANet across seven datasets. The average performance degradation of each variant model is 6.82%, 4.37%, 4.36%, and 5.23%, respectively. (B) Distribution of AUPRC values for the four variant models and GRANet across seven datasets. The average performance degradation of each variant model is 6.45%, 4.12%, 4.76%, and 4.98%, respectively.

To further investigate the contribution of each feature type, we conduct additional ablation experiments in the Supplementary Text S2, where the standardized, smoothed, and discretized features are evaluated independently. Three variants of GRANet are shown in Supplementary Table S1, and the AUPRC values under four different cases in the Ablation Study are shown in Supplementary Fig. S1.

Impact of different parameters on the GRANet model

To investigate the sensitivity of GRANet to hyperparameters, we systematically adjusted several key hyperparameters, including the number of attention heads in the RCGAT, the embedding dimensions of TFs, the model’s learning rate, and the dropout rates across various modules. The performance was evaluated on seven benchmark scRNA-seq datasets. Finally, we validated GRANet’s performance under some alternate settings of critical hyperparameters, with results presented in the Supplementary Table S2.

Impact of the number of attention heads in the RCGAT layer on the GRANet model

We varied the number of attention heads in the second RCGAT layer (1, 2, 3, 6, and 8), keeping other hyperparameters fixed. Performance was measured using average AUROC and AUPRC over 10 experiments (Table 3). Increasing the number of heads from 2 to 3 improved AUROC by 3.2% and AUPRC by 4.5%, indicating enhanced feature capture. However, further increases to the number of heads to 6 yielded diminishing returns, and using 8 heads slightly degraded performance while increasing training time by 35%. This suggests a trade-off between feature representation and computational efficiency, with 3 heads being optimal.

Table 3.

 The impact of varying the number of heads in the multi-head attention layer.

Number of heads	AUROC	AUPRC	Time
1	0.723	0.418	4m07s
2	0.777	0.402	4m52s
3	0.802	0.421	5m36s
6	0.723	0.411	7m29s
8	0.735	0.396	8m35s
16	0.621	0.337	12m24s

Impact of embedding dimensions on the GRANet model

We tested embedding dimensions of 8, 16, 32, and 64 for TFs and target genes. An embedding dimension of 16 achieved the best performance in both the AUROC and the AUPRC (Table 4). Lower dimensions (e.g. 8) limited the model’s representational capacity, while higher dimensions (e.g. 32, 64) introduced redundancy and increased computational costs without providing significant improvements. An embedding dimension of 16 offered an optimal balance between performance and efficiency, making it the ideal choice for modeling gene regulatory networks.

Table 4.

 The impact of different embedding dimensions for gene features and TFs.

Feature dimensions	AUROC	AUPRC	Time
8	0.628	0.323	5m25s
16	0.802	0.421	5m36s
32	0.786	0.431	6m37s
64	0.734	0.419	7m16s

Impact of discretization level on the GRANet model

We evaluated the impact of varying discretization levels (5, 10, 20, 25, and 30) in the multi-feature extraction module, keeping other hyperparameters constant. Performance was measured using the average AUROC and AUPRC over 10 experiments (Table 5). Increasing the discretization level from 5 to 20 improved AUROC by 5.8% and AUPRC by 7.1%, indicating that higher discretization levels capture more detailed features. However, when the discretization level exceeded 20 (e.g. 25 or 30), a slight decrease in performance was observed, accompanied by a significant increase in computational cost. These results suggest that a discretization level of 20 offers an optimal balance between feature representation and computational efficiency.

Table 5.

 The impact of different discretization levels in multi-feature extraction.

Discretization level	AUROC	AUPRC	Time
5	0.728	0.373	4m51s
10	0.788	0.407	5m12s
20	0.802	0.421	5m36s
25	0.791	2.00	5m43s
30	0.760	1.00	6m08s

Case study

In this section, we further evaluated the predictive capability of GRANet in identifying potential target genes of specific TFs through a case study [40]. The experiments were conducted on cell-type-specific datasets (mHSC-GM 500). Specifically, we used the trained model to predict unknown TF–gene pairs and selected the top 25 TF–gene interaction pairs with the highest prediction scores. These predictions were validated using the publicly available hTFtarget database, and the results are summarized in Fig. 6.

Figure 6.

Application of GRANet to predict potential target genes in the mHSC-GM dataset. (A) Ranking plots of the top 25 predicted potential target gene scores for CBFB. (B) Ranking plots of the top 25 predicted potential target gene scores for ELF1. (C) Ranking plots of the top 25 predicted potential target gene scores for EGR1.

We focused on three transcription factors: EGR1 (Early Growth Response Protein 1), CBFB (Core-Binding Factor Subunit Beta), and ELF1 (E74-Like Factor 1). EGR1 is a key TF involved in regulating various biological processes, including cell proliferation, differentiation, and apoptosis [43]. CBFB is a key component of the core-binding factor complex, playing a critical role in hematopoiesis and skeletal development [44]. ELF1, a member of the ETS family of transcription factors, regulates immune response, cell differentiation, and cancer progression [45]. Dysregulation or mutations in EGR1, CBFB, and ELF1 are closely associated with various diseases, including cancer and immune system disorders [46, 47].

The validation of the top 25 predicted target genes for EGR1, CBFB, and ELF1 using the hTFtarget database revealed highly accurate predictions. Specifically, 20 out of 25 target genes for EGR1, all 25 for CBFB, and 24 out of 25 for ELF1 were experimentally validated, as shown in Fig. 6. These results highlight the robustness and reliability of GRANet in identifying potential TF–gene regulatory relationships, demonstrating its ability to effectively predict both known and novel regulatory relationships, as shown in Fig. 7. The high validation rates for CBFB and ELF1, in particular, highlight the model’s precision in capturing biologically relevant interactions. Notably, the unvalidated target genes, especially those for EGR1 and ELF1, may represent novel regulatory mechanisms or pathways that have not yet been documented in existing databases or experimentally explored. These findings suggest that GRANet excels not only in predicting well-established TF–gene interactions but also holds significant potential for uncovering new biological insights. Further experimental validation of these unverified targets could lead to the discovery of previously unknown regulatory networks or functional roles, deepening our understanding of transcriptional regulation and its implications in health and disease.

Figure 7.

Visualization of the gene regulatory network predicted by GRANet. This figure presents the visualization of a gene regulatory network predicted by the GRANet model, highlighting the regulatory relationships and their predicted strengths between transcription factors CBFB, EGR1, and ELF1 and their potential target genes. In the visualization, nodes represent genes, edges represent potential regulatory relationships, and the edge weights (e.g. 0.693, 0.532) indicate the predicted regulatory strengths.

Discussion and conclusion

This study introduces GRANet, a GRN inference method based on GNNs and residual attention mechanisms. By integrating multiple feature sets and incorporating a graph residual multi-head attention mechanism, GRANet effectively addresses the performance limitations of traditional methods when handling complex biological data. The model extracts and integrates standardized, smoothed, and discretized gene expression features to capture global expression patterns, mitigate noise, and highlight regulatory transitions. Extensive experimental evaluations demonstrate that GRANet outperforms current state-of-the-art GRN inference models across multiple benchmark datasets.

However, despite its strong performance in most scenarios, GRANet has some limitations. For instance, its performance degrades when processing extremely sparse datasets, particularly when the number of gene samples is insufficient [48].

Although GRANet has achieved impressive results in gene regulatory network inference, its full potential remains largely untapped. Future research could explore integrating additional types of biological data, such as proteomics and metabolomics, to construct more comprehensive multi-omics GRNs [49]. Additionally, incorporating self-supervised learning methods could reduce reliance on large amounts of labeled data, further enhancing the model’s generalization capabilities [50].

In summary, the application of artificial intelligence in single-cell biology [52–60] has substantially advanced the inference of GRNs. GRANet presents a powerful tool for this purpose, offering innovative solutions that open new avenues for research in bioinformatics [51]. While further optimization is needed, the model’s breakthroughs across multiple aspects provide fresh insights into the study of GRNs. In future work, we aim to enhance GRN inference by adopting a multi-omics approach, integrating data from various omics layers to better capture gene regulatory interactions. This multi-omics integration will not only improve the accuracy of GRN analysis but also enhance its interpretability.

Key Points

• We propose a comprehensive multi-feature extraction strategy specifically designed for single-cell RNA-seq data.

• We introduce GRANet, an advanced deep learning framework that integrates multi-head graph attention mechanisms, convolutional layers, and residual connections.

• To address the challenges posed by noisy data and limited sample sizes, GRANet incorporates internal residual connections that stabilize training and improve generalization, thereby reducing the risk of overfitting.

• We systematically assess GRANet’s performance across multiple benchmark datasets. Additionally, we conduct ablation studies to quantify the individual contributions of each architectural component, demonstrating the framework’s modular effectiveness.

• In a focused case study involving key transcription factors such as EGR1, CBFB, and ELF1, GRANet not only accurately recapitulates known regulatory interactions but also predicts previously unreported mechanisms, offering new biological insights.

Author contributions

H.W. and J.Z. (Conceived the experiments), H.W., J.Z., and N.G. (Conducted and analyzed the experiments), H.W. and J.Z. (Wrote the manuscript), and H.W., Y.H., H.Y. and G.W. (Reviewed the manuscript).

Funding

This work is supported by the National Natural Science Foundation of China (Grant No. 62272278 & 61972322), the National Key Research and Development Program (Grant No. 2021YFF0704103), and Guangdong Basic and Applied Basic Research Foundation (Grant No. 2024A1515012775). The funders did not play any role in the design of the study, the collection, analysis, and interpretation of data, or the writing of the manuscript.

Data availability

The datasets utilized in this study, along with comprehensive details of the GRANet code, are available for access at https://github.com/HaoWuLab-Bioinformatics/GRANet.