Cell Decoder: decoding cell identity with multi-scale explainable deep learning

Abstract

Background

Cells are the fundamental units of life, and understanding their diversity and functionality requires detailed characterization. The rise of single-cell omics data enables this, yet current deep learning approaches lack multi-scale interpretability.

Results

We introduce Cell Decoder, a model that integrates biological prior knowledge to provide a multi-scale representation of cells. Using automated machine learning and post hoc analysis, Cell Decoder decodes cell identity and outperforms existing methods. It offers multi-view interpretability and facilitates data integration.

Conclusions

Applied to human bone and mouse embryonic data, Cell Decoder reveals the multi-scale heterogeneity of cell identities, providing a powerful framework for advancing our understanding of cellular diversity.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13059-025-03832-y.

Background

Cells are the basic structural and functional units of life [1]. The complex functions of different tissues and organs are rooted in cellular composition, and studying the organization and function of various cells can help understand how organisms achieve normal life functions. Cell types have been defined according to their structures and functions for centuries. Usually, the same kind of cells exhibit similar characteristics and functions. Classifying and annotating cells can significantly aid in understanding their organization and functions [2]. The increasing application of single-cell transcriptomic technologies in biological research has greatly advanced the study of cell types [3–6]. However, the varying properties exhibited in different cell types at multiple scales present substantial challenges for precise cell-type definition and annotation.

The identification of cell types in single-cell transcriptomics data is usually reliant on a multi-step process. It includes the preprocessing of transcriptomics profiles, dimensionality reduction, and unsupervised clustering. Subsequently, category annotation is conducted based on manually curated differentially expressed marker genes [7, 8]. Cell identification based on the traditional approach is time-consuming and laborious, and the selection of marker genes heavily relies on researchers’ domain knowledge, which is empirical and easily biased. As the accumulation of annotated single-cell transcriptomics data provides a large number of reference datasets for cell type identification, some representative deep learning models, including fully connected neural networks, autoencoders, and transformers, are applied for mapping and migrating from reference datasets to new datasets for cell-type identification [9–12]. While the aforementioned methods have achieved commendable performances across different datasets, their “black box” nature renders them largely unexplainable. The essence of the model learning process and human reasoning is significantly different, making it difficult to understand how deep learning models learn [13] from single-cell data. However, for biological research, the transparency of the model is as important as its accuracy. A clear understanding of a model’s workings is indispensable for interpreting the biological significance of its findings.

A significant trend in machine learning is the development of explainable deep learning methods (XAI) [14]. For instance, the incorporation of domain knowledge into the model for drug response prediction, tumor typing, and biomarker discovery [15–19]. However, these methods do not fully leverage biological domain knowledge, particularly the interactions between proteins and the interdependencies among biological pathways. Therefore, constructing a multi-scale interpretable model for cell-type identification remains a challenge.

Here, we propose an interpretable deep learning model called Cell Decoder, which embeds multi-scale biological knowledge into the graph neural network, enabling the decoding of distinct cell identity features. Cell Decoder constructs a hierarchical graph structure based on the interactions between genes, the mapping relationships between genes and pathways, and the hierarchical pathway information. Through the application of automated machine learning techniques, the model’s representation power is enhanced, facilitating precise and robust cell-type identification and multi-scale data integration. Moreover, we have developed a multi-view attribution interpretation method, elucidating the model’s learning and decision-making processes and mapping them to biological explanations. Cell Decoder facilitates the understanding of the interactions, pathways, and biological processes that distinguish different cell types, providing significant implications for deeper exploration of cell identity and function.

Results

Overview of Cell Decoder

Traditional deep neural networks exhibit heightened capacity at the cost of reduced model interpretability. Here, we aim to design a model that can maintain robust representational power while offering biologically interpretable insights for identifying cell identity features. Cell Decoder designs an explainable graph neural network to model the multi-scale biological interactions and gene expressions for cell-type identification.

First, Cell Decoder leverages biological domain knowledge from curated databases and gene expressions as inputs. This biological domain knowledge includes protein–protein interaction (PPI) networks, gene-pathway maps, and pathway-hierarchy relationships (Fig. 1a). Cell Decoder processes these relationships to construct multi-scale interactions, including the gene–gene graph, gene-pathway graph, pathway-pathway graph, pathway-biology process (BP) graph, and BP-BP graph (Fig. 1b), represented as graph structures and fed into the model as inputs. Concurrently, gene expressions are used as features for each node.

Fig. 1.

An overview of Cell Decoder. a Cell Decoder explicitly leverages biological domain knowledge in cell-type identification, which consists of protein–protein-interaction networks, gene-pathway maps, and pathway-hierarchy relationships. These networks are represented as graphs (adjacency matrices) that serve as inputs to Cell Decoder. b In addition, Cell Decoder incorporates gene expressions as features for each cell. Cell Decoder employs intra-scale and inter-scale messaging passing layers to integrate information across different scales, thereby obtaining the representations of genes, pathways, and biological processes. The specific layers, as well as other hyper-parameters and architecture modifications, are automatically designed using AutoML techniques to adapt to various cell-type identification scenarios. GAT: Graph Attention Networks; GIN: Graph Isomorphism Networks. c The output of Cell Decoder can be applied for cell annotation, reference mapping and incorporates post hoc interpretability modules to provide explainable multi-view cell identity. Grad-CAM: Gradient-weighted Class Activation Mapping

To integrate information within the same resolution or across different scales effectively, Cell Decoder designs intra-scale and inter-scale message passing layers respectively (Fig. 1b). The former shares messages within homogeneous biological entities such as different genes, pathways, or BPs while the latter aggregates information from a fine-grained resolution to a coarse-grained one, i.e., from genes to pathways or from pathways to BPs. Then, Cell Decoder utilizes mean pooling to summarize the node representations of the BPs in the last graph layer into cell representations and adopts a multi-layer perceptron classifier for cell-type identification. Cell Decoder is trained end-to-end by minimizing the cross-entropy loss between predicted and ground-truth cell labels.

To adapt to various intricate cell-type identification scenarios, Cell Decoder utilizes an automated machine learning (AutoML) module to search the model design automatically, encompassing the choices of intra-scale and inter-scale layers, hyper-parameters, and architecture modifications. The searched Cell Decoder instantiation is specifically tailored to fit the targeted cell-type identification scenario, consequently leading to improved results.

Lastly, to provide model interpretability and gain insights into the identified cell types, Cell Decoder incorporates post hoc interpretability modules (Fig. 1c). Through hierarchical Gradient-weighted Class Activation Mapping (Grad-CAM) [20] analysis of the fitted model, a diverse set of biological features can be identified, including pathways and biological processes crucial for predicting different cell types. This provides a multi-view biological characterization that enhances our ability to decode cell identity.

Cell Decoder achieves superior performances and robustness for cell-type identification

We benchmarked Cell Decoder on seven different datasets against nine popular cell identification methods [9, 11, 12, 29–34]. The evaluation metrics are prediction accuracy and Macro F1 score (the average of F1 scores computed per cell type). Given that certain cell types can be rare in single-cell datasets, Macro F1 provides a more balanced measure of each method’s ability to recognize diverse cell types than accuracy alone.

To ensure fair comparisons, we also optimized hyperparameters for each baseline method, employing grid searches for learning rate and batch size in deep learning–based methods (e.g., ACTINN, TOSICA) and adjusting relevant tuning parameters for probabilistic/statistical methods (e.g., CHETAH, scmap, scPred, SingleCellNet, SingleR, Seurat). Despite these optimizations, Cell Decoder continued to outperform all baselines. On average, Cell Decoder achieved the highest accuracy (0.87) compared with the second-best method (SingleR at 0.84), as well as the highest Macro F1 score (0.81), followed by Seurat v5 at 0.79. In most datasets, Cell Decoder maintained the top performance in terms of Macro F1 (Fig. 2a) and accuracy (Additional file 1: Fig. S1 and Additional file 2: Table S1), demonstrating that the model’s advantages are not solely attributable to hyperparameter tuning but are instead driven by its multi-scale biological integration and graph-based architecture.

Fig. 2.

Systematic benchmarks and robustness evaluation of Cell Decoder for cell identification. a Macro F1 score for different cell identification methods on human and mouse datasets. Columns are sorted by the mean Macro F1 of each method on all datasets in an increasing order (top). b Robustness evaluation (Macro F1) for cell identification based deep learning methods at 10 different feature perturbation rates (0, 0.01, 0.05, 0.1, 0.5, 0.6, 0.7, 0.8, 0.9, and 1). The evaluation was repeated three times with different random seeds for feature noise (n = 3). The x-axis represents the weight of the injected noise ($\mathrm{\lambda }$), which controls the level of feature perturbation. c Imbalanced epithelial cell type ratio in mouse lung. d Confusion matrix of different cell identification methods for predicting imbalanced epithelial cell type (c). Values are normalized within each row, and values greater than 0.5 are noted. e Data shifted between reference and query in human liver dataset. f Recall score of each cell type (indicated in red if its proportion in the query is greater than that in the reference), and the average recall scores for each method are labeled on the upper right corner

Considering the inherent noise in single-cell datasets, the feasibility of model transfer has been limited. To gauge the capacity of fitted model transferring across diverse datasets, we conducted feature perturbation experiments (Methods). These experiments introduced random noise with varying rates of perturbations into the test data. As the level of data perturbation increases, all models exhibit a certain degree of decline in prediction performance. Compared to other models with transfer capabilities, Cell Decoder has demonstrated remarkable improvements in robustness across all 7 datasets (Fig. 2b and Additional file 1: Fig. S2). This indicates that Cell Decoder can recognize the efficient identity of different cell types and has strong resistance to data noise.

For cell identification, dealing with imbalanced cell-type proportions within datasets and distribution shifts between reference and query datasets are common challenges. Therefore, evaluating the performance of models under the two scenarios is significantly important. The MU_Lung dataset exhibits an imbalanced distribution of epithelial cell types, with AT2 cells comprising 82% of the reference data, while AT1 cells, Ciliated cells, and Club cells make up 8%, 8%, and 2% respectively, as shown in Fig. 2c. Cell Decoder outperforms other deep learning models in predicting accuracy for the four imbalanced cell types (Fig. 2d). In scenarios with imbalanced cell types of Endothelial cells, Immune cells, and Mesenchymal cells in MU_Lung, Cell Decoder also achieves the highest prediction accuracy for the minority cell types (Additional file 1: Fig. S1b-g and Additional file 1: Fig. S3) and best sensitivity compared with other deep-learning-based methods (Additional file 2: Table S2). In the HU_Liver dataset, a clear data shift is observed, with the proportions of cell types in the reference and query datasets showing opposite trends (Fig. 2e). Specifically, cell types such as Hepatocyte, Plasma, Mono/Macro, and Portal Endothelial (highlighted in red in Fig. 2f) have a higher proportion in the query dataset compared to the reference dataset. Cell Decoder achieves a recall of 0.88 on the query dataset, marking a 14.3% improvement over the second-best methods, ACTINN and scANVI, both of which attain a recall of 0.77. In contrast, Cell BLAST and TOSICA show recalls of 0.69 and 0.68, respectively. Additionally, we evaluated the deep learning models based on precision and Macro F1 scores. Cell Decoder shows the superior performance with a precision of 0.86 and a Macro F1 score of 0.85. (Additional file 1: Fig. S1h-i and Additional file 1: Fig. S4).

We investigated the sparsity of the hierarchical graph constructed with prior knowledge in Cell Decoder (Additional file 2: Table S3 and S4). We then conducted ablation experiments on the model by randomly removing biological prior knowledge (nodes and edges in the graph) and testing the retrained model (Methods). When the perturbation rate is 100%, it implies that the edges of the model are fully removed, with the exception of self-loop. As the rate of graph node perturbation increases, model performance decreases in almost all datasets, except HU_Pancreas and MU_Lung, where it remains relatively stable (Additional file 1: Fig. S5a). When randomly perturbing the edges of the model, there is also a noticeable decrease in prediction performance. In the case of the MU_Lung dataset, the Macro F1 score and accuracy slightly decrease. For both HU_Blood and HU_Bone, the Macro F1 score and accuracy initially decline but eventually exhibit a modest recovery (Additional file 1: Fig. S5b). We also replaced Cell Decoder’s architecture with fully connected (dense) layers, resulting in a baseline model Cell Decoder (Dense), and conducted benchmark experiments. The results are shown in Additional file 2: Table S5. This suggests that biologically informed modelling improves deep learning performance of predicting cell types.

Cell Decoder enables multi-scale data integration

In-depth cell identification typically involves integrating data from multiple sources, which often introduces batch effects due to variations in experimental protocols. By incorporating multi-scale biological prior knowledge into Cell Decoder, we obtain low-dimensional embeddings of cells at both pathway (PW) and biological process (BP) layers. We evaluated Cell Decoder—in both label-based and label-free modes—on the human immune cell dataset [21–25] with 10 different batches, comparing it with with Harmony (label-based) [26], Scanorama (label-based) [27], scVI (label-based) [28], and Monae (label-free) [29]. As shown in Fig. 3a, b and Additional file 1: Fig. S6, Cell Decoder effectively removes batch effects between individuals and platforms while preserving biological variation.

Fig. 3.

Multi-scale biologically informed data integration. a, b UMAP visualization of the multi-scale integrated cell embeddings by cell identity (a) and batch annotation (b). Cell Decoder (BP) + Harmony: Embeddings from Cell Decoder, which leverages biological process (BP) layer, combined with the Harmony method that uses batch labels. Cell Decoder (PW) + Harmony: Embeddings from Cell Decoder, which leverages pathway (PW) layer, combined with Harmony using batch labels. Cell Decoder (PW) and Cell Decoder (BP): Label-free methods that integrate data without requiring batch labels, using either pathway (PW) or biological process (BP) layer embedding. c Comparing the data integration performance of various methods, including Cell Decoder (CD) variants, label-based approaches, and the label-free method Monae. The table reports bio-conservation metrics (e.g., ARI, Silhouette) alongside batch correction metrics (e.g., kBET, PCR). The aggregation scores are calculated using the scib default method. CD: Cell Decoder. scVI, Harmony and Scanorama with batch labels

We assessed its performance using 10 metrics from scIB [30] that cover both batch correction and biological variation preservation. When batch labels are available, Cell Decoder (BP) + Harmony achieves the best batch correction performance, followed by scVI and Cell Decoder (Pathway) + Harmony. In terms of biological variation preservation, Harmony performs the best, with Cell Decoder (Pathway) and Cell Decoder (BP) close behind. Considering all metrics, Cell Decoder (BP) + Harmony ranks highest overall, followed by Cell Decoder (Pathway) + Harmony and then scVI. Meanwhile, under label-free conditions, Cell Decoder still demonstrates strong integration capabilities (Fig. 3c).

Discovery of novel cell types using Cell Decoder

One significant challenge in cell identification tasks is discovering cell types within the query dataset that are absent from the reference dataset. Most existing methods tend to categorize novel classes by forcefully aligning them with the closest known class. Such approaches are not able to discover novel cell types or cell states in the query dataset. Benefiting from biologically informed modelling, Cell Decoder possesses significant potential for capturing subtle differences between novel and known cell types. By predicting the probabilities of different cell types, it can automatically uncover potential new cell types. Moreover, it can decode the identity features of new cell types based on prior knowledge and post hoc analysis (Methods).

To verify Cell Decoder’s ability to discover novel cell types, we masked “Mono/Macro cells” in the training dataset HU_Kidney, while this cell type exists in the test set, thus simulating the scenario of encountering new cell types in the query dataset (Fig. 4a). Cell Decoder provides the predicted probabilities for each cell. If the highest probability falls below a threshold (0.95), the cell is classified as “Novel,” suggesting it is likely a new cell type not included in the training set (Fig. 4a, right). Despite their high predictive accuracies, the methods such as SingleR and ACTINN (Fig. 2a) are unable to automatically identify newly emerged cell types in the query dataset. Instead, they forcibly categorize them as existing cell types in the reference. TOSICA (cutoff = 0.95) and Cell BLAST (P < = 0.05) also identify novel cells in the query dataset by predicting the probabilities of different cells. On the masked Mono/Macro cell type, Cell BLAST achieves a recall of 0.20, while the remaining Mono/Macro cells are predicted as CD4T, CD8T cells, with a small portion being labeled as ambiguous (Fig. 4b middle). TOSICA correctly labels 37% of Mono/Macro cells but predicts a larger portion of cells as B cells (Fig. 4b right). In contrast, Cell Decoder achieves a recall of 0.94 for Mono/Macro cells, correctly identifying the vast majority of Mono/Macro cells in the query dataset. This represents a significant improvement compared to the other two methods (Fig. 4b left).

Fig. 4.

Identification of novel cell type and refinement of cell annotations. a, b Performance of different methods on the human liver for predicting novel cell type by masking Mono/Macro cell during model training (a). The recall of prediction novel cell type (b). Sankey plots comparing predictions on known and novel cell types across different methods (Left: original labels. Right: predicted labels). c UMAP visualization illustrating the re-classification of originally annotated HSPC cells as erythrocytes (HSPC_Er) by Cell Decoder. Er_Er denotes cells originally and predictively labeled as erythrocytes, and HSPC_HSPC denotes those originally and predictively labeled as HSPC. d Hierarchical clustering of differentially expressed genes (DEGs) among HSPC_HSPC, HSPC_Er, and Er_Er. e UMAP plots showing the expression levels of classical erythroid markers of HSPC_Er cells (HBB, HBA2, and HBA1)

Identifying different cell types is important, as a single cell type may occupy a range of biological states. Transcriptomic changes during these state transitions tend to be more gradual than those distinguishing distinct cell types [31]. Consequently, in low-dimensional embedding spaces, such cells often appear in a continuum rather than forming well-separated clusters. Manual annotation methods based solely on marker genes can struggle to detect these finer distinctions within a single cell type. By contrast, Cell Decoder leverages biological prior knowledge rather than relying on specific marker genes, enabling it to refine labeling and potentially correct misannotations.

In the HU_Bone dataset, a subset of cells originally annotated as Hematopoietic Stem and Progenitor Cells (HSPC) was predicted by Cell Decoder to be Erythrocytes (Fig. 4c). We labeled these cells as HSPC_Er, while those aligning with the original HSPC or Erythrocyte annotations were designated as HSPC_HSPC or Er_Er, respectively. UMAP visualization revealed that HSPC_HSPC, HSPC_Er, and Er_Er cells form a gradient, with HSPC_Er positioned in an intermediate zone between HSPC_HSPC and Er_Er (Fig. 4c) using Cell Decoder’s BP embedding. Differential expression analysis and hierarchical clustering (Fig. 4d) showed that HSPC_Er cells are more transcriptionally similar to Er_Er. In addition, established erythroid markers—including HBB, HBA2, and HBA1—showed comparable expression levels in both HSPC_Er and Er_Er cells (Fig. 4e and Additional file 1: Fig. S7). Collectively, these findings indicate that Cell Decoder effectively corrects misannotations in the dataset, suggesting that the subset initially labeled as HSPC is more accurately classified as erythroid cells.

Cell Decoder reveals cellular dynamic changes in mouse embryogenesis

Understanding the lineage relationships between cells and cell types, as well as the molecular programs governing the emergence of each cell type, constitutes a fundamental goal in developmental biology. For the mouse embryogenesis dataset [32–35], the entire embryo has been profiled using scRNA-seq. However, systematically integrating the associated data remains a challenge. This challenge is attributed to technical issues such as varying technologies and batch effects, as well as the complexity of mouse development [35]. We integrated data from three stages of mouse embryo development at E6.25, E6.5, and E6.75, obtained from different technologies. In the mouse embryogenesis dataset, there are a total of seven cell types, namely epiblast (3302 cells), extraembryonic ectoderm (1220 cells), primitive streak and adjacent ectoderm (1214 cells), extraembryonic visceral endoderm (606 cells), embryonic visceral endoderm (295 cells), nascent mesoderm (159 cells), and parietal endoderm (44 cells).

Cell Decoder successfully integrated data from different developmental stages and demonstrated its batch correction capability through UMAP visualization computed on the biological process embeddings (Fig. 5a, b). We randomly partitioned the data, allocating 80% to the training set and 20% to the validation set for model fitting. The visceral endoderm encompasses the extraembryonic visceral endoderm (ExVE) and the embryonic visceral endoderm (EmVE). We learned the differences between ExVE and EmVE (E6.5 and E6.75) in Fig. 5c across various biological processes using Cell Decoder. The notably higher Grad-CAM scores of EmVE in cell–cell communication and extracellular matrix categories, as opposed to ExVE, suggested a heightened activity in these biological processes. Furthermore, EmVE at the E6.75 stage exhibited a significant level of programmed cell death, while ExVE showed pronounced activity in the immune system (Fig. 5d). Cell Decoder learns representations of EmVE and ExVE in the BP embedding, allowing for further clustering into four subtypes, which is a more refined approach compared to subclustering using Scanpy (Fig. 5e and Additional file 1: Fig. S8). At E6.5, C0 is the predominant subtype, followed by C1 and C3, while C2 is the smallest subtype. However, at E6.75, the highest proportion is observed for C1, followed by C2 cells (Fig. 5e right). In addition, we further characterized the four ExVE/EmVE subtypes defined by Cell Decoder validated them using pathway scores (Additional file 1: Fig. S9). First, we performed differential expression analysis for each subtype and visualized the top five markers in Fig. 5f. Next, we carried out pathway enrichment on the top 50 differentially expressed genes per subtype (Additional file 1: Fig. S10-13). This revealed coherent, subtype-specific programs: C0 is enriched for lipid and nutrient handling; C1 shows strong proliferative and translational signatures; C2 highlights pathways involved in structural remodeling; and C3 is marked by neuroepithelial and cell–cell junction organization. Together, these results demonstrate that Cell Decoder not only attenuates batch effects but also robustly captures and amplifies subtle cell-state differences, underscoring its utility for uncovering multi-scale heterogeneity in single-cell data.

Fig. 5.

Decoding dynamic cellular changes during mouse embryogenesis using Cell Decoder. a, b UMAP projections of the mouse embryo dataset based on Cell Decoder embeddings, colored by developmental cell type (a) and by stage (b). c Subset highlighting Extraembryonic visceral endoderm (ExVE) and Embryonic visceral endoderm (EmVE) populations. d Median Grad-CAM scores for key biological processes across developmental stages in ExVE and EmVE, illustrating how Cell Decoder attributes pathway activity over time. e Clustering of ExVE and EmVE cells into four subtypes using Cell Decoder embeddings (left), with stacked barplots showing the relative abundance of each subtype at E6.5 versus E6.75 (right). f UMAP overlays of the top five differentially expressed genes for each subtype, with rows corresponding to subtypes 0–3, highlighting the transcriptional programs that define each population

Multi-view interpretability in Cell Decoder for cell identification

Due to the diverse properties exhibited by different cell types at multiple scales, defining, categorizing, and understanding them pose significant challenges [31]. Probabilistic statistical and traditional deep learning-based cell identification methods can classify different cell types. However, most of these methods [9–12, 36] lack interpretability. While TOSICA [9] explains the mapping from gene to pathway through an attention mechanism, it falls short in decoding the identity features of a cell type. A cell type is generally considered to express an assemblage of cellular modules (protein complexes, pathways, and molecular machines constitute the structure and function of the cell) responsible for discrete subfunctions [37]. Cell Decoder leverages biological prior knowledge (protein–protein interactions, gene-pathways, biological process) to enhance the transparency of its network structure (Fig. 1a). Additionally, after training Cell Decoder, we perform a multi-view post hoc analysis (see Methods) from different perspectives: at the level of genes, pathways, and protein–protein interactions. This approach enables us to map the model’s classification decisions onto specific biological factors, thereby revealing how distinct combinations of genetic, pathway-level, and interaction-level features characterize cell types or states. For instance, in mouse embryonic tissues, it pinpoints crucial interaction pairs like Lefty1-Nodal (Additional file 1: Fig. S14). Together, these findings offer a more comprehensive biological explanation of how Cell Decoder arrives at its classifications, underscoring the value of multi-view attribution analysis in bridging computational predictions with mechanistic insights.

Discussion

We propose a novel biologically informed graph neural network architecture that integrates protein–protein interactions, gene-pathway mappings, and hierarchical pathway information into a hierarchical graph neural network for decoding cell identity. This approach aims to simulate the intracellular gene/protein interactions and aggregate information from genes to pathways, then to biological processes. We utilized automated machine learning techniques to enhance model performance and streamline the cumbersome process of deep learning parameter tuning. Cell Decoder exhibits strong transparency and interpretability. Through multi-view attribution analysis of the fitted model, we gained deeper insights into different cell types across scales and interactions.

Cell Decoder has outperformed existing advanced methods across diverse benchmarks. In human and mouse cell identification tasks, Cell Decoder achieves accurate and stable knowledge transfer, demonstrating strong resilience against data noise and data shift. By learning cell-type-specific features, Cell Decoder achieves accurate data integration at different scales (pathways embedding and biological process embedding) without batch labels. We also explored a novel data integration approach, i.e., pretraining on one batch and transferring the model to others. This method not only accomplishes cell identification but also enables cross-batch data integration. Biologically informed modelling has effectively boosted the ability to discover new cell types or states.

Despite having the above advantages in decoding cell identity, Cell Decoder may face potential limitations in selecting biological knowledge for modelling. Our model leverages domain knowledge from multiple databases [38–40] to enhance the interpretability of deep learning. However, the effectiveness of prior knowledge varies across different datasets. The results of knowledge ablation experiments on 7 datasets indicated that different tissues and organs required varying biological knowledge, demonstrating that the choice of biological knowledge modelling has a certain impact on the model’s performance. We speculate that this variation arises from differences in cellular heterogeneity and the availability of well-defined markers: in tissues with more homogeneous cell populations and clear lineage markers (e.g., pancreas), pathway priors yield greater gains, whereas in highly heterogeneous tissues the signal-to-noise ratio at the pathway level may be lower. Then, although multi-view attributions provide transparency, gradient-based methods such as GradCAM can be inherently unstable—small perturbations in input or model parameters may cause large swings in attribution scores, potentially leading to misleading interpretations. Consequently, we recommend that any insights from such analyses be corroborated by complementary, data-driven approaches (e.g., clustering metrics, differential expression, pathway enrichment).

Meanwhile, with the development of single-cell multi-omics [41] technologies that can capture protein expression levels within individual cells, Cell Decoder can naturally extend to multimodal datasets, thereby gaining more insights into cell-type regulation. Due to the incorporation of gene/protein interaction information, Cell Decoder opens up the possibility of predicting cell responses to genetic perturbations [42]. We believe this can bring new interpretability and biological insights to define and understand the identity features of different cell types.

Conclusions

In this paper, we introduce Cell Decoder, a biologically informed hierarchical graph neural network that integrates protein–protein interactions, gene-pathway mappings, and hierarchical pathway information to decode cell identities. By simulating intracellular gene/protein interactions and aggregating information from genes to pathways, then to biological processes, Cell Decoder provides a powerful framework for deciphering complex biological systems. The inclusion of automated machine learning significantly enhances model performance and alleviates the burden of deep learning parameter tuning, while multi-view attribution analysis offers valuable insights across scales and interactions. Our evaluations demonstrate that Cell Decoder consistently outperforms advanced methods, showing robustness to data noise and shifts across diverse datasets. This biologically informed modelling approach also supports the discovery of novel cell types, highlighting its potential for broad applications in complex biological contexts.

Methods

The architecture of cell decoder

We integrated the STRING v11.5 (https://string-db.org), MSigDB (https://www.gsea-msigdb.org/gsea/msigdb), and Reactome (https://reactome.org) databases to construct a biologically informed multi-scale hierarchical graph $G=\left(V,,,E\right)$. This graph integrates genes, pathways, and biological processes into a unified framework. The node set $V$ is composed of three distinct subsets: $V^{\mathit{gn}}$ (gene nodes), $V^{\mathit{pw}}$ (pathway nodes), and $V^{\mathit{bp}}$ (biological process nodes). The edge set $E$ represents the interactions between these nodes across scales, capturing gene–gene interactions, gene-pathway relationships, and pathway-biological process relationships.

To define the graph at the gene level, we used high-confidence protein–protein interactions (PPI) from STRING (score ≥ 850) as proxies for gene–gene interactions. The adjacency matrix $A^{\mathit{gn}}\in \{\text{0,1}{\}}^{n\times n}$ encodes these interactions, where $A_{i,j}^{\mathit{gn}}=1$ indicates that the protein corresponding to the $i$ th gene interacts with the protein corresponding to the $j$ th gene, and $A_{i,j}^{\mathit{gn}}=0$ otherwise.

For pathway and biological process levels, we used Reactome and MSigDB databases to map relationships. The gene-pathway relationships are represented by the adjacency matrix $A^{gn\to pw}$, where $A_{i,j}^{gn\to pw}=1$ indicates that the $i$ th gene is associated with the $j$ th pathway. Similarly, the pathway-biological process relationship is taken into account and denoted as$A^{pw\to bp}$. Then, Cell Decoder constructs the pathway adjacency matrix by utilizing the multi-scale biological knowledge:

$$A^{\mathit{pw}}=\text{min}\left(A^{pw\to gn},\times ,A^{\mathit{gn}},\times ,A^{gn\to pw},,,1\right),$$

where the operator $\times$ is the matrix multiplication, and $A^{pw\to gn}$ is the transpose of $A^{gn\to pw}$. The biological process adjacency matrix is similarly constructed, i.e.,

$$A^{\mathit{bp}}=\text{min}\left(A^{bp\to pw},\times ,A^{\mathit{pw}},\times ,A^{pw\to bp},,,1\right).$$

The resulting multi-scale graph $G=\left(V,,,E\right)$ captures the complex relationships across genes, pathways, and biological processes, providing a comprehensive representation of biological systems for Cell Decoder (Additional file 1: Fig. S15).

Multi-scale graph

Cell Decoder captures the interactions among genes, pathways, and biological processes using a multi-scale graph. The gene expressions and interactions between genes are represented as an undirected graph with an adjacency matrix ${\mathbf{A}}^{\text{gn}}\in {\left\{0,,,1\right\}}^{n\times n}$ and a feature matrix $\mathbf{X}\in {\mathbb{R}}^{n\times f}$, where $n$ denotes the number of genes and ${\mathbf{A}}_{i,j}^{\text{gn}}$ indicates whether the $i$ th gene interacts with the $j$ th gene. Cell Decoder also considers the gene-pathway relationship ${\mathbf{A}}^{\text{gn}\to \text{pw}}$, where ${\mathbf{A}}_{i,j}^{\text{gn}\to \text{pw}}$ indicates whether the $i$ th gene belongs to the $j$ th pathway. Similarly, the pathway-biological process relationship is taken into account and denoted as ${\mathbf{A}}^{\text{pw}\to \text{bp}}$. Then, Cell Decoder constructs the pathway adjacency matrix by utilizing the multi-scale biological knowledge:

$${\mathbf{A}}^{\text{pw}}=\text{min}\left({\mathbf{A}}^{\text{pw}\to \text{gn}},\times ,{\mathbf{A}}^{\text{gn}},\times ,{\mathbf{A}}^{\text{gn}\to \text{pw}},,,1\right),$$

where the operator $\times$ is the matrix multiplication, and ${\mathbf{A}}^{\text{pw}\to \text{gn}}$ is the transpose of ${\mathbf{A}}^{\text{gn}\to \text{pw}}$. The biological process adjacency matrix is similarly constructed, i.e.,

$${\mathbf{A}}^{\text{bp}}=\text{min}\left({\mathbf{A}}^{\text{bp}\to \text{pw}},\times ,{\mathbf{A}}^{\text{pw}},\times ,{\mathbf{A}}^{\text{pw}\to \text{bp}},,,1\right).$$

Multi-scale message passing

The objective of the multi-scale message passing layers in Cell Decoder is to effectively aggregate information from genes, pathways, and biological processes. To incorporate multi-scale information, Cell Decoder consists of two types of GNN layers: intra-scale layers and inter-scale layers. For intra-scale layers, Cell Decoder aggregates messages from the neighborhoods within each scale and subsequently updates the node representations as follows:

$$H^{\mathit{gn}}\leftarrow GN{N}_{\mathit{Intra}}\left(H^{\mathit{gn}},,,A^{\mathit{gn}}\right),H^{\mathit{pw}}\leftarrow GN{N}_{\mathit{Intra}}\left(H^{\mathit{pw}},,,A^{\mathit{pw}}\right),H^{\mathit{bp}}\leftarrow GN{N}_{\mathit{Intra}}\left(H^{\mathit{bp}},,,A^{\mathit{bp}}\right),$$

where $\mathbf{H}\in {\mathbb{R}}^{n\times d}$ is the node representation, and $d$ denotes the dimensionality. For inter-scale layers, Cell Decoder updates the node representations by aggregating the neighborhood messages from preceding scales, i.e.,

$${\mathbf{H}}^{\text{pw}}\leftarrow {\text{GNN}}_{\text{Inter}}\left({\mathbf{H}}^{\text{gn}},,,{\mathbf{A}}^{\text{gn}\to \text{pw}}\right),{\mathbf{H}}^{\text{bp}}\leftarrow {\text{GNN}}_{\text{Inter}}\left({\mathbf{H}}^{\text{pw}},,,{\mathbf{A}}^{\text{pw}\to \text{bp}}\right).$$

The execution of intra-scale layers and inter-scale layers follows an alternating pattern, starting from genes, then pathways, and finally biological processes. Finally, Cell Decoder utilizes mean pooling to summarize the node representations into cell representations, which are subsequently employed for cell-type identification, i.e.,

$$\text{g}\leftarrow \text{MEAN}\left({\mathbf{H}}^{\text{bp}}\right),\widehat{\mathit{y}}\leftarrow \text{MLP}\left(\text{g}\right),$$

where $\text{g}\in {\mathbb{R}}^{1\times d}$ represents the cell representation, MLP denotes a multi-layer perceptron classifier, $\text{MEAN}\left(·\right)$ is the mean pooling operator, $\widehat{\mathit{y}}\in {\mathbb{R}}^{C\times 1}$ represents the predicted probability distribution of the cell types, and $C$ is the number of the cell types. To learn the model parameters, Cell Decoder adopts the cross-entropy loss function:

$$\ell \left(\mathit{y},,,\widehat{\mathit{y}}\right)=-\sum _{c=1}^C{\mathit{y}}_c\text{log}{\widehat{\mathit{y}}}_c+\left(1,-,{\mathit{y}}_c\right)\text{log}\left(1,-,{\widehat{\mathit{y}}}_c\right),$$

where $\mathit{y}$ denotes the ground-truth cell types.

Automated model design

Given that biological mechanisms in various cells could show intricate patterns, it becomes crucial to identify optimal hyper-parameters and GNN layers for improved cell-type identification. In this regard, we employ AutoML techniques to automate the process of architecture design and hyper-parameter selection. To maintain simplicity, we adopt a classical grid search method [43], which systematically enumerates and selects the best architectures and hyper-parameters from the entire set of possibilities. The search space of Cell Decoder encompasses intra-scale layers, inter-scale layers, hyper-parameters, and other architectural modifications, defining the feasible ranges for exploration and selection.

Intra-scale layers

Our search space includes two types of intra-scale layers: Graph Attention Networks (GAT) [44] and Graph Isomorphism Networks (GIN) [45].

GAT employs an attention mechanism for neighborhood aggregation, and its formulation is as follows:

$$\mathbf{H}\leftarrow \sigma \left(\alpha ,\mathbf{H},{\mathbf{W}}^{⊺}\right),{\alpha }_{i,j}=\frac{{\mathbf{A}}_{i,j}\text{exp}\left(\sigma ,\left({\mathit{a}}^{⊺},\left[\mathbf{W},{\mathbf{h}}_i,||,\mathbf{W},{\mathbf{h}}_j\right]\right)\right)}{{\sum }_{k=1}{\mathbf{A}}_{i,k}\text{exp}\left(\sigma ,\left({\mathit{a}}^{⊺},\left[\mathbf{W},{\mathbf{h}}_i,||,\mathbf{W},{\mathbf{h}}_k\right]\right)\right)},$$

where $\mathbf{W}$ represents a learnable parameter matrix used to transform the node representations, $\mathit{a}$ is a learnable vector for calculating the attention weights $\alpha$ for each edge, ${\left(·\right)}^{⊺}$ denotes the transpose operation, ${\mathbf{h}}_i$ denotes the feature vector of the $i$ -th node, $||$ denotes the concatenation operation, and $\sigma$ represents the LeakyReLU activation function. Multi-head attention mechanism is also utilized to stabilize the training process and improve performance.

On the other hand, GIN, which adopts a sum aggregation, can be formulated as follows:

$$\mathbf{H}\leftarrow \text{MLP}\left(\left(\mathbf{A},+,\left(1,+,ϵ\right),·,\mathbf{I}\right),\mathbf{H}\right),$$

where $ϵ$ is a learnable parameter, $\mathbf{I}$ denotes the identity matrix, and MLP represents a multi-layer perceptron with learnable parameters.

Inter-scale layers

For inter-scale layers, our search space contains two operations: mean pooling [46] and attention pooling [47].

The mean pooling computes the average of the neighbor representations from the previous layers to obtain the node representation in the current layer. For instance, to obtain the representation of the ith pathway in gene-pathway message passing, the model aggregates all of its related gene representations and calculates the average as follows:

$${\mathbf{h}}_i^{\text{pw}}=\text{MEAN}\left(\left\{{\mathbf{h}}_j^{\text{gn}},:,{\mathbf{A}}_{i,j}^{\text{gn}\to \text{pw}},=,1\right\}\right).$$

In the case of attention pooling, the calculation is similar to Eq. (7), with the difference that we first employ mean pooling to obtain the node representation used in calculating attention, and then utilize the inter-scale adjacency matrix to calculate and aggregate the messages. For example, the hidden representation for the pathways is calculated as:

$${\mathbf{h}}_i^{\text{pw}}\leftarrow \sum _j{\alpha }_{i,j}\mathbf{W}{\mathbf{h}}_j^{\text{gn}},{\alpha }_{i,j}=\frac{{\mathbf{A}}_{j,i}^{\text{gn}\to \text{pw}}\text{exp}\left(\sigma ,\left({\mathit{a}}^{⊺},\left[\mathbf{W},{\mathbf{h}}_j^{\text{gn}},||,\mathbf{W},{\mathbf{h}}_{i,\text{mean}}^{\text{pw}}\right]\right)\right)}{{\sum }_{k=1}{\mathbf{A}}_{k,i}^{\text{gn}\to \text{pw}}\text{exp}\left(\sigma ,\left({\mathit{a}}^{⊺},\left[\mathbf{W},{\mathbf{h}}_k^{\text{gn}},||,\mathbf{W},{\mathbf{h}}_{i,\text{mean}}^{\text{pw}}\right]\right)\right)},$$

where ${\mathbf{h}}_{i,\text{mean}}^{\text{pw}}$ is calculated by the mean pooling.

Hyper-parameters

Additionally, our search space encompasses commonly used hyper-parameters, including the dimensionality of the node representation, the learning rate, and the number of samples per class utilized for training. If there exist some classes that have fewer or more training samples than $K$, we conduct over-sampling or down-sampling techniques to obtain exact $K$ training samples for each class.

Architecture modification

Lastly, our search space includes two architecture modification techniques to facilitate more flexible architecture design: one-hot encoding of genes and jumping connections.

For the one-hot encoding, we introduce a learnable embedding matrix for each gene node, which is concatenated with the node features as follows:

$$\mathbf{X}\leftarrow \left[\mathbf{X},,,{\mathbf{X}}_{\text{para}}\right],$$

where ${\mathbf{X}}_{\text{para}}$ denotes the trainable embedding matrix. This enables the model to effectively capture specific information pertaining to each gene, enhancing its ability to learn gene-specific characteristics. Regarding the jumping connections, we allow the model to concatenate the raw node features with the final node representation learned by the GNN layers. The concatenated features are then fed into the classifier as follows:

$$\widehat{\mathit{y}}\leftarrow \text{MLP}\left(\left[\mathbf{g},,,{\mathbf{g}}^t,_{}^{\prime }\right]\right).$$

where $\mathbf{g}_{}^{_{}^{\prime }}$ is obtained by flattening the raw features $\mathbf{X}$ for each graph into a vector. This strategy draws inspiration from the widely employed residual connections in deep learning [48] and the jumping knowledge network in GNNs [49]. By incorporating jumping connections, we aim to improve the expressiveness of the model and enhance training stability.

Multi-view model interpretability

In order to interpret the model, we employ a post hoc analysis to assess the importance of features.

Feature importance

We utilize two methods to measure the feature importance: gradient-based methods (referred to as Grad) and Gradient-weighted Class Activation Mapping (GradCAM) [20]. For Grad, we calculate the L2 norm of the feature gradients with respect to the loss, which quantifies the model’s sensitivity to the features. GradCAM extends this analysis by incorporating the feature maps prior to graph convolutions, considering the influence of features on the classifiers.

Robustness evaluation

We evaluate the robustness of our model through three perturbation strategies, which involve perturbation on features, nodes, and structures. We focus on the robustness of model [50], where all perturbation occur during test time by manipulating the input data.

Feature perturbation

To perturb a feature $x$, we denote its mean and standard deviation as $\mu$ and $\sigma$, respectively. The feature is modified as $x\leftarrow \text{max}\left(x,+,\lambda ,·,ϵ,,,0\right)$, where the noise $ϵ$ is sampled from a Gaussian distribution $\mathcal{N}\left(\mu ,,,\sigma \right)$, and $\lambda \in \mathbb{R}$ is a hyper-parameter that controls the perturbation rate.

Node perturbation

In this perturbation, we randomly remove nodes and denote $p\in \left[\text{0,100}\right]$ as a hyper-parameter to control the perturbation rate. We randomly select $p\text{\%}$ of the gene nodes and remove all their edges from the graph, except for the self-loops.

Edge perturbation

In this perturbation, we randomly remove edges and denote $p\in \left[\text{0,100}\right]$ as a hyper-parameter to control the perturbation rate. We randomly select $p\text{\%}$ of the edges from the adjacency matrices ${\mathbf{A}}^{\text{gn}},{\mathbf{A}}^{\text{gn}\to \text{pw}},{\mathbf{A}}^{\text{pw}\to \text{bp}}$, and remove these edges from the graph.

Datasets and preprocessing

For all datasets, we applied a unified preprocessing workflow. First, we removed genes expressed in fewer than three cells and excluded any low-quality cells or unknown cell types, along with cell types represented by fewer than 1000 cells. We then perform log-scale normalization of the filtered datasets. Following normalization, we identified the top 3000 highly variable genes using Scanpy’s default HVG detection method for consistency across both human and mouse datasets. This pipeline ensures uniform filtering and normalization procedures, facilitating reproducibility in our downstream analyses.

Baselines for cell identification and data integration

All baseline methods for cell-type identification were trained on the same training set, generally following their recommended parameters. However, to ensure optimal performance, we performed a grid search on key hyperparameters (e.g., learning rate and batch size). In the data integration task, we employed methods implemented in scIB. The integrated data were then evaluated using scIB’s suite of metrics to compare performance across different integration strategies. This approach ensures consistent configuration of each method, enabling a fair and transparent comparison between Cell Decoder and existing single-cell analysis workflows.

Metrics for cell identification evaluation and integration

We use accuracy and Macro F1 to evaluate the performance and robustness of models in predicting cell types. For the data integration task, we employ average silhouette width (ASW), Batch ASW, kBET, Graph Connectivity, and Graph iLISI to assess the model’s batch correction capability. Additionally, we utilize normalized mutual information (NMI), adjusted Rand index (ARI), Cell Type ASW, and Isolated Label F1 to evaluate the model’s ability to preserve biological differences using scIB. The overall score was computed as the average of all scores.

Authors’ contributions

J.Z. and Zeyang.Z. conceived the project and designed the architecture of Cell Decoder. J.Z., Zeyang.Z. and Y.X. implemented the method and performed the analyses. C.C. and Ziwei.Z. co-supervised the project. J.Z., Y.X. and B.X. performed data analysis. X.D., R.R., X.W. and Y.L. collected the public datasets. F.H. and W.Z. coordinated the study. L.X. and P.Z. helped to revise the manuscript. All authors contributed to the writing of the manuscript. All authors read and approved the final manuscript.

Funding

This work was financially supported by the National Key Research and Development Program of China (2024YFA1210400), the National Natural Science Foundation of China (32088101 and 62472018), and CAMS Innovation Fund for Medical Sciences (CIFMS, 2019-I2M-5-063).

Data availability

Datasets used for cell identification benchmarking include: HU_Blood (GSE171555), HU_Bone (GSE134355), HU_Liver (GSE136103, GSE115469), HU_Kidney (GSE145927), HU_Pancreas (GSE81608, E-MTAB-5061, GSE85241, GSE84133), MU_Pancreas (GSE132188), and MU_Lung (GSE252225). Each dataset is annotated with species and tissue information (see Additional file 2: Table S6 for details). Published datasets used in this study include human immune cell datasets from the 10X Genomics website (https://support.10xgenomics.com/single-cell-gene-expression/datasets/3.0.0/pbmc_10k_v3) and GEO (GSE120221, GSE107727, GSE115189, GSE128066, GSE94820), as well as mouse embryo datasets obtained from GEO (GSE109071, GSE100597) and ArrayExpress (E-MTAB-6967), with detailed descriptions provided in Additional file 2: Table S7. The source code, documentation, and tutorials of Cell Decoder are available at GitHub (https://github.com/PHOENIXcenter/CellDecoder) [51] under the MIT License. The version of Cell Decoder used in the manuscript is available at Zenodo (https://zenodo.org/records/15369546) [52].

Declarations

Competing interests

The authors declare no competing interests. Peijie Zhou is affiliated with the AI for Science Institute, Beijing, 100084, China.