scRSSL: Residual semi‐supervised learning with deep generative models to automatically identify cell types

Abstract

Single‐cell sequencing (scRNA‐seq) allows researchers to study cellular heterogeneity in individual cells. In single‐cell transcriptomics analysis, identifying the cell type of individual cells is a key task. At present, single‐cell datasets often face the challenges of high dimensionality, large number of samples, high sparsity and sample imbalance. The traditional methods of cell type recognition have been challenged. The authors propose a deep residual generation model based on semi‐supervised learning (scRSSL) to address these challenges. ScRSSL creatively introduces residual networks into semi‐supervised generative models. The authors take advantage of its semi‐supervised learning to solve the problem of sample imbalance. During the training of the model, the authors use a residual neural network to accomplish the inference of cell types so that local features of single‐cell data can be extracted. Because of the semi‐supervised learning approach, it can automatically and accurately predict individual cell types in datasets, even with only a small number of cell labels. Experimentally, the authors’ method has proven to have better performance compared to other methods.

(1) ScRSSL is based on semi‐supervised residual networks that use neural network methods to infer unknown labels. (2) ScRSSL uses the network connectivity of the residual network in place of the fully connected neural network structure. (3) ScRSSL utilises the feature of semi‐supervised learning to improve the problem of sample imbalance in single‐cell datasets.

1. INTRODUCTION

With the rapid development of single‐cell high‐throughput sequencing technology, we have been able to accurately sequence individual cells [1]. The traditional bulk sequencing technology sequences a collection of cells to obtain gene expression information. However, its limitation is that the sequencing sample is a collection of tissues or cells, which cannot meet the latest development in biology and precision medicine. Single‐cell sequencing has improved researchers' understanding of the unique heterogeneity of individual cells. It has facilitated the understanding and study of the physiological mechanisms and functions of single cells, especially its differentiation process and response to external disturbances [2]. As technology advances, more and more projects for large‐scale single‐cell atlases are beginning to take root, such as the Human Cell Atlas (HCA) [3] and the Human Integrated Cell Atlas (hECA) [4]. These projects make significant contributions to research in the biological and medical communities. It can help researchers obtain a wealth of information that can help reveal the heterogeneity of cells in normal and diseased tissues and help predict disease associations [5, 6]. As a result, the focus of scRNA‐seq research has also shifted from the discovery of new cells to the need to accurately identify and predict cell types [7]. Accurate identification and prediction of cell types [8, 9] are of great significance for the accurate analysis of cell properties, revealing heterogeneity between cells, in‐depth understanding of cell development and differentiation, and cell–cell interactions. In addition, the accurate identification and prediction of efficient cell types can help doctors analyse the pathogenesis of diseases at the single‐cell level and improve clinical diagnosis.

Identifying individual cell types is a critical task when analysing single‐cell transcriptomics data. In the traditional approach, it is common to use unsupervised clustering algorithms to cluster cells and then manually assign labels to them [10]. This approach is suitable for cases where the number of samples is small and the data are not particularly complex. With the development of single‐cell sequencing technology and the reduction of sequencing costs, we can obtain many large‐scale single‐cell datasets [11]. Because high‐throughput single‐cell datasets has the characteristics of particularly large sample size, high dimension, high sparsity and unbalanced sample [12], single‐cell datasets are no longer suitable for traditional manual annotation cases [13]. This makes it a challenge to obtain labels for single‐cell datasets, which in turn makes it difficult for researchers to predict cell types [14]. This challenge makes semi‐supervised learning particularly important in the study of single‐cell datasets. Currently, most semi‐supervised learning methods predict unknown datasets using existing labels and then continue to train the model on the predicted labels [15]. These methods determine decision boundaries by labelling, potentially increasing the proportion of incorrect predictions [16]. For the single‐cell automatic recognition algorithms proposed in recent years, we can broadly classify them into 2 categories. One type is to determine which type of single cell belongs to based on the similarity of the nearest neighbour samples, such as single‐cell mouse cell Atlas (scMCA) [17]. Another class is based on supervised learning algorithms, such as single‐cell hierarchical progressive learning (scHPL) [18]. scHPL is based on support vector machines [19] and has the advantage of using an iterative learning process to learn the hierarchy of single‐cell data [20]. Most algorithms predict new single‐cell labels based on reference datasets, such as Seurat is a powerful single‐cell processing tool built on the Random Forest algorithm [21]. scDHA, which uses a deep learning based layered autoencoder model for single‐cell decomposition, uses a stacked autoencoder to remove data noise and a K‐Nearest Neighbour (KNN) to classify cells [22]. Unlike them, scANVI is a semi‐supervised learning method that uses variational inference for single‐cell annotation [23]. However, its neural network is connected in a fully connected way and does not use convolutional layers to extract local features of the single‐cell transcriptome. So, its network structure needs to be further improved.

Deep learning technology has developed rapidly and has broad prospects in recent years [24]. It is also widely used in the field of bioinformatics, for example, Large Language Models [25] are integrated into single‐cell applications to infer gene networks and reveal regulatory associations between genes through single‐cell language models. Predicting the protein structure through deep learning models [26] can help researchers analyse hyperscale proteomic data. In this paper, according to the characteristics of single‐cell data, a deep residual generation model based on semi‐supervised learning is introduced to automatically recognise cell types [27]. Since semi‐supervised learning can reduce labelling bias, we use semi‐supervised learning models to effectively mitigate the sample imbalance problem [28]. Compared with the general classification model, the deep generative model based on semi‐supervised learning has the following advantages: First, it does not simply use labelled data to build a classifier. It uses the properties of the data itself, both labelled and unlabelled, to optimise the decision boundaries [16]. Secondly, it can use its ability to generate data to the imputation of data to solve the problem of zeros or dropout of data, that is, the problem of sparse data [29]. We use the encoder neural network in the variational autoencoder for non‐linear dimensionality reduction of high dimensional data. This can solve the problem of the high dimensional data while removing excess noise in the process [30]. In the inference network of single‐cell labelling, we use residual connectivity to connect neurons [31]. This allows us to extract local features of the single‐cell transcriptome through the convolutional layers in the residual module, which can also help us solve the network degradation problem. After we make the neural network layers deeper, this can enhance the ability of the model to extract features, and thus the accuracy of prediction is improved [32]. After we derive the posterior probabilities using the residual neural network, we decide whether to perform the rejection procedure for a cell based on the posterior probabilities.

Convolutional Neural Network (CNN) is a deep learning method. It has the ability of a biological vision to perceive the local features of images and is an algorithm developed specifically for image recognition [33]. The algorithm is now also used in some bioinformatics fields, such as in medical image analysis. In scRSSL, we introduce convolutional neural networks to extract local features of single‐cell data [34]. Since the negative binomial distribution is consistent with the over‐dispersed distribution characteristic of scRNA‐seq [35], in the decoder, we map the low‐dimensional representation of the data to the negative binomial distribution. We have validated it on several real single‐cell datasets, and the experiments show that our proposed model has higher accuracy compared to other models. We also verify that our model has a relatively short runtime compared to other models (Supplemental Figure S1). More importantly, the scRSSL model can handle very large datasets (Supplemental Figure S2). In summary, this article makes three main contributions: (1) Our model has the ability of semi‐supervised learning, which can learn single‐cell datasets with a small number of labels and then label larger single‐cell datasets without labels. (2) We use the network connectivity of the residual network instead of the fully connected neural network structure. This has two benefits, one is that the network can be made deeper, enhancing the model's ability to extract single‐cell features. The second is that the model can extract local detailed features of the single‐cell transcriptome, thus better optimising the classification decision boundary and making the prediction results more accurate. (3) Our model can alleviate the sample imbalance problem of single‐cell data.

2. MATERIAL AND METHODS

2.1. Datasets

Presented in Table 1 are the 10 major datasets used in the analysis of this study, which we divided into three categories. The first data is the inter‐dataset; they are all cells extracted from the human pancreas, so they have similar labels and common genes. They are from different batches and different protocols in the experiments: Wang (457 cells) [35], Xin (1600 cells) [36], Baron (Human) (8569 cells) [37], Segerstolpe (2209 cells) [38], and Muraro (2126 cells) [39]. The experimental design about them is as follows: we use one of the datasets at a time as the reference dataset and then use the other four datasets as the query dataset for prediction. It is equivalent to doing five‐fold cross‐validation, which not only verifies the robustness of the model but also its resistance to batch effects. The second data are the intra‐dataset; they are single‐cell data from the same tissue and the same experimental setting. They are Baron (Mouse) (1886 cells) [37], Klein (2717 cells) [40], Romanov (2881 cells) [41], and Zeisel (3005 cells) [42], respectively, for which we performed a five‐fold cross‐validation. The third dataset is the unbalanced single‐cell dataset hECA*, which is obtained from the large‐scale dataset hECA [4] by stratified random sampling. It contains 14,190 cells and 15,500 genes. We can observe the degree of imbalance in this dataset in Supplementary Figure 3. Overall, we performed experiments on 10 real single‐cell datasets to validate the effectiveness of scRSSL. From Table 1, we can observe more detailed information about the dataset and information about the source of the data.

TABLE 1.

Description of the 10 single‐cell data sets used to assess the performance of computational methods.

Datasets	Tissue	Cells	Genes	Class	Protocol	Reference
Wang	Human pancreas	457	19,950	7	SMARTer	Wang et al., 2016
Xin	Human pancreas	1600	39,851	8	SMARTer	Xin et al., 2016
Baron (Huamn)	Human pancreas	8569	20,125	14	inDrop	Baron et al., 2016
Segerstolpe	Human pancreas	2209	25,525	14	Smart‐Seq2	Segersto et al., 2016
Muraro	Human pancreas	2126	19,127	10	Cel‐Seq2	Muraro et al., 2016
Baron (mouse)	Mouse pancreas	1886	14,878	13	inDrop	Baron et al., 2016
Klein	Mouse embryo stem	2717	24,175	4	inDrop	Klein et al., 2015
Romanov	Mouse brain	2881	24,341	7	SMARTer	Roman et al., 2017
Zeisel	Mouse brain	3005	19,972	9	STRT‐Seq	Zeisel et al., 2015
hECA*	Mixed	14,190	15,500	8	Multiple	Chen S et al., 2021

We take a uniform pre‐processing approach for each of the three data, for the five datasets in the first data. We only filter some low‐quality cells, such as the dropped cells in “Wang”. The data are then normalised using a log transformation. For the second dataset, we use the method proposed in Seurat to pre‐process the dataset. The specific processing is similar to automatic deep embedding clustering (ADClust) [43]. For the third dataset, we use the method proposed in Scanpy to filter the genes. Then, it is screened for highly differentially expressed genes using scVI, and the top 15,500 highly differentially expressed genes are finally selected.

2.2. The architecture of scRSSL

In this paper, we propose a deep residual generation model scRSSL based on semi‐supervised learning. Its underlying structure is a variational autoencoder (VAE) [44, 45]. The model framework is shown in Figure 1. ScRSSL consists of three modules. First, an encoder module to reduce the dimensionality of the data, and then the data are labelled using a semi‐supervised residual network [46] module. After labelling, another encoder neural network is used to map the latent variables to the negative binomial distribution [47], and then a decoder is used to convert the data to the original dimensionality. The whole training process is accomplished by minimising three losses, including the common reconstruction loss and Kullback–Leibler Divergence (KL) loss in VAE and our new classification loss. Please see Figure 1 for the detailed structure of the model.

FIGURE 1.

Model framework summary figure. Firstly, part of the labelled data was preliminarily pre‐processed before entering the neural network model. Then, the first hidden layer is compressed to produce an initial possible representation, designated as z1. After separating the labelled cells from the unlabelled cells in z1, we consider the cell type of the unlabelled cell data as a latent variable and introduce the variable y for the labelled cell data. After the cell type is inferred by the residual neural network architecture, the unlabelled cells are created together with the labelled data as the second latent representation z2. Finally, the decoder neural network converts it into a negative binomial distribution of the original dataset.

2.3. Variational autoencoder module

In this paper, a variational autoencoder is used as the base model [48]. Variational autoencoder is one of the most used deep generating models, and its function can be summarised as follows: using lower sample dimensions to represent higher dimensional samples [49]. It is based on the autoencoder, which is a data compression algorithm based on deep learning. The autoencoder is composed by an encoder neural network and a decoder neural network whose loss term is the reconstruction loss of the data, and ensures that the compressed data is not distorted by minimising the reconstruction loss [50]. The major difference between the variational autoencoder and the autoencoder is that the former incorporates a noise based on a normal distribution in the low‐dimensional space. Thus, the variational autoencoder needs to take into account a KL loss term while considering the reconstruction error loss term [51].

The variational autoencoder contains an original variable $x$ and a latent variable $z$. Under the assumption that the prior $p\left(z\right)$ is a normal distribution, we use it and the conditional distribution to find the joint probability $p_{\sigma }\left(x|z\right)$, where $p_{\sigma }\left(x,z\right)=p\left(z\right)p_{\sigma }\left(x|z\right)$ is the parameter of $\sigma$. This joint probability $p_{\sigma }\left(x|z\right)$ represents its original distribution. The process of data generation can be understood as follows: the generation of low‐dimensional features $z$ generated from the standard orthogonal distribution as a condition of the original variable $x$. The variational autoencoder first uses the Bayesian formula to calculate $p\left(z|x\right)=\frac{p\left(x|z\right)p\left(z\right)}{p\left(x\right)}$, and the potential variable $z$ can be obtained from the original variable $x$ by doing so. The likelihood function $p_{\sigma }\left(x\right)$ of the original variables is then maximised by computing $p_{\sigma }\left(x\right)={\mathrm{E}}_{zp\left(z\right)}\left[p_{\sigma }\left(x|z\right)\right]$. Since computing $p\left(z|x\right)$ is difficult, we introduce a parameterised $q_{\omega }\left(z|x\right)$ to approximate it, and the parameter $\omega$ refers to $\mu {\sigma }^2$, denoting the mean and variance. Then, we need to calculate a log‐likelihood, also known as Evidence Lower BOund (ELBO), which is calculated as follows [52]:

$$\log p_{\sigma }\left(x\right)\ge {\mathrm{E}}_{z{q}_{\omega }\left(z|x\right)}\left[\log p_{\sigma }\left(x|z\right)\right]-KL\left(q_{\omega }\left(z|x\right)\Vert p\left(z\right)\right)=ELBO\left(x;\omega ,\sigma \right)$$ (1)

When we are maximising this ELBO, it is equivalent to minimising $KL\left(q_{\omega }\left(z|x\right)\Vert p\left(z|x\right)\right)$ at the same time. This also meets our need to let the introduced distribution $q_{\omega }\left(z|x\right)$ be the closest to $p\left(z|x\right)$. It is very convenient for us to implement this computation with the power of deep learning. We only need to design two neural network structures, an encoder mainly responsible for optimising the parameters of $q_{\omega }\left(z|x\right)$ and a decoder mainly responsible for optimising the parameters of $p_{\sigma }\left(x|z\right)$. It is worth noting that we need to perform a resampling trick after adding the noise of the Gaussian distribution to the latent space [53]. Making it derivable so as to ensure the continuity of the training. The general formula of the variational autoencoder [54] is as follows:

$$\mathrm{F}\left(x;\omega ,\sigma \right)=-E_{z{q}_{\omega }\left(z|x\right)}\left[\log p_{\sigma }\left(x|z\right)\right]+KL\left(q_{\omega }\left(z|x\right)\Vert p\left(z\right)\right),$$ (2)

2.4. Semi‐supervised learning module

The semi‐supervised generative model represents the data as the data generated from the continuous variable $z$ and the category variable $y$ obtained from the VAE potential space, which is defined by [55]

$$p\left(y\right)=Cat\left(y|\pi \right);p\left(z\right)=\mathrm{N}\left(z|0,I\right);p_{\theta }\left(x|z,y\right)=f\left(x;y,z,\theta \right)$$ (3)

where $Cat\left(y|\pi \right)$ is the multinomial distribution [56]. For a single‐cell without labels, $y$ is a potential type variable and $z$ is a potential continuous variable. These potential variables are untangled and they can represent certain properties used to distinguish single cells. $f\left(x;y,z,\theta \right)$ is our assumed likelihood function, which is assumed to be negatively binomially distributed for the single‐cell dataset [57]. It is obtained by a non‐linear parameterisation of the latent variables, which we implement here by means of a decoder neural network in the model. We integrate single cells without labels, make classification inferences by training the neural network, and predict the type of unlabelled cells from the inferred posterior distribution $p_{\theta }\left(y|x\right)$. After obtaining the posterior probabilities, we reject the cells with posterior below the threshold $\gamma$.

By stacking the variational autoencoder model and the semi‐supervised learning model, we can obtain a semi‐supervised learning model based on the deep generative model [58]. The specific procedure is to replace the original data $x$ with latent variables obtained from the variational autoencoder and then learn a semi‐supervised model. The final composition is a deep generative model with two layers of random variables:

$$p_{\theta }\left(x,y,z_1,z_2\right)=p\left(y\right)p\left(z_2\right)p_{\theta }\left(z_1|y,z_2\right)p_{\theta }\left(x|z_1\right)$$ (4)

where $p\left(y\right)$ and $p\left(z_2\right)$ are equivalent to $y$ and $z$ in Equation (3), respectively. $p_{\theta }\left(z_1|y,z_2\right)$ and $p_{\theta }\left(x|z_1\right)$ are parameterised using our constructed neural network structure.

2.5. Deep residual learning module

The features learned when the input is $x$ in some network hierarchy are $H\left(x\right)$. Assuming that multiple non‐linear layers can asymptotically approximate some complex function, they can also asymptotically approximate some residual function: $H\left(x\right)-x$, where the output and input are of equal dimensions [59]. Assume that the residual function is defined as $F\left(x\right)=H\left(x\right)-x$. The original function is $F\left(x\right)+x$. The degeneracy problem illustrates the difficulties that may exist when approximating a constant mapping with multiple non‐linear layers. In this case, residual learning is used every few stacked modules, defined as follows:

$$y=F\left(x,\left\{W_i\right\}\right)+x$$ (5)

where $x$ denotes the input vector, $y$ denotes the output vector, and $F\left(x,W_i\right)$ denotes the residual mapping that needs to be learned. $F=W_2\alpha \left(W_{1}x\right)$, where $\alpha$ denotes the activation function ReLU [60] and $F+x$ represents a shortcut connection. Note that the dimensions of $x$ and $F$ must be equal, and if they are not, a linear mapping $W_s$ needs to be performed to dimensionally match them:

$$y=F\left(x,W_i\right)+W_{s}x$$ (6)

Our residual network is composed of 20 convolutional layers, a maximisation pooling layer, an average pooling layer and a fully connected layer. It consists of eight residual modules in total, and the stacked residual modules are connected to each other by (Eq 5) in the above equation. The design obeys the principle that the number of filters is doubled whenever the size of the feature map is halved. This keeps the time complexity of each layer unchanged [61].

2.6. Convolutional neural network (CNN) module

Compared with the traditional neural network connection method, CNN adds a new convolutional layer, pooling layer [62]. Among them, the convolutional layer is the key module of the CNN structure, which is composed of multiple feature maps as input, and each feature map is composed of multiple neurons [63]. We convert the input data into the form of feature maps through the tensor broadcast mechanism. The convolution layer scans the feature map with a convolution kernel to extract the information of each adjacent part of the feature map, and then the model is trained by batch normalisation and activation function. The formula for the convolution operation is as follows:

$$X_j^{l+1}=f\left(\sum _{i\in N_j}{X}_i^l\otimes k_{ij}^{l+1}+b_i^{l+1}\right)$$ (7)

where $X_j^{l+1}$ denotes the $j\text{th}$ feature map of the $\left(l+1\right)\text{th}$ convolution layer, $X_j^l$ denotes the input data, $f$ denotes the activation function, $N$ represents the set of input feature maps, $\otimes$ represents the operation of the convolution operation, $k$ represents the convolution kernel, and $b$ denotes the bias term. Pooling layer is used to reduce the dimensionality of the feature graph by combining some of the features in the feature graph together. Pooling operations generally include average pooling [64], maximum pooling [65] and Lp pooling [66]. The pooling layer is defined as follows:

$$X_j^{l+1}=X_j^l\odot {\mathrm{k}}_j^{l+1}+b_j^{l+1}$$ (8)

where $X_j^{l+1}$ is the $j\text{th}$ feature of the $\left(l+1\right)$th layer, $\odot$ denotes the pooling operation, $k$ denotes the convolution kernel, and $b$ denotes the bias term. Finally, the results of single‐cell type prediction are output through a fully connected layer [67].

2.7. Hyperparameters setting

ScRSSL is implemented using PyTorch. For the variational autoencoder module, we set the dimensionality transformation as input‐128‐100‐128‐output. That is, the dimensionality of the potential space is set to 100, the batch size during training is set to 128, and when the number of samples of the input data is larger than 10,000, it is expanded to 1024 to reduce the training time. We choose the Adam optimiser [68] to optimise the model and the learning rate is set to 0.001. We set the training epoch to 100 times. The activation function is set to Relu [60]. For the semi‐supervised module, that is, the residual network training module, we set the data of 100 dimensions in the potential space as [batch size, 1, 10, 10], and the last fully connected layer of the residual network module as [512, number of types of cells], and all activation functions in the module use Relu [60], and the last fully connected layer uses Sigmoid [69] as the activation function.

2.8. Benchmarking classification methods

To evaluate the performance of scRSSL, we use seven methods as a baseline to compare with our method, and they are Random Forest (RF) [70], Support Vector Machine (SVM) [71], K‐Nearest Neighbour (KNN) [72], Adaptive Boosting (AdaBoost) [73], Decision Tree (DT) [73], Naive Bayes (NB) [74] and scANVI [23] For the first experiment, we use the inter‐dataset mentioned in the text datasets that belong to human pancreatic cells and therefore have similar genes. They are Wang (457 cells), Baron (Human) (8569), Xin (1600), Segerstolpe (2209), and Muraro (2126). We choose one of these datasets at a time as the reference dataset and the other four as the query dataset, which is equivalent to a five‐fold cross‐validation. For example, we first use Wang as the reference dataset and then test the trained model on each of the four query datasets: baron, Xin, Segerstolpe, and Muraro. Next, we use baron as the reference dataset and then use the trained model for each of the four query datasets: Wang, Xin, Segerstolpe, Muraro etc. For the second class of the intra‐dataset, we experiment it with a five‐fold cross‐validation and compare it with the evaluation of the baseline model. For the third class of the single‐cell dataset, which is characterised by severe sample imbalance, we evaluate the effectiveness of scRSSL on it.

Evaluation metric: We use f1‐score [75] to evaluate the test results.

$$f1-score=\frac{2\left(P\times R\right)}{P+R}$$ (9)

In Equation (9), $P$ represents the accuracy rate, $R$ represents the recall rate, and $f1-score$ considers both the model accuracy rate and the recall rate, which represents the harmonic average of the two. The higher the value of $f1-score$, the better the test result.

3. RESULTS

We evaluate the performance of scRSSL on different types of real datasets. For the inter‐dataset, we test five datasets. As shown in Figure 2, our proposed model scRSSL outperforms other algorithms in all the universal datasets. One of them, the Wang dataset, contains 457 cells, and it is also the smallest dataset in all datasets. When we use it to predict the largest single‐cell dataset, Baron (8569 cells), it still has the highest results among all methods. We can see that in this extreme case, ordinary machine learning algorithms have failed, such as SVM, KNN, AdaBoost, decision trees, and Naive Bayes. This shows the good performance of our model. We can see that the scANVI model, which is specifically designed to handle single‐cell datasets, does not perform as well as the scRSSL model in most cases. Only two cases lead scRSSL by a slight margin, which can indicate that the model has a stronger ability to extract features when we make the layers of the neural network deeper. There is also a role played by the convolutional neural network in the residuals module, which can capture the local features of the single‐cell transcriptome through the filter, which makes the model's performance further improved. In some special cases, ordinary machine learning methods have failed, mainly due to factors such as batch effects between single‐cell datasets. For example, when the Segerstolpe dataset is used as the training set to train the model and then the training model is used to predict the cell types of the Xin dataset, the SVM, KNN, and Naive Bayesian algorithms have completely failed. However, the scRSSL model still maintains a high prediction score: 0.92. This indicates that our model is robust to the single‐cell dataset.

FIGURE 2.

Inter‐dataset experiment result figure. The box plot of the experimental results of scRSSL compared with seven other baseline methods, using the f1‐score as the evaluation metric. The first row of the graph represents the use of the Xin dataset as the reference dataset, with the remaining four datasets as the query dataset for prediction.

For the intra‐dataset, we used four datasets. Figure 3 represents a box plot of the experimental results of scRSSL compared with seven other baseline methods, where the metric for evaluating the effectiveness of the model is the f1‐score. The compared methods include two types, one is a general machine learning method and the other is a classification algorithm specifically for single‐cell datasets. The general machine learning methods include AdaBoost, Decision Tree, KNN, Naive Bayes, Random Forest, and SVM. ScANVI is the method specifically for single‐cell datasets. Algorithms for single cells include scANVI and JLONMFSC [76]. scANVI is chosen as the baseline model because it is also a depth‐based model generation method like scRSSL. Notably, it is a very competitive approach. To the best of our knowledge, this model is the most advanced single‐cell semi‐supervised learning algorithm. JLONMFSC is an unsupervised learning method specifically designed for processing single‐cell RNA sequencing (scRNA‐seq) data. We observed that the scRSSL model has higher accuracy compared to other baseline model algorithms. One of the advantages of scRSSL over scANVI is due to its network structure which has better feature extraction capabilities compared to fully connected networks. For the imbalance dataset, we used a subset of the hECA single‐cell dataset to illustrate the mitigation of sample imbalance. Figure 4 shows the confusion matrix obtained by each model predicting the hECA dataset, where 0–7 represent: Adipocyte, Cardiomyocyte cell, Endothelial cell, Fibroblast, Lymphoid cell, Myeloid cell, Pericyte, Smooth muscle cell, and other 8 cell types. Among them, Cardiomyocyte cell cells occupied 36.44\%, which was the most abundant cell type, followed by Fibroblast type and Endothelial cell type, which together accounted for 74.28\% of the whole single‐cell dataset. Adipocyte type, Lymphoid cell type, and Smooth muscle cell type were the least abundant, accounting for: 1.58\%, 2.58\%, and 2.72\%, respectively. For such single‐cell datasets with high dimensionality and unbalanced samples, general supervised machine learning methods, such as those in Figure 4b–d, predict a relatively high rate of false positives for the samples. In contrast, the scRSSL model in Figure 4 outperforms the other models, and from the confusion matrix case, we can observe that it predicts correctly more minority single‐cell types, such as Adipocyte cell types and Lymphoid cell types.

FIGURE 3.

Intra‐dataset experiment result figure. Boxplot of the experimental results of scRSSL against seven other baseline methods on four datasets: Zeisel, Baron, Klein, and Romanov.

FIGURE 4.

Confusion matrix figure of experimental results. The confusion matrix obtained by each model predicting the hECA dataset, where 0–7 represent: Adipocyte, Cardiomyocyte cell, Endothelial cell, Fibroblast, Lymphoid cell, Myeloid cell, Pericyte, Smooth muscle cell, and other 8 cell types.

We can see more specifically the correctness of each method for each class prediction from the ROC curve plot in Figure 5. We can see that scRSSL represents the highest AUC curve for predicting Lymphoid cell types compared to all the other three methods. The ROC value of our method for predicting the Smooth muscle cell type is 0.97, which is higher than 0.93 for RF, 0.84 for KNN, and 0.8 for AdaBoost, indicating that the model can significantly reduce the sample bias when predicting few cell types, thus improving the impact caused by sample imbalance. In Figure 6, we observe the comparison of ROC curves for all models, where scRSSL is largely dominant but slightly lower than the ROC curve of KNN. The reason for this situation may be that KNN fetches a better number of neighbour nodes. However, looking at the category prediction curves of each model (Figure 5a–d), scRSSL is superior to the KNN method in the microscopic case, which is mainly since the residual network can use the convolutional layer to capture the detailed features of the data.

FIGURE 5.

Visualisation of the prediction performance comparison of different cell types on four models: scRSSL, RF, KNN, and AdaBoost. In the ROC plot, the closer the curve is to the upper left corner, the better the performance of the model.

FIGURE 6.

Visualisation of the prediction performance comparison of the four models scRSSL, RF, KNN, and AdaBoost. In the ROC plot, the closer the curve is to the top‐left corner, the better the performance of the model.

4. ABLATION EXPERIMENT

scRSSL and scRSSL variants without the residual network module are applied to four datasets: Zeisel, Baron, Klein, and Romanov for the comparative analysis to verify the effectiveness of the residual module. As shown in Figure 7, the effectiveness of the scRSSL method is higher than the scRSSL variant without the residual network module. The residual module has two advantages: on the one hand, the residual module can extract local features in single‐cell data. On the other hand, the residual structure can solve the degradation problem of the network. Therefore, the introduction of the residual module can enhance the feature extraction ability of the model. The experimental results of scRSSL and scRSSL variants without the residual network module on four datasets also demonstrate the effectiveness of the residual module.

FIGURE 7.

Ablation experiment. F1 score visualisation results of two models, ScRSSL and scRSSL variants without the residual network module on four datasets: Zeisel, Baron, Klein, and Romanov.

5. DISCUSSION

With the advent of large‐scale single‐cell datasets, there is a need for an accurate and scalable algorithm to automatically annotate cell types. ScRSSL is developed to address this problem. ScRSSL first introduces a semi‐supervised generation model to effectively improve the sample bias problem and then constructs the semi‐supervised inferential network as a residual network structure to enhance the ability of the neural network to extract features from single‐cell data. The advantages of scRSSL are as follows: Firstly, the perceptual field unique to convolutional neural networks helps to extract local detailed features of single‐cell data, and secondly, in the model of the residual network structure, the way of passing residuals can solve the degradation problem of neural networks, thus making the layers of the network deeper and helping to further abstract the features. Experiments show that the neural network model with the residual structure has better performance compared with the neural network model with a fully connected structure similar to scANVI.

When we are training the model, we need to convert the data at the dimensional level. We use the broadcast mechanism of deep learning to convert the two‐dimensional single‐cell transcriptome data into a feature map with one channel and then train the residual network on it. However, this is not the only way to process the data, as the conversion process involves different data representations, such as image data versus tabular data. We will explore more ways of data transformation in the next research, which may involve more aspects of the principles; however, we think it is very valuable. At the same time, since the semi‐supervised generation model has the natural imputation capability of missing data, we will continue to explore and improve the imputation capability of scRSSL.

AUTHOR CONTRIBUTIONS

Yanru Gao: Methodology; Writing ‐ original draft. Hongyu Duan: Methodology; Writing ‐ review and editing. Fanhao Meng: Writing ‐ review and editing. Conghui Zhang: Visualisation. Xiyue Li: Visualisation. Feng Li: Funding acquisition; Methodology; Writing ‐ review and editing.

CONFLICT OF INTEREST STATEMENT

The authors declare no potential conflicts of interests.

Supporting information

Supplementary Material

Gao, Y. , et al.: scRSSL: Residual semi‐supervised learning with deep generative models. IET Syst. Biol. e12107. (2025). 10.1049/syb2.12107

DATA AVAILABILITY STATEMENT

The source code and dataset can be obtained through the following links: https://github.com/FengLi1028/scRSSL.git.