FRMC: a fast and robust method for the imputation of scRNA-seq data

ABSTRACT

The high-resolution feature of single-cell transcriptome sequencing technology allows researchers to observe cellular gene expression profiles at the single-cell level, offering numerous possibilities for subsequent biomedical investigation. However, the unavoidable technical impact of high missing values in the gene-cell expression matrices generated by insufficient RNA input severely hampers the accuracy of downstream analysis. To address this problem, it is essential to develop a more rapid and stable imputation method with greater accuracy, which should not only be able to recover the missing data, but also effectively facilitate the following biological mechanism analysis. The existing imputation methods all have their drawbacks and limitations, some require pre-assumed data distribution, some cannot distinguish between technical and biological zeros, and some have poor computational performance. In this paper, we presented a novel imputation software FRMC for single-cell RNA-Seq data, which innovates a fast and accurate singular value thresholding approximation method. The experiments demonstrated that FRMC can not only precisely distinguish ‘true zeros’ from dropout events and correctly impute missing values attributed to technical noises, but also effectively enhance intracellular and intergenic connections and achieve accurate clustering of cells in biological applications. In summary, FRMC can be a powerful tool for analysing single-cell data because it ensures biological significance, accuracy, and rapidity simultaneously. FRMC is implemented in Python and is freely accessible to non-commercial users on GitHub: https://github.com/HUST-DataMan/FRMC.

1. Introduction

Single cell RNA sequencing (scRNA-seq) has been becoming one of the most widely used technologies in the biomedical investigation over the past few years. Using the traditional bulk RNA sequencing method dealing with RNA isolated from millions of cells, the final gene expression value should be considered as the average of all input cells, and is therefore more suitable for revealing the global view of gene expression [1,2]. However, bulk RNA sequencing cannot accurately quantify RNA content, and besides, it suffers from measurement bias when samples contain multiple heterogeneous cell populations (e.g. at early embryonic developmental stages where the number of cells is relatively limited and multiple lineages exist) [3]. In contrast, scRNA-seq overcomes the disadvantage of traditional bulk RNA sequencing, and enables analysis of transcriptome data at the single-cell level, thereby obtaining information on gene sequences, transcripts, and epigenetics of a single cell [4–6].

With the development of high-throughput sequencing technology and the reduction of sequencing costs, the number of studies using scRNA-seq technology has exploded, revealing many important biological discoveries [7–9]. A growing number of studies have demonstrated the irreplaceable application of identifying different cell subtypes of seemingly similar cells, including discriminating cancer heterogeneity [10,11], discovering and identifying novel cell populations, and elucidating changes of transcriptional dynamics in cellular developmental trajectories [12–18].

Despite its many advantages, the scRNA-seq technique is still plagued by relatively higher technical noises. One of the general troubles is the insufficient RNA input. For example, some low-abundance transcripts are often lost in the reverse transcription step due to their own low quantity [19]. In addition, the expression counts we observed are usually only a small random sample (typically 5–15%) of the transcriptome of each cell because of the low capturing and sequencing efficiency [2,20,21]. These technical factors affect scRNA-seq result data sets with high sparsity in the gene – cell expression matrix, also known as ‘zero inflation’, i.e. a large percentage (even exceeding 90%) of zero or very low values in the expression matrix. Some of these zeros are ‘true zeros’, defined as genes that were never expressed in the first place, and some are ‘false zeros’, also known as dropout events, which refer to situations where transcripts are physically present but not amplified and detected, whose essence is a special type of missing value [19].

Current commonly used scRNA-seq statistical tools handle dropout events in different ways [3]. Excluding genetic data containing dropout events directly from the analysis would inevitably lead to the loss of information and sacrifice the opportunities to discover numerous biologically valuable findings [22]. Some approaches directly use clustering algorithms such as PCA to aggregate genes or cells to create ‘meta-genes’ or collapse thousands of cells into a small number of clusters, which addresses the sparsity of scRNA-seq data to some extent, but simultaneously loses the resolution at single cell or single gene level [23]. Therefore, imputation of dropout events is an indispensable step in processing scRNA-seq datasets. From a biological point of view, a reasonable imputation strategy should have the following discriminatory properties: it should keep the ‘true zero’ counts unaffected, while correctly recovering the expression counts of the genes in which the dropout event occurred [22].

In response to the above challenges, there are two main types of imputation. The first type is based on predetermined statistical models, and the limitation of such algorithms is their requirement to make appropriate statistical distribution assumptions on the data to be processed in advance. Examples include scImpute based on a gamma-normal mixed statistical model [24], VIPER based on a non-negative sparse regression model [25], 2DImpute based on the assumption of interrelationships between genes and cells [26], etc. The other type is built on a mathematical optimization model for low-rank matrix recovery without the need to make a priori statistical distribution assumptions, such as McImpute [22] and ALRA [27].

To tackle the drawbacks and limitations of existing imputation methods, we proposed a novel fast and robust imputation software based on matrix completion, called FRMC, for scRNA-seq data in this paper. This method FRMC achieves simultaneously superior performance in accuracy and speed by innovating a singular value thresholding approximation method and further combining a mathematical optimization model for low-rank matrix completion and a prejudgment algorithm for distinguishing dropout events. The experiments we performed demonstrated that our FRMC method can not only precisely distinguish ‘true zeros’ from dropout events and correctly impute missing values attributed to dropout events, but also effectively enhance intracellular and intergenic connections and achieve accurate clustering of cells in biological applications.

2. Methods

2.1. Algorithms and implementation

The FRMC method imputes scRNA-seq data by taking a two-step approach: (1) first evaluate the ‘similar cell set’ and the proportional index of gene expression on this set on the basis of Jaccard similarity coefficients to further predict the biological ‘true zero’ and dropout events; (2) transform the imputation problem of expression matrix into a low-rank matrix optimization problem, then minimize the augmented Lagrangian function of this constrained minimization problem through a singular value thresholding iteration to recover the complete gene expression matrix.

2.1.1. Prejudgment method

Before the imputation algorithm starts, we prejudge whether each zero in the data expression matrix is a dropout that requires imputation, or a ‘true zero’ that does not require any subsequent changes and mark them.

We first identify a set of ‘similar cells’ that are similar to the cells they belong to, defined as having the same genes expressed (with positive values) or not expressed (with zero values) as far as possible. To quantify the similarity between cell $i$ and cell $j$, we use the Jaccard similarity coefficient based on a binary expression value (0 for non-expression and 1 for gene expression), defined as

$$J_{ij}=\frac{G_i\cap G_j}{G_i\cup G_j}$$

Here, $G_i$ and $G_j$ stands for the set of genes expressed in cell $i$ and cell $j$, respectively. A group $i$ of ‘similar cells’ is defined as a collection of all cells that have a Jaccard similarity coefficient greater than 0.5 with cell $i$. If the number of cells in this group $i$ is less than 10, zero is held constant because of insufficient confidence. In each group of similar cells, we calculated the proportion of cells where each gene is expressed (>0) as an indicator. If this indicator exceeds a critical threshold of 20% for a gene, we prejudge that the expression zero value for that gene is a dropout event, otherwise it is true zero.

2.1.2. Imputation algorithm

The imputation of scRNA-seq expression matrix is to use the partially observed gene expression matrix $X$ (where rows represent individual cell IDs and columns represent genes) to recover the missing unknown values (i.e. zero values). Assume that the observed scRNA-seq expression matrix $M\in R^{m\times n}$ is a sampled version of the complete expression matrix $X\in R^{m\times n}$. The mathematical representation is: $M=P_{\mathrm{\Omega }}\left(X\right)$. Here $P_{\mathrm{\Omega }}:R^{m\times n}\to R^{m\times n}$ is the orthogonal projection matrix, which is a binary mask matrix containing only 0 and 1 of the same dimension as $X$, where 0 indicates that no gene expression is observed at the corresponding position in $X$ and 1 indicates that gene expression is observed at the corresponding position in $X$. Its mathematical expression is $[P_{\mathrm{\Omega }}\left(X\right){]}_{ij}=\left\{\begin{array}{c}{X}_{ij,\hspace{0.17em}}\left(i,j\right)\in \mathrm{\Omega }\\ 0,\left(i,j\right)\notin \mathrm{\Omega }\end{array}\right.$, where $\mathrm{\Omega }$ is the set of indices of the known values in the data matrix. In summary, the imputation problem can be translated as follows: given the gene expression observations $M$ and the binary mask matrix $P_{\mathrm{\Omega }}$, one recovers the original complete gene expression matrix $X$.

It has been proven that a matrix $X$ of rank $r$ can be recovered by solving the following nuclear norm minimization problem (NNM) [28,29]:

$$\underset{x}{min}{∥X∥}_{\ast }\hspace{0.17em}s.t.\hspace{0.17em}{X}_{ij}=\hspace{0.17em}{M}_{ij},\hspace{0.17em}\mathrm{\forall }\left(i,j\right)\in \mathrm{\Omega }$$ (1)

where ${∥.∥}_{\ast }$ denotes the nuclear norm of the matrix, i.e. the sum of all singular values of the matrix. This motivates us to extend the model to the imputation of dropout events in the actual scRNA-seq data expression matrix. To solve the NNM problem easily by using the existing optimization theory and Lagrangian methods, FRMC introduces an error matrix $E\in R^{m\times n}$. The NNM problem is further transformed into the following optimization problem with equality constraints:

$$\mathrm{m}\mathrm{i}{\mathrm{n}}_x\hspace{0.17em}{∥X∥}_{\ast }\hspace{0.17em}s.t.\hspace{0.17em}X+E=M,P_{\mathrm{\Omega }}\left(E\right)=0$$ (2)

According to the constrained optimization theory [30], by introducing a Lagrangian multiplier matrix $Y$ to remove the equality constraint, the optimal solutions of (2) can be approximated by unconstrained minima of the following augmented Lagrangian function:

$$L\left(X,E,Y,\mu \right)=X{ }_{\ast }+\hspace{0.17em}⟨Y,M-X-E⟩+\frac{\mu }{2}M-X-E_F^2,\hspace{0.17em}{P}_{\mathrm{\Omega }}\left(E\right)=0$$ (3)

where $\mu$ is a positive penalty parameter and ${∥.∥}_{\mathrm{F}}$ denotes the Frobenius norm of the matrix, i.e. ${∥.∥}_{\mathrm{F}}$ for matrix ${∥A∥}_F=\sqrt{\sum _{i,j}{a}_{ij}^2}$. The details of the augmented Lagrangian method for solving the optimization problem with equality constraints were demonstrated in Section 4.2 [31].

From Theorem 2.1 and its proof process in Cai’s work [32], it can be seen that

$$U{S}_{{\mu }_k^{-1}\hspace{0.17em}}\left[S\right]V^T=arg\underset{X}{min}\hspace{0.17em}\left({∥X∥}_{\ast }+\hspace{0.17em}⟨Y,M-X-E⟩+\frac{{\mu }_k}{2}{∥M-X-E∥}_F^2\right),$$

namely, the optimal solution of $L\left(X,E,Y,{\mu }_k\right)$ about $X$ is $X_{k+1}=U{\mathbb{S}}_{{\mu }_k^{-1}}\left[S\right]V^T$ where $\left(U,S,V\right)=\mathrm{s}\mathrm{v}\mathrm{d}\left(M-E_k+{\mu }_k^{-1}{Y}_k\right)$. The notation svd is singular value decomposition of one matrix. To solve the problem (2), the FRMC algorithm in this paper iteratively updates $X$ and $E$ via minimizing the augmented Lagrangian function (3) with respect to $X$ and $E$ respectively, with Lagrangian multiplier matrix $Y$ fixed. The specific algorithmic steps are as follows:

Input: scRNA-seq gene expression matrix $M\in {\mathbb{R}}^{m\times n}$ obtained by the prejudgment method.

Step0Initialization: $\begin{array}{rl}& Y_0=0_{m\times n};E_0=0_{m\times n};{\mu }_0>0;\rho >1;k=0;\\ & tol1=1e^{-4},tol2=1e^{-3}\end{array}$.

Step 1 If convergence conditions $\frac{M-X_{k+1}-{E_{k+1}}_F}{M{ }_F}<tol1$ or $\frac{M-X_{k+1}-{E_{k+1}}_F}{M{ }_F}<tol1$ holds, STOP

Step 2 Solve $X_{k+1}=argmi{n}_x\mathcal{L}\left(X,E_k,Y_k,{\mu }_k\right)$:

$$\left(U,S,V\right)=\mathrm{s}\mathrm{v}\mathrm{d}\left(M-E_k+{\mu }_k^{-1}{Y}_k\right)$$ (4)

$$X_{k+1}=U{\mathbb{S}}_{{\mu }_k^{-1}}\left[S\right]V^T$$ (5)

where $S_{{\mu }_k^{-1}}\hspace{0.17em}\left[x\right]=\left\{\begin{array}{c}x-{\mu }_k^{-1},ifx-{\mu }_k^{-1}>0\\ 0,otherwise\end{array}\right.$.

Step 3 Solve $E_{k+1}=\mathrm{a}\mathrm{r}\mathrm{g}\underset{P_{\mathrm{\Omega }}\left(E\right)=0}{min}\mathcal{L}\left(X_{k+1},E,Y_k,{\mu }_k\right)$:

$$E_{k+1}=P_{\bar{\mathrm{\Omega }}}\left(M-X_{k+1}+{\mu }_k^{-1}{Y}_k\right)$$ (6)

where $\bar{\mathrm{\Omega }}\hspace{0.17em}$ is the set of indices of the unknown values in $M$.

Step 4 Update Lagrangian multiplier matrix $Y_{k+1}$:

$$Y_{k+1}=Y_k+{\mu }_k\left(M-X_{k+1}-E_{k+1}\right)$$ (7)

Step 5 Update ${\mu }_{k+1}$:

$${\mu }_{k+1}=\left\{\begin{array}{c}\rho {\mu }_k,ifmin\left\{{\mu }_k,\hspace{0.17em}\sqrt{{\mu }_k}\right\}\ast \frac{{∥E_{k+1}-E_k∥}_F}{{∥M∥}_F}<tol2\\ {\mu }_k,otherwise\end{array}\right.$$ (8)

Step 6 Set $k=k+1$. Return to Step 1.

Output: Imputed gene expression matrix $X_{k+1}$.

2.2. Performance evaluation

2.2.1. Datasets

This paper uses scRNA-seq datasets from four different studies to conduct the evaluation experiments, all of which are available for download from public websites.

(1) Usoskin dataset [33]: This dataset is mouse neuronal data, obtained by RNA-Seq of 799 dissociated single cells dissected from mouse dorsal lumbar ganglia (DRG). After principal component analysis (PCA) of the expression of all cells and genes, 622 cells were classified as neuronal, 109 as non-neuronal and 68 as unknown cell types. We considered neuronal clusters containing NF, non-peptidergic injury receptors (NP), peptidergic injury receptors (PEP) and tyrosine hydroxylase (TH) of mouse lumbar DRG neurofilaments. RPM normalized expression data are available under NCBI Gene Expression Omnibus (GEO) GSE59739.

(2) Jurkat-293T dataset [34]: This dataset contains the expression profiles of Jurkat and 293T cells, which were mixed in equal proportions (50%:50%) in vitro. All approximately 3,380 cells were annotated according to the expression of cell type-specific markers. Cells expressing the gene CD3D were assigned to Jurkat and cells expressing the gene XIST were assigned to 293T. The dataset is available from the 10x Genomics website: https://support.10xgenomics.com/single-cell-gene-expression/datasets/1.1.0/jurkat:293t_50:50.

(3) InDrop dataset [35]: This single-cell RNA-seq dataset contains immune cells from eight patients with primary breast cancers and matched normal tissues. Two cell sets were selected for this study, InDrop-BC6 consisting of 3498 cells from one primary breast cancer of patient BC6, and InDrop-BC4 consisting of 13,266 cells from the blood of the patient BC4. This dataset can be downloaded from NCBI GEO with the ID number GSE114725.

(4) Smart-Seq2 dataset [36]: This dataset contains single-cell RNA sequencing profiles from 33 human melanoma tumours, including 17 newly collected patient tumours and 16 previously reported patient tumours. Data containing 2987 cells from the new cohort were selected for this study. This data is accessible from NCBI GEO GSE115978.

(5) NeuronCell dataset: This dataset contains more than 100,000 neuronal cells derived from temporal cortices of nine epileptic and ten non-epileptic subjects. The data is available on github: https://github.com/khodosevichlab/Epilepsy19.

2.2.2. Data preprocessing

Data preprocessing is divided into three steps: first, low abundance genes are removed before imputing and other downstream analyses [20,37]. We filter the data expression matrix for genes with extremely low expression, so that the retained genes are expressed (at least three reads) in at least 20 cells. Next, the filtered data expression matrix is normalized by the total number of libraries: for each cell, the expression of each gene was divided by the total expression of all genes and then multiplied by 10,000. Finally, the normalized single-cell data expression matrix is $\mathrm{l}\mathrm{o}\mathrm{g}\left(x+1\right)$ transformed to obtain a new expression matrix, which is used as the input to FRMC to perform imputation.

2.2.3. Dimensions and metrics

To comprehensively evaluate the performance of FRMC, we evaluated and compared the performance and accuracy of FRMC with the existing methods McImpute, ScImpute, 2Dimpute, and ALRA. All these baseline imputation methods take a gene expression matrix as input and output an imputed gene expression matrix with the same dimensions, and the implementation of the baseline algorithms is listed in Supplementary Table S1. Five dimensions were evaluated: imputation accuracy, identification of ‘true zero’ and dropout events, enhancement of intergenic linkage, improvement of intracellular linkage, and computation time.

3. Results

3.1. Accuracy of imputation

To investigate the accuracy of imputation, we simulated a certain percentage of missing values using the real single-cell dataset mentioned above to compare the five imputation methods FRMC, McImpute, ScImpute, 2Dimpute, and ALRA in terms of error and correlation.

For each gene in the original data matrix in the InDrop dataset, the set of cells with non-zero expression is selected, and cells with different masking ratios (2%, 5%, 10%, 20%) are randomly selected based on Bernoulli distribution, and the gene expression of these cells is masked to zero to simulate dropout, forming a new and different dataset. Here Masking Ratio refers to the percentage of non-zero elements in the data matrix that are masked to missing values. Then, five methods are used to impute these four datasets with different masking ratio. The Normalized Root Mean Square Error (NRMSE, defined as $NRMSE=\frac{{∥M-X_k∥}_F}{{∥M∥}_F}$, where $M$ is the real original data matrix, $X_k$ is the imputed matrix. [38]) and Pearson correlation coefficient (PCC) are used to compare the error level and correlation level between the real original data matrix and the imputed data matrix.

Under different mask rate gradients in dataset Indrop-BC6, FRMC has a lower error level with NRMSE as low as 0.522, followed by McImpute method, while other methods show a larger error, especially the ALRA method has the largest error of 0.998 (Fig. 1(a)). In addition, FRMC has a high Pearson correlation coefficient between the imputed data matrix and the true original data matrix, reaching a minimum of 0.838, followed by the McImpute method, while the other methods exhibit lower correlation levels, especially the ALRA method with a correlation coefficient of only 0.6 (Fig. 1(b)). Considering that the large number of zero values included in these matrices may bias the final PCC scores, the scores of PCC after removing zeros of the corresponding positions between the actual matrix and imputed matrix were calculated and the validation was added in both dataset Jurkat-293T and dataset Usoskin (Supplementary Table S2), and the results are consistent with those in the dataset Indrop-BC6. The above results suggested the recovered data values obtained from the missing values in the data expression matrix are closer to the true original values after the processing of FRMC.

Figure 1.

Accuracy comparison of FRMC with other 4 imputation methods on the dataset indrop-BC6. Error levels and correlation levels between the original values and the imputed values of the expression matrices with different artificial random masking ratios (2%, 5%, 10%, and 20%), were measured by using: (a) Normalized root-mean-square error (NRMSE) and (b) Pearson correlation coefficients

3.2. Identification of ‘true zero’ and dropout events

The performance of FRMC and four other algorithms in identifying ‘true zero’ and dropout events was evaluated using the InDrop-BC6 dataset to compare the expression changes of specific genes before and after imputation.

The gene PTPRC (Recombinant Protein Tyrosine Phosphatase Receptor Type C), also known as leukocyte common antigen (LCA) or CD45, is widely present on the surface of leukocyte surface; the cells in the InDrop-BC6 dataset were subjected to fluorescence-activated cell sorting (FACS) purification process, and PTPRC is its fluorescence-tagged gene which is necessarily expressed, so it can be assumed that all zero values of this gene are dropout events in the measured expression data. FRMC and 2Dimpute accurately predicted the deletion value of this gene as dropout event and impute it. In contrast, after the processing of ScImpute and ALRA, the gene expression values still had a large number of missing values without imputation, which indicating that these two methods did not accurately distinguish between ‘true zero’ and dropout events (Fig. 2(a)).

Figure 2.

Violin plots of distributions before and after the imputation of FRMC and other 4 algorithms in different cell type marker genes on the dataset indrop-BC6. (a) PTPRC: universally expressed on all leukocytes; (b) CD68:macrophage-specific; (c) CD4:T cell-specific; (d) CD14: Monocyte-specific; (e) ITGAX: Dendritic cell-specific;(f)GAPDH: a house-keeping gene universally expressed on all human cells

The other gene CD68 is macrophage-specific, and InDrop-BC6 dataset involves multiple types of leukocytes including macrophages, so there were a large number of ‘true zeros’ in the zero values of the gene CD68 in the expression matrix. The four imputation methods, FRMC, 2Dimpute, ScImpute and ALRA, correctly predicted the missing expression values of CD68 gene as ‘true zeros’ without imputation, while McImpute identified all ‘zeros’ in CD68 gene as dropout events and imputed them without exception, which is incorrect and deviates from the actual biological significance (Fig. 2(b)). In addition, we also validated this result on CD4 (a T cell-specific gene,) CD14 (a monocyte-specific gene), ITGAX (a dendritic cell-specific gene) and GAPDH (a house-keeping gene universally expressed on all human cells) (Fig. 2).

Taken together, these comparisons show that only FRMC and 2Dimpute have the best ability to discriminate between ‘true zero’ and dropout events, and only impute missing values that need to be imputed, which is consistent with the actual biological knowledge of the cells and genes in the dataset.

3.3. Enhancement of intergenic linkage

The heavy presence of missing values in the scRNA-seq dataset resulted in much weaker true gene-to-gene associations and did not facilitate the subsequent identification of marker genes in cell subpopulations. To assess whether the algorithm is able to strengthen and recover associations between genes, we used biologically significant co-expressed genes (FTL and CD68 in the InDrop-BC6 dataset) to do a comparative analysis [35].

The gene CD68 in the original dataset showed weak correlation with the gene FTL with the PCC of 0.72. After imputation by the FRMC, McImpute, and ALRA imputation algorithms, the linear correlation between the co-expressed genes FTL and CD68 was enhanced with Pearson correlation coefficients reaching 0.8349, 0.8465, and 0.8954, respectively (Fig. 3). After 2Dimpute’s imputation, the correlation coefficient of this pair of co-expressed genes slightly increased to 0.8019. Worst of all, the expression data lost their original co-expression under the imputation treatment of the ScImpute method, showing a weak correlation. As can be seen, this confirms that FRMC can enhance the association between genes by imputation.

Figure 3.

Scatter plots of co-expressed genes FTL and CD68 before and after the imputation of FRMC and other 4 algorithms on the dataset indrop-BC6. The straight (black) line represents the regression line fitted by a linear model

3.4. Improvement of intracellular linkage

Both PCC within cells before and after imputation and cell clustering maps were analysed as a way to assess whether the imputation algorithm can enhance cell-to-cell correlations.

We used a Jurkat-293T dataset containing two in vitro equal proportions of mixed cell types (Jurkat cell type and 293T cell type). In the unimputed expression data, the mean Pearson coefficients between cells in cell types 293T and Jurkat were 0.74 and 0.71 with standard deviations of 0.056 and 0.054, respectively (Fig. 4). After FRMC imputation, the mean PCC between cells in both cell types 293T and Jurkat increased to 0.82 and 0.80, respectively, and the standard deviations decreased to 0.039 and 0.036, respectively (Fig. 4), illustrating that FRMC enhanced the intracellular correlations (i.e. intracellular connections) between the two cell types Jurkat and 293T. The other four imputation algorithms also enhanced intracellular correlations to some extent (Fig. 4).

Figure 4.

Intra-cellular correlation between ‘Jurkat’ and ‘293T’ cell types before and after the imputation of FRMC and other 4 algorithms on the dataset Jurkat-293T. Error bars represent 1 SD

To further investigate the accuracy of FRMC algorithm to enhance intracellular linkage, we randomly selected 50 cells from the Jurkat-293T dataset for demonstrating the clustering of cells before and after imputation. In the following, the cell name with ‘_1’ identifier stands for cell type ‘293T’, and the cell name with ‘_2’ identifier stands for cell type ‘Jurkat’. Fifty cells were randomly sampled from the original gene expression matrix, and the same 50 cells were extracted from the FRMC-treated gene expression matrix, and subsequently the two new data were analysed by cell clustering using the R package pheatmap. It was found that incorrect clusterings occurred in the clustering graph of the data before imputation, that cells CL2505_1 and CL2019_2 had their separate clustering branches. Presumably, this is due to a large number of missing values in the single-cell dataset resulting in diminished intracellular correlation. However, after imputation by the FRMC algorithm, both cells CL2505_1 and CL2019_2 were correctly classified into their true cell types, and the clustering relationships between the two types of cells were more clearly defined (Fig. 5). Obviously, after the imputation process of FRMC, the single-cell gene expression matrix can more accurately reflect the true intracellular connections and facilitate the downstream single-cell analysis.

Figure 5.

Cell clustering map based on gene expression before and after the imputation of FRMC on the dataset Jurkat-293T. Yellow cell names identified with ‘_1’ represent cell type ‘293T’, and blue cell names identified with ‘_2’ represent cell type ‘Jurkat’

3.5. Runtime

We evaluated the computational efficacy of the algorithms on six single-cell datasets, including four datasets with cells less than 10,000 (Jurkat-293T, InDrop-BC6, Usoskin, and Smart-Seq2) and two large-scale single-cell datasets Indrop-BC4 with about 13k cells and NeuronCell with more than 100k cells. These datasets have different data characteristics: number of cells and genes, and the percentage of missing values, with the dataset Indrop-BC4 yielding the highest percentage of missing values at 94.22% and the number of genes at 9218. On the five datasets except the dataset NeuronCell, we implemented five algorithms with 2Dimpute and ScImpute setting the number of parallel CPU cores parameter (2Dimpute setting cores to 22, i.e. using 22 CPU cores in parallel, and ScImpute setting cores to 20) and the other algorithms not setting the number of CPU cores. Specifically, on the dataset NeuronCell, because ScImpute and 2DImpute methods require more memory as the set cores increase, and the memory of the machine cluster used for testing is 128 G. When processing dataset NeuronCell in this environment, ScImpute and 2DImpute would fail to run due to the insufficient memory if the ncores was set higher than 1, thus the ncores could only be set to 1.

Compared among the datasets with close cell counts (Smart-Seq2, InDrop-BC6, and Jurkat-293T), the algorithm has a longer runtime on the Smart-Seq2 dataset, which has the higher zero ratio among three datasets (Table 1), explaining the higher imputing difficulty of the single-cell dataset with a high number of genes and a high percentage of missing values. On this dataset, ALRA can finish running in as fast as 0.8 minutes; the new method FRMC is second only to ALRA and can be completed in 2.5 minutes, which is close to the runtime of McImpute; the most time-consuming algorithm is 2Dimpute which take 6021 minutes. Furthermore, the FRMC method can also impute successfully in large-scale single-cell datasets. When dealing with datasets InDrop-BC4 with 13,266 cells, FRMC run second only slower than ALRA; and when dealing with datasets NeuronCell with 100,000 cells, FRMC is one of the only three methods that can run successfully within 7 days.

Table 1.

The runtime of FRMC and other four algorithms on six different datasets

Datasets	Cells×Genes	Zero ratio	Runtime (min)
			FRMC	McImpute	ScImpute	2DImpute	ALRA
Smart-Seq2	2987 × 16,727	72.40%	64.5	64.2	58.9	6021	2.5
Indrop-BC6	3498 × 2423	70.03%	21	19.5	34	204	0.8
Jurkat-293T	3388 × 3702	36.00%	10.1	28	72.8	448	1
Usoskin	622 × 10,554	64.40%	2.5	3	13	504	0.8
InDrop-BC4	13,266 × 9218	94.22%	203.7	838	224	4675	4.5
NeuronCell	100,000 × 8746	78.16%	9703	1766	NA*	NA*	29

*runtime longer than 7 days (10,080 minutes).

4. Discussions

The high-resolution feature of single-cell transcriptome sequencing technology allows researchers to observe cellular gene expression profiles at the single-cell level, offering numerous possibilities for subsequent studies, but also poses a serious problem of missing data, with expression matrices of high missing values instead hindering downstream analysis. To address this problem, it is essential to develop a more rapid and stable data imputation method with greater accuracy, which should not only be able to recover the missing data, but also effectively facilitate the subsequent biological mechanism analysis.

Although several relatively mature imputation algorithms exist for processing scRNA-seq data, each of them has its own bias and drawbacks. As mentioned earlier, a priori-type algorithms based on statistical models need to make appropriate statistical distribution assumptions on the data to be processed, but such algorithms also do not adopt uniform distribution models, such as gamma-normal mixed distribution (ScImpute), Poisson-gamma mixed distribution (SAVER) or zero-inflated Poisson distribution (VIPER). Evidently, there does not exist an established distribution model that can accurately describe all scRNA-seq features. Therefore, we are eager to seek a more ubiquitous algorithm that can solve this problem unbiasedly. In fact, imputation is a technique that is widely used in various fields, such as Netflix’s movie recommendation system, speech recovery, and image reconstruction [39–42]. The cardinal goal of all these algorithms is to recover an unknown low-rank matrix from a very limited amount of information. Further, whether the imputation problem of scRNA-seq data can be transformed into a problem of solving low-rank matrices is the key to determine whether the imputation techniques from other fields described previously can be applied to scRNA-seq data processing.

Numerous studies have confirmed that gene expression does not behave in isolation, but is interdependent as part of a complex network structure [43–45]. Different genes may be highly correlated with each other due to shared regulatory mechanisms, functional interdependence, or possessing the same tissue origin [46,47]. It has also been suggested that it is reasonable and feasible to assume that the expression values of genes lie in a linear subspace of low dimensionality [22]. Therefore, a low-rank matrix model is feasible for imputing single-cell transcriptome data matrices. There are various alternative approaches in order to solve this low-rank matrix model. One is the singular value thresholding iterative method to solve NNM [32], which mainly preforms singular value shrinkage iterations, namely, a soft-thresholding technique on the singular values of iterative matrices at each step until a stopping criterion is reached. It is efficient at addressing NNM problem, but it requires a very large number of iterations to converge, which limits its applicability. The other is FPCA method (Fixed Point Continuation with Approximate SVD) [48], which incorporates fixed point continuation into a Bregman iteration, but has limitations in solving large-scale NNM problem. Besides, the accelerated proximal gradient approach [49] is an accelerated proximal gradient singular value thresholding algorithm that is relatively efficient and robust in solving large-scale NNM problem. However, it has only sub-linear convergence speed theoretically. Based on this, FRMC chose a more efficient way by using the existing optimization theory and the Lagrangian method and then solved the transformed problem through a singular value thresholding iteration.

To address the shortcomings and limitations of existing imputation methods for single-cell transcriptomics datasets, we have developed a new, fast imputation software FRMC that transforms the single-cell gene expression matrix imputation problem into a low-rank matrix optimization problem, whose kernel innovation is using a fast and accurate singular value threshold approximation method combined with a mathematical optimization model for low-rank matrix recovery and a ‘true zero’ prediction algorithm. It is worth mentioning that some algorithms integrate the imputation step into other data processing processes, performing imputation simultaneously in processes such as normalization, batch effect correction, and clustering [50,51]. While FRMC is a separate session used to process dropout events and can be flexibly integrated into the single-cell data analysis pipeline to allow researchers to analyse with their own data characteristics and adopt appropriate correction strategies. FRMC is implemented in Python and is freely accessible to non-commercial users on GitHub: https://github.com/HUST-DataMan/FRMC. Also, FRMC can be flexibly integrated into single-cell data analysis pipelines.

The results revealed that FRMC can not only accurately distinguish ‘true zero’ from dropout events and correctly impute the missing values generated by dropout events, but also can effectively enhance intracellular and intergenic connections and achieve accurate clustering of cells. In addition, its operation speed is relatively fast. In conclusion, FRMC is an optimized scRNA-seq imputation software that combines accuracy and rapidity performance compared with existing 2Dimpute, ScImpute, McImpute and ALRA imputation algorithms.

Finally, FRMC also has its limitations. It can currently only be used to impute missing values in single-cell transcriptomic data and is not suitable for other datasets. In the future, we will extend the FRMC method to more application scenarios, such as for solving the imputation problem of GWAS genotype data [52] and also for solving the imputation problem of missing values in macro-genomics data [53]. Another room for improvement lies in the use of parallelization to accelerate the operation of the algorithm.

Biographies

H W, X W and M C designed the study, summarized the findings, interpreted the data and drafted the manuscript.

X W implemented the algorithms and the computational analyses.

R X did visualizations.

K Z supervised the project.

Disclosure statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary material

Supplemental data for this article can be accessed here.