Single-cell differential expression analysis between conditions within nested settings

Abstract

Differential expression analysis provides insights into fundamental biological processes and with the advent of single-cell transcriptomics, gene expression can now be studied at the level of individual cells. Many analyses treat cells as samples and assume statistical independence. As cells are pseudoreplicates, this assumption does not hold, leading to reduced robustness, reproducibility, and an inflated type 1 error rate. In this study, we investigate various methods for differential expression analysis on single-cell data, conduct extensive benchmarking, and give recommendations for method choice. The tested methods include DESeq2, MAST, DREAM, scVI, the permutation test, distinct, and the t-test. We additionally adapt hierarchical bootstrapping to differential expression analysis on single-cell data and include it in our benchmark. We found that differential expression analysis methods designed specifically for single-cell data do not offer performance advantages over conventional pseudobulk methods such as DESeq2 when applied to individual datasets. In addition, they mostly require significantly longer run times. For atlas-level analysis, permutation-based methods excel in performance but show poor runtime, suggesting to use DREAM as a compromise between quality and runtime. Overall, our study offers the community a valuable benchmark of methods across diverse scenarios and offers guidelines on method selection.

Introduction

Differential gene expression analysis is a fundamental task in single-cell and bulk transcriptomics to detect differences between sample groups. While tools for bulk RNA-seq have matured over the last two decades, there is currently no consensus on differential gene expression analysis on single-cell RNA-seq (scRNA-seq) datasets. In the past, simple statistical tests, such as t-tests or Wilcoxon tests, have commonly been applied to compare expression levels between groups of cells. Since cells of the same sample are not independent observations, this violates the assumption of independence underlying most statistical tests. Ignoring this assumption results in pseudoreplication bias, which can lead to false-positive results [1, 2].

One solution to avoid pseudoreplication bias is to aggregate the single-cell data by sample and perform statistics on the resulting “pseudobulks.” [2] To retain the advantage of scRNA-seq, this can be done for each cell-type separately. Moreover, linear mixed effects (LME)-models explicitly model the biological replicate as a random effect [1, 3, 4]. Alternatively, resampling-based methods like distinct [5] have been proposed.

Since hierarchical bootstrapping has been applied to similar problems in other scientific fields [6, 7], we adapt hierarchical bootstrapping to single-cell data and compare it with LME-models and bulk differential gene expression analysis methods using both simulated and real-world data, covering different analysis scenarios. We provide a comprehensive benchmark, including advanced simulation scenarios, such as atlases, highly unbalanced datasets, and varying proportions of differentially expressed genes. In contrast to previous benchmarks, which concentrated on simpler simulated datasets, our study offers deeper insights into the performance and applicability of these methods in complex biological contexts and guidelines for the community (Fig. 1) [1, 4].

Figure 1.

Flowchart of the benchmarking pipeline. Differential gene expression testing methods are applied to simulated and real datasets. The data are preprocessed or simulated and preprocessed. A correlation analysis (supplementary note 2) and a runtime and performance benchmark, as well as a negative control, are then performed. For determining the performance on the real dataset, the Reactome database is used [33]. Additionally, the reproducibility of the methods across datasets is evaluated on the lung cancer atlas [27].

Statistical framework and consequences of pseudoreplication

Historically, simple statistical procedures such as the t-test or Wilcoxon rank-sum test have been commonly used to compare gene expression between groups of cells [8]. For single-cell studies this naive practice treats every cell as an independent observation and can therefore drive the false discovery rate far above the nominal level [1, 9, 10]. Since cells obtained from the same biological sample generally constitute dependent observations, they are pseudoreplicates, rather than actual independent replicates. Ignoring this dependence violates the independence assumption that underlies these tests, leading to underestimated variability and inflated type 1 error, which ultimately compromises the reproducibility of statistical findings [11, 12].

Single-cell data arise from a two-stage sampling design. First, one draws an independent sample of biological specimens, and extracts a tissue piece from each, and second, one profiles many individual cells from every extracted tissue piece. The resulting hierarchical structure induces group dependencies among cells that originate from the same sample [13]. Statistical procedures that ignore this structure implicitly treat the total number of profiled cells as the effective sample size, even though the true number of independent experimental units is the number of samples . Those inflated sample sizes can lead to underestimated standard errors as well as overconfident statements on statistical significance of observed effects and an inflated type 1 error rate [9].

A mixed effect formulation makes the problem explicit. Let

denote the expression of gene in cell from sample under condition . The fixed effects capture the mean expression per condition. The random sample effect accounts for variation at the level of the sampled tissues, and the residual cell noise represents within sample variability. For simplicity assume normal distributions with and all mutually independent [14].

The total variance is and consists of two components, the sample-level and cell-level variability. For each gene , this induces a correlation

where and refer to two cells and from the same sample. This intraclass correlation coefficient is the fraction of variance attributable to between sample heterogeneity. Cells from the same sample can only be independent if there is no sample-level variation. Consider two conditions with samples per condition and cells per sample. Writing for the average expression of gene within condition , the variance of the difference in means estimator is

The multiplier shows how strongly correlation inflates variance. For and the variance grows by a factor of 50, turning nominally significant findings into noise. In the extreme case a two sample t-test would wrongly use degrees of freedom instead of the correct . Mixed models or sample level aggregation (pseudobulks) mitigate this bias because they treat the sample, not the cell, as the fundamental replicate.

Glossary

Linear model A linear model, or more specifically linear regression, estimates the linear relationship between a response variable () and one or more explanatory variables (), by choosing the parameters () such that the error term () in the following equation is minimal: . The fundamental assumption is that a constant change in one of the explanatory variables leads to a constant change in the response variable (i.e. “linear relationship”).

Generalized linear model A generalized linear model is an extension of linear models that allows for response variables that follow a distribution incompatible with the assumption of a linear relationship, such as binary variables or exponential distributions. Gene expression data are often modeled as a response variable that follows a negative binomial distribution.

Variance Stabilizing Transform (VST) For some distributions, the variance depends on the mean. Specifically, in the case of gene expression count data, the higher the gene expression, the higher the variance. VST transforms the data to ensure variance is independent of the mean, allowing the application of simpler statistical models to the data.

Wald test The Wald test assesses the statistical significance of a regression coefficient () by comparing the estimated coefficient with its standard error.

Empirical Bayes shrinkage Empirical Bayes shrinkage is a technique to improve parameter estimates by leveraging information from all variables (i.e. genes), which is particularly useful when there are not enough replicates to reliably estimate the parameter. An initial estimate of the parameter based on the whole dataset is used as prior, which is updated using Bayes’ rule with the data seen for the specific variable.

Random effects See mixed effects.

Mixed effects A linear mixed effects model includes both fixed and random effects and is typically used for accounting for nested structures, such as repeated measurements. The fixed effects are captured by explanatory variables’ coefficients like in a standard linear regression model. The random effects part controls for heterogeneity that is not correlated with the explanatory variables.

Bayes factor In testing a null hypothesis against an alternative hypothesis , the null hypothesis is typically rejected if the -value falls below a significance threshold (typically 0.05). Bayesian statistics, instead, compares the likelihood of two and . The ratio of the likelihoods is called Bayes Factor. Bayes factors can be interpreted more gradually rather than focusing on a single significant threshold, do not require adjustments for multiple hypothesis testing, and are arguably more appropriate than -values for exploratory analyses [15].

Differential gene expression analysis methods

We distinguish between methods that can be applied to single-cell data directly and methods that require pseudobulking to mitigate the pseudoreplication bias [1]. While parametric methods assume a certain data distribution, nonparametric tests can be applied to data independent of its underlying distribution (Table 1). If the underlying assumptions are met, parametric tests are considered more powerful than nonparametric ones [16].

Table 1.

Methods partitioned by the statistical framework and used input data.

	Parametric	Nonparametric
Pseudobulk	DESeq2 [17]	Permutation est
	DREAM [18]
Single-cell	MAST [19]	Hierarchical bootstrapping distinct [5]
	scVI [20]
	t-test [21]

DESeq2

DESeq2 [17] is one of the most widely used methods for differential expression analysis of bulk RNA-seq data. It models gene counts as a negative binomial distribution and applies empirical Bayes shrinkage to calculate gene-wise dispersion estimates—using information from all genes to stabilize the dispersion estimates, especially for genes with a low expression. By default, it uses the Wald test to obtain -values for each gene. Based on a linear model, it allows complex designs, including continuous and categorical covariates, but no random effects.

DREAM

DREAM (Differential expression for repeated measures) [18] is an extension of the limma/voom workflow [22] for bulk RNA-seq analysis. On top of continuous and categorical covariates, it allows modeling mixed effects such as multi-experiment data or repeated measures of the same individual. First, this workflow applies a variance-stabilizing transformation (VST) to the gene counts. Then, a linear mixed effects model is fitted to the transformed values, and an empirical Bayes shrinkage is applied to the variance estimates, similar to DESeq2.

Permutation test

The permutation test is a simple, nonparametric test for comparing the distribution between two groups of samples. It was proposed by Fisher as early as the 1930s [23]. Even though it is agnostic of the underlying distribution, it still assumes independence of the samples. Therefore, it cannot be applied to single cells directly but requires pseudobulk-aggregation. When applied to differential gene expression analysis, the difference in means between the two groups is calculated for each gene. Then, the group labels are randomly shuffled times to obtain a background distribution of the difference in means. The -value (the probability that an observed difference could have occurred by chance) is the number of permutations that resulted in a greater or equal difference than the one observed divided by . Of note, the smallest -value achievable by this method is , making this test computationally expensive, especially in combination with multiple testing correction.

MAST

MAST [19] is a generalized linear mixed effects model for single-cell data. MAST allows explicitly modeling biological replicates as a random effect, thereby treating single cells as repeated measures of the same sample [1]. It employs a two-component hurdle model that treats single-cell data as “zero-inflated,” i.e. it assumes that zeros are observed more often than expected by a standard distribution. The first component models the discrete expression rate of each gene across cells, and the second component captures the continuous gene expression level conditioned on the expressed gene. MAST expects variance-stabilized counts as input.

scVI

scVI [20, 24] is an autoencoder for single-cell data, i.e. an artificial neural network that learns a low-dimensional embedding of the single-cell data while removing batch effects and other technical covariates. It has excelled in a benchmark assessing batch-effect removal [25] and has been successfully used to build tissue and disease atlases with >1 million cells from heterogenous datasets [26, 27]. From the decoder network, posterior distributions of the corrected gene expression can be retrieved and used for Bayesian differential gene expression analysis. To this end, the probability of two hypotheses ( and ) is calculated and compared using a Bayes factor. The Bayes factor is used to rank the genes instead of a -value.

Distinct

Distinct [5] is a nonparametric method for differential distribution analysis. Unlike the other methods mentioned in this work that focus solely on changes in the mean, it can detect other changes in the gene expression distribution that do not, or only subtly, change the mean, such as an increase of variance or shifts between bimodal distributions. To do so, it computes an empirical cumulative distribution function (ECDF) for each sample based on the expression of single cells. To compare two groups, the ECDFs of all samples in a group are averaged, and test statistics capturing the distance between the two curves are calculated. A null distribution of the test statistics is obtained using a permutation approach to derive -values. While distinct can detect changes in distributions beyond their mean, we only use distinct to compare means in this work.

Hierarchical bootstrapping

Hierarchical bootstrapping is a nonparametric method for obtaining confidence estimates for differences between groups that has been successfully applied in other fields for dealing with complex, hierarchically structured data [7]. We have adopted this approach to single-cell transcriptomics. Given a dataset with a specified hierarchical structure (e.g. cell < patient < dataset), we randomly sample with replacement for each condition bootstraps with an equal number of cells from each category of all categorical covariates. We obtain empirical gene expression distributions from these samples for both conditions. To obtain a -value, we count the number of samples in which the first condition exceeds the second condition and vice versa and divide the smaller of the two values by the number of iterations . A more detailed description of our implementation of hierarchical bootstrapping, including pseudocode, is available in supplementary note 3. Similar to the permutation test, the smallest possible -value is limited to . We adopted an adaptive sampling strategy to optimize computational efficiency where, initially, 10 000 bootstrap iterations are drawn for each gene. For genes achieving the smallest possible -value in this step, the number of iterations is increased to 100 000.

t-test

The t-test is a simple parametric method that compares the means of gene expression between groups [21]. It assumes statistical independence of observations, which means it does not control for pseudoreplication bias when applied to single-cell data, where cells within a sample are not independent. In this work, the t-test is included as a baseline method to evaluate the performance of more advanced approaches. While its simplicity and computational efficiency are advantageous, its inability to account for nested data structures limits its suitability for single-cell differential expression analysis.

Benchmark

Single-cell analysis is a fast-evolving field in which many conceptually similar methods have been published. We limited our study to include representative and widely used methods for single-cell differential expression analysis that address pseudoreplication bias in one way or another, including parametric methods like linear models and linear mixed effects models, as well as nonparametric methods (see Table 1). To facilitate future extensions, our benchmark was implemented as a fully reproducible Nextflow pipeline with containerized environments [28], making it easy for method developers to integrate additional tools.

We simulated data using a modified version of the R package splatter with its extension splatPop [29, 30]. To corroborate the findings, we further assess the methods’ performance on experimental data [31, 32], where we consider Reactome pathway genes as ground truth [33]. In addition to the runtime, we assess the false positive rate (FPR) of the tested tools by simulating data without differentially expressed genes and permute the labels of the experimental data. An overview of all benchmark tasks is provided in Fig. 1.

Materials and methods

The benchmark is implemented as a Nextflow pipeline [28] with predefined containerized environments for each process. Data simulation and plotting, along with the methods DESeq2, distinct, DREAM, and MAST, are implemented in R. All other methods and processes are implemented in Python. Genes expressed in of the cells were removed from the expression matrix because MAST and its linear model require a certain proportion of cells with expression for each gene to work reliably. The individual methods partly require specific preparation steps, where we follow the recommendations in their documentation or vignette. For methods expecting bulk RNA-seq data as input, such as DESeq2, the permutation test, and DREAM, we created pseudobulk samples as detailed below.

Simulation

We selected Splatter and SplatPop as the basis for our simulation due to its extensive customization options, support for nested hierarchies, and comprehensive documentation. We adapted SplatPop with respect to the condition assignment for this work (https://github.com/LeonHafner/sc-guidelines). The simulation process can be divided into two parts. First, we generate the gene means for each sample and the condition-specific expression of the differentially expressed genes. Second, we generate a selected number of cells and expression values for each sample and gene.

Dataset scenario

The dataset scenario serves as a balanced baseline with a single batch and consists of two groups (e.g. diseased patient versus healthy control) with five samples of 250 cells each. We simulated datasets with 5000 genes of which either 250 () or 25 () were differentially expressed between the two conditions. The fold change for the differential expression is drawn from a log-normal distribution with a mean of and a standard deviation of . The quantile of the values is and the quantile is , thus providing favorable conditions for detecting the vast majority of differentially expressed genes. Unless otherwise noted, these fold-change values apply to this simulation and all other datasets simulated in this work.

Atlas scenario

The atlas consists of 3750 cells evenly distributed across three batches of five samples each. Each batch contains at least one sample from each of the two conditions to facilitate comparisons within the batch. The batch effects for the three batches were drawn from log-normal distributions with the following parameters for location and scale: , and . The resulting log-normal distributions display little to no overlap, ensuring clear differences between batches.

Dataset with varying cell numbers scenario

The above scenarios assume a constant number of cells, which does not reflect reality. We hence investigate a modification of the dataset scenario with a variable number of cells per sample taken from a gamma distribution with shape and rate and rounded to integer values. These parameters were chosen to ensure that samples with high and low cell counts are simulated, resulting in large differences with a mean cell count of and a standard deviation of . Pseudobulk samples, in particular, typically neglect the number of cells of the contributing samples, possibly leading to an uneven weighting of differential expression. The distribution of the cells per sample is shown in Fig. 2B.

Figure 2.

Simulated scenarios. (A) shows an overview of the four main scenarios simulated for this study. (B) depicts a histogram of the number of cells in all samples of 20 simulations of the dataset with varying cell numbers per sample scenario. These numbers were drawn using a gamma distribution with shape 0.8 and rate 0.0035.

Atlas with unbalanced conditions scenario

In the plain atlas scenario, we consider evenly balanced conditions within batches. To assess if methods are biased by strongly unbalanced batches, we modified the Atlas scenario. We considered a batch consisting of nine samples of condition 1 and one of condition 2 and vice versa in a second batch.

Pathway perturbation dataset

To further assess real-world performance, we used a publicly available scRNA-seq data set with peripheral blood mononuclear cells from eight patients [32]. Here, each sample was divided into two aliquots before sequencing. One aliquot was stimulated with interferon-beta, while the other served as a control. Since the interferon-beta pathway is well studied, we can use the pathway (obtained from the Reactome database [33], Fig. S1) as a silver standard for differential gene expression analysis. Following the example of the seurat vignette [31], we considered only B cells to avoid cell type bias in the analysis. In addition, genes expressed in of cells were removed, resulting in the detection of 1615 genes across 1319 cells.

Negative control

To assess how many false-positive genes are detected, we repeated the atlas simulation scenario without conditions, i.e. we expect no significant difference between the arbitrarily assigned conditions. Similarly, we repeated the analysis of the real data set after randomly permuting the condition labels. We count all differentially expressed genes as false positives and assess method performance systematically across different significance thresholds. Since scVI employs Bayesian statistics instead of generating classical -values, we evaluated its results at 2000 equidistant cutoffs using the “target FDR” parameter. We relied on scVI’s classification of genes based on these cutoffs.

Performance evaluation

Genes we expect to be differentially expressed and recognized by a method are counted as true positives (TP). Similarly, genes that we do not expect to be differentially expressed and are not reported as such are counted as true negatives (TN). Falsely classified genes are counted as false positives or false negatives (FN), respectively [34]. To compare the methods in each scenario, a precision-recall curve (PRC) is used instead of the widely used Receiver-Operating-Characteristic (ROC) because it allows a reasonable interpretation even for very unbalanced data [35, 36]. This is particularly important in this classification problem since only (or ) of genes are differentially expressed. Moreover, our data may contain genes that show random differential expression. Hence, precision, which focuses on TPs, is likely less biased. We repeated our evaluation of the area of the precision-recall curve (AUPRC) over 20 independent simulations to ensure robustness.

Performance evaluation on real-world data

Since there is no ground truth for the real dataset, and it is not known how strongly each gene is differentially expressed, we rely on the Reactome interferon-beta stimulation pathway as a silver standard (Fig. S1, R-HSA-909733) [33]. A total of transcripts from the Reactome database can be found in our dataset. These genes are assumed to be differentially expressed by interferon-beta stimulation and are used as ground truth for creating precision-recall curves.

Method reproducibility

We evaluated the reproducibility of the tested methods using the lung cancer atlas [27], which includes eight datasets containing samples from both of the two conditions: lung adenocarcinoma and lung squamous cell carcinoma. We discarded cells from other conditions. Additionally, we randomly selected 5000 genes to achieve a reasonable runtime for all tested methods. Each dataset was then divided into smaller subsets, with each subset containing no more than 4000 cells. We made sure that each subset included at least 20 cells from each sample in the original dataset to preserve the balance of the original data. This process resulted in a total of 42 final subsets for analysis. Each method was then applied to every subset, and the reproducibility of the results was assessed based on the overlap of the top N genes ranked by their -values. We quantified reproducibility using the Jaccard index, a metric that measures the similarity between two sets. Comparisons were performed with a fixed and variable value of N, which allowed us to assess the consistency of results within and across methods.

Runtime benchmark

We fixed the number of genes to 1000 and investigated each method’s runtime across an increasing number of cells; vice versa, we fixed the number of cells and increased the number of genes. Each combination was tested times and averaged to obtain the final runtime. We further considered the additional runtime needed for preprocessing and pseudobulking. The runtime was measured using the Unix system time during the execution of each process in the Nextflow pipeline. Each Nextflow task was executed on a single core for comparability, despite some methods supporting multi-core usage. The HPC used for generating the figures consists of several machines, each equipped with an Intel Xeon Gold 6148 processor with a clock speed of 2.40 GHz.

Method implementations

We have structured the methods as separate processes within a Nextflow pipeline, where each method has its own containerized environment tailored to meet the recommended use. Detailed descriptions of the implementations and specifications for each benchmarked method are provided in supplementary note 1.

Results

Performance evaluation on simulated scenarios

Before utilizing the simulated data for benchmarking, we conducted a correlation analysis. This analysis confirmed the presence of pseudoreplication bias in the simulated data, as the transcriptomes of cells within a sample were more highly correlated than those of cells between samples (supplementary note 2). We then evaluated the performance of the methods on the simulated scenarios described in Fig. 2A. Most methods performed well on the dataset scenario, with DESeq2 leading the field with a mean AUPRC of 0.93 followed by scVI with a mean AUPRC of 0.87 and the remaining methods ranging between a mean AUPRC of 0.71 and 0.81 (Fig. 3A).

Figure 3.

Performance of the methods on the simulated data scenarios. Performance of the methods on the simulated scenarios over 20 independent simulations and runs each for (A) the Dataset scenario, (B) the Atlas scenario, (C) the Dataset scenario with varying cell numbers, and (D) the Atlas scenario with unbalanced conditions. The methods were sorted based on their performance. The random baseline is indicated by the dashed line. Full precision-recall curves for a randomly selected run are shown in Figure S7. We excluded distinct from those benchmarks, as it computed the lowest possible -value for the majority of the genes and therefore did not provide useful results.

In the Atlas scenario, the methods generally performed worse, likely owing to the batch effects. Here, the permutation test performs best with an AUPRC of 0.78. DREAM, DESeq2, hierarchical bootstrapping, and MAST perform comparably with a mean AUPRC between 0.73 and 0.77, while scVI and the t-test showed a significantly worse performance AUPRC of 0.66 and 0.68, respectively (Fig. 3B). ScVI’s poor performance in this scenario is surprising, as the batch information was provided using the batch_key parameter and sample information was passed using the categorical_covariates parameter.

Since making pseudobulks from samples with identical cell numbers seems unrealistic in practice, we used the Dataset with varying cell numbers scenario to ask how different cell counts per sample impact the results. Overall, the results are comparable with the regular dataset scenario, except the performance of scVI, which is slightly worse, and the t-test, which scores last (Fig. 3C). Therefore, we conclude that at least within the range of cells per sample tested here, varying numbers of cells per sample only have a minor impact on the results of a pseudobulk DE workflow. For the parametric pseudobulk methods, we additionally tried passing the logarithmized number of cells in a pseudobulk as a continuous, fixed effect covariate to the methods. However, this resulted in significantly worse performance (Fig. S2).

It is not always possible to obtain single-cell data in a well-controlled experimental setting with (almost) equal number of samples in each condition, especially when compiling a single-cell atlas from heterogeneous data sources [27]. In the atlas with unbalanced conditions scenario, we tested how methods perform in an extreme case, with only one sample per batch being from the first or second condition, respectively. As in the atlas scenario, the permutation test achieves the best result with a mean AUPRC of 0.94, followed by hierarchical bootstrapping and the t-test with an AUPRC of 0.81 and 0.77. MAST and DREAM achieve a mean AUPRC between 0.73 and 0.74, DESeq2 of 0.65, and scVI performs the worst with an AUPRC of 0.22 (Fig. 3D). While some methods show a better performance on the unbalanced atlas than on the balanced atlas, this might be due to the different number of batches. Noticeably, the two nonparametric methods (permutation test and hierarchical bootstrapping) have an edge over parametric methods in this scenario, while in particular scVI appears to have difficulties. The reason might be that in this extreme case (only one sample per batch for one of the groups), parametric methods struggle to obtain reliable parameter estimates. While the permutation test does not have these issues, its good performance was still surprising to us, as it does not account for batch effects, which could, in theory, lead to false-positive results when the change in gene expression is contradicting between the two datasets (e.g. downregulated in batch 1 and upregulated in batch 2). An inspection of the simulated gene expression revealed that very little such contradicting cases are present within the set of the truly differentially expressed genes (Fig. S3), therefore, explaining the good performance of the simplistic permutation test.

We additionally examined the change in performance with only 0.5% (instead of 5%) differentially expressed genes (Fig. S4). Expectedly, the performance of all methods decreased. In the dataset scenario, parametric methods tended to be less affected than the nonparametric ones. Furthermore, we determined the performance of the methods on filtered datasets using only 10% of the genes with the highest variability, which was recommended by the authors of scVI. However, this led to a decreased performance of all methods, including scVI (Fig. S5).

Distinct’s performance did not outperform random baseline in any of the scenarios (Fig. S6). We observed that running distinct resulted in a -value of 0.0001 for 63% to 89% of the genes, which is the minimal achievable -value at default setting limited by the number of permutations.

Evaluation on pathway perturbation dataset

We applied the methods to a single-cell dataset of 1319 B-cells partially stimulated with interferon-beta, considering the genes in the corresponding Reactome pathway as differentially expressed. In this setting, the t-test and MAST perform best with an AUPRC of 0.302 and 0.282, closely followed by scVI, DESEq2, and DREAM with an AUPRC between 0.25 and 0.27. The permutation test achieves an AUPRC of 0.11, while both hierarchical bootstrapping and distinct achieve an AUPRC of 0.073. Notably all three nonparametric methods perform worse than the parametric methods on this scenario. We acknowledge the limitations of using an imperfect ground-truth. Reactome contains all genes known to be involved in interferon-beta stimulation. While this pathway is well studied, there might still be further genes involved. Additionally, secondary genes outside the pathway could be perturbed, too. Moreover, not all genes necessarily alter their expression level upon pathway stimulation, or could be regulated at protein rather than mRNA level. In this light it is not surprising that the methods achieve a significantly worse performance than in a well-controlled simulation experiment. Nevertheless, it remains informative to assess the relative ranking in this setting.

Negative control shows that not all methods are well-calibrated

In this section, we performed a “negative control” benchmark to assess if the methods appropriately control for false discoveries when no differentially expressed genes are present by comparing -value cutoffs with the observed false discovery rate. This allows us to detect whether the methods were overly conservative in their classification. DREAM and the permutation test aligned very well with the diagonal in the simulated data (Fig. 5), making them neither too conservative nor too liberal. DESeq2 and MAST slightly exceed the nominal FDR in the range of small -values. scVI is over-conservative for -values but exceeds the nominal FDR for all cutoffs above. Hierarchical bootstrapping exceeds the nominal FDR at all cutoffs and the -values of the t-test are inflated, as expected. For the real dataset with permuted labels (Figure S8B), the trends were similar; the permutation test and DESeq2 appeared slightly conservative, while DREAM and MAST controlled their FDR reasonably well.

Figure 5.

Negative control based on raw -value cutoff and FPR. This plot shows -value cutoffs versus the observed FPR (defined as the proportion of genes falsely classified as positive). In a perfect case, the observed FDR corresponds to the -value cutoff. Values below the diagonal indicates that a method is over-conservative, while values above the diagonal indicate that a method does not appropriately control for false discoveries. Since scVI does not generate classical -values but employs Bayesian statistics, we passed 2000 equidistant cutoffs to scVI via its “target FDR” parameter and relied on scVI’s classification of positive and negatives. This plot only shows the results up to a -value of 0.15, the plot over the entire axis range can be seen in Fig. S8A.

Figure 4.

Precision-recall plot of the performance on a real data set. Performance of all methods on a single-cell dataset of 1319 B-cells stimulated with interferon beta. The corresponding Reactome pathway serves as the ground truth for determining the precision-recall curve.

Method reproducibility

To assess the reproducibility of the methods, we first analyzed the pairwise Jaccard indices across 42 subsets of the lung cancer atlas with considering the top genes (Fig. S10). The results revealed that all methods, except scVI, achieved a median Jaccard index below 0.25, indicating relatively low reproducibility across subsets. In contrast, scVI stood out with a notably higher median Jaccard index of 0.39. To investigate this pattern further, we examined how reproducibility changes with varying (Fig. S11). Most methods exhibited similar trends. However, scVI consistently demonstrated higher Jaccard indices across nearly all values of , underscoring its distinct behavior in maintaining greater overlap across subsets. Finally, we explored reproducibility within and between methods (Fig. S12). This analysis revealed that splits from the same dataset were generally more similar to each other than splits from different datasets of the lung cancer atlas, highlighting that differential gene expression across datasets in complex diseases and tissues can lead to quite different results irrespective of the method used. We note that results from scVI show very low agreement with all other methods and that reproducibility alone does not imply that the results are accurate.

Runtime benchmark

We first varied the number of genes while maintaining a constant cell count of 1000 (Fig. 6A). DESeq2, DREAM, and the t-test display only minimal linear increase in runtime, despite a 100-fold increase in the number of genes. In contrast, all other methods exhibit a steeper, yet still linear, increase in runtime with higher gene numbers. The permutation test is slowest taking almost 5 h to run for 10000 genes, followed by hierarchical bootstrapping, MAST, distinct, and scVI.

Figure 6.

Runtime comparison of the methods. The plots show the runtime of the different methods averaged over 10 independent runs. Although pseudobulking had a minimal impact on the overall runtime, it was included in the assessments for DESeq2, the permutation test, and DREAM, as these methods work with pseudobulked data. (A) Runtime on data simulated to have a fixed cell count of 1000 and between 100 and 10000 genes. (B) Runtime on data simulated to have a fixed gene count of 1000 and between 100 and 10000 cells.

Next, we evaluated the performance with a constant number of genes and an increasing number of cells (Fig. 6B). DESeq2, DREAM, the permutation test, hierarchical bootstrapping, and the t-test demonstrated constant runtimes, while distinct, MAST, and scVI showed a linear increase in runtime depending on the number of cells. DESeq2, DREAM, and the permutation test are independent of the number of cells, because they work with pseudobulked data. Hierarchical bootstrapping samples a fixed number of bootstrapping iterations from the cell pool, making it largely independent of the pool size. Its runtime would likely increase with additional bootstrapping iterations and samples per hierarchical level. MAST, distinct, and scVI work on single cells and therefore depend on the cell count; however scVI is significantly faster than the latter two methods.

Overall, as expected, resampling-based methods were slower than parametric methods and pseudobulk-methods were faster than single-cell methods, with the exception of scVI, which is almost as fast as DESeq2 and DREAM, despite working on single cells.

Discussion and conclusion

This work explores the performance of differential gene expression analysis methods in hierarchically structured single-cell datasets such as atlases. A novel contribution is the adaptation of the hierarchical bootstrapping method, previously successful in neuroscience, to single-cell biology. This method and six other current methods were thoroughly investigated for its performance in the same scenarios. The aim was to provide recommendations for the best method. Four simulated scenarios and a real dataset were used for benchmarking and their effects on the methods were investigated.

While this work thoroughly examines the most common scenarios in single-cell analysis, it is important to acknowledge its limitations. For instance, it could be further explored whether the performance of the methods decreases continuously as the number of batches increases or whether methods that have not shown the best performance so far work better with larger batch numbers. Another limitation is that, in reality, single-cell datasets typically encompass a substantially greater number of cells than what we have simulated here. Certain benchmarked methods have already reached the upper limits of their scalability, making it challenging to increase the simulated cell number. In addition, further research could be done regarding the cell number of a pseudobulk. Knowing the minimum reasonable number of cells in a pseudobulk would be particularly interesting. One must also be aware of the uncertainty inherent in using simulated data. Some methods might excel due to their concordance with simulated data characteristics. For instance, drawing expression values from distributions may favor parametric methods. We also note that methods such as MAST assume zero-inflated counts, which do not represent single-cell data as well as previously thought [37]. We also restricted our benchmark to methods that test for differences in mean gene expression for a fair comparison. Future work may also consider alternative approaches such as sigEMD [38], which investigate shifts in the distribution or other differential expression patterns.

Regarding the real dataset, additional evaluation methods would also be possible. For example, genes considered to be differentially expressed could also be found in the literature or data sources other than the Reactome database could be included. Nevertheless, when comparing differential expression methods, the main focus will be on simulated data, since only for these the real differential expression is known and real biological data are rather laborious to generate or may contain too many confounding factors to perform a meaningful benchmark.

Based on our comprehensive research, we provide a practical guideline to the community. This guideline is specifically designed to empower researchers, scientists, and practitioners in the field of single-cell biology and gene expression analysis to select the most appropriate method for their specific dataset and research objectives. First, a user should consider whether their data contain a nested hierarchy or whether they are a planar dataset with individual samples. If the latter is the case, the current state-of-the-art method DESeq2 should be used after pseudobulking, as it delivers the best results on the dataset scenario and performs well on the real-world dataset combined with a very fast runtime. If the data consist of a nested hierarchy, the user should consider the number of genes as this massively impacts runtime. For increasingly larger datasets, the user should consider DREAM. If the data have a small number of genes, then the permutation test can be used for both balanced and unbalanced batches (Fig. 7).

Figure 7.

Decision tree supporting users in selecting an appropriate method for single-cell differential expression analysis.

In conclusion, pseudobulking, followed by classical methods for bulk RNA-seq data, is often the most effective strategy. More sophisticated methods tailored toward single-cell data can add benefits in complex scenarios such as the integration of multiple unbalanced batches. Since atlas-level data integration becomes more prevalent, further development in refining such methods is warranted. Some methods, such as MAST, cannot cope with current atlas-level data sets with millions of cells. Our extensible Nextflow benchmarking framework allows researchers to quickly evaluate new methods with respect to the state-of-the-art, thus facilitating rapid and continuous method evaluation in a rapidly moving field.

Key points

• Pseudobulk differential expression analysis methods demonstrate comparable performance with single-cell-specific approaches while offering significantly reduced computational runtimes.

• DREAM provides a good balance between accuracy and runtime for large and nested datasets, whereas permutation-based methods show higher performance but at the cost of longer runtimes.

• Hierarchical bootstrapping, adapted for nested single-cell transcriptomic data, offers a robust approach for analyzing complex hierarchically structured datasets.

Competing interests

G.S. is an employee of Boehringer Ingelheim International Pharma GmbH & Co KG. M.L. consults for mbiomics GmbH. L.H., S.L. and M.D. declare no competing interests.

Data availability

All figures and results presented in this paper can be reproduced using our Nextflow pipeline with containerized environments, which is available on GitHub: https://github.com/LeonHafner/sc-guidelines. Figures and Singularity containers are additionally available for download from Zenodo: 10.5281/zenodo.14639638.