An introduction to representation learning for single-cell data analysis

Summary

Single-cell-resolved systems biology methods, including omics- and imaging-based measurement modalities, generate a wealth of high-dimensional data characterizing the heterogeneity of cell populations. Representation learning methods are routinely used to analyze these complex, high-dimensional data by projecting them into lower-dimensional embeddings. This facilitates the interpretation and interrogation of the structures, dynamics, and regulation of cell heterogeneity. Reflecting their central role in analyzing diverse single-cell data types, a myriad of representation learning methods exist, with new approaches continually emerging. Here, we contrast general features of representation learning methods spanning statistical, manifold learning, and neural network approaches. We consider key steps involved in representation learning with single-cell data, including data pre-processing, hyperparameter optimization, downstream analysis, and biological validation. Interdependencies and contingencies linking these steps are also highlighted. This overview is intended to guide researchers in the selection, application, and optimization of representation learning strategies for current and future single-cell research applications.

High-dimensional data generated by single-cell systems biology (omics) methods require powerful representation learning approaches to enable interpretation and interrogation of the structures, dynamics, and regulation of cell heterogeneity. In this perspective, Gunawan et al. elucidate key steps involved in representation learning to guide optimized method application by researchers analyzing diverse single-cell data modalities.

Introduction

Dramatic advances in single-cell analysis technologies have brought cellular heterogeneity into sharp focus. The combination of quantitative, multi-parametric systems biology techniques, termed “omics,” with single-cell-resolved measurement capabilities has permitted the delineation of cellular states and their transitional trajectories, as well as inference of mechanisms controlling cell-state dynamics.¹ Single-cell omics provide insight by leveraging information inherent within cellular heterogeneity: in the quantitative variation and co-variation of cellular features,² accessible whether induced by experimental perturbations, driven by disease processes, or naturally occurring. Insights into cell subpopulation subtypes and spectra, state dynamics, and molecular control, as a result of single-cell data analysis, have been influential in diverse fields, including but not limited to cancer research,³ immunology,⁴ developmental biology,⁵ neuroscience,⁶ and drug discovery.⁷ Given these advantages, single-cell omics technologies have rapidly diversified to encompass proteomics,⁸ metabolomics,⁹ genomics,¹⁰ epigenetics,¹⁰ and especially transcriptomics, where single-cell RNA sequencing (scRNA-seq) technologies are reaching maturity.¹¹ Maturing in parallel, high-throughput, high-content single-cell microscopy, also termed systems microscopy,^12,13 is becoming increasingly powerful for the systematic quantification of cell phenotypes in time and space.^14,15 Now, emerging interfaces between single-cell omics and systems microscopy has begun to permit interrogation of cellular heterogeneity defined by spatiotemporally resolved measures of both molecular and cellular states. This fusion will illuminate the diversity and the control of cellular phenotypes as never before.¹⁶

The wide and diverse deployment of single-cell measurement technologies has produced a deluge of data requiring corresponding advances in computational methods for data analysis.¹⁷ A significant challenge faced by researchers analyzing omics and imaging-based single-cell data is high data dimensionality. With numerous (100s–10,000s) features simultaneously measured per cell, a vast data landscape of potential cell states is created, requiring (theoretically) a correspondingly vast dataset to achieve a well-sampled state space. Real-world experimental constraints often lead to relative sparsity in the true data landscape, obscuring relationships between cells and between features.^18,19,20 Additionally, intrinsic biological and technical noise can further convolute meaningful signals in an under-sampled space.²¹ In single-cell data, cell atlas initiatives have produced huge omics datasets, some reaching hundreds of thousands of cells, creating the potential for powerful analyses.²² Yet, increases in the number of measurable features impose ever-greater computational and scaling challenges, often resulting in intractable analyses.²¹

To address these challenges, dimension-rich datasets are now routinely interrogated via some form of representation learning, a term encompassing a suite of analytical techniques that project high-dimensional data into a lower-dimensional space, or “representation.” Such low-dimensional embedding is achieved by stripping out redundancies and noise in the data that occupy dimensions beyond the intrinsic structure of the phenomena under investigation, i.e., cell diversity. Representation learning is not only statistically motivated but is also biologically intuitive. For example, regulatory modules formed by genes are expressed in a coordinated manner, and thus the dimensionality needed to represent the expression of highly correlated genes can naturally be compressed,²³ better revealing the underlying metaparameters driving biological phenomena. Generally, representation learning is used to transform data into configurations amenable to various types of downstream analysis. This coupling of representation learning with downstream analytical tasks is exemplified by software tools such as semisoft clustering with pure cells (SOUP),²⁴ scPred,²⁵ and Monocle,²⁶ which apply representation learning for dimension reduction prior to downstream clustering (e.g., subpopulation detection), classification (e.g., supervised cell-type labeling), or pseudo-time construction (e.g., inferring the main trajectories of dynamic state changes), respectively (Figure 1A).

Figure 1.

Two-dimensional representation learning from three-dimensional data

(A) Data are sampled from an underlying biological structure or state space. Single-cell experiments often measure thousands of features simultaneously, resulting in a high-dimensional space, here exemplified in only three dimensions for visualization (left). Degrees of freedom in high dimensions create a vast state-space volume, leading to sparsity, which makes identifying relationships challenging (center). Projection by representation learning to a lower dimension reduces sparsity to reveal relationships in the data for further analysis, but fidelity to the original state-space structure is not guaranteed (right). The choice of representation learning method is critical to determining analytical outcomes. Alternate analytical goals (e.g., clustering and/or classification or lineage inference) may encourage selection of distinct, task-matched representation learning methods.

(B) Overview of common stages in a general single-cell analysis pipeline. Data require preparation and pre-processing before they can undergo representation learning to reveal relationships and compress dimensions. The application of representation learning requires hyperparameter optimization to achieve optimal results. Further downstream analyses then leverage learned embeddings. Result interpretation requires an understanding of the quality of the analysis and, often, further biological validation. Iterative refinement of each stage, or the entire pipeline, may be required.

In practice across the biological sciences, representation learning is often a central component in analytical processes comprising several common steps: (1) data preparation and pre-processing, (2) selection of representation learning methods, (3) hyperparameter optimization, (4) downstream analyses, and (5) evaluation and interpretation of results. Such pipelines are typically iterative, with analyses often requiring cycles of refinement.

Here, we provide an end-to-end overview of techniques and options in the composition of single-cell analysis pipelines centered on representation learning. We highlight key considerations aligned to the common steps of such pipelines (Figure 1B), exploring influences on the nature and quality of learned embeddings and how these impact downstream analysis outcomes. Constituting a guide for those less familiar with the field, we step through each stage of a typical single-cell investigation. We first discuss data preparation and pre-processing and give an overview of representation learning approaches and methods including their optimization via hyperparameter tuning. We highlight how downstream analytical goals should influence upstream representation learning strategies and, finally, emphasize the challenges inherent to evaluating and interpreting results.

Common steps in representation learning-centered single-cell analysis pipelines

Upstream data preparation and pre-processing

Upstream pre-processing prepares a dataset so that it can be embedded optimally for further analysis. Pre-processing of data, including transformations/filtration, denoising, imputation, and integration, occurs prior to representation learning to improve the quality of embeddings. At times overlooked, pre-processing can significantly influence analysis. While each data modality has its own specific concerns and methods, we highlight common themes and examples.

Feature transformation, filtration, and denoising

Raw data are often unsuitable for embedding and further analysis. Measured features may be at different scales and between different ranges, requiring normalization or standardization to make features comparable and to prevent domination by a subset of features. Additionally, confounding factors can be minimized by transformations or feature extraction. For example, log transformations are often applied to scRNA-seq data to remove mean-variance dependencies that can be problematic for some representation learning methods including principal-component analysis (PCA).²⁷ Other transformations may highlight information, such as contextual relationships emphasized by fish-eye transformation in single-cell microscopy,²⁸ that is otherwise insufficiently represented in learned embeddings. Feature filtration is also commonly used in single-cell microscopy, removing features that contain (in many cases) irrelevant spatial information, such as a cell’s angular orientation within the image field, while retaining features such as mean intensity or nuclear-cytoplasmic intensity ratios, associated with marker (e.g., protein) levels and/or localization, respectively. Notably, single-cell image segmentation and analysis methods are routinely used to transform cell images into high-dimensional quantitative cell descriptors.^29,30 This permits scalable and robust interpretation, as well as the use of representation learning techniques that would otherwise be restricted to omics-style numerical matrix data.^31,32 Interestingly, while image data are typically transformed into quantitative data for analysis, omics researchers have begun experimenting with “imagification” of omics data, bringing potent image modeling methods, especially convolutional neural networks, to bear on the analysis of omics data.^33,34

Removal of background variation and biological noise with respect to the phenomena under investigation can also improve embeddings and subsequent analysis. This is demonstrated in the comparison of methods to filter out “nuisance genes” in scRNA-seq data from Drosophila wing disc myoblast cells³⁵ and by Satjia et al., who exclude known housekeeping genes in scRNA-seq data to study spatial patterning of gene expression in zebrafish embryos.³⁶ Technical noise is also unavoidable and is specific to any measurement system. Such noise obfuscates signals of interest and is exacerbated by high-dimensional spaces. Hence, raw data generally require some form of denoising before further analysis. Imaging artifacts such as skews caused by optical objectives create systematic aberrations requiring corrections. Random noise is also common, exemplified by salt and pepper noise and imaging backgrounds, which can also be minimized^37,38 (Figure 2A). In scRNA-seq measurements, random noise is exemplified by the spurious zero counts (“dropouts”) that are commonly accounted for by imputation methods.^39,40,41 However, denoising can be particularly challenging in rare cell populations or where scRNA-seq sequencing depth is low, as noted by Wang et al., who use transfer learning to improve data quality.⁴² Denoising and imputation strategies for scRNA-seq data have been reviewed by Patruno et al., identifying versatile tools as well as context-dependent performers.⁴³

Figure 2.

Influence of pre-processing on representation learning

(A) Imaging artifacts obscuring cells in label-free imaging (top; original data) are removed by background flattening through Gaussian smoothing and subtraction, creating a more consistent image (bottom; original data).

(B) Batch-effect correction performed by CytoNorm in mouse bone marrow cytometry data to form a unified dataset while maintaining informative cell-type clusters (adapted from Ashhurst et al. with permission⁴⁴).

While not all representation learning methods require the same pre-processing steps, it is necessary to carefully consider these in order to suppress spurious or irrelevant variation and make data more amenable to effective representation learning and subsequent analyses.⁴⁵ Because pre-processing impacts the performance of any subsequent representation learning tools, iterative refinement of pre-processing on the basis of downstream outcomes may be important as an investigation matures.

Integrating heterogeneous data from multiple batches and measurement modes

Even with single omics, heterogeneity exists between experiments, which could be due to different measurement systems, different cells, and/or experimental procedures. For a unified analysis, corrections between batches are required (“horizontal integration”) (Figure 2B). Several approaches have been proposed for this including alignment through data alignment and empirical Bayesian modeling.⁴⁶ This is demonstrated by Lotfollahi et al., who use transfer learning with conditional variational autoencoders to produce joint embeddings between query datasets and reference atlases, allowing large cell numbers in such atlases to be leveraged. However, even though individual omics technologies have brought deep insights, each separate omics method unveils a distinct perspective on the cell. Only by bringing these views together can we achieve a more comprehensive understanding of cellular states, machinery, and their interplay. Such multi-modal data integration is generally made possible either through experimental methods enabling concurrent measurement of alternate data modalities in the same individual cells (“vertical integration”) or through emerging computational methods to integrate multi-modal data derived in parallel from different cell populations⁴⁷ (“diagonal integration”).

Measurement of multiple omics data types in the same cells has so far included single-cell methylome and transcriptome sequencing (scM&T-seq), as well as single-cell nucleosome, methylation, and transcription sequencing (scNMT-seq).^16,48 The capacity to diversify data modalities concurrently accessed per cell is strongly limited by the distinct and often incompatible experimental requirements for each measurement modality. Thus, more diverse (less constrained) data integration relies on computational approaches for multi-modal data integration, which commonly leverage representation learning methods. For example, Yang et al. performed manifold alignment using deep learning autoencoders to integrate unmatched scRNA-seq and chromatin images of naive CD4⁺ T cells.⁴⁹ Other computational tools include manifold alignment to characterize experimental relationships (MATCHER),⁵⁰ maximum mean discrepancy manifold alignment (MMD-MA),⁵¹ and UnionCom⁵²; however, their general use and interpretation remain challenging.⁵³ We direct the reader to a review by Argelaguet et al.⁴⁷ for further details.

Finally, we highlight multi-modal data integration as a key research opportunity as well as a data preparation/pre-processing challenge, with several integration methods inherently using representation learning approaches to output joint embeddings.⁵⁴ Undoubtedly, optimized methods for computational data integration will permit cogent interrogation of ever more diverse data sources irrespective of inflexible experimental requirements, providing a pathway for increasingly comprehensive analyses of cellular processes.

State-of-the-art representation learning methods and their characteristics

Representation learning projects data into spaces amenable to further analysis (often lower dimensional). The emphasis on the development and comparison of representation learning methods has increased dramatically in recent years due to the pivotal role of these methods in analyzing increasingly complex single-cell datasets. Here, we group the diverse selection of popular representation learning methods (Table 1) into three generalized categories wherein methods broadly share similar characteristics: (1) statistical/probabilistic approaches, including matrix decomposition, (2) manifold learning, and (3) neural networks. Each method has advantages and limitations highly dependent on their circumstance of use and the dataset of interest. Below, we discuss the characteristics of each group in turn, seeking to provide the reader with some starting rationale and intuition for method selection and comparison.

Table 1.

Selective summary of representation learning methods

	Method	Example usage in single-cell analyses	Linearity
Statistical methods	Principle component analysis (PCA)55	sc-qPCR,56 microscopy features57	linear
	Non-negative matrix factorization (NMF)58	scRNA-seq59	linear
	Similarly weighted non-negative embedding (SWNE)60	scRNA-seq60	non-linear
	Singular value decomposition (SVD)61	scRNA-seq45	linear
	Latent semantic analysis (LSI)62	scATAC-seq63,64	linear
	Independent component analysis (ICA)65	scRNA-seq26	linear
	Zero inflated factor analysis (ZIFA)66	scRNA-seq66	linear
	Latent dirichlet allocation (LDA)67	scRNA-seq,68 scATAC-seq69	non-linear
	Gaussian process latent variable models (GPLVM)70	scRNA-seq71	non-linear
Manifold learning	t-distributed stochastic neighbor embedding (t-SNE)72	scRNA-seq,73 microscopy features74	non-linear
	Isomap75	scRNA-seq76	non-linear
	Locally linear embedding (LLE)77	scRNA-Seq78	non-linear
	Uniform manifold approximation and projection (UMAP)79	scRNA-seq,80 microscopy features81	non-linear
	Potential of heat-diffusion affinity-based transition embedding (PHATE)82	CyTOF, scRNA-Seq82	non-linear
	Laplacian eigenmaps83	scRNA-seq84	non-linear
	Diffusion maps85	sc-qPCR, scRNA-seq85	non-linear
	Local tangent space alignment (LTSA)86	scRNA-seq87	non-linear
	PhenoGraph88	mass cytometry88	non-linear
	Markov affinity-based graph imputation of cells (MAGIC)40	scRNA-seq40	non-linear
Neural networks	Sparse autoencoder for unsupervised clustering, imputation, and embedding (SAUCIE)89	CyTOF, scRNA-seq89	non-linear
	Deep variational autoencoder for scRNA-seq data (VASC)90	scRNA-seq90	non-linear
	Scvis91	scRNA-seq91	non-linear
	Two-stage variational autoencoder92	microscopy92	non-linear
	Batch-adversarial variational auto-encoder (BAVARIA)93	scATAC-seq93	non-linear
	Ivis94	CyTOF, scRNA-seq94	non-linear
	Cellular in-painting95	microscopy95	non-linear
	ScSemiGAN96	scRNA-seq96	non-linear
	scNym97	scATAC-seq97	non-linear
	Treatment convolutional neural network98	microscopy98	non-linear
	Cytoself99	microscopy99	non-linear

Statistical/probabilistic methods

Representation learning has strong roots in statistical modeling and can be traced back to Pearson’s 1901 PCA,⁵⁵ which remains relevant and widely used today. Statistical methods often make strong assumptions, such as the relationships between features being linear. This assumption generates methods that are more robust to noise, outliers in data, and hyperparameter choice. Moreover, the algorithms themselves (and their logical motivations) are often highly interpretable. As an example, probabilistically motivated linear discriminant analysis (LDA) is able to distinguish “recurrent cellular neighborhoods” in skin samples with pre-cancer or cancer histology using multiplexed immunofluorescent imaging, aiding in the identification of discrete cell states whose frequency and proximity varied throughout cancer progression¹⁰⁰ (Figure 3A). However, these same assumptions limit the modeling of more complex (non-linear) relationships that are commonly present in single-cell data.⁸² Recognition of this limitation has motivated the development of non-linear approaches more suited to learning the underlying structure of single-cell datasets. These include both manifold learning methods and those based on deep neural networks.

Figure 3.

Application of example representation learning methods

(A) LDA latent space shows 10 recurring neighborhoods with different cell-type composition signatures in skin cancer samples (adapted from Nirmal et al.¹⁰⁰).

(B) UMAP embedding of scRNA-seq data of CD45⁺ cells shows four clusters of major cell types with a further 23 subgroupings (adapted from Zhao et al.¹⁰¹).

(C) Ivis embedding of scRNA-seq from vascular myeloid cells reveals 12 clusters for hypothesis generation and further investigation of Clec4a2 macrophage regulation (adapted from Park et al.¹⁰²).

Manifold learning

A common and often useful assumption is that cellular states, and ideally empirical cellular data, lie on a “manifold,” defined as a locally smooth landscape that has relatively lower dimensionality compared to the ambient data space. Such a landscape was famously conceptualized in the 1950s by Conrad Waddington¹⁰³ as a way to conceive the relationship between cellular state diversity and underlying regulatory mechanisms. Formal validation of the manifold assumption in relation to single-cell data remains challenging. Yet, it is both rational and intuitive that cellular states move in incremental steps within a state space intrinsically compressed through coordinated gene expression.¹⁰⁴ Successful applications of manifold learning further empirically support manifold modeling of single-cell data.^40,88 For example, Zhao et al. has demonstrated the use of manifold learning using uniform manifold approximation and projection (UMAP) to visualize the top 50 principal components of scRNA-seq data of CD45⁺ cells, showing clustering of major cell types and further subgroupings in the population¹⁰¹ (Figure 3B). In general, the manifold assumption is less restrictive than assumptions made by statistical methods; hence, manifold learning is thought to better capture true cell state-space structures. However, this flexibility can make the more complex projections learned less interpretable.¹⁰⁴

Neural networks

In the single-cell context, neural networks make the least assumptions about underlying data relationships. This implies broad application generalizability, as exemplified by cross-domain use of neural network-based natural language processing techniques in biological sequence data.¹⁰⁵ A vast range of different neural network training paradigms and architectures have been proposed, including autoencoders, contrastive learning models, and semi-supervised models (Table 1). As an example, Park et al. demonstrated contrastive learning using ivis on scRNA-seq of vascular myeloid cells, identifying 12 clusters for further analysis, with three sharing resident macrophage genes. Further characterization of these three macrophage clusters revealed that Clec4a2 potentially had a role in modulating macrophage activation and homeostasis, generating a hypothesis for further investigation by the authors¹⁰² (Figure 3C). Though the flexibility given by fewer assumptions make them powerful tools capable of learning highly complex relationships, neural networks often require demanding hyperparameter optimization and are more sensitive to variations in data, thus demanding larger datasets for successful model training. What precisely is learned by neural networks may be even less interpretable (than manifold learning methods). As such, neural networks are often considered “black box” models, a limitation widely recognized by the field.

Hyperparameter optimization

Representation methods typically contain adjustable parameters that govern the model fitting process, referred to as hyperparameters. Hyperparameter optimization depends on the chosen tools and can have substantial impacts on the resultant embedding (Figure 4A) Neglecting to optimize hyperparameters will impact downstream performance and can even lead to nonsensical outputs, especially when using hyperparameter-sensitive methods.¹⁰⁶ Though the importance of hyperparameters is recognized by some expert users and method developers,^107,108 many studies applying existing methods—and even some detailing new methods—describe little hyperparameter optimization. This suggests a limited focus on the key role such optimization may play.¹⁰⁹ In many circumstances, such as multi-method benchmarking, researchers rely on default hyperparameter values.^45,87 Here, we briefly describe the generalized hyperparameter optimization process and note how different challenges are minimized or exacerbated by different representation learning methods, potentially influencing method suitability for a given user or analysis task.

Figure 4.

Hyperparameter tuning in representation learning

(A) Independent-component analysis (ICA) embeddings of scRNA-seq data. Each embedding reflects a different number of input dimensions, revealing how the dimension hyperparameter influences embedding. With 20, 30, or 50 dimensions, clustering is relatively straightforward compared to using 100, 200, or 300 dimensions. This highlights hyperparameter influence on downstream (in this case, clustering) performance (adapted from Feng et al.¹⁰⁶).

(B) The hyperparameter k (number of nearest neighbors; x axes) in the context of different data subsample sizes alters identification accuracy for manual gates in the neural network ivis. Ivis is relatively robust to data subsampling, especially for small values of k. However, the value choice of k has a considerable impact when less data are available (adapted from Szubert et al.⁹⁴).

(C) Dataset influences on representation learning. Embeddings of scRNA-seq of the mouse neocortex by PCA and t-SNE. In comparison to PCA, t-SNE captures local structure at the expense of global structure (adapted from Kobak and Berens⁷³).

Hyperparameter optimization is often framed as the search for a good, or even the best, set of hyperparameter values from an array of all possible value combinations. This constitutes an exploration of the so-called hyperparameter “search space,” typically done by repeated applications of a representation learning method configured with different hyperparameter values sampled either randomly or exhaustively or by some more sophisticated schemes.¹¹⁰ State-space representations from each model instance undergo comparison to find a set of hyperparameter values producing the most desirable embedding according to available criteria.

Key challenges in optimization include practical restrictions relating to computational costs for hyperparameter search space exploration. Methods with larger hyperparameter numbers or extent define search spaces with many more value combinations. This is especially compounded by interdependencies between hyperparameters, causing the search space to quickly undergo a combinatorial explosion. This can be compounded by the inherent speed and resource demands of a given method, such as memory limits (video RAM) associated with graphical processor unit (GPU) hardware used in neural network approaches. Another challenge reflects the difficulty in assessing and interpreting results to find the most desirable embedding. This is particularly difficult when ground-truth annotations are inaccessible, as further discussed below.

Where researchers have limited capacity for hyperparameter optimization, or a limited basis to evaluate results, it may be advisable to favor methods with limited hyperparameter sensitivity and/or to apply hyperparameter values that are relatively robust to changes in the dataset (e.g., cell population). Such robust characteristics have, for example, been validated for PhenoGraph, markov affinity-based graph imputation of cells (MAGIC), and ivis (Figure 4B), which reduce risks of overfitting and lack of generalizability.^40,88,94 In general, the use of less complex and more robust representation learning methods is a practical strategy to achieve reasonable outcomes with a lower risk of defective embeddings, which may not be readily discernible to the user. This robust approach may nonetheless cost peak performance.

Downstream analyses

Ultimately, in the context of single-cell analysis, representation learning is used to derive a data embedding that facilitates one or more downstream analysis tasks. Given that downstream analysis tasks may be substantially different, it follows that the functional utility of any individual embedding may be highly variable depending on the downstream task. For example, clustering and lineage analysis are common downstream tasks performed in single-cell analyses. In the case of clustering, splitting the dataset into informative cell subpopulations is a desirable outcome. Conversely, when viewed through the lens of lineage analysis, an embedding with this discretized character may be undesirable because continuity and smoothness in the embedding are required to map continuous cell-state changes.

It is important to recognize that computational tools are often developed for a specific purpose that will significantly inform (or inhibit) their broader utility. For instance, PCA excels at revealing structures in global cell states but has been noted to struggle to maintain complex local structural details. This is exemplified in the embedding of embryoid body differentiation.⁸² Similarly, t-distributed stochastic neighbor embedding (t-SNE) was originally intended for visualization of high-dimensional data and hypothesis generation rather than deep analysis⁷² and, unlike PCA, fragments the global structure of the embedding of mouse neocortex scRNA-seq data (Figure 4C). Repurposing tools for different downstream analysis tasks require consideration and understanding of their original rationale, data-dependent performance, and behavior.

As a general example of this, numerous benchmarking studies on scRNA-seq datasets have sought to define optimal representation learning methods with respect to particular downstream tasks.^{45,107,87,111,112} Yet despite substantial efforts and rigor, outcomes have been mixed, depending significantly on the underlying data used. This is highlighted by Heisner and Lau, who stress that benchmarking results are highly dataset specific, ultimately questioning the premise of benchmarking efforts purposed to find a “best” method since result generalizability appears minimal.¹¹² Instead, benchmarking typically provides a starting point for further evaluation and validation rather than crowning a winning, one-size-fits-all tool. For the practitioner, methods validated to perform well on similar datasets might best be prioritized where possible. Best practice dictates testing of multiple methods, with careful evaluation and validation of results for each dataset and analysis task.

Evaluation and biological validation

Ground-truth annotations for the biology of interest are often absent in single-cell investigations. Yet, interpreting an analysis depends on understanding the quality of the results. Hence, evaluating learned representations is a crucial, though often challenging, step. Though computational tools may reveal phenomena of interest, biological validation is required to validate indications. We next consider the role of technical metrics and biological validation in such result evaluation.

In some cases, it is possible for domain experts to qualitatively assess embeddings. For instance, single-cell imaging data can sometimes be visually assessed by experts (e.g., using a depiction of images within the learned representation).¹¹³ This is demonstrated by Bryce et al., who, after mapping cellular phenotypes in a t-SNE space, examine phenotypic differences in each cluster by inspection of microscopy images⁷ (Figure 5A) and use this approach to match learned representations to visual perception and thus avoid model over-/under-fitting. For other data modalities, such as scRNA-seq, direct human assessment of high-dimensional RNA expression data is impractical. Validation is thus limited to qualitative expectations, such as those that might pertain to the relative proximity of known cell types¹¹⁴. Qualitative evaluations can be informative but are generally not scalable and lack the quantitative comparative power required for robust optimization. Hence, quantitative metrics are often preferentially used.

Figure 5.

Evaluating embeddings and metrics

(A) Example actin phenotypes induced by chemical compound library screening; representative of numbered clusters in the t-SNE map in (C) (adapted from Bryce et al.⁷).

(B) FICA and UMAP embeddings of scRNA-seq datasets are shown, with colors representing different cell types. The UMAP embedding compresses and fractures the data considerably more than FICA (adapted from Koch et al.⁴⁵).

(C) t-SNE phenotypic embedding of 114,400 compounds, with distinct phenotypic clusters numbered. The total number of actin phenotypes (see A) and positive control compound-induced phenotype self-clustering metrics were evaluated against t-SNE perplexity, showing combinatorial use of supervised and unsupervised metrics to evaluate the embedding (adapted from Bryce et al.⁷).

Quantitative metrics often score embeddings based on specific criteria associated with downstream analysis tasks, including clustering (e.g., cluster compactness, separability, dispersion, and cell-type numbers),¹¹⁵ classification (e.g., accuracy and robustness),⁸⁷ and trajectory inference analyses (topological similarity, branch assignment, and correlation).¹¹⁶ Thus, metric selection may depend primarily on the expectations of the data. Moreover, metrics can be unsupervised—based solely on the embedding irrespective of any additional annotations—or supervised, assessing the embedding through the lens of available labels like cell type or treatment condition. It is important to keep in mind that although annotations can help interpret an embedding, over-reliance on them can also introduce a bias toward established dogmas. This is especially true when annotations are only weakly related to the phenomenon under investigation.⁹⁵ For example, Koch et al. show embeddings of scRNA-seq data across multiple datasets using fast independent-component analysis (FICA) and UMAP.⁴⁵ Though the UMAP embedding is well clustered with respect to the cell-type labels used—yielding a high clustering score—the embedding is severely fractured and compresses the nuance and variability among other morphological properties not measured during the establishment of the cell-type definition (Figure 5B). Ideally, some combination of supervised and unsupervised metrics should be applied, with the former leveraging validation of available ground-truth knowledge to support novel inferences illuminated by the latter. Such an approach was exemplified in a systems microscopy analysis of drug-response phenotypes wherein the number and coherence of known drug phenotypes were used as part of an adversarial strategy to optimize representation-learned embeddings of a library of 114,400 compounds with unknown effects⁷ (Figure 5C). Through such combinatorial approaches counterposing both supervised and unsupervised metrics, researchers deploying representation learning may more confidently extrapolate from the known to the unknown.

Finally, we mention subsampling evaluation schemes, such as bootstrapping and cross-validation, as a method to evaluate the generalizability of any findings. The consistency of embeddings on subsamples of the data suggests robustness to noise and spurious variations. This provides a heuristic for model reliability, as demonstrated by Moon et al., who show the robustness of potential of heat-diffusion affinity-based transition embedding (PHATE) in subsampling of Splatter-simulated scRNA-seq data.⁸²

Conclusion and future outlook

Single-cell technologies continue to mature, enriching and transforming our understanding of the cell. As increasingly powerful biological data measurement methods are applied, the role of computational tools for enabling complex data interpretation becomes ever more central. Representation learning is already widely used in the biomedical community for this purpose, yet it remains a complex challenge to navigate the selection, optimization, application, and comparison of representation learning methods for single-cell analyses. Ever-expanding applications are driving an explosion of new methods encompassing more nuanced understandings of the cell and increasingly sophisticated computational processes. Thus, the complexity involved in applying representation learning for single-cell analysis continues to escalate. Here, we hope to have provided a broad roadmap covering the end-to-end process of representation learning, helping to orient new practitioners in the field.

In this exciting era, new perspectives on, and aspects of, the cell are being revealed by proliferating modalities of data measurement. Representation learning lies at the intersection of these data modalities, providing the means to achieve a mature and integrated characterization of the cell. In particular, very diverse omics and imaging data types require merging in a way that is interpretable for further analyses. Excitingly, representation learning techniques have recently been applied to graph representations of single-cell relationships, such as molecular similarity graphs or gene regulatory networks.¹¹⁷ The ubiquity of graph-style relationships and the ability to integrate them in single-cell investigations suggest a powerful capability. Thus, active research continues to develop tools that generalize and scale well,⁵³ helping biomedical researchers obtain systemic views incorporating unique, unbiased, coherent perspectives. With the stage set for transformational advances in our understanding of the cell, the need for biomedical researchers to understand, select, and effectively deploy representation learning methods becomes an ever more significant priority.