Partial alignment of multislice spatially resolved transcriptomics data

Abstract

Spatially resolved transcriptomics (SRT) technologies measure messenger RNA (mRNA) expression at thousands of locations in a tissue slice. However, nearly all SRT technologies measure expression in two-dimensional (2D) slices extracted from a 3D tissue, thus losing information that is shared across multiple slices from the same tissue. Integrating SRT data across multiple slices can help recover this information and improve downstream expression analyses, but multislice alignment and integration remains a challenging task. Existing methods for integrating SRT data either do not use spatial information or assume that the morphology of the tissue is largely preserved across slices, an assumption that is often violated because of biological or technical reasons. We introduce PASTE2, a method for partial alignment and 3D reconstruction of multislice SRT data sets, allowing only partial overlap between aligned slices and/or slice-specific cell types. PASTE2 formulates a novel partial fused Gromov-Wasserstein optimal transport problem, which we solve using a conditional gradient algorithm. PASTE2 includes a model selection procedure to estimate the fraction of overlap between slices, and optionally uses information from histological images that accompany some SRT experiments. We show on both simulated and real data that PASTE2 obtains more accurate alignments than existing methods. We further use PASTE2 to reconstruct a 3D map of gene expression in a Drosophila embryo from a 16 slice Stereo-seq data set. PASTE2 produces accurate alignments of multislice data sets from multiple SRT technologies, enabling detailed studies of spatial gene expression across a wide range of biological applications.

Spatially resolved transcriptomics (SRT) technologies measure messenger RNA (mRNA) expression simultaneously at thousands of locations within a tissue. These technologies include both sequencing-based approaches, such as 10x Genomics Visium (https://www.10xgenomics.com/products/spatial-gene-expression) and Slide-seq (Rodriques et al. 2019; Stickels et al. 2021), as well as hybridization and fluorescent approaches such as MERFISH (Chen et al. 2015) and seqFISH (Lubeck et al. 2014). Nearly all of these technologies measure expression at two-dimensional (2D) locations within a thin tissue slice ( ≈ 10 μm), and we will use the term spatial transcriptomics (ST) as a generic term to refer to any of these technologies. ST provides spatial context that is missing from single-cell RNA sequencing (scRNA-seq) measurement of mRNAs from disassociated cells, and has been widely used to study both normal (Baccin et al. 2020; van den Brink et al. 2020) and diseased tissues, such as cancer (Ståhl et al. 2016; Thrane et al. 2018; Ji et al. 2020) and Alzheimer's disease (Chen et al. 2020). However, similar to scRNA-seq, ST data suffers from high rates of sparsity. Moreover, recording only the x, y coordinates on a 2D tissue slice loses information along the z (orthogonal) direction of the 3D tissue, hindering a comprehensive analysis of the whole tissue (Fig. 1).

Figure 1.

PASTE2 partial alignment of overlapping slices. Four thin slices (red, green, blue, purple) are dissected from the same tissue and placed on an ST array. However, these slices only partially overlap in the z-coordinate direction. The inputs to PASTE2 are the four ST slices, including gene expression, spot locations, and optionally, histology images. PASTE2 computes a partial alignment of each pair of adjacent slices by selecting subsets of spots from each slice that preserve transcriptional, spatial, and image similarity. PASTE2 uses the partial alignment to create a 3D spatial reconstruction of the tissue.

Spatial transcriptomics is often applied to multiple sequential 2D slices from the same tissue (Fig. 1), thus opening the possibility of performing integrative analysis of all slices. Such joint analysis of multiple slices not only helps with the data sparsity problem in individual slices, but also enables innovative downstream tasks such as 3D spatial expression analysis, 3D cell-cell communication, and 3D clustering (Lin and Yang 2022; Zeira et al. 2022). However, aligning multiple slices from the same tissue along the orthogonal direction to recover spot-spot correspondence across slices is a challenging task because of morphological differences across slices as well as technical variability in mRNA capture between experiments.

Several approaches have been used for alignment of multiple ST slices. One approach is to apply methods developed for scRNA-seq and multi-omics data integration, such as Seurat (Stuart et al. 2019), SCOT (Demetci et al. 2022; Demetçi et al. 2022), or Pamona (Cao et al. 2022). Another approach is to use methods that align an scRNA-seq data set onto an ST data set, such as Tangram (Biancalani et al. 2021) or RCTD (Cable et al. 2022). However, these methods are designed for different alignment tasks and ignore the spatial information within or across slices. Another method, STUtility (Bergenstråhle et al. 2020) is designed to align a pair of ST slices, but aligns only the histology images, ignoring both gene expression and spatial information. Moreover, this method can only be applied to 10x Genomics Visium data. Another recent method, GPSA (Jones et al. 2022) integrates multiple ST slices into a common coordinate system, but does not output a mapping between spots that can be used for downstream analysis, and the common coordinate system it produces is different from the 3D coordinates of the tissue.

Another possible solution is to use histological and medical image registration (Maes et al. 2003) toolkits such as ITK (McCormick et al. 2014) and SimpleITK (Beare et al. 2018). However, many image registration methods are supervised and often require manually selected landmarks, creating an extra burden on the user. The spatial alignment problem has also been studied in the context of functional magnetic resonance imaging (fMRI) data registration (Lancaster et al. 2000; Brett et al. 2001; Klein et al. 2009), but these methods are not easily extensible to the spatial genomics setting (Jones et al. 2022). Finally, many ST technologies do not have matching histological images.

Recently, PASTE, a method that performs probabilistic alignment of ST slices using both spatial and transcriptional similarities, was introduced (Zeira et al. 2022). However, PASTE assumes that the slices overlap over the full 2D assayed region, with a similar field of view and similar number and proportion of cell types. Essentially, PASTE assumes the two slices are biological/technical replicates of a 2D tissue, an assumption that is often violated in real ST experiments because of technical difficulties in tissue dissection and array placement, or differences in tissue morphology between nearby slices. For example, two slices may only partially overlap along the z axis because of different placements of the array on the tissue, and hence only a part of both slices should be aligned (Fig. 1). Furthermore, two slices may have different compositions of cell types, leading to slice-specific cell types and spots that should not be aligned.

Here, we introduce PASTE2, a method to align multiple adjacent ST slices from the same tissue with several substantial improvements over existing methods. First, PASTE2 performs partial pairwise alignment, selecting and aligning only a subset of spots. PASTE2 thus addresses the important case in which adjacent slices do not fully overlap in space or have different cell type compositions. To solve the partial alignment problem, we introduce the partial fused Gromov-Wasserstein (partial-FGW) optimal transport framework. We derive a conditional gradient (Frank-Wolfe) algorithm (Frank and Wolfe 1956) to optimize the partial-FGW objective. Partial-FGW is the partial extension (Chapel et al. 2020) of the fused Gromov-Wasserstein optimal transport (Titouan et al. 2019) and allows only a fraction of the total mass to be transported between the two distributions. To the best of our knowledge, PASTE2 is the first to formulate the partial-FGW problem and provide an optimization procedure for it. Second, PASTE2 includes a model selection procedure to estimate the fraction of overlap between two slices to align, which is in general a very difficult problem. Third, PASTE2 optionally uses the histological images. Some ST technologies, such as the 10x Genomics Visium platform, produce a Hematoxylin and Eosin (H&E) stained image of the same tissue slice in which gene expression is measured. The information in this image can aid in alignment of slices by identifying spots with similar histology. Finally, we provide a generalized Procrustes analysis (Wahba 1965) method for 3D spatial reconstruction of the tissue from partially aligned 2D slices. We show PASTE2's advantages on both simulated and real ST data sets.

Methods

An ST experiment on a 2D tissue slice yields a pair (X, Z), where X ∈ $\mathbb{N}$^n×p is the gene expression matrix of the tissue slice, and Z ∈ $\mathbb{R}$^2×n is the spatial location matrix of each spot on the slice, where the jth column z_·j is the x-y coordinate of spot j on the 2D array³ used by the ST experiment. Here, n is the number of spots on the slice and p is the number of genes measured. x_ij ∈ $\mathbb{N}$ is the transcript count of gene j in spot i. Each row vector x_i· of X is the expression profile of spot i. Following Zeira et al. (2022), we encode the spatial location of each spot in a pairwise distance matrix $D=\left[d_{ij}\right]\in {\mathbb{R}}_{+}^{n\times n}$, where d_ij is the Euclidean distance between spot i and spot j on the slice, calculated from the 2D coordinates z_·i and z_·j. Thus, we represent an ST slice of n spots and p genes by a tuple (X, D).

Partial pairwise slice alignment problem

Given a pair (X, D) and (X′, D′) of ST slices, our goal is to compute a partial pairwise slice alignment, that is, to find a probabilistic spot-spot correspondence between spots in the two slices while accounting for the fact that some spots should not be mapped (Fig. 1). The probabilistic mapping is a matrix $\pi =\left[{\pi }_{ij}\right]\in {\mathbb{R}}^{n\times n^{\mathrm{\prime }}}$ between the n spots in one slice and n′ spots in the other slice, where π_ij describes the probability (or relative fraction) that a spot i in the first slice is aligned to a spot j in the second slice.

We begin by describing the solution given in Zeira et al. (2022) to the pairwise slice alignment problem, implemented in the PASTE algorithm. PASTE uses a formulation based on optimal transport to compute the mapping π. Specifically, given probability distributions g and g′ over the spots in slice X and X′, respectively, PASTE finds the map π (also known as the transport matrix) that minimizes the following transport cost:

$$F\left(\pi ;X,D,X^{\mathrm{\prime }},D^{\mathrm{\prime }},c,\alpha \right)=\left(1-\alpha \right)\sum _{i,j}c\left({\mathbf{x}}_{i\cdot },{{\mathbf{x}}^{\mathrm{\prime }}}_{j\cdot }\right){\pi }_{ij}+\alpha \sum _{i,j,k,l}{(d_{ik}-{d^{\mathrm{\prime }}}_{jl})}^2{\pi }_{ij}{\pi }_{kl}$$ (1)

subject to the regularity constraint that π has to be a probabilistic coupling between g and g′:

$$\pi \in \mathcal{F}\left(g,g^{\mathrm{\prime }}\right)=\{\pi \in {\mathbb{R}}^{n\times n^{\mathrm{\prime }}}|\pi \ge \mathbf{0},\pi {\mathbf{1}}_{n^{\mathrm{\prime }}}=g,{\pi }^T{\mathbf{1}}_n=g^{\mathrm{\prime }}\}.$$ (2)

Here, c:$\mathbb{R}$^p × $\mathbb{R}$^p → $\mathbb{R}$₊ is an expression cost function that gives a nonnegative dissimilarity score between the expression profiles of two spots over the same genes. 1_n is an all-one vector of length n. Typically, g and g′ are chosen to be uniform distributions over spots in each slice, although other distributions can be used (Zeira et al. 2022).

The PASTE objective function F is composed of an expression similarity term (first summand) and a spatial similarity term (second summand) weighted by a parameter α. The first term, also called the Wasserstein distance in the OT literature (Peyré and Cuturi 2019), represents the cost of moving one unit of probability mass from each spot i to each spot j, with the cost being the gene expression dissimilarity between spots. The second term, also called the Gromov-Wasserstein distance (Mémoli 2011; Peyré et al. 2016), approximately preserves the intraslice spatial distances between spots. Together, the convex combination of the two terms in F is known as the FGW optimal transport objective (Titouan et al. 2019).

The regularity condition (2) forces a rigid structure on π such that all spots from both slices must be aligned. However, such constraints may not be appropriate for ST slices with considerable differences in field of view or cell type composition because of both biological variation across tissue sections as well as differences caused by the manual nature of tissue dissection. Therefore, spots containing cell types or tissue regions that are unique to only one slice will be forced to be mapped to somehow arbitrary spots on the other slice.

Thus, in PASTE2, we propose to solve the partial pairwise slice alignment problem by minimizing the same objective function as PASTE (Equation 1), but with a different set of constraints that allow for unmapped spots. Specifically, given a parameter s ∈ [0, 1] describing the fraction of mass to transport between g and g′, we define a set $\mathcal{P}\left(g,g^{\mathrm{\prime }},s\right)$ of s-partial couplings between distributions g and g′ as

$$\mathcal{P}\left(g,g^{\mathrm{\prime }},s\right)=\{\pi \in {\mathbb{R}}^{n\times n^{\mathrm{\prime }}}|\pi \ge \mathbf{0},\pi {\mathbf{1}}_{n^{\mathrm{\prime }}}\le g,{\pi }^T{\mathbf{1}}_n\le g^{\mathrm{\prime }},{\mathbf{1}}_n^T\pi 1_{n^{\mathrm{\prime }}}=s\}.$$ (3)

The parameter s ∈ [0, 1] is interpreted as the overlap percentage between the two slices to align. The constraint ${\mathbf{1}}_n^T\pi {\mathbf{1}}_{n^{\mathrm{\prime }}}=s$ ensures that only the fraction of s probability mass is transported. Equivalently, if $g_i={\displaystyle \frac{1}{n}}$ is a point mass for each spot, then roughly the s fraction of the spots in each slice are aligned. The feasibility constraints π ≥ 0, $\sum _j{\pi }_{ij}\le g_i$ for all spots i in the first slice, and $\sum _i{\pi }_{ij}\le g_j^{\mathrm{\prime }}$ for all spots j in the second slice make sure that each spot only transports probability mass that it already has according to g and g′, hence ensures π to be a valid transport plan. In PASTE2 we require that the map π belongs to $\mathcal{P}\left(g,g^{\mathrm{\prime }},s\right)$, thus replacing the set $\mathcal{F}$ (defined in Equation 2) by the set $\mathcal{P}\left(g,g^{\mathrm{\prime }},s\right)$ (defined in Equation 3). In analogy to sequence alignment, PASTE calculates a global alignment, whereas PASTE2 calculates a local alignment.

The general concept of partial optimal transport (Caffarelli and McCann 2010) extends optimal transport theory to allow the transportation of only a specified fraction of mass between distributions. Here, we adapt the idea of partial optimal transport to the fused Gromov-Wasserstein objective, hence the PASTE2 optimization problem is a novel partial fused Gromov-Wasserstein (partial-FGW) optimal transport problem. Although there are existing solutions to the partial Wasserstein and partial Gromov-Wasserstein problem, to the best of our knowledge, PASTE2 is the first to state and formulate the partial-fused Gromov-Wasserstein problem and provide an optimization procedure to solve this problem.

PASTE2 has two parameters: α, the balance between the gene expression dissimilarity and the spatial dissimilarity, and the overlap percentage parameter s indicating the fraction of mass to transport. Unless otherwise specified, we set α = 0.1 following Zeira et al. (2022). We choose the value of s using a model selection procedure described in Supplemental Material S1. The choice of c, the expression dissimilarity function, is described in Supplemental Material S2.

An iterative conditional gradient algorithm for optimization

We derive an optimization algorithm to minimize the objective (1) subject to the constraint (3). This problem is a large-scale (each slice contains thousands of spots) nonconvex quadratic program with a convex and compact feasible region. Our algorithm is based on the Frank-Wolfe optimization algorithm (Frank and Wolfe 1956), also known as the conditional gradient (Levitin and Polyak 1966) algorithm. This algorithm has been widely adopted in the optimal transport community (Ferradans et al. 2014; Flamary et al. 2014; Titouan et al. 2019; Chapel et al. 2020) to compute transport plans because of its ability to handle large-scale quadratic programs (Jaggi 2013). The optimization problem in PASTE2 is thus particularly suitable for the conditional gradient algorithm.

The conditional gradient algorithm is an iterative first-order algorithm for constrained optimization. To fit in the conditional gradient scheme, we first write Equation 1 in matrix form, following Peyré et al. (2016):

$$F\left(\pi \right)=\left(1-\alpha \right){⟨\mathbf{C},\pi ⟩}_F+\alpha {⟨\mathbf{L}\left(D,D^{\mathrm{\prime }}\right)\otimes \pi ,\pi ⟩}_F,$$ (4)

where $\mathbf{C}\in {\mathbb{R}}^{n\times n^{\mathrm{\prime }}}$ encodes the gene expression dissimilarity $c_{ij}=c\left({\mathbf{x}}_{i\cdot },{{\mathbf{x}}^{\mathrm{\prime }}}_{j\cdot }\right)$ between each spot i in the first slice and each spot j in the second slice, and $\mathbf{L}\left(D,D^{\mathrm{\prime }}\right)\in {\mathbb{R}}^{n\times n^{\mathrm{\prime }}\times n\times n^{\mathrm{\prime }}}$ is a 4D tensor defined by ${\mathbf{L}}_{i,j,k,l}\left(D,D^{\mathrm{\prime }}\right)={\left(D_{ik}-D_{jl}^{\mathrm{\prime }}\right)}^2$. ⊗ is the tensor-matrix multiplication operator; that is, L ⊗ π is an n × n′ matrix whose (i, j)th element is $(\sum _{k,l}{\mathbf{L}}_{i,j,k,l}\cdot {\pi }_{k,l})$. <,> denotes the Frobenius dot product of matrices.

In each iteration, the algorithm moves in the direction that minimizes a linear approximation of the objective function while remaining in the feasible region. The mathematical details of the derivation of each step, as well as the pseudocode, are provided in Supplemental Material S3.

Using histological image data in alignment

We further extend the PASTE2 partial-FGW framework to incorporate image information. Specifically, we replace the gene expression dissimilarity matrix $\mathbf{C}\in {\mathbb{R}}^{n\times n^{\mathrm{\prime }}}$ in Equation 4 by a sum of two n × n′ dissimilarity matrices ${\displaystyle \frac{1}{2}{\mathbf{C}}_{gene}+{\displaystyle \frac{1}{2}{\mathbf{C}}_{image}}}$, where C_gene is the gene expression dissimilarity matrix as defined above and C_image encodes the dissimilarity between the image information at each spot. Thus, we seek a map π that minimizes the following objective function:

$$F\left(\pi \right)=\left(1-\alpha \right){⟨{\displaystyle \frac{1}{2}{\mathbf{C}}_{gene}+{\displaystyle \frac{1}{2}{\mathbf{C}}_{image},\pi }}⟩}_F+\alpha {⟨\mathbf{L}\left(D,D^{\mathrm{\prime }}\right)\otimes \pi ,\pi ⟩}_F.$$ (5)

Note that to avoid an extra parameter we give equal weight ${\displaystyle \frac{1}{2}}$ to both gene expression and image information, although substituting other weights is straightforward. Also, because C_gene and C_image may not be on the same scale, we scale C_image such that the maximum entry of C_image equals the maximum entry of C_gene. In our implementation, we define [C_image]_ij to be the Euclidean distance between the mean RGB values of the spots. See Supplemental Material S4 for further details.

3D Reconstruction based on the partial alignment matrix

Given a series of consecutive, (partially) overlapping slices from the same tissue, we aim to reconstruct the spatial expression of the tissue in 3D by transforming PASTE2 partial pairwise alignments into a common coordinate system. Specifically, given a series of consecutive slices we first find partial alignments between adjacent slices by solving the partial pairwise slice alignment problem as above. To project all slices onto a common coordinate system, we extend the generalized weighted Procrustes analysis (Wahba 1965; Kabsch 1976) approach in Zeira et al. (2022) to sequentially project each pair of adjacent slices. Whereas Zeira et al. (2022) projects a pair of slices onto the same coordinate system by centering both slices followed by calculating a rotation matrix, we derive the centering step for each slice separately to address the case in which the alignment matrix π is partial and the aligned regions of the two slices have unique barycenters. The details of the projection are in Supplemental Material S5.

Results

Evaluation on simulated ST data

We first compared PASTE2 and PASTE on a simulated ST data set based on a human dorsolateral prefrontal cortex (DLPFC) tissue slice from Maynard et al. (2021). Specifically, we extracted two partially overlapping subslices from a single DLPFC slice (sample 151674) with varying overlap percentages 90%, 70%, 50%, 30% (Fig. 2A). To perturb the gene expression, we resample the gene expression profile of each spot in one of the subslices by sampling from a multinomial distribution with added pseudocount δ, which controls the noise level (Supplemental Material S6). We vary the pseudocount δ in the range⁴ from 0.1 to 2 with an increment of 0.1. In total, we generated 4 × 30 = 120 pairs of subslices with different overlap percentages and noise levels δ. For each pair of subslices, we ran PASTE2 with α = 0, 0.1, 1 and using the ground-truth value for the overlap fraction s (we evaluated model selection separately in Supplemental Material S9), as well as full PASTE with default parameters (α = 0.1). We evaluated the alignment using Label Transfer Adjusted Rand Index (LTARI). Given a labeling of cell type/spatial region of spots, LTARI measures how well the alignment preserves the label between the aligned spots. LTARI first defines a new spot labeling for the second slice by assigning to each aligned spot the label of the most likely corresponding spot in the first slice, then calculates the ARI between the induced spot labeling of the second slice and the ground-truth labeling (Supplemental Material S7). We used the manual cortical layer annotation from Maynard et al. (2021) as ground-truth spot labeling (Fig. 2A).

Figure 2.

Comparison of PASTE2 and PASTE on simulated partially overlapping subslices. (A) DLPFC slice 151674 with spots colored according to the manual annotations of cortical layers from Maynard et al. (2021). Red box and blue box indicate two partially overlapping subslices, with central overlapping region containing some fraction of spots from each slice. (B) Label Transfer ARI of the alignments produced by PASTE2 with α = 0 (gene expression information only), PASTE2 α = 1 (spatial information only), PASTE2 α = 0.1 (both), and PASTE (full alignment, α = 0.1) as a function of the pseudocount (δ) for overlap percentage 70%. (C) Label Transfer ARI for overlap percentage 50%.

We found that PASTE2 with the default parameter setting of α = 0.1, which uses both gene expression information and spatial information, outperforms PASTE across most values of the added noise δ for every overlap percentage (Fig. 2B,C; Supplemental Fig. S1). Specifically, for all four overlap percentages, PASTE2 (α = 0.1) achieves the highest LTARI when δ < 2.0, and achieves almost perfect LTARI when δ is small. Note that PASTE obtains constant accuracy because it aligns overlapping regions well but nonoverlapping regions arbitrarily. The gap in accuracy between PASTE2 and PASTE is larger when the overlap is smaller. This indicates that PASTE, which finds an alignment between all pairs of spots, is not suitable for the partial alignment task. In contrast, PASTE2 has high accuracy in partial alignment across a wide range of overlap percentages and gene expression noise. In particular, PASTE2 achieves near perfect LTARI when the pseudocount δ is in the range ≈0.1−0.2 that matches the variability in read counts observed in real data (Zeira et al. 2022), although the performance of PASTE2 does degrade for substantially larger pseudocount values δ > 1.

To investigate the effect of the misspecification of the value of the overlap percentage parameter s on the result of PASTE2, we ran PASTE2 with s ranging from 0.1 to 1, with a step size of 0.1, on a simulated pair in which the ground-truth overlap percentage is 50% and the added pseudocount is 0.1. We found that PASTE2 aligns correctly when s is lower than the ground truth, whereas the performance degrades for larger values of s (Supplemental Fig. S2). This is expected because with an overestimation of the overlap percentage, the PASTE2 alignment becomes more similar to the PASTE alignment which includes all the spots. Thus, in selecting a value for s, it is preferable to use a model selection procedure that slightly underestimates s rather than overestimates s. We propose a heuristic for selecting s in Supplemental Material S1.

Because the ground-truth 1-1 spot alignment for these simulated pairs is known, we also evaluated PASTE2 alignments using another metric, precision, which is defined as the amount of mass transported between each pair of corresponding spots in the ground-truth alignment over the total mass transported. For a given alignment, the precision measures the percentage of aligned spot pairs that are ground-truth correspondence, and the result shows similar trends as LTARI, where PASTE2 achieves the highest performance in the reasonable range of δ (Supplemental Fig. S3).

Finally, we emphasize the importance of using both gene expression and spatial information in computing accurate partial alignments. PASTE2 (α = 1.0) has consistently low LTARI, indicating that using spatial coordinates alone cannot recover alignment across slices. The performance of PASTE2 (α = 0), which only uses gene expression information of each spot for alignment, drops more quickly than PASTE2 (α = 0.1) with increasing pseudocount δ indicating more noise in gene expression. Using both gene expression and spatial information, PASTE2 is able to accurately align two partially overlapping ST slices. We investigated the effect of different intermediate values of α on the alignment performance (Supplemental Fig. S4) and found that α between 0.1 and 0.5 results in good performance.

Human dorsolateral prefrontal cortex (DLPFC) slices

We next compared PASTE2 to PASTE (Zeira et al. 2022) and two other transcriptomics alignment methods—Pamona (Cao et al. 2022) and Tangram (Biancalani et al. 2021)—on the full human DLPFC data set containing 10x Genomics Visium ST data from three individuals (labeled sample 1, 2, 3) with four slices (labeled slice A, B, C, D) per individual (Maynard et al. 2021). For each individual, slice A and B are 10 µm apart, the same as slice C and D. Slice B and C are further apart at a distance of 300 µm, hence slice pairs AB and CD are more similar to each other than slice pair BC. Note that Pamona (Cao et al. 2022), a manifold alignment algorithm for multi-omics data sets, is also based on partial optimal transport; Tangram (Biancalani et al. 2021) is a deep learning–based method that aligns scRNA-seq data onto ST data. We create partial ST alignment problems by generating two partially overlapping DLPFC data sets as follows. For each individual, we extracted the left portions of slice A and C, and the right portions of slice B and D such that the extracted pairs AB, BC, and CD have ≈70% overlap in area (Fig. 3A). We also created another set of partially overlapping data using horizontal slices (Supplemental Fig. S5A). We ran each of the methods as described in Supplemental Material S8. We evaluate the accuracy of each method by computing the label transfer ARI (LTARI) as previously described.

Figure 3.

Comparison of alignment methods on partially overlapping DLPFC slices. (A) Vertical subslices were obtained by cropping subslices (blue dotted boxes) from four adjacent slices from DLPFC sample 3, with indicated distances between adjacent slices. Each pair of adjacent subslices overlaps in 70% of their areas. (B) LTARI of pairwise alignments computed by PASTE2, PASTE, Pamona, and Tangram for each pair of adjacent vertical subslices from three samples. (C) Optimal projection of vertical subslices from slice AB of sample 3 onto the same 2D coordinate system using the PASTE2 partial alignment.

We find that PASTE2 achieves the highest LTARI on all adjacent subslices of all individuals, for both vertical partial slices and horizontal partial slices, with the exception of one pair (Fig. 3B; Supplemental Fig. S5B). About 70%–75% of the spots from each subslice is aligned in each pair indicating the parameter s corresponds well with slice overlap. For most pairs, PASTE2 has more than twice the LTARI than all other methods, demonstrating PASTE2's ability to identify the overlap region of the two ST slices and align the overlap region reliably. On one pair, Pamona has slightly higher LTARI than PASTE2 (Supplemental Fig. S5B), but all methods have very low LTARI (<0.1), suggesting that this pair has low spatial coherence. PASTE is the second-highest performing method on most pairs, indicating that even though PASTE does not model partially overlapping slices, it is still more suitable for aligning spatial transcriptomics data than methods designed for different purposes. Although Pamona is designed to align data sets with both shared and data-set-specific cells (Cao et al. 2022)—the analog of the partial pairwise slice alignment problem for single cell data sets—Pamona does not model spatial constraints, perhaps explaining its lower performance. Tangram assumes the single-cell gene expression data set and the spatial data set come from the same anatomical region (Biancalani et al. 2021); hence, the partial slice alignment task violates the Tangram assumption, leading to a low alignment accuracy.

For a more intuitive demonstration of PASTE2's advantage and accuracy, we projected the vertical subslices of sample 3 pair AB onto the same coordinate system, computed as described in Methods based on the alignment matrix computed by PASTE2 (Fig. 3C), as well as the optimal projection of the same pair based on the alignment computed by PASTE (Supplemental Fig. S6A). Qualitatively, the projection of PASTE2 correctly stacks the overlap area of the two slices, with spots from the same cortical layer stacking on top of each other, although PASTE fails to find the corresponding layers in the two slices. Additionally, PASTE2 correctly identifies and aligns the overlap area of all four partial slices of an individual while leaving the rest unaligned, leading to a visually correct 3D reconstruction of the tissue from partial slices (Supplemental Fig. S6B).

We also ran STUtility (Bergenstråhle et al. 2020), a method to align H&E stained images that are generated as part of the 10x Genomics Visium ST workflow. STUtility outputs new coordinates of the aligned slices and does not produce a mapping between pairs of spots; thus, we visualized the alignment results by plotting each pair of partial subslices according to the new coordinates output by STUtility (Supplemental Fig. S7). The image masking function used by STUtility failed for the partial slices of sample 3, so we only visualized the results for samples 1 and 2. STUtility correctly identifies that each pair of input slices are partially overlapping, but it does not align the correct overlapping region, and the output alignment seems quite arbitrary. These results might be because STUtility aligns images by identifying and finding correspondences between edges of the two input tissues, but when two tissues are partially overlapping, the edges do not provide information about spot correspondences. On the other hand, PASTE2 correctly aligns the overlapping region (Fig. 3C). This shows that using transcriptomic similarity, spatial similarity, and image information yields better partial alignments than H&E images alone.

Finally, we compared PASTE2's running time with other methods on the vertical subslices of sample 3. PASTE2 finished in under 10 min for all subslice pairs on a Macbook Pro with 2.4GHz Intel Core i5 CPU, with most of the running time spent on the GLM-PCA subroutine (Supplemental Fig. S8). The conditional gradient optimizer in PASTE2 runs in less than half of the time of Pamona and Tangram, and only runs slightly slower than PASTE. We also used the DLPFC data sets to evaluate the accuracy of PASTE2's model selection procedure for estimating the overlap percentage s (Supplemental Material S9), and found that PASTE2 correctly estimates the overlap percentage in many scenarios (Supplemental Figs. S9, S10).

Incorporating histology information improves alignment

We compared PASTE2's alignment performance when using both gene expression and histological image (Equation 5) versus using only gene expression data (Equation 4). Note that spatial information is included in both analyses. We ran the two modes of PASTE2 on pairs of horizontal and vertical subslices from DLPFC sample 3. We found that using the histological image substantially improved the alignment performance for pair CD (Fig. 4A,B), increasing the LTARI from 0.34 to 0.46. Examining the alignment obtained on this pair using only gene expression information (Fig. 4C) to the alignment obtained with both gene expression and the histology image (Fig. 4D), we observe that the alignment obtained using the images is more spatially contiguous. In particular, there is a curve of unaligned spots (blue spots in Fig. 4C) in subslice D inside the yellow region. This curve corresponds to spots that are manually annotated as Layer 6 of the DLPFC (Maynard et al. 2021). Notably, the spots in this layer have lower total UMI counts than other layers: the mean total UMI counts for spots from Layer 6 is 2915 compared to total UMI counts ≈4500 in the other layers. This suggests that the gene expression signal is weaker in these spots. In contrast, the PASTE2 alignment obtained using both gene expression and image information (Fig. 4D) does not have the same curve of unaligned spots, demonstrating the advantages of using the histological image for spots with a weak gene expression signal.

Figure 4.

Evaluating the benefit of using histological image information in PASTE2 alignment. (A) The label transfer ARI (LTARI) of PASTE2 partial alignments of pairs of vertical subslices extracted from DLPFC sample 3 using only gene expression (blue) and using both gene expression and image information (orange). (B) Histological images of sample 3 slice C and slice D. The red boxes bound the vertical subslices extracted for partial alignment. The right part of subslice C should be aligned to the left part of subslice D. (C) Visualization of PASTE2 alignment of the subslice pair CD using gene expression and spatial information. Yellow spots are aligned by PASTE2, whereas blue spots are unaligned. Thin black lines connect pairs of spots that are aligned by PASTE2 with high weight. (D) Visualization of PASTE2 alignment of the same subslice pair when gene expression, histological image, and spatial information are all used.

In the horizontal slices, we see cases in which using the image information reduces the alignment performance. For example, the LTARI drops from 0.56 (expression) to 0.50 (expression and image) for horizontal subslices of pair AB (Supplemental Fig. S11A,B). The aligned part of subslice B in Supplemental Figure S11D shows that many spots are left unaligned in the actual overlap region, and there is a clear stripe of unaligned blue spots towards the left part of the subslice. Looking at the H&E image of subslice B in Supplemental Figure S11B, we see a clear dark stain on the left of the subslice that is missing from the image of subslice A, at exactly the same location of the unaligned stripe. This indicates that the stain on the H&E image is the cause for the worse alignment performance.

For the other pairs, the LTARI for PASTE2 alignments with and without images is approximately the same. This is not too surprising because the H&E images of DLPFC slices do not display strong heterogeneity across different layers (Fig. 4B; Supplemental Fig. S11B). Thus, gene expression appears to be the dominant signal in computing the alignment in this data set. To evaluate further, we ran PASTE2 with only image information on the six pairs of sample 3, and found that the LTARI is lower than using only expression or using both expression and image on all pairs (Supplemental Fig. S12), which shows that, on this data set, histological images alone cannot provide enough signal for finding an accurate alignment. However, the fact that utilizing image information corrects the alignment of low UMI spots shows the potential for histological images to guide PASTE2 alignment. The image information can help overcome the sparsity of gene expression, and when the histological images have greater variation across spots, using the images should further improve the alignment quality by complementing the gene expression signal.

Spatial transcriptomics of Drosophila embryo

We applied PASTE2 to analyze a Stereo-seq data set from a Drosophila embryo (Chen et al. 2022). Stereo-seq is a new SRT technology with 500-nm resolution, two orders of magnitude smaller than the 10x Visium platform, but with lower UMIs per spot. Wang et al. (2022) applied Stereo-seq to two late-stage Drosophila embryos 14–16 h and 16–18 h after egg laying (labeled E14–16 and E16–18) and three stages of larvae (labeled L1–L3). Each slice has ≈1000 spots with median UMI per spot of ≈2000, compared to ≈4000 spots and ≈5000 median UMI per spot in the 10x Visium DLPFC data set. They derived the cell type of each spot by unsupervised clustering of gene expression followed by annotation based on marker genes. The publication used PASTE to align all slices from the same stage and obtain a 3D map of spatial expression of each stage. However, slices from the same stage vary in size and cell type compositions and do not fully overlap in space. For example, inspection of annotated cell types shows that adjacent slices from the E14–16 sample do not fully overlap (Supplemental Fig. S13). Therefore, it is appropriate to use PASTE2 to realign the adjacent slices respecting the different composition of cell types across slices, and to obtain a more accurate 3D reconstruction of the Drosophila embryo.

We applied PASTE2 to compute a partial alignment for each pair of 16 adjacent slices from the E14–16 sample, estimating the overlap percentage using the PASTE2 model-selection heuristic (Supplemental Material S1). Slices 7 and 8 have clear differences in the composition of cell types annotated by Wang et al. (2022), with the carcass cells showing the largest difference in proportion (Fig. 5A). PASTE2 addresses this imbalance by aligning a similar proportion of carcass cells across slices, leaving the excess cells in slice 8 unaligned. The spots from the two slices included in the PASTE2 partial alignment show similar spatial organization (Fig. 5B) and cell type composition (Supplemental Fig. S14B). For example, the proportions of carcass cells in slices 7 and 8 differ by 10% before alignment (Supplemental Fig. S14A), but after alignment the difference is <3% (Supplemental Fig. S14B). The differences in proportions shrink for salivary gland cells as well, indicating PASTE2 correctly identifies and aligns the overlapping parts. PASTE2 optimal projection of the two slices to the same coordinate system puts slice 8 slightly higher in y coordinates than slice 7, consistent with the observation that slice 8 has unaligned carcass cells at the top (Fig. 5C). The LTARI obtained by PASTE2 for this pair is 0.49, compared to an LTARI of 0.39 for PASTE, again showing the advantages of partial alignment. Examination of pair of adjacent slices 14 and 15 shows a similar advantage of partial alignment. Slice 15 has a stripe of carcass cells that is absent in slice 14 (Supplemental Fig. S15A). PASTE2 leaves the stripe unaligned across slices (Supplemental Fig. S15B,C), increasing the LTARI from 0.29 for PASTE to 0.52 for PASTE2. Because PASTE computes an alignment for all spots, the extra carcass cells in slice 15 are mapped somewhere on slice 14, creating false correspondences between spots (Supplemental Fig. S16).

Figure 5.

PASTE2 alignment of Stereo-seq data from an E14–16 Drosophila embryo from Wang et al. (2022). (A) Stereo-seq slices 7 and 8 with spots labeled by cell types annotated in Wang et al. (2022). (B) Spots from slices 7 and 8 that are included in the partial alignment computed by PASTE2. Spots selected by PASTE2 have similar proportions of cell types and spatial locations. (C) Optimal projection of slices 7 and 8 onto the same 2D coordinate system using the PASTE2 partial alignment. (D) PASTE2 3D reconstruction using all 16 slices from the Drosophila embryo.

We compared the LTARI of the PASTE2 alignment with the LTARI scores of PASTE, Pamona, and Tangram on every pair of adjacent slices. PASTE2 achieves the highest LTARI for most pairs, with the largest gain in pairs in which the two slices have different compositions of cell types, such as slice 14 and 15 (Supplemental Fig. S17). Pairs in which PASTE2 does not obtain the highest LTARI, such as slice 2 and 3, have relative similar sizes and cell types, and PASTE2 still achieves comparable LTARI with the highest performing method. This indicates that PASTE2 not only aligns partially overlapping slices correctly, but also performs well on pairs of similar slices.

We used PASTE2 to generate a 3D reconstruction of all 16 slices of the E14–16 Drosophila embryo, in which adjacent slices have on average 70% of overlapping spots (Fig. 5D). The PASTE2 3D reconstruction will be useful for refining the analyses presented in Wang et al. (2022) who showed that the PASTE-generated 3D expression helped detect functional subregions and uncover the dynamics of cell state changes and tissue-specific gene regulation.

Discussion

We present PASTE2, a method to perform pairwise alignment and 3D reconstruction of multislice spatial transcriptomics data. PASTE2 addresses the important situation in which slices partially overlap in space or have different cell type compositions, which is the case for most real data sets. We formulate the ST partial pairwise alignment problem using a partial fused Gromov-Wasserstein optimal transport framework and derive an optimization algorithm to solve this problem. We further design a model selection procedure to determine the overlap between slices, and extend the framework to incorporate both gene expression and imaging information.

We show on simulated data that PASTE2 achieves accurate alignment and outperforms PASTE when slices do not fully overlap. We found that PASTE2 outperforms multiple other methods for alignment of spatial transcriptomics or single cell data including PASTE, Pamona, Tangram, and STUtility on ST data set from the human DLPFC (Maynard et al. 2021). We show that PASTE2's use of histology images can further improve alignments, although the results are variable depending on the quality of the images. We expect that PASTE2 will achieve much higher accuracy incorporating image information in data sets in which histological images display stronger signal across spots—in preliminary results on unpublished cancer data sets with high-quality H&E images we observed even larger gains. Finally, we show PASTE2's capabilities on larger data sets from another SRT technology by generating a 3D reconstruction of a Drosophila embryo from 16 slices of Stereo-seq data.

There are multiple directions for future work. First, is to extend the partial alignment framework to integrate multiple slices into a single consensus slice to address the data sparsity issue by pooling counts from corresponding spots (Zeira et al. 2022). Second, one could stitch together multiple partially overlapping slices into a larger 2D slice. This stitching would be helpful in cases in which adjacent tissue slices are close in the z-coordinate which is often the case with thin tissue slices ( ≈ 10 μm). In addition, one could incorporate additional spatial regularization terms to enforce more contiguous overlapping regions. Third, it would be interesting to apply PASTE2 to integrated spatial transcriptomics and imaging data from other platforms such as Slide-seq (Rodriques et al. 2019; Stickels et al. 2021), or combined Stereo-seq and imaging data which Wang et al. (2022) noted as a future technology development. Finally, it would be interesting to examine the effectiveness of other optimal transport frameworks such as unbalanced OT (Séjourné et al. 2021) that impose soft constraints rather than hard constraints on partial alignments.

We anticipate that PASTE2 will be a useful tool for integrating transcriptomic information across multislice ST data sets and for building 3D tissue atlas across both normal and diseased tissues, such as in the Human Tumor Atlas Network and related projects.

Software availability

PASTE2 has been implemented in an open-source, publicly available Python package that is available at GitHub (https://github.com/raphael-group/paste2). All the code to reproduce the analysis can be found in Supplemental Code.

Competing interest statement

The authors declare no competing interests.