CO-CLUSTERING OF SPATIALLY RESOLVED TRANSCRIPTOMIC DATA

Abstract

Spatial transcriptomics is a groundbreaking technology that allows the measurement of the activity of thousands of genes in a tissue sample and maps where the activity occurs. This technology has enabled the study of the spatial variation of the genes across the tissue. Comprehending gene functions and interactions in different areas of the tissue is of great scientific interest, as it might lead to a deeper understanding of several key biological mechanisms, such as cell-cell communication or tumor-microenvironment interaction. To do so, one can group cells of the same type and genes that exhibit similar expression patterns. However, adequate statistical tools that exploit the previously unavailable spatial information to more coherently group cells and genes are still lacking.

In this work, we introduce SpaRTaCo, a new statistical model that clusters the spatial expression profiles of the genes according to a partition of the tissue. This is accomplished by performing a co-clustering, i.e., inferring the latent block structure of the data and inducing two types of clustering: of the genes, using their expression across the tissue, and of the image areas, using the gene expression in the spots where the RNA is collected. Our proposed methodology is validated with a series of simulation experiments and its usefulness in responding to specific biological questions is illustrated with an application to a human brain tissue sample processed with the 10X-Visium protocol.

1. Introduction.

1.1. The rise of spatial transcriptomics.

In the last few years, we have witnessed a dramatic improvement in the efficiency of DNA sequencing technologies that ultimately gave rise to new advanced protocols for single-cell RNA sequencing (scRNA-seq) and, more recently, spatial transcriptomics. In particular, spatial transcriptomics has been chosen as method of the year 2020 (Marx, 2021). With respect to scRNA-seq, spatial transcriptomic platforms are able to provide, in addition to the abundance, the locations of thousands of genes in a tissue sample.

Righelli et al. (2021) classify spatial transcriptomic protocols into molecule-based and spot-based methods. Among molecule-based methods, seqFISH (Lubeck et al., 2014) and similar methods, such as MERFISH (Chen et al., 2015), are capable of providing the spatial expression of thousands of transcripts at a sub-cellular level, but the setup necessary to perform this kind of spatial experiments is often complex and expensive to recreate. Spot-based methods, such as Slide-seq (Rodriques et al., 2019) or the 10X Genomics Visium platform (Rao, Clark and Habern, 2020), have substantially lower resolution than seqFISH, but allow scientists to measure close to the whole transcriptome of (small pools of) cells across a tissue in a relatively easy manner.

Briefly, in the Visium platform, the data collection process is performed by placing a slice of the tissue of interest over a grid of spots, so that every spot contains a few neighboring cells. The gene expression of each spot is then characterized, resulting in a dataset made of tens of thousands of genes for each spot, together with the spatial location of the spots. Figure 1 shows an example of human dorsolateral prefrontal cortex (DLPFC) processed with Visium at the Lieber Institute for Brain Development (Maynard et al., 2021). The colored dots denote a manual annotation of the spots performed by Maynard et al. (2021). The dataset is available in the R package spatialLIBD (Pardo et al., 2021).

FIG 1.

Tissue sample of LIBD human dorsolateral prefrontal cortex (DLPFC) processed with Visium platform and stored in the R package spatialLIBD. The dots represent the spots over the chip surface. Different colors denote a manual annotation of the areas performed by Maynard et al. (2021): they recognize a White Matter (WM) stratum in the bottom-left part of the image, and 6 Layers (from L6 to L1) moving toward the top-right.

The rise of spatial transcriptomics has motivated the development of new statistical methods that handle the identification of spatially expressed (s.e.) genes, i.e., genes with spatial patterns of expression variation across the tissue. Specific inferential procedures for detecting such kind of genes, such as SpatialDE (Svensson, Teichmann and Stegle, 2018) and Trendsceek (Edsgärd, Johnsson and Sandberg, 2018), have been proposed only in the last years. These methods are widely computationally efficient, but sometimes they reach discordant inferential conclusions, and additionally they fail to account for the correlation of the genes. The very recent algorithm by Sun, Zhu and Zhou (2020), called SPARK, has addressed some of the limitations of the earlier methods. However, the additional information brought by the new spatial transcriptomic platforms has raised several questions, both on the biological and the statistical side: detecting the s.e. genes is thus not the end of the analysis but just its beginning. In this article, we want to focus on three specific research questions, i.e., to determine:

1. the clustering of the areas of the tissue sample according to the spatial variation of the genes;

2. the existence of clusters of genes which are s.e. only in some of the areas discovered from i.);

3. the highly variable genes in the areas discovered from i.) net of any spatial effect.

Research question i.) is fundamental for the analysis of tissue samples because it is the starting point for successive downstream analyses. The recent GIOTTO (Dries et al., 2021) and BayesSpace (Zhao et al., 2021) methods are unsupervised clustering algorithms for spot-based spatial transcriptomics, designed for inferring the cell types making up a tissue. They perform a clustering based on the principle that neighboring spots are likely to be annotated with the same label, without exploiting the information carried by s.e. genes. Thus, these methods respond to a substantially different research question than i.).

Research question ii.) is of great scientific interest, but, to the best of our knowledge, has not been tackled yet. Discovering that some genes are s.e. only in some areas of the tissue would play a core role in comprehending some fundamental biological mechanisms, and ultimately discovering new ones. Even the very recent SPARK method for detecting s.e. genes is not designed to state if the spatial expression activity of a gene is restricted to specific areas of the tissue. With the existing statistical tools, one can approach this issue with a two-step analysis, first clustering the image using BayesSpace or GIOTTO, and then applying SPARK to each of the discovered clusters. However, such heuristic procedure has some severe limitations. First, repeating the tests in each of the image cluster requires to control for multiple testing, e.g., by controlling the False Discovery Rate (Benjamini and Hochberg, 1995). Second, even after the s.e. genes are isolated, an additional clustering of the genes is necessary to perform specific downstream analyses (Svensson, Teichmann and Stegle, 2018; Sun, Zhu and Zhou, 2020). Last, if indeed there are clusters of genes, such information should be accounted for in the first step of the procedure, when the image is clustered. However, this is something that cannot be accomplished with BayesSpace or GIOTTO.

Finally, research question iii.) has the goal of determining which genes are active in each of the image cluster. Thanks to the spatial mapping of the spots, it will be possible to separate the presence of spatial effects from the total variation of each gene, providing a more accurate list of highly variable genes.

1.2. A co-clustering perspective.

In this article, we consider the problem of modelling and clustering gene expression profiles in a tissue sample processed with a spot-based spatial transcriptomic method, such as 10X Visium, and measured over a set of spatially located sites.

In the remainder of the article, we use “spots” to denote the spots in the tissue from which RNA is extracted and “genes” to denote the variables measured in each spot, using a terminology typical of the Visium platform. However, the method presented here is more general and can be applied to any spatial transcriptomic technology and, more broadly, to any dataset for which the rows or the columns are measured in some observational sites with known coordinates.

We tackle the research questions outlined above as a single, two-directional clustering problem: of the genes, using spots as variables, and of the spots, using genes as variables. This kind of procedure is known in the literature as co-clustering (or block-clustering, Bouveyron et al., 2019) and denotes the act of clustering both the rows and the columns of a data matrix, which, in this way, is partitioned into rectangular, non-overlapping sub-matrices called co-clusters (or blocks).

Bouveyron et al. (2019) distinguish between deterministic and model-based co-clustering approaches. Model-based methods are designed to simultaneously perform the clustering and reconstruct the probabilistic generative mechanism of the data. The model-based co-clustering literature is centered around the Latent Block Model (LBM; Govaert and Nadif, 2013), an extension of the standard mixture modelling approach when both rows and columns of a data matrix are deemed to come from some underlying clusters. Thanks to the ease of interpretation and to the raise of new advanced computational methods, the LBM has been extensively explored as a tool for modelling continuous (Govaert and Nadif, 2013, Chapter 5), categorical (Keribin et al., 2015), count (Govaert and Nadif, 2010), binary (Govaert and Nadif, 2008) and recently even functional data (Bouveyron et al., 2018; Casa et al., 2021). In addition, both frequentist (Govaert and Nadif, 2008; Bouveyron et al., 2018) and Bayesian (Wyse and Friel, 2012; Keribin et al., 2015) approaches have been proposed for fitting these models. The conditional independence assumption of LBM states that the observations within the same co-cluster are independent. Surely, this hypothesis is computationally attractive, yet it is incompatible with the high correlation levels shown by gene expression data (Efron, 2009).

Tan and Witten (2014) overcome the conditional independence assumption proposing a co-clustering model based on the matrix variate Gaussian distribution (Gupta and Nagar, 2018), which accounts for the dependency across the rows and the columns in a block with two non-diagonal covariance matrices. Their model represents a first attempt to extend k-means-type algorithms for co-clustering to the case where the data entries in a block are not independent. The estimation of the needed covariance matrices is challenging; a challenge that can be overcome with the aid of a penalization term, such as the LASSO (Witten and Tibshirani, 2009), to avoid singularity problems. However, with spatial data, it is natural to leverage the spatial dependencies observed in the data to aid the covariance matrix estimation.

Here, we propose SpaRTaCo (SPAtially Resolved TrAnscriptomics CO-clustering), a novel co-clustering technique designed for discovering the hidden block structure of spatial transcriptomic data. Since the spots in which gene expression is measured are spatially located on a grid, our model expresses the correlation across transcripts in different spots as a function of their distances. As a consequence, differently from the rest of the co-clustering models proposed in the literature, SpaRTaCo divides the data matrix into blocks based on the estimated means, variances, and spatial covariances. In addition, we use gene-specific random effects to account for the remaining covariance not explained by the spatial structure.

Although the published literature is not always clear about the distinction between co-clustering and biclustering, in accordance with the recent works of Moran, Ročková and George (2021) and Murua and Quintana (2021) here we adopt the following terminology: both co-clustering and biclustering are families of techniques used to group the rows and the columns of a data matrix. However, in biclustering the groups formed, called biclusters, can take any possible shape, while co-clustering is limited to rectangular, non-overlapping blocks. In addition, biclustering algorithms do not necessarily allocate all the data entries into one of the existent biclusters, and so some entries can be left unassigned. Although biclustering methods are more flexible, the main advantage of co-clustering is that the returned blocks are often easier to interpret both from a statistical and practical perspective.

1.3. Outline.

The rest of the manuscript is structured as follows. Section 2 illustrates the SpaRTaCo modelling approach and reviews some competing co-clustering models, highlighting the similarities and the differences with our proposal. Section 3 discusses some identifiability issues, illustrates our classification-stochastic EM (CS-EM) algorithm for parameter estimation, proposes a measure to quantify the clustering uncertainty, and derives a model selection criterion based on the integrated completed log-likelihood (Biernacki, Celeux and Govaert, 2000). Section 4 proposes five simulated spatial experiments of growing complexity with whom we compare SpaRTaCo with other co-clustering models. Section 5 shows how our proposal allows to answer our three research questions using the human brain tissue sample displayed in Figure 1. The manuscript is concluded by some considerations of the possible future extensions.

2. The statistical model.

Let $\text{X}={\left(x_{ij}\right)}_{1\le i\le n\text{,}1\le j\le p}$ be the $n\times p$ matrix of a spatial experiment processed by a spot-based spatial transcriptomic platform, i.e, containing the expression of $n$ genes over a grid of $p$ spots on the chip surface. The spatial location of the spot $j$ over the chip surface is known through its spatial coordinates ${\mathrm{s}}_j=\left(s_{jx}\text{,}\hspace{0.17em}{s}_{jy}\right)$; we name as $S={\left(s_j\right)}_{1\le j\le p}$ the $p\times 2$ matrix containing the coordinates of the $p$ spots. From this point, we assume that the data entries in $\mathrm{X}$ have been properly pre-processed, and so $x_{ij}\in ℝ$ for any $i$ and $j$ (see Section 5).

2.1. Model formulation.

We assume there exist $K$ clusters of rows of $\mathrm{X}$, and $R$ clusters of columns of $\mathrm{X}$, forming a latent structure of $KR$ blocks. The vectors of random variables $\mathcal{Z}={\left({\mathcal{Z}}_i\right)}_{1\le i\le n}$ and $\mathcal{W}={\left({\mathcal{W}}_j\right)}_{1\le j\le p}$ denote to which cluster the rows and the columns belong, respectively. Thus, ${\mathcal{C}}_k=\left\{i=1\text{,}\dots \text{,}\hspace{0.17em}n\hspace{0.17em}\text{:}\hspace{0.17em}{\mathcal{Z}}_i=k\right\}$ is the $k$-th row cluster, with $k=1\text{,}\dots \text{,}\hspace{0.17em}K$, and ${\mathcal{D}}_r=\left\{j=1\text{,}\dots \text{,}\hspace{0.17em}p\hspace{0.17em}\text{:}\hspace{0.17em}{\mathcal{W}}_j=r\right\}$ is the $r$-th column cluster, with $r=1\text{,}\dots \text{,}\hspace{0.17em}R$. The cluster dimensions are $n_k=\left|{\mathcal{C}}_k\right|$ and $p_r=\left|{\mathcal{D}}_r\right|$. The notation used to refer to subsets of $\mathrm{X}$ is the following: ${\text{X}}^{kr}={\left(x_{ij}\right)}_{i\in {\mathcal{C}}_k\text{,}\hspace{0.33em}j\in {\mathcal{D}}_r}$ is the $kr$-th co-cluster (block), ${\text{X}}^{k\text{.}}={\left(x_{ij}\right)}_{i\in {\mathcal{C}}_k\text{,}1\le j\le p}$ is the $n_k\times p$ matrix formed by all the rows in ${\mathcal{C}}_k$, and ${\text{X}}^{\text{.}r}={\left(x_{ij}\right)}_{1\le i\le n\text{,}j\in {\mathcal{D}}_r}$ is the $n\times p_r$ matrix formed by all the columns in ${\mathcal{D}}_r$. When it comes to access the elements of a block, we use the notation ${\text{X}}^{kr}={\left(x_{ij}^{kr}\right)}_{1\le i\le n_k\text{,}1\le j\le p_r}$. So, the $i$-th row vector and the $j$-th column vector of ${\mathrm{X}}^{kr}$ are respectively $x_{i\text{.}}^{kr}={\left(x_{ij}^{kr}\right)}_{1\le j\le p_r}$ and $x_{\text{.}j}^{kr}={\left(x_{ij}^{kr}\right)}_{1\le i\le n_k}$.

The vector $x_{i\text{.}}^{kr}$ contains the expression of the $i$-th gene in the cluster ${\mathcal{C}}_k$ across the $p_r$ spots in the cluster ${\mathcal{D}}_r$. We model $x_{i\text{.}}^{kr}$ as

$${\text{x}}_{i\text{.}}^{kr}={\mu }_{kr}{1}_{p_r}+{\sigma }_{kr\text{,}\hspace{0.17em}i}{ϵ}_{i\text{.}}^{kr}\text{,}\hspace{1em}{ϵ}_{i\text{.}}^{kr}\sim {\mathcal{N}}_{p_r}\left(0\text{,}{\text{Δ}}_{kr}\right)\text{,}$$ (1)

$${\text{Δ}}_{kr}={\tau }_{kr}\mathcal{K}\left(S^r\text{;}\hspace{0.17em}{\varphi }_r\right)+{\xi }_{kr}{\mathbb{I}}_{p_r}\text{,}$$ (2)

where ${\mu }_{kr}$ is a scalar mean parameter, $1_{p_r}$ is a vector of ones, ${\sigma }_{kr\text{,}i}^2$ is a gene-specific variance, and ${\text{Δ}}_{kr}$ is the covariance matrix of the columns. Following Svensson, Teichmann and Stegle (2018) and Sun, Zhu and Zhou (2020), Formula (2) expresses ${\text{Δ}}_{kr}$ as a linear combination of two matrix terms: ${\mathbb{I}}_{p_r}$ is a diagonal matrix of order $p_r$, $\mathcal{K}\left(S^r\text{;}{\varphi }_r\right)={\left(k\left(‖s_j^r-s_{j^{\prime }}^r‖\text{;}{\varphi }_r\right)\right)}_{1\le j\text{,}\hspace{0.17em}{j}^{\prime }\le p_r}$ is the spatial covariance matrix, where $k\left(\cdot \text{;}\hspace{0.17em}{\varphi }_r\right)$ is an isotropic spatial covariance function (Cressie, 2015) parametrized by a vector ${\varphi }_r$, and $S^r={\left(s_j\right)}_{j\in {\mathcal{D}}_r}$ is the sub-matrix of $\mathrm{S}$ containing the spots in ${\mathcal{D}}_r$. The term isotropic denotes that the covariance between two points $j$, $j^{\prime }\in {\mathcal{D}}_r$ depends just on the distance between their two sites, $‖s_j^r-s_{j^{\prime }}^r‖$. The positive parameters ${\tau }_{kr}$ and ${\xi }_{kr}$ in Formula (2) handle the linear combination between $\mathcal{K}$ and ${\mathbb{I}}_{p_r}$: the former measures the spatial dependence of the data, the latter is the so-called nugget effect, a residual variance.

According to Section 2.4 of Cressie (2015), to select an adequate spatial covariance kernel for the data, one can explore the empirical spatial dependency through the variogram and then select a kernel from a vast list of proposals (see for example Rasmussen and Williams, 2006). However, under our model, this strategy would be unfeasible because only the columns within the same cluster are spatially dependent, so the selection of the spatial covariance kernel should be performed simultaneously with the clustering of the data. As a compromise, SpaRTaCo considers the same covariance model $k\left(\cdot \text{;}\hspace{0.17em}{\varphi }_r\right)$ for every column cluster ${\mathcal{D}}_r$; the only difference among the kernels of the clusters is the value of the model parameters ${\varphi }_r$.

The scale parameters ${\sigma }_{kr\text{,}\hspace{0.17em}i}^2$ in (1) aim to capture the variability left unexplained by the spatial covariance model (2), and possibly the extra source of variability due to the dependency across genes. In the longitudinal data framework, De la Cruz-Mesía and Marshall (2006) and Anderlucci and Viroli (2015) consider a random effect model to account for the systematic dependency across subjects in the same group of study. We follow the same approach and we assume that every ${\sigma }_{kr\text{,}\hspace{0.17em}i}^2$ is a realization of an Inverse Gamma distribution $\mathcal{I}\mathcal{G}\left({\alpha }_{kr}\text{,}\hspace{0.17em}{\beta }_{kr}\right)$, where ${\alpha }_{kr}$ and ${\beta }_{kr}$ denote the shape and the rate, respectively. The Inverse Gamma is chosen for its conjugacy with the Gaussian distribution and allows to derive the marginal probability density of $x_{i\text{.}}^{kr}$, that is

$$f\left({\text{x}}_{i\text{.}}^{kr}\text{;}\hspace{0.17em}{\theta }_{kr}\text{,}{\varphi }_r\right)=\frac{1}{\sqrt{{\left(2\pi \right)}^{p_r}\text{det}\left({\text{Δ}}_{kr}\right)}}\frac{\text{Γ}\left({\alpha }_{kr\text{,}i}^{\ast }\right)}{\text{Γ}\left({\alpha }_{kr}\right)}\frac{{\beta }_{kr}^{{\alpha }_{kr}}}{{\beta }_{kr\text{,}i}^{\ast }{}^{{\alpha }_{kr\text{,}i}^{\ast }}}\text{,}$$ (3)

where $\text{det}\left(\cdot \right)$ denotes the matrix determinant, ${\alpha }_{kr\text{,}i}^{\ast }=p_r/2+{\alpha }_{kr}$ and ${\beta }_{kr\text{,}i}^{\ast }={\left(x_{i\text{.}}^{kr}-{\mu }_{kr}{1}_{p_r}\right)}^T{\text{Δ}}_{kr}^{-1}\left(x_{i\text{.}}^{kr}-{\mu }_{kr}{1}_{p_r}\right)/2+{\beta }_{kr}$. Note that this formulation corresponds to the probabilistic model $x_{i\text{.}}^{kr}\sim t_{2{\alpha }_{kr}}\left({\mu }_{kr}{1}_{p_r}\text{,}\hspace{0.17em}{\alpha }_{kr}^{-1}{\beta }_{kr}{\text{Δ}}_{kr}\right)$ and is similar to that employed to shrink the gene variances in the popular limma model (Smyth, 2004). The set of parameters ${\theta }_{kr}=\left\{{\mu }_{kr}\text{,}{\tau }_{kr}\text{,}{\xi }_{kr}\text{,}{\alpha }_{kr}\text{,}{\beta }_{kr}\right\}$ is specific of the data into the $\left(k\text{,}\hspace{0.17em}r\right)$-th co-cluster, while ${\varphi }_r$ is a parameter that is descriptive of the entire $r$-th column cluster.

The model in Formula (1) can be rephrased with a probability distribution over the entire $kr$-th block, ${\text{X}}^{kr}|{\text{Σ}}_{kr}\sim \mathcal{M}\mathcal{V}\mathcal{N}\left({\mu }_{kr}{1}_{n_k\times p_r}\text{,}\hspace{0.17em}{\text{Σ}}_{kr}\text{,}\hspace{0.17em}{\text{Δ}}_{kr}\right)$, where $\mathcal{M}\mathcal{V}\mathcal{N}$ denotes the matrix-variate normal distribution and ${\text{Σ}}_{kr}=\text{diag}\left({\sigma }_{kr\text{,}1}^2\text{,}\dots \text{,}{\sigma }_{kr\text{,}{n}_k}^2\right)$ is the (diagonal) covariance matrix of the genes. A consequence of the matrix-variate normal model is that every row, column and sub-matrix of ${\mathrm{X}}^{kr}$ is Gaussian (Gupta and Nagar, 2018). For instance, the following model formulation is equivalent to Formula (1):

$$x_{\text{.}j}^{kr}\text{|}{\text{Σ}}_{kr}\sim {\mathcal{N}}_{n_k}\left\{{\mu }_{kr}{1}_{n_k}\text{,}\left({\tau }_{kr}+{\xi }_{kr}\right){\text{Σ}}_{kr}\right\}\text{,}\hspace{1em}\text{Cov}\left(x_{\text{.}j}^{kr}\text{,}\hspace{0.17em}{x}_{\text{.}{j}^{\prime }}^{kr}\right)={\tau }_{kr}k\left(‖s_j^r-s_{j^{\prime }}^r‖\text{;}\hspace{0.17em}{\varphi }_r\right){\text{Σ}}_{kr}\text{,}$$

with $j$, $j^{\prime }\in {\mathcal{D}}_r$.

Last, the clustering labels $\mathcal{Z}$ and $\mathcal{W}$ are unknown independent random variables. Figure 2 represents the relations across the elements of the model with a DAG.

FIG 2.

DAG of the SpaRTaCo co-clustering model. Grey circle denotes the data, white circles are the latent random variables, and white rectangles are the model parameters.

2.2. A comparison with other co-clustering models.

We review in this section some advanced co-clustering techniques that have some similarities with our proposal. The goal is to highlight, starting from the existing literature, how SpaRTaCo has been designed specifically for detecting and clustering data based on their spatial covariance in some groups of observational sites. With respect to the distinction between deterministic and model-based co-clustering techniques we already discussed in Section 1.2, we choose to compare SpaRTaCo only with model-based techniques because they offer a clear advantage in the interpretation of the results. Some of the methods that we review here are named as biclustering models, but in practice they segment the data matrix into rectangular blocks.

Sparse Biclustering (sparseBC, Tan and Witten, 2014) extends the k-means algorithm to the co-clustering framework. The model corresponds to a probabilistic assumption on the block of the type ${\text{X}}^{kr}\sim \mathcal{M}\mathcal{V}\mathcal{N}\left({\mu }_{kr}{1}_{n_k\times p_r}\text{,}\hspace{0.17em}{\mathbb{I}}_{n_k}\text{,}\hspace{0.17em}\xi {\mathbb{I}}_{p_r}\right)$, where $\xi$ is an unknown scale parameter. In sparseBC, the estimation of ${\mu }_{kr}$, for any $k$ and $r$, is regulated by a LASSO penalization. We thus distinguish the sparse estimation from the case of null penalization (BC).

Matrix-Variate Normal Biclustering (MVNb, Tan and Witten, 2014) extends sparseBC by taking a probabilistic model on the blocks of the type ${\text{X}}^{kr}\sim \mathcal{M}\mathcal{V}\mathcal{N}\left({\mu }_{kr}{1}_{n_k\times p_r}\text{,}\hspace{0.17em}{\text{Σ}}_k^{\text{MVNB}}\text{,}\hspace{0.17em}{\text{Δ}}_r^{\text{MVNB}}\right)$, where both ${\text{Σ}}_k^{\text{MVNB}}$ and ${\text{Δ}}_r^{\text{MVNB}}$ are non-diagonal covariance matrices with respectively $n_k\left(n_k+1\right)/2$ and $p_r\left(p_r+1\right)/2$ free parameters. Together with the LASSO penalization on the centroids, handled by a parameter $\lambda$, the authors deploy a graphical LASSO penalization (Witten and Tibshirani, 2009) to practically solve the singularity problems in the estimate of ${\text{Σ}}_k^{\text{MVNB}}$ and ${\text{Δ}}_r^{\text{MVNB}}$. The penalization parameters involved are denoted by ${\rho }_{\text{Σ}}$ and ${\rho }_{\text{Δ}}$. With respect to the MVNb, SpaRTaCo has specific row and column covariance matrices ${\text{Σ}}_{kr}$ and ${\text{Δ}}_{kr}$ for each block, whose structure is described in Section 2.1. The total number of free parameter, $KR\left|{\theta }_{kr}\right|+R\left|{\varphi }_r\right|$, does not grow either with $n$ or $p$. As a direct consequence, the parameter estimation of SpaRTaCo, conditioning on the clustering labels $\mathcal{Z}$ and $\mathcal{W}$, remains much less computationally prohibitive than the one of the MVNb, specially when the sample size becomes considerably large.

Latent Block Model is a vast class of statistical models that can be seen as an extension of the mixture model for co-clustering problems. The model for continuous data (Govaert and Nadif, 2013, Chapter 5) can be written using the Matrix Variate Normal representation as ${\text{X}}^{kr}\sim \mathcal{M}\mathcal{V}\mathcal{N}\left({\mu }_{kr}{1}_{n_k\times p_r}\text{,}\hspace{0.17em}{\mathbb{I}}_{n_k}\text{,}\hspace{0.17em}{\xi }_{kr}{\mathbb{I}}_{p_r}\right)$ and so it is based on the assumption that the data entries in a block are independent given the clustering labels (conditional independence). The intra-block model is thus a special case of SpaRTaCo when ${\text{Σ}}_{kr}=\mathbb{I}$ and ${\tau }_{kr}=0$, for all $k$ and $r$. However, the LBM is more general on the probabilitistic assumptions over the clustering variables. In fact, it assumes $\text{Pr}\left({\mathcal{Z}}_i=k\right)={\pi }_k$ and $\text{Pr}\left({\mathcal{W}}_j=r\right)={\rho }_r$, where $\left({\pi }_1\text{,}\hspace{0.17em}\dots \text{,}\hspace{0.17em}{\pi }_K\right)$ and $\left({\rho }_1\text{,}\hspace{0.17em}\dots \text{,}\hspace{0.17em}{\rho }_R\right)$ are probability vectors such that ${\displaystyle {\sum }_{k=1}^K{\pi }_k}={\displaystyle {\sum }_{r=1}^R{\rho }_r=1}$, while SpaRTaCo implicitly assumes that $\text{Pr}\left({\mathcal{Z}}_i=k\right)=1/K$ and $\text{Pr}\left({\mathcal{W}}_j=r\right)=1/R$ for any $k$ and $r$.

Supplementary Figure 1 (Sottosanti and Risso, 2022) gives a summary of the relations across SpaRTaCo and the co-clustering models discussed in this section.

3. Inference.

3.1. Identifiability.

The model as expressed in Formula (1) is not identifiable in the covariance term: in fact, for any $a>0$, ${\sigma }_{kr\text{,}i}^2\cdot {\text{Δ}}_{kr}=a{\sigma }_{kr\text{,}i}^2\cdot {\text{Δ}}_{kr}/a={\tilde{\sigma }}_{kr\text{,}i}^2\cdot {\widetilde{\text{Δ}}}_{kr}$. This issue generates in practice an infinite number of solutions for the parameter estimate.

A typical workaround to get unique parameter estimates consists in setting the value of some covariance parameters. In our model, this would mean taking ${\sigma }_{kr\text{,}i}^2=c$, for one $i$ in $\left\{1\text{,}\hspace{0.17em}\dots \text{,}\hspace{0.17em}{n}_k\right\}$, using an arbitrary positive constant $c$. Incidentally, this is equivalent to constraint $\text{tr}\left({\text{Σ}}_{kr}\right)$, the trace of the matrix ${\text{Σ}}_{kr}$ (Allen and Tibshirani, 2010; Caponera et al., 2017). However, we discard this solution as, under our model, the rows of the data matrix are involved into a clustering procedure. Thus, it is not possible to define which $i$ in a cluster should take the constraint.

The solution we adopt for our model puts the identification constraint on ${\text{Δ}}_{kr}$ (Anderlucci and Viroli, 2015). Since $\text{tr}\left({\text{Δ}}_{kr}\right)=p_r\left({\tau }_{kr}+{\xi }_{kr}\right)$, we constraint the quantity ${\tau }_{kr}+{\xi }_{kr}=c_{\text{Δ}}$, where $c_{\text{Δ}}$ is an arbitrary positive constant. Such constraint has a notable practical consequence: in fact, once the estimate ${\widehat{\tau }}_{kr}$ is determined within the constrained domain ($0$, $c_{\text{Δ}}$), then ${\widehat{\xi }}_{kr}$ is simply taken by difference as ${\widehat{\xi }}_{kr}=c_{\text{Δ}}-{\widehat{\tau }}_{kr}$. Hence, we can only interpret ${\widehat{\tau }}_{kr}$ and ${\widehat{\xi }}_{kr}$ in relation to each other and not in absolute terms. According to Svensson, Teichmann and Stegle (2018), in our applications (Sections 4 and 5) we will consider the quantity ${\tau }_{kr}/{\xi }_{kr}$ that we called spatial signal-to-noise ratio. This ratio is easily interpretable because it represents the amount of spatial expression of the genes in a cluster with respect to the nugget effect.

3.2. Model estimation.

To estimate SpaRTaCo, we propose an approach based on the maximization of the classification log-likelihood, that is

$$\text{log}\mathcal{L}\left(\text{Θ},\hspace{0.17em}\mathcal{Z},\hspace{0.17em}\mathcal{W}\right)=\sum _{i=1}^n\sum _{k=1}^{K}1\left({\mathcal{Z}}_i=k\right)\left\{\sum _{r=1}^R\text{log}\hspace{0.17em}f\left(x_{i\text{.}}^{\text{.}r}\text{;}\hspace{0.17em}{\theta }_{kr},\hspace{0.17em}{\varphi }_r\right)\right\},$$ (4)

where $\text{Θ}={\cup }_r\left\{{\cup }_k{\theta }_{kr},{\varphi }_r\right\}$, $x_{i\text{.}}^{\text{.}r}$ is the $i$-th row of the matrix ${\mathrm{X}}^{.r}$ and $f\left(\cdot \text{;}\cdot \right)$ is given in Formula (3). Notice that the correlation across the columns does not allow to write the $\mathcal{W}$ explicitly. This issue does not concern the $\mathcal{Z}$, because the rows are independent.

Chapter 2 of Bouveyron et al. (2019) makes a clear distinction between the classification and the complete log-likelihood (the latter includes an additional part related to the distribution of the clustering labels). However, since SpaRTaCo implicitly assumes that $\text{Pr}\left({\mathcal{Z}}_i=k\right)=1/K$ and $\text{Pr}\left({\mathcal{W}}_j=r\right)=1/R$ for any $k$ and $r$, then there is no practical difference between classification and complete log-likelihood.

The classification log-likelihood can be maximized with a classification EM algorithm (CEM, Celeux and Govaert, 1992), a modification of the standard EM which allocates the observations into the clusters during the estimation procedure. The CEM is an iterative algorithm which alternates between a classification step (CE Step), where the estimates of $\mathcal{Z}$ and $\mathcal{W}$ are updated, and a maximization step (M Step), which updates the parameter estimates of $\text{Θ}$. The benefits brought by such algorithm are particularly visible when complex models as the LBM are employed, because the joint conditional distribution $p\left(\mathcal{Z},\mathcal{W}|\text{X};\text{Θ}\right)$ is not directly available (Govaert and Nadif, 2013).

Under SpaRTaCo, a direct update of $\mathcal{W}$ through a CE step is unfeasible due to the correlation across the columns, and so the estimation algorithm requires some modifications. This issue was already discussed by Tan and Witten (2014) for their MVNb model; however, their solution consists in an heuristic estimation algorithm with no guarantees of convergence. We propose to perform a stochastic allocation (SE step), where the column clustering configuration $\mathcal{W}$ is sampled from a Markov chain whose limit distribution is the conditional distribution $p\left(\mathcal{W}\text{|}\mathcal{Z},\text{X};\text{Θ}\right)$. This step can be performed using the Metropolis-Hastings algorithm. A stochastic version of the EM algorithm was previously employed also for estimating the LBM by Keribin et al. (2015), Bouveyron et al. (2018) and Casa et al. (2021). Because of the alternation of a classification move, a stochastic allocation move and a maximization move, we name our algorithm classification-stochastic EM (CS-EM). We denote with ${\left(\text{Θ},\mathcal{Z},\mathcal{W}\right)}^{\left(t-1\right)}$ the estimate of the model parameters and of the clustering labels at iteration $t-1$. At step $t$, the algorithm executes the following steps:

• CE Step: keeping fixed ${\left(\mathcal{W},\hspace{0.17em}\text{Θ}\right)}^{(t-1)}$, update the row clustering labels with the following rule:

  $${\mathcal{Z}}_i^{(t)}=\underset{k=1\text{,…,}K}{\text{argmax}}\frac{\prod _{r=1}^{R}f\left(x_{i\text{.}}^r;{\theta }_{kr}^{\left(t-1\right)},{\varphi }_r^{\left(t-1\right)}\right)}{\sum _{k^{\prime }=1}^K\left\{\prod _{r=1}^{R}f\left(x_{i\text{.}}^r;{\theta }_{k^{\prime }r}^{\left(t-1\right)},{\varphi }_r^{\left(t-1\right)}\right)\right\}},\hspace{1em}i=1,\dots ,\hspace{0.17em}n\text{.}$$

• SE Step: keeping fixed ${\mathcal{Z}}^{\left(t\right)}$ and ${\text{Θ}}^{(t-1)}$, generate a candidate clustering configuration ${\mathcal{W}}^{\ast }$ by randomly changing some elements from the starting configuration ${\mathcal{W}}^{\left(t-1\right)}$. Let $m$ be the number of elements of ${\mathcal{W}}^{\left(t-1\right)}$ that we attempt to change: $m$ can be either fixed or randomly drawn from a discrete distribution. To formulate ${\mathcal{W}}^{\ast }$, we exploit two moves.

  (M1) Two clustering labels $g_1\sim \mathcal{U}\left(\left\{1\text{,…,}\hspace{0.17em}R\right\}\right)$ and $g_2\sim \mathcal{U}\left(\left\{1\text{,…,}\hspace{0.17em}R\right\}\setminus \left\{g_1\right\}\right)$ are drawn. The candidate configuration ${\mathcal{W}}^{\ast }$ is made by selecting $m$ observations from ${\mathcal{W}}^{\left(t-1\right)}$ at random with label $g_1$ and changing their label to $g_2$. The quantity

  $$\frac{q\left({\mathcal{W}}^{(t-1)}\text{|}{\mathcal{W}}^{\ast }\right)}{q\left({\mathcal{W}}^{\ast }\text{|}{\mathcal{W}}^{(t-1)}\right)}=\frac{p_{g_1}\text{!}{p}_{g_2}\text{!}}{\left(p_{g_1}-m\right)\text{!}\left(p_{g_2}+m\right)\text{!}}$$

  is the ratio of transition probabilities employed by the Metropolis-Hastings algorithm to evaluate ${\mathcal{W}}^{\ast }$, where $q\left({\mathcal{W}}^{\ast }\text{|}{\mathcal{W}}^{(t-1)}\right)$ and $q\left({\mathcal{W}}^{(t-1)}\text{|}{\mathcal{W}}^{\ast }\right)$ are respectively the probabilities of passing from configuration ${\mathcal{W}}^{\left(t-1\right)}$ to ${\mathcal{W}}^{\ast }$ and vice-versa. This move almost coincides with the (M2) move of Nobile and Fearnside (2007).
  (M2) For $h=1\text{,…,}\hspace{0.17em}m$, the clustering $g_{1h}\sim \mathcal{U}\left(\left\{1\text{,…,}\hspace{0.17em}R\right\}\right)$ and $g_{2h}\sim \mathcal{U}\left(\left\{1\text{,…,}\hspace{0.17em}R\right\}\setminus \left\{g_{1h}\right\}\right)$ are drawn. Let $b_{lr}={\displaystyle \sum _{h=1}^m}1\left(g_{lh}=r\right)$, for $l=1,\hspace{0.17em}2$ and $r=1,\hspace{0.17em}\dots ,\hspace{0.17em}R$. Then the candidate configuration ${\mathcal{W}}^{\ast }$ is made by changing the labels of $b_{1r}$ observations selected at random from the group $r$, when $b_{1r}>0$, to $g_{2_{\kappa \left(r\right)}}$, where $\kappa \left(r\right)=\left\{h=1\text{,…,}\hspace{0.17em}m\hspace{0.17em}\text{:}\hspace{0.17em}{g}_{1h}=r\right\}$. The ratio of transition probabilities is

  $$\frac{q\left({\mathcal{W}}^{\left(t-1\right)}\text{|}{\mathcal{W}}^{\ast }\right)}{q\left({\mathcal{W}}^{\ast }\text{|}{\mathcal{W}}^{\left(t-1\right)}\right)}=\prod _{r\text{:}{b}_{2r}>0}\frac{b_{2r}\text{!}\left(p_r-b_{1r}\right)\text{!}}{\left(p_r-b_{1r}+b_{2r}\right)\text{!}}/\prod _{r\text{:}{b}_{1r}>0}\frac{b_{1r}\text{!}\left(p_r-b_{1r}\right)\text{!}}{p_r\text{!}}\text{.}$$
  The choice between (M1) and (M2) is random. The candidate configuration ${\mathcal{W}}^{\ast }$ is accepted with probability $\text{min}\left\{1,\hspace{0.17em}A\right\}$, where $A$ is the following Metropolis-Hastings ratio:

  $$A=\frac{\mathcal{L}\left({\text{Θ}}^{\left(t-1\right)},{\mathcal{Z}}^{\left(t\right)},{\mathcal{W}}^{\ast }\right)}{\mathcal{L}\left({\text{Θ}}^{\left(t-1\right)},{\mathcal{Z}}^{\left(t\right)},{\mathcal{W}}^{\left(t-1\right)}\right)}\frac{q\left({\mathcal{W}}^{(t-1)}\text{|}{\mathcal{W}}^{\ast }\right)}{q\left({\mathcal{W}}^{\ast }\text{|}{\mathcal{W}}^{\left(t-1\right)}\right)}\text{.}$$

  Within the same iteration $t$, the SE Step can be run for an arbitrary large number of times to accelerate the exploration of the space of clustering configurations and so the convergence of the estimation algorithm to a stationary point. From our experience, we suggest to repeat the SE Step for at least 100 times per iteration.

• M Step: using the rows in ${\mathcal{C}}_k^{\left(t\right)}$ and the columns in ${\mathcal{D}}_r^{\left(t\right)}$, update the parameter estimates ${\theta }_{kr}^{\left(t\right)}$ and ${\varphi }_r^{\left(t\right)}$. The derivative of the log-likelihood with respect to (${\theta }_{kr}$, ${\varphi }_r$) does not lead to closed solutions for updating the model parameters, and for this reason a numerical optimizer must be applied. We exploit the L-BFGS-B algorithm of Byrd et al. (1995) implemented in the stats library of the R computing language, which allows constrained optimization; this aspect is particularly useful to estimate ${\tau }_{kr}$ under the identifiability constraint described in Section 3.1.

Following Tan and Witten (2014), our implementation of the estimation algorithm alternates each allocation step, either the CE Step and the SE Step, with an M Step. As pointed by Keribin et al. (2015), the SE Step is not guaranteed to increase the classification log-likelihood at each iteration, but it generates an irreducible Markov chain with a unique stationary distribution which is expected to be concentrated around the maximum likelihood parameter estimate. The estimation algorithm must be run for a sufficiently large number of iterations. We additionally implemented a convergence criterion that stops the algorithm if the increment of the classification log-likelihood is smaller than a certain threshold for a given number of iterations in a row. The final estimates of ($\widehat{\text{Θ}}$, $\widehat{\mathcal{Z}}$, $\widehat{\mathcal{W}}$) are the values obtained at the iteration from which (4) is maximum.

Notice that the criterion to form the co-clusters that SpaRTaCo uses has also a geometrical interpretation; in fact, in the same way that k-means minimizes the Euclidean distance between the observations and the centroids, SpaRTaCo minimizes the Mahalanobis distance of the observations from the block centroids, embedding the spatial structure of the data into the covariance matrix. Therefore, even when the data do not fully respect the probabilistic assumptions, the model is still valid, as a distance-based clustering algorithm.

3.3. Measuring the clustering uncertainty.

The proposed estimation procedure should be run multiple times from different starting points to check if the algorithm encounters some local maxima. In addition, the parallel runs can be used to quantify the uncertainty of the estimated co-clustering structure. In fact, if the analyzed data carry large evidence in favor of a unique clustering configuration, then the parallel runs will return approximately the same row and column clusters. If instead the clustering structure of the data that SpaRTaCo searches for is not evident, then the multiple runs of the algorithm will tend to discover different but equally likely solutions.

Let us suppose to run the CS-EM algorithm $S$ times on the same dataset: (${\widehat{\text{Θ}}}^{\left(s\right)}$, ${\widehat{\mathcal{Z}}}^{\left(s\right)}$, ${\widehat{\mathcal{W}}}^{\left(s\right)}$) is the solution to the parameter estimate returned by the $s$-th run, for $s=1,\hspace{0.17em}\dots ,\hspace{0.17em}S$, and ${\ell }^{\left(s\right)}=\text{log}\hspace{0.17em}\mathcal{L}\left({\widehat{\text{Θ}}}^{\left(s\right)},\hspace{0.17em}{\widehat{\mathcal{Z}}}^{\left(s\right)},\hspace{0.17em}{\widehat{\mathcal{W}}}^{\left(s\right)}\right)$. In addition, Let $s^{\ast }={\text{argmax}}_s{\ell }^{\left(s\right)}$: since the co-clustering structure (${\widehat{\mathcal{Z}}}^{\left(s^{\ast }\right)}$, ${\widehat{\mathcal{W}}}^{\left(s^{\ast }\right)}$) has found the largest evidence across the $S$ runs on the current data, it is the final estimate returned by the algorithm. The co-clustering uncertainty can be thought of as a function of the distances between the final estimate, (${\widehat{\mathcal{Z}}}^{\left(s^{\ast }\right)}$, ${\widehat{\mathcal{W}}}^{\left(s^{\ast }\right)}$), and the other estimates of lower evidence, (${\widehat{\mathcal{Z}}}^{\left(s\right)}$, ${\widehat{\mathcal{W}}}^{\left(s\right)}$), for $s\ne s^{\ast }$. Let ${\mathcal{I}}_k={\left\{1\left({\widehat{\mathcal{Z}}}_i^{\left(s^{\ast }\right)}=k\right)\right\}}_{1\le i\le n}$ be the binary vector denoting which rows belong to the $k$-th row cluster given by the run $s^{\ast }$, for $k=1,\hspace{0.17em}\dots ,\hspace{0.17em}K$, and ${\mathcal{I}}_{h_s\left(k\right)}=\left[1{\text{{}{\widehat{\mathcal{Z}}}_i^{\left(s\right)}=h_s\left(k\right)\text{]}}_{1\le i\le n}\right.$ be the binary vector denoting which observations belong to the cluster $h_s\left(k\right)$ given by the $s$-th run, where $h_s\left(k\right)={\text{argmax}}_{h=1\text{,…,}\hspace{0.17em}K}{\displaystyle \sum _{i=1}^n}1\left({\mathcal{Z}}_i^{\left(s^{\ast }\right)}=k,\hspace{0.17em}{\mathcal{Z}}_i^{\left(s\right)}=h\right)$ and $s\ne s^{\ast }$. In addition, let us consider the weights ${\omega }_s=1/\left({\ell }^{\left(s^{\ast }\right)}-{\ell }^{(s)}\right)$. The uncertainty of the row cluster $k$ is measured as

$${\epsilon }_k^{\text{rows}}=\frac{{\displaystyle {\sum }_{s\ne s^{\ast }}{\omega }_s\text{CER}\left({\mathcal{I}}_k,{\mathcal{I}}_{h_s\left(k\right)}\right)}}{{\displaystyle {\sum }_{s\ne s^{\ast }}{\omega }_s}},$$ (5)

where $\text{CER}\left(\cdot ,\cdot \right)$ denotes the clustering error rate (Witten and Tibshirani, 2010), an index that measures the disagreement between a reference and an estimated clustering configuration: the closer is $\text{CER}$ to 0, the larger is the agreement between the true and the estimated clusters. The ${\left\{{\omega }_s\right\}}_{s\ne s^{\ast }}$ give a large weight to the $\text{CER}$ between ${\mathcal{I}}_k$ and ${\mathcal{I}}_{h_s\left(k\right)}$ when ${\ell }^{\left(s^{\ast }\right)}-{\ell }^{\left(s\right)}$ is small, and vice-versa. The reason is intuitively that, if both ${\omega }_s$ and $\text{CER}\left({\mathcal{I}}_k,\hspace{0.17em}{\mathcal{I}}_{h_s\left(k\right)}\right)$ are large, then there are two considerably different clustering configurations that yield approximately the same log-likelihood value. Thus, the clustering structure of the data is uncertain. If instead ${\omega }_s$ is small, the difference between ${\widehat{\mathcal{Z}}}^{\left(s^{\ast }\right)}$ and ${\widehat{\mathcal{Z}}}^{\left(s\right)}$ is in practice irrelevant, because the evidence arising from the data clearly leans in favor of ${\widehat{\mathcal{Z}}}^{\left(s^{\ast }\right)}$.

Formula (5) can be applied also for computing the uncertainties of the column clusters $\left({\epsilon }_1^{\text{cols }}\text{,…,}\hspace{0.17em}{\epsilon }_R^{\text{cols }}\right)$, just replacing ${\widehat{\mathcal{Z}}}^{\left(s\right)}$ with ${\widehat{\mathcal{W}}}^{\left(s\right)}$. The uncertainty measure introduced here can be interpreted similarly to the $\text{CER}$ index: the closer are ${\epsilon }_k^{\text{rows}}$ and ${\epsilon }_r^{\text{cols}}$ to 0, the larger is the evidence of a unique co-clustering structure of the data.

3.4. Model selection.

SpaRTaCo can be run with different spatial covariance models $k\left(\cdot ;\cdot \right)$ and with different combinations of $K$ and $R$. We consider the problem of selecting the best model for the data, both in terms of the number of clusters and the spatial covariance function, using an information criterion. The most common criteria, the AIC and the BIC, cannot be derived under Model (1) because the likelihood of the data $p\left(\text{X};\hspace{0.17em}\text{Θ}\right)$, marginalized with respect to the latent variables $\mathcal{Z}$ and $\mathcal{W}$, is not available in closed form.

In this work, we propose to guide the model selection using the integrated completed log-likelihood (ICL, Biernacki, Celeux and Govaert, 2000). The ICL is a well-established criterion for selecting the number of clusters (Bouveyron et al., 2019) which has become popular in the co-clustering framework for selecting the size of LBM (Keribin et al., 2015; Bouveyron et al., 2018; Casa et al., 2021). Under Model (1)-(2), its expression is

$$\text{ICL}=\text{log}\hspace{0.17em}\mathcal{L}\left(\widehat{\text{Θ}},\hspace{0.17em}\widehat{\mathcal{Z}},\hspace{0.17em}\widehat{\mathcal{W}}\right)-n \text{log}\hspace{0.17em}K-p \text{log}\hspace{0.17em}R-\frac{4KR+\text{dim}\left(\varphi \right)R}{2}\text{log}\hspace{0.17em}np,$$ (6)

where $\text{dim}\left(\varphi \right)$ is the dimension of the parameter vector ${\varphi }_r$ , which does not depend on $r$. The derivation of (6) is described more in details in Supplementary Section 1. Operatively, the best model from a list of candidates corresponds to the one with the largest value of (6).

In the presence of mixed effects, Delattre et al. (2014) argue that the actual sample size is not trivial to define, and thus the classical information criteria need to be modified. In particular, they derive an alternative formulation of the BIC which includes a term that depends only on the parameters involved with the random effects. However, their model specification assumes that the marginal distribution of the data with the random parameters integrated out cannot be derived in closed form. Although the presence of the random variances ${\sigma }_{kr, i}^2$ makes SpaRTaCo a random effect model, the integration of ${\sigma }_{kr, i}^2$ from the density function of $x_{i\text{.}}^{kr}\text{|}{\sigma }_{kr, i}^2$ leads to the marginal density (3). For this reason, we do not implement any modification based on the random effects into our information criterion (6).

4. Simulation studies.

4.1. Simulation model.

We study the performance of SpaRTaCo with five simulated spatial experiments that recreate some possible scenarios that can be found in real data. We generate the latent blocks using the matrix-variate normal distribution (Gupta and Nagar, 2018) as follows: given the number of row and column clusters $K^{\text{true}}$ and $R^{\text{true}}$ (for convenience, we considered here $K^{\text{true}}=R^{\text{true}}=3$ in every simulation experiment), the clustering labels ${\mathcal{Z}}^{\text{true}}$ and ${\mathcal{W}}^{\text{true}}$, and the clusters ${\mathcal{C}}_k^{\text{true}}=\left\{i=1\text{,…,}\hspace{0.17em}n\hspace{0.17em}\text{:}\hspace{0.17em}{\mathcal{Z}}_i^{\text{true}}=k\right\}$ and ${\mathcal{D}}_r^{\text{true}}=\left\{j=1\text{,…,}\hspace{0.17em}p\hspace{0.17em}\text{:}\hspace{0.17em}{\mathcal{W}}_j^{\text{true}}=r\right\}$, the $\left(k,\hspace{0.17em}r\right)$-th block is drawn from

$${\text{X}}^{kr}\sim \mathcal{M}\mathcal{V}\mathcal{N}\left({\mu }_{kr}^{\text{true}}{1}_{n_k\times p_r},\hspace{0.17em}{\text{Σ}}_{kr}^{\text{true}},\hspace{0.17em}{\text{Δ}}_{kr}^{\text{true}}\right),\hspace{1em}{\text{Δ}}_{kr}^{\text{true}}={\tau }_{kr}^{\text{true}}{\mathcal{K}}_r^{\text{true}}\left(S^r;\hspace{0.17em}{\varphi }_r^{\text{true}}\right)+{\xi }_{kr}^{\text{true}}{\mathbb{I}}_{p_r},$$ (7)

where ${\mathcal{K}}_r^{\text{true}}\left(S^r;\hspace{0.17em}{\varphi }_r\right)={\left(k_r^{\text{true}}\left(‖s_j^r-s_{j^{\prime }}^r‖;\hspace{0.17em}{\varphi }_r^{\text{true}}\right)\right)}_{1\le j,j^{\prime }\le p_r}$, and $k_r^{\text{true}}\left(\cdot ;\hspace{0.17em}{\varphi }_r^{\text{true}}\right)$ is an isotropic spatial covariance kernel parametrized by ${\varphi }_r^{\text{true}}$. Note that, differently from (2), the presence of the subscript $r$ into the kernel matrix ${\mathcal{K}}_r^{\text{true}}$ denotes that the spatial covariance function can be different for any column cluster. In our simulations, we employed the Exponential kernel with scale ${\theta }_E$ for the columns in ${\mathcal{D}}_1^{\text{true}}$, the Rational Quadratic kernel with parameters (${\theta }_R$, ${\alpha }_R$) for the columns in ${\mathcal{D}}_2^{\text{true}}$, and the Gaussian kernel (known also as Squared Exponential) with scale ${\theta }_G$ for the columns in ${\mathcal{D}}_3^{\text{true}}$. Their formulation is reported in Supplementary Section 2 and it is further discussed in Chapter 4 or Rasmussen and Williams (2006). The simulation model (7) implies the following marginal distributions of the genes and of the spots:

$${\text{x}}_{i\text{.}}^{k\text{.}}\text{|}{\mathcal{Z}}^{\text{true}},\hspace{0.17em}{\mathcal{W}}^{\text{true}}\sim {\mathcal{N}}_p\left\{\left({\mu }_{k1}^{\text{true}}{1}_{p_1}\text{,…,}\hspace{0.17em}{\mu }_{k3}^{\text{true}}{1}_{p_3}\right),{\text{Σ}}_{ii}^{\text{true}}\text{diag}{\left({\text{Δ}}_{kr}^{\text{true}}\right)}_{r=1,2,3}\right\},$$ (8)

$${\text{x}}_{\text{.}j}^{\text{.}r}\text{|}{\mathcal{Z}}^{\text{true}},\hspace{0.17em}{\mathcal{W}}^{\text{true}}\sim {\mathcal{N}}_n\left\{\left({\mu }_{1r}^{\text{true}}{1}_{n_1}\text{,…,}\hspace{0.17em}{\mu }_{3r}^{\text{true}}{1}_{n_3}\right),\hspace{0.17em}{c}^{\text{true}}\text{diag}{\left({\text{Σ}}_k^{\text{true}}\right)}_{k=1,\hspace{0.17em}2,\hspace{0.17em}3}\right\},$$ (9)

where ${\text{Σ}}_{ii}^{\text{true}}$ is the variance parameter of the $i$-th row and does not depend on $k$, and the notation $\text{diag}{\left({\text{Δ}}_{kr}^{\text{true}}\right)}_{r=1,\hspace{0.17em}2,\hspace{0.17em}3}$ denotes a block diagonal matrix formed by the matrices ${\text{Δ}}_1^{\text{true}}\text{,…,}\hspace{0.17em}{\text{Δ}}_3^{\text{true}}$. Notice that, from Formula (9), the marginal distribution of the spots does not carry any information on the column clusters. The cross-covariance matrix of two rows $i$, $i^{\prime }\in {\mathcal{C}}_k^{\text{true}}$ is $\text{Cov}\left({\mathrm{x}}_{i\text{.}}^{k\text{.}},{\mathrm{x}}_{i^{\prime }}^k\right)={\text{Σ}}_{k,i{i}^{\prime }}^{\text{true}}\text{diag}{\left({\text{Δ}}_{kr}^{\text{true}}\right)}_{r=1,\hspace{0.17em}2,\hspace{0.17em}3}$, and the cross-covariance of two columns $j$, $j^{\prime }\in {\mathcal{D}}_r^{\text{true}}$ is $\text{Cov}\left(x_{\text{.}j}^{\text{.}r},\hspace{0.17em}{x}_{\text{.}{j}^{\prime }}^{\cdot r}\right)=\text{diag}{\left\{{\tau }_{kr}^{\text{true}}{k}_r^{\text{true}}\left(s_j^r-s_{j^{\prime }}^r;\hspace{0.17em}{\varphi }_r^{\text{true}}\right){\text{Σ}}_k^{\text{true}}\right\}}_{k=1,\hspace{0.17em}2,\hspace{0.17em}3}$.

We took the sets of spatial coordinates (${\mathrm{S}}_1$, ${\mathrm{S}}_2$, ${\mathrm{S}}_3$) from the brain tissue sample of the subject with ID 151507 contained in the R package spatialLIBD and processed with Visium. As we briefly discussed in Section 1.1, the spots in these experiments have been manually annotated into layers. We extracted 200 spots from each of the three layers appearing in the top-right region of the image. The resulting map of 600 spots is shown in the left plot of Figure 3; the clustering labels ${\mathcal{W}}^{\text{true}}$ correspond to the labels assigned with the manual annotation. Note that, although we took the spot annotation from the real data, the image clusters in the simulation experiments have a substantially different meaning: in fact, under the simulation model (7), they denote regions of the tissue in which some genes are expressed with specific spatial variation profiles, while, in the real data, the manually annotated regions identify the morphological structure of the tissue. In addition, the right plot of Figure 3 shows the covariance functions used for the simulations. We set the covariance parameters (${\theta }_E$, ${\theta }_R$, ${\alpha }_R$, ${\theta }_G$) according to how much the clusters extend over the plane: the covariance function of ${\mathcal{D}}_1^{\text{true}}$ is steeper than the one of ${\mathcal{D}}_2^{\text{true}}$ because ${\mathcal{D}}_1^{\text{true}}$ covers a smaller distance. Because ${\mathcal{D}}_3^{\text{true}}$ is made of two distinct groups of spots appearing in the top and in the bottom of Figure 3 (left), we specify $k_3^{\text{true}}\left(\cdot \hspace{0.17em};\hspace{0.17em}\cdot \right)$ in such a way that only the spots within the same group are spatially correlated, while spots from different groups are poorly correlated. Details on the covariance parameters are given in the caption of Figure 3.

FIG 3.

Left: map of the spots used to generate the simulation experiments, extracted from the subject 151507 contained in the package spatialLIBD. The clusters are of equal size, $p_1=p_2=p_3=200$. Right: comparison of the covariance functions used in the three clusters of spots. When $r=1$, the covariance is Exponential with scale ${\theta }_E=50$, when $r=2$, it is Rational Quadratic with ${\theta }_R=50$ and ${\alpha }_R=2$, and when $r=3$ it is Gaussian with scale ${\theta }_G=70$.

Last, we set the values of the spatial signal-to-noise ratios ${\tau }_{kr}^{\text{true}}/{\xi }_{kr}^{\text{true}}$. The additional identifiability constraint ${\tau }_{kr}^{\text{true}}+{\xi }_{kr}^{\text{true}}=c_{kr}^{\text{true}}$ leads to a unique value of the parameters ${\tau }_{kr}^{\text{true}}$ and ${\xi }_{kr}^{\text{true}}$. Note that, due to the identifiability issue described in Section 3.1, which holds also for the simulation model, the value assigned to $c_{kr}^{\text{true}}$ is in practice irrelevant. For this reason, without loss of generality we assumed $c_{kr}^{\text{true}}=c^{\text{true}}=10$, for any $k$ and $r$. In our simulations, we considered three cases: (i) no spatial effect, ${\tau }_{kr}^{\text{true}}/{\xi }_{kr}^{\text{true}}=0$; (ii) the spatial effect is as much as the nugget effect, ${\tau }_{kr}^{\text{true}}/{\xi }_{kr}^{\text{true}}=1$; and (iii) the spatial effect is considerably larger than the nugget effect, ${\tau }_{kr}^{\text{true}}/{\xi }_{kr}^{\text{true}}=3$. Finally, we set ${\mu }_{kr}^{\text{true}}=0$ to test if SpaRTaCo is able to recover the co-clusters using the covariance of the data without being driven by the effect of the mean.

4.2. Competing models and evaluation criteria.

We fit SpaRTaCo on the simulated data taking $k\left(\cdot ;\hspace{0.17em}\cdot \right)$ in Formula (2) as the exponential kernel, which has a lower decay than the more common Gaussian kernel considered by Svensson, Teichmann and Stegle (2018) and Sun, Zhu and Zhou (2020). The estimation is carried running the algorithm described in Section 3.2 five times in parallel to avoid local maxima. The procedure is run for 5,000 iterations, and if the classification log-likelihood function is still growing, it is run until reaching 10,000 iterations. In addition to SpaRTaCo, we consider also the following co-clustering models:

• two independent K-MEANS, applied separately to the rows and to the columns of the data matrix, using the R function kmeans;

• the biclustering algorithm BC, and its sparse version sparseBC with $\lambda =1$, 10, 20, using the R package sparseBC;

• the matrix variate normal algorithm MVNb with the following setups: 1) $\lambda =1$, $\rho \text{Σ}=\rho \text{Δ}=0.25$, 2) $\lambda =10$, $\rho \text{Σ}=\rho \text{Δ}=2.5$ and 3) $\lambda =20$, $\rho \text{Σ}=\rho \text{Δ}=5$. We had to implement a slight modification of the function matrixBC contained in the R package sparseBC, as its original form could not handle the computation of the logarithm of the determinant of some matrices.

• LBM, using the R package blockcluster;

Tan and Witten (2014) do not give any indication on how to select the penalization parameters $\rho \text{Σ}$ and $\rho \text{Δ}$ of MVNb. In their simulation experiments and real data applications, they simply set $\lambda$ to be much larger than $\rho \text{Σ}$ and $\rho \text{Δ}$. For this reason, in our simulations we fit MVNb with three setups, where the $\lambda$ values are the same of sparseBC, and $\rho \text{Σ}$ and $\rho \text{Δ}$ are taken equal to a quarter of $\lambda$. We measure the clustering accuracy by comparing the estimated row and column clusters with the true ones using the $\text{CER}$. In this section, we do not focus on the parameter estimates returned by SpaRTaCo, because the principal goal is evaluating the classification accuracy of the models. We leave the interpretation of the parameter estimates to Section 5.

4.3. Simulation 1.

We generated 9 blocks of size $n_k=200\times p_r=200$, for every $k$ and $r$. We assume that the variances and covariances of the genes do not change with respect to the spot clusters, thus ${\text{Σ}}_{kr}^{\text{true}}={\text{Σ}}_k^{\text{true}}$ for all $r$. We draw ${\text{Σ}}_k^{\text{true}}$ as follows:

$${\text{Σ}}_1^{\text{true}}\sim \mathcal{W}i\left(210,\hspace{0.17em}0.03{\mathbb{I}}_{200}\right),\hspace{1em}{\text{Σ}}_2^{\text{true}}\sim \mathcal{W}i\left(230,\hspace{0.17em}0.05{\mathbb{I}}_{200}\right),\hspace{1em}{\text{Σ}}_3^{\text{true}}\sim \mathcal{W}i\left(200,\hspace{0.17em}{\text{Σ}}_1^{\text{true}}/150\right),$$ (10)

where $\mathcal{W}i\text{(}a,\hspace{0.17em}b\text{)}$ denotes a Wishart distribution with degrees of freedom $a$ and scale matrix $\mathrm{b}$. Generating the covariance matrices from a Wishart distribution ensures that the draws are positive definite. The simulation setup in Formula (10) was selected after both numerical and graphical evaluations. More details on the motivations which led to this setup are given in Supplementary Section 3.

We designed a spatial experiment in which three clusters of genes have a grade of spatial expression that changes in three different areas of the tissue sample. The tessellation of the data matrix into blocks and the values of the spatial signal-to-noise ratios appear in Figure 4 (a). Figure 5 displays a spatial experiment generated under this framework, to show how the average gene expression changes across the 9 blocks. For example, in the left panel $\left(k=1\right)$ there is an evident spatial expression across the spots from clusters $r=2$ and $r=3$, while the spots in $r=1$ are randomly positive or negative due to the absence of spatial dependency. Different spatial expression profiles across the image are distinguishable also in real data, as seen in Supplementary Figure 3, which displays the expression of three genes on the subject 151507. The real and simulated experiments appear very similar, confirming that our simulations are realistic and can be used for testing methods designed for 10X Visium data. We simulated 10 replicates of this experiment and we fitted the co-clustering models listed in Section 4.2 using $K=R=3$. The boxplots of the row and the column $\text{CER}$ over the 10 replicates appear in the first line of Figure 6. SpaRTaCo outperforms the competing models and leads to no clustering errors. Good results on the rows are achieved also by the LBM, while on the columns the k-means type algorithms (K-MEANS, BC and sparseBC) and the MVNb with $\rho \text{Σ}=\rho \text{Δ}=5$ perform better than the other competitors. A further confirmation of the accuracy of SpaRTaCo for modelling this spatial experiment comes from the value of estimated clustering uncertainties, which are ${\epsilon }_k^{\text{rows}}<0.001$ and ${\epsilon }_r^{\text{cols}}<0.001$, for every $k$ and $r$. A graphical representation of these quantities across the 10 replicates is given in Supplementary Figure 4.

FIG 4.

Representation of the latent block structures used to generate the simulation experiments. All the blocks in Panels (a)-(c) have the same size and are colored according to the value of the spatial signal-to-noise ratio ${\tau }_{kr}^{\text{true}}/{\xi }_{kr}^{\text{true}}$. The setup in Panel (a) is used in Sections 4.3 and 4.6, Panel (b) is used in Section 4.4, Panel (c) in Section 4.5 and Panel (e) in Section 4.7. Panel (d) gives the hidden block structure of Simulation 4.7. Within the columns 1 and 2, the row clusters have the same size (200), while in the third column it is $n_{13} =100$, $n_{23} =200$ and $n_{33} =300$. The numbers from 1 to 6 on the right denote the alternative clusters ${\mathcal{C}}_1^{\ast \text{true}}\text{,…,}\hspace{0.17em}{\mathcal{C}}_6^{\ast \text{true}}$.

FIG 5.

Examples of a spatial experiment generated under Simulation 1. The spots are coloured according to $n_k^{-1}{\left({\text{X}}^{k\text{.}}\right)}^T{1}_{n_k}$, the average expression of the $k$-th gene cluster. The three spot clusters are displayed with different symbols. The co-clusters with no spatial expression are ($k=1$, $r=1$), ($k=2$, $r=2$) and ($k=3$, $r=3$), and the co-clusters with the largest spatial signal-to-noise ratio are ($k=1$, $r=2$), ($k=2$, $r=3$) and ($k=3$, $r=1$).

FIG 6.

Results from Simulations 1–4. For each scenario, we generated 10 datasets and we applied the co-clustering models listed in Section 4.2. Every figure gives the boxplots of the $CER$ obtained on the rows and on the columns.

This experiment has demonstrated that the presence of spatial covariance patterns, if not properly accounted for, heavily impacts on the performance of the standard co-clustering models. Since the MVNb is designed to flexibly estimate the covariance of the blocks, in theory it should be the best candidate for such complex experiments. However, the formulation of ${\widehat{\text{Σ}}}_k^{\text{MVNB}}$ and ${\widehat{\text{Δ}}}_r^{\text{MVNB}}$ is too generic for capturing the spatial correlation across the spots, causing a poor clustering result. As a confirmation of this statement, we notice that the smallest classification error made by MVNb is reached when the penalization parameters $\rho \text{Σ}$ and $\rho \text{Δ}$ are large, leading the estimated matrices ${\widehat{\text{Σ}}}_k^{\text{MVNB}}$ and ${\widehat{\text{Δ}}}_r^{\text{MVNB}}$ to be diagonal.

As a second step of this experiment, we tested the model selection criterion based on the ICL proposed in Section 3.4. Using the same 10 replicates of the experiment, we ran SpaRTaCo with $K$ and $R$ taking values in $\left\{2,\hspace{0.17em}3,\hspace{0.17em}4\right\}$. Supplementary Figure 5 shows that the proposed ICL always selects the correct model dimension, while the classification log-likelihood favors models with a larger number of co-clusters than the truth.

While the ICL criterion accurately selects the number of co-clusters, it is a computationally expensive procedure due to the large number of times that the estimation must be run. Hence, we compared our model selection method with two faster alternatives: the first selects the number of row and column clusters separately by combing a dimension reduction method with K-MEANS (details are given in Supplementary Section 4), the second, proposed by Tan and Witten (2014), performs a 10-fold cross-validation using sparseBC; a function that implements this last method can be found into the R package sparseBC. The first criterion selected 6 row clusters on the 90% of the replicates of Simulation 1, and 5 clusters in the remaining 10%; on the columns, it selected 3 clusters on the 33% of the replicates, and 4 clusters on the remaining 77%. The second criterion was applied with $K$ and $R$ taking values in $\left\{2,\hspace{0.17em}\dots ,\hspace{0.17em}6\right\}$ and fixing $\lambda =10$, but it has revealed to be inadequate for this kind of data, as it selected $K=6$ and $R=6$ on every replicate of the experiment.

4.4. Simulation 2.

The second simulation experiment differs from the first in the values of the spatial signal-to-noise ratios, which are now taken as in Figure 4 (b). For any $r$, the signal-to-noise ratios $\left\{{\tau }_{kr}^{\text{true}}/{\xi }_{kr}^{\text{true}},\hspace{0.17em}k=1\text{,…,}\hspace{0.17em}{K}^{\text{true}}\right\}$ have all the same value. As a consequence, ${\text{Δ}}_{kr}^{\text{true}}={\text{Δ}}_r^{\text{true}}$ for any $k$. Under the current setup, the marginal distribution of a row $i\in {\mathcal{C}}_k^{\text{true}}$ under the data generating model given in Formula (8) does not depend on $k$ and so it is not informative of the row clustering. The only discriminating facets are the cross-covariances of the rows and of the columns, which carry the information about the row clusters through the matrices ${\text{Σ}}_k^{\text{true}}$. This framework is thus meant to evaluate the performance of SpaRTaCo when all the genes have the same spatial expression profiles across the tissue. A representation of a spatial experiment generated under this framework is given in the top row of Supplementary Figure 6.

We ran the co-clustering models using $K=R=3$ on 10 replicates on the proposed experiment; the results are displayed in the second line of Figure 6. Our model outperforms the competitors: on the rows, the median $\text{CER}$ from SpaRTaCo is less than 0.2, while on the columns it returns a perfect classification on all replicates. The estimated row clustering uncertainty is low (${\epsilon }_k^{\text{rows}}<0.15$, $\forall k$), while the column clustering uncertainty is practically null. Details are given in the second row of Supplementary Figure 4. Both Simulations 1 and 2 have shown that SpaRTaCo works properly even if the spatial covariance function employed by the fitted model in Formula (2) does not match the covariance functions of the data generating process. In particular, Simulation 2 has highlighted this remarkable result because the only cluster of columns for which the spatial covariance function is correctly specified is $r=1$, which however is devoid of any spatial effect, as ${\tau }_{k1}^{\text{true}}=0$ for any $k$.

The best competitor on the rows is the LBM, with a median $\text{CER}$ of 0.44. On the columns, the best results are from the k-means type models, or alternatively from the MVNb with $\lambda =20$ and $\rho \text{Σ}=\rho \text{Δ}=5$. Considerable results are obtained also with the LBM; however, its classification accuracy is more variable. This experiment hence confirms what we have already observed in Simulation 1, namely that, in the presence of spatial covariance patterns in the data, the model of Tan and Witten (2014) tends to fail in recovering the correlation structure, at least in our simulation setup. This is demonstrated by the diagonal estimated covariance matrices $\left\{{\widehat{\text{Σ}}}_k^{\text{MVNB}},\hspace{0.17em}k=1,\hspace{0.17em}2,\hspace{0.17em}3\right\}$ and $\left\{{\widehat{\text{Δ}}}_r^{\text{MVNB}},\hspace{0.17em}r=1,\hspace{0.17em}\text{2,}\hspace{0.17em}\text{3}\right\}$.

4.5. Simulation 3.

The third simulation experiment assumes that the spatial signal-to-noise ratio ${\tau }_{kr}^{\text{true}}/{\xi }_{kr}^{\text{true}}$ is constant across the blocks within the same row cluster $k$; as a consequence, ${\tau }_{kr}={\tau }_k$ for any $r$. This case is illustrated in Figure 4 (c). Notice for example that the rows in ${\mathcal{C}}_1^{\text{true}}$ are not spatially expressed in any of the three column clusters. Under the current simulation setup, the marginal distribution of a row $i\in {\mathcal{C}}_k^{\text{true}}$ given in Formula (8) is informative on the column clusters only through the different spatial kernels $k_r^{\text{true}}\left(\cdot \hspace{0.17em};\hspace{0.17em}{\varphi }_r^{\text{true}}\right)$, while, as already discussed in Section 4.1, the marginal distribution of a column $j\in {\mathcal{D}}_r^{\text{true}}$ is never informative on the column clusters. The cross-covariances of the rows and of the columns are informative of both rows and column clustering. Under this framework, it is challenging to determine the image areas with spatial interaction, because all the genes in a cluster ${\mathcal{C}}_k^{\text{true}}$ are spatially expressed with the same intensity over the whole tissue. An example of a spatial experiment generated under this simulation setup is given in the bottom row of Supplementary Figure 6.

We ran the co-clustering models on 10 replicates of the experiment using $K=R=3$; the results appear in the third line of Figure 6. On the rows, SpaRTaCo outperforms the competitor models returning a $\text{CER}$ of zero for all replicates. On the columns, its clustering accuracy is highly variable: the median $\text{CER}$ is 0.21, the first and the third quartiles are 0.08 and 0.25, and extremes are 0 and 0.36. The competitor models, and in particular the k-means type models, are substantially less variable than SpaRTaCo. Their median column $\text{CER}$ is 0.13. However, none of them ever returns a perfect classification.

Even if SpaRTaCo has returned unstable results on the columns, the advantages brought by our model against the competitors are many, and are particularly visible from the results on the rows. The column clustering changes considerably across the replicates because, in the current setup, our estimation algorithm is more sensible to the starting points. This aspect is highlighted also by the estimated column clustering uncertainties ${\epsilon }_r^{\text{cols}}$, whose values across the 10 replicates are now mainly between 0.3 and 0.4 (see Supplementary Figure 4). From our experience, if independent runs of the estimation algorithm reach distant stationary points, both the number of starting points and the number of iterations of the SE Step should be increased to favor a faster exploration of the space of the configurations.

4.6. Simulation 4.

Up to now, we built the simulation experiments under the framework in which SpaRTaCo is designed to work properly, that is the case where the genes/spots in a cluster are correlated only with the other genes/spots of the same cluster. In this section, we violate this assumption and we design a spatial experiment where both the genes and the spots are correlated also with genes and spots from other clusters. This experiment aims to study the effects of an additional dependency structure across the data that is not accounted by the fitted model.

Let ${\mathrm{X}}_s$ be a $600\times 600$ spatial experiment made of 9 equally sized blocks, generated as in Simulation 1, and ${\text{X}}_b\sim \mathcal{M}\mathcal{V}\mathcal{N}\left(0,\hspace{0.17em}{\text{Σ}}_b,\hspace{0.17em}{\text{Δ}}_b\right)$. Both ${\text{Σ}}_b$ and ${\text{Δ}}_b$ are squared matrices of size 600: the first is drawn from ${\text{Σ}}_b\sim \mathcal{W}\left(600,\hspace{0.17em}0.015{\mathbb{I}}_{600}\right)$, the second is ${\text{Δ}}_b={\tau }_b{\mathcal{K}}^b\left(S;\hspace{0.17em}{\sigma }_b\right)+{\xi }_b{\mathbb{I}}_{600}$, where ${\mathcal{K}}^b\left(S;\hspace{0.17em}{\sigma }_b\right)={\left(k^b\left(‖s_j-s_{j^{\prime }}‖;\hspace{0.17em}{\sigma }_b\right)\right)}_{1\le j,j^{\prime }\le 600}$ and $k^b\left(\cdot \hspace{0.17em};\hspace{0.17em}{\sigma }_b\right)$ is a Gaussian kernel with scale ${\sigma }_b$ We set ${\tau }_b={\xi }_b=c^{\text{true}}/2$ and ${\sigma }_b=50$. The final simulation experiment is made as follows: $\text{X}={\lambda }_s{\text{X}}_s+{\lambda }_b{\text{X}}_b$, where ${\lambda }_s$, ${\lambda }_b\ge 0$. We generated 10 replicates of the current experiment, each time drawing first the matrices ${\mathrm{X}}_s$ and ${\mathrm{X}}_b$, and then combining them to form $\mathrm{X}$. Supplementary Figure 7 shows a single realization of ${\mathrm{X}}_s$, ${\mathrm{X}}_b$ and $\mathrm{X}$ using ${\lambda }_s={\lambda }_b=\sqrt{0.5}$. This value satisfies the constraint ${\lambda }_s^2+{\lambda }_b^2=1$ that we imposed to keep the variance of the current experiment comparable with the previous experiments proposed in this work.

We ran the co-clustering models using $K=R=3$; results appear in the last row of Figure 6. Despite the additional correlation structure in the data brought by the nuisance signal ${\mathrm{X}}_b$, SpaRTaCo outperforms its competitors on both the row and the column clustering. In the right plot, the $\text{CER}$ boxplots are more variable than in the left plot, therefore, the nuisance component has affected more the column than the row clustering of the employed models. Among the competitors, K-MEANS and MVNb with $\lambda =10$ and $\rho \text{Σ}=\rho \text{Δ}=2.5$ are the least affected by the nuisance: the former because it performs the clustering on the two dimensions of the data matrix separately, the latter because it regulates the estimate of the row and column covariances with a moderate shrinkage factor. The effect of the additional dependency structure is visible also on the distributions of ${\epsilon }_k^{\text{rows}}$ and ${\epsilon }_r^{\text{cols}}$, which are displayed in the last line of Supplementary Figure 4: over the 10 replicates, the row clustering uncertainties spread between 0 and 0.17, and column uncertainties between 0 and 0.5.

4.7. Simulation 5.

In the last experiment, we intentionally violate two important assumptions made by SpaRTaCo: the first states that the latent block structure of an experiment corresponds to a segmentation of the data matrix into $K$ row clusters and $R$ column clusters, the second states that the spatial covariance functions change only across the spots and not across the genes. For instance, we generate a spatial experiment creating first the $R^{\text{true}}$ column clusters, and then generating the $K^{\text{true}}$ row clusters independently for each column cluster. From a biological perspective, this setup simulates the case where the expression profiles of some genes are similar only in some specific areas of the tissue sample. In addition, following the discoveries of Svensson, Teichmann and Stegle (2018) and Sun, Zhu and Zhou (2020) that different genes are s.e. according to different spatial covariance functions, we consider a data generating model where the spatial kernels change with respect to the gene cluster index $k$ and no longer with respect to the spot cluster index $r$.

Let ${\mathcal{C}}_{kr}^{\text{true}}$ and ${\mathcal{D}}_r^{\text{true}}$ be the actual row and column clusters, with $k=1\text{,…,}\hspace{0.17em}{K}^{\text{true}}$ and $r=1\text{,…,}\hspace{0.17em}{R}^{\text{true}}$, where ${\mathcal{C}}_{kr}^{\text{true}}=\left\{i=1\text{,…,}\hspace{0.17em}n\hspace{0.17em}\text{:}\hspace{0.17em}{\mathcal{Z}}_{\text{ir }}^{\text{true}}=k\right\}$ is the $k$-th row cluster within the $r$-th column cluster, and $\left|{\mathcal{C}}_{kr}^{\text{true}}\right|=n_{kr}$ . Under the current setup, we draw ${\text{X}}^{kr}\sim \mathcal{M}\mathcal{V}\mathcal{N}\left({\mu }_{kr}{1}_{n_k\times p_r},\hspace{0.17em}{\text{Σ}}_{kr}^{\text{true}},\hspace{0.17em}{\text{Δ}}_{kr}^{\text{true}}\right)$, where the covariance across the spots is now equal to ${\text{Δ}}_{kr}^{\text{true}}={\tau }_{kr}^{\text{true}}{\mathcal{K}}_k^{\text{true}}\left(S^r;\hspace{0.17em}{\varphi }_k^{\text{true}}\right)+{\xi }_{kr}^{\text{true}}{\mathbb{I}}_{p_r}$. Notice that, differently from Section 4.3, the covariance matrices of the rows ${\text{Δ}}_{kr}^{\text{true}}$ change with respect to $r$ because the number of observations in the cluster is $n_{kr}$ (and no longer $n_k$). In addition, the model assumes that the $kr$-th block has mean µkr . The tessellation of the data matrix into blocks is shown in Figure 4 (d). The size of the clusters is $n_{kr}=200$ for $k=1$, 2, 3 and $r=1$, 2, while $n_{13}=100$, $n_{23}=200$ and $n_{33}=300$. The covariance matrices of the rows are drawn as follows:

$${\text{Σ}}_{1r}^{\text{true}}\sim \mathcal{W}i\left(n_{1r}+10,\hspace{0.17em}0.03{\mathbb{I}}_{n_{1r}}\right),\hspace{1em}{\text{Σ}}_{2r}^{\text{true}}\sim \mathcal{W}i\left(n_{2r}+30,\hspace{0.17em}0.05{\mathbb{I}}_{n_{2r}}\right),\hspace{1em}{\text{Σ}}_{3r}^{\text{true}}\sim \mathcal{W}i\left(n_{3r},\hspace{0.17em}{\text{Σ}}_{3r}^{\ast }/150\right),$$

where ${\text{Σ}}_{3r}^{\ast }\sim \mathcal{W}i\left(n_{3r}+10,\hspace{0.17em}0.03{\mathbb{I}}_{n_{3r}}\right)$. Notice that this setting is nothing but a generalization of what appears in Formula (10). Calling ${\mu }_{k\text{.}}^{\text{true}}=\left({\mu }_{k1}^{\text{true}},\hspace{0.17em}{\mu }_{k2}^{\text{true}},\hspace{0.17em}{\mu }_{k3}^{\text{true}}\right)$, we set the mean values equal to ${\mu }_{1.}^{\text{true}}=\left(-3,\hspace{0.17em}0,\hspace{0.17em}3\right)$, ${\mu }_{2.}^{\text{true}}=\left(3,-3,\hspace{0.17em}0\right)$ and ${\mu }_{3.}^{\text{true}}=\left(0,\hspace{0.17em}3,-\text{3}\right)$. Finally, the employed spatial signal-to-noise ratio values $\left\{{\tau }_{kr}/{\xi }_{kr}\right\}$ are shown in Figure 4 (e).

To facilitate the model evaluation and the interpretation of the results, we assign to every row $i$ an alternative clustering label ${\mathcal{Z}}_i^{\ast \text{true}}$ such that ${\mathcal{Z}}_i^{\ast \text{true}}={\mathcal{Z}}_{i^{\prime }}^{\ast \text{true}}$ if $i$, $i^{\prime }\in \left({\mathcal{C}}_{k_{1}1}^{\text{true}}\cap {\mathcal{C}}_{k_{2}2}^{\text{true}}\cap {\mathcal{C}}_{k_{3}3}^{\text{true}}\right)$, for some $k_1$, $k_2$, $k_3\in \left\{1,\hspace{0.17em}2,\hspace{0.17em}3\right\}$. In words, this means that the new clusters are formed by the rows that belong to the same cluster in all of the three column clusters. The new row clustering labels appear on the right side of Figure 4 (d). In our experiment, every ${\mathcal{Z}}_i^{\ast \text{true}}\in \left\{1\text{,…,}\hspace{0.17em}6\right\}$, and ${\mathcal{C}}_b^{\ast \text{true}}=\left\{i=1\text{,…,}\hspace{0.17em}n\hspace{0.17em}\text{:}\hspace{0.17em}{\mathcal{Z}}_i^{\ast \text{true}}=b\right\}$ is the $b$-th alternative cluster with size $\left|{\mathcal{C}}_b^{\ast \text{true}}\right|=100$, for $b=1\text{,…,}\hspace{0.17em}6$.

To reduce the computational cost spent on the simulation, we generated a single replicate of the experiment, and we fitted SpaRTaCo using $K=3,\dots ,\hspace{0.33em}9$, while the number of column clusters is kept equal to its real value, $R=3$. Supplementary Figure 8 (a) shows that the ICL criterion selects $K=8$ as the optimal model dimension; using the log-likelihood, we would have wrongly picked $K=9$, confirming the importance of using a suitable information criterion to drive the model selection. In addition, one could consider also a model with a smaller number of row clusters: for example, $K=5$ looks also a reasonable choice, because it corresponds to a local maximum. SpaRTaCo with $K=8$ returns a row $\text{CER}$ of 0.028 and a column $\text{CER}$ of 0. In details, the model correctly recovers the gene clusters 2, 4, 5 and 6, while the genes in ${\mathcal{C}}_1^{\ast \text{true}}$ and ${\mathcal{C}}_3^{\ast \text{true}}$ are split into two separate groups. The estimated clustering uncertainty is ${\epsilon }_r^{\text{cols}}<0.004$, for $r=1$, 2, 3, while on the rows it varies between 0 and 0.19. Thus, some of the genes clusters are clearly visible, while others are unstable. As a comparison, we give also the results using $K=5$. The $\text{CER}$ on the rows is 0.056, and it is 0 on the columns; the estimated clustering uncertainties are ${\epsilon }_k^{\text{rows}}<0.001$, for $k=1,\dots ,\hspace{0.17em}5$, and ${\epsilon }_r^{\text{cols}}<0.06$, for $r=1$, 2, 3. The fact that the row clusters of the model with $K=5$ are more stable than the ones with $K=8$ gives additional support to the idea of selecting the model with a smaller number of blocks, but both models yield reasonably good results.

We finally run the competing models using $K=5$ and $R=3$; results are shown in Supplementary Figure 8 (b). Thanks to the difference in mean across the blocks, all the competing models can clearly distinguish the clustering structure of the spots. However, due to the spatial dependency effects, their performance in clustering the genes is poor, confirming, once again, the improvement brought by SpaRTaCo.

5. Application.

In this section, we analyze the human dorsolateral prefrontal cortex sample from the subject 151673 studied by Maynard et al. (2021) that we briefly described in Section 1.1 and shown in Figure 1. The dataset has 33,538 genes measured over 3,639 spots. Similarly to 10X scRNA-seq protocols, 10X Visium yields unique molecular identifier (UMI) counts as gene expression values. As a first step, we sought to exclude uninformative genes and reduce the analysis to a lower dimensional problem. We applied the gene selection procedure for UMI count data proposed by Townes et al. (2019), i.e., we fit a multinomial model on every vector of gene expression and compute the deviance. Based on the criterion that large deviance values are associated to informative genes, we kept the first 500 genes and discarded the remaining ones. Supplementary Figure 9 shows that the deviance, which is very high for the top genes, reaches a plateau after 200 genes. To normalize the data, we computed, for each selected gene, the deviance residuals based on the binomial approximation of the multinomial distribution as done in Townes et al. (2019). The result of this procedure is the expression matrix $\mathrm{X}$ whose entries are $x_{ij}\in ℝ$ and whose row vectors ${\mathrm{x}}_i$. yield approximately symmetric histograms. Boxplots of the transformed gene expression vectors are given in Supplementary Figure 10, where it is shown also that there is no practical difference between using the binomial or the Poisson for computing the the residuals.

We fitted SpaRTaCo with all the configurations in $\left\{\left(K,\hspace{0.17em}R\right)\hspace{0.17em}\text{:}\hspace{0.17em}K=2,\hspace{0.17em}R=7\text{,…,}\hspace{0.17em}12\right\}$, starting the estimation of each model from five different initial points. More details about the setup of the estimation algorithm and the computational costs are given in Supplementary Section 5. The range of column cluster values reflects the number of biological layers that appear in Figure 1. As we already mentioned in Section 1.1, SpaRTaCo performs a substantially different image clustering than BayesSpace or GIOTTO; thus, we do not expect the clusters discovered by SpaRTaCo to match the cortical layers. However, we believe that their number could still be indicative of the biological diversity of this specific area. Supplementary Figure 11 (a) gives the ICL values of the models with $K=2$. Although our criterion selects $K=2$ and $R=12$, we believe that the local maximum in correspondence of ($K=2$, $R=9$) represents also a valid solution. In fact, a large value of $R$ would result in too many small clusters, complicating the biological interpretation. Furthermore, we fixed $R=9$ and we explored the options $K\in \left\{1,\hspace{0.17em}3,\hspace{0.17em}4\right\}$ to investigate the absence of gene clusters $\left(k=1\right)$ and the presence of multiple clusters. However, the ICL selects $K=2$. Figure 7 (a) displays the tissue map colored according to the estimated clusters. The White Matter spots are covered by clusters ${\mathcal{D}}_1$, ${\mathcal{D}}_7$, ${\mathcal{D}}_8$ and ${\mathcal{D}}_9$; this last one is placed at the border between the White Matter and Layer 6. The remaining clusters cover the surface within the Layers 2–6. Last, Layer 1 is covered by ${\mathcal{D}}_4$ and mostly by ${\mathcal{D}}_5$ Incidentally, we note that the spot clusters within the White Matter are the ones with the smallest grade of uncertainty (see the right plot in Supplementary Figure 11 (b)).

FIG 7.

Results on the human dorsolateral prefrontal cortex data. The first row displays the 3,639 spots: in Panel (a) they are colored according to the clusters returned by SpaRTaCo and shaped according to the clustering uncertainty ${\epsilon }_r^{\text{cols}}$, in Panel (b) they are colored according to the average gene expression in the estimated cluster ${\mathcal{C}}_2$. Panels (c) and (d) represent the data matrix tessellated into the 18 discovered blocks. Both the genes and the spots are reordered based on the estimated clusters for visualization purposes. The graphs are colored according to the estimated mean ${\widehat{\mu }}_{kr}$ (c) and to the estimated spatial signal-to-noise ratio ${\widehat{\tau }}_{kr}/{\widehat{\xi }}_{kr}$ (d).

As for the row clustering, 109 of the genes in Cluster ${\mathcal{C}}_2\left(n_2=129\right)$ were ranked within the top 200 most informative genes by the deviance procedure of Townes et al. (2019). Figure 7 (b) displays the spots colored according to $n_2^{-1}{\left({\text{X}}^{2.}\right)}^T{1}_{n_2}$, the average expression of the genes in ${\mathcal{C}}_2$, from which it emerges that the expression tends to be larger within the White Matter than in the rest of the cortical area. Panels (c) and (d) in Figure 7 display the estimated means ${\widehat{\mu }}_{kr}$ and spatial signal-to-noise ratios ${\widehat{\tau }}_{kr}/{\widehat{\xi }}_{kr}$ within each block. It appears that the spatial activity of the genes in ${\mathcal{C}}_2$ is largely evident within the internal area of the White Matter $\left({\widehat{\tau }}_{21}/{\widehat{\xi }}_{21}=3.45\right)$ and progressively decreases approaching Layer 6 (${\widehat{\tau }}_{28}/{\widehat{\xi }}_{28}=1.55$ and ${\widehat{\tau }}_{29}/{\widehat{\xi }}_{29}=0.58$). These genes show also a moderate spatial expression on the rest of the cortical area (${\widehat{\tau }}_{2r}/{\widehat{\xi }}_{2r}\in \text{[}0.39,\hspace{0.17em}0.9\text{]}$, for $r=2,\dots ,\hspace{0.17em}6$). Last, cluster ${\mathcal{D}}_7$ denotes a restricted group of spots that are present both within and outside the White Matter, with a non-negligible spatial effect $\left({\widehat{\tau }}_{27}/{\widehat{\xi }}_{27}=1.60\right)$. On the contrary, the genes in ${\mathcal{C}}_1\left(n_1=371\right)$ show a small spatial variation in every spot cluster expect in ${\mathcal{D}}_1$ (${\widehat{\tau }}_{11}/{\widehat{\xi }}_{11}=0.71$ and ${\widehat{\tau }}_{1r}/{\widehat{\xi }}_{1r}\le 0.31$ for all $r\ne 1$), suggesting a constant variation of these genes throughout the cortical area. In fact, ${\mathcal{C}}_1$ is enriched for housekeeping genes with respect to ${\mathcal{C}}_2$ (chi-square test, $p=2.6 \times {10}^{-4}$). Housekeeping genes are maintainers of the cellular functions and their activity is not restricted to a specific cell type (Eisenberg and Levanon, 2003). It is therefore expected that these genes show a small spatial variation across the tissue. We notice also from Figure 7 (c) that the estimated means $\left\{{\widehat{\mu }}_{1r},\hspace{0.17em}r=1\text{,…,}\hspace{0.17em}9\right\}$ are complementary to $\left\{{\widehat{\mu }}_{2r},\hspace{0.17em}r=1\text{,…,}\hspace{0.17em}9\right\}$: the expression level is smaller within the White Matter area than outside. To ensure that the co-clustering was not driven only by the mean effects, we run also sparseBC using the same number of blocks and $\lambda =10$: the $\text{CER}$ between the gene clusters returned by SpaRTaCo and sparseBC is 0.44, confirming that the two methods perform a substantially different grouping of the data. A further confirmation of the evidence of our gene clustering is given by the very small uncertainty displayed in the left panel of Supplementary Figure 11 (b).

The results discussed above allow us to answer the first two research questions listed in Section 1.1 that motivated our work. We now turn our attention to the third research question, namely the identification of genes that exhibit high specific variation. To do so, for every spot cluster $r$, we investigate the conditional random variables ${\sigma }_{{\widehat{\mathcal{Z}}}_{i}r,i}^2\text{|}{x}_{i\text{.}}^{{\widehat{\mathcal{Z}}}_{i}r}$, for $i=1,\dots ,\hspace{0.17em}n$, to determine which genes are most highly variable in each block. We display their density in Supplementary Figures 12 , highlighting in red the twenty genes with the largest $\mathbb{E}\left({\sigma }_{{\widehat{\mathcal{Z}}}_{i}r,i}^2\text{|}{\mathrm{x}}_{i\text{.}}^{{\widehat{\mathcal{Z}}}_{i}r}\right)$, for every $r$. We expect that genes with a large gene-specific variance in some areas are likely to be informative of the biological mechanisms occurring there.

First, we notice that all the most variable genes in each of the nine spot clusters belong to ${\mathcal{C}}_2$. Among the highly variable genes in ${\mathcal{D}}_1$, ${\mathcal{D}}_8$ and ${\mathcal{D}}_9$ there are MBP and PLP1, which are responsible, respectively, for the production and the maintenance of myelin, the covering sheath of the nerve fibers in the White Matter. Conversely, among the highly variable genes in ${\mathcal{D}}_2$ and ${\mathcal{D}}_7$, we notice PCP4 and CCK: these are markers of distinct subtypes of excitatory neurons present in Layers 5–6 (Hodge et al., 2019). We display the expression of the four genes discussed here in Supplementary Figure 13, showing their pattern in the spot clusters where they appear to be highly variable.

Supplementary Figure 12 highlights some important differences between ranking genes according to the posterior distribution of our gene-specific variance ${\sigma }_i^2$ and the method of Townes et al. (2019) that only ranks genes based on variability without considering the spatial context. This analysis may be used to highlight important genes that would have been missed if the spatial structure of the data would not have been taken into account. Two examples are CERCAM and SAA1: their ranks according to Townes et al.’s method were 465 and 271, while SpaRTaCo places them among the most variable genes in the White Matter area (cluster ${\mathcal{D}}_1$) and in a region covering the Layers 3, 5 and 6 (cluster ${\mathcal{D}}_6$), respectively. We display their expression over the whole tissue in Supplementary Figure 14. CERCAM encodes a cell adhesion protein involved in leukocyte transmigration across the blood-brain barrier (Starzyk et al., 2000), while SAA1 is highly expressed in response to inflammation in mouse glial cells (Barbierato et al., 2017).

Taken together, these results convincingly show that our model is able to partition the tissue in coherent clusters, which exhibit cluster-specific gene expression, both spatially coordinated and otherwise, and to detect highly variable genes of potential biological interest in specific areas of the tissue that would not have been found without considering their spatial variability.

6. Discussion.

The growing demand of appropriate statistical methods to analyze spatial transcriptomic experiments has driven us to develop SpaRTaCo, a model-based co-clustering tool that groups genes with a similar profile of spatial expression in specific areas of a tissue. SpaRTaCo brings the concepts of spatial modelling into the co-clustering framework, and thus it can be applied to any dataset with entries in the real domain and whose row or column vectors are multivariate observations recorded at some fixed sites in space. The inference is carried out via maximization of the classification log-likelihood function. To do so, we put together two variants of the EM algorithm, the classification EM and the stochastic EM, forming what we called the classification-stochastic EM. We completed our proposal deriving the formulation of the ICL for our model to drive the model selection.

A series of simulation studies have highlighted that, in the presence of spatial covariance patterns, the major co-clustering models become inadequate to recover the hidden block structure of the data. On the contrary, SpaRTaCo has shown remarkable results in each simulation, managing to distinguish different spatial expression profiles in different areas of the image. It further revealed to be robust to the presence of a nuisance component into the data. The model selection driven by the ICL revealed to be precise but computationally expensive, due to the large number of times the model must be run. On the contrary, other criteria that do not exploit the spatial information of the data are computationally attractive but less accurate. We conclude that the two approaches can be used jointly, using the results given by a fast model selection criterion, such as the PCA-k-means method discussed in Section 4.3, to restrict the range of $K$ and $R$ values to be tested with SpaRTaCo’s ICL criterion. Lastly, we demonstrated how our proposal is capable of answering specific biological research questions using a human brain tissue sample processed with the Visium protocol. Our model has identified two clusters of genes with different spatial expression profiles in nine different areas of the tissue. A subsequent downstream analysis has allowed us to determine the highly variable genes in each of the nine pinpointed areas. We additionally showed that some of the genes considered as poorly informative by the deviance method of Townes et al. (2019) are revealed by SpaRTaCo to be highly variable in specific areas of the tissue sample.

Although this article has introduced a complete solution to answer some relevant questions in the analysis of spatial transcriptomics, we believe that there is space for further extensions. To use SpaRTaCo on spatial transcriptomic experiments, the UMI counts must be transformed through a real-valuated function as discussed at the beginning of Section 2. We performed this step using the pre-processing techniques of Townes et al. (2019), which in our application have led to approximately symmetric distributions of the gene expression vectors ${\mathrm{x}}_i$. In addition, our model is theoretically robust with respect to the presence of heavy tail distributions thanks to the random parameters ${\sigma }_{kr,i}^2$, that allow to go beyond the normal assumption. Nevertheless, SpaRTaCo could be extended to directly model UMI counts, similarly to how SPARK (Sun, Zhu and Zhou, 2020) has extended SpatialDE (Svensson, Teichmann and Stegle, 2018). Second, to overcome the limitations of the stochastic EM presented in Section 4.5, we could explore the simulated annealing algorithm (Van Laarhoven and Aarts, 1987), to reduce the chances of converging to local maxima.