A multiperspective evaluation framework of spatial transcriptomics clustering methods

Abstract

Spatial transcriptomics (ST) allows the exploration of gene expression within tissue microenvironments, driving the development of multiple computational approaches for spatial domain identification. Evaluating these methods typically relies on label-dependent metrics, such as contingency matrices and information-theoretic measures, which require ground-truth annotations, and label-independent metrics, which assess transcriptomic similarity or spatial organization. However, annotations are often incomplete or unavailable, while label-independent metrics fail to jointly evaluate the integration of transcriptomic and spatial information, a core feature of ST clustering methods. To address these limitations, we introduce MultimetricST, a Python-based framework that provides a unified, flexible evaluation strategy integrating both cutting-edge and state-of-the-art label-dependent and label-independent metrics. We applied MultimetricST on two generated synthetic datasets and thirteen datasets derived from seven ST technologies to systematically evaluate the spatial domains identified by eleven state-of-the-art deep learning methods. Our framework highlights the strengths and limitations of each assessment strategy, providing an accessible and reproducible tool for comparative and robust evaluation, and method selection.

Graphical Abstract

Graphical Abstract.

Introduction

Spatial transcriptomics (ST) enables the study of gene expression within the architecture of native tissue, providing insights into how the spatial context influences cellular function. ST allows the characterization of the organization of the cell subpopulation within their spatial contexts, providing information on cellular arrangement and interactions within local tissue microenvironments. ST is useful for studying complex diseases such as cancer, where tissue spatial organization influences tumor development, metastasis, and treatment response [1–4]. It is also applied in the study of neurodegenerative disorders, facilitating the analysis of layer-specific transcriptional changes in the brain [5].

Developments in ST technologies for sequencing and the broad spectrum of clinical applications have expanded the range of computational approaches for the joint analysis of spatial and transcriptional data. Clustering represents a central step in these analyses, enabling a range of downstream tasks such as cell-type annotation, spatial network reconstruction, functional enrichment [6], cell–cell communication analysis [7], and comparative spatial studies between samples or experimental conditions [8].

The early methods were primarily based on distance measures, such as Stardust [9], or on probabilistic models, such as BayesSpace [10]. More recent approaches employ graph-based deep learning techniques, in which ST data are modeled as graphs where nodes represent spatial spots and edges connect those that are spatially adjacent or transcriptionally similar. Many of these methods incorporate contrastive learning strategies to improve the quality of learned representations. Representative examples include GIST [11], GraphST [12], ConST [13], CCST [14], SCAN-IT [15], SpaceFlow [16], and SpaGCN [17]. Other deep learning frameworks that do not rely on contrastive learning include DeepST [18], HERGAST [19], STAGATE [20], and SEDR [21].

Although numerous computational methods have been developed for spatial domain identification, their evaluation remains challenging. Current evaluations of spatial domain identification methods rely on two main categories of metrics: label-dependent metrics, which compare predicted clusters to ground-truth annotations, and label-independent metrics, which assess clustering quality based on transcriptomic or spatial information. The latter are particularly valuable for datasets where annotations are incomplete or unavailable.

Method assessment studies [22, 23] have focused on the clustering accuracy with respect to ground-truth labels and the spatial compactness of clusters using label-independent metrics. However, for datasets lacking reliable annotations, jointly assessing transcriptomic and spatial information remains a major challenge. As a result, current label-independent evaluation strategies often fail to fully capture the complex relationship between spatial organization and gene expression patterns. For example, as highlighted by the SpaGCN authors [17], their spatial domain identification approach relies mainly on gene expression signals, which may lead to inconsistencies between inferred domains and true anatomical structures. Such discrepancies are not adequately reflected by existing label-independent metrics, underscoring the need for more comprehensive evaluation frameworks that jointly consider spatial and transcriptomic features to assess performance across diverse and unlabeled datasets.

To address these limitations, we propose MultimetricST, a Python-based framework that employs twelve metrics to evaluate ST clustering performance. When ground-truth annotations are available, MultimetricST applies clustering metrics derived from contingency matrices, including the Adjusted Rand Index (ARI) [24], which measures agreement between predicted domains and annotated classes, and Purity [25], which quantifies the proportion of correctly assigned labels within each domain. Leveraging information-theoretic criteria, it also computes the Adjusted Mutual Information (AMI) [26], which quantifies the shared information between predicted and annotated domains; Homogeneity [27], which assesses whether each domain contains only members of a single class; Completeness [27], which evaluates whether all members of a class are assigned to the same domain; and the V-Measure [27], which balances homogeneity and completeness. For both annotated and unannotated datasets, the framework evaluates clusters using label-independent metrics. Spatial compactness is assessed using Clustering Homogeneity Across Spots (CHAOS) [28], which measures spatial proximity within clusters; Percentage of Abnormal Spots (PAS) [29], which identifies deviations from spatial neighbors; and the Average Silhouette Width (ASW) [30], which evaluates within-cluster spatial compactness relative to between-cluster spatial separation. To assess transcriptomic coherence, MultimetricST computes the Davies–Bouldin Index (DBI) [31], which evaluates intra-cluster cohesion and inter-cluster separation in gene expression space, and the Silhouette Coefficient (SC) [30], which quantifies the trade-off between transcriptomic cohesion and separation across clusters. Given that spatial compactness metrics capture only spatial organization while ignoring gene expression, and that transcriptomic coherence metrics overlook spatial continuity, MultimetricST includes the Silhouette Spatial Coefficient (SSC) [11], a recently developed extension of the SC that jointly evaluates transcriptomic similarity and spatial continuity, providing a unified measure that captures both molecular and spatial structure in cluster assessment.

MultimetricST provides evaluation results through an interactive dashboard, enabling intuitive visualization and comparison of computed metrics alongside user-supplied clustering results. The dashboard allows method developers and end users to identify the relative strengths and limitations of different evaluation approaches.

To demonstrate the functionalities of MultimetricST and to highlight metric behavior, we adopted two strategies. The first uses synthetic ST datasets in which key properties such as the cluster geometry, the number of clusters and the degree of fragmentation can be explicitly controlled enabling an empirical and verifiable assessment. The second assesses clustering results across real datasets derived from multiple ST technologies, including 10x Visium [32], Visium HD [32], Xenium [32], CosMx [33], STARmap [34], Stereo-seq [35], and BaristaSeq [36], spanning a range of spatial resolutions. These real ST datasets were analyzed using eleven state-of-the-art deep learning methods for spatial domain identification, GIST, GraphST, conST, CCST, SCAN-IT, SpaceFlow, SpaGCN, DeepST, STAGATE, HERGAST, and SEDR, integrated into the framework. These strategies allowed us to show that each evaluation metric captures distinct aspects of spatial domain identification within the complex context of ST data.

Finally, by combining visualization and comparison capabilities, we show that MultimetricST supports informed model selection and systematic assessment of method generalizability across diverse datasets and metrics, establishing it as a versatile and powerful tool for the ST community.

Methods

Framework acrhitecture

MultimetricST is a Python-based framework designed for the evaluation of spatial domain identification methods. The framework enables consistent and reproducible method assessment by providing an integrated environment for repository cloning or method installation, method execution, ST data preprocessing, and comprehensive clustering evaluation. The repositories of methods evaluated in this study, and described in section ‘Spatial domain identification methods’, are integrated into the framework, allowing users to easily clone, execute, and compare them.

The framework is organized into three main components: the method execution module, the cluster label evaluation module, and the dashboard module. Depending on a user-specified parameter, these modules can be run independently or sequentially as part of a complete pipeline through a main Python script, named MultimetricST.py, executable from the command line. The execution parameter supports three modes: (i) running the full pipeline by invoking clustering.py and evaluate.py and visualizing the results with dashboard.py, (ii) executing only evaluate.py and visualizing the results with dashboard.py, and (iii) visualizing the results from a user-provided CSV file containing precomputed cluster evaluation scores with dashboard.py.

Method execution module

To use the method execution module and obtain cluster labels, users can run the clustering.py script from the command line. Each method is executed in a consistent environment, with runtime and memory usage automatically monitored and recorded. This module enables users to clone GitHub repositories of spatial domain identification methods and execute them within a version-controlled Conda environment. New Python-based methods can be added by specifying the URL of the GitHub repository and adapting the corresponding run function, which is method-specific and follows the original execution routine provided by the method’s authors. Each method takes as input a ST AnnData object [37] and outputs the same object with cluster labels and a Python dictionary of additional information, including runtime and memory usage.

Cluster label evaluation module

Users can evaluate clustering results by executing the evaluate.py module, which takes as input a ST AnnData object, the corresponding cluster labels generated by the method execution module, and ground-truth annotations when available. The module also supports the evaluation of precomputed cluster labels provided by the user, along with raw expression and spatial matrices or an AnnData object containing ground-truth labels, if available. The evaluate.py script computes 12 metrics described in the ‘Evaluation metrics’ section: ARI, AMI, Purity, Homogeneity, Completeness, V-Measure, CHAOS, PAS, ASW, DBI, SC, SSC, and also reports the Average Dispersion Index (ADI). For the systematic evaluation of clustering performance using DBI, SC, and SSC, since it is recommended to preprocess transcriptomic data, the script first normalizes, log-transforms, and scales the data, selects the most highly variable genes (default: 3000 genes), and performs dimensionality reduction via principal component analysis (PCA; default: 20 components) on the transcriptomic data. The output of this module is a Python dictionary containing the score of each metric for every evaluated method, which is subsequently saved as a CSV file directly available to the user and passed to the dashboard module.

Dashboard module

The module dashboard.py enables visualization of method evaluation results from a user-provided CSV file containing precomputed cluster evaluation scores through an interactive dashboard implemented with Panel [38] and Plotly [39]. When available, the user can provide an Anndata object or a spatial matrix along with the corresponding cluster labels and tissue image to display the clusters on the spatial grid. The dashboard enables users to select specific methods for visualization. For the selected methods and group of metrics, it provides visualizations of metric behavior. In a radar chart, metric behavior is summarized by metric category. Specifically, annotation-dependent metrics display ARI, AMI, Completeness, Homogeneity, V-Measure, and Purity scores (Best = Max). Annotation-independent metrics for transcriptomic coherence and spatial compactness include SC, SSC, and ASW scores (Best = Max), while their counterparts comprise Davies–Bouldin and CHAOS metrics (Best = Min). Annotation-independent metrics for spatial compactness and fragmentation present ADI and PAS scores (Best = Min). Trade-offs between metrics are illustrated with a scatter plot, and spatial cluster distributions are visualized directly on the tissue grid.

In addition, the dashboard also provides a dynamic interpretation table that links each metric to its biological relevance and highlights top-performing methods, enabling a deeper contextual understanding of clustering quality (Table 4). A summary table presents detailed metric scores, and a bar chart compares computational efficiency across methods in terms of execution time and memory usage.

Table 4.

Biological relevance and interpretation of evaluation metrics with their top two performing methods on the Dorsolateral Prefrontal Cortex (DLPFC) 151673 dataset

Metric category	Top two performing methods	Biological relevance/interpretation
ARI	GIST, GraphST	Captures faithful domain recovery by measuring agreement between predicted clusters and known tissue domains.
AMI	GIST, GraphST	Captures the amount of shared information between predicted and reference tissue domains.
Hom	GIST, GraphST	Relevant for detecting overmixing and consistent reference tissue domain labeling.
Com	SEDR, GIST	Prevents splitting of the same tissue domain.
V-M	GIST, GraphST	Balances avoiding overmixing and undersplitting of tissue domains.
Pur	GraphST, GIST	Reflects the extent to which each predicted cluster contains spots from only one reference tissue domain.
CHAOS	GIST, SCAN-IT	Captures gradual tissue transitions and anatomical boundaries of smooth, contiguous spatial domains.
PAS	CCST, GIST	Captures interactions between adjacent cell populations.
ASW	CCST, GraphST	Beneficial for tissue domains with topologically distinct regions that optimize spatial local density.
DBI	SpaGCN, SEDR	Relevant for molecularly distinct regions, capturing their low intra-cluster variance and high inter-cluster variance.
SC	SpaGCN, STAGATE	Valuable for datasets where transcriptomic distinction dominates.
SSC	SpaGCN, GIST	Relevant for tissues where local microarchitecture matters by balancing spatial adjacency with expression similarity.

Evaluation metrics

Evaluation metrics for spatial domain identification methods can be broadly categorized based on whether they require ground-truth labels, as outlined in Table 1. Metrics that do not require annotated labels mostly focus on the intrinsic structure of the data, emphasizing compactness, or how closely related the spots within a cluster are, and separation, or how distinct or well-separated each cluster is from the others. MultimetricST includes both annotation-dependent and annotation-independent metrics.

Table 1.

Overview of evaluation metrics, including the value range, optimum value, annotation flag, and evaluation purpose

Metric	Values	Optimum	Annotation	Evaluation
Contingency matrix
ARI	[−1, 1]	Max	Yes	Accuracy of domain to annotated class
Purity (Pur)	[0, 1]	Max	Yes	Accuracy of assigning labels to a single class
Information theory
AMI	[0, 1]	Max	Yes	Mutual information of annotated class and domain
Homogeneity (Hom)	[0, 1]	Max	Yes	Each domain contains only members of a single class
Completeness (Com)	[0, 1]	Max	Yes	All members of a class are assigned to the same domain
V-Measure (V-M)	[0, 1]	Max	Yes	Harmonic mean of homogeneity and completeness
Spatial compactness
CHAOS	[0, 1]	Min	No	Spatial proximity of clusters
PAS	[0, 1]	Min	No	Label deviation from its spatial neighbors
ASW	[−1, 1]	Max	No	Cohesion versus separation of spatial neighbors
Transcriptomic coherence
DBI	[0, )	Min	No	Intra-cluster and inter-cluster variance in the expression space
SC	[−1, 1]	Max	No	Cohesion versus separation in the expression space
Joint spatial compactness and transcriptomic coherence
SSC	[−1, 1]	Max	No	Combines spatial and gene expression coherence

In the following formulations, we use and to denote individual spots and to indicate the total number of spots. The ground-truth classes are represented by the set , and the predicted cluster labels are represented by the set , where is the total number of predicted clusters. is the set of spots belonging to the ground-truth class , and is the set of spots assigned to the predicted cluster .

Annotation-dependent metrics

When annotations are available, contingency matrix-based metrics are typically used to directly compare predicted clustering results with true labels.

The ARI [24] assesses pairwise agreements between predicted and true labels by evaluating how often pairs of spots are assigned consistently, with values ranging from −1 to 1. The ARI is calculated as:

where RI is the Rand Index and is its expected value under randomness. is the number of possible pairs of spots, is the number of pairs that are in the same cluster in both ground-truth class and predicted cluster labels, and is the number of pairs that are in different clusters in both ground-truth class and predicted cluster labels.

Purity (Pur) [25] evaluates the extent to which clusters contain a single class in the annotation. For each cluster, it counts the number of spots that match the majority ground-truth class, sums over all clusters, and divides by the total number of spots:

where represents the number of spots in the predicted cluster that also belong to ground-truth class , is the size of the majority ground-truth class within cluster .

Other metrics rely on information-theoretic approaches to evaluate the uncertainty or disorder in clustering assignments.

The AMI [26] adjusts for chance in shared information, that is, how much knowing the predicted cluster reduces uncertainty about the ground-truth class, and is calculated as:

where is the mutual information, and and are the entropies of the labels, with . The number of spots in both sets and is represented by , and .

Homogeneity (Hom) [27] assesses whether each cluster contains only members of a single class, making it useful for identifying misclustered spots within clusters. The Homogeneity is calculated as:

where is the entropy of ground-truth classes and is the conditional entropy of the classes given the clusters.

Completeness (Com) [27] evaluates whether all spots of a given class are assigned to the same cluster, helping to detect when a class is fragmented across multiple clusters. The Completeness is calculated as:

where is the entropy of clusters and is the conditional entropy of the clusters given the classes.

V-Measure (V-M) [27] combines homogeneity and completeness into a single harmonic mean score, providing a balanced assessment of the contribution of the two metrics, as indicated in the formula:

where is the Homogeneity and is the Completeness.

All the presented metrics, except for ARI, have values ranging from 0 to 1. Higher values in ARI, AMI, Pur, Hom, Com, and V-M indicate better spatial domain identification performance.

Annotation-independent metrics

Among annotation-independent metrics, some quantify the spatial continuity of predicted clusters by leveraging the underlying spatial coordinates of the data, whereas others measure the transcriptomic coherence of predicted spatial domains using transcriptomic data.

CHAOS [22, 28, 40] measures the spatial compactness of clusters by calculating the average shortest path distance between all pairs of spots within the same cluster on a spatial graph. Let indicate the shortest path distance between spots and :

CHAOS values range from 0 to 1, where lower values indicate more spatially compact clusters. To enhance score interpretability and ranking, we scaled the CHAOS values by a factor of 100, resulting in a range between 0 and 100.

PAS [22, 29] quantifies the proportion of spots whose cluster label disagrees with the majority label of their spatial neighbors. PAS values range from 0 to 1, and lower values reflect higher local label consistency. Given the neighborhood of spot , its assigned label such that , and the majority label within its neighborhood :

ASW [22, 30] evaluates cluster cohesion and separation by comparing intra-cluster distances with nearest-cluster distances for each spot :

where spot is the pairwise Euclidean distance between spot i and spot j, is the set of spots assigned to cluster , and . ASW values range from −1 to 1, with values near 1 indicating well-separated, compact clusters, and values near or below 0 suggesting highly fragmented clustering.

The DBI [31] evaluates the average transcriptomic coherence between each cluster and its most similar counterpart, with values ranging from 0 to infinity. Lower DBI values suggest clearer separation between clusters:

where , is the intra-cluster scatter of spots assigned to cluster , , and are cluster centroids, is the distance of spot from its cluster centroid.

The SC [30] measures the gene expression coherence of a spot to its own cluster relative to other clusters, with values ranging from −1 to 1. A value close to 1 indicates that the gene expression profile of the spot is much more similar to its own cluster than to any other, suggesting a well-assigned spot. A value near 0 implies that the spot lies near the boundary between two clusters, while a negative value suggests potential misclustering, as the spot’s expression is more similar to a different cluster than its assigned one. The transcriptomic coherence computed using the SC for each spot is defined as:

where is the intra-cluster cosine distance and is the nearest-cluster cosine distance.

To jointly assess spatial continuity and transcriptomic consistency, the SSC [11] extends the traditional SC by incorporating a spatial penalty that accounts for the contiguity of cluster assignments. Given a spatial connectivity graph, it computes a cluster-level penalty score that reflects how spatially cohesive or fragmented the cluster is:

where is the size (number of spots) of the -th connected component. is the penalty score of the cluster . Higher penalties indicate greater spatial fragmentation as described in Supplementary section S1 and shown in Supplementary Fig. S1.

Given the penalty associated with the spatial fragmentation of cluster , the ADI is the mean penalty across all clusters:

Its values range from 0 to 1, where lower values indicate spatially contiguous clusters with low fragmentation, while higher values reflect spatially fragmented clusters. The ADI provides an auxiliary measure of spatial fragmentation, serving as an evaluator to quantify and compare the overall fragmentation level of spatial clusters.

Finally, the SSC measures both the transcriptomic coherence of a spot relative to its assigned cluster and its spatial proximity to neighboring spots within the same cluster, and is calculated as:

where the inflated intra-cluster distance is defined as . This increases relative to , penalizing clusters with high spatial fragmentation and revealing reduced spatial continuity. The SSC values range from −1 to 1, where a higher score indicates both strong transcriptomic consistency and coherent spatial organization within clusters.

Collectively, these metrics provide a comprehensive assessment of clustering quality in terms of accuracy, annotation fidelity, transcriptomic coherence, and spatial compactness.

Spatial domain identification methods

MultimetricST includes eleven state-of-the-art deep learning–based methods for spatial domain identification (Table 2). GIST [11] employs graph-based learning and contrastive optimization to enhance structural representations of ST data. It reconstructs gene expression from the reduced features of PCA [41], applies contrastive learning with adversarially sampled negatives, and enforces spatial coherence through graph structure learning. GraphST [12] adopts a three-step pipeline: it begins by perturbing gene expression data while preserving the graph structure, then encodes the data using a GNN, and finally fine-tunes the resulting embeddings using contrastive learning by comparing similar and dissimilar spot pairs. ConST [13] also leverages contrastive learning by integrating gene expression, spatial location, and tissue morphology. It processes morphological data using MAE, reduces dimensionality with PCA, and employs a two-stage training strategy that first reconstructs the input data and then maximizes the similarity between related spatial regions. CCST [14] integrates spatial and gene expression data into an adjacency and expression matrix, which are processed by a Deep Graph Infomax [42] network to learn joint embeddings. SCAN-IT [15] treats cells like image pixels, with gene expression as multichannel input. It constructs a spatial graph using an alpha complex and learns embeddings with Deep Graph Infomax. SpaceFlow [16] combines spatial and transcriptomic data using deep GNNs with spatial regularization to generate low-dimensional embeddings for clustering. SpaGCN [17] integrates gene expression, spatial coordinates, and histology into a unified graph representation. It uses graph convolutional layers, followed by a clustering algorithm, to delineate spatial domains. DeepST [18] enhances spatial domain detection by incorporating morphological features extracted using a pre-trained deep neural network. These features are combined with spatial and gene expression data and passed through autoencoders to denoise the signals and better capture spatial patterns. HERGAST [19] proposes a DIC framework specifically designed for ultra-large and ultra-high-resolution ST data. It splits large datasets into manageable patches for iterative training and constructs a heterogeneous relational graph that integrates both spatial proximity and gene expression similarity. STAGATE [20] constructs a spatial neighbor network and uses an attention-based GAE to adaptively learn spot similarities, thus improving spatial domain detection. SEDR [21] further improves spatial modeling by combining a variational GAE with a MAE to improve its understanding of spatial organization.

Table 2.

Overview of spatial domain identification methods evaluated in MultrimetricST, including the deep learning model, input type, and clustering method

Method	Neural network model	Input type	Clustering algorithm
CCST	Graph neural network (GNN) + DGI	Gene + Spatial	Leiden
conST	Contrastive GNN + masked autoencoder (MAE)	Gene + Spatial + Morphology	Leiden
DeepST	AE + Pretrained CNN	Gene + Spatial + Morphology	Leiden
GIST	Contrastive GNN + Structural Learning	Gene + Spatial	mclust
GraphST	GNN + Contrastive Learning	Gene + Spatial	mclust or Leiden
HERGAST	Divide–Iterate–Conquer (DIC) Relational graph autoencoder (GAE)	Gene + Spatial	Leiden
SCAN-IT	GNN + DGI	Image-style expression + Spatial	Leiden
SEDR	VAE + GAE + MAE	Gene + Spatial	mclust
SpaceFlow	GNN + Spatial Regularization	Gene + Spatial	Leiden
SpaGCN	GCN + Clustering	Gene + Spatial + Histology	Louvain + DEC
STAGATE	Attention-based GAE	Gene + Spatial	mclust

Results

We evaluated the utility and effectiveness of MultimetricST as a user-friendly framework for assessing ST clustering results using a comprehensive panel of evaluation metrics. Specifically, the framework was used to compute both annotation-based and annotation-independent metrics (Table 1) across synthetic datasets and real ST datasets (Table 3), using clustering outputs generated by integrated state-of-the-art methods (Table 2). Our analysis highlights the importance of evaluating clustering performance using complementary groups of metrics, and underscores the limitations of relying on a single measure. To this end, we structured the evaluation into two main components: (i) annotation-dependent metrics, and (ii) annotation-independent metrics, including measures that capture transcriptomic structure, spatial organization, or their combination.

Table 3.

Overview of ST technologies, including dataset name, annotation availability, number of spots, and spatial resolution

Technology	Dataset Name	Annotated	# Spots	Resolution (m)
10x Visium	Mouse Brain Anterior (MBA)	Yes	2695	55
	Dorsolateral Prefrontal Cortex (DLPFC) 151508	Yes	4381
	Dorsolateral Prefrontal Cortex (DLPFC) 151669	Yes	3658
	Dorsolateral Prefrontal Cortex (DLPFC) 151673	Yes	3638
	Dorsolateral Prefrontal Cortex (DLPFC) 151674	Yes	3670
STARmap	Mouse Visual Cortex (MVC)	Yes	1207	0.5–2
BaristaSeq	Mouse Primary Cortex (MPC)	Yes	4432	0.5–1
Stereo-seq	Axolotl Brain (AB)	Yes	8243	0.5
	Mouse Organogenesis Spatiotemporal Transcriptomic Atlas	Yes	8494
	(MOSTA) E10.5_E2S1
	Mouse Organogenesis Spatiotemporal Transcriptomic Atlas	Yes	30 124
	(MOSTA) E11.5_E1S1
Xenium	Xenium Mouse Partial Coronal (xMPC)	No	31 506	0.05–0.2
CosMx	Simulated Human Lung Cancer (sHLC)	Yes	87 589	0.1
Visium HD	Human Colorectal Cancer (HCC)	Yes	131 228	0.16
SRTsim	Synthentic1	Yes	2025	-
SRTsim	Synthentic2	Yes	1024	-
Total	15	–	1024–131 228	–

We compiled a collection of 13 ST datasets from multiple technologies, each providing different spatial resolutions and spot densities. These datasets include five 10x Visium [32] samples, such as the Dorsolateral Prefrontal Cortex (DLPFC 151508, 151669, 151673, 151674 datasets), and MBA; the Visium HD [32] Human Colorectal Cancer (HCC) dataset; the Xenium [32] Mouse Partial Coronal (xMPC) dataset; the STARmap [34] Mouse Visual Cortex (MVC) dataset; three Stereo-seq [35] datasets, including AB, and two time-point samples from the Mouse Organogenesis Spatiotemporal Transcriptomic Atlas (MOSTA E10.5_E2S1, E11.5_E1S1); the CosMx [33] simulated Human Lung Cancer (sHLC) dataset; and the BaristaSeq [36] MPC dataset.

In addition, we generated two synthetic ST datasets using the SRTsim R package [43]. The first dataset comprises spots arranged in nine different clustering configurations, with the number of clusters ranging from two to sixteen. The second dataset consists of spots evenly divided into two clusters.

For reproducibility, MultimetricST executed all integrated methods using the default parameters provided by their authors. We adopted a baseline hardware configuration based on CPU execution, with up to 200 GB of RAM. All computations were performed on an Ubuntu server equipped with 48 CPU cores. Under this configuration, the MultimetricST cluster label evaluation module showed that evaluation time (Supplementary Fig. S2) and memory usage (Supplementary Fig. S3) varied primarily across datasets as a function of dataset size (Supplementary Section S2 and Table 3). The computational time and memory usage of the method execution module depend on the architecture and implementation of each individual spatial domain identification method. The framework remains compatible with GPU-enabled environments, allowing users to leverage GPU acceleration when supported; however, the selection and management of CPU or GPU execution are handled internally by each method.

Assessing annotated datasets

MultimetricST incorporates several annotation-dependent metrics: ARI, AMI, Homogeneity (Hom), Purity (Pur), Completeness (Com), and V-Measure (V-M) (Table 1). To demonstrate how MultimetricST reveals the complementary strengths and weaknesses of each metric in evaluating clustering performance, we applied the framework to both sythentic datasets (Fig. 1) and real ST datasets including MBA (Fig. 2, Supplementary Fig. S4 and Supplementary Table S1), HCC (Supplementary Fig. S5), sHLC (Supplementary Fig. S6 and Supplementary Table S2), and DLPFC 151668 (Supplementary Fig. S7 and Supplementary Table S3).

Figure 1.

Annotation-dependent metrics behavior under different simulated (Synthetic1) clustering scenarios. Cluster configurations are considered relative to a ground-truth annotation with four clusters (A), over-segmentation into 32 clusters (B), moderate segmentation into 8 clusters (C), and under-segmentation into 2 clusters (D). The evaluated annotation-based metrics include ARI, AMI, Homogeneity (Hom), Completeness (Com), V-measure (V-M), and Purity (Pur).

Figure 2.

Annotation-dependent metrics (ARI, AMI, Pur, Hom, Com, V-M) behavior on the MBA dataset. Visualization of the reference annotation of the MBA and clustering results obtained by three state-of-the-art deep learning methods (CCST, SCAN-IT, and SEDR). In all cases, the expected outcome was set to 52 clusters.

Results on simulated datasets show that ARI is highly sensitive to cluster fragmentation and discrepancies between the number and distribution of predicted clusters relative to the ground truth. When the number of predicted clusters exceeds the number of annotated classes, the proportion of spot pairs belonging to the same ground-truth class (Fig. 1A) but assigned to different predicted clusters (Fig. 1B) increases substantially. Conversely, when fewer clusters are predicted than those present in the annotation, many spot pairs assigned to the same predicted cluster originate from different ground-truth classes (Fig. 1D). In both situations, these discordant spot pairs do not contribute positively to the RI (see the ‘Methods’ section), leading to strong penalization of both over-segmentation and under-segmentation (Supplementary Fig. S8). This behavior is consistently observed in real datasets. In the MBA dataset generated using 10x Visium technology, the reference annotation contains 52 clusters (Fig. 2A). Methods predicting a substantially different number of clusters yield lower ARI values. For example, CCST (Fig. 2B) identifies only eight clusters and consequently achieves an ARI of 0.24 relative to the 52 annotated regions. In addition, the MBA dataset presents a strong imbalance in cluster sizes, with annotation cluster 11 representing a large proportion of spots. Under these conditions, fragmentation of the dominant cluster produces a particularly strong ARI penalty because splitting a large cluster generates many discordant spot pairs. Although SCAN-IT (Fig. 2C) and SEDR (Fig. 2D) both predicted the same number of clusters as the reference annotation, SEDR achieved a higher ARI because it better preserved the integrity of large annotation regions such as cluster 11, whereas SCAN-IT fragmented this dominant structure. A complete overview of clustering results on the MBA dataset and the corresponding annotation-based scores is provided in the Supplementary Material. In contrast, AMI appears less sensitive to fragmentation than ARI [44] because it is based on mutual information rather than pairwise concordance. As observed in the synthetic datasets, increasing fragmentation (Fig. 1B and C) results in a less pronounced decrease in AMI compared to ARI. A similar trend is observed in the MBA dataset, where CCST, despite predicting only eight clusters, is less severely penalized by AMI than by ARI (Fig. 2).

Furthermore, when examining Completeness (Com), which measures the extent to which all elements of a given reference class are assigned to the same predicted cluster, we observe that it is strongly influenced by the number of predicted clusters. As shown in Fig. 1D, solutions with fewer predicted clusters, where individual clusters contain spots belonging to multiple annotated classes (Fig. 1A), tend to achieve higher Com scores. In contrast, when reference class become fragmented (Fig. 1B), the score decreases. This behavior is also evident in the MBA dataset (Fig. 2), where Completeness does not consistently reflect clustering quality. Higher Com values are frequently associated with methods that estimate substantially fewer clusters than those present in the reference annotation, highlighting the metric’s inherent bias toward cluster number relating to undersegmentation. For example, when many annotated regions are merged into a single predicted cluster, as observed with CCST (Fig. 2B), the Com score becomes artificially high because a number of reference classes are contained within a single cluster (Fig. 2).

In contrast, the Homogeneity (Hom) score, which measures the extent to which each predicted cluster contains elements from a single reference class, exhibits an opposite behavior. As shown in Fig. 1B, when the number of predicted clusters exceeds that of the reference annotation (over-segmentation), Hom can increase even when the clustering does not accurately reflect the ground truth (Fig. 1A). Conversely, when fewer clusters are predicted (Fig. 1D), Hom decreases, as multiple reference classes are merged within the same predicted cluster. Overall, Hom remains strongly influenced by the number of clusters, and increased cluster granularity can artificially inflate its value. This behavior is also observed in the MBA dataset (Fig. 2). Methods that predict substantially fewer clusters than expected, such as CCST (Fig. 2B), achieve low Hom scores. In contrast, methods with a number of predicted clusters closer to the reference annotation, such as SCAN-IT (Fig. 2C) and SEDR (Fig. 2D), show higher Hom values.

To overcome the biases of these two complementary measures (Homogeneity and Completeness), the V-Measure (V-M) computes their harmonic mean to balance the score, as shown in Fig. 1B and D. By combining both properties, V-M reduces the tendency of Hom to favor over-segmentation and Com to favor under-segmentation, providing a more balanced evaluation of clustering quality. On the MBA dataset, SEDR (Fig. 2D) stands out, achieving the highest V-M score, which reflects a well-balanced trade-off between homogeneity (Hom) and completeness (Com).

Moreover, by looking at the Pur score, we observe that the measure tends to favor clusters in which groups of spots from different predicted clusters correspond to a single ground-truth class (Fig. 1B). Lower Pur scores can occur when large regions are grouped into fewer clusters whose majority composition does not perfectly overlap with the corresponding class (see Figs 1D and 2B method CCST). In general, Pur assigns higher values when the majority of spots within some clusters match a dominant ground-truth class containing a large number of spots. Under this evaluation, a relatively high Pur score is observed for SEDR in Fig. 2D, where Pur reaches 0.62. Consistently, higher or competitive values across complementary metrics, such as V-measure (V-M), Purity (Pur), AMI, and ARI, suggest that a method achieves a balanced clustering outcome, capturing both cluster homogeneity and completeness while maintaining agreement with the reference annotation (Supplementary Table S1).

Additional evaluations using ARI, AMI, Pur, Hom, Com, and V-M on the HCC, sHLC, and DLPFC 151668 datasets are provided in the Supplementary Material (Supplementary section S3).

In summary, each metric captures unique aspects of clustering performance but has inherent limitations. However, when considered together, they provide a more comprehensive and reliable assessment, since they are grounded in reference annotations. At the same time, relying on annotations introduces a key limitation; in fact, these metrics assume that reference labels are accurate and faithfully represent the true tissue architecture. Moreover, in most practical applications, newly generated ST datasets lack high-quality reference annotations, making such metrics unusable. In these cases, MultimetricST computes alternative annotation-independent approaches and complementary metrics that are necessary to properly evaluate clustering quality.

Assessing nonannotated datasets

Annotation-independent metrics allow the evaluation of predicted spatial domains without relying on external reference labels. Within our framework, these metrics quantify cluster quality by exploiting the intrinsic structure of the data through graph-based representations, which capture the internal spatial and transcriptional relationships that underpin the clustering results. MultimetricST evaluates spatial compactness using CHAOS, PAS, and ASW, transcriptomic coherence using DBI and SC, and both aspects together through the SSC. We applied the framework to both synthetic datasets (Fig. 3) and a range of real ST datasets. Specifically, as described in Supplementary Material (Supplementary sections S4, S5, and S6), spatial compactness was assessed on DLPFC 151508 (Fig. 4, Supplementary Fig. S9, and Supplementary Table S4) and MBA (Supplementary Fig. S4 and Supplementary Table S5) Transcriptomic coherence was evaluated on MPC (Fig. 5, Supplementary Fig. S10, and Supplementary Table S6), MVC (Supplementary Fig. S11 and Supplementary Table S7), MOSTA E10.5_E2S1 (Supplementary Fig. S12 and Supplementary Table S8), and MOSTA E11.5_E1S1 (Supplementary Fig. S13 and Supplementary Table S9). Spatially aware transcriptomic coherence was examined using synthetic datasets (Fig. 6), DLPFC 151674 (Fig. 7, Supplementary Fig. S14, and Supplementary Table S10), and xMPC (Supplementary Fig. S15 and Supplementary Table S11) Finally, joint assessment of annotation-based and annotation-independent was conducted on AB (Fig. 8, Supplementary Fig. S16, and Supplementary Table S12) and DLPFC 151673 (Supplementary Fig. S17 and Table 4).

Figure 3.

Annotation-independent metrics behavior under different simulated (Synthetic1) clustering scenarios. Four structurally distinct, unannotated cluster configurations are considered: (A) four structurally regular clusters with cluster mixing, (B) four structurally regular clusters with cluster size imbalance, (C) twelve structurally regular clusters with heterogeneous boundaries, and (D) six structurally irregular clusters. The evaluated annotation-independent metrics for spatial compactness include CHAOS, PAS, and ASW.

Figure 4.

Annotation-independent spatial compactness metrics (CHAOS, PAS, and ASW) behavior on the Dorsolateral Prefrontal Cortex (DLPFC) 151508. Visualization of clustering results from two spatial domain identification methods (CCST and conST). The ground-truth annotation is shown alongside for comparison.

Figure 5.

Annotation-independent transcriptomics coherence metrics (DBI and SC) behavior on the MPC dataset. Visualizations of clustering results from two spatial domains identification methods (DeepST and SpaGCN). The ground-truth annotation is shown alongside for comparison.

Figure 6.

Annotation-independent metric combining transcriptomic similarity and spatial contiguity. The SSC is shown alongside the ADI and the SC under different simulated clustering scenarios. (A) Clusters without fragmentation; (B) induced fragmentation of the original clusters through random shuffling of spot spatial coordinates.

Figure 7.

Annotation-independent spatial compactness metrics Annotation-independent metric combining transcriptomic similarity and spatial contiguity, the SSC, shown alongside the ADI and the SC behavior on the Dorsolateral Prefrontal Cortex (DLPFC) 151674 dataset. Visualizations of clustering result from three spatial domain identification methods (GIST, SpaGCN, and STAGATE).

Figure 8.

Heatmap showing the evaluation of multiple methods on the AB dataset based on ARI, AMI, Hom, Com, V-M, Pur, CHAOS, PAS, ASW, DBI, SC, and SSC. Normalized scores for each metric are displayed, with the best value for each metric highlighted in darker color, and methods ordered according to the row mean score. Parallel dashed lines separate annotation-dependent metrics (ARI, AMI, Hom, Com, V-M, Pur), transcriptomic coherence metrics (DBI, SC), spatial compactness metrics (CHAOS, PAS, ASW), and the spatially aware transcriptomic coherence metric (SSC).

Evaluating spatial compactness

MultimetricST implements spatial compactness metrics to evaluate how well clusters align with the physical organization of spots within tissue. This class of metrics is grounded in the expectation that, in ST, spots belonging to the same cluster should be spatially proximate, reflecting underlying anatomical or functional domains.

In particular, CHAOS measures spatial discontinuity by computing the spatial distance between a spot and its nearest neighbor within the same cluster. However, a key limitation is that it relies on a k-nearest neighbor strategy with k = 1 to assess spatial continuity. As such, CHAOS is more effective at identifying isolated or scattered spots within clusters, as illustrated in Fig. 3A. Nevertheless, it remains unaware of the broader structural organization of the tissue. This limitation is evident in Fig. 3B–D, where structurally distinct spatial configurations receive identical CHAOS scores. Consequently, CHAOS is primarily sensitive to local spot density and fails to capture higher-order spatial arrangement within clusters.

PAS assesses the smoothness of tissue transitions and boundaries between adjacent domains by evaluating, for each spot, whether the majority of its spatial neighbors belong to the same cluster. However, as illustrated in Fig. 3C, PAS tends to penalize clusters with mixed or irregular boundaries, where local neighborhood agreement is reduced. At the same time, PAS assigns more favorable scores to clusters with higher local spot density, as shown in Fig. 3A, B, and D. In these cases, its majority-vote mechanism is more easily satisfied due to dense neighborhood agreement. Consequently, PAS is influenced by local neighborhood composition and favors compact, high-density regions while penalizing biologically plausible transitional boundaries.

ASW compares the average spatial distance between a spot and its neighbors within the same cluster to the distance from the nearest neighboring cluster, thereby evaluating both compactness and separation. However, ASW tends to favor fewer and more compact clusters, as illustrated in Fig. 3A and B. In contrast, it penalizes clusters with irregular or nonconvex shapes, as shown in Fig. 3C and D. Under such configurations, the average intra-cluster distance increases and separation from neighboring clusters may decrease, leading to lower ASW scores. Consequently, ASW may obscure fine-grained or biologically relevant spatial domains that do not conform to compact and well-separated geometric structures.

We applied these metrics using MultimetricST to the DLPFC 151508 dataset (Fig. 4). In this case study, CCST (Fig. 4B) achieved the lowest PAS and CHAOS scores and the highest ASW score, suggesting that it produces clusters that are spatially compact. However, these clusters are misaligned with true tissue boundaries, highlighting a mismatch between spatial compactness and biological accuracy. By contrast, conST (Fig. 4C) performed poorly across all three measures, with higher CHAOS and PAS values and lower ASW scores, reflecting clusters that are both fragmented and spatially dispersed.

In summary, CHAOS, PAS, and ASW provide an objective means of quantifying the spatial proximity and compactness of clusters. However, high scores in these metrics do not necessarily correspond to biologically meaningful predictions. For instance, assigning an identical label to all spots would produce maximal spatial continuity and compactness, despite failing to capture any true underlying tissue structure [22].

Evaluating transcriptomic coherence

Annotation-independent metrics for transcriptomic coherence are based on the principle that within anatomical regions or tissues, transcriptomic similarity tends to be high. Transitions in gene expression typically align with anatomical boundaries such that spatially adjacent spots with similar transcriptomes cluster together, while sharp changes in expression delineate anatomical or developmental borders, as observed during mouse embryo development [45]. The DBI quantifies transcriptomic dissimilarity between clusters. Because it relies on cluster centroids to approximate inter-cluster distances, DBI may not accurately capture true cluster boundaries in heterogeneous transcriptomic data. Additionally, DBI is sensitive to cluster size imbalance, as clusters with uneven sizes can distort the separation metric [46]. As shown in Supplementary Table S6, DeepST (Fig. 5B) attains the lowest (best) DBI score; however, this likely reflects DBI’s bias rather than a genuine improvement in clustering quality (Fig. 5).

The SC, being independent of cluster tissue spatial geometry when evaluating transcriptomic similarity, can be applied equally to highly irregular structures as well as to compact and regular ones. However, both DBI and SC implicitly assume that the underlying transcriptomic profiles form well-separated and coherent groups in feature space. In situations with limited gene coverage, particularly in rare disease conditions, gene sparsity may distort transcriptomic similarity patterns [44]. This characteristic helps describe why SpaGCN (Fig. 5C), despite producing spatially scattered clusters, achieves the highest SC score (Supplementary Table S6). Indeed, methods such as SpaGCN place more emphasis on transcriptomic signals [17], which explains their consistently strong performance in transcriptomic coherence metrics. These results highlight the limitation of SC in penalizing spatial fragmentation, as it does not account for the continuity of cluster neighborhoods.

Overall, though transcriptomic-based annotation-independent metrics effectively capture expression similarity, they do not incorporate spatial context, an omission that is particularly critical in ST, where spatial continuity and neighborhood relationships are central to assessing clustering quality.

Evaluating jointly transcriptomic similarity and spatial contiguity

We evaluated the consistency between spatial and transcriptomic clustering through the SSC. In particular, we applied MultimetricST to examine how variations in spatial cluster geometry, and especially different levels of cluster fragmentation, affect the penalties imposed on the SC. To this end, we designed two complementary experiments.

In the first experiment, we evaluated the SC and the SSC under two different configurations of the generated synthetic dataset. In the first configuration (Fig. 6A), we considered a scenario where the two clusters are well separated and not fragmented, resulting in a null fragmentation penalty and an ADI of zero. In this case, SC and SSC yielded identical values. In the second configuration (Fig. 6B), we randomly permuted the spot coordinates while keeping their cluster memberships fixed. As expected, this shuffling introduced substantial cluster fragmentation, leading to a higher fragmentation penalty in the SSC score. Importantly, SC remained unchanged at 0.75, as cluster assignments were preserved. However, the additional spatial penalty lowered the overall SSC: while in the nonpermuted configuration, SSC equaled SC (0.75), in the permuted configuration, SSC dropped to 0.57, with ADI equal to 0.74.

In the second experiment, we applied MultimetricST to evaluate how the SSC behaves across different computational tools when applied to the DLPFC Visium dataset 151674. Based on tissue morphology (Fig. 7A), we expected clusters to be stratified and not fragmented across the tissue. Consequently, a combination of a high SC and a low fragmentation penalty should reflect a good clustering outcome. In this experiment, we compared the performance of the methods (Table 2), each executed to produce seven distinct clusters, consistent with the reference annotation (Fig. 7). As shown in Supplementary Table S10, SSC is consistently lower than or equal to SC, since it incorporates a spatial penalty for cluster fragmentation. For some methods, such as GIST (Fig. 7b), the difference between SC and SSC is minimal, and ADI is close to zero, reflecting a low fragmentation penalty (Fig. 7). In contrast, some methods that achieve high SC values, most notably SpaGCN (Fig. 7C) and STAGATE (Fig. 7D), are strongly penalized due to the fragmentation level, resulting in very low SSC values and higher ADI. This analysis highlights that high SC values alone do not necessarily imply better clustering quality. By considering both SC and clustering fragmentation, SSC provides a more realistic measure of spatial and transcriptomic coherence, yielding results that, in this case study, better align with the reference tissue annotation. The effectiveness of SSC should always be interpreted in light of the expected level of fragmentation within the tissue. However, in a ST context, where spots with similar transcriptomic profiles are typically spatially close in the tissue [12], the SSC represents a highly valuable metric for evaluating clustering results in the absence of a ground-truth baseline.

Therefore, given the strengths and limitations of each metric discussed above, the MultimetricST joint evaluation provides a more nuanced perspective on clustering quality. For instance, CCST often excels based on the annotation-independent metrics for spatial compactness, achieving the better PAS, CHAOS, and ASW scores. Yet, its clusters are frequently misaligned with true tissue boundaries and perform poorly in annotation-based metrics, as also observed in the AB dataset (Fig. 8, Supplementary Fig. S16, and Supplementary Table S12). Conversely, methods such as SpaGCN often achieved strong performance based on annotation-independent metrics for transcriptomic coherence (SC and DBI). However, these gains are not always reflected in annotation-dependent metrics, highlighting discrepancies between transcriptomic similarity and biological accuracy in terms of tissue organization. These examples illustrate how relying on a single class of metrics may lead to misleading conclusions.

Our evaluation framework addresses this challenge by enabling the joint assessment of annotation-dependent and annotation-independent metrics. This integration allows for an unbiased and comprehensive assessment of clustering quality, uncovering biologically relevant subtleties in method behavior that would otherwise remain hidden (Table 4).

Taken together, these findings highlight the importance of considering each metric both individually, in terms of its evaluative focus, and collectively, as part of a broader assessment framework that properly evaluates the multifaceted nature of ST data.

Discussion

We introduced MultimetricST, a framework for the comprehensive evaluation of ST clustering results. The open-source tool, publicly available on GitHub, provides a comprehensive and systematic clustering assessment by applying multiple clustering methods on a ST dataset or starting from existing clustering outputs. The framework integrates a broad set of evaluation metrics, grouped into annotation-dependent and annotation-independent categories. In addition, it includes the SSC [11], a recently developed annotation-independent metric that jointly evaluates transcriptomic consistency and spatial contiguity, thereby enabling robust assessment in scenarios where ground-truth annotations are not available.

MultimetricST functionality and metric behavior were evaluated through two strategies. Firstly, through the generation and use of synthetic ST datasets with controlled characteristics—such as cluster shape, number, and fragmentation—allowing a systematic and verifiable analysis of metric responses under predefined conditions; and secondly, by using the framework to assess the clustering results of real ST datasets from multiple ST technologies, analyzed by 11 state-of-the-art deep learning based methods. Results demonstrate the importance of a multimetric evaluation strategy. Since each metric has specific limitations and applicability constraints, relying on multiple complementary criteria is essential for an accurate and balanced assessment of clustering performance.

Furthermore, a careful consideration of both the clustering strategy and the resulting number of clusters is necessary for a fair method comparison, as downstream clustering methods vary considerably among methods and directly influence both cluster quantity and quality. Some methods (STAGATE, SEDR, GraphST, GIST) employ Gaussian Mixture Models (via mclust), which are supervised clustering methods, while other methods use community detection algorithms such as Leiden (conST, DeepST, SpaceFlow, SCAN-IT, CCST, HERGAST) or Louvain combined with DEC (SpaGCN), which typically require resolution parameter tuning and may not recover the desired cluster count. Another important aspect to consider when evaluating the results of different tools is the intrinsic heterogeneity of ST technologies in terms of sequencing depth and spatial resolution. As extensively discussed in the literature [47], different platforms exhibit variable capture efficiency, gene detection sensitivity, and spot or pixel resolution, leading to substantial differences in data sparsity and library complexity. Consequently, assessment results across datasets should be interpreted with caution, as algorithmic performance is not only a function of the method design but also of the underlying data characteristics. This highlights the necessity of evaluation frameworks that account for both biological and technical variability, ensuring that comparisons between methods remain reproducible and informative across diverse experimental contexts.

Beyond accuracy, execution time and memory consumption are also key elements to take into account in the method selection process [48]. Therefore, the time and memory used are also reported by MultimetricST.

It is important to note that high-resolution data usually correspond to a high computational effort that may not necessarily be manageable by every method. For instance, since GNN-based methods typically require loading the entire graph into memory, such inefficiencies limit their practicality for large-scale datasets (hundreds of thousands to millions of spots) without substantial memory resources [19].

Moreover, in its current implementation, MultimetricST focuses on clustering results evaluated in transcriptomic and spatial spaces, which are consistently defined across methods and directly related to biological interpretation. In contrast, latent embeddings are inherently method-dependent, reflecting different learning objectives and geometric properties, making their direct comparison nontrivial. Evaluating the extent to which these representations preserve biologically meaningful transcriptional relationships is an important and emerging challenge in both single-cell and ST [49]. Addressing this will require the development of dedicated, method-aware evaluation strategies, and we identify this as an important avenue for future extensions of the framework.

Lastly, MultimetricST is fully reproducible and extensible, supporting both sequential and independent use of its method execution, cluster evaluation, and dashboard modules. Users can easily add new methods, which are seamlessly integrated and output cluster assignments. Cluster evaluation can be performed automatically across all methods, a single selected method, or directly on user-provided cluster labels and ST data. The interactive dashboard allows real-time selection and visualization of methods and metrics, facilitating the evaluation of emerging spatial domain identification approaches.

Supplementary data

Supplementary data is available at NAR Genomics & Bioinformatics online.

Funding

This work has been supported by Mission 4, Component 2, Investiment 1.3 - NEXTGENERATIONEU and the Italian Ministry of University and Research, National Recovery and Resilience Plan (PNRR), HEAL ITALIA (PE0000019) CUP: B33C22001030006.

Data availability

Spatial transcriptomics datasets used in this study are available at the following links: https://figshare.com/articles/dataset/Visium_DLPFC_preprocessed/22004273 for 10x Visium, https://figshare.com/articles/dataset/STARmap/28195031 for STARmap, https://spacetx.github.io/data.html for BaristaSeq, https://zenodo.org/records/11120307 for Xenium, https://zenodo.org/records/11402686 for Visium HD, https://drive.google.com/file/d/1oe_6i3kYlawwNeZktckHd2tF1rVMK5f4/view?usp=drive_link and https://github.com/GYQ-form/HERGAST/blob/main/analysis/SMI_Lung/HERGAST_pipeline.py for CosMx, https://db.cngb.org/stomics/mosta/download/ and https://doi.org/10.6084/m9.figshare.28195031 for StereoSeq. The 13 spatial transcriptomics datasets and the 2 synthentic datasets are available at https://zenodo.org/records/19153414.

Code availability

Our MultimetricST framework DOI: 10.5281/zenodo.20383825 is also available on github https://github.com/InfOmics/MultimetricST.