Spot-Based Global Registration for Sub-pixel Stitching of Single-Molecule Resolution Images for Tissue-Scale Spatial Transcriptomics

Abstract

Single-molecule imaging at the tissue scale has revolutionized our understanding of biology by providing unprecedented insight into the molecular expression of individual cells and their spatial organization within tissues. However, achieving precise image stitching at the single-molecule level remains a challenge, primarily due to heterogeneous background signals and dim labeling signals in single-molecule images. This paper introduces Spot-Based Global Registration (SBGR), a novel strategy that shifts the focus from raw images to identified molecular spots for high resolution image alignment. The use of spot-based data enables straightforward and robust evaluation of the credibility of estimated translations and stitching performance. The method outperforms existing image-based stitching methods, achieving sub-pixel accuracy (83 ± 36 nm) with exceptional consistency. Furthermore, SBGR incorporates a mechanism to surgically remove duplicate spots in overlapping regions, maximizing information recovery from duplicate measurements. In conclusion, SBGR emerges as a robust and accurate solution for stitching single-molecule resolution images in tissue-scale spatial transcriptomics, offering versatility and potential for high-resolution spatial analysis.

Graphical Abstract

Tissue-scale single-molecule imaging studies have provided unparalleled insight into biology. Single-molecule imaging enables molecular counting, offering a straightforward method to quantify molecular expression, and probes their accurate spatial location. Tissue-scale imaging provides the spatial context of molecules and cells within a tissue, critical information for understanding cell functions and interactions within the tissue context. For instance, single-molecule fluorescence in situ hybridization (smFISH) across the whole developing Drosophila embryo precisely characterized the stochasticity in RNA transcription and precise pattern of gene expression during development ¹. Recently developed signal encoding schemes have significantly increased the throughput of single-molecule imaging technologies by allowing the simultaneous profiling of a large number of targets and expanding the imaging area. With the combinatorial fluorescence signal encoding schemes, researchers can now profile hundreds to tens of thousands of targets at single-molecule resolution across mm to cm tissue sections ^2–9. The development of massively parallelized single-molecule imaging technologies has transformed the field of spatial biology and these technologies are actively being used to characterize the spatial architecture of cells within tissues ¹⁰, cell-cell interactions ^{11, 12}, disease biology ^{13, 14}, and many more.

Achieving sensitive detection in single-molecule imaging necessitates the use of high magnification objectives. To capture areas ranging from millimeters to centimeters with high magnification objectives, it is necessary to image a large number of field of view (FOV)s that overlap slightly at their edges. These overlapping portions are then stitched together by aligning and merging neighboring FOVs. The translation values between neighboring FOVs recorded by a microscope have significant errors due to mechanical limitations such as stage repeatability and actuator backlash ¹⁵, requiring additional corrective stitching methods. While various methods have been developed, image stitching with sub-pixel accuracy remains a challenge. Existing image stitching methods ^15–17 compute relative translations between adjacent FOVs by computing cross-correlations of pairwise images themselves, and these image-based stitching methods are used to register global spot locations in large-area single-molecule imaging experiments ^{5, 18, 19}. These image-based stitching methods effectively align morphological features between neighboring FOVs but are not designed to achieve pixel-level accuracy. Achieving this high level of precision is crucial to accurately registering images at the resolution required for single-molecule analysis.

Properties that contribute to stitching inaccuracy include weak signals from individual labeled molecules, heterogeneous background signals resulting from autofluorescence and non-specific binding of labeling probes, uneven illumination across the FOV, and varying degrees of photobleaching in the overlapping regions during the time-sequence composite imaging ²⁰. An additional complication arises in the effective merging of the overlapping regions. Image-based stitching methods utilize linear or nonlinear blending of pixel values to smoothly blend the morphological features in the overlapping regions ¹⁵. However, this approach is not applicable to single-molecule imaging experiments as their final data format is the position and identity of each identified spot. A simple solution is to remove cells that reside in the overlapping regions ²¹.

Here we present a spot-based stitching strategy called Spot-Based Global Registration (SBGR). This method utilizes the identified spot data for alignment instead of raw images. Aligning images using identified spot locations holds multiple benefits. First, spot locations are probed at sub-pixel resolution resulting in a highly accurate estimate of the relative translation between adjacent FOVs. Second, image alignment is solely based on identified spots and not on spurious background or noise, forcing alignment to consider only target objects. Third, the use of precisely defined spot locations for alignment offers straightforward and quantitative metrics for assessing stitching performance. Spot-based alignment allows direct interpretation of the distance between duplicate spots as a stitching error, whereas image-based alignment often requires manual selection of reference points on the images for quantitative evaluation of stitching performance. Lastly, this method enables the reliable detection and removal of duplicate spots. Duplicate spots identification is performed along each overlapping edge prior to stitching, eliminating the errors associated with aligning a large number of FOVs. In addition, the system takes spot identity into account when identifying duplicates, allowing adjacent spots with different identities to be distinguished.

For method development and benchmarking, we used five imaging-based spatial transcriptomics datasets generated using a custom MERFISH platform that probes the identity and location of hundreds to tens of thousands of RNA species at single-molecule resolution ². The datasets comprise spatially resolved transcriptome maps generated from three honeybee brain sections and two cultured human osteosarcoma U2-OS cell samples. The bee brain samples represent highly heterogeneous tissue data featuring both dense regions containing a large number of labeled targets and sparse regions with a limited number of detected spots. The additional feature is highly heterogeneous background signals in the tissues. Whole bee brain sections span 200-400 FOVs, presenting an additional challenge for stitching, as satisfying the estimated translation of all edges for such a large number of FOVs is not trivial. On the contrary, cultured U2OS cells tend to display a relatively uniform distribution across the entire image field, with high cell density and few sparse regions. In addition, cultured cells exhibit lower and less heterogeneous background signal levels compared to tissue samples.

The first step in SBGR is to generate binary spot images for overlapping regions of adjacent FOVs. Initially, overlapping regions are defined by the recorded position of the microscope stage. In spatial transcriptomics imaging experiments, multi-round raw fluorescence images are first transformed into spot images commonly involves high-pass filtering and deconvolution for background removal. Then, the on-off pattern observed in each pixel is then translated into a spot identity by referencing a predefined codebook ^{2, 3, 7, 9}. The final spot data consisting of the location and identity of the detected spots is then converted into binary spot images. SBGR is then processed through the following steps: computing the translation between adjacent FOVs for each edge, identifying duplicate spot pairs in each edge, quantitatively evaluating the confidence of the computed translations, updating unreliable translations, and stitching FOVs. These steps are repeated until the “distance error” stabilizes, defined by the average distance between the same spots in adjacent FOVs after stitching (Fig. 1).

Figure 1. Schematic of Spot-Based Global Registration.

For overlapping regions of adjacent FOVs, we computed translation using phase correlation. With calculated translation, we applied it and identified duplicate spots with Gaussian Mixture Model clustering. With duplicate spots identified, we evaluated the credibility of each calculated translation. Uncredible translations were updated and FOVs are stitched with minimum spanning tree algorithm with iteration. After FOVs are stitched, duplicate spots are removed surgically.

The first step is to compute the translation between adjacent FOVs by applying the Phase Correlation Method (PCM) ^{22, 23}. Performing PCM on spot images achieves sub-pixel accuracy. After that, the two images are temporarily combined to identify duplicate spots. In the combined images, duplicate spots are identified as spots having the same gene identity and are within a short distance. The distance threshold for detecting duplicate spots is determined for each dataset separately by calculating the distance of all same-identity spot pairs present in the overlapping regions and performing Gaussian Mixture Model (GMM) clustering. In all our test cases, we observed that each log-transformed distance histogram shows a clear bimodal distribution, with one peak for duplicate spots and the other for distinct spots (Fig. S1). The identified distance threshold for duplicate spots for our five MERFISH-based spatial transcriptomics data were 3.4, 3.0, 3.1, 3.6, and 3.3 pixels with the mean distance between duplicate spots being 0.8 pixels (80 nm).

Identifying duplicate spot pairs in each overlapping region serves three purposes. First, distance between the duplicate spots provides a simple metric for evaluating the stitching error after global registration. Second, their abundance in the overlapping region is positively correlated with the credibility of the estimated translation; the more duplicate spot pairs exist in the overlapping region bestows the higher credibility of the computed translation value. Evaluating the credibility of the calculated translation is particularly critical in tissue samples, which often have highly variable target density across the sample. In such cases, areas of low-density target molecules may not have enough labeled signals that can be used for alignment, yielding inaccurate prediction of translations. Finally, the list of duplicate spots for each edge is used to surgically consolidate duplicate spots while retaining those spots that are detected exclusively in one of the two overlapping FOVs. This method maximizes information recovery from overlapping images.

After duplicate spot pairs are identified, the credibility of the translation computed for each edge is assessed and any estimates with low confidence are updated. An anticipated cause of low-confidence estimation is the low density of spots in the overlapping area. To establish robust criteria for determining ‘reliable edges’, we conducted an analysis encompassing the distribution of computed translation values as well as the number of duplicate spot pairs, the proportion of duplicate spots, and the normalized cross-correlation (NCC) in the overlapping regions (Fig. 2). The number of duplicate spot pairs is a critical metric for assessing credibility, as they serve as reference points for image alignment. Similarly, the proportion of duplicate spots in combined images is pertinent to the credibility of the measurements. Ideally, all spots in the aligned overlapping regions should be duplicate, but photobleaching, measurement errors, and other factors cause some spots not to have their duplicate counterpart in the combined image. While it is natural for the ratio of duplicate spots to deviate from 1, an exceptionally low ratio might indicate low credibility of the estimated translation. For computed translation values, we anticipate a consistent discrepancy between the recorded stage position and the actual position, stemming from a probable systematic error in position recording coupled with random deviations. The distribution of computed translation values, depicted in Fig. 2a, demonstrates a prominent singular peak at 11.81 μm with a standard deviation of 0.42 μm (10 – 13μm), aligning with our expectations. Translation values that deviate significantly from the median are unlikely to be real considering the repeatability of the microscope stage and its controller and are likely to be unreliable estimation.

Figure 2. Evaluation metrics for the reliability of computed translations.

The distribution of estimated translation values shows a strong singular peak at 11.82 μm. The translation values including the outliers are 15.95± 15.47 μm (a). The number of duplicate spot pairs, ratio of duplicate spots, estimated translation, and NCC score of each overlapping region are analyzed to identify a metric to distinguish credible from non-credible edges. Outlier translation values are observed for edges with low number of duplicate spot pairs and low ratio (b). Non-credible edges are defined by the number of duplicate spot pairs ≤ 2 and their ratio ≤ 0.2 (c). The number of duplicate spots and NCC values for each edge colored by (d) credible edge classification and (e) translation values show that NCC does not discriminate between credible and non-credible edges.

Fig. 2b summarizes the translation values, the number of duplicate spot pairs, and ratio of the duplicate spots in each edge. As expected, edges having a low number of duplicate spot pairs and a low ratio of duplicate spots exhibit significant outliers in the computed translation values. We established a threshold for reliable translation estimation so that all edges satisfy the criteria to have estimated translations within the peak distribution. These criteria include a minimum count of duplicate spots exceeding 2 and a ratio of duplicate spots greater than 0.2. Fig. 2d presents the distribution of duplicate spot pair counts and ratio for credible versus non-credible edges. It highlights instances where non-credible estimations originate from edges featuring either a high duplicate spot ratio (i.e., 0.5) but an extremely low pair counts (i.e., 2) or conversely, a high count of duplicate spot pairs (i.e., 420) but a low ratio (i.e., 0.054). This observation underscores the importance of employing both metrics as criteria for credibility assessment.

Fig. 2d-e shows that NCC, a common metric for evaluating the similarity between feature-based images, fails to serve as a reliable metric for spot-based images. The NCC scores of the non-credible edges span the entire range from 0 to 1.0. Some of the edges display remarkably high NCC scores (0.7-1.0) while the estimated translation is > 30 μm, which is likely to be a false estimation. The inefficacy of NCC in distinguishing credible from non-credible edges stems from the sparse information inherent in spot-based images. Unlike feature-based images, spot-based images consist of sparse spots without continuous features across the image. In scenarios with a limited number of spots, inaccurately aligned images can yield high NCC scores. In an extreme case, an NCC score of 1 can be obtained with arbitrary translations if no duplicate pair exists within an overlapping region. Indeed, the edges characterized by extremely high NCC with unreasonably high translation values in our analysis had zero duplicate spots.

After identifying the credible edges, the translation value for each edge is finalized. The credible edges are assigned from the computed translations of the PCM. However, for unreliable edges, which often result from sparse spot density, it is crucial to update the unreliable translation values to prevent disrupting of the alignment of connected FOVs. To address this issue, the SBGR method replaces unreliable estimates with approximations derived from the translation pattern observed in reliable edges. Specifically, Fig. S2 shows that vertical edges of each row have similar translations, thus we replace unreliable estimates for vertical edges with the averaged translation value of reliable vertical edges in that row. The snake scan pattern explains the consistent translation between vertical edges in the same row. In contrast, translation values for horizontal edges lack consistent patterns. Therefore, we replace unreliable estimates for horizontal edges with an average translation of all reliable horizontal translations. This adaptive approach allows effective handling of images containing sparse regions, which are common in tissue samples.

To achieve high accuracy for global registration, the FOV stitching process employs the Minimum Spanning Tree (MST) algorithm ²⁴. In this method, each FOV is regarded as a node, and the weight of an edge is determined by the number of duplicate spots between them. The MST algorithm connects all nodes within a weighted graph without forming cycles, while aiming to maximize the total weight of the edges. In essence, edges are connected in descending order based on their weights, with credible edges being prioritized for stitching. The accuracy of the stitching is assessed using the distance error (D_err), which is calculated as the average distance between duplicate spot pairs in each credible edge. To examine the impact of stitching FOVs in different orders, we compared D_err after stitching each test sample in three different orders: descending order of credibility (MST), ascending order of credibility (inverse MST), and a random order (Fig. S3). While the MST method achieved the lowest average D_err (MST: 0.083 μm; Inverse MST: 0.187 μm; Random: 0.142 μm), the lowest standard deviation of D_err (std_D_err), and the lowest maximum D_err (max_D_err) which probes the registration error of the least accurately registered edge. This result highlights the significance of prioritizing credible edges based on the number of duplicate spots to consistently achieve minimal stitching errors across all edges.

The aforementioned steps can be repeated iteratively until D_err reaches a plateau. In all our samples, the initial iteration leads to a significant reduction in average D_err, from 11.81um to 0.08 um. However, subsequence iterations minimally affect the average D_err and the maximum value of D_err (Fig. S4). Thus, we have chosen to employ a single iteration for these MERFISH datasets. Notably, the attained D_err (83±36nm) is smaller than the reported technical error for MERFISH (100 nm), highlighting the accuracy of SBGR². Moreover, this value is smaller than the size in a single pixel of an image obtained using a 63X oil objective and sCMOS camera.

The final step of SBGR is consolidating duplicate spots within the overlapping regions. Following the final stitching, duplicate spots in the combined images are consolidated at their midpoint while preserving all other spots (Fig. 3). The small D_err obtained by SBGR ensures minimal error in pinpointing the accurate location of duplicate spots. The surgical removal of duplicate spots aims to maximize the retention of information gathered from duplicate measurements within the overlapping regions, ensuring that all spots detected in only one FOV are included in the ultimate image. Spots in FOV edges often exhibit dimmer fluorescence signals compared to those situated in the middle of the FOV due to imperfect flat fielding and photobleaching, resulting in reduced detection efficiency in these areas. Notably, the MERFISH technology utilizes multi-round measurements and a detection scheme capable of rectifying measurement errors², suggesting that singly detected spots are authentic and not measurement errors. This strategic edge merging technique significantly enhances detection efficiency at the FOV edges. In our analysis of honeybee section images, 32.5% of spots within the overlapping region appear in only one FOV, and our edge merging method successfully preserved all these spots.

Figure 3. Application of SBGR to spatial transcriptomics data from cultured cells.

(a) Detected spots in the overlapping region. Colors encode gene identity. The enlarged view of a small region is plotted with coloring of FOV identity (top, red: FOV 1; green: FOV2) and gene identity (bottom). To help visualize the translation between two FOVs, three pairs of regions with clear duplicate spots are shown in boxes: R1-R1’, R2-R2’, R3-R3’ (dashed: FOV1; solid: FOV2). (b) Post stitching images of the overlapping region. The duplicate regions overlap well. (c) After consolidation of the duplicate spots. The overlapping region is plotted with coloring of FOV identity (top: enlarged, left: full) and gene identity (bottom, right: full). The spots detected only in FOV1 and FOV2 are colored in red and green, respectively. The consolidated duplicate spots are colored blue. (d) Spot density plot before and after duplicate spot removal. The count density is calculated by counting the number of spots per 1.5μm x 1.5μm grid.

We compared the performance of SBGR with existing image stitching methods that are commonly used in the field, including those integrated into microscope software (e.g. Zen) and a freely available package (e.g. MIST) ¹⁵. The evaluation of these existing methods also involved the calculation of D_err, an average distance error between duplicate spots within credible overlapping regions. The most common approach is to apply the stitching software to raw DAPI images and use the calculated translations to register single molecule images (methods 1 and 2)^{5, 18, 19}. These methods leverage the clear staining patterns of nuclei to register images. Alternatively, the stitching software can be applied directly to single molecule images, which are inherently noisier and dimmer (method 3 and 4). To evaluate the stitching accuracy of different methods, we identified duplicate spots for all methods and calculated D_err of the credible edges. Fig. 4 presents these values for each method probed for tissue and cultured cell samples. For both Zen and MIST, DAPI-based registration yields significantly higher D_err (p-value < 0.001, Wilcoxon Rank-Sum test) than single molecule images. This result is likely due to the finer feature size of single molecule images. Importantly, our SBGR method significantly outperforms all other methods (p-value < 0.001, Wilcoxon Rank-Sum test), achieving subpixel accuracy (87.37 nm for tissues and 68.68 nm for cultured cells, total 83 nm). Additionally, our SBGR method consistently achieves the lowest standard deviation of D_err and max_D_err, emphasizing the consistency of registration accuracy across all edges. The identification of significantly higher outlier D_err values in tissue samples compared to cell samples in Zen and MIST suggests instability in the stitching process, particularly when dealing with sparse and heterogeneous data that SBGR addresses.

Figure 4. Quantitative evaluation of the performance of different stitching methods.

(a) D_err of three honeybee brain sections for each stitching method, (b) D_err of two cultured U2-OS cell samples for each stitching method. ***p < 0.001 as determined by Wilcoxon Rank-Sum test.

We extended our investigation to assess the stability and adaptability of our approach by systematically reducing the number of detected spots by randomly dropping them (Fig. S5). In the first simulation, spots were randomly omitted while maintaining the ratio of duplicate spots within the overlapping region (Fig. S5a). Remarkably, the D_err remained stable (143.12 nm) even with the omission of up to 95% of the total spots, underscoring the robustness of the proposed method in dealing with sparse datasets. In the second simulation, the maintenance of the duplicate spot ratio was not taken into account, resulting in an exponentially decreasing proportion of duplicate spots in the over-lapping regions by randomly dropping the spots (Fig. S5b). Even under these circumstances, D_err remained within a pixel until 70% of the total spots were removed. Thereafter, the ratio of credible overlapping regions dropped below 50%, as the ratio of duplicate spots dropped below 0.2. These results highlight the power of systematically identifying credible and non-credible edges, which could make our method highly stable. Specifically, we computed the exact translation based solely on credible edges and updating the approximate translation of non-credible edges with the values computed from credible edges results in a highly robust global registration result.

The newly developed SBGR introduces a new class of registration methods that align images based on identified spot patterns rather than the images themselves. This method is ideal for single-molecule resolution image datasets that require particularly high-resolution alignment, rather than lower resolution feature alignment, and strategic merging of overlapping regions for accurate molecular counting. To achieve exceptional alignment precision, SBGR systematically identifies duplicate spots, quantifies the credibility of estimated translations, and accounts for them during stitching. Application of SBGR to both tissue and cultured cell samples has resulted in robust registration of hundreds of FOVs, achieving subpixel registration accuracy (83±36nm). Notably, the obtained registration error is well below the reported technical errors for spatial transcriptomics measurements (100 nm)², emphasizing the minimal interference with single molecule localization. The systematic identification of duplicate spots also allows their surgical consolidation.

SBGR relies on identified spots for image alignment and therefore requires data with sufficiently high spot density for credible registration. Imaging-based spatial transcriptomics data, with their high spot density and large-scale images, are an ideal data type for SBGR. The exceptional accuracy of SBGR enables high- resolution spatial analysis, such as distance-based interactome profiling ¹¹, and its strategic edge merging approach ensures accurate molecular counting in cells located at FOV boundaries. Furthermore, SBGR presents additional advantages. Firstly, it does not require reference images for quantitative evaluation of performance, unlike image-based registration methods, which rely on comparing merged images to reference images acquired with shifted frames. SBGR evaluates performance simply by computing the distance between duplicate spots in each overlapping region. Eliminating the need for reference images significantly reduces imaging time, and processing reconstructed binary spot images instead of raw images saves computational resources. Secondly, SBGR compensates for low spot detection efficiencies near FOV edges due to uneven illumination by combining spot information from duplicate measurements and surgically removing duplicate spots. Additionally, SBGR allows stitching of non-rectangular shape images, making it adaptable to various tissue shapes.

The proposed method is aimed for single-molecule resolution imaging, requiring the capability to identify individual molecular spots and their locations accurately. Consequently, it may not be suitable for images that do not allow for the clear identification of individual spots such as low-resolution images, where intensity-based analysis prevails. Also, targeting highly packed molecules will limit the ability of detecting individual molecules because of optical crowd. Also, the detected spots data which is input of our methods need enough duplicate spots to get enough number of credible edges (>50%) to build the MST. Therefore, it is particularly well-suited for multi-omics data, which provides multiple measurements for accurate spot detection. For example, if we target only one gene which expressed sparse across the cell, the performance of SBGR could be limited by the low number of spots in the overlapping region.

SBGR offers a promising solution for precise global registration in single-molecule resolution images, catering to a wide range of applications, including various in situ hybridization and single-molecule resolution protein staining images. Its adaptability to diverse imaging types marks a notable step forward in achieving high-resolution alignment, potentially driving advances in multiple scientific disciplines that rely on accurate imaging techniques.

Data availability

The software codes to generate the results of this work are available at https://github.com/SeokJinYeo/SBGR.