Skip to main content
The Company of Biologists Open Access logoLink to The Company of Biologists Open Access
. 2026 Oct 14;139(21):jcs265152. doi: 10.1242/jcs.265152

NucleiSky enables cross-scale multimodal registration of microscopy data using nuclei constellations

Iván Hidalgo-Cenalmor 1,2,3, Adán Olguín-Olguín 1,2,3, Carolina Prieto 4, Johannes Kumra Ahnlide 5, Pontus Nordenfelt 5, Ricardo Henriques 4, Mario Del Rosario 2, Guillaume Jacquemet 1,2,3,6,✉
PMCID: PMC13741949  PMID: 42598871

ABSTRACT

Integrating tissue-level organisation with subcellular resolution and molecular information often requires combining multiple microscopy modalities and scales. However, aligning images acquired with different modalities, settings or instruments remains challenging. Here, we introduce NucleiSky, a microscopy image registration framework that utilises the spatial arrangement of nuclei or other landmarks as an intrinsic biological fingerprint. NucleiSky represents images as constellations of centroids and aligns them using geometric algorithms and spatial consensus scoring. In benchmark datasets, NucleiSky could localise query regions within larger reference images using as few as five nuclei. We show that NucleiSky can locate high-magnification fields of view within low-magnification overview scans, map these alignments to additional channels, support live brightfield-to-fixed registration using synthetic nuclear labels and guide microscope re-targeting. We further show that the same constellation-matching principle can be extended to 3D localisation and to non-nuclear landmarks. These findings establish local landmark geometry as an intrinsic spatial fingerprint that enables localisation and registration across imaging scales, modalities and microscopy platforms. NucleiSky is available as an open-source Python package and as notebook-based applications.

Keywords: Multimodal image registration, Cross-scale localisation, Nuclei constellations, Correlative microscopy, Smart microscopy


Summary: NucleiSky uses nuclei constellations as spatial fingerprints to localise microscopy images across scales, modalities and instruments, supporting automated field of view re-targeting.

INTRODUCTION

Biological imaging increasingly relies on integrating information across modalities and scales. In correlative and multimodal imaging workflows, a reference image is often acquired first, for example, from a tissue section, stitched mosaic or stack, and then used to guide more specialised measurements. These can include high-resolution imaging using confocal microscopy, super-resolution microscopy, electron microscopy or atomic force microscopy, as well as higher-content molecular readouts, including multiplexed protein imaging and spatial omics approaches (Enninful et al., 2026; Goltsev et al., 2018; Kim et al., 2023; Krentzel et al., 2025; Semba and Ishimoto, 2024). A related re-localisation problem arises when the same field of view must be revisited multiple times. For example, live-to-fixed imaging can link dynamic behaviours to endpoint molecular states (Almada et al., 2019), whereas iterative staining enables multiplexed imaging by repeatedly imaging the same field of view after sequential rounds of labelling (Lin et al., 2015; Radtke et al., 2020). Across these workflows, a central requirement is to align images acquired under different conditions, so that molecular, mechanical or ultrastructural measurements can be interpreted together.

Although the image-registration field is mature (Fernandez and Moisy, 2021; Guizar-Sicairos et al., 2008; Klein et al., 2010; Pylvänäinen et al., 2023; Saraiva et al., 2025; Thévenaz et al., 1998; Yaniv et al., 2018), aligning microscopy images across different scales and modalities without external fiducials can remain a significant challenge. Intensity-based registration frequently fails when appearances differ between modalities, such as haematoxylin and eosin (H&E) versus fluorescence, or electron microscopy versus light microscopy. Feature-based methods are vulnerable to staining differences, section distortions and the complexity of searching extensive tissue areas. Consequently, multimodal registration often relies on manual landmark selection, grid dishes, and repeated trial and error, which reduce throughput, reproducibility and the feasibility of automated microscopy workflows.

Cell organelles provide valuable opportunities for aligning microscopy images. For instance, mitochondria have proven useful for aligning high-resolution fluorescent images with focused ion beam–scanning electron microscopy (FIB-SEM) datasets (Krentzel et al., 2025). Interestingly, nuclei can serve as widely accessible intrinsic fiducials. They are common in biological samples, show distinct local arrangements, and can be easily stained with dyes or inferred via label transfer (Follain et al., 2026; von Chamier et al., 2021). For example, nuclei have been successfully used to align serial histopathology slices (Jeyasangar et al., 2024; Nasir et al., 2027). Additionally, nuclei are typically straightforward to segment using conventional image-processing techniques or deep-learning workflows (Hollandi et al., 2020; Pachitariu et al., 2025 preprint; Schmidt et al., 2018).

Inspired by astronomical methods that identify unknown images by matching star asterisms despite unknown translation, rotation and scale, we developed NucleiSky (https://github.com/CellMigrationLab/NucleiSky) to use nuclei as a modality-agnostic coordinate system for microscopy registration (Beroiz et al., 2020; Groth, 1986; Lang et al., 2010; Pál and Bakos, 2006). NucleiSky performs localisation and registration by matching constellations of nuclei (or other landmarks) between a query image, such as a high-resolution crop and a larger reference image, such as a low-resolution overview or tiled scan. Recognising that no single approach is likely to capture local landmark geometry across all experimental settings, NucleiSky incorporates multiple complementary constellation matchers that represent neighbourhood structures in distinct ways. These matchers propose candidate similarity transforms based on landmark positions or, when available, per-landmark features, and evaluate them using a spatial-consensus step. This design enables robust matching across sparse constellations, repetitive patterns and large search spaces, allowing for the localisation of queries with as few as five nuclei under favourable conditions. To accommodate diverse datasets without manual parameter tuning, NucleiSky also provides an adaptive matching strategy that prioritises rapid initial searches and increases computational effort only when necessary, thereby supporting practical cross-scale navigation across large microscopy datasets.

RESULTS

The NucleiSky pipeline

In astronomy, an unknown field of view can be mapped onto a star chart by matching the relative arrangement of stars. Analogously, nuclei can serve as biological landmarks for aligning microscopy images (Jeyasangar et al., 2024). Nuclei are prevalent across tissues and samples, can be visualised using various methods and are generally straightforward to segment. NucleiSky applies this constellation-based approach to locate a query field of view within a larger reference image and to estimate the similarity transform that aligns the two sets of nuclear points (Fig. 1).

Fig. 1.

Fig. 1.

Overview of the NucleiSky processing pipeline. (A) Overview of the NucleiSky workflow. (B) A query field of view (or crop) is localised within a larger reference image. The query can be acquired at a different scale and in a different modality from the reference. (C) Nuclei are segmented using user-supplied masks, a classical threshold-and-watershed pipeline or deep learning-based backends (such as Cellpose-SAM or InstanSeg). The segmentation is reduced to a constellation of nuclear centroids and optional per-nucleus descriptors that serve as a modality-agnostic coordinate system. Centroids provide geometry, whereas morphology and neighbourhood features help disambiguate repetitive nuclear patterns. (D) Matching engine. Multiple complementary matchers propose candidate similarity transformations based on local nuclear configurations. These matchers trade off robustness and speed across regimes of landmark density, segmentation noise and search-space size. In the schematics, p denotes landmark points and v denotes vectors. For the triangle matcher, v_b and v_h denote the normalised base-projection and signed-height components of the local triangle descriptor. For the graph matcher, d_i denotes the distance from a nucleus to its ith local neighbour and Δθ_i denotes the relative angular offset of that neighbour; distances are normalised by the median local neighbour distance (local_med). For geometric hashing, r_norm denotes the normalised distance of p3 in the local coordinate frame and θ denotes its angular coordinate. (E) Candidate transforms are then ranked by spatial consensus after mapping the query constellation into the reference frame. Matched nuclei form an inlier set within a fixed tolerance, enabling the selection and optional refinement of the best transform. NucleiSky returns the final alignment alongside the predicted bounding box for visual validation and downstream analysis. Each candidate registration is accompanied by quantitative confidence metrics, including the inlier fraction and localisation error. This approach enables users to distinguish reliable registrations from ambiguous matches.

The NucleiSky workflow (Fig. 1A) begins by assembling landmark constellations in both the query field and the larger reference image, either by detecting nuclei from existing label masks or by segmenting them using one of the built-in segmentation backends (Fig. 1B,C). As nuclear segmentation is a common image analysis task, many methods exist, including several deep learning models that perform well across diverse images. We implemented a classical threshold-and-watershed workflow in the NucleiSky segmentation backends, alongside popular deep learning models such as Cellpose-SAM (Pachitariu et al., 2025 preprint) and InstanSeg (Goldsborough et al., 2024 preprint). When segmentation occurs within NucleiSky and the nuclear size varies widely across inputs, images can optionally be rescaled before detection to normalise nuclear size, thereby enhancing robustness for scale-sensitive segmentation models (Ferreira et al., 2025 preprint). From the resulting masks, NucleiSky extracts nuclear centroids and, when possible, per-nucleus descriptors that summarise morphology and neighbourhood context (for example, size and shape descriptors and local neighbour statistics). These descriptors are then standardised and combined with rotation- and scale-normalised geometric encodings used by the matchers.

Rather than comparing image intensities directly, NucleiSky reduces each image to a geometric representation whose identity is preserved across imaging modalities. The query constellation must be aligned with the reference coordinate system despite unknown translations, rotations and scales. We implemented four matchers in NucleiSky. These matchers differ in how they generate candidate similarity transforms from local geometric evidence (Fig. 1D). The graph matcher fuses per-nucleus feature vectors with graph- and triangle-derived local geometry; the triangle matcher uses local triangles for matching; the quad matcher uses four-point configurations to produce rotation- and scale-invariant descriptors; and the geometric hashing matcher builds an index of local reference frames. Each matcher suggests plausible correspondences, converts them into transform hypotheses and then uses the same consensus strategy to evaluate candidates. After applying a candidate transform to the query centroids, NucleiSky identifies inliers as transformed nuclei that fall within a fixed tolerance of a reference nucleus. It then scores hypotheses based on spatial inlier support and reports the inlier fraction and mean localisation error (Fig. 1E). The best transform can optionally be refined via iterative updates. The final output includes both the estimated alignment and the predicted query footprint, represented as a bounding box in reference pixel coordinates, enabling quick verification and downstream analysis (Fig. 1E).

Benchmarking of the NucleiSky pipeline

With the NucleiSky pipeline in place, we next asked whether landmark constellations contain sufficient spatial information to localise a small query field of view within a larger reference image. To test this, we cropped query patches of increasing size from four datasets (all datasets in this paper are available from the accompanying Zenodo deposit, with the exception of datasets 3 and 9, doi:10.5281/zenodo.20721416) and used NucleiSky to recover the position of each patch in the original image (Fig. 2; also see Materials and Methods). Datasets 1 and 2 comprise tiled images of fixed DAPI-stained endothelial monolayers, with nuclei segmented either by classical intensity thresholding (Dataset 1; 4387 nuclei; 1266×2970 pixels) or by Cellpose-SAM (Dataset 2; 14,418 nuclei; 5520×5520 pixels). Dataset 3 was a substantially larger bright-field whole-slide image of an H&E-stained mouse colon section, segmented with InstanSeg (247,107 nuclei; 32,201×28,329 pixels; dataset: ‘Fresh Frozen Mouse Colon with Xenium Multimodal Cell Segmentation’; https://www.10xgenomics.com/datasets), providing a more challenging search space. Dataset 4 tested whether the same principle could extend beyond nuclei by using a tiled image of DAPI-stained E. coli containing 81,379 detected bacteria, segmented with Cellpose-SAM.

Fig. 2.

Fig. 2.

Benchmarking NucleiSky localisation performance. (A–D) Evaluation of the ability of NucleiSky to localise small query fields of view within larger reference images using landmark constellations. Query patches ranging from 50 to 512 pixels were cropped from each reference image and then re-localised with NucleiSky. For each dataset, the panels show, from left to right: the full reference image, a magnified region of interest, the corresponding segmentation mask, the localisation probability as a function of query patch size and the localisation probability as a function of the number of detected landmarks in the query field of view. For each dataset and patch size, n=50 unique query crops were evaluated per matcher; identical crops were used across matchers. (A) Dataset 1: tiled image of a fixed DAPI-stained endothelial monolayer, measuring 1266×2970 pixels and containing 4387 nuclei segmented using classical intensity thresholding. (B) Dataset 2: larger tiled image of a DAPI-stained endothelial monolayer, measuring 5520×5520 pixels and containing 14,418 nuclei segmented using the Cellpose-SAM backend. (C) Dataset 3: bright-field whole-slide image of an H&E-stained mouse colon section, measuring 32,201×28,329 pixels and containing 247,107 nuclei segmented using InstanSeg. (D) Dataset 4: tiled image of DAPI-stained E. coli, measuring 11,788×11,788 pixels and containing 81,379 detected objects segmented with Cellpose-SAM. In the localisation performance plots, points represent the observed localisation probability and shaded bands represent 95% Wilson confidence intervals. For these plots, successful localisation was defined as matcher-reported success, an inlier fraction≥0.80, SSIM≥0.80 and a translation error <20 pixels. Patch-size plots show success as a function of query field size in pixels, whereas landmark-count plots show success as a function of the number of detected landmarks in the query field of view. The landmark-count x-axis is shown on a logarithmic scale to highlight the low-landmark regime, where localisation limits are most apparent. Smooth curves indicate logistic fits in log10(landmark count) space and are shown only as visual summaries. (E) Segmentation-error robustness benchmark using Dataset 1. Nuclei centroid sets were systematically degraded by removing true nuclei to mimic missed detections or by adding synthetic centroids to mimic spurious detections. Perturbations were applied to both the query and reference segmentations. The benchmark was conducted on 512×512-pixel query patches using the quad, triangle and hashing matchers. Points indicate the observed probability of correct localisation and shaded bands indicate 95% Wilson confidence intervals. For this benchmark, success was defined solely by image-level agreement between the original query crop and the predicted reference crop (SSIM≥0.80).

Across datasets, localisation depended primarily on the uniqueness of the local landmark geometry rather than on image appearance or modality. Once a sufficient number of landmarks were present, however, localisation became highly reliable (Fig. 2). To compare matchers, we summarised performance using the fitted landmark threshold required to reach 90% localisation probability (Table 1). In Dataset 1, the quad matcher reached this threshold with as few as five nuclei, whereas the triangle, graph and hashing matchers required 16, 26 and 44 nuclei, respectively. Dataset 2 showed a similar pattern – quad reached 90% success with approximately five nuclei, whereas triangle and graph required 17.5 and 25.1 nuclei, respectively; in this dataset, hashing reached the same threshold with 11.4 nuclei. In the larger H&E whole-slide image, all matchers required larger constellations, consistent with the increased search space and more repetitive tissue architecture. Nevertheless, quad and triangle remained the most sample-efficient, achieving 90% success with 9.6 and 18.3 nuclei, respectively, whereas graph and hashing required ∼40 and ∼94 nuclei, respectively. In the E. coli dataset, quad again required the fewest landmarks (12.7 objects), followed by triangle (27 objects) and graph (38.8 objects), whereas hashing did not reach the 90% success threshold.

Table 1.

Estimated landmark thresholds for 90% localisation probability across matchers and benchmark datasets

Triangle Quad Graph Hashing
Dataset 1: tiled image of a fixed DAPI-stained endothelial monolayer 16 5 26 44
Dataset 2: larger tiled image of a DAPI-stained endothelial monolayer 17.5 5 25.1 11.4
Dataset 3: bright-field whole-slide image of an H&E-stained mouse colon section 18.3 9.6 40 94
Dataset 4: tiled image of DAPI-stained E. coli 27 12.7 38.8 n/a

For each matcher and dataset, the column reports the fitted number of landmarks required to achieve a 90% probability of localisation. Thresholds were estimated from logistic fits in log10(landmark count) space, as used for the landmark-count plots in Fig. 2. Hashing did not reach the 90% localisation threshold in Dataset 4 and is reported as n/a. The lowest threshold in each dataset is shown in bold.

We next assessed runtime under the same benchmark conditions. Localisation was generally fast on a standard laptop workstation, with search times below 1 s for Dataset 1 and ∼2 s for Dataset 2. By contrast, the much larger H&E reference required ∼20 s per query (Fig. S1A; also see Materials and Methods). Runtime also varied by matcher, with the triangle matcher generally providing the fastest searches (Fig. S1A). These results indicate that constellation-based localisation is computationally practical for tiled microscopy images and remains feasible even with whole-slide-scale references.

Because segmentation quality can vary, we next tested the sensitivity of NucleiSky to segmentation errors. Using Dataset 1, we selected 512×512-pixel query patches containing at least 50 nuclei and degraded both the query and reference centroid sets. To do so, we removed true nuclei and added synthetic centroids to mimic missed or spurious detections, respectively (Fig. 2E; also see Materials and Methods). We evaluated the quad, triangle and hashing matchers individually and defined success solely by image-level recovery of the correct field of view, measured by the structural similarity index measure (SSIM; Wang et al., 2004). Under this stringent condition, the quad matcher was most robust to missing nuclei, maintaining at least 90% success even with up to 60% landmark loss, compared with 20% for both the hashing and triangle matchers. Conversely, hashing was most tolerant of spurious detections, maintaining at least 90% success with up to 500% extra centroids, compared with 200% for quad and 10% for triangles.

Because benchmarking indicated that no single matcher was optimal across all datasets, we implemented an adaptive controller to reduce the need for manual matcher selection (Fig. 2). The adaptive mode selects and escalates matchers based on the number of query landmarks and the availability of feature vectors, prioritising matchers suited to small constellations and escalating when additional search effort is required (also see Materials and Methods).

Collectively, these results demonstrate that relatively small groups of nuclei contain sufficient geometric information to uniquely identify their spatial context within much larger images, even across datasets that differ substantially in imaging modality, landmark density and scale.

Localising high-magnification fluorescence fields of view within low-magnification tile scans

Having established the baseline localisation for NucleiSky performance under controlled benchmarking conditions, we next tested whether the same approach could support practical microscopy re-targeting scenarios. In these settings, query and reference images are often acquired under different conditions, including different magnifications and stage orientations. We therefore asked whether NucleiSky could localise high-magnification fluorescence fields of view within low-magnification overview tile scans. To do so, we used data previously prepared for a correlative light- and electron-microscopy workflow on confluent epithelial monolayers of DCIS.com cells (Grobe et al., 2026 preprint; Miller et al., 2000; Peuhu et al., 2022) (Fig. 3).

Fig. 3.

Fig. 3.

Cross-scale localisation of high-magnification acquisitions within overview tiles. (A) Dataset 5. A high-magnification query (maximum-intensity projection from a 63× stack) is localised within a lower-magnification 20× tile-scan reference acquired from a confluent DCIS.com epithelial monolayer. From left to right: the query and reference inputs, the predicted query footprint overlaid on the full reference and a magnified view of the matched region (reference in green, query in magenta). This highlights the consistent nuclear geometry despite differences in sampling and a slight relative rotation. The overlay is rendered in reference-pixel units, with the query downsampled to match the reference sampling. (B) Dataset 5, channel transfer. The saved transform from A is applied to additional query channels to position high-magnification molecular readouts within the reference context. From left to right: reference ROI DAPI, registered query DAPI, registered membrane channel, registered MYO10HMM–GFP signal (a truncated Myosin-X construct used as a filopodia-tip marker) and a merged view. The panel is displayed in reference-pixel units (query downsampled), with 20-pixel padding around the query footprint to visualise the continuity of the monolayer beyond the original crop. (C) Dataset 6. A higher-magnification 63× query is localised within a 10× tile-scan reference, demonstrating successful placement even when the reference contains less detail per unit area. As in A, panels show the inputs, the predicted footprint on the full reference and a magnified overlay of the matched region (reference in green, query in magenta). Here, the overlay is rendered in query-pixel units, with the reference crop upsampled to match the sampling of the query. (D) Dataset 6, channel transfer. The transform estimated in C is reused to register additional query channels onto the matched reference ROI. Images are displayed in query-pixel units, with the reference crop upsampled. All images were acquired on an Airyscan confocal microscope in standard super-resolution mode.

In the first example (Dataset 5), the query was a maximum-intensity projection from a high-magnification 3D stack acquired with a 63× objective (2372×2372 pixels; 0.056 μm/pixel). The reference was a single Z-plane from a 20× tile scan (6966×7143 pixels; 0.179 μm/pixel) (Fig. 3A,B). In the second example (Dataset 6), the query was captured at an even higher magnification from a 63× stack (2372×2372 pixels; 0.031 μm/pixel), whereas the reference was a lower-magnification 10× tile scan (6844×6904 pixels; 0.267 μm/pixel) that provided fewer structural details per unit area (Fig. 3C,D). In both datasets, nuclei were segmented using InstanSeg, yielding comparable centroid representations despite the drastic differences in sampling resolution.

Across these cross-scale comparisons, the quad and geometric hashing matchers produced accurate transformations, whereas the graph and triangle matchers failed to do so (using default settings). Notably, the query regions in both datasets showed a slight rotation relative to the reference, reflecting common experimental conditions. Once the transformation was estimated using the nuclear constellation, the same mapping was applied to other channels from the corresponding acquisitions. This enabled multimodal overlays, rendered either in the coordinate system and pixel size of the reference image (Fig. 3B) or in the coordinate system of the query image (Fig. 3D).

Collectively, these examples illustrate that NucleiSky can localise high-resolution fluorescence fields of view within low-resolution overview scans.

Live-to-fixed relocalisation using synthetic nuclear labels

Following a live cellular event and then imaging the same field of view using high-resolution immunofluorescence microscopy can be challenging. We next tested NucleiSky in a second relocalisation scenario, aligning live-cell imaging video with endpoint stainings obtained after fixation. To assess whether NucleiSky could support a live-to-fixed imaging workflow, we used bright-field movies of cancer cells attaching to endothelial cells under controlled flow conditions in microfluidic devices (Follain et al., 2026). In these experiments, live imaging was performed in bright-field to avoid fluorescence phototoxicity. After live acquisition, samples were fixed and stained using a cell painting kit, including a nuclear channel, and imaged as fluorescence tile scans (Fig. 4A).

Fig. 4.

Fig. 4.

Live-to-fixed registration using bright-field-derived nuclear constellations. (A) Input images for live-to-fixed registration. The live query was taken from the final frame of a transmitted-light bright-field movie acquired during cancer-cell perfusion over an HPMEC monolayer in a microfluidic channel. After live imaging, the same sample was fixed, stained using a Cell Painting protocol and re-imaged by spinning-disk confocal microscopy. The fixed reference was generated as a maximum-intensity projection. The query measured 2048×2044 pixels at 0.3245 μm per pixel and the reference measured 9421×9421 pixels at 0.3169 μm per pixel. (B) Synthetic nuclear landmark generation and NucleiSky matching. A pix2pix artificial-labelling model was used to infer a DAPI-like nuclear image from the live bright-field input. The pix2pix-derived query nuclei and Hoechst-positive reference nuclei were segmented using the Cellpose-SAM backend in NucleiSky. The query constellation was matched to the fixed reference constellation using the quad matcher. Yellow squares highlight regions of interest, which are magnified. The magenta square highlights the query image within the reference image. (C) Application of the live-to-fixed transform to endpoint fluorescence channels. The similarity transform estimated from the nuclear constellations was applied to fixed-sample fluorescence channels (as indicated) acquired after live imaging. The yellow square highlights a region of interest, which is magnified.

Because no fluorescent nuclear channel was acquired during live imaging, we generated synthetic nuclear landmarks from the bright-field movie using a pix2pix artificial-labelling model developed for FlowVision (Follain et al., 2026; Isola et al., 2018). The model inferred a DAPI-like nuclear image from the transmitted-light input, producing a synthetic nuclear channel. The pix2pix-derived nuclei image and the fixed Hoechst 34580 reference were both segmented with Cellpose-SAM to generate the query and reference constellations. NucleiSky then matched these two constellations and estimated a similarity transform that maps the live field of view into the fixed fluorescence coordinate system (Fig. 4B). In this example, the quad matcher successfully localised the live-imaged region within the fixed reference tile scan.

Finally, the same transform was applied to additional endpoint fluorescence channels acquired after fixation (Fig. 4C). This linked the live bright-field field of view to post-fixation Cell Painting stainings, including membrane, cytoskeletal and mitochondrial labels. Thus, using synthetic nuclear labels inferred from label-free live imaging, NucleiSky can determine where the live imaging was performed in the sample, recover the corresponding fixed-cell field of view across changes in both modality and microscope system, and transfer the alignment to multiplexed endpoint molecular readouts. This approach shows that NucleiSky can operate even when nuclei are never experimentally labelled during live imaging, provided that biologically meaningful landmarks can be computationally inferred. It provides a strategy to correlate dynamic behaviours observed during live imaging, such as cancer-cell arrest under flow, with cellular and molecular organisation acquired post-fixation.

Smart microscopy using nuclei constellations

After demonstrating that NucleiSky can localise microscopy fields of view post-acquisition, we next tested whether the same strategy could be applied directly on the microscope to guide acquisition. We integrated NucleiSky into a Nikon NIS-Elements JOBS workflow, in which NIS-Elements controls stage movement, autofocus and image acquisition, while NucleiSky performs target recognition based on nuclear constellations (Fig. 5; Movie 1; also see Materials and Methods).

Fig. 5.

Fig. 5.

Smart microscopy using nuclei constellations to re-target fields of view across microscopes. (A) Query generation on Microscope A. A field of view is acquired on the first microscope and the nuclei channel is selected from the image stack to generate the query image for downstream re-targeting. This query image contains the nuclear constellation that NucleiSky searches for on the second microscope. (B) Automated re-targeting workflow on Microscope B. The user defines a scanning area, loads the query image and moves the stage to the starting scan position. The microscope then enters a scanning loop, acquiring low-magnification overview images with a 4× objective. At each position, the image stack is converted into a maximum-intensity projection, denoised and passed to NucleiSky with the query image. If no acceptable match is found, the workflow advances to the next scan position. When NucleiSky identifies the query constellation, the microscope moves to the matched coordinates and acquires a higher-resolution image with a 20× objective. The final panels show successful re-localisation of the original field of view, with the matched region highlighted in the low-magnification overview and confirmed by the corresponding high-magnification acquisition.

The test workflow begins by acquiring a field of view containing nuclei on the first microscope to generate the query image (Fig. 5A). This query contains the nuclear constellation to be re-identified later. The sample is then transferred to a second microscope, where the user defines a scanning area and loads the query image into the JOBS workflow (Fig. 5B). The microscope enters a low-magnification scanning loop, in which, at each position, it autofocuses and acquires an overview image stack with a 4× objective, converts the nuclei channel into a maximum-intensity projection, denoises the image and passes the resulting reference image and the query image to NucleiSky. If no acceptable match is found, the stage advances to the next scanning position. When a match is detected, the loop stops, the stage moves to the predicted coordinates, performs a double-pass autofocus and acquires a higher-resolution multichannel image with a 20× objective.

As a proof of concept, we used this workflow to image an endothelial monolayer stained with a Cell Painting kit on two microscopes. A region of interest (ROI) was first acquired on a widefield microscope using a 20× objective. The same sample was then transferred to a spinning-disk confocal microscope, where the NucleiSky JOBS workflow performed low-magnification exploratory scanning and automatically identified the original nuclear constellation. The matched position was then re-imaged at higher resolution, confirming recovery of the original field of view on the second microscope (Fig. 5B; Movie 1).

This workflow is, of course, only an example and it can be fully customised. For instance, we use maximum-intensity projection here for simplicity and speed, but this may not be suitable in very crowded environments. These results demonstrate that NucleiSky can serve as a real-time re-targeting engine for smart microscopy. By using nuclear constellations as persistent spatial signatures, the workflow enables fields of view to be re-identified across instruments or imaging sessions without manual searching, gridded coordinates or external fiducials. This, in turn, provides a path towards closed-loop acquisition workflows in which overview imaging, target recognition and high-resolution re-imaging are linked automatically.

Extending nuclei-constellation matching to volumetric datasets

We next asked whether constellation-based localisation could be extended from 2D images to volumetric microscopy data (Fig. 6A). In NucleiSky3D, each volume is represented as a constellation of nuclear centroids in calibrated 3D physical coordinates. NucleiSky3D retains the core architecture of the 2D pipeline, in which local landmark constellations identify candidate transforms that are then evaluated via spatial consensus. For 3D point clouds, we implemented two complementary geometric matchers with natural 3D formulations (Fig. 6B). The pyramid matcher extends the local configuration strategies used by the 2D triangle and quad matchers to four-point tetrahedra formed from k-nearest-neighbour relationships, whereas 3D geometric hashing uses anchor-defined local coordinate frames to index neighbouring landmarks in 3D hash bins. These approaches were selected because four-point matching was most sample-efficient in 2D, whereas hashing was most tolerant of spurious detections. We did not prioritise a 3D graph matcher because, in the 2D benchmarks, the graph matcher was consistently less sample-efficient than the quad and triangle matchers, and extending it to 3D would require a new rotation-invariant descriptor. Compatible configurations generate candidate transforms, enabling the selected transform to localise the query subvolume within the reference volume (Fig. 6C). As in the 2D pipeline, NucleiSky3D includes an adaptive controller that selects the matcher order based on query size.

Fig. 6.

Fig. 6.

Extending nuclei-constellation matching to 3D stacks. 3D data input. A query volume is localised within a larger reference volume by reducing both image stacks to calibrated 3D nuclear-centroid constellations. The example panels show the reference and query images, with scale bars indicated. (B) 3D matching engine. NucleiSky3D implements two volumetric geometric matchers. The tetrahedron-based pyramid matcher constructs local four-point configurations from a 3D k-nearest-neighbour graph and encodes each non-degenerate tetrahedron using a normalised volume term and six sorted, normalised edge lengths. The 3D geometric hashing matcher defines a local coordinate frame from three anchor nuclei and stores the normalised position of a fourth nucleus in a 3D hash table. This enables the retrieval of compatible local configurations and the generation of candidate 3D similarity transforms. (C) Alignment between the query image and the reference image. An example of successful alignment between a random 3D crop from the Platynereis-Nuclei-CBG dataset and the full image. Successful alignment is demonstrated both with a single Z-plane and with a maximum-intensity projection. (D,E) Evaluation of NucleiSky3D's ability to localise query subvolumes within larger reference volumes using 3D nuclear constellations. For each dataset, query subvolumes of increasing size were cropped from the reference volume and re-localised with the pyramid, 3D geometric hashing and adaptive 3D matchers. The panels show a single Z-plane of the dataset, along with the corresponding segmentation examples and benchmark performance as functions of query subvolume size and the number of detected landmarks in the query volume. For each dataset and subvolume size, n=100 unique query subvolumes were evaluated per matcher; identical subvolumes were used across matchers. (D) Dataset 9: Platynereis-Nuclei-CBG reference volume, measuring 700×660×113 pixels (x, y, z) and containing 392 segmented nuclei. (E) Dataset 10: DAPI-stained mouse brain slice acquired as a spinning-disk confocal Z-stack, measuring 1840×1840×224 pixels (x, y, z) and containing 864 segmented nuclei. In the 3D localisation performance plots, points represent the observed localisation probability and shaded bands represent 95% Wilson confidence intervals. Successful localisation was defined as matcher-reported success, an inlier fraction ≥0.50 and a translation error <20 pixels. Patch-size plots show success as a function of query subvolume size in pixels, whereas landmark-count plots show success as a function of the number of detected nuclei in the query subvolume. The landmark-count x-axis is shown on a logarithmic scale to highlight the low-landmark regime, where localisation limits are most apparent. Smooth curves indicate logistic fits in log10(landmark count) space.

NucleiSky3D can use existing 3D label masks or generate landmarks from raw volumes using a 2.5D segmentation workflow. In this mode, individual planes are segmented with the same 2D backends described above and detections are linked across adjacent Z-slices using an intersection-over-union criterion to reconstruct 3D nuclear objects.

To benchmark NucleiSky3D, we used the same patch-localisation strategy as in 2D, extended to query volumes. We tested two datasets with different landmark densities and imaging characteristics. Dataset 9 was the publicly available Platynereis-Nuclei-CBG dataset (Lalit et al., 2022). The accompanying nuclear instance masks allowed us to evaluate matching independently of segmentation quality. This reference measured 700×660×113 pixels (x, y, z) and contained 392 segmented nuclei (Fig. 6D). Dataset 10 was a DAPI-stained mouse brain slice acquired as a spinning-disk confocal Z-stack, measuring 1840×1840×224 pixels and containing 864 segmented nuclei (Fig. 6E). For each dataset, query subvolumes of increasing size were cropped from the reference and re-localised using the pyramid, 3D geometric hashing and adaptive matchers.

As in the 2D benchmark, performance was primarily determined by the number of landmarks in the query. In the Platynereis dataset, 3D geometric hashing was the most sample-efficient, reaching 90% localisation probability with five nuclei. The pyramid matcher required larger constellations, reaching the same threshold at 34 nuclei (Fig. 6D). In the mouse brain dataset, 3D geometric hashing again performed best in the low-landmark regime, requiring 6 nuclei to reach 90% localisation probability, whereas pyramid matching required 33 nuclei (Fig. 6E). Thus, across both datasets, adaptive and 3D geometric hashing could localise sparse subvolumes from only a few nuclei, whereas pyramid matching became reliable only when larger 3D constellations were available.

Runtime remained compatible with practical volumetric localisation. In the Platynereis benchmark, median total query times were ∼0.12 s for adaptive mode, ∼0.12 s for pyramid matching and ∼1.19 s for 3D geometric hashing. In the mouse brain benchmark, adaptive searches ranged from ∼0.3 to ∼7 s across the tested patch sizes, whereas 3D geometric hashing was more stable, at ∼2.6–3.9 s per query.

Together, these results demonstrate that NucleiSky3D extends constellation-based localisation from 2D images to volumetric data. Although the tested 3D references remain modest compared with whole-organ or cleared-tissue volumes, they establish the feasibility of geometry-based 3D localisation and provide a foundation for scaling to larger volumetric datasets.

DISCUSSION

Here, we introduce NucleiSky, an open-source toolbox that uses landmark constellations as a coordinate system for microscopy image registration. By reducing images to calibrated landmark constellations, NucleiSky localises small query fields of view within much larger reference images, achieving successful localisation from as few as five nuclei under favourable conditions. We show that this strategy enables cross-scale registration of high-magnification fluorescence images within low-magnification overview scans, transfer of the resulting transforms to additional channels, live-to-fixed registration, automated field-of-view re-targeting on the microscope and also works with volumetric data. Together, these findings indicate that nuclei can serve not only as objects for segmentation and quantification but also as endogenous spatial landmarks for preserving image identity across scales, modalities, instruments and dimensions.

NucleiSky draws inspiration from blind astrometry, in which an unknown astronomical field is identified from local star geometry despite unknown translations, rotations and scales. This concept has a long history, from triangle-invariant matching of two-dimensional coordinate lists to Delaunay-based triangle spaces and astrometry.net, which uses hashing of four- and five-star asterisms, followed by statistical verification against a reference catalogue (Groth, 1986; Lang et al., 2010; Pál and Bakos, 2006). A related software tool to NucleiSky is astroalign (https://astroalign.quatrope.org/), which uses triangle invariants for direct star alignment (Beroiz et al., 2020). NucleiSky follows the same central idea – that local point geometry can identify a field of view, but applies it to a different scientific field. Unlike stars, nuclei can be variably segmented and can be locally repetitive.

Constellation-based point-cloud matching also has an important precedent in microscopy. Preibisch and colleagues have demonstrated that multiview selective plane illumination microscopy (SPIM) datasets can be registered by detecting fluorescent beads and matching their constellations using geometric descriptors, accelerated by geometric hashing and filtered by random sample consensus (RANSAC) (Preibisch et al., 2010). However, their task was to align multiple overlapping views of the same SPIM volume using externally introduced fiducials. NucleiSky addresses a different localisation problem – identifying where a small query field of view or subvolume belongs within a much larger reference image, even when acquired using different magnification and modalities. NucleiSky builds on the same broader principle that local landmark geometry can identify corresponding fields, but applies it to endogenous biological landmarks rather than fluorescent beads. Unlike beads, nuclei are dense and often locally repetitive. We therefore implemented multiple complementary matchers, spatial-consensus scoring and adaptive matcher selection in NucleiSky to support localisation across microscopy samples, scales and modalities.

Nuclei have previously been used as landmarks to support the alignment of serial tissue sections (Jeyasangar et al., 2024; Nasir et al., 2027). Typically, previous approaches use nuclei for local refinement after the corresponding tissue region has been approximately identified. For instance, a previous workflow converts detected nuclei into point sets and combines rigid alignment, non-rigid point-set registration and image warping to align serial tissue sections (Jeyasangar et al., 2024). Another whole-slide registration framework, CORE, uses nuclear centroids and shape information to register images after a separate coarse alignment stage (Nasir et al., 2027). These methods establish nuclei as useful registration landmarks, but they require many nuclei to work; for example, one nuclei-location workflow recommends tiles containing ∼100 nuclei (Jeyasangar et al., 2024). NucleiSky addresses a different problem. Rather than using nuclei only to refine already corresponding fields, it treats small groups of nuclei as searchable constellations to identify where a query field belongs within a larger reference. NucleiSky can therefore operate in a partial-overlap regime without requiring individual nuclei to be uniquely identifiable or the target region to be known in advance.

Retargeting the same field of view is a common challenge in microscopy. Existing strategies address this by preserving or externalising spatial identity. In live-to-fixed imaging, the sample is often kept on the microscope while treatment, fixation and staining are performed in situ using microfluidics (Almada et al., 2019). In such continuous workflows, re-localisation might not be required. However, microfluidics does not address experiments in which the sample is transferred between instruments. Cyclic imaging methods such as CycIF, CODEX and 4i similarly preserve correspondence by repeatedly imaging the same specimen through sequential staining or readout cycles (Goltsev et al., 2018; Gut et al., 2018; Lin et al., 2015). In correlative light and electron microscopy, retargeting often relies on gridded coverslips, laser-inscribed or micro-moulded coordinate systems or fiducial markers that remain visible across modalities and support high-precision correlation (Karreman et al., 2016; Kukulski et al., 2011; Schorb and Briggs, 2014). These strategies are effective because they make spatial identity recoverable by design, but they require that the coordinate frame, carrier or engineered landmarks remain valid. By using nuclear constellations as endogenous landmarks, NucleiSky provides another localisation strategy directly encoded in the sample itself. This sample-intrinsic approach reduces dependence on external fiducials, dedicated marker channels or carrier-based coordinate systems. Because the core NucleiSky matching functions can be called from Python, NucleiSky can also be incorporated into microscope-control environments that expose Python interfaces, providing a route towards automated field-of-view recognition and re-targeting.

The current implementation of NucleiSky has several limitations. It estimates global similarity transforms in calibrated physical coordinates and therefore assumes that the relative arrangement of landmarks is largely preserved across images. This makes it well-suited for localisation and retargeting. However, it does not support affine shear, anisotropic scaling or local non-rigid deformations introduced by moving cells, swelling or tissue distortion. NucleiSky also depends on the quality and uniqueness of the landmark constellation. Our benchmarking indicates that NucleiSky can tolerate missing nuclei and spurious detections to some extent, but these will reduce confidence or produce ambiguous matches. Finally, although NucleiSky has been extended to 3D and benchmarked on two volumetric datasets, this mode remains less extensively validated than the 2D workflow. The current 3D implementation estimates global similarity transforms, relies on 3D centroid constellations generated either from existing masks or 2.5D slice-linked segmentation, and has so far been tested on modest reference volumes rather than whole-organ or cleared-tissue datasets. Future work will focus on scalable OME-Zarr-based workflows, lazy loading and chunked export, parallel-assisted search and hybrid pipelines that combine constellation-based localisation with downstream deformable refinement.

Altogether, we envision constellation-based localisations as a general strategy for bridging microscopy modalities, enabling biological datasets acquired across instruments, resolutions and experimental conditions to be linked through intrinsic spatial organisation rather than shared image appearance.

MATERIALS AND METHODS

NucleiSky implementation

NucleiSky is implemented as a Python package, with separate 2D and 3D pipelines accessible via nucleisky2d and nucleisky3d (https://github.com/CellMigrationLab/NucleiSky). The repository includes reproducible Jupyter notebooks that demonstrate complete 2D and 3D registration workflows. The same functionality is available through a programmatic API suitable for batch processing and integration into acquisition or analysis workflows. NucleiSky depends on standard scientific Python libraries for array computing and data management, including NumPy (Harris et al., 2020) and pandas. It uses SciPy for spatial queries and numerical routines (Virtanen et al., 2020) and scikit-image for image processing utilities and resampling (van der Walt et al., 2014). Several matchers use NetworkX-based kNN graph representations (Hagberg et al., 2008) and Numba (Lam et al., 2015). Microscopy file I/O and export are handled by tifffile, while plotting utilities use matplotlib (Hunter, 2007). Optional dependencies include segmentation backends (Cellpose and InstanSeg), Zarr I/O for large volumetric datasets (Moore and Kunis, 2023) and SimpleITK to enhance 3D feature extraction (Yaniv et al., 2018).

Additionally, we provide Jupyter notebooks to run NucleiSky2D and NucleiSky3D, along with associated benchmarks. These notebooks can be run in Google Colab or installed locally using the NucleiSky app, which was created with LabConstrictor (Hidalgo-Cenalmor et al., 2026).

Image calibration, nuclei extraction and constellation features

NucleiSky performs matching in physical units to support cross-magnification registration and reduce sensitivity to pixel sampling. Users provide the pixel size (μm/px) for 2D images and the voxel size for 3D volumes, or NucleiSky retrieves them from metadata when available. Centroid coordinates follow (y, x) in 2D and (z, y, x) in 3D. The estimated similarity transforms map query coordinates to the reference coordinate system. When built-in segmentation backends are used, images can optionally be rescaled before segmentation to ensure nuclei have comparable apparent diameters in pixels. This rescaling stabilises segmentation, whereas matching and export operate on centroids expressed in micrometres.

Nuclei segmentation can be performed using a threshold-and-watershed method or via learning-based techniques such as Cellpose [with a pretrained configuration called CellPose-SAM (Pachitariu et al., 2025 preprint)] or InstanSeg (Goldsborough et al., 2024 preprint). Alternatively, users might provide precomputed 2D or 3D label masks. For volumetric data, NucleiSky also supports a 2.5D approach that segments each Z-slice independently and then stitches labels across slices using intersection-over-union matching to produce a 3D label volume.

From labelled masks, NucleiSky extracts nuclear centroids and per-nucleus descriptors for use with feature-driven matchers. These descriptors include region-based shape measures and local neighbourhood context, such as k-nearest-neighbour distances and density estimates in micrometre space. For 3D labels, similar centroid and neighbourhood features are extracted using SimpleITK when available or provided directly as centroid tables. In both cases, consistency with the provided voxel calibration is preserved.

Graph matcher

The graph matcher constructs a k-nearest-neighbours graph from nuclear centroids measured in micrometres and generates a rotation-resistant node descriptor that encodes locally normalised neighbour distances and relative angles. This descriptor, derived from the graph, can be combined with per-nucleus morphological and neighbourhood feature vectors obtained from the segmentation mask. Candidate matches between query and reference nuclei are identified via nearest-neighbour search in the combined feature space. Optionally, multiple candidates per nucleus might be retained to handle ambiguous matches. These candidates can be filtered using ratio-based ambiguity tests and robust feature-distance thresholds. The matcher then proposes similarity transforms by sampling correspondence triplets in a RANSAC-like process, discarding degenerate triplets with very small triangle areas and evaluating hypotheses based on spatial inlier consensus. The best hypothesis can be refined using the full set of inliers and further adjusted via iterative closest-point (ICP)-like updates to the similarity transform. To avoid overly strict matching in dense tissues, the graph matcher enforces a minimum inlier radius based on the typical nearest-neighbour spacing in the reference constellation.

Triangle matcher

The triangle matcher derives local geometric descriptors from triangles within a k-nearest-neighbour neighbourhood. For each nucleus, triangles are formed by anchoring on a local neighbour and combining it with additional neighbours. Near-degenerate configurations below a predefined area threshold are discarded. Triangle descriptors are aggregated per nucleus and normalised using statistics from the reference constellation. Candidate correspondences are found via a nearest-neighbour search in triangle-descriptor space, and similarity transforms are proposed via RANSAC-style sampling of correspondence triplets. Candidate hypotheses are filtered by configured scale and rotation limits and scored by spatial consensus among inliers. The best hypothesis can optionally be refined with ICP-like updates.

Quad matcher

The quad matcher analyses multiple local four-point configurations around each nucleus by selecting neighbour triplets from a k-nearest-neighbours set. Each configuration is encoded with a rotation- and scale-invariant descriptor derived from normalised distances and a deterministic neighbour ordering. Near-degenerate quads are discarded using a minimum-area criterion. Query quad descriptors are matched to reference quad descriptors via nearest-neighbour retrieval in descriptor space, yielding candidate configuration pairs. Candidate similarity transforms are derived from matched quads. Hypotheses from multiple three-point subsets of each quad might be generated to tolerate a single incorrect neighbour within a local configuration. These hypotheses are screened using scale and optional rotation constraints and scored based on inlier consensus. Early stopping can be applied once an inlier target is reached and the final transform can be refined using the inliers.

Geometric hashing matcher

The geometric hashing matcher accelerates localisation in large reference sets by indexing local coordinate frames in a hash table. In 2D, local frames are generated from short anchor pairs selected from k-nearest-neighbour lists. Additional neighbours are encoded in a scale-normalised polar representation relative to the anchor-defined frame. These signatures are quantised into radial and angular bins and stored in a hash map with per-bin limits to manage memory and runtime. During querying, similar local arrangements are sampled from the query set. Hash bins are searched using a configurable neighbour-bin radius to reduce quantisation sensitivity, and retrieved hash entries are ranked and tested to generate candidate transform hypotheses. These hypotheses are filtered by scale and optional rotation bounds and scored using spatial inlier consensus. An optional pre-test on a subset of nuclei allows early rejection of weak hypotheses. The selected transform can be refined through ICP-like updates.

Adaptive matching controller

To accommodate variations in nuclear density, segmentation quality and query size, NucleiSky provides an adaptive controller that runs a sequence of matchers and stops at the first successful localisation. The controller selects the order of matchers based on the number of query nuclei and skips matchers whose prerequisites are unmet. For example, the graph matcher is skipped when per-nucleus feature vectors are unavailable. If no matcher succeeds, the controller returns the highest-scoring candidate under a policy that prioritises transforms with high inlier fractions and low mean inlier errors, enabling best-effort localisation while preserving diagnostic information about failure modes.

NucleiSky3D

NucleiSky extends constellation matching to volumetric datasets by representing nuclei as 3D centroids in micrometre space and estimating a 3D similarity transform comprising a uniform scale, a 3D rotation and a translation. The 3D pipeline features two matchers, the pyramid matcher and 3D geometric hashing. The pyramid matcher constructs local tetrahedra from k-nearest-neighbour neighbourhoods and encodes each with a descriptor based on normalised edge-length patterns and a volume term normalised by the typical edge scale, providing invariance to rotation and uniform scaling while rejecting degenerate tetrahedra. Candidate tetrahedra are matched in descriptor space using mutual filtering, and 3D similarity transforms are estimated by sampling tetrahedron correspondences and scoring them via inlier consensus. The 3D geometric hashing matcher defines local 3D coordinate frames from three anchor points and encodes the relative coordinates of a fourth point within that frame, which are quantised into bins for hash-based retrieval. Candidate hypotheses are generated by querying the hash map with configurations, filtering by scale and optional rotation limits, and scoring them using inlier consensus, with optional pre-test pruning and iterative refinement. An adaptive 3D controller chooses between pyramid-first and 3D geometric hashing-first strategies based on the query constellation size, using a threshold of 1000 nuclei in the current implementation.

Outputs, export and reproducibility

For each successful run, NucleiSky provides estimated similarity transform parameters and match-quality metrics, including inlier statistics and localisation error estimates derived from nearest-neighbour distances in micrometre space. The pipeline can export aligned overlays and derived outputs to support validation and downstream analysis. For example, predicted query footprints in reference pixel coordinates and transform records suitable for reuse across sessions. When segmentation masks are provided, they can be saved alongside transforms and overlays to enhance reproducibility and facilitate auditing of segmentation-dependent effects.

Matcher benchmarking and performance analysis

We evaluated NucleiSky using a patch-localisation task in which square query fields of view were cropped from a larger reference image, ensuring known ground-truth coordinates for each query. For each reference image, landmark centroids were precomputed and stored in both pixel and micrometre units. Query landmarks were defined as those whose centroid coordinates fell within the crop. Their coordinates were then transformed into the local coordinate system for the query patch and converted to micrometres using the pixel size of the image, whereas reference landmarks remained in the coordinate system of the full reference image. Because the query and reference shared the same pixel calibration in this benchmark, correct localisation meant recovering the translation for the crop in the reference image.

For the localisation benchmark, square query patches of increasing sizes were tested. For each patch size, 50 candidate query locations were generated deterministically and reused across matchers. Patch origins were anchored to landmarks to ensure informative configurations – a landmark was sampled, a random position within the patch was selected, and the resulting top-left coordinate was clipped to the image bounds. Duplicate candidates were removed and, when enabled, candidates with fewer than three landmarks were discarded. The graph, quad, triangle, hashing and adaptive matchers were evaluated on the same query patches. For each patch, we recorded the estimated translation, matcher-reported success and match-quality statistics, including inlier fraction and mean inlier error. Translation accuracy was measured as the Euclidean distance between the estimated translation and the ground-truth crop origin, in pixels. When a translation was available, we also computed the structural similarity index measure (SSIM) between the query patch and the predicted reference crop after identical percentile-based intensity normalisation. Localisation was considered successful on the standard benchmark only when the matcher reported success and all predefined quality thresholds were met: inlier fraction≥0.80, SSIM≥0.80 and translation error <20 pixels. Success probabilities were summarised as a function of query patch size and as a function of the number of landmarks in the query field of view. Confidence intervals were calculated using the Wilson score interval. For nuclei-count summaries, observed success probabilities were binned by landmark count and plotted on a logarithmic x-axis; logistic fits in log10 (landmark count) space were used to estimate the number of landmarks required to reach 90% localisation probability.

To evaluate sensitivity to segmentation errors, we conducted a separate benchmark using 512-pixel query patches. Candidate patches were selected to contain at least 50 clean landmarks before perturbation. Adaptive mode was excluded from this benchmark because the goal was to compare the robustness of individual matchers under controlled landmark corruption. In addition, the adaptive controller uses internal success criteria that are not appropriate when false-positive or false-negative landmarks are deliberately introduced. We therefore evaluated the quad, triangle and hashing matchers individually. Two segmentation-error modes were tested independently. To mimic missed detections, a specified fraction of true landmarks was randomly removed from the set of centroids. To mimic spurious detections, synthetic centroids were added uniformly within the relevant image domain. Extra detections were expressed relative to the number of true detections after any removal; for example, 100% extra detections added one synthetic centroid per true centroid, whereas 1000% added ten synthetic centroids per true centroid. Missed-detection levels of 0, 10, 20, 40, 60, 80, 90 and 95% were tested, as were extra-detection levels of 0, 10, 25, 50, 100, 200, 500 and 1000%.

Segmentation perturbations were applied to both the query and reference centroid sets. For query perturbations, synthetic detections were sampled uniformly within the 512-pixel query patch. For reference perturbations, synthetic detections were uniformly sampled across the full reference image, with an upper limit on the number of added reference points to avoid excessive memory usage and runtime. Patches or perturbed centroid sets with fewer than three landmarks after perturbation were recorded as failures. For the segmentation-error benchmark, matcher-reported success and inlier fraction were recorded only as diagnostics and were not used to define benchmark success. This was necessary because removing true landmarks changes the maximum attainable inlier fraction, whereas adding spurious detections changes the consensus denominator and can distort internal success flags. Instead, success was defined solely by image-level agreement: the predicted reference crop was compared with the original unperturbed query image using SSIM after identical percentile-based intensity normalisation, and a trial was considered successful when SSIM≥0.80. Success probabilities were reported with 95% Wilson confidence intervals. Extra-detection levels were plotted as tested categories because the perturbation series was intentionally uneven.

Runtime was measured as both matcher time and total per-patch time, with total time including patch extraction and metric calculation. Benchmarks were run using the NucleiSkyApp benchmarking notebooks on a laptop workstation running Ubuntu 26.04 LTS with an Intel Core i7-13700H CPU, 64 GB RAM and an NVIDIA GeForce RTX 4060 Laptop GPU.

3D benchmark and performance analysis

We evaluated NucleiSky3D using a volumetric patch-localisation task similar to the 2D benchmark. Query subvolumes were cropped from a larger reference volume, providing known ground-truth positions for each query. For each reference, 3D landmark centroids were extracted from label masks and stored in both pixel and calibrated micrometre coordinates. Query landmarks were defined as nuclei whose centroids fell within the cropped subvolume. Their coordinates were transformed into local coordinate system for the query by subtracting the crop origin and converting into micrometres using the voxel size, whereas the reference landmarks remained in the coordinate system of the full reference volume. Because the query and reference came from the same calibrated volume in this benchmark, correct localisation corresponded to recovering the 3D crop translation within the reference.

For each tested subvolume size, candidate query locations were generated deterministically and reused across matchers. Patch origins were anchored to landmarks to ensure informative configurations: a landmark was sampled, assigned a random position within the subvolume and the resulting z, y, x crop origin was clipped to the reference bounds. Duplicate candidates were removed and subvolumes with fewer than three landmarks were discarded before matching. Patch sizes were specified in z, y, x order and automatically filtered to evaluate only subvolumes that fit within the loaded reference volume. Unless otherwise stated, 100 candidate subvolumes were tested per patch size.

We evaluated the pyramid, 3D geometric hashing and adaptive 3D matchers on the same query subvolumes. The adaptive controller used the pyramid and 3D geometric hashing matchers, stopping once successful localisation was achieved. For each query, we recorded the matcher-reported success, the estimated 3D translation, the inlier fraction, the mean inlier error, the estimated scale, the estimated rotation magnitude, the per-axis translation errors and the runtime. Translation accuracy was measured by comparing the estimated translation with the known crop origin in both calibrated coordinates and pixels. Total runtime included patch extraction, matching and metric calculation, whereas matcher runtime measured the matching call itself.

For plotting and summary statistics, successful 3D localisation was defined as matcher-reported success, an inlier fraction≥0.50 and a 3D translation error <20 pixels. The xy maximum-intensity-projection SSIM between the query subvolume and the predicted reference subvolume was computed after identical percentile-based intensity normalisation. It was recorded only as an image-level diagnostic and not used as a volumetric success criterion. Success probabilities were summarised as a function of query subvolume size and of the number of landmarks in the query subvolume. Confidence intervals were calculated using the Wilson score interval. Landmark-count summaries were plotted on a logarithmic x-axis, and logistic fits in log10(landmark count) space were used to estimate the number of landmarks required to reach 90% localisation probability. Benchmark results were checkpointed separately for each matcher and patch size, allowing interrupted runs to resume from existing result files.

Runtime was measured as both matcher time and total per-patch time, with total time including patch extraction and metric calculation. Benchmarks were run using the NucleiSkyApp benchmarking notebooks on a laptop workstation running Ubuntu 26.04 LTS with an Intel Core i7-13700H CPU, 64 GB RAM and an NVIDIA GeForce RTX 4060 Laptop GPU.

Cells

MCF10 DCIS.com cell lines (DCIS.com; invasive T24 c-Ha-ras oncogene-transfected breast epithelial cells; Miller et al., 2000) were cultured in a 1:1 mix of DMEM (Merck) and F12 (Merck) supplemented with 5% horse serum (Gibco, cat. no. 16050122), 20 ng/ml human EGF (Merck, cat. no. E9644), 0.5 μg/ml hydrocortisone (Merck, cat. no. H0888-1G), 100 ng/ml cholera toxin (Merck, cat. no. C8052-1MG), 10 μg/ml insulin (Merck, cat. no. I9278-5ML) and 1% (v/v) penicillin-streptomycin (Merck, cat. no. P0781-100ML) at 37°C, 5% CO2. DCIS.com cells were provided by John F. Marshall (Barts Cancer Institute, Queen Mary University of London, London, England, UK).

Human umbilical vein endothelial cells (HUVECs; PromoCell, C-12203) were cultured in ready-to-use endothelial cell growth medium (ECGM; PromoCell, C-22010 and C-39215), supplemented with 1% penicillin-streptomycin (Sigma-Aldrich, P0781). Human pulmonary microvascular endothelial cells (HPMEC; PromoCell, C-12281) were cultured in ready-to-use Endothelial Cell Growth Medium MV (ECGM MV; PromoCell, C-22020 and C-39225) supplemented with 1% penicillin-streptomycin (Sigma-Aldrich, P0781). Primary endothelial cells from ‘P0’ (commercial vial) were expanded to a P3 stock and stored at −80°C until used to standardise experimental replicates. Vials were thawed, cells were seeded into a 10-cm culture dish, and the dish was incubated for at least 2 days.

Pancreatic ductal adenocarcinoma SU.86.86 cells (ATCC CRL-1837) and AsPC-1 cells (ATCC CRL-1682) were cultured in RPMI 1640 (Sigma-Aldrich, R6504) supplemented with 10% fetal bovine serum (FBS; BioWest, S1810), 1% L-glutamine (Sigma-Aldrich, G7513) and 1% penicillin-streptomycin (Sigma-Aldrich, P0781). AsPC-1 and SU.86.86 cells were authenticated by the Leibniz Institute DSMZ (Deutsche Sammlung von Mikroorganismen und Zellkulturen) via STR profiling.

All cell lines were routinely tested for mycoplasma infections and found to be free of mycoplasma.

Dataset 1: tiled widefield image of a DAPI-stained HUVEC monolayer

HUVECs were seeded in an ibidi chamber to form a confluent endothelial monolayer, then fixed with 4% paraformaldehyde (Thermo Fisher Scientific, 28908) and stained with DAPI (Thermo Fisher Scientific, D1306) to label nuclei. Widefield fluorescence images were captured on a Nikon Eclipse Ti2-E inverted microscope, controlled by Nikon NIS-Elements AR (v6.10), using a 10×/0.3 Plan Fluor objective and a Hamamatsu Orca Flash4.0 V3 sCMOS camera. Illumination was provided by a Lumencor Spectra/AuraII (Spectra X) light engine. The dataset used for benchmarking was acquired as a stitched mosaic in NIS-Elements and exported for analysis. For the nuclei channel, excitation was delivered via the 395 nm LED line and emission was collected through the Nikon DAPI filter set. The camera was operated with 2×2 binning and 12-bit acquisition, with data stored in a 16-bit container. The reference image used for benchmarking measured 1266×2970 pixels, corresponding to 1671.8 μm×3922.0 μm with a pixel size of 1.3205 μm per pixel. For benchmarking, nuclei were segmented using intensity thresholding (implemented via the NucleiSky threshold-and-watershed backend), identifying 4387 nuclei. This dataset has been deposited on Zenodo (doi:10.5281/zenodo.20721416).

Dataset 2: spinning-disk montage of a HUVEC monolayer co-incubated with SU.86.86 cells

HUVECs were seeded in an ibidi chamber to form a confluent endothelial monolayer, after which SU.86.86 cells were added for 2 h. Monolayers were washed, fixed with 4% paraformaldehyde (Thermo Fisher Scientific, 28908) and stained with DAPI (Thermo Fisher Scientific, D1306) to label nuclei. Imaging was performed on a 3i/Zeiss Marianas CSU-W1 spinning-disk confocal inverted microscope equipped with a 20×/0.8 Plan-Apochromat air objective and an Orca Flash4.0 sCMOS camera, controlled by 3i SlideBook (v6). Acquisition was performed using the 405-nm channel for nuclei. The exposure time for the 405-nm channel was 50 ms. Z-stacks were acquired and stitched into a montage, and the dataset was analysed as a maximum-intensity projection. The resulting projected montage had dimensions of 5520×5520 pixels with a calibrated field of view of 3500.488 μm×3500.488 μm (pixel size 0.6341 μm/pixel). For benchmarking, nuclei were segmented from the DAPI channel using the Cellpose backend (Cellpose-SAM configuration), identifying 14,418 nuclei. This dataset has been deposited on Zenodo (doi:10.5281/zenodo.20721416).

Dataset 3: public H&E whole-slide image of mouse colon

Dataset 3 is an H&E bright-field image of a mouse colon section, obtained from the 10× Genomics datasets portal (dataset: ‘Fresh Frozen Mouse Colon with Xenium Multimodal Cell Segmentation’; https://www.10xgenomics.com/datasets). The image was provided as an OME-TIFF RGB image (8-bit; three channels corresponding to RGB) with dimensions of 32,201×28,329 pixels and a calibrated sampling of 0.27377 μm/pixel (corresponding to an imaged area of ∼8.82 mm×7.76 mm). For benchmarking, nuclei were segmented from the H&E image using the InstanSeg backend, identifying 247,107 nuclei.

Dataset 4: widefield tiled imaging of fixed E. coli cells

E. coli cells were cultured under standard conditions at 37°C in LB broth, sedimented onto glass coverslips, fixed with 4% paraformaldehyde for 10 min and stained with DAPI (Thermo Fisher Scientific, D1306) to label bacterial nucleoids. Imaging was performed on a Zeiss Axio Observer 7 inverted microscope equipped with a dual-filter wheel and a Prime 95B camera, using a Plan-Apochromat 40×/0.95 Korr M27 objective with a 1.6× optovar, resulting in an effective magnification of 64× and a calibrated pixel size of 0.171875 μm/pixel. For benchmarking, the stitched reference image used measured 11,788×11,788 pixels, corresponding to ∼2.03×2.03 mm. DAPI-stained bacterial nucleoids were segmented using the Cellpose-SAM backend in NucleiSky, identifying 81,379 detected objects. This dataset has been deposited on Zenodo (doi:10.5281/zenodo.20721416).

Datasets 5 and 6: DCIS.com monolayers for cross-scale localisation

Datasets 5 and 6 consist of confluent monolayers of DCIS.com cells that transiently express MYO10HMM–GFP (a truncated Myosin-X construct used to label filopodia tips; Grobe et al., 2026 preprint) and are stained to visualise the plasma membrane and nuclei. Briefly, DCIS.com cells transiently expressing MYO10HMM–GFP were seeded into ibidi 2-well culture inserts (ibidi, 81176) positioned at the centre of MatTek 35-mm gridded glass-bottom dishes (P35G-1.5-14-CGRD; MatTek) coated with poly-L-lysine and fibronectin. Cells were fixed for 25 min at room temperature in 2% formaldehyde and 0.05% glutaraldehyde prepared in 0.1 M HEPES supplemented with 2 mM CaCl2 and 2 mM MgCl2. Samples were stained with wheat germ agglutinin (WGA; 1:1000) to label the cell surface and with DAPI (Thermo Fisher Scientific, D1306) to label nuclei. Between steps, samples were washed three times with 0.1 M HEPES supplemented with 2 mM CaCl2 and 2 mM MgCl2. Images were acquired on a Zeiss LSM 880 inverted microscope equipped with an Airyscan detector and controlled with Zeiss ZEN 2.3. Airyscan processing was performed in standard super-resolution mode and tiled acquisitions were stitched in ZEN as indicated below.

Dataset 5 includes a 20× overview mosaic and a 63× query. For the reference overview, a tiled acquisition was performed with a Plan-Apochromat 20×/0.8 air objective and processed using Airyscan/Zen stitching and fusion, yielding a field of view of 6966×7143 pixels and a calibrated pixel size of 0.179214 μm/pixel (x, y). For the high-magnification query, a confocal Z-stack was captured with a C Plan-Apochromat 63×/1.4 oil objective, yielding 2372×2372 pixels at 0.056893 μm/pixel (x, y) and a Z sampling of 0.184964 μm. For localisation, the stack was converted to a 2D view using a maximum-intensity projection of the nuclei channel.

Dataset 6 includes a 10× overview mosaic and a 63× query. The reference overview was captured with a Plan-Apochromat 10×/0.3 air objective and processed using Airyscan/Zen stitching and fusion. This produced a fused mosaic of 6844×6904 pixels, with a calibrated pixel size of 0.2679 μm/pixel, covering ∼1.83×1.85 mm. The high-magnification query involved acquiring a 63× Z-stack with a C Plan-Apochromat 63×/1.4 oil objective, yielding an image of 2372×2372 pixels with voxel sampling of 0.0316×0.0316×0.1850 μm3 (x, y, z) across 95 Z-planes. For localisation, a single Z-plane from this stack was selected.

For both datasets, nuclei were segmented independently in the reference and query images using the InstanSeg backend. To harmonise nuclear appearance across magnifications, images were rescaled with the NucleiSky rescaling option before segmentation. Both datasets have been deposited on Zenodo (doi:10.5281/zenodo.20721416).

Dataset 7: live-to-fixed imaging using bright-field-derived nuclear labels

Dataset 7 was used to test live-to-fixed registration between a label-free bright-field movie and a post-fixation fluorescence tile scan. The experiment was conducted using FlowVision, in which circulating pancreatic cancer cells are perfused over an endothelial monolayer under controlled flow conditions and imaged using fast transmitted-light bright-field microscopy (Follain et al., 2026).

For live imaging, AsPC-1 cells in suspension (2×105 cells/ml) were perfused over a confluent HPMEC monolayer within a fibronectin-coated microfluidic channel, following the FlowVision endothelial arrest assay, in which circulating cells are exposed to controlled capillary-like flow over an endothelial monolayer. The assay uses a microfluidic channel connected to a perfusion system to deliver cancer cells over endothelial monolayers under defined flow conditions while imaging in a temperature- and CO2-controlled microscope chamber (Follain et al., 2026). Live imaging was performed in bright-field mode to avoid fluorescence phototoxicity and preserve downstream endpoint staining options. In this experiment, cells were perfused for 8 min, with the flow speed changed every 2 min from 300 μl/min to 200 μl/min, then to 100 μl/min, followed by a return to 300 μl/min as a wash step to assess adhesion stability. The live bright-field movie used for NucleiSky registration was acquired as a 16-bit time series of 12,000 frames, with a 40 ms frame interval. Each frame measured 2048×2044 pixels, corresponding to 664.6×663.3 μm, with a calibrated pixel size of 0.3245 μm/pixel.

Because nuclei were not directly acquired as a fluorescent channel during live imaging, synthetic nuclear labels were generated from the bright-field data using the pix2pix artificial-labelling model developed for FlowVision (Follain et al., 2026; Isola et al., 2018). This model was trained on paired bright-field and fluorescence images to infer DAPI-like nuclear signal from bright-field input images. The pix2pix-derived nuclear image used for NucleiSky analysis measured 1024×1024 pixels and served as the query image. It was segmented using the Cellpose-SAM backend in NucleiSky to generate the live-derived nuclear centroid constellation.

After live imaging, the same sample was immediately fixed for 10 min with 4% PFA at a flow rate of 200 μl/min. The PFA was washed with PBS for 10 min at a flow rate of 200 μl/min, then stored overnight at 4°C. Immunostaining for TOMM20 was performed prior to Cell Painting staining. Cells were permeabilised with 0.2% Triton X-100 for 10 min at room temperature on a rocker, washed three times with 150 μl PBS and quenched with 0.2 M glycine for 30 min at room temperature on a rocker. After three washes with PBS, cells were incubated with the mouse anti-TOMM20 antibody (1:200; Abcam, cat. no. ab56783) for 1 h at room temperature on a rocker. Cells were then washed three times with PBS and incubated with Alexa Fluor 647 goat anti-mouse-IgG secondary antibody (1:400; Thermo Fisher Scientific, cat. no. A21235) for 30 min at room temperature on a rocker. After three further PBS washes, cells were stained with the Image-iT Cell Painting Kit (Invitrogen, cat. no. I65000) according to the manufacturer's protocol, omitting MitoTracker. Briefly, cells were incubated with Cell Painting blocking solution (1% BSA+0.1% Triton X-100 in HBSS, filtered through a 0.2 μm membrane) for 20 min at room temperature on a rocker, washed three times with PBS and incubated for 30 min at room temperature on a rocker with the Cell Painting dye mixture containing Hoechst 34580, concanavalin A–Alexa Fluor 488, WGA–Alexa Fluor 555, Alexa Fluor 568–phalloidin and SYTO 14. The final working concentrations were 5 μg/ml Hoechst 34580, 100 μg/ml concanavalin A–Alexa Fluor 488, 3 μg/ml WGA–Alexa Fluor 555, 50 ng/ml Alexa Fluor 568–phalloidin and 6.5 μM SYTO 14. Cells were washed three times with PBS and stored at 4°C, protected from light, until imaging.

Images were acquired on a 3i/Zeiss Marianas CSU-W1 spinning-disk confocal microscope equipped with a Yokogawa CSU-W1 scanner, a Hamamatsu ORCA-Flash4.0 sCMOS camera and a Zeiss Plan-Apochromat 20×/0.8 air objective. Images were acquired using Slidebook 6 software with 1×1 binning, a calibrated pixel size of 0.317 μm/pixel and a Z-step size of 0.630 μm. Each field of view measured 649×649 μm (2048×2048 pixels). The four-channel Z-stacks were acquired as a 5×5 tile montage comprising 25 fields of view, and the montage was generated with Slidebook 6.

The fixed nuclei (Hoechst 34580) reference was exported as a maximum-intensity projection and used as the reference image for NucleiSky registration. The reference image measured 9421×9421 pixels, corresponding to 2985.3×2985.3 μm, with a calibrated pixel size of 0.3169 μm/pixel. The image was 16-bit and segmented from the nuclei channel using the Cellpose-SAM backend in NucleiSky.

For NucleiSky analysis, the pix2pix-derived live nuclear centroids served as the query constellation and the Hoechst-derived fixed nuclear centroids as the reference constellation. Both centroid sets were converted into calibrated physical coordinates before matching. NucleiSky was then used to estimate a 2D similarity transform that mapped the live bright-field-derived query field of view onto the fixed fluorescence reference tile scan. The resulting transform located the live-imaged field of view within the post-fixation reference image and enabled transfer of the alignment to additional endpoint fluorescence channels. This dataset has been deposited on Zenodo (doi:10.5281/zenodo.20721416).

Dataset 8: dataset used for smart microscopy

Dataset 8 was used to test real-time smart microscopy re-targeting across microscopes. The dataset consisted of three acquisition stages: a query image acquired on Microscope A, low-magnification overview images acquired during automated scanning on Microscope B and high-resolution images acquired after NucleiSky identified the matched field of view. HUVEC cells were seeded on 13 mm circular coverslips coated with fibronectin (10 μg/ml) at a density of 50,000 cells per coverslip and cultured in 12-well plates. After 48 h, cells were fixed with 4% PFA in culture medium for 10 min at room temperature, washed three times with PBS and stained using the Image-iT Cell Painting Kit, omitting MitoTracker. Cells were incubated with Cell Painting blocking solution for 20 min at room temperature on a rocker, washed three times with PBS and incubated for 30 min at room temperature with Hoechst 34580, concanavalin A–Alexa Fluor 488, WGA–Alexa Fluor 555, Alexa Fluor 568–phalloidin and SYTO 14 at final working concentrations of 5 μg/ml, 100 μg/ml, 3 μg/ml, 50 ng/ml and 6.5 μM, respectively. Cells were then washed three times with PBS and stored at 4°C protected from light until imaging. The query image was acquired on a Nikon Eclipse Ti2-E inverted microscope using NIS-Elements AR (v6.10) and a 20×/0.75 Plan Apo Lambda objective, with a Hamamatsu Orca Flash4.0 V3 camera. Illumination was provided by a Lumencor Spectra/AuraII light engine. Images were acquired in three fluorescence channels: Hoechst342, AF568 and SYTO 14, using excitation wavelengths of 395, 550 and 475 nm, respectively. All channels were acquired with a 100 ms exposure time. The query was acquired as a seven-plane Z-stack with 0.9 μm spacing. Each plane measured 2044×2048 pixels, corresponding to 663.28×664.58 μm, with a calibrated pixel size of 0.3245 μm/pixel. For NucleiSky matching, the nuclei channel was converted into a maximum-intensity projection and used as the query image.

Overview scanning and matched high-resolution imaging were performed on a Nikon Ti2-E Crest V3 inverted microscope controlled by NIS-Elements AR (v6.20). Low-magnification overview images were acquired using a 4×/0.2 CFI Plan Apochromat Lambda D objective and a Photometrics Kinetix camera. Overview images were acquired from the nuclei channel. Each overview position was captured as a five-plane Z-stack with 12.5 μm spacing. Within the NIS-Elements JOBS workflow, each overview stack was processed using a maximum-intensity-projection block followed by a Denoise.ai block before being passed to NucleiSky. The processed overview images measured 2048×2048 pixels, corresponding to 3430.51×3430.51 μm, with a calibrated pixel size of 1.6751 μm/pixel. NucleiSky compared the processed overview image with the query image to identify the matching nuclear constellation. When no acceptable match was detected, the JOBS workflow advanced to the next overview position. When a match was detected, the microscope moved to the predicted position and, using a 20×/0.8 CFI Plan Apochromat Lambda D objective and a Photometrics Kinetix sCMOS camera, acquired a three-channel image (Hoechst 34580, AF568 and SYTO 14, excited at 406, 545 and 476 nm, respectively) as a 3×3 stitched mosaic with 30% overlap and a 9-plane Z-stack with 0.8 μm spacing. This resulted in a 5735×5735-pixel image with a calibrated pixel size of 0.3332 μm/pixel. This dataset has been deposited on Zenodo (doi:10.5281/zenodo.20721416).

Dataset 9: Platynereis-Nuclei-CBG

Dataset 9 was the publicly available Platynereis-Nuclei-CBG dataset (Lalit et al., 2022), used to evaluate 3D localisation with pre-existing nuclear instance masks. The reference volume measured 700×660×113 pixels (x, y, z) and contained 392 segmented nuclei after conversion of the 3D label mask into nuclear centroids. Pre-existing 3D instance masks were used directly; no additional nuclear segmentation was performed. Label objects were converted to calibrated 3D centroids and used as the reference landmark set for NucleiSky3D benchmarking.

Dataset 10: DAPI-stained mouse brain 3D benchmark dataset

Dataset 10 was generated previously, and no new animal experiments were performed in this study (Ball et al., 2024). Briefly, brains from female mice (Hsd: Athymic Nude-Foxn/1nu, Envigo) were dissected, rinsed in PBS and fixed overnight at 4°C in 4% paraformaldehyde (Thermo Fisher Scientific, 28908). Fixed brains were embedded in low-melting-point agarose and 100–200 μm sections were cut using a vibratome. Brain slices were permeabilised, blocked and stained. Samples were mounted in ProLong Glass Antifade Mountant (P36980; Thermo Fisher Scientific). Images were acquired on a Marianas spinning-disk confocal microscope using a Yokogawa CSU-W1 spinning-disk unit on an inverted Zeiss Axio Observer Z1 microscope controlled by SlideBook 6, with an Orca Flash 4 sCMOS camera and a 63×/1.4 oil Plan-Apochromat objective.

The analysed volume measured 1840×1840×224 pixels (x, y, z), corresponding to 185.2×185.2×67.2 μm, with voxel sampling of 0.1007×0.1007×0.2998 μm (x, y, z). The DAPI channel was used to generate the 3D nuclear landmark reference. Individual Z-planes were segmented using the Cellpose-SAM backend in NucleiSky and 3D nuclear objects were reconstructed by merging labels across adjacent Z-slices using the NucleiSky 2.5D label-linking workflow. This produced 864 segmented nuclei, which were reduced to calibrated 3D centroids for benchmarking. Query subvolumes of increasing size were cropped from this reference and re-localised using the pyramid, 3D geometric hashing and adaptive NucleiSky3D matchers. This dataset has been deposited on Zenodo (doi:10.5281/zenodo.20721416).

Manuscript preparation and generative AI use

Figures were prepared using Fiji software (Schindelin et al., 2012) and Inkscape (https://inkscape.org/). PlotsOfData was used to generate the box plot shown in Fig. S1 (Postma and Goedhart, 2019). GPT-5.5 (OpenAI) and Grammarly (Grammarly, Inc.) served as writing aids during manuscript preparation. After using these services, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication. The preprint version of this manuscript was formatted with Rxiv-Maker (Saraiva et al., 2026).

Supplementary Material

Supplementary information
DOI: 10.1242/joces.265152_sup1

Acknowledgements

Imaging was performed at the Advanced Imaging Core of the Turku Bioscience Centre. This facility received support from Turku BioImaging, Biocenter Finland and the Finnish Advanced Microscopy Node of Euro-BioImaging Finland (funded by the Research Council of Finland, FIRI 2023 grant decision numbers 359073 and 358879 and FIRI 2024 grant decision numbers 367582 and 367577).

Footnotes

Author contributions

Conceptualization: I.H.-C., J.K.A., P.N., G.J.; Data curation: G.J.; Formal analysis: I.H.-C., A.O.-O., G.J.; Funding acquisition: G.J.; Investigation: I.H.-C., A.O.-O., C.P., G.J.; Methodology: I.H.-C., A.O.-O., G.J.; Project administration: G.J.; Resources: G.J.; Software: I.H.-C., G.J.; Supervision: R.H., M.D.R., G.J.; Validation: G.J.; Visualization: I.H.-C., G.J.; Writing – original draft: G.J.; Writing – review & editing: I.H.-C., A.O.-O., C.P., P.N., R.H., M.D.R., G.J.

Funding

This study was funded by the Research Council of Finland (338537, 371287 and 374180 to G.J.), the Sigrid Juselius Foundation (to G.J.), the Cancer Society of Finland (Syöpäjärjestöt; to G.J.) and the Solutions for Health strategic funding for Åbo Akademi University (to G.J.). The research was also supported by the InFLAMES Flagships Program of the Research Council of Finland (decision numbers: 337530, 337531 and 357910). G.J. is supported by the Finnish Cancer Institute (K. Albin Johansson Professorship). I.H.-C. is funded by the Finnish Doctoral Program Network in Artificial Intelligence, AI-DOC (decision number VN/3137/2024-OKM-6). P.N. is supported by the Fru Berta Kamprads Foundation [FBKS-2024-8-(584)]. R.H., M.D.R. and C.P. acknowledge funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (SelfDriving4DSR, grant agreement No. 101001332). R.H. is further supported by the European Union through Horizon Europe (RT-SuperES, grant agreement No. 101099654); by a European Molecular Biology Organization (EMBO) Installation Grant (EMBO-2020-IG-4734); by a Chan Zuckerberg Initiative Visual Proteomics Imaging award (vpi-0000000044, doi:10.37921/743590vtudfp); by a joint Wellcome Trust, Chan Zuckerberg Initiative and Kavli Foundation Essential Open Source Software for Science Cycle 6 award (Wellcome 313383/Z/24/Z; CZI EOSS6-0000000260); and by the La Caixa Foundation (CaixaResearch Health 2025, VirusAwareScopes, HR25-00453). R.H. and C.P. were also supported by Fundação para a Ciência e a Tecnologia, through the MOSTMICRO-ITQB R&D Unit (doi:10.54499/UID/04612/2025, UID/PRR/4612/2025) and the LS4FUTURE Associated Laboratory (doi:10.54499/LA/P/0087/2020). The Chan Zuckerberg Initiative awards are made through the Chan Zuckerberg Initiative DAF, an advised fund of Silicon Valley Community Foundation. This study was funded by the European Union; however, the views and opinions expressed are those of the authors only and do not necessarily reflect those of the European Union or the granting authority. Neither the European Union nor the granting authority can be held responsible for them. Open Access funding provided by the University of Turku. Deposited in PMC for immediate release.

Data and resource availability

All datasets used in this study are either already publicly available (as indicated) or deposited on Zenodo (doi:10.5281/zenodo.20721416). All code used in this study is available under the MIT license at https://github.com/CellMigrationLab/NucleiSky and archived on Zenodo (doi:10.5281/zenodo.21071484). Any additional information needed to reanalyse the data reported in this paper is available from the corresponding authors. All other relevant data and details of resources can be found within the article and its supplementary information.

Special Issue

This article is part of the Special Issue ‘Imaging Cell Architecture and Dynamics’, guest edited by Francesca Bottanelli and Giulia Zanetti. See related articles at https://journals.biologists.com/jcs/issue/139/21.

Peer review history

The peer review history is available online at https://journals.biologists.com/jcs/lookup/doi/10.1242/jcs.265152.reviewer-comments.pdf

References

  1. Almada, P., Pereira, P. M., Culley, S., Caillol, G., Boroni-Rueda, F., Dix, C. L., Charras, G., Baum, B., Laine, R. F., Leterrier, C.et al. (2019). Automating multimodal microscopy with NanoJ-Fluidics. Nat. Commun. 10, 1223. 10.1038/s41467-019-09231-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ball, N. J., Ghimire, S., Follain, G., Pajari, A. O., Wurzinger, D., Vaitkevičiūtė, M., Cowell, A. R., Berki, B., Ivaska, J., Paatero, I.et al. (2024). TLNRD1 is a CCM complex component and regulates endothelial barrier integrity. J. Cell Biol. 223, e202310030. 10.1083/jcb.202310030 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Beroiz, M., Cabral, J. B. and Sanchez, B. (2020). Astroalign: a Python module for astronomical image registration. Astron. Comput. 32, 100384. 10.1016/j.ascom.2020.100384 [DOI] [Google Scholar]
  4. Enninful, A., Zhang, Z., Klymyshyn, D., Ingalls, M., Yang, M., Zong, H., Bai, Z., Farzad, N., Su, G., Baysoy, A.et al. (2026). Integration of imaging-based and sequencing-based spatial omics mapping on the same tissue section via DBiTplus. Nat. Methods. 23, 1827-1839. 10.1038/s41592-025-02948-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Fernandez, R. and Moisy, C. (2021). Fijiyama: a registration tool for 3D multimodal time-lapse imaging. Bioinformatics 37, 1482-1484. 10.1093/bioinformatics/btaa846 [DOI] [PubMed] [Google Scholar]
  6. Ferreira, M. G., Saraiva, B. M., Brito, A. D., Pinho, M. G., Henriques, R. and Gómez-de-Mariscal, E. (2025). ReScale4DL: balancing pixel and contextual information for enhanced bioimage segmentation. bioRxiv 2025.04.09.647871. 10.1101/2025.04.09.647871 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Follain, G., Ghimire, S., Pylvänäinen, J. W., Vaitkevičiūtė, M., Hidalgo-Cenalmor, I., Wurzinger, D., Guzmán, C., Conway, J. R. W., Dibus, M., Härkönen, J.et al. (2026). Fast label-free live imaging with FlowVision reveals key principles of cancer cell arrest on endothelial monolayers. EMBO J. 45, 1381-1421. 10.1038/s44318-025-00678-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Goldsborough, T., O'Callaghan, A., Inglis, F., Leplat, L., Filby, A., Bilen, H. and Bankhead, P. (2024). A novel channel invariant architecture for the segmentation of cells and nuclei in multiplexed images using InstanSeg. bioRxiv 2024.09.04.611150. 10.1101/2024.09.04.611150 [DOI] [Google Scholar]
  9. Goltsev, Y., Samusik, N., Kennedy-Darling, J., Bhate, S., Hale, M., Vazquez, G., Black, S. and Nolan, G. P. (2018). Deep profiling of mouse splenic architecture with CODEX multiplexed imaging. Cell 174, 968-981.e15. 10.1016/j.cell.2018.07.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Grobe, H., Pineda, M. X. R., Ghimire, S., Laisne, M.-C., Nylund, A., Vaitkevičiūtė, M., Vihinen, H., Prakash, A., Tammi, J., Ojalill, M.et al. (2026). Filopodia-mediated trans-endocytosis. bioRxiv 2026.02.09.703982. 10.64898/2026.02.09.703982 [DOI] [Google Scholar]
  11. Groth, E. J. (1986). A pattern-matching algorithm for two-dimensional coordinate lists. Astron. J. 91, 1244-1248. 10.1086/114099 [DOI] [Google Scholar]
  12. Guizar-Sicairos, M., Thurman, S. T. and Fienup, J. R. (2008). Efficient subpixel image registration algorithms. Opt. Lett. 33, 156-158. 10.1364/OL.33.000156 [DOI] [PubMed] [Google Scholar]
  13. Gut, G., Herrmann, M. D. and Pelkmans, L. (2018). Multiplexed protein maps link subcellular organization to cellular states. Science 361, eaar7042. 10.1126/science.aar7042 [DOI] [PubMed] [Google Scholar]
  14. Hagberg, A. A., Schult, D. A. and Swart, P. J. (2008). Exploring network structure, dynamics, and function using NetworkX. Python Sci. Conf. 10.25080/TCWV9851 [DOI] [Google Scholar]
  15. Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J.et al. (2020). Array programming with NumPy. Nature 585, 357-362. 10.1038/s41586-020-2649-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Hidalgo-Cenalmor, I., Pineda, M. X. R., Saraiva, B. M., Henriques, R. and Jacquemet, G. (2026). Packaging Jupyter notebooks as installable desktop apps using LabConstrictor. arXiv 2603.10704 [cs.SE]. 10.48550/arXiv.2603.10704 [DOI]
  17. Hollandi, R., Szkalisity, A., Toth, T., Tasnadi, E., Molnar, C., Mathe, B., Grexa, I., Molnar, J., Balind, A. and Gorbe, M.et al. (2020). nucleAIzer: a parameter-free deep learning framework for nucleus segmentation using image style transfer. Cell Syst. 10, 453-458.e6. 10.1016/j.cels.2020.04.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hunter, J. D. (2007). Matplotlib: a 2D graphics environment. Comput. Sci. Eng. 9, 90-95. 10.1109/MCSE.2007.55 [DOI] [Google Scholar]
  19. Isola, P., Zhu, J.-Y., Zhou, T. and Efros, A. A. (2018). Image-to-image translation with conditional adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 5967-5976. 10.1109/CVPR.2017.632 [DOI] [Google Scholar]
  20. Jeyasangar, A., Alsalemi, A. and Raza, S. E. A. (2024). Nuclei-location based point set registration of multi-stained whole slide images. In Medical Image Understanding and Analysis (ed. Yap M. H., Kendrick C., Behera A., Cootes T.,and Zwiggelaar R.), pp. 372-386. Cham: Springer Nature Switzerland. [Google Scholar]
  21. Karreman, M. A., Mercier, L., Schieber, N. L., Solecki, G., Allio, G., Winkler, F., Ruthensteiner, B., Goetz, J. G. and Schwab, Y. (2016). Fast and precise targeting of single tumor cells in vivo by multimodal correlative microscopy. J. Cell Sci. 129, 444-456. 10.1242/jcs.181842 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kim, E. N., Chen, P. Z., Bressan, D., Tripathi, M., Miremadi, A., di Pietro, M., Coussens, L. M., Hannon, G. J., Fitzgerald, R. C., Zhuang, L.et al. (2023). Dual-modality imaging of immunofluorescence and imaging mass cytometry for whole-slide imaging and accurate segmentation. Cell Rep. Methods 3, 100595. 10.1016/j.crmeth.2023.100595 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Klein, S., Staring, M., Murphy, K., Viergever, M. A. and Pluim, J. P. W. (2010). elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29, 196-205. 10.1109/TMI.2009.2035616 [DOI] [PubMed] [Google Scholar]
  24. Krentzel, D., Elphick, M., Domart, M.-C., Peddie, C. J., Laine, R. F., Shand, C., Henriques, R., Collinson, L. M. and Jones, M. L. (2025). CLEM-Reg: an automated point cloud-based registration algorithm for volume correlative light and electron microscopy. Nat. Methods 22, 1923-1934. 10.1038/s41592-025-02794-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kukulski, W., Schorb, M., Welsch, S., Picco, A., Kaksonen, M. and Briggs, J. A. G. (2011). Correlated fluorescence and 3D electron microscopy with high sensitivity and spatial precision. J. Cell Biol. 192, 111-119. 10.1083/jcb.201009037 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lalit, M., Tomancak, P. and Jug, F. (2022). EmbedSeg: embedding-based instance segmentation for biomedical microscopy data. Med. Image Anal. 81, 102523. 10.1016/j.media.2022.102523 [DOI] [PubMed] [Google Scholar]
  27. Lam, S. K., Pitrou, A. and Seibert, S. (2015). Numba: a LLVM-based Python JIT compiler. In Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC, pp. 1-6. New York, NY, USA: Association for Computing Machinery. 10.1145/2833157.2833162 [DOI] [Google Scholar]
  28. Lang, D., Hogg, D. W., Mierle, K., Blanton, M. and Roweis, S. (2010). Astrometry.net: blind astrometric calibration of arbitrary astronomical images. Astron. J. 139, 1782. 10.1088/0004-6256/139/5/1782 [DOI] [Google Scholar]
  29. Lin, J.-R., Fallahi-Sichani, M. and Sorger, P. K. (2015). Highly multiplexed imaging of single cells using a high-throughput cyclic immunofluorescence method. Nat. Commun. 6, 8390. 10.1038/ncomms9390 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Miller, F. R., Santner, S. J., Tait, L. and Dawson, P. J. (2000). MCF10DCIS.com xenograft model of human comedo ductal carcinoma in situ. J. Natl. Cancer Inst. 92, 1185-1186. 10.1093/jnci/92.14.1185a [DOI] [PubMed] [Google Scholar]
  31. Moore, J. and Kunis, S. (2023). Zarr: a cloud-optimized storage for interactive access of large arrays.In Proceedings of the Conference on Research Data Infrastructure, 1. 10.52825/cordi.v1i.285. [DOI] [Google Scholar]
  32. Nasir, E. S., Elhaminia, B., Eastwood, M., King, C., Cain, O., Harper, L., Moss, P., Chanouzas, D., Snead, D., Rajpoot, N.et al. (2027). CORE: A coarse-to-fine registration engine for multi-stain whole-slide images. Biomed. Signal Process. Control 129, 111292. 10.1016/j.bspc.2026.111292 [DOI] [Google Scholar]
  33. Pachitariu, M., Rariden, M. and Stringer, C. (2025). Cellpose-SAM: superhuman generalization for cellular segmentation. bioRxiv 2025.04.28.651001. 10.1101/2025.04.28.651001 [DOI] [Google Scholar]
  34. Pál, A. and Bakos, G. Á. (2006). Astrometry in wide-field surveys. Publ. Astron. Soc. Pac. 118, 1474-1483. 10.1086/508573 [DOI] [Google Scholar]
  35. Peuhu, E., Jacquemet, G., Scheele, C. L. G. J., Isomursu, A., Laisne, M.-C., Koskinen, L. M., Paatero, I., Thol, K., Georgiadou, M., Guzmán, C.et al. (2022). MYO10-filopodia support basement membranes at pre-invasive tumor boundaries. Dev. Cell 57, 2350-2364.e7. 10.1016/j.devcel.2022.09.016 [DOI] [PubMed] [Google Scholar]
  36. Postma, M. and Goedhart, J. (2019). PlotsOfData—A web app for visualizing data together with their summaries. PLoS Biol. 17, e3000202. 10.1371/journal.pbio.3000202 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Preibisch, S., Saalfeld, S., Schindelin, J. and Tomancak, P. (2010). Software for bead-based registration of selective plane illumination microscopy data. Nat. Methods 7, 418-419. 10.1038/nmeth0610-418 [DOI] [PubMed] [Google Scholar]
  38. Pylvänäinen, J. W., Laine, R. F., Saraiva, B. M. S., Ghimire, S., Follain, G., Henriques, R. and Jacquemet, G. (2023). Fast4DReg – fast registration of 4D microscopy datasets. J. Cell Sci. 136, jcs260728. 10.1242/jcs.260728 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Radtke, A. J., Kandov, E., Lowekamp, B., Speranza, E., Chu, C. J., Gola, A., Thakur, N., Shih, R., Yao, L., Yaniv, Z. R.et al. (2020). IBEX: A versatile multiplex optical imaging approach for deep phenotyping and spatial analysis of cells in complex tissues. Proc. Natl. Acad. Sci. USA 117, 33455-33465. 10.1073/pnas.2018488117 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Saraiva, B. M., Cunha, I., Brito, A. D., Follain, G., Portela, R., Haase, R., Pereira, P. M., Jacquemet, G. and Henriques, R. (2025). Efficiently accelerated bioimage analysis with NanoPyx, a Liquid Engine-powered Python framework. Nat. Methods 22, 283-286. 10.1038/s41592-024-02562-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Saraiva, B. M., Carlota, R., Brito, A. D., Hildago-Cenalmor, I., Jacquemet, G. and Henriques, R. (2026). Rxiv-Maker: an automated template engine for streamlined scientific publications. J. Cell Sci. jcs263183. 10.1242/jcs.265183 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Schindelin, J., Arganda-Carreras, I., Frise, E., Kaynig, V., Longair, M., Pietzsch, T., Preibisch, S., Rueden, C., Saalfeld, S., Schmid, B.et al. (2012). Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676-682. 10.1038/nmeth.2019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Schmidt, U., Weigert, M., Broaddus, C. and Myers, G. (2018). Cell Detection with Star-Convex Polygons. In Medical Image Computing and Computer Assisted Intervention - MICCAI 2018 - 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part II, pp. 265-273. 10.1007/978-3-030-00934-2_30 [DOI] [Google Scholar]
  44. Schorb, M. and Briggs, J. A. G. (2014). Correlated cryo-fluorescence and cryo-electron microscopy with high spatial precision and improved sensitivity. Ultramicroscopy 143, 24-32. 10.1016/j.ultramic.2013.10.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Semba, T. and Ishimoto, T. (2024). Spatial analysis by current multiplexed imaging technologies for the molecular characterisation of cancer tissues. Br. J. Cancer 131, 1737-1747. 10.1038/s41416-024-02882-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Thévenaz, P., Ruttimann, U. E. and Unser, M. (1998). A pyramid approach to subpixel registration based on intensity. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 7, 27-41. 10.1109/83.650848 [DOI] [PubMed] [Google Scholar]
  47. van der Walt, S., Schönberger, J. L., Nunez-Iglesias, J., Boulogne, F., Warner, J. D., Yager, N., Gouillart, E. and Yu, T., and Scikit-image contributors. (2014). scikit-image: image processing in Python. PeerJ 2, e453. 10.7717/peerj.453 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J.et al. (2020). SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods 17, 261-272. 10.1038/s41592-019-0686-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. von Chamier, L., Laine, R. F., Jukkala, J., Spahn, C., Krentzel, D., Nehme, E., Lerche, M., Hernández-Pérez, S., Mattila, P. K., Karinou, E.et al. (2021). Democratising deep learning for microscopy with ZeroCostDL4Mic. Nat. Commun. 12, 2276. 10.1038/s41467-021-22518-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wang, Z., Bovik, A. C., Sheikh, H. R. and Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600-612. 10.1109/TIP.2003.819861 [DOI] [PubMed] [Google Scholar]
  51. Yaniv, Z., Lowekamp, B. C., Johnson, H. J. and Beare, R. (2018). SimpleITK image-analysis notebooks: a collaborative environment for education and reproducible research. J. Digit. Imaging 31, 290-303. 10.1007/s10278-017-0037-8 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary information
DOI: 10.1242/joces.265152_sup1

Articles from Journal of Cell Science are provided here courtesy of Company of Biologists

RESOURCES