Beyond the LUMIR challenge: The pathway to foundational registration models

Abstract

Medical image challenges have played a transformative role in advancing the field, catalyzing innovation and establishing new performance benchmarks. Image registration, a foundational task in neuroimaging, has similarly advanced through the Learn2Reg initiative. Building on this, we introduce the Large-scale Unsupervised Brain MRI Image Registration (LUMIR) challenge, a next-generation benchmark for unsupervised brain MRI registration. Previous challenges relied upon anatomical label maps, however LUMIR provides 4,014 unlabeled T1-weighted MRIs for training, encouraging biologically plausible deformation modeling through self-supervision. Evaluation includes 590 in-domain test subjects and extensive zero-shot tasks across disease populations, imaging protocols, and species. Deep learning methods consistently achieved state-of-the-art performance and produced anatomically plausible, diffeomorphic deformation fields. They outperformed several leading optimization-based methods and remained robust to most domain shifts. These findings highlight the growing maturity of deep learning in neuroimaging registration and its potential to serve as a foundation model for general-purpose medical image registration.

1. Introduction

Medical image registration is a cornerstone of medical image analysis, enabling voxel correspondence between images to support a wide range of critical clinical applications. For brain magnetic resonance imaging (MRI) in particular, accurate registration plays a pivotal role in longitudinal monitoring (Xue et al., 2006; Holland et al., 2011), assessing treatment effects (Vaarkamp et al., 2000), planning interventions (Rosenman et al., 1998), and conducting population-level studies of brain structure (Gerber et al., 2010) and function (Toga and Thompson, 2001; Bilgel et al., 2015). Deformable brain image registration is widely recognized as a fundamental tool in neuroimaging for applications ranging from Alzheimer’s disease research to functional mapping for surgical planning. Although the brain is encased within the rigid skull, which limits large deformations and constrains anatomical variability both within and between subjects after affine pre-alignment, this apparent simplicity belies significant challenges. Structures like the hippocampus, putamen, and amygdala, often lack strong intensity-based contrast, complicating voxel-based alignment. In addition, neurodegenerative conditions introduce cortical thinning, atrophy, and other subtle deformations that demand highly sensitive and robust registration algorithms to capture clinically meaningful changes.

Medical image challenges have played a crucial role in advancing the field by providing rigorous evaluation frameworks and standardized benchmarks, fostering collaboration, driving innovation, and encouraging the development of robust and generalizable algorithms for various imaging tasks (Murphy et al., 2011; Sirinukunwattana et al., 2017; Maier-Hein et al., 2018; Halabi et al., 2019; Antonelli et al., 2022; Hering et al., 2023; Hansen et al., 2025). However, despite the fundamental role of image registration in many clinical and research applications, it has been relatively underrepresented in medical image challenges, particularly those focused on brain MRI. While existing challenges have extensively addressed segmentation tasks involving tumors (Menze et al., 2015), lesions (Menze et al., 2015; Maier et al., 2017; Carass et al., 2017; Commowick et al., 2018; Kuijf et al., 2019), brain tissue (Mendrik et al., 2015; Kuijf et al., 2019), and the parcellation of fine brain structures (Carass et al., 2018), fewer efforts have been dedicated to systematically evaluating registration algorithms for brain MRI. Notable exceptions, such as Hippocampus-MR and OASIS from previous editions of the Learn2Reg challenge (Hering et al., 2023), have offered valuable contributions but were limited in scope. For example, they focus on small regions of interest or rely on label maps that can inadvertently bias models towards over-optimizing label overlap (Rohlfing, 2011) at the expense of generating smooth and physically plausible deformations (Rohlfing, 2011; Chen et al., 2024). Moreover, their evaluations were restricted to in-domain datasets, without assessing generalizability. To address these limitations, we introduce the Large-scale Unsupervised Brain MRI Image Registration (LUMIR) challenge. LUMIR leverages data from publicly available brain MRI datasets, which have been carefully prepared and preprocessed to support unsupervised learning of inter-subject registration. It emphasizes both anatomical accuracy and deformation smoothness without reliance on label map supervision. By providing gold-standard evaluation metrics alongside this curated dataset, the LUMIR challenge establishes a critical platform for benchmarking and advancing the next generation of brain image registration models, with the ultimate goal of enhancing clinical workflows and supporting high-impact neuroimaging research.

The challenge features 4,014 T1-weighted (T1-w) brain MRI images, with 3,384 training images that generate more than 5.7 million unique image pairs. This unprecedented scale promotes the development of foundational registration models while enabling fair benchmarking of traditional optimization-based and deep learning-based approaches. Furthermore, to complement the label maps used to assess accuracy, the LUMIR challenge also provides a subset of 130 test images, each manually annotated with 32 anatomical landmarks. These landmarks enable gold-standard evaluations of registration accuracy and allow analyses of correlations between landmark-based and label map-based measures. Detailed descriptions of the LUMIR challenge dataset are provided in Fig. 1. Through extensive evaluation, the results reveal that deep learning-based methods achieve superior performance and efficiency, with the top-performing approaches employing a single-step, optimization-free strategy while producing smooth and even diffeomorphic deformations, suggesting that effective model design can be sufficient for strong performance, though instance optimization may offer further gains in some settings. These findings underscore the advances and maturity of deep learning methods for brain image registration.

Fig. 1.

The top panel provides a breakdown of the training, validation, and testing data used in the LUMIR Challenge; with each source listed and the quantity of data in parentheses. The bottom panel shows from left to right, a tri-planar view of a typical T1-weighted MRI after our preprocessing, the manually identified landmarks overlaid on the same T1-weighted MRI, and finally the corresponding SLANT labels of the T1-weighted MRI. See Sec. 2.1 for complete details.

Beyond the in-domain test set, the LUMIR challenge provides the opportunity to explore a broader and more ambitious question: Can deep learning models trained on this large-scale dataset effectively register brain images across various modalities, acquisition protocols, and even species? This exploration is grounded in the unique nature of image registration, which focuses on identifying spatial correspondences between image pairs. Unlike tasks that rely on the semantic interpretation of individual images, registration models prioritize inter-image relationships. This suggests that such models may be inherently robust to out-of-distribution samples, as previous studies have shown that those trained on random shapes can still perform well in brain MRI registration (Hoffmann et al., 2021; Chen et al., 2025). To further investigate this, we evaluated challenge submissions and baseline registration models on extensive out-of-distribution brain MRI datasets. The results are striking: deep learning methods trained solely on T1-w brain MRI images of healthy subjects showed exceptional generalization capability. Moreover, the top-performing deep learning methods outperformed traditional pairwise optimization methods, despite the latter being theoretically immune to domain shift. Our results directly address concerns that deep learning-based registration methods may struggle to generalize across datasets (Jena et al., 2024b). In contrast, our findings demonstrate that with sufficient training data and appropriate preprocessing pipelines, learning-based registration models evaluated in this study exhibit excellent out-of-domain generalization to intra-contrast registration on unseen MRI data, including data from MR sequences not encountered during training, even surpassing models designed to be robust across image contrasts, such as SynthMorph (Hoffmann et al., 2021). This robustness suggests that specialized contrast-agnostic training strategies may become less critical when large-scale datasets and consistent preprocessing pipelines are available.

The LUMIR challenge underscores the transformative potential of deep learning and large scale data in creating robust and generalizable solutions for medical image registration. By addressing critical limitations in training data, evaluation methods, and application scope, this challenge establishes an unparalleled platform for benchmarking and advancing brain image registration models. This work not only summarizes the challenge outcomes but also provides empirical insights to guide the development of robust and adaptable registration models, unlocking new possibilities for clinical and research advancements. To help ensure that the LUMIR challenge has a meaningful impact, the submission website and associated evaluation of results will remain open for the foreseeable future; which is in keeping with the tradition of previous Learn2Reg Challenges. This fosters reproducibility across iterations of technological innovation within the medical image registration community.

For the rest of the paper, in Sec. 2, we introduce the datasets and evaluation metrics employed in our experiments. Sec. 3 describes the design and organization of the LUMIR challenge. In Sec. 4, we summarize both the participating methods and the baseline methods included in the challenge. Sec. 5 presents detailed quantitative and qualitative results. We then discuss key findings and limitations in Sec. 6, before concluding the paper in Sec. 7.

2. Materials

2.1. Datasets

2.1.1. Challenge Dataset

The LUMIR challenge training, validation, and test data is built from three primary public sources, these include ten different public datasets (Di Martino et al., 2014, 2017; Biomedical Image Analysis Group, Imperial College London, 2013; Zuo et al., 2014; Holmes et al., 2015; Nastase et al., 2021; Babayan et al., 2019; Sunavsky and Poppenk, 2020; Li and Clariana, 2019; Follmer et al., 2018; Orfanos et al., 2017) that are part of the OpenBHB dataset (Dufumier et al., 2022) and the OASIS dataset (Marcus et al., 2007), in combination with public landmarks (Taha et al., 2023a) and our own privately held landmarks. The OpenBHB dataset contains T1-weighted brain MRIs that have been skull-stripped and affinely aligned to the isotropic 1mm MNI template using FreeSurfer (Fischl, 2012). Subsequently, we applied kernel-based intensity normalization (Reinhold et al., 2019) to normalize the mean white matter value to 110 and used the automated segmentation tool SLANT (Huo et al., 2019) to segment the brains into 133 cortical and subcortical structures. The AFIDs-OASIS dataset (Taha et al., 2023b) is a subset of the OASIS-1 dataset (Marcus et al., 2010), comprising 30 T1-weighted 3T MR images from cognitively intact subjects, including 17 female and 13 male participants, with a mean age of 58.0 years and a standard deviation of 17.9 years. Each image is annotated with 32 manually identified anatomical landmarks and underwent the same preprocessing steps. The final LUMIR dataset contains 4,014 images, with 3,984 from OpenBHB and 30 from AFIDs-OASIS. The 3,984 OpenBHB images are acquired from 64 different sites and 3,984 different healthy control subjects. From 3,984 images, 3,637 are acquired on 3T MRI and 347 are acquired on 1.5T MRI. From 3,984 subjects, 2,087 are males and 1,897 are females. The average age of these subjects is 24.9 years, with the standard deviation of age being 14.3. From these, we randomly allocated 3,384 OpenBHB images for training; 40 images (30 from OpenBHB, 10 from AFIDs-OASIS) for validation via the grand-challenge.org public leaderboard; and 590 images (570 from OpenBHB, 20 from AFIDs-OASIS) for testing. Additionally, following AFIDs protocols (Taha et al., 2023b), a neurologist (Y. Dai) and two radiologists (J. Wu, H. Bai) manually annotated landmarks on 110 OpenBHB images in the test set for landmark-based evaluation. Specifically, the final breakdown includes:

• Training: 3,384 unlabeled brain MRI images.

• Validation: 50 brain MRI images, comprising 40 with segmentation label maps and 10 with landmarks.

• Testing: Privately held 590 brain MRI images, including 460 images with segmentation labels and 130 images with landmark annotations.

In line with our focus on unsupervised image registration, only imaging data were provided for training (i.e., no label maps or deformation fields), allowing participants to freely create inter-subject pairs or deformation fields as they see fit. This setup yields 5,724,036 unique unordered image pair combinations from the 3,384 training images, or up to 11,448,072 possible ordered moving-to-fixed training pairs, where participants were free to construct training pairs according to their own training strategy from the available 3,384 training images. Anatomical label maps and landmarks for the training and validation sets were kept private. However, a small subset (label maps from five images) was made available to participants for preliminary sanity checks before submitting their results to the Grand Challenge platform.

2.1.2. Zero-shot Datasets

• ADHD (Lytle et al., 2020, 2021): This dataset includes MRI scans from 47 children diagnosed with attention deficit hyperactivity disorder (ADHD) in two previous studies. The cohort consists of 35 males (ages 8.59–11.85) and 12 females (ages 9.8–12.3). MRI scans were acquired using 3T and 1.5T systems with the MP-RAGE sequence.

• ADNI1 1.5T&3T: Subsets of the ADNI1 dataset were used, specifically the “Complete 1Yr 1.5T” and “Complete 1Yr 3T” cohorts, comprising 639 and 89 subjects, respectively, all diagnosed with Alzheimer’s disease. Scans were acquired using 1.5T and 3T MRI systems. The age range for the 1.5T cohort is 55–91 years, and for the 3T cohort, 55–89 years. ADNI1 data was obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). ADNI was launched in 2003 as a public-private partnership, led by Principal Investigator Michael W. Weiner, MD.

• NIMH (Nugent et al., 2022): This dataset includes MRI scans of healthy subjects aged 18–72 years. The scans were categorized by imaging modality, comprising 249 subjects with T1-weighted scans, 247 with T2-weighted scans, 241 with T2*-weighted scans, and 240 with FLAIR scans. Notably, T2*-weighted and FLAIR scans typically exhibit larger voxel sizes along the slice-select (z-axis) direction, leading to significantly lower resolution along this axis.

• UltraCortex (Mahler et al., 2024): This data set consists of 9.4T ultra-high-field T1 weighted MRI scans from 78 healthy adults (28 females, 50 males) aged 20–53 years. MRI acquisition was performed using a 9.4T whole-body scanner with the MP-RAGE and MP2RAGE sequences.

• Atlas: The ICBM 2009c Nonlinear Symmetric T1-weighted template with 1 × 1 × 1 mm isotropic resolution was used for atlas based registration. This template is part of the ICBM 152 Nonlinear atlases (2009 version) (Fonov et al., 2009).

• Macaque (Milham et al., 2018): This dataset includes T1-weighted MRI scans from 57 macaques, collected from five different sources: 1) Aix-Marseille Université: Contains 4 scans from 3 male and 1 female Macaca mulatta, aged 7–8 years, acquired using a 3T MRI system with the MPRAGE sequence. 2) Mount Sinai School of Medicine (Philips): Includes 9 scans from 8 males and 1 female Macaca mulatta and Macaca fascicularis, aged 3.4–8 years, acquired on a 3T MRI scanner with the MPRAGE sequence. 3) McGill University: Comprises 3 scans from 3 female Macaca mulatta and Macaca fascicularis, aged 10–12 years, acquired using a 3T system with the MP2RAGE sequence. 4) Stem Cell and Brain Research Institute: includes 22 scans from 9 male and 13 female Macaca mulatta and Macaca fascicularis, aged 3.6–14 years, acquired using 1.5T and 3T systems with the MPRAGE sequence. 5) University of California, Davis: includes 19 scans from 19 female Macaca mulatta, aged 18.5–22.5 years, acquired using a 3T system with the MPRAGE sequence.

For consistency with the challenge dataset, a similar preprocessing pipeline was applied to the zero-shot datasets. This included skull-stripping, affine registration to a 1 × 1 × 1 mm MNI template (Mazziotta et al., 2001), bias field correction using ANTs (Avants et al., 2009), and intensity normalization based on kernel density estimation (Reinhold et al., 2019). As the macaque brain images were initially resampled to a resolution of 1 × 1 × 1 mm but were significantly smaller than human brains, we scaled them up by a factor of 1.5. To ensure proper affine alignment, a macaque brain was first affinely registered to the MNI template and qualitatively evaluated; all remaining macaque brains were subsequently aligned to this initial macaque image. T1-weighted MRI scans from human participants were segmented into 133 cortical and subcortical anatomical structures using SLANT (Huo et al., 2019). The resulting label maps were propagated from T1-weighted scans to T2*-weighted and FLAIR scans. For non-human primates (ie., macaques), brain segmentation was performed using nBEST (Zhong et al., 2024), an automated method that delineates nine anatomical structures. The resulting label maps from the segmentation algorithms were reviewed by the challenge organizers, who have extensive experience in neuroimaging research. There is no overlap between the Challenge Training Dataset and the Zero-shot Dataset.

2.2. Comparisons

We compute a range of comparisons to evaluate registration performance that assess the accuracy, robustness, plausibility, and speed of the algorithms. We specifically use the following:

• DSC: The Dice similarity coefficient (DSC) (Dice, 1945) measures the overlap of two sets of segmentation labels. The DSC provides a reference for the general accuracy of the methods, although it has limitations (Carass et al., 2020) specifically related to registration evaluation (Rohlfing, 2011).

• DSC30: To assess robustness, DSC30 considers the 30^th percentile of DSC scores on all anatomical structures and cases. It serves to evaluate the methods’ performance on challenging scenarios, representing the lower tail of the distribution.

• HD95: The Hausdorff distance (HD) (Hausdorff, 1914) indicates the maximum distance in a metric space between two sets, in this case segmentation labels of the fixed and warped moving images. For a robust score, it is customary to consider the 95^th (HD95) percentile instead of the maximum distance.

• TRE: The target registration error (TRE) is defined as the mean of the Euclidean distance across all corresponding landmarks in the fixed and warped moving images.

• TRE30: Similarly to the DSC30 score, the TRE30 reports the 30^th percentile of the largest distances from the landmark.

• NDV: The non-diffeomorphic volume (NDV) (Liu et al., 2024a) quantifies the size of the non-diffeomorphic space in 3D caused by a digital transformation. It avoids the incorrect estimation arising from the central differences based Jacobian determinant computation by computing the volume of folded space in 3D. Since larger brains inherently contain more voxels and are therefore more likely to exhibit higher absolute NDV values independent of algorithmic performance, we report the %NDV relative to the total brain volume. Ideally, it would have a value of zero, as a non-zero value indicates that there has been a fold in the transformation between the two images, which is not physically feasible.

• ACC: To provide a summary statistic of registration accuracy (ACC), we pool the rankings of DSC, HD95, and TRE (when available). ACC is intended as a summary measure across complementary measures of registration accuracy.

• Runtime: The average runtime of each method is computed from ten repeated measurements performed on CPU, including image I/O operations. This ensures fair comparisons with traditional optimization-based methods, which typically have CPU-only implementations. All experiments were conducted on a Linux workstation equipped with an Intel^® Xeon^® Silver 4410Y processor and 535 GB of RAM. We note that, in practical settings where GPUs are widely available, participant methods and baseline models with GPU acceleration can achieve substantially lower runtimes. Accordingly, the reported CPU runtimes are intended to provide a relative point of reference rather than to reflect deployment performance, and were not factored into the final ranking.

In line with previous editions of the Learn2Reg Challenge (Hering et al., 2023), which followed the standard set by the Medical Decathlon (Antonelli et al., 2022), we rank the methods using the Wilcoxon signed-rank test. Specifically for each of DSC, HD95, TRE, and NDV, all methods are compared against each other using the paired Wilcoxon signed rank test with a one-sided alternative hypothesis, a per comparison α = 0.05, and no adjustment for multiple comparisons. The ranking for DSC, HD95, TRE, and NDV, is then determined based on the number of comparisons a method “won”, with the number of wins for each method assigned to values between 0.1 and 1 with the possibility of multiple methods having the same rank. Finally, a task rank score is obtained by taking the geometric mean of the individual rank scores, which determines the ACC ranking. An almost identical procedure is used to rank the methods based on their DSC30 and TRE30, with the modification that an unpaired Wilcoxon rank-sum test (Mann–Whitney U test) is used. The unpaired test is used, as the 30^th percentile of data represented in the DSC30 or TRE30 is different between methods. The rankings for each of the methods across the challenge test set and zero-shot evaluations are reported in Fig. 4. The summary results for each of the evaluations for the challenge test set are presented in Table 2.

Fig. 4.

Shown for each subtask is a ranking for registration accuracy (ACC), 30^th percentile of Dice Similarity Coefficient (DSC30), non-diffeomorphic volume (NDV), and 30^th percentile of total registration error (TRE30), across all 21 reported methods and the baseline of no registration (ZeroDisplacement ). The reported rankings aggregate all zero-shot sub-tasks within each category (inter-subject, atlas-to-subject, and subject-to-atlas), including unseen MRI contrasts and intra-species macaque registration used to evaluate cross-species generalization of human-trained models. Detailed per-dataset results are reported in the Appendix. For brief descriptions of the evaluated methods see Sec. 4, with more detailed descriptions in Appendix A. A description of the ranking scheme is given in Sec. 2.2, with a detailed example of the ranking in Appendix B.2.

Table 2.

Shown for the challenge test set are the mean and standard deviation for the DSC, HD95, TRE, NDV, DSC30, and TRE30. Also shown is the ranking based on registration accuracy (ACC) which for the Testing set is a combination of the rankings of DSC, HD95, and TRE. For a complete description of the reported statistics and the how the ranking scheme was computed see Sec. 2.2, with a detailed example of the ranking in Appendix B.2.

3. LUMIR Challenge Setup

The LUMIR challenge consisted of three phases (mainly organized on grand-challenge.org).

• Phase 1 – Validation Phase: The challenge began on April 1^st, 2024, allowing participants to download the training and validation datasets and train their registration networks using the training images. Participants then generated displacement fields for the validation cases, which were subsequently submitted and automatically evaluated via grand-challenge.org. Participants whose submissions ranked within the top 10 on the leaderboard were invited to provide method descriptions by June 24^th, 2024, and present posters at the Workshop on Biomedical Image Registration (WBIR), held during the 27^th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2024).

• Phase 2 – Test phase: After the validation phase, participants had until September 8^th, 2024 to submit a Docker container to the organizers. This container included their registration models in executable form, designed to read input image pairs from a provided file listing all test set pairs, and subsequently generate the corresponding displacement fields. The organizers evaluated these displacement fields using both label maps and anatomical landmarks. The performance metrics and final rankings were then computed and presented at the MICCAI conference.

• Phase 3 – Further analysis phase: After the challenge concluded at MICCAI 2024, the organizers conducted further analyses of the submitted algorithms by evaluating their zero-shot performance on out-of-domain datasets, whose data distributions differed from those used during training. The results and findings of this analysis are summarized in this paper.

The LUMIR challenge attracted a total of 39 participants across 31 teams, each submitting at least one set of computed displacement fields during the validation phase. Collectively, these participants contributed 305 valid entries to the challenge leaderboard, see Table D.1 for a summary of the number of submissions from each team. In the subsequent test phase, 12 teams submitted their trained models for evaluation. Notably, while one team consisted of industry professionals, the remaining teams were affiliated with academic institutions.

4. Methods

4.1. Submissions

The evaluation phase of the challenge received submissions from 11 teams, of which only one employed a purely optimization-based method; the remaining ten submissions were deep learning-based. Among these, two incorporated instance-specific optimization, where the deformation fields predicted by the network were further refined through pairwise optimization at inference time. All submitted methods employed cross-correlation as the similarity metric for registration, with VROC using normalized cross-correlation (NCC) computed over the entire moving and fixed images, whereas all others used local NCC (LNCC) as the similarity metric. All participant methods were required to be trained or developed using the LUMIR dataset. The use of external data was permitted only if explicitly disclosed; based on the submitted method descriptions, none of the participants relied on external datasets. Some approaches, specifically lukasf and LoRA-FT , were initialized from previously pretrained models but were subsequently retrained or fine-tuned using the LUMIR dataset.

In addition to participant submissions, we evaluated ten widely used baseline methods, including traditional optimization-based approaches such as ANTsSyN (Avants et al., 2008) and representative deep learning models such as VoxelMorph (Balakrishnan et al., 2019a). We include an eleventh baseline method of no deformation, ZeroDisplacement, to establish how the images differ prior to registration.

We summarize all evaluated registration methods in Table 1, highlighting their image similarity metrics, deformation regularization techniques, model design choices, runtime, and availability as open-source models. For comprehensive descriptions of image similarity metrics and deformation regularization methods, readers are referred to a recent survey (Chen et al., 2024). The main architectural design choices listed in the table are defined as follows:

• Dual-Stream Encoders: Two separate encoders extracting features from moving and fixed images, which may either share weights or be trained independently.

• Coarse-to-Fine Pyramid: Models (either deep learning- or optimization-based) that generate deformation fields or perform registration in a coarse-to-fine manner. Note that this is distinct from whether the underlying network employs multi-resolution architectures.

• Progressive Registration: Models that progressively refine deformation fields. For deep learning methods, this involves using multi-stage networks, with each subsequent network refining the deformation produced by the previous stage. For optimization-based methods, progressive refinement is inherently built-in, as each iteration updates and improves the deformation estimate.

• Cost Volume: Computation of a cost volume containing similarity or distance measures (e.g., correlation or Euclidean distance) between intensity or feature values at corresponding voxels or patches in the fixed and moving images.

• Attention: Models incorporating attention mechanisms (e.g., self-attention, cross-attention, or other variants) within their architectures.

• Inverse Consistency by Construction: Models explicitly designed to guarantee inverse consistency, such that the forward deformation ϕ_A→B is the inverse of the backward deformation ϕ_B→A. In deep learning-based methods, this is often achieved by exponentiating the difference between intermediate fields computed by an auxiliary network (Greer et al., 2023b); in optimization-based approaches, symmetric optimization schemes are used, as in (Avants et al., 2008).

• Diffeomorphic Registration: Models that employ formulations (e.g., time-varying velocity fields or stationary velocity fields) to achieve diffeomorphic (smooth, invertible) deformation fields.

• Instance-Specific Optimization: Models that iteratively refine deformation fields for each specific image pair by optimizing the registration objective function individually per instance.

Table 1.

Overview of the 21 registration methods evaluated in the LUMIR challenge. The image similarity measures, regularization terms used in the registration formulation, and model design choices for each method are marked with a ●. For diffeomorphic registration, methods that do not mathematically guarantee diffeomorphisms but adopt alternative formulations that approximate them are denoted by ○ The average runtime per image pair (measured on CPU) and the public availability of the source code are also provided. The runtime for zhuoyuanw210 is omitted, as the implementation requires a GPU and cannot be fairly compared with CPU-only methods. We note that the reported runtimes are intended as points of reference; with GPU acceleration, most participant methods and several baseline models can complete registration in roughly 30 seconds or less, depending on the use of instance-specific optimization.

Following this overview, we introduce each method individually using a concise three-line summary: the first line includes a colored square (that is used in plots and figures for quick reference) and the name of the method; second is a one line summary of the method; and finally in parentheses is the Participant(s) that contributed the method. Each summary is then followed by a single sentence description of the respective method. In Appendix A, we provide more detailed descriptions of each of the submission methods.

honkamj

Symmetric, inverse consistent, and topology preserving by construction

(Joel Honkamaa & Pekka Marttinen (Honkamaa and Marttinen, 2024))

honkamj is a multi-resolution deep learning registration architecture which is inverse consistent, symmetric, and preserves topology by construction.

MadeForLife

Dual-stream encoders with a specialized decoder for coarse-to-fine pyramid and progressive registration

(Yichao Zhou & Zuopeng Tan)

MadeForLife uses a combination of GroupMorph (Tan et al., 2024) and VoxelMorph (Balakrishnan et al., 2019b) for their registration framework.

LYU1

Pyramid network with auxiliary decoder

(Hongchao Zhou & Shunbo Hu)

LYU1 proposes a shared encoder, a shared auxiliary decoder, and a fusion pyramid decoder.

next-gen-nn

Robust backbone network

(Yuxi Zhang & Hang Zhang)

next-gen-nn uses a robust backbone network based on Zhang et al. (Zhang et al., 2024a) which comprises an encoder with a series of convolutional layers and a cascaded mechanism designed to refine the prediction of deformation fields.

zhuoyuanw210

Dual stream pyramid encoder and motion decomposition transformer

(Zhuoyuan Wang & Yi Wang)

This method employs a dual-stream pyramid encoder and a GPU-accelerated motion decomposition Transformer (ModeTv2) (Wang et al., 2024) decoder.

DutchMasters

Gradient projection optimization

(Yi Zhang & Qian Tao (Zhang et al., 2024b))

DutchMasters uses gradient projection techniques to address the multi-objective optimization (MOO) challenge that is at the core of image registration.

lukasf

TransMorph refinement

(Lukas Förner & Thomas Wendler)

The authors propose a fine-tuning of a pre-trained TransMorph (Chen et al., 2022b) model to improve convergence stability and deformation smoothness.

Bailiang

Dual-stream encoders, coarse-to-fine pyramid, correlation layers, and progressive registration

(Bailiang Jian & Benedikt Wiestler (Jian et al., 2024))

Bailiang uses a ConvNet architecture with dual-stream encoders to independently extract image features from moving and fixed images, followed by a decoder that integrates these features with a coarse-to-fine optimization pyramid to progressively refine deformations across scales.

VROC

Variational Registration on Crack

(Frederic Madesta & Thilo Sentker)

VROC uses classic registration components like Demons based forces (Thirion, 1998) and image preprocessing steps to perform the registration.

TimH

ConstrICON framework multistep inverse consistency

(Tim Hable)

TimH uses the ConstrICON framework (Greer et al., 2023a) to build an inverse consistent multistep registration model.

LoRA-FT

Low-rank adaptation of foundation models

(Jin Kim & Matthew S. Brown)

LoRA-FT uses a Low-rank Adaptation (LoRA) (Hu et al., 2021) technique to adapt the uniGradICON (Tian et al., 2024) foundation models to the challenge datasets.

4.2. Baseline Methods

For each of the Baseline methods we also provide a three line summary: the first line includes a colored square (that is used in plots and figures for quick reference) and the name of the method; the second is a one line summary of the method; and finally in parentheses is the reference for the method. Following the one line summaries is a single sentence description of the respective methods. Table 1 includes additional details about the methods and complete descriptions of each of the baseline methods is included in Appendix A. Among the deep learning-based baseline methods, TransMorph , VFA , and VoxelMorph were trained using the LUMIR training data. In contrast, the remaining approaches, including SynthMorph and uniGradICON , were treated as registration foundation models, and we used the pretrained weights provided by the respective authors. Note that SynthMorph was developed specifically for brain image registration, whereas uniGradICON is a more general foundation model that has demonstrated applicability across multiple anatomical regions. All optimization-based methods do not require training and were directly applied to the test set, performing instance-specific optimization for each test image pair. All baseline deep learning based methods, except the pretrained foundation registration models (i.e., SynthMorph and uniGradICON /uniGradICONiso ), were trained on the LUMIR training data, which was brain-specific.

ANTsSyN

Optimization-based, symmetric, time-varying diffeomorphic registration

(Avants et al. (2007, 2008); Avants and Tustison (2018))

The symmetric image normalization method (SyN), packaged as part of the Advanced Normalization Tools (ANTs), is a traditional registration framework that maximizes the cross-correlation (CC) within the space of diffeomorphic maps.

deedsBCV

Optimization-based, discrete registration

(Heinrich et al. (2013))

deedsBCV uses Markov random field (MRF)-based discrete optimization strategies to address large motion of small features, sliding motions between organs, and changes in image contrast due to compression in lung CT registration.

FireANTsGreedy / FireANTsSyN

Optimization-based, GPU-accelerated, time-varying diffeomorphic registration

(Jena et al. (2024a))

FireANTsSyN and FireANTsGreedy use multi-scale adaptive Riemannian optimization for diffeomorphic image matching that overcome both the slow convergence of traditional optimization based registration and the poor out-of-distribution generalization typical of deep learning based registration.

SynthMorph

Learning contrast-agnostic registration without acquired images

(Hoffmann et al. (2021))

SynthMorph is a general strategy for learning contrast-agnostic registration, which uses a generative model to provide random label maps during raining.

TransMorph

Vision Transformer-based registration model

(Chen et al. (2022b))

TransMorph is a deep learning-based image registration framework that incorporates a Transformer-based architecture for both affine and deformable registration.

uniGradICON / uniGradICONiso

Inverse consistent, learning-based foundation registration model

(Tian et al. (2024))

uniGradICON uses gradient inverse consistency regularization (Tian et al., 2023) to train a universal foundation model for registration. uniGradICONiso incorporates instance-specific optimization through iterative fine-tuning of the network parameters for each image pair.

VFA

Dual-stream encoders, vector field cross-attention decoder, multi-scale registration model

(Liu et al. (2024b))

VFA uses neural networks to extract multiresolution feature maps from the fixed and moving images and then retrieves pixel-level correspondences based on an attention module without the need for learnable parameters.

VoxelMorph

U-Net-based registration model

(Balakrishnan et al. (2019a))

VoxelMorph is a learning-based framework for deformable, pairwise medical image registration that uses parameterized CNNs.

5. Results

In the following, we summarize the results from both the challenge test set (Sec. 5.1) and the zero-shot out-of-domain datasets (Sec. 5.2). Figures 2 and 3 present representative qualitative results from the challenge test set and the zero-shot evaluation tasks (respectively), showcasing the outputs of the top three performing methods for each main task. A brief description of all the evaluated methods is provided in Sec. 4 with a detailed presentation of the methods in Appendix A, and a summary of the properties of each method is in Table 1. We note that across many of the challenge and zero-shot evaluations, the strongest methods were often closely clustered across the reported metrics. Accordingly, the rankings are most informative when interpreted together with the underlying metric values, which provide additional context on the magnitude of the observed differences.

Fig. 2.

Representative registration results from the challenge evaluation task. Shown are the moving and fixed images, the deformation fields and deformed moving images for the top three methods (selected from Fig. 4 and shown left to right in rank order), and the corresponding anatomical landmarks.

Fig. 3.

Representative registration results from the zero-shot evaluation tasks. From top to bottom the three panels show the Intersubject, Atlas to Subject (Atlas2Subject) and Subject to Atlas (Subject2Atlas) results. Each panel shows an example moving and fixed image, with the deformation fields from the top three methods (selected from Fig. 4 and shown left to right in rank order). Also shown are the corresponding anatomical label maps for the moving and fixed images, as well as the propagated anatomical label maps for the top three methods.

5.1. Challenge Test Set

The LUMIR challenge test set comprises images drawn from the same distribution as the training data, which consists exclusively of T1-w brain MRIs from healthy controls. The test set includes 590 images, of which 460 are annotated with anatomical label maps and 130 with landmark points. Using these annotations, we evaluated the registration performance based on segmentation overlap and landmark alignment. Details of the dataset are provided in Sec. 2.1.1, and a detailed description of the evaluation metrics is presented in Sec. 2.2.

The test set results are summarized in Table 2, with a more detailed ranking presented in Fig. 4. The top ten methods demonstrated comparable registration accuracy, with DSC scores ranging from 0.76 to 0.79 and TRE between 3.15 mm and 3.00 mm. Even for more challenging cases, reflected by DSC30 and TRE30, the performance differences among the top ten methods remained similar.

Overall, deep learning-based methods outperformed traditional optimization-based approaches, with all top ten methods being DL-based. Among these, eight were developed by participants, and two were baseline models. The top three methods in overall registration accuracy were honkamj (1^st), MadeForLife (2^nd), and LYU1 (3^rd). For DSC30, the top performers were honkamj (1^st), followed by MadeForLife , LYU1 , and next-gen-nn , which were all tied for 2^nd. Of particular note, next-gen-nn ranked 5^th overall, however, it performed better on the more difficult cases. For TRE30, the top three methods were MadeForLife (1^st), honkamj (2^nd), and DutchMasters (3^rd), reflecting a slight change in the classification between segmentation-based and landmark-based evaluations. The highest-ranked optimization-based method was FireANTsGreedy , which placed 12^th, followed by the participant-developed optimization method VROC in 13^th place.

In terms of deformation regularity, the widely used deep learning baseline VoxelMorph produced approximately 1% non-diffeomorphic volumes within the brain region. All other top-performing methods generated smoother fields, with the top three methods achieving an NDV well below 1%, approaching diffeomorphic quality. Notably, all optimization-based methods excelled in generating smooth deformation fields.

5.2. Zero-shot Out-of-domain Evaluation Tasks

We evaluated the zero-shot performance on a diverse set of out-of-domain datasets: (1) T1-w MRI from a different acquisition site (NIMH), (2) pediatric T1-w MRI scans from ADHD patients, (3) T1-w MRI scans with neurodegenerative pathology from ADNI, (4) intra-contrast registration for unseen MRI contrasts (T2-w, T2*-w, FLAIR), (5) ultra-high-field T1-w MRI at 9.4T, and (6) macaque brain MRI. Details of the dataset and the atlas are provided in Sec. 2.1.2 Across these datasets, we assessed inter-subject, atlas-to-subject, and subject-to-atlas registration tasks.

Performance across subtasks is summarized in Table 3, ranking methods by the number of gold, silver, and bronze medals earned for top-three placement in each evaluation setting. Consistently strong performers across zero-shot tasks include honkamj , MadeForLife , and DutchMasters . Despite the overall success of deep learning-based methods, several optimization-based methods, such as FireANTsGreedy , performed competitively in specific domains, notably in the macaque dataset where anatomical priors diverge significantly from the training data. Contrast-specific generalization remained an open challenge, as all methods exhibited a noticeable drop in performance when evaluated on unseen contrasts, especially T2*-w and FLAIR images. This is discussed in further detail in Appendix B.1.4.

Table 3.

We present a medal table for the methods that covers all the sub-tasks presented in this paper. Medals are assigned for 1^st, 2^nd, and 3^rd place based on their performance in the various subtasks. We include only the top five medal receiving methods.

To facilitate a more detailed analysis, we provide a task-wise breakdown and separate discussions for each zero-shot setting in the Appendix. These evaluations include out-of-domain T1-weighted data acquired from sources distinct from the training set (Appendix B.1.1) and MRI acquisition field strengths unseen during training (Appendix B.1.5); the application of models trained on LUMIR healthy controls to pathological brains with neurodegenerative disease (Appendix B.1.3) and across different age groups (Appendix B.1.2); inter-contrast registration involving out-of-domain MRI contrasts such as T2-, T2*-weighted, and FLAIR images (Appendix B.1.4); and evaluations under different anatomical priors using macaque brain data (Appendix B.1.6). Several of these datasets, particularly those involving out-of-domain MRI contrasts, exhibit heterogeneous image quality, including thick-slice acquisitions that were resampled to an isotropic voxel size of 1 mm. Detailed task definitions, evaluation metrics, and per-method results are provided in Appendix B.1.

6. Discussion

6.1. Deep Learning Models Excel in Zero-Shot Registration.

Deep learning models tend to degrade when evaluated on data from a distribution different from the one they were trained on (Torralba and Efros, 2011). Addressing such domain shift remains an active research area, prompting the development of specialized techniques for domain generalization (Zhou et al., 2022) and domain adaptation (Wang and Deng, 2018). In the LUMIR challenge, models were trained exclusively on inter-subject registration tasks using T1-w brain MRIs of healthy controls. The zero-shot evaluations, in contrast, exposed these models to distributional shifts including different MRI contrasts (T2-, T2*-weighted, and FLAIR), atlas-to-subject registration, images from pathological populations, and scans from non-human species.

Under these evaluation settings, several learning-based registration methods exhibited strong performance relative to traditional optimization-based approaches. In particular, methods such as honkamj , MadeForLife , and next-gen-nn consistently ranked among the top-performing approaches across zero-shot tasks. As summarized in Fig. 4, learning-based methods occupied the majority of the top-five rankings and outperformed established optimization-based baselines, including ANTsSyN , DeedsBCV , and the recently proposed FireANTs variants. While performance degradation was observed for more challenging contrasts such as T2* and FLAIR, the relative ranking of the top-performing deep learning methods remained stable.

We emphasize that these observations should be interpreted as empirical trends rather than as evidence of universal robustness. In particular, all zero-shot datasets were processed using the same preprocessing pipeline as the training data, including pre-affine alignment, resampling to a common 1×1×1 mm resolution, and cropping to a uniform image size. Deep learning-based registration methods are sensitive to preprocessing discrepancies, and their performance is not expected to generalize out of the box when preprocessing pipelines differ substantially. For example, changes in image resolution alter the effective spatial context observed by convolutional kernels or Transformer patches, leading to differences in foreground-to-background ratios and anatomical scale relative to the learned receptive fields. Such mismatches can substantially degrade performance, even when the underlying anatomy remains similar. Accordingly, the generalization observed in this study should be understood as conditional on consistent preprocessing rather than as an inherent property of the learned models.

Within this context, we offer two interpretations that may help explain the observed behavior. First, as the training data aggregates images from over 60 imaging centers it thus encompasses substantial variability in anatomy and acquisition. While this factor was not isolated through controlled experiments, the scale and diversity of the training set may contribute to the improved generalization compared to prior studies relying on more homogeneous data. Second, we hypothesize that the formulation of image registration itself may play a role. Registration emphasizes learning relative spatial correspondences between image pairs, rather than modeling the semantic content of individual images. This focus on geometric alignment may reduce sensitivity to appearance variations across contrasts, a behavior also reported in prior work such as SynthMorph (Hoffmann et al., 2021). However deep learning methods, despite their observed robustness, have clear limitations which we describe in Sec. 6.2.

Finally, despite excellent performance on zero-shot tasks, all tasks evaluated in the challenge remained within a single MRI contrast type. We anticipate that performance would degrade substantially on inter-contrast registration tasks (e.g., T1-w to T2-w MRI). Despite the comprehensive nature of our Zero-shot Dataset, there are still unanswered questions. For example, none of our presented data sources represent severe pathologies such as hemispherectomy or neuroblastoma. The pathological cases included in this study primarily involve neurodegenerative changes that preserve overall brain topology. Scenarios involving substantial topological alterations, such as large tumors or edema, would likely violate the assumptions underlying deformable registration and are beyond the scope of the present evaluation. For example, given two subjects with tumors in different brain regions, a registration algorithm should not (incorrectly) align these tumors. Likewise, when registering images between healthy controls and a subject with a tumor, there is no clear or universally accepted definition of what constitutes successful registration.

6.2. Limitations of Deep Learning-based Methods.

While learning-based methods demonstrate strong performance on unseen contrasts, several limitations remain. In this study, we show that models trained solely on large-scale T1-w images can generalize robustly to unseen contrasts and datasets. However, this evaluation relies on the assumption that zero-shot datasets undergo the same preprocessing pipeline as the training data, including voxel-size standardization via resampling and cropping to a uniform image size. Under different conditions commonly encountered in practice, such as substantially different spatial resolutions, sampling schemes, image corruptions, or input image sizes arising from non-standardized acquisition protocols, learning-based methods may struggle to operate out of the box. This limitation is particularly evident when such variations are not represented during training, as recently discussed in (Jena et al., 2024b, 2025). In this sense, learning-based methods are not yet sufficiently robust to generalize to all unseen tasks through training on T1-w images alone. Additionally, the deep-learning based approaches do have the significant drawback of requiring high-end GPUs for their training. This is mitigated in many cases as these methods provide publicly available models, however it is not clear at this time how well these pre-trained models behave when applied to a never before seen organ or other more complex registration task (ie. intermodal registration).

6.3. Optimization-based Methods: Strengths and Limitations.

Optimization-based refers to those methods that rely on pairwise optimization without any prior training stage, and they have historically seen widespread use in neuroimaging studies. Common pipelines for statistical parametric mapping (Penny et al., 2011), voxel-based morphometry (Ashburner and Friston, 2000), brain segmentation (Fischl, 2012), and computational anatomy (Younes et al., 2009) heavily depend on optimization-based methods. In the LUMIR challenge, the optimization-based methods evaluated included VROC , ANTsSyN , DeedsBCV , and FireANTs /. Despite their historical significance and conceptual appeal, these methods consistently underperformed compared to learning-based approaches.

On the challenge test set, ANTsSyN , a widely used optimization-based method, ranked 20^th out of 21 methods, outperforming only VoxelMorph . This is consistent with the original VoxelMorph publication, which reported comparable or slightly inferior performance relative to ANTsSyN . DeedsBCV followed closely, ranking 19^th. FireANTs /, a recent method introduced as a strong optimization-based alternative to deep learning models, achieved rankings of 12^th for its Greedy variant (FireANTsGreedy ) and 16^th for its SyN variant (FireANTsSyN ). Another optimization-based method, VROC , specifically tuned for the LUMIR test set by challenge participants, ranked 13^th overall.

On the zero-shot datasets, optimization-based methods demonstrated certain advantages. Because these methods optimize deformation fields independently for each image pair, they are inherently less sensitive to domain shifts. As such, ANTsSyN and FireANTs / achieved higher accuracy and modest improvements in their overall rankings (see Fig.4). Nevertheless, they consistently remained outside the top ten, with the best-performing method on each zero-shot task being deep learning-based.

Since the introduction of VoxelMorph in 2019 (Balakrishnan et al., 2019a)—when deep learning-based registration was only comparable to ANTsSyN—the field has advanced markedly. Optimization-based approaches still hold value, particularly for their well-defined properties such as symmetric deformations and explicit modeling of time-varying velocity fields. However, the deep learning methods that perform the best in the LUMIR challenge now substantially outperform the traditional techniques on a wide range of tasks.

That said, as discussed in Sec. 6.2, the zero shot datasets evaluated in this study were processed using the same preprocessing pipeline as the LUMIR challenge training data. Prior work has shown that learning-based registration methods are sensitive to preprocessing discrepancies and can underperform when applied to data processed with different pipelines (Jena et al., 2025). As a result, these methods are unlikely to generalize out-of-the-box to arbitrary zero-shot scenarios. In contrast, optimization-based approaches can be more adaptable to unseen cases, as their performance can often be recovered through careful tuning of hyperparameters or loss functions, highlighting a complementary role for these methods in practical deployment.

6.4. Identifying Effective Model Designs.

In this section, we explore common architectural patterns among the top-performing models, based on observed trends, as direct ablation studies are not feasible. Notably, the top five models on the challenge test set—honkamj , MadeForLife , LYU1 , VFA , and next-gen-nn —all adopted a dual-stream encoder design. In this setup, separate encoders are used to extract features from the moving and fixed images independently, rather than jointly feeding both images into a shared encoder. This design encourages each encoder to focus on extracting features specific to its input, without early fusion of information across the image pair. The responsibility of identifying differences between the two images is thus shifted to the decoder, which estimates the deformation field based on the contrast between the separately encoded feature maps. The results suggest a strong correlation between this architectural design and effective registration performance. Another common feature among these models is a coarse-to-fine strategy for deformation estimation, where the deformation fields are estimated at multiple resolutions and then composed to form the final output. This hierarchical approach is beneficial for capturing both global and local deformations effectively.

Additionally, progressive registration appears to play a key role in promoting generalization across domains. Models employing this design—where the model iteratively estimates full-resolution deformation fields, gradually refining the alignment by progressively warping the moving image toward the fixed image over multiple steps—tended to maintain consistent performance between the challenge and zero-shot tasks. In contrast, models that did not adopt progressive registration, such as LYU1 and next-gen-nn , exhibited notable performance drops, often falling out of the top ten on zero-shot tasks.

Notably, DutchMasters and Bailiang , both employing a progressive registration design, ranked 7^th and 11^th, respectively, on the challenge test set and showed substantial gains on zero-shot tasks. DutchMasters , in particular, placed among the top three in atlas-based registration. These results highlight progressive registration as a promising strategy for developing robust and generalizable registration models, which warrants further investigation through controlled ablation studies.

In this challenge, several deep learning-based registration foundation models were evaluated as baseline methods, including uniGradICON /uniGradICONiso and SynthMorph , the latter of which was developed specifically for brain registration. We note that these models were used as pretrained baselines and were not trained on the LUMIR dataset. On the challenge test set, they did not outperform the strongest challenge-specific methods and generally ranked in the lower half of the evaluated methods. Compared with the top-performing method, honkamj , the mean DSC was lower by 0.063 for SynthMorph and by 0.043 for uniGradICON . Even with instance-specific optimization enabled, uniGradICONiso ranked 9^th overall. On the zero-shot evaluations, these models largely retained similar relative ranks, with the notable exception of uniGradICONiso , which achieved third place overall on the atlas-to-subject tasks across the zero-shot datasets. However, the strongest challenge-specific deep learning methods also generalized well to the zero-shot settings and continued to outperform the evaluated foundation models in most cases. Taken together, these results suggest that, in this benchmark, challenge-specific models remained the most competitive overall, while pretrained foundation models are an attractive alternative in settings where developing or retraining a task-specific model may not be practical. Additionally, as uniGradICON was trained beyond brain registration, it may offer broader utility across anatomical regions.

6.5. Impact of Instance-specific Optimization on Registration Performance

Learning-based methods can further refine the deformation field at test time by iteratively optimizing the registration loss function for each specific image pair. This process, known as instance-specific optimization (ISO), treats the DNN as an implicit optimizer corresponding to an unrolled iterative optimization procedure (Gregor and LeCun, 2010). Several previous studies have shown that ISO can significantly improve the performance of learning-based registration models (Balakrishnan et al., 2019a; Tian et al., 2024).

The top-performing methods on both the “in-domain” challenge test set and the “out-of-domain” zero-shot tasks did not employ ISO. Among ISO-based approaches, the best-performing method was DutchMasters , achieving the rank of 7^th on the challenge test set. For the zero-shot tasks, DutchMasters exhibited enhanced robustness, even reaching top performance in several instances. Such improvements could be attributed to ISO bridging performance gaps caused by data distribution shifts.

Although direct ablation studies were not feasible for participant-submitted methods, comparison between uniGradICON and uniGradICONiso highlights the benefit of the ISO component. Incorporating ISO improved the rankings by five places (14^th → 9^th) for the challenge test set, alongside noticeable gains for the zero-shot set. However, a clear disadvantage of ISO is its substantial computational burden, resulting in significantly increased runtimes. For instance, the runtime for honkamj was 56.87s per image, compared to 1604.29s for DutchMasters using a CPU, although employing GPUs is expected to significantly alleviate this runtime overhead. Overall, these findings emphasize that while ISO can enhance robustness and model performance, optimal DNN design that integrates inductive bias relevant to image registration continues to play a critical role in achieving the best performance.

6.6. Inverse Consistency.

Let the deformation field that maps image A to image B be denoted as ϕ_A→B. Inverse consistency (IC) requires that the inverse transformation ϕ_B→A to satisfy that ${\varphi }_{A\to B}\circ {\varphi }_{B\to A}^{-1}={\varphi }_{B\to A}^{-1}\circ {\varphi }_{A\to B}=\mathit{\text{id}}$, where id is the identity transformation. The rationale for the importance of IC is the preservation of anatomical landmarks when moving between the images of two patients; several other benefits and implications of IC that are discussed in greater detail in (Chen et al., 2024; Greer et al., 2023b). Without explicit architectural designs or dedicated loss terms enforcing IC, neural networks typically generate different deformation fields when the order of the input images is swapped, even when the generated deformation fields are diffeomorphic.

To study IC, we computed the mean absolute error (MAE) between the composed deformation fields (forwards and backwards) and the identity map. Specifically, we evaluated the performance of each method in paired subtasks, “atlas-to-subject” and “subject-to-atlas”, across the zero-shot datasets. These tasks differ only in the order of input images; thus, an inverse-consistent method should produce deformation fields whose composition approximates the identity transformation. A lower MAE indicates stronger IC, with a value of zero corresponding to perfect consistency. Table 4 summarizes these results, with scores averaged across five datasets: ADHD, ADNI1.5T, ADNI3T, NIMH, and UltraCortex.

Table 4.

Mean absolute error (MAE) quantifying the deviation of the composed forward and inverse deformation fields from the identity, computed as |ϕ_A→B ∘ ϕ_B→A − Id|. The forward and inverse fields are obtained from the “Subject to Atlas” and “Atlas to Subject” tasks, respectively. The dagger (†) next to ANTsSyN and FireANTsSyN indicates that both the forward and the inverse deformation fields were generated from a single symmetric run on the “Subject to Atlas” task, whereas entries without the dagger correspond to MAE values computed from deformation fields obtained in separate runs for the two tasks.

In the LUMIR challenge, four evaluated methods explicitly guarantee IC by design: honkamj , TimH , ANTsSyN , and FireANTsSyN (see Table 1). Specifically, honkamj and TimH achieve IC through an auxiliary network structure that computes deformation fields by exponentiating the difference of intermediate fields generated with reversed input orders. In contrast, the optimization-based methods ANTsSyN and FireANTsSyN ensure IC by simultaneously optimizing forward and inverse deformation maps in a symmetric manner. For these optimization-based methods, we report MAE values computed from the forward and inverse maps obtained from the “subject-to-atlas” task alone, since both maps can be generated within a single run of the algorithms. We additionally report MAE values computed from separate runs, where the forward map was obtained from the “subject-to-atlas” task and the inverse map from the “atlas-to-subject” task with swapped input order. Because optimization-based methods are sensitive to initialization, IC is not necessarily preserved across separate runs.

None of the methods that theoretically guarantee IC achieved perfect IC, as indicated by non-zero MAE values. Minor deviations are observed, likely due to interpolation errors introduced during the resampling step. Nevertheless, these methods consistently exhibited the lowest MAE values, with honkamj , TimH , ANTsSyN , and FireANTsSyN achieving mean MAEs of 0.049, 0.094, 0.057, and 0.098 mm, respectively.

In addition, four methods encourage IC through the GradICON regularizer (Tian et al., 2023): LoRA-FT , DutchMasters , uniGradICON , and uniGradICONiso . However, as noted in (Tian et al., 2023), GradICON represents a relaxed version of IC regularization, penalizing derivatives of deformation rather than strictly enforcing inverse consistency. Nonetheless, as observed in Table 4, their MAE values typically fall within the 0.3–0.5 mm range, which is the lowest among methods that do not explicitly enforce IC, while other methods without such regularization generally exhibit MAE values above 0.5 mm.

Interestingly, the best-performing method in terms of IC is SynthMorph , achieving an MAE of 0.052. This improvement is consistent with updates in the latest SynthMorph model, which explicitly enhanced inverse consistency, as noted in its Docker release documentation .

6.7. Correlation between DSC and TRE

Torsten Rohlfing’s 2012 paper (Rohlfing, 2012) makes a compelling argument for not relying on tissue label overlap to evaluate registration accuracy. Given the broad range of registration methods available to us as organizers and the presence of both labels and landmark data in our Test Set, we have the opportunity to return to Rohlfing’s premise and see if we can offer any further insight. We choose to see if there is any correlation between DSC and TRE on the challenge test set. We computed the Pearson correlation coefficient between the mean DSC and the mean TRE across our challenge test set, as the slope of the linear fit. The correlation and slope are reported in Table 5.

Table 5.

The Pearson correlation coefficient (Correlation) and slope of a linear fit between the mean Dice Similarity Coefficient and the mean Total Registration Error on the Challenge Test dataset.

The slopes, in Table 5, are all negative, which is expected—as DSC improves, it goes up and TRE goes down, and vice versa for worsening results. However, the correlations are poor, indicating that the relationship between DSC and TRE is not particularly strong. This is not particularly surprising; first the landmarks are sparse, and the labeled structures run the gambit of small to large. Second, the landmarks are not in any reasonable correspondence with the available labels. As such, this simple analysis has informed us about how to better evaluate the relationship between DSC and TRE, if any such relationship exists. Moving forward with future challenges, we anticipate using several dedicated landmarks that are readily focused on specific structures, with the landmarks capturing the important boundary positions of the structures under investigation.

6.8. Deformation Regularity.

Deformation regularization plays a crucial role in ensuring that estimated transformations are anatomically plausible. Without appropriate constraints, deformable registration algorithms may produce biologically unrealistic mappings, such as foldings, even when achieving high image similarity. While the assumption of smooth, invertible deformations may not hold in cases of severe brain pathology, it is well-justified in the context of this challenge, which focuses primarily on healthy subjects and individuals with neurodegenerative conditions such as Alzheimer’s disease and mild cognitive impairment. In these populations, overall brain topology is largely preserved, and the transformations used to register them are expected to be diffeomorphic.

To quantify deformation regularity, we compute the percentage of non-diffeomorphic volume (%NDV) (Liu et al., 2024a) for each estimated transformation, defined as the percentage of folded volume within the total brain volume. A zero %NDV confirms the absence of folding, but does not imply that the registration is accurate. Conversely, a non-zero %NDV provides direct evidence of anatomically implausible behavior in the affected region(s). However, the practical interpretation of %NDV depends not only on whether it is non-zero, but also on the extent of the folded volume. Therefore, %NDV should not be interpreted as a purely binary measure in which any non-zero value is equally problematic. Although non-zero folded volume is undesirable in principle, prior literature does not establish a universal threshold for when such behavior becomes practically unacceptable; rather, acceptable tolerance depends on the intended application and the extent of the affected volume. For example, in radiotherapy applications (Nenoff et al., 2023), suggested tolerances for deformation smoothness at locations with negative Jacobian have been discussed on the order of 0.2% to 2.0%, highlighting the application-dependent nature of such criteria.

Across the challenge test set and the zero-shot datasets, the observed %NDV values fall into several practical regimes. First, some methods consistently produced exactly diffeomorphic transformations in practice, including the deep learning-based TimH and the optimization based ANTsSyN , both of which achieved %NDV=0 across all evaluated cases. Second, several methods yielded extremely small non-zero %NDV values, including FireANTsSyN , SynthMorph , uniGradICON , DeedsBCV , and next-gen-nn ; these values are effectively negligible in practice and indicate only minimal local folding. Third, a group of methods, including honkamj , DutchMasters , zhuoyuanw210 , MadeForLife , LYU1 , and Bailiang , exhibited small but measurable %NDV values, indicating that folding was present but remained spatially limited. In contrast, methods such as lukasf , VROC , VFA , TransMorph , and especially VoxelMorph , produced substantially larger %NDV values, corresponding to more extensive non-diffeomorphic volumes. Accordingly, the NDV rankings should be interpreted with care, especially among methods whose %NDV values are all very small and close to zero, as differences in numerical rank in this regime may correspond to only minor practical differences in the extent of folded volume.

To aid interpretation, we relate representative %NDV values to absolute folded volume. For a typical skull-stripped brain volume of 1,886.54 cc in 1×1×1 mm³ MNI space, a %NDV of 0.01% corresponds to approximately 0.19 cc of non-diffeomorphic volume, whereas a %NDV of 1% corresponds to approximately 18.86 cc. Thus, although several methods yielded non-zero %NDV values, the practical extent of folding varies substantially across methods. Very small non-zero values indicate limited local folding that is negligible relative to the full brain volume, whereas larger values reflect increasingly extensive non-diffeomorphic regions that are more likely to compromise transformation plausibility.

Across the test and zero-shot datasets, %NDV ≥ 1% was exceedingly rare. The only method reaching this regime was VoxelMorph . This suggests that, for nearly all evaluated methods, folding was either absent or spatially limited, even when explicit diffeomorphic guarantees were not built into the formulation. Additionally, methods that incorporate inverse-consistency constraints, as discussed in the following paragraph, generally performed favorably in terms of deformation regularity. While deformation regularization has traditionally been viewed as being in tension with registration accuracy, the top-performing methods on each dataset were not those with the highest %NDV. This indicates that, with thoughtful architectural or algorithmic design, it is possible to simultaneously achieve strong registration accuracy and maintain highly regular deformations.

6.9. Use of automated label maps

An additional criticism of the benchmark design is our use of automatically generated anatomical label maps to enable large-scale benchmarking of all the submitted results. We believe that this approach makes systematic and reproducible evaluation across thousands of scans feasible, however we acknowledge that it also introduces the possibility that segmentation inaccuracies may influence Dice- and HD95-based scores. As Rohlfing (2012) notes, tissue overlap measures are an imperfect surrogate for registration validation. To mitigate these limitations, we used fine-grained parcellation of the brain into 133 cortical and subcortical structures to reduces the gap between Dice and registration performance. Even if an automatic tool introduces errors, consistent errors across images can still support valid registration assessment, since overlapping mislabeled regions continue to reflect alignment accuracy. SLANT has been shown to be robust in this regard. We further complemented segmentation-based metrics with landmark-based TRE, providing an orthogonal evaluation that is independent of the segmentation quality.

6.10. Interpreting Rankings and Practical Significance

We pool DSC, TRE, and HD95 to evaluate the overall robustness of each method. The inclusion of HD95 is particularly important because it captures extreme spatial outliers and penalizes localized boundary errors that may not be reflected by a high overall DSC. Although the combined ranking provides a useful summary of general-purpose performance, many of the top-performing methods achieved very similar metric values. Therefore, small differences in rank should be interpreted together with the underlying metric magnitudes and should not necessarily be taken to imply large practical differences. Method selection should ultimately be guided by the intended downstream clinical application. For instance, applications such as radiation oncology or surgical planning may place greater emphasis on boundary accuracy and spatial correspondence, making HD95 and TRE especially relevant, whereas general volumetric estimation may place greater emphasis on DSC. It is also important to distinguish statistical significance in the rankings from practical significance in clinical settings. A statistically significant difference in TRE may have limited practical relevance if both errors are substantially smaller than the image voxel size. In contrast, differences in HD95 or DSC30 may be practically meaningful when they separate clinically acceptable performance from substantial failure on challenging cases.

7. Conclusion

The LUMIR challenge has highlighted the transformative impact of deep learning on brain MRI image registration, by presenting both promising insights and remaining challenges. The LUMIR challenge is characterized by two major focuses: large-scale dataset and unsupervised nature. By incorporating datasets with substantial distributional variations relative to the training data, the challenge establishes a rigorous benchmark for assessing model robustness and adaptability. Additionally, by deliberately excluding label maps from training, the challenge encourages the development of models that inherently learn spatial correspondences and adapt to diverse data conditions without relying on predefined labels, further promoting the creation of generalizable and scalable registration solutions. This framework also fosters innovation in the development of foundational models that can operate effectively in environments where annotated data is scarce.

From our comprehensive experiments and comparisons, we found that deep learning-based registration methods significantly outperform traditional optimization-based approaches in inter-subject and atlas-based brain MRI registration. Sec. 6 provided additional insights into the promising advancements and remaining challenges of various deformable registration approaches. Our evaluation of zero-shot registration revealed that deep learning models consistently outperform traditional optimization-based techniques, even in scenarios with substantial domain shifts. Furthermore, our results suggest an empirical trend in which architectural choices, such as dual-stream encoders, are associated with more effective registration performance; however, without controlled ablation studies it is not possible to establish whether this architectural design is the causal factor, or whether other co-occurring design decisions (e.g., training strategies, loss functions, data augmentation) play an equally or more important role. Achieving perfect inverse consistency also remains an open challenge: even methods that encourage inverse consistency by construction exhibited small but non-zero deviations. Finally, while instance-specific optimization can enhance performance in certain settings, the top-ranked single-step approaches in this benchmark achieved competitive accuracy with substantially lower runtime, suggesting—though not conclusively proving—that well-designed single-step approaches can deliver a favorable trade-off between accuracy and computational cost. Collectively, these insights from the LUMIR challenge advance our understanding of medical image registration and guide future research directions in model development and optimization.

In conclusion, the LUMIR challenge embodies the ongoing advances in medical image registration. As the field progresses, we believe LUMIR will play an important role in shaping the future of medical image registration, ultimately unlocking new possibilities for clinical and research advancements.

Appendix A. Methods

Appendix A.1. Submissions

The challenge submissions are listed in the order in which they ranked, see Table 2 for details.

honkamj

Symmetric, inverse consistent, and topology preserving by construction

(Joel Honkamaa & Pekka Marttinen (Honkamaa and Marttinen, 2024))

honkamj is a novel multi-resolution deep learning registration architecture which is inverse consistent, symmetric, and preserves topology by construction. These properties are fulfilled for the whole multi-resolution pipeline and not just separately at each resolution.

honkamj is designed to be cycle-consistent. This means that the deformation map between two images A and B is denoted by ϕ(A, B), and we want the composition of ϕ(A, B) and ϕ(B, A) to be the identity. The authors achieve this through an auxiliary network u that predicts deformations and defines ϕ as ϕ(A, B) = ϕ_u(A, B) ∘ ϕ_u(B, A)⁻¹. By construction, ϕ(A, B) = ϕ(B, A)⁻¹, up to numerical precision. This approach of using an intermediate network and creating the desired deformation field through an inverse consistent construction has been previously proposed by Estienne et al. (Estienne et al., 2021). Inverse consistency is not guaranteed by Estienne et al. as they enforce it via a loss term, whereas the formulation of honkamj is symmetric by construction. To implement this inverse consistent approach effectively, the authors encode features of the inputs separately before feeding them to the deformation extraction network ϕ.

The approach is implemented in a multi-scale manner; from the lowest resolution, features are extracted and used to create the coarsest deformation field. Then the approach recursively builds deformations between the inputs at each level using the extracted features. To ensure symmetry, two deformations are built: one deforming the first image half-way towards the second image, and the other deforming the second image half-way towards the first image. The final deformation is the composition of these two deformations at the highest resolution level. For more details, see Honkamaa and Marttinen (Honkamaa and Marttinen, 2024); we note that the model trained for the challenge used the group consistency loss over image triplets (Gu et al., 2020).

MadeForLife

Dual-stream encoders with a specialized decoder for coarse-to-fine pyramid and progressive registration

(Yichao Zhou & Zuopeng Tan)

The authors use the GroupMorph (Tan et al., 2024) structure for their approach. Using a dual encoder with shared weights to extract multilevel features for both the fixed and moving images. The encoder adopts a structure similar to that of VoxelMorph (Balakrishnan et al., 2019b). The features extracted by the dual encoder are fed into grouping update modules (GUMs) for the production of deformation fields and contextual features, where the contextual features provide explicit prior information and intergroup correspondence for the generation of the deformation field. The GUMs consist of two parts: the grouping module (GM), which calculates feature correlation and generates deformation fields, and the contextual fusion module (CFM), which fuses contextual features to facilitate inter-group communication for the grouping field estimator.

In summary, GMs use a different number of convolutional layers to compute multiple deformation subfields per level, with these subfields having different receptive fields. The intralevel multiscale fields, along with the interlevel multiscale fields, form the final deformation fields. Compared to the manner of predicting one field per level, the finer grained decomposing can further promote model optimization, thereby obtaining an optimal alignment. The CFMs facilitate collaboration among subfields within each group, the CFMs fuse the contextual features from adjacent groups, thus their intergroup communication is strengthened. This enables every group to have access to explicit prior information from the other groups while estimating the deformation fields.

LYU1

Pyramid network with auxiliary decoder

(Hongchao Zhou & Shunbo Hu)

LYU1 proposes a shared encoder, a shared auxiliary decoder, and a fusion pyramid decoder. The fusion pyramid decoder includes five scales. Except for the smallest scale, each scale is constructed from a multi-scale feature fusion block (MSFB). The pyramid decoder first merges the smallest-scale low-level features and predicts a coarse deformation field. This deformation field is then upsampled and applied to the moving feature map at the current scale. The moving feature map, the fixed feature map, and the high-level moving and fixed features from the shared auxiliary decoder at the same scale are input into the MSFB. This process removes redundant information and produces fused features, predicting the deformation field at the current scale. This process is repeated at each scale, with the final refined deformation field obtained by iteratively combining the coarse deformation fields from each scale.

next-gen-nn

Robust backbone network

(Yuxi Zhang & Hang Zhang)

next-gen-nn uses a robust backbone network based on Zhang et al. (Zhang et al., 2024a). Their network architecture comprises an encoder with a series of convolutional layers and a cascaded mechanism designed to refine the prediction of deformation fields. Specifically, next-gen-nn uses modifications to the base network, including a co-attention mechanism (Chen et al., 2021), large kernel convolutions (Jia et al., 2022), and an increased number of convolutional channels in the prediction stage of the deformation field. Additionally, next-gen-nn uses an image-guided bilateral filtering technique (Wagner et al., 2022), to refine the predicted registration results.

zhuoyuanw210

Dual stream pyramid encoder and motion decomposition transformer

(Zhuoyuan Wang & Yi Wang)

This method employs a dual-stream pyramid encoder and a GPU-accelerated motion decomposition Transformer (ModeTv2) (Wang et al., 2024) decoder. The dual-stream pyramid encoder extracts features from both the moving image and the fixed image, generating corresponding feature hierarchies for each image. The feature hierarchies are used from coarse to fine to generate deformations between the moving and fixed image at each scale level.

DutchMasters

Gradient projection optimization

(Yi Zhang & Qian Tao (Zhang et al., 2024b))

DutchMasters introduces a novel instance optimization (IO) algorithm that addresses the multi-objective optimization (MOO) challenge that is inherent to image registration. The approach applies gradient projection techniques, inspired by advances in MOO (Yu et al., 2020), to enhance the effectiveness of IO. IO effectively combines the generalization capabilities of deep learning with the fine-tuning advantages of instance-specific optimization. The authors framework uses gradient projection to mitigate conflicting updates in the MOO. The technique projects conflicting gradients into a common space, better aligning the dual objectives and enhancing optimization stability. The authors consider the components of the loss being optimized (i.e. similarity and regularization) as components in a MOO. The multiple losses can sometimes be in conflict with each other, affecting the optimization which is reflected in the direction and magnitude of the loss gradients. DutchMasters address this by using the gradient projection technique from multi-task learning (Yu et al., 2020). The conflicting direction of the loss gradients is corrected by randomly projecting one gradient to the normal space of the other. Conflicting gradients are identified by evaluating the cosine similarity between two gradients. Unlike mini-batch calculation (Yu et al., 2020), the authors focus on instance optimization, thus reducing the scope from multi-task learning to multi-objective optimization. If the gradients are not conflicting, the update will remain unchanged compared to gradient-based methods. If, however, the gradients are in conflict, one of the gradients will be randomly projected to the normal space of the other, and that gradient will be used for the update. For complete details, see Zhang et al. (Zhang et al., 2024b).

lukasf

TransMorph refinement

(Lukas Förner & Thomas Wendler)

The authors propose a fine-tuning of a pre-trained TransMorph (Chen et al., 2022b) model to improve convergence stability and deformation smoothness. Convergence stability is achieved through the Fisher Adam (FAdam) (Hwang, 2024) optimizer, and the deformation smoothness comes from the addition of gradient correlation (Hiasa et al., 2018) in the similarity measure, which improves anatomical alignment.

The authors use the FAdam optimizer to improve optimization during training. FAdam is a variant of the Adam optimizer that incorporates principles from natural gradient descent and Riemannian geometry to improve convergence and stability. The key modifications in FAdam include adjustments to momentum, bias correction, adaptive gradient scaling, and a refined weight decay that respects the geometry of the parameter space. lukasf also incorporates the gradient correlation (Hiasa et al., 2018) in their loss computation along with the local normalized cross-correlation, as part of the similarity loss between the fixed and moving images.

Bailiang

Dual-stream encoders, coarse-to-fine pyramid, correlation layers, and progressive registration

(Bailiang Jian & Benedikt Wiestler (Jian et al., 2024))

Bailiang introduces a ConvNet architecture designed with registration-specific components. It employs dual-stream encoders to independently extract image features from moving and fixed images, followed by a decoder that integrates these features with a coarse-to-fine optimization pyramid to progressively refine deformations across scales. The network also employs progressive registration to iteratively compose deformation fields to enable large deformations, while correlation layers establish voxel-wise correspondences.

VROC

Variational Registration on Crack

(Frederic Madesta & Thilo Sentker)

The Variational Registration on Crack (VROC ) toolbox employs a classic registration approach using task-specific hyperparameters and image preprocessing steps to improve performance. For the LUMIR Challenge the pre-processing includes Gaussian smoothing with a σ of 1 to reduce the noise. Intensity-based label maps are generated using a multi-Otsu thresholding with seven classes. A registration mask is created by taking the union of the fixed and moving images where pixel values are greater than zero, ensuring that only relevant regions are considered during the registration process. Deformable registration is performed for the first time using NCC forces applied to the fixed and moving label maps based on thresholds. This is followed by fine-tuning the initial registration result with Demons forces (Thirion, 1998) (on the original intensity-based images) to achieve precise alignment and handle more complex deformations.

TimH

ConstrICON framework multistep inverse consistency

(Tim Hable)

This approach addresses the challenge of unsupervised registration of anatomical structures in brain MRI by using the ConstrICON framework (Greer et al., 2023a), which is used to build an inverse consistent multistep registration model. The approach employs residual U-Nets and applies additional smoothing to the stationary velocity fields to reduce the inverse consistency error.

The approach leverages the TwoStepConsistent (TSC) operator from the ConstrICON framework. In contrast to other approaches, that utilize penalty terms during training to approximate inverse consistency, ConstrICON achieves this property intrinsically through its architectural design and by parameterizing the output transformations of the neural networks by a Lie group. Parameterizing the output of a neural network by a Lie group is assumed to be sufficient, since many types of transformations that are used in medical image registration, such as affine or diffeomorphic transforms, are Lie groups. The TSC operator preserves inverse consistency by construction—see Greer et al. (Greer et al., 2023a) for complete details—and can be recursively applied, thereby extending the procedure from a two-step registration to an N-step approach that remains inverse consistent. The proposed model uses three residual U-Nets, denoted as $\tilde{\Xi },\tilde{\Phi }$, and Ψ, each producing an intermediate transformation to align the moving image m to the fixed image f. These networks are integrated into a three-step architecture of the form TSC{Ξ, TSC{Φ, Ψ}}. Here, Ξ and Φ are defined by composing the outputs of $\tilde{\Xi }$ and $\tilde{\Phi }$ with themselves, e.g., $\Phi (m,f)=\tilde{\Phi }(m,f)\circ \tilde{\Phi }(m,f)$. This ensures that $\tilde{\Xi }$ and $\tilde{\Phi }$ each learn only the square root of the corresponding transformation required by the TSC formulation. In contrast, Ψ directly predicts a full intermediate transformation. The stationary velocity field is smoothed via average pooling (kernel size 3, applied twice), which empirically reduces inverse consistency error. Each registration network also includes selective Jacobian determinant regularization (Mok and Chung, 2020) to mitigate folding. The final output of the combined registration networks is regularized using bending energy loss (Johnson and Christensen, 2001) and selective Jacobian determinant regularization loss, with LNCC as the similarity metric.

LoRA-FT

Low-rank adaptation of foundation models

(Jin Kim & Matthew S. Brown)

Adapting uniGradICON (Tian et al., 2024) foundation models to new domains or datasets typically requires extensive fine-tuning of millions of parameters, which is computationally expensive and can lead to catastrophic forgetting. LoRA-FT addresses this limitation by using Low-rank Adaptation (LoRA) (Hu et al., 2021) to achieve parameter-efficient adaptation. Rather than fine-tuning all network parameters, LoRA-FT decomposes weight updates into low-rank matrices, reducing the number of trainable parameters by orders of magnitude while maintaining registration performance. The approach enables rapid adaptation of the foundation model to any dataset without compromising its general registration capabilities or requiring direct supervision of anatomical label maps.

Appendix A.2. Baseline Methods

For each of the Baseline methods we also provide a three line summary: the first line includes a colored square (that is used in plots and figures for quick reference) and the name of the method; second is a one line summary of the method; and finally in parentheses is the reference for the method. Following the one line summaries is a brief overview of the respective methods and Table 1 includes additional details about the methods. All baseline deep learning based methods, except the pretrained foundation registration models (i.e., SynthMorph and uniGradICON /uniGradICONiso ), were trained on the LUMIR training data, which was brain-specific.

ANTsSyN

Optimization-based, symmetric, time-varying diffeomorphic registration

(Avants et al. (2007, 2008); Avants and Tustison (2018))

Originally proposed circa 2007 (Avants et al., 2007, 2008), the symmetric image normalization method (SyN), now packaged as part of the Advanced Normalization Tools (ANTs), is a traditional registration framework that maximizes the cross-correlation (CC) within the space of diffeomorphic maps using Euler-Lagrange equations during optimization.

ANTsSyN uses a symmetric, diffeomorphic formulation in which the local normalized CC similarity metric is optimized jointly in the forward and inverse directions. The symmetric energy leads to Euler–Lagrange update equations that ensure inverse-consistent registration. It works by iteratively applying transformations to align the images in a symmetric manner, thus optimizing the transformation in both directions simultaneously. The symmetry used in ANTsSyN ensures that the transformation is optimal in both directions, which removes bias from the directionality of the transformation. ANTsSyN coming from the pre-deep learning era can be computationally expensive, which is often reported as a criticism despite its continued good results on a wide range of imaging data and organs (Klein et al., 2009; Murphy et al., 2011; Tustison et al., 2011).

deedsBCV

Optimization-based, discrete registration

(Heinrich et al. (2013))

deedsBCV was designed to address three outstanding issues of lung CT registration: large motion of small features, sliding motions between organs, and changes in image contrast due to compression. The approach takes advantage of Markov random field (MRF)-based discrete optimization strategies while also keeping computational complexity tractable without reducing the registration accuracy.

deedsBCV made three distinct contributions, first the use of an image-derived minimum spanning tree as a simplified graph structure to handle the complexity of sliding motion and allowed for the estimation of the global optimum very efficiently. Secondly, a stochastic sampling approach for the similarity cost between images is introduced within a symmetric, diffeomorphic B-spline transformation model with diffusion regularization. This helps reduce the complexity of the computation by orders of magnitude and enables the minimization of much larger label spaces. Thirdly, the addition of proxy-labels, dubbed hyper-labels, are introduced which capture the local intensity variations arising within this task. deedsBCV was originally designed for lung CT registration; however, it has been shown to be quite adapt at a broad range of tasks (Hering et al., 2023).

FireANTsGreedy / FireANTsSyN

Optimization-based, GPU-accelerated, time-varying diffeomorphic registration

(Jena et al. (2024a))

Traditional registration algorithms such as ANTsSyN (Avants et al., 2007, 2008) have endured over the years due to their precision, reliability, and robustness in a wide spectrum of modalities and acquisition settings (Klein et al., 2009; Murphy et al., 2011; Tustison et al., 2011). However, these algorithms converge slowly, are prohibitively expensive to run, and their usage requires a steep learning curve, limiting their scalability to larger clinical and scientific studies. The authors of the FireANTs approach have developed a multiscale adaptive Riemannian optimization algorithm for diffeomorphic image registration. Their framework leverages mathematical correspondences to avoid expensive operations like the Riemannian metric tensor, and parallel transport of the optimization state, which are needed for implementing first-order adaptive algorithms. FireANTs can compose transformations, avoiding resampling artifacts across transformations. All of this with the aim of improving scalability in image registration.

SynthMorph

Learning contrast-agnostic registration without acquired images

(Hoffmann et al. (2021))

SynthMorph is a general strategy for learning contrast-agnostic registration. SynthMorph enables the registration of real images within and between contrasts, learning only from synthetic data. During training, SynthMorph synthesizes images from label maps and trains the registration framework to align these synthetic images. The approach uses a generative model to provide random label maps of variable geometric shapes. Conditioned on these generated label maps, they synthesize images with arbitrary contrasts, deformations, and artifacts. SynthMorph can then train with a contrast-agnostic loss that measures label overlap, rather than an image-based loss.

TransMorph

Vision Transformer-based registration model

(Chen et al. (2022b))

TransMorph is a deep learning-based image registration framework that incorporates a Transformer-based architecture for both affine and deformable registration. Its encoder consists of a three-stage Swin Transformer (Liu et al., 2021), which extracts hierarchical features from the input images. These features are then passed to a decoder that progressively refines the deformation field, composing each new estimate with the previously generated field (Chen et al., 2022a). TransMorph is trained using an unsupervised learning paradigm similar to that of VoxelMorph , with both fully unsupervised and semi-supervised variants (leveraging anatomical label maps during training) described in the literature.

uniGradICON / uniGradICONiso

Inverse consistent, learning-based foundation registration model

(Tian et al. (2024))

The authors propose to train a universal foundation model for registration. A hurdle to developing a foundation model for registration is the obstacle of training a universal registration network over many different datasets to obtain such a model. It is incredibly difficult to pick a set of registration hyperparameters (for regularizers and similarity measures) that provide good performance across a wide-range of registration tasks. The issue of different registration tasks requiring different hyperparameters is obviously in direct conflict with a universal registration network being trained with a fixed set of hyperparameters. However, this is made possible by replacing conventional regularizers with gradient inverse consistency (GradICON) regularization (Tian et al., 2023). GradICON acts as a weak regularizer that only encourages invertibility of the transform. This weak constraint, allows the network to discover what transformations are supported by the data and thereby facilitates training with the same hyperparameters across different datasets. The uniGradICONiso version incorporates instance-specific optimization, iteratively fine-tuning the network parameters for 50 iterations for each image pair.

VFA

Dual-stream encoders, vector field cross-attention decoder, multi-scale registration model

(Liu et al. (2024b))

Vector field attention (VFA ) is an extension of the previously published im2grid (Liu et al., 2022). VFA uses neural networks to extract multiresolution feature maps from the fixed and moving images and then retrieves pixel-level correspondences based on feature similarity. Retrieval of the correspondences is achieved with a novel attention module without the need for learnable parameters. VFA is trained end-to-end in either a supervised or unsupervised manner.

VoxelMorph

U-Net-based registration model

(Balakrishnan et al. (2019a))

VoxelMorph is a fast learning-based framework for deformable, pairwise medical image registration. In contrast to traditional registration methods, VoxelMorph is formulated as a function that maps an input image pair to a deformation field that aligns the input images. The underlying network is parameterized by CNNs. Both supervised and unsupervised versions have been reported in the literature.

Appendix B. Results

Appendix B.1. Detailed Results on Zero-shot Evaluation

Appendix B.1.1. Diverse T1-w MRI Data Sources

The first and smallest step we can take in establishing if these methods are approaching foundation model ability is to see how they behave on a T1-w dataset that is distinct from the available T1-w training data made available for the Challenge. In essence, this is the first (and easiest) obstacle that these methods have to overcome to be considered foundational. To assess this, we compare the results on the Challenge test set with T1-w inter-subject results from the NIMH dataset. These results are shown in Tables 2 and B.4, specifically by looking at the comparable columns of DSC, HD95, and also DSC30. The top four methods for the Challenge test set are honkamj (1^st), MadeForLife (2^nd, next-gen-nn (3^rd), and VFA (4^th). These same four methods are also the top four methods on the T1-w NIMH inter-subject data with the order changing slightly: honkamj (1^st), VFA (2^nd), MadeForLife (3^rd), and next-gen-nn (4^th). For all four methods, their DSC values are almost identical between the two datasets, the biggest change being MadeForLife with a difference of 0.003 between the mean DSCs for the two cohorts. For HD95, honkamj , MadeForLife , next-gen-nn , and VFA , all experienced an almost identical drop in performance of approximately 0.3 mm. Across nearly all the methods, there is stability in the DSC and HD95 between the two cohorts of T1-w images. With respect to the DSC, the biggest changes between the two cohorts occur with ANTsSyN and VoxelMorph , with changes of approximately 0.04 and 0.01, respectively. With three methods seeing a change of approximately 0.007 (LoRA-FT , DutchMasters , and uniGradICON ), with other methods seeing a change of 0.003 or less. For HD95 the change for all methods, except ANTsSyN , is very uniform with an approximate change of 0.3 mm; with ANTsSyN having a change of 0.5 mm. However, the most interesting observation from these results is that while the DSC results saw a drop-off in performance, the HD95 saw a uniform improvement, which is a most surprising finding.

Appendix B.1.2. Pathological Brain: ADHD

Next, we evaluate the methods on the same T1-w contrast in which they were trained but in distinct cohorts, with attention deficit hyperactivity disorder (ADHD) in this section and subjects from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) in the next. Although ADHD typically does not cause distinct anatomical changes that can be clearly observed at the individual level in structural MRI scans, we included this dataset because it focuses on a specific age group, namely children aged 8 to 12, whose brain anatomy can differ substantially from that of adults. Although the training dataset does include subjects within this age range, as shown in Sec. 2.1.1, they represent a relatively small portion of the overall data. This setup allows us to assess whether the registration models can perform well on this specific subset, despite the imbalance in age representation during training.

The detailed scores for inter-subject, atlas-to-subject, and subject-to-atlas registration are shown in Tables B.5, B.6, and B.7, respectively. For inter-subject registration, the top three performing methods are honkamj , VFA , and MadeForLife , all of which are deep learning-based. Compared to the challenge test set, we observed a slight degradation in DSC and HD95 performance, with DSC decreasing by less than 0.01 and HD95 by less than 0.05 mm. This suggests potential influences from dataset shift, as well as the focus on a younger age group. Interestingly, ANTsSyN performed notably better on this dataset than on the challenge test set, showing an improvement of 0.031 (4%) in DSC and 0.205 mm (6%) in HD95. Despite this improvement, it still did not outperform deep learning-based methods and was ranked 15^th overall.

For atlas-based registration tasks, the top three performing methods are honkamj , DutchMasters , and zhuoyuanw210 for atlas-to-subject registration, and Bailiang , next-gen-nn , and zhuoyuanw210 for the subject-to-atlas registration task. Interestingly, we observe that all methods perform better, or at least equally well (with equal scores likely due to the inverse consistency constraint enforced by some methods), on the subject-to-atlas task than the atlas-to-subject task. However, the specific reason for this performance asymmetry remains unclear.

Appendix B.1.3. Pathological Brain: ADNI

Subjects with mild cognitive impairment and Alzheimer’s disease often present with significant brain atrophy, ventricular enlargement, and other structural changes compared to healthy controls. These anatomical changes introduce a significant domain shift relative to structurally normal training data, posing substantial challenges for deep learning-based registration models. Evaluating zero-shot performance on these pathological cases provides valuable insight into how effectively deep learning-based registration models generalize beyond their training domain. Our evaluation includes both 1.5T and 3T MRI scans, allowing for an assessment of model robustness across different field strengths. The detailed scores for inter-subject, atlas-to-subject, and subject-to-atlas registration are reported in Tables B.8, B.9, B.10, B.11, B.12, and B.13.

For all three registration tasks, the top-performing methods were based on deep learning. For inter-subject registration, the top-performing methods for both 1.5T and 3T scans were honkamj , MadeForLife , and next-gen-nn . For atlas-to-subject registration, the top-performing methods across both field strengths were DutchMasters , honkamj , and VFA . For subject-to-atlas registration, the top three methods for 1.5T were DutchMasters , MadeForLife , and honkamj , while for 3T they were DutchMasters , honkamj , and Bailiang . Despite the anatomical variability associated with neurodegeneration, the top-performing deep learning-based methods consistently outperformed traditional optimization-based approaches across all registration tasks. The similar performance observed between the 1.5T and 3T results further suggest that incorporating training data from both field strengths enhances model generalizability across different acquisition protocols.

Appendix B.1.4. Out-of-Domain MRI Contrasts

Out-of-domain contrast tests include intra-contrast registration tasks within one of T2-weighted (T2-w), T2*-weighted (T2*-w), and FLAIR, all of which come from the NIMH dataset; this is an out-of-domain task as the methods have been trained on only T1-weighted (T1-w) intra-contrast registration. We want to see if the methods generalize well to unseen contrasts and maintain the same level of accuracy as the in-domain contrast. Using DSC and HD95 as our proxies for accuracy, we see an almost uniform drop in the performance characteristics of the various methods between the intra-contrast registration on the T1-w data and the intra-contrast registration with the other contrasts. We see this based on comparing the intra-contrast results on the T1-w data (Table B.4) with any of the intra-contrast results for the other three contrasts (Table B.14 for T2*-w, Table B.15 for T2-w, or Table B.16 for FLAIR). The only exception is DeedsBCV , which had slightly better mean DSC and HD95 for intra-contrast registration with the T2-w images than it had for intra-contrast registration with the T1-w images.

A key observation from examination of Tables B.4 through B.16, is that while all methods see a degradation in performance (except DeedsBCV in one sub-task), the impact varies considerably; this is readily discernible from the change in the ordering of the methods in these tables. Consider the top three performing methods with respect to the T1-w inter-subject task, they are honkamj , VoxelMorph , and MadeForLife . However, the same methods have dramatically different rankings (and as noted above poorer performance) on the inter-subject task for the T2*-w images. Specifically, honkamj drops to 7^th, VoxelMorph falls dramatically to 20^th, while MadeForLife manages to move up to rank 2^nd despite its own drop in performance; this is due to the much greater performance drop of the other methods. Similar changes in the rankings are obvious when considering any methods across the NIMH collection of sub-tasks. It might be argued that the issue is not with the methods, but rather with the choice of statistics; however, this is not borne out by a review of the statistics used or by replacing those statistics with more robust counterparts. Firstly, the issue of the inconsistency in the rankings exists in HD95 which itself is already a robust statistic on the Haudorff distance. Secondly, if we look at the DSC30 which is a robust statistic on Dice score, we again see drops in performance and large changes in the rankings between the T1-weighted contrast and any of the other contrasts. For example, the DSC30 on the intra-contrast sub-task on the NIMH data has the top three methods being honkamj , MadeForLife , and next-gen-nn . While for the T2*-w registration sub-task, with respect to DSC30, honkamj drops to 7^th, MadeForLife moves up to 1^st, and next-gen-nn drops to 9^th. In fact, for each of the inter-subject sub-tasks on the NIMH data, the ranking of the methods for both DSC and DSC30 is almost identical; which is not the case when looking at rankings between the different sub-tasks. These drops in performance and chaotic changes in the ranking when applying the methods to the NIMH zero-shot tasks highlight that there is still work to be done to establish foundational models for registration that are robust to imaging contrast.

Appendix B.1.5. Ultra-High-Field MRI

Recent advances in high-field and ultra-high-field MRI technologies are enabling neuroimaging at substantially improved spatial resolution and tissue contrast compared to conventional 1.5T or 3T MRI scanners. These improvements allow for better separation of fine anatomical structures, but they also introduce domain shifts in image appearance that can challenge registration algorithms. In neuroimaging studies using high- or ultra-high-field MRIs, accurate registration is critical, and it is desirable for top-performing deep learning-based registration models to generalize to these new acquisition settings without requiring retraining.

To assess the robustness of the submitted models, we evaluated their zero-shot performance on the UltraCortex 9.4T dataset across three registration tasks: inter-subject, atlas-to-subject, and subject-to-atlas registration. The results of each task are summarized in Tables B.17, B.18, and B.19, respectively.

Inter-subject registration is inherently the most challenging due to anatomical variability. As expected, overall Dice scores were lower across the three subtasks. In the inter-subject registration task, honkamj achieved the top performance, leading both in terms of DSC and HD95, earning 1^st place overall. Close competitors included MadeForLife and VFA , which also demonstrated high DSC scores and reasonably low HD95 values, indicating strong spatial alignment. In particular, VFA achieved these results with a relatively high NDV, suggesting some trade-off between anatomical plausibility and spatial accuracy.

In the atlas-to-subject registration task, honkamj again led with the highest accuracy ranking, while maintaining anatomical plausibility with a low NDV. MadeForLife and VFA closely followed in performance, showing similarly high DSC scores and low HD95 values, affirming their robustness across tasks. Notably, VFA ’s relatively high NDV shows again its tendency towards less regularized deformation.

The subject-to-atlas registration task shows slightly better performance across methods compared to atlas-to-subject registration, indicating that the direction of registration affects accuracy. MadeForLife claimed the highest accuracy ranking, closely followed by next-gen-nn and VFA . These top-performing methods consistently achieved high overlap and low boundary errors, while generally keeping NDV within acceptable ranges. Notably, next-gen-nn had one of the lowest NDVs, which emphasizes its effectiveness in preserving diffeomorphic properties while delivering high accuracy.

Overall, consistent top performers across all tasks include honkamj , MadeForLife , and VFA , which show robust registration accuracy and good diffeomorphic behavior. Registration accuracy declines from subject-to-atlas to atlas-to-subject, emphasizing the importance of evaluating across multiple directions. Deep learning methods outperform traditional methods in alignment metrics, but often at the cost of increased non-diffeomorphic deformations, unless specifically regularized.

Appendix B.1.6. Non-Human Species

The macaque brain inter-subject registration task involves anatomical structures that differ significantly from those in the human brain dataset. Evaluating on this dataset introduces a domain shift in structural anatomy and imaging content, beyond the intensity differences assessed in the out-of-domain contrast experiments. However, due to the differences in anatomical structures, it is not feasible to evaluate the alignment of the same types of anatomical labels as those used in the human brain datasets. Additionally, the annotations in the macaque dataset are relatively coarse, making direct performance comparisons with human brain registration results inappropriate.

Given these limitations, we discuss the results of this task separately and focus on comparing the methods directly. As shown in Table B.20, without registration (i.e., ZeroDisplacement ), the DSC is already 0.665. Most methods perform reasonably close to each other, with DSC values ranging from 0.75 to 0.78 and HD95 from 4.6 mm to 4.3 mm. An exception is VFA , which, despite consistently ranking in the top ten on human brain tasks, was the only method to fall below 0.7 in DSC and above 5 mm in HD95, ranking lowest among all methods on this task.

The three best performing methods on the macaque dataset are MadeForLife , FireANTsGreedy , and honkamj . Notably, this is the only task in which the optimization-based method FireANTsGreedy ranked among the top three, outperforming several deep learning-based methods that had surpassed it in the other tasks.

Appendix B.2. Explaining the Ranking

We provide a simplified example to explain how the rankings are determined. Consider the case in which we have three algorithms A, B, & C and we are evaluating these algorithms with respect to two measurements I & II. Using the Wilcoxon signed rank test (WSRT) we compare algorithm A against B with respect to measure I. Algorithm A has a better measure I score than B and it is significant based on the WSRT, thus A wins the comparison with B. For algorithm A and C with respect to I, we have A has a better I score than C which is significant and thus A wins the comparison with C. For algorithm B and C, B has a better I score than C which is significant and thus B wins the comparison with C. Thus for measure I, A has two wins and B has one win, thus the ranking would be A, B, C. This would mean for measure I, that A gets assigned the value 0.1 (for winning) and C get assigned the value 1.0 (for coming last). B would get a value of 0.55 for coming second in the comparison (in the instance of more methods between the first and last algorithms, the range [0.1, 1.0] is evenly divided up and values are assigned in order from 0.1 to 1.0). This is summarized in the two columns under Measure I in Table B.1.

For measure II, algorithm A is again significantly better than both B and C, and is denoted as having two wins in the appropriate column of Table B.1. Whereas neither algorithm B or C is significantly better than the other resulting in neither of them having a win. Thus the ranking with respect to measure II is A first, with B and C tied in last place. The corresponding scores are 0.1 for A and 1.0 for both B and C. It is then straightforward to compute the corresponding geometric means and determine the final ranking of the three algorithms over the two measures. This procedure was used to compute all of our reported rankings.

Appendix B.3. Detailed Quantitative Results for Each Zero-Shot Subtask

Appendix B.3.1. NIMH - T1w

Table B.2.

NIMH T1w Atlas to Subject.

Table B.3.

NIMH T1w Subject to Atlas.

Table B.4.

NIMH T1w Inter-subject.

Appendix B.3.2. ADHD

Table B.5.

ADHD Inter-subject.

Table B.6.

ADHD Atlas to Subject.

Table B.1.

A worked example of how the ranking is computed: In this instance for two measures across three algorithms. See Sec. Appendix B.2 for complete details.

	Measure I		Measure II
Algorithm	Wins	Score	Wins	Score	Geometric Mean	Final Ranking
A	2	0.1	2	0.1	0.1	1st
B	1	0.55	0	1.0	0.742	2nd
C	0	1.0	0	1.0	1.0	3rd

Table B.7.

ADHD Subject to Atlas.

Appendix B.3.3. ADNI 1.5T

Table B.8.

ADNI 1.5T Inter-subject.

Table B.9.

ADNI 1.5T Atlas to Subject.

Table B.10.

ADNI 1.5T Subject to Atlas.

Appendix B.3.4. ADNI 3T

Table B.11.

ADNI 3T Inter-subject.

Table B.12.

ADNI 3T Atlas to Subject.

Table B.13.

ADNI 3T Subject to Atlas.

Table B.14.

NIMH T2*w Inter-subject.

Table B.15.

NIMH T2w Inter-subject.

Table B.16.

NIMH FLAIR Inter-subject.

Appendix B.3.5. UltraCortex - 9.4T

Table B.17.

UltraCortex - 9.4T Inter-subject.

Table B.18.

UltraCortex - 9.4T Atlas to Subject.

Table B.19.

UltraCortex - 9.4T Subject to Atlas.

Appendix B.3.6. Macaque (Non-human Brain)

Table B.20.

Macaque (Non-human Brain) Inter-subject.

Appendix C. Statistical significance matrices for all evaluated tasks

We present the significance matrices for all evaluated tasks below. Overall differences among methods were first assessed using the Friedman test. When statistical significance was detected, pairwise comparisons were performed using Wilcoxon signed-rank post-hoc tests with Bonferroni correction, which is the most conservative p-value correction method.

Fig. C.1.

Significance matrix for pairwise method comparisons on the ADHD dataset. The upper row shows results for DSC, and the lower row shows results for HD95. Columns correspond to three registration tasks: Intersubject, Atlas to Subject (Atlas2Subject), and subject to atlas (Subject2Atlas) registration. Each cell indicates the statistical significance of performance differences between a pair of methods, assessed using Wilcoxon signed-rank post-hoc tests with Bonferroni correction. Color encodes the significance level, ranging from non-significant (NS) to p < 0.001. Only the upper triangular matrix is shown to avoid redundancy.

Fig. C.2.

Significance matrix for pairwise method comparisons on the ADNI 1.5T dataset. The upper row shows results for DSC, and the lower row shows results for HD95. Columns correspond to three registration tasks: Intersubject, Atlas to Subject (Atlas2Subject), and subject to atlas (Subject2Atlas) registration. Each cell indicates the statistical significance of performance differences between a pair of methods, assessed using Wilcoxon signed-rank post-hoc tests with Bonferroni correction. Color encodes the significance level, ranging from non-significant (NS) to p < 0.001. Only the upper triangular matrix is shown to avoid redundancy.

Fig. C.3.

Significance matrix for pairwise method comparisons on the ADNI 3T dataset. The upper row shows results for DSC, and the lower row shows results for HD95. Columns correspond to three registration tasks: Intersubject, Atlas to Subject (Atlas2Subject), and subject to atlas (Subject2Atlas) registration. Each cell indicates the statistical significance of performance differences between a pair of methods, assessed using Wilcoxon signed-rank post-hoc tests with Bonferroni correction. Color encodes the significance level, ranging from non-significant (NS) to p < 0.001. Only the upper triangular matrix is shown to avoid redundancy.

Fig. C.4.

Significance matrix for pairwise method comparisons on the UltraCortex dataset. The upper row shows results for DSC, and the lower row shows results for HD95. Columns correspond to three registration tasks: Intersubject, Atlas to Subject (Atlas2Subject), and subject to atlas (Subject2Atlas) registration. Each cell indicates the statistical significance of performance differences between a pair of methods, assessed using Wilcoxon signed-rank post-hoc tests with Bonferroni correction. Color encodes the significance level, ranging from non-significant (NS) to p < 0.001. Only the upper triangular matrix is shown to avoid redundancy.

Fig. C.5.

Significance matrices for pairwise method comparisons on the NIMH dataset. For T1-weighted images, results are shown for three registration tasks, Intersubject, Atlas to Subject (Atlas2Subject), in the upper panel separated by the dashed line. For the T2-weighted, T2*-weighted, and FLAIR images, results are shown for Intersubject registration only in the lower panel. For each panel, the upper row shows results for DSC, and the lower row shows results for HD95. Each cell represents the statistical significance of performance differences between a pair of methods, assessed using Wilcoxon signed-rank post-hoc tests with Bonferroni correction. Color encodes the significance level, ranging from non-significant (NS) to p < 0.001. Only the upper triangular matrices are shown.

Fig. C.6.

Significance matrix for pairwise method comparisons on the macaque dataset. Results are shown for Intersubject registration only. The left plot shows results for DSC, and the right plot shows results for HD95. Each cell indicates the statistical significance of performance differences between a pair of methods, assessed using Wilcoxon signed-rank post-hoc tests with Bonferroni correction. Color encodes the significance level, ranging from non-significant (NS) to p < 0.001. Only the upper triangular matrix is shown to avoid redundancy.

Appendix D. Challenge Uploads

In Table D.1, we provide a summary of the total submissions to the challenge website through August 31, 2025.

Table D.1.

Summary of submission attempts by team (as of August 31, 2025). Organizers denotes the LUMIR organizers. Others denotes teams that submitted results via the Grand Challenge platform but did not participate in the final challenge testing and competition.