Unraveling Normal Anatomy via Fluid-Driven Anomaly Randomization

Abstract

Data-driven machine learning has made significant strides in medical image analysis. However, most existing methods are tailored to specific modalities and assume a particular resolution (often isotropic). This limits their generalizability in clinical settings, where variations in scan appearance arise from differences in sequence parameters, resolution, and orientation. Furthermore, most general-purpose models are designed for healthy subjects and suffer from performance degradation when pathology is present. We introduce UNA (Unraveling Normal Anatomy), the first modality-agnostic learning approach for normal brain anatomy reconstruction that can handle both healthy scans and cases with pathology. We propose a fluid-driven anomaly randomization method that generates an unlimited number of realistic pathology profiles on-the-fly. UNA is trained on a combination of synthetic and real data, and can be applied directly to real images with potential pathology without the need for fine-tuning. We demonstrate UNA’s effectiveness in reconstructing healthy brain anatomy and showcase its direct application to anomaly detection, using both simulated and real images from 3D healthy and stroke datasets, including CT and MRI scans. By bridging the gap between healthy and diseased images, UNA enables the use of general-purpose models on diseased images, opening up new opportunities for large-scale analysis of uncurated clinical images in the presence of pathology. Code is available at https://github.com/peirong26/UNA.

1. Introduction

Recent machine learning based methods have significantly advanced the speed and accuracy of brain image analysis tasks, such as image segmentation [11, 26, 37, 41], registration [3, 9, 55], and super-resolution [46, 49]. Human brain imaging in vivo is primarily dominated by Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) [22]. CT is faster and preferred in emergency cases, while MRI provides superior contrast for soft tissues such as the brain. Unlike CT, which is a standardized modality that produces quantitative measurements in Hounsfield units, MRI is generally not calibrated and can generate a wide range of imaging contrasts (e.g., T1w, T2w, FLAIR) to visualize different tissues and abnormalities. This diversity in contrast and the lack of standardization complicate the quantitative analysis of MRI scans. As a result, most existing MRI analysis methods are contrast-specific and often suffer from performance degradation when voxel size or MRI contrast differs between training and testing datasets [52]. This limits the generalizability of machine learning models and leads to redundant data collection and training efforts for new datasets. Recent contrast-agnostic models that leverage synthetic data [5,20,23,24,33] have demonstrated impressive results, significantly extending their applicability to diverse clinical acquisition protocols. However, these models are primarily designed for analyzing healthy brain anatomy and typically struggle to produce reliable results in the presence of extensive abnormalities (Figs. 3 and 4).

Figure 3.

Qualitative comparisons on healthy anatomy reconstruction, between UNA, and the state-of-the-art modality-agnostic T1w synthesis method. Testing images are generated from real healthy subjects encoded with randomly simulated pathology profiles. Pathology regions are circled in red.

Figure 4.

Qualitative comparisons on healthy anatomy reconstruction between UNA and state-of-the-art modality-agnostic synthesis models. Testing images are from real stroke datasets (ISLES [19] and ATLAS [31]), where the stroke lesion annotations are provided, yet the ground truth healthy anatomy is unavailable. The last row shows a failure case of UNA, where it “over-corrects” the diseased anatomy. Pathology regions are circled in red.

To the best of our knowledge, the recently proposed PEPSI [34] is the only contrast-agnostic brain MRI analysis method that is compatible with extensive pathology. PEPSI leverages synthetic data to estimate T1w and FLAIR MRI from input scans containing pathology. However, it has several limitations: (i) It relies on paired pathology segmentation map associated with each brain anatomy during training, which limits its application to datasets that provide pathology annotations; (ii) It requires access to pre-trained pathology segmentation models to compute the implicit pathology segmentation loss; and (iii) It requires additional fine-tuning to detect anomalies.

Here, we introduce UNA, the first modality-agnostic learning method for Unraveling Normal Anatomy. UNA leverages the power of synthetic data, and can be applied to real images (CT and MRI) of both healthy and diseased populations, without the need for fine-tuning (Fig. 1).

1. We propose fluid-driven anomaly randomization (Sec. 3) to overcome the scarcity of pathology segmentation annotations. Using only limited existing pathology segmentations as initial conditions, our fluid-driven anomaly generator generates unlimited new pathology profiles on-the-fly through advection-diffusion partial differential equations (PDEs). This formulation offers a continuous and controllable trajectory for pathology evolution and also naturally enforces realistic constraints on brain abnormalities through boundary conditions (Fig. 1 (left)).

2. We introduce a modality-agnostic learning framework to reconstruct healthy brain anatomy from images with potential pathology (Sec. 4). Our framework leverages symmetry priors of brain anatomy and incorporates subject-specific anatomical features from contralateral healthy tissue in a self-contrastive learning fashion.

3. We extensively evaluate the healthy anatomy reconstruction performance of UNA on simulated and real images with stroke lesions, in both CT and different MR contrasts (T1w, T2w, and FLAIR) (Secs. 5.1 and 5.2). We further demonstrate the direct application of UNA to anomaly detection, without fine-tuning (Sec. 5.3). UNA achieves state-of-the-art performance in all tasks and modalities.

Figure 1.

Powered by the proposed fluid-driven anomaly randomization, UNA can handle a range of pathological patterns without requiring paired pathology annotations for training. (i) By bridging the gap between healthy and diseased anatomy, UNA enables the use of general analysis models for images containing pathology; (ii) By reconstructing anatomy in a modality-agnostic manner, UNA facilitates analysis with standard tools designed for high-resolution, healthy T1w MRI.

By bridging the gap between healthy and diseased anatomy UNA enables the use of general-purpose models for images containing pathology, unlocking the tremendous potential for analyzing clinical images with pathology.

2. Related Work

Foundation Models in Medical Imaging.

Large-scale datasets in medical imaging require significantly more effort to compile than those in natural imaging or language due to varying acquisition protocols and privacy requirements across institutions. Consequently, medical foundation models are not as well developed as their natural image counterparts. There have been, nevertheless, some notable efforts. SAM-Med3D-MoE [51] provides a 3D foundation model for medical image segmentation, trained on 22,000 scans. The MONAI [1] project includes a model zoo with pre-trained models, which are highly task-specific and sensitive to particular image contrasts. Zhou et al. [57] constructed a medical foundation model designed for detecting eye and systemic health conditions from retinal scans. Still, it only functions with color fundus photography and optical coherence tomography modalities. Recently, generalist biomedical AI systems, e.g., GMAI [39] and Med-PaLM M [44, 50], have demonstrated significant potential in biomedical tasks within a vision-language context, including visual question answering, image classification, and radiology report generation. However, they have not tackled more complex dense 3D prediction tasks such as reconstruction, segmentation, and registration.

Contrast-Agnostic Learning for MRI.

MRI scans acquired across sites vary substantially in appearance due to differences in contrast, resolution, and orientation. This heterogeneity leads to duplicate training efforts for approaches that are sensitive to specific MR contrast. Classical approaches in brain segmentation used Bayesian inference for contrast robustness [14, 29], but require long processing times and struggle with resolutions that are not high and isotropic [23, 40]. SynthSeg [5, 6] achieves contrast- and resolution-agnostic segmentation with a synthetic generator that simulates widely diverse contrasts and resolutions. The same generator has been used to achieve contrast invariance in tasks like image registration [10, 20], super-resolution [24], or skull stripping [21]. Brain-ID [33] explored contrast-agnostic feature representations that generalize across various fundamental medical image analysis tasks, including image synthesis, segmentation, and super-resolution. However, all these general-purpose methods are either trained exclusively on healthy anatomical labels, or require paired anatomy-pathology annotations, which limits their application primarily to healthy subjects or every specific pathology (e.g., white matter lesions) – as opposed to previously unseen pathology profiles (Figs. 3 and 4).

Fluid-Based Dynamics Modeling.

Fluid dynamics is a fundamental topic in physics and plays a crucial role in various real-world applications such as weather forecasting, airflow analysis [8], optical flow [45, 47], image registration [43, 48, 56], and perfusion analysis [32]. In fluid dynamics, advection-diffusion PDEs are commonly employed to describe the fluid transport processes. Liu et al. [35] introduced regularization-free representations to ensure the compressibility and positive semi-definiteness of estimated velocity and diffusion fields. Franz et al. [16] simulated 3D density and velocity fields from single-view data without 3D supervision. Xing et al. [54] proposed to learn the velocity field from past physical observations using Helmholtz dynamics, eliminating the need for ground truth velocity. In these studies, the inverse problem of velocity estimation provides interpretable insights for predicting future fluid behavior. We build upon the concept of fluid flow simulation and frame anomaly pattern randomization as a forward process of advection-diffusion PDEs. This formulation naturally enables us to ensure that simulated anomaly outcomes are well posed, through controllable velocity fields and established boundary conditions (Sec. 3.1).

3. Fluid-Driven Anomaly Randomization

Manually annotating pathology to create gold-standard segmentation is extremely costly, particularly for 3D medical images. This process not only requires specialized expertise from clinicians, but is also highly time-consuming and not reproducible. Consequently, large-scale datasets with gold-standard pathology annotations are almost inexistent (BraTS [36] being a notable exception). In addition, discrepancies often arise among the gold-standard pathology segmentation maps provided by different datasets. To address these issues, we seek to design an anomaly randomization approach that is:

1. Expressive: the generated anomaly profiles should exhibit diverse and expressive shapes and intensities that sufficiently reflect the variety of pathological appearances encountered in clinical practice.

2. Realistic: the randomized abnormalities must conform to realistic constraints. For example, abnormalities in white matter should not appear in other tissue structures, brain tumors should be localized within the brain region.

To achieve these two aims, we propose randomizing unlimited, diverse anomaly profiles by formulating the generation as a forward mass transport process, with realistic constraints naturally guaranteed by boundary conditions. Our anomaly randomization consists of three steps (Alg. 1): (i) Initializations of random anomaly (P₀), velocity (V), and diffusion (D) for anomaly transport; (ii) Forward transport of abnormal intensities for random time steps; (iii) Appearance encoding of the generated anomaly on healthy images of any modality. Sec. 3.1 below describes the generation of abnormal profiles (i-ii), and Sec. 3.2 introduces the encoding of abnormalities on healthy images (iii).

Algoritham 1:

Fluid-Driven Anomaly Randomization

3.1. Anomaly Profile Randomization

Background.

Advection-diffusion PDEs describe a large family of fluid dynamics processes, e.g., heat conduction, wind dynamics, and blood flow [8, 32, 54]. In general, the advection term refers to the mass transport driven by fluid flow, while the diffusion term refers to the gradient of mass concentration. Inspired by the advection-diffusion process, which computes the natural progression of mass intensities, we propose to randomize an unlimited variety of anomaly profiles by formulating the generation as a forward advection-diffusion, starting from either a single realistic pathology annotation map or a random shape.

Problem Setup.

Let $P(\mathbf{x},t)$ denote the pathology probability at location $\mathbf{x}$ in a bounded domain of interest $\mathrm{\Omega }\subset {ℝ}^3$ (e.g., brain), at time $t$. The local pathology probability changes of an anomaly randomization process are described by the advection-diffusion PDE:

$$\frac{\partial P(x,t)}{\partial t}=\underset{\text{Flow}}{\underbrace{-\nabla (V(x)\cdot P(x,t))}}+\underset{\text{Diffusion}}{\underbrace{\nabla \cdot (D(x)\nabla P(x,t)),}}$$ (1)

$$\mathit{s.t.}\underset{\text{Initial Condition}}{\underbrace{P(x,0)=P_0(x),}}\underset{\text{Zero-Neumann}}{\underbrace{{\frac{P(x,t)}{\partial n}|}_{\partial {\text{Ω}}_p}=0,}}t\le T_{\max },$$ (2)

where $t\left(T_{\text{max}}\right)$ refers to the (maximum) time steps used for the generation of new anomaly profiles. The spatially varying velocity field $\mathbf{V}(\mathbf{x})\in {ℝ}^3$ and diffusion scalar field $D(\mathbf{x})\in ℝ$ govern the advection and diffusion process of an initial anomaly, $P_0(\mathbf{x})$. The zero Neumann boundary condition ensures that the randomization process of $P_0$ satisfies pre-assumed bounds of the anomaly developing regions. To ensure that the dynamics of anomaly changes are well posed, we impose the incompressible flow and non-negative diffusion constraints on $\mathbf{V}$ and $D$ [35], and rewrite the advection-diffusion process in Eq. (1) as:

$$\begin{gathered} \frac{\partial P(x,t)}{\partial t}=-V(x)\cdot \nabla P(x,t)+\nabla \cdot (D(x)\nabla P(x,t)) \\ =\underset{\text{Incompressible Flow}}{\underbrace{-\nabla \times \Psi (x)\cdot \nabla P(x,t)}}+\underset{\text{Non-Negative Diffusion}}{\underbrace{\nabla \cdot \left({\text{Φ}}^2(x)\nabla P(x,t)\right),}} \end{gathered}$$ (3)

where $\mathbf{\Psi }\in L^3(\mathrm{\Omega }{)}^3$ and $\mathrm{\Phi }\in {ℝ}^{+}(\mathrm{\Omega })$ refer to the potential fields for representing $\mathbf{V}$ and $D$, respectively, such that the resulting flow and diffusion will be incompressible and non-negative by construction.

Initializations of P₀, V, D.

To enrich the diversity of abnormal profiles, we initialize the anomaly (P₀ in Eq. (2)) from two sources: (i) Publicly available pathology annotations from the ATLAS [31] and ISLES [19] stroke datasets, which include high-quality gold-standard segmentation of stroke lesions. (ii) Random shapes using randomly thresholded Perlin noise, a widely used procedural generation algorithm known for creating rich textures. We further generate random Perlin noise for creating random potentials $\mathbf{\Psi }$ for $\mathbf{V}$, and $\mathrm{\Phi }$ for $D$.

Forward Scheme.

We employ a first-order upwind scheme [30] to approximate the differential operators associated with the advection term, and a nested central-forward-backward difference scheme for the diffusion term in Eq. (3). Discretizing the spatial derivatives leads to a system of ordinary differential equations that can be solved with numerical integration. To enhance numerical stability and ensure compliance with the Courant-Friedrichs-Lewy (CFL) condition [17, 30], we apply the RK45 method for adaptive time-stepping ( $\delta t$) in advancing to $P^{t+\delta t}$.

As shown in Fig. 1 (left), we can generate infinite variations from a single pathology profile via the introduced fluid-driven anomaly transport, while naturally satisfying boundary conditions imposed by the brain contour.

3.2. Anomaly Apprearance Randomization

As mentioned in Sec. 2, large-scale annotation of 3D medical imaging data requires tremendous effort. UNA is instead trained on a combination of synthetic and real images (many of them labeled automatically). Specifically, we encode the generated pathology profiles, P, into normal anatomy of healthy control scans, enabling the generation of diverse images with random modalities, each exhibiting a distinct appearance introduced by P.

Random Modality Generation.

To generate healthy images with complex structural details, we first leverage domain randomization [33] to synthesize images of random modality and resolution with healthy anatomy (Fig. 2 (left)). Specifically, we randomly sample intensities on 3D neuroanatomical segmentation (label maps L), where the intensities are conditioned on the label at each location:

$$\{\begin{array}{l}{I}_0(x)\sim 𝒩({\mu }_l,{\sigma }_l),l\in L,\hfill \\ {\mu }_l\sim 𝒰(0,1\mid {\theta }_{\mu },{\theta }_l),{\sigma }_l\sim 𝒰(0,1\mid {\theta }_{\sigma },{\theta }_l),\hfill \end{array}$$ (4)

where ${\mu }_l$ and ${\sigma }_l$ refer to the mean and variance of the uniform distribution of each label l. ${\theta }_l,{\theta }_{\mu },{\theta }_{\sigma }\in \mathrm{\Theta }$ control the shifts and scales. A random deformation field is then generated for augmentation purposes, comprising linear and non-linear transformations [24, 33].

Figure 2.

UNA’s framework overview for modality-agnostic learning of healthy anatomy, supported by fluid-driven anomaly randomization.

Anomaly Profile Encoding.

We encode the random anomaly profiles from Sec. 3.1 into the generated healthy anatomy $I_0$, based on a priori knowledge on the white and gray matter intensities of $I_0$ [28, 34]:

$$I(x)=I_0(x)+\text{Δ}I(x)\text{*}P(x),$$ (5)

$$\mathit{s.t.}\text{Δ}I(x)\sim \{\begin{array}{ll}\{0\},\hfill & x\notin {\text{Ω}}_P\hfill \\ 𝒩(-{\mu }_{\text{w}}/2,{\mu }_{\text{w}}/2),\hfill & x\in {\text{Ω}}_P,{\mu }_{\text{w}}>{\mu }_{\text{g}}\hfill \\ 𝒩({\mu }_{\text{w}}/2,{\mu }_{\text{w}}/2),\hfill & x\in {\text{Ω}}_P,{\mu }_{\text{w}}\le {\mu }_{\text{g}}\hfill \end{array}$$ (6)

${\mu }_{\mathrm{w}}\left({\mu }_{\mathrm{g}}\right)$ is the mean of $I_0$’s white (gray) matter intensities. A higher ${\mu }_{\mathrm{w}}$ resembles T1w, where pathology appears darker, while a lower ${\mu }_{\mathrm{w}}$ resembles T2w/FLAIR, where pathology is typically brighter. Considering extreme scenarios, we randomly assign the sign of $\mathrm{\Delta }I(\mathbf{x})20\mathrm{\%}$ of the time. $I$ further undergoes a standard augmentation pipeline [23], introducing partial voluming [5] and various resolutions, noise, scanning artifacts commonly found in clinical practice.

4. Learning Anatomy Beyond Gold Standards

In this section, we present UNA’s end-to-end training framework, which learns to unravel normal anatomy from images of random modality containing potential pathology.

Contralateral-Paired Input.

Healthy human brain anatomy typically exhibits a high degree of symmetry in structure. Based on this fact, we combine the original input image ($I$) with its contralateral-mirrored image $(\bar{I})$ to create paired inputs for UNA’s healthy anatomy reconstruction learning. This approach allows our model to “borrow” healthy information from the contralateral counterpart, thereby enhancing subject-specific healthy anatomy reconstruction. To ensure structural correspondence and minimize computational complexity during training, we pre-compute the deformation (${\varphi }_{I\to \bar{I}}$) between each training subject’s scan and its axial-flipped image using NiftyReg [38, 42]. As a result, the contralateral-paired input for each subject sample is represented as $\left(I,{\varphi }_{I\to \bar{I}}^{-1}\circ \bar{I}\right)$.

Modality-Agnostic Healthy Anatomy Reconstruction.

To enhance model generalizability, UNA is trained on both real datasets containing pathology (${𝔻}_{\text{Real}}$) and synthetic images $\left({𝔻}_{\text{Synth}}\right)$ generated from fluid-driven anomaly randomization (Sec. 3), featuring varying simulated modalities and abnormality conditions. During training, we define the following healthy anatomy reconstruction loss, which takes into account both the subject-level and the voxel-level abnormality of the input image ($I$):

$$\begin{array}{l}{ℒ}_{\text{Recon}}={\int }_{\text{Ω}}k(x)\{|\tilde{I}(x)-I(x)|+{\lambda }_{\nabla }|\nabla \tilde{I}(x)-\nabla I(x)|\}dx\\ \mathit{s.t.}k(x)=\{\begin{array}{ll}1-d\cdot p(x),\hfill & x\in {\text{Ω}}_P,\hfill \\ (1+{\lambda }_p)\cdot (1-d)\cdot p(x),\hfill & x\notin {\text{Ω}}_P,\hfill \end{array}\end{array}$$ (7)

where $d=\left\{1:I\in {𝔻}_{\text{Real}};0:I\in {𝔻}_{\text{Synth}}\right\}$ indicates whether the current image is sourced from real datasets $\left({𝔻}_{\text{Real}}\right)$ or generated synthetically (${𝔻}_{\text{Synth}}$). The parameters ${\mathrm{\lambda }}_{\nabla }$ and ${\mathrm{\lambda }}_p$ control the training weights for gradient L1 loss and attention to pathology, respectively. Specifically: (i) if the current training input image ($I$) is generated by UNA, i.e., the ground truth healthy anatomy of the entire brain region is accessible, we compute the anatomy reconstruction loss across the whole brain ($\mathrm{\Omega }$). (ii) Conversely, if $I$ is sourced from real datasets, the ground truth healthy anatomy of the entire brain is not available. In this case, we compute the voxel-wise reconstruction loss exclusively for the healthy regions, while masking out any abnormalities.

Intra-Subject Self-Contrastive Learning.

In Eq. (7), the anatomy reconstruction in abnormal regions is not supervised when dealing with real images containing pathology. To enhance the performance of learning healthy anatomy, we propose an intra-subject learning strategy that exploits the (approximate) symmetry of the brain with a contrastive loss that encourages two properties:

1. Similarity in appearance between the reconstructed healthy anatomy and its contralateral healthy counterpart.

2. Distinctiveness between the reconstructed anatomy and the original regions that exhibit abnormalities.

Specifically, we define this intra-subject contrastive loss as:

$${ℒ}_{\text{Contrast}}=-\mathit{log}\frac{{\int }_{{\text{Ω}}_{p\setminus \overline{p}}}{e}^{\tilde{I}\cdot \left({\varphi }_{I\to \overline{I}}^{-1}\circ \overline{I}\right)/\alpha }dx}{{\int }_{{\text{Ω}}_{p\setminus \overline{p}}}{e}^{\tilde{I}\cdot \left({\varphi }_{I\to \overline{I}}^{-1}\circ \overline{I}\right)/\beta }+e^{\tilde{I}\cdot I/\gamma }dx},$$ (8)

where ${\mathrm{\Omega }}_{p\setminus \bar{p}}={\mathrm{\Omega }}_p\setminus \left({\mathrm{\Omega }}_p\cap {\mathrm{\Omega }}_{{\varphi }_{I\to \bar{I}}^{-1}\circ \bar{P}}\right)$, ensuring that we exclude pathologies that appear at the same contralateral location on both hemispheres. $\alpha ,\beta ,\gamma$ represent the corresponding temperature scaling factors of each term.

Thus, UNA’s end-to-end healthy anatomy reconstruction training loss is obtained by the sum of Eqs. (7) and (8):

$$ℒ={ℒ}_{\text{Recon}}+{\mathrm{\lambda }}_{\text{Contrast}}{ℒ}_{\text{Contrast}},$$ (9)

where ${\mathrm{\lambda }}_{\text{Contrast}}$ is the weight of self-contrastive learning loss.

As shown in Fig. 1, as a general model for healthy anatomy reconstruction, UNA also addresses the following tasks: (i) Given an input image without any abnormalities, UNA performs anatomy reconstruction; (ii) Given a T1w MRI of any resolution, UNA performs super-resolution.

5. Experiments

We evaluate UNA’s performance and demonstrate its impact from three perspectives. (i) The reconstruction of anatomy from healthy images. This enables analysis with standard tools made for high-resolution T1w MRI, such as segmentation and parcellation using FreeSurfer [13], registration with NiftyReg [38, 42], ANTs [2], etc. (ii) The synthesis of healthy anatomy from images with pathology. This allows for the application of well-established general-purpose models to images with extensive pathology. For a more comprehensive assessment, we test on both synthetic data – where ground truth healthy images are available (Sec. 5.1) – and real images from two public stroke datasets – where the ground truth healthy anatomy is unknown (Sec. 5.2). (iii) We further demonstrate UNA’s direct application to anomaly detection (Sec. 5.3). Our test data includes CT and various MRI modalities (T1w, T2w, FLAIR).

Datasets.

We conducted experiments using eight public datasets: ADNI [25], ADNI3 [53], HCP [12], ADHD200 [7], AIBL [15], OASIS3 [27], ATLAS [31], ISLES [19]. ATLAS and ISLES include stroke patients, associated with gold-standard manual segmentations of stroke lesions (referred to as ${𝔻}_{\text{Stroke}}$ hereafter). The other datasets contain subjects with healthy anatomy (${𝔻}_{\text{Healthy}}$). These datasets cover both MR (T1w, T2w, FLAIR) and CT images. The train/test subject splits for each dataset are listed in Tab. 2.

Table 2.

Quantitative comparisons of healthy anatomy reconstruction performance between UNA and state-of-the-art, contrast-agnostic T1w synthesis models, evaluated on real images. Since we do not have ground truth anatomy for the stroke datasets, we only report the reconstruction performance within healthy regions. (ISLES [19] stroke dataset does not provide T1w MRI scans, therefore we only show qualitative results on ISLES in Fig. 4.)

Modality	Dataset (Train/Test)	Method	Recon struction (on Healthy)
			L1 (↓)	PSNR (↑)	SSIM (↑)
T1w MRI	ADNI [25] (1841/204)	SynthSR [23]	0.014	26.78	0.984
		Brain–ID [33]	0.012	33.82	0.993
		PEPSI [34]	0.014	31.25	0.989
		UNA	0.012	32.96	0.995
	HCP [12] (808/87)	SynthSR [23]	0.033	22.13	0.854
		Brain–ID [33]	0.020	27.47	0.957
		PEPSI [34]	0.023	28.20	0.971
		UNA	0.017	31.61	0.986
	ADNI3 [53] (298/33)	SynthSR [23]	0.023	23.60	0.928
		Brain–ID [33]	0.021	29.89	0.966
		PEPSI [34]	0.020	26.67	0.935
		UNA	0.019	30.01	0.975
	ADHD200 [7] (865/96)	SynthSR [23]	0.035	21.67	0.882
		Brain–ID [33]	0.011	32.48	0.996
		PEPSI [34]	0.015	29.87	0.976
		UNA	0.012	30.12	0.980
	AIBL [15] (601/67)	SynthSR [23]	0.026	22.95	0.916
		Brain–ID [33]	0.009	33.73	0.972
		PEPSI [34]	0.012	29.86	0.950
		UNA	0.010	32.89	0.964
	* Stroke * ATLAS [31] (590/65)	SynthSR [23]	0.030	23.50	0.881
		Brain–ID [33]	0.027	26.09	0.892
		PEPSI [34]	0.025	26.73	0.905
		UNA	0.020	29.10	0.974
T2w MRI	HCP [12] (808/87)	SynthSR [23]	0.034	21.46	0.833
		Brain–ID [33]	0.016	28.10	0.934
		PEPSI [34]	0.018	26.45	0.915
		UNA	0.016	28.62	0.949
	AIBL [15] (272/30)	SynthSR [23]	0.033	20.08	0.805
		Brain–ID [33]	0.022	23.99	0.861
		PEPSI [34]	0.024	22.93	0.859
		UNA	0.021	24.76	0.892
FLAIR MRI	ADNI3 [53] (298/33)	SynthSR [23]	0.026	22.77	0.919
		Brain–ID [33]	0.017	26.44	0.927
		PEPSI [34]	0.023	25.62	0.929
		UNA	0.015	27.43	0.965
	AIBL [15] (302/34)	SynthSR [23]	0.029	21.77	0.902
		Brain–ID [33]	0.019	27.25	0.936
		PEPSI [34]	0.021	25.43	0.914
		UNA	0.017	27.76	0.967
CT	OASIS3 [27] (795/88)	SynthSR [23]	0.041	20.93	0.758
		Brain–ID [33]	0.023	25.49	0.891
		PEPSI [34]	0.027	22.98	0.842
		UNA	0.022	25.68	0.897

Synthetic Data Generation.

We use the anatomical labels of training subjects from ${𝔻}_{\text{Healthy}}$ for random modality generation (Sec. 3.2). The synthetic abnormal profiles are generated using UNA’s fluid-driven anomaly randomization (Sec. 3), with initial profiles either sampled from the gold standard lesion segmentation maps of training subjects in ${𝔻}_{\text{Stroke}}$, or Perlin noise (Sec. 3.1). For evaluation on simulated data in Sec. 5.1, we employ our synthetic generator to create 1,000 testing samples from ${𝔻}_{\text{Healthy}}$, encoded with random anomaly profiles from ${𝔻}_{\text{Stroke}}$. This generation is solely for providing ground truth healthy anatomy; therefore, we encode random anomaly profiles without applying any additional deformation and corruption.

Metrics.

For anatomy reconstruction and synthesis, we use L1 distance, PSNR, and SSIM. For anomaly detection, we assess performance using Dice scores.

Implementation Details.

For fair comparisons, we adopt the same 3D UNet [41] as utilized in the models [23, 33, 34] we compare with. The training sample images are sized at 160³, with a batch size of 4. We use the AdamW optimizer, beginning with a learning rate of 10⁻⁴ for the first 300,000 iterations, which is then reduced to 10⁻⁵ for the subsequent 100,000 iterations. The additional attention parameter (${\mathrm{\lambda }}_p$ in Eq. (7)) is set to 1 for healthy anatomy reconstruction in pathological regions. The intra-subject contrastive learning weight (${\mathrm{\lambda }}_{\text{contrast}}$ in Eq. (9)) is set to 2. The training process took approximately 14 days on an NVIDIA A100 GPU.

Competing Models.

UNA is the first model achieving modality-agnostic healthy anatomy synthesis and reconstruction. We compare UNA with the closest state-of-the-art modality-agnostic models for image reconstruction and anomaly detection: (i) SynthSR [23], a modality-agnostic super-resolution model; (ii) Brain-ID [33], a modality-agnostic feature representation and T1w synthesis model; (iii) PEPSI [34], a modality-agnostic pathology representation model for T1w and FLAIR MRI synthesis. Note that PEPSI does not synthesize healthy tissue in regions of pathology; (iv) VAE [4], an unsupervised anomaly detection variational autoencoder model for brain MRI; (v) LDM [18], an out-of-distribution detection model for 3D medical images using latent diffusion.

5.1. Simulations with Ground Truth Anatomy

To better evaluate UNA’s performance in healthy anatomy reconstruction, we first conduct experiments using 1,000 healthy images encoded with simulated pathologies, for which ground truth segmentations are available for quantitative assessment. To explicitly assess the model performance in pathology regions, we report reconstruction scores not only for the entire brain but also separately for areas that are originally healthy and diseased in the input image.

Tab. 1 reports the quantitative comparison results between UNA and the state-of-the-art modality-agnostic synthesis models. UNA yields the best performance across all metrics, modalities, and regions of interest – including the full brain, healthy anatomy, and pathological regions. Remarkably, UNA outperforms competing models by a large margin in anatomy reconstruction within diseased tissue. Visualization results for each test modality are provided in Fig. 3. UNA demonstrates consistent performance across modality and resolution. Notably, other models either fail to capture any anatomy (SynthSR [23]) or generate unrealistic patterns around the pathology (Brain–ID [33] and PEPSI [34]) when given a noisy CT scan (4^th row in Fig. 3), whereas UNA successfully reconstructs plausible healthy anatomy.

Table 1.

Quantitative comparisons of healthy anatomy reconstruction performance between UNA and state-of-the-art contrast-agnostic T1w synthesis models, using images with simulated pathology. PEPSI [34] is designed to emphasize the abnormalities, therefore we do not report its scores within diseased regions.

Modality	Method	L1 (↓)			PSNR (↑)			SSIM (↑)
		F	H	D	F	H	D	F	H	D
T1w MRI	SynthSR [23]	0.0285	0.0253	0.0010	20.71	22.90	36.59	0.823	0.879	0.895
	Brain–ID [33]	0.0231	0.0219	0.0007	22.86	23.71	40.22	0.859	0.890	0.904
	PEPSI [34]	0.0257	0.0194	N/A	21.78	23.21	N/A	0.831	0.872	N/A
	UNA	0.0147	0.0143	0.0003	31.98	33.25	45.61	0.981	0.992	0.998
T2w MRI	SynthSR [23]	0.0362	0.0337	0.0016	18.25	20.66	35.47	0.816	0.864	0.880
	Brain–ID [33]	0.0277	0.0269	0.0008	20.98	22.31	39.62	0.844	0.881	0.892
	PEPSI [34]	0.0295	0.0279	N/A	19.33	23.18	N/A	0.820	0.845	N/A
	UNA	0.0184	0.0182	0.0003	25.14	26.22	45.69	0.938	0.981	0.998
FLAIR MRI	SynthSR [23]	0.0327	0.0300	0.0016	19.30	21.04	34.88	0.823	0.869	0.895
	Brain–ID [33]	0.0285	0.0242	0.0010	19.98	20.32	38.76	0.840	0.879	0.907
	PEPSI [34]	0.0301	0.0287	N/A	19.82	21.59	N/A	0.842	0.850	N/A
	UNA	0.0202	0.0194	0.0007	28.34	28.93	42.91	0.921	0.982	0.996
CT	SynthSR [23]	0.0541	0.0536	0.0029	13.97	13.13	28.50	0.712	0.763	0.725
	Brain–ID [33]	0.0339	0.0357	0.0018	20.15	21.20	32.87	0.811	0.824	0.843
	PEPSI [34]	0.0473	0.0420	N/A	16.72	16.90	N/A	0.723	0.782	N/A
	UNA	0.0259	0.0266	0.0010	25.63	25.70	42.53	0.883	0.897	0.895

(F: full brain; H: healthy region; D: diseased region.)

5.2. Real-World Datasets with Potential Pathology

We further evaluate UNA’s performance on all the real datasets as introduced in Sec. 5, among which ATLAS [31] and ISLES [19] contain stroke patients. Tab. 2 reports the reconstruction scores over all datasets and their available modalities: (i) For anatomy reconstruction of originally healthy subjects, UNA achieves the highest scores across most datasets, with the remaining scores on par with Brain–ID [33], which is specifically designed for healthy anatomy; (ii) On the ATLAS stroke dataset, UNA outperforms competing models by a larger margin (≈ 10%).

As shown in Fig. 4, other models tend to generate unrealistic patterns within and around abnormalities, whereas UNA’s reconstructions are notably more visually coherent. Additionally, we present a failure case (4^th row in Fig. 4), where we observe that UNA tends to “over-distinguish” the reconstructed healthy anatomy from the diseased regions, particularly in challenging scenarios where the pathology pattern completely occludes the underlying anatomy.

5.3. Direct Application: Anomaly Detection

UNA’s ability to synthesize diseased-to-healthy anatomy naturally equips it with the potential for application to anomaly detection. To demonstrate its effectiveness, we directly use the reconstructed healthy anatomy from UNA to detect abnormalities. Specifically, we follow the standard evaluation pipeline for unsupervised anomaly detection in medical images [4,18] and compute UNA’s anomaly estimation maps by calculating the voxel-wise absolute differences between the diseased input and the reconstructed output. The anomaly detection Dice scores are then obtained by comparing the ground truth pathology segmentations with the computed anomaly estimation maps, scaled to the range [0, 1] such that they represent the normalized abnormality. The same procedure is applied to other competing models.

As shown in Fig. 5, UNA’s difference maps clearly identify anomalies with varying shapes and sizes. Quantitative comparisons are provided in Tab. 3, where UNA: (i) out-performs other modality-agnostic synthesis models, and the state-of-the-art anomaly detection models; and (ii) demonstrates consistent performance across various datasets.

Figure 5.

Visualizations of directly applying UNA’s healthy anatomy reconstruction for anomaly detection. The estimated anomaly is computed as the absolute difference between diseased T1w MRI scans and UNA’s reconstructed healthy anatomy.

Table 3.

Dice scores (↑) of downstream anomaly detection performance based on the voxel-wise absolute differences between the diseased input and the reconstruction. The testing images include healthy T1w MRI scans with simulated pathology, and real T1w MRI images from stroke patients in ATLAS [31] dataset.

Image Source	Dataset	SynthSR [23]	Brain–ID [33]	VAE [4]	LDM [18]	UNA
Healthy T1w with Simulated Pathology	ADNI [25]	0.27	0.26	0.18	0.23	0.36
	HCP [12]	0.28	0.28	0.13	0.21	0.33
	ADHD200 [7]	0.23	0.25	0.15	0.23	0.34
	ADNI3 [53]	0.27	0.28	0.17	0.24	0.37
	AIBL [15]	0.25	0.24	0.12	0.20	0.32
Stroke T1w	ATLAS [31]	0.24	0.24	0.11	0.22	0.31

5.4. Ablation Study

To assess the contributions of UNA’s individual components, we perform an ablation study with several variants: (a) Training without fluid-driven anomaly randomization, i.e., training exclusively with real images with pathology; (b) Training with fluid-driven anomaly randomization, but initializing the anomaly profiles with random noise; (c) Training without contralateral-paired input, i.e., using only a single image without its contralateral counterpart; (d) Training without the intra-subject self-contrastive loss.

As shown in Fig. 6 and Tab. 4, training without fluid-driven anomaly randomization (UNA-(a)) results in the largest performance drop, showing only slight improvement over Brain–ID [33] (reported in Fig. 3), which does not train on diseased inputs at all. Introducing fluid-driven anomaly randomization improves overall performance, but performance gaps remain evident when compared to the proposed UNA when no real pathology profiles are used for initialization (UNA-(b)). Leveraging subject-specific contralateral information (UNA-(c), UNA-(d)) further enhances reconstruction results, particularly within diseased regions.

Figure 6.

Ablations on UNA’s healthy anatomy reconstruction.

Table 4.

Ablation study on UNA. Testing images are real T1w MRI encoded with simulated pathology (same as first-row group in Tab. 1).

Method	L1 (↓)			PSNR (↑)			SSIM (↑)
	F	H	D	F	H	D	F	H	D
UNA-(a)	0.0229	0.0193	0.0008	23.71	25.09	38.92	0.859	0.890	0.904
UNA-(b)	0.0195	0.0182	0.0005	25.79	27.30	42.35	0.903	0.925	0.950
UNA-(c)	0.0155	0.0163	0.0004	30.00	31.92	43.61	0.959	0.977	0.982
UNA-(d)	0.0195	0.0182	0.0005	27.13	28.04	42.97	0.931	0.950	0.969
UNA	0.0147	0.0143	0.0003	31.98	33.25	45.61	0.981	0.992	0.998

(F: full brain; H: healthy region; D: diseased region.)

6. Limitations and Future Work

Handling Extreme Cases.

As discussed in Sec. 5.2, UNA appears to “over-correct” its reconstructed healthy anatomy, especially in extreme cases where the pathology in the input image heavily occludes the underlying anatomy. This issue will be further investigated in our future work.

Broader Applications.

By bridging the gap between healthy and diseased anatomy, UNA opens up a wide range of applications beyond anomaly detection. For example, it could enable modality-agnostic image registration in the presence of pathology, as well as stroke treatment outcome prediction based on UNA’s reconstructed healthy anatomy. We plan to further explore these applications of UNA.

7. Conclusion

We introduce UNA, a modality-agnostic model for reconstructing healthy anatomy that works both with healthy subjects and images with varying degrees of pathology. Our fluid-driven anomaly randomization approach enables the generalization of an unlimited number of anomaly profiles from just a few real pathology segmentations. UNA can be directly applied to real images containing pathologies without fine-tuning. We demonstrate UNA’s superior performance across eight public datasets, including MR and CT images from healthy subjects and stroke patients. Additionally, we showcase UNA’s direct applicability to anomaly detection tasks. By bridging the gap between different modalities and the underlying anatomy, as well as between healthy and diseased images, we believe UNA opens up exciting opportunities for general image analysis in clinical practice, particularly for images with diverse pathologies.