TransUNET-DDPM: A Transformer-Enhanced Diffusion Model for Subject-Specific Brain Network Generation and Classification

Abstract

Generative AI for image synthesis has significantly progressed with the advent of advanced diffusion models. These models have set new benchmarks in creating high-quality and meaningful visual information. In this paper, we introduce TransUNET-DDPM, a novel framework that fuses transformer-based architectures with denoising diffusion probabilistic models (DDPMs) to generate high-quality, 2D and 3D intrinsic connectivity networks (ICNs). This architecture addresses limitations of traditional linear methods like independent component analysis (ICA) by leveraging the nonlinear modeling capabilities of DDPMs, further enhanced through transformer blocks that enable attention-driven feature encoding. To produce subject-specific 3D ICNs, an image-conditioned variant of TransUNET-DDPM is employed, utilizing a spatiotemporal encoder to incorporate resting-state fMRI (rs-fMRI) conditional information. Efficient training is achieved through a transfer learning strategy in which a large-scale, unconditional TransUNET-DDPM is first pretrained to capture general spatial and temporal patterns, followed by fine-tuning on a smaller, condition-specific neuroimaging dataset. Additionally, a class-conditioned version of the model is introduced for data augmentation in schizophrenia classification. By generating synthetic ICNs based on diagnostic labels, this variant enhances the robustness of classifiers, particularly in data-scarce scenarios. Furthermore, quantitative and qualitative evaluations demonstrate that our framework surpasses existing generative models in producing anatomically and functionally meaningful ICNs, with external dataset validation confirming its generalizability.

1. Introduction

Diffusion models [1] have become the new benchmark in deep generative modeling [2, 3], demonstrating exceptional performance across various applications. These generative models provide an alternative to existing generative adversarial networks (GANs) [4], variational autoencoders (VAEs) [5], and normalizing flow models [6]. For generative models to be utilized in real-world applications, they must meet several key criteria: high-quality sampling, diversity of samples, and fast, efficient sampling. While GANs can produce high-quality medical images rapidly, they often lack diversity and result in mode collapse [7]. In contrast, VAEs [8] and normalizing flows [9] excels at capturing the variation within data distributions but tend to produce lower-quality samples. Diffusion models address the limitations of VAEs and GANs by effectively capturing the variations in data distribution and generating high-quality samples. In recent years, denoising diffusion probabilistic models (DDPMs) [10, 11] have emerged as the leading algorithms for both unconditional and conditional image generation in the medical imaging domain. In the case of fMRI data, the application of high-fidelity fMRI image generation focused on 2D fMRI due to the computationally expensive nature of early diffusion models. A notable 2D medical data synthesis model [12] utilizes a DDPM framework integrated with a Swin vision transformer [13] to enhance the training set through augmentation. Later, another study proposed a 3D-DDPM for generating high-resolution MRIs of brain tumors [14]. Another alternative approach for the controllable generation of 3D MRI images is the use of latent diffusion models (LDMs) [15, 16]. This method produced 3D synthetic MRI images of the adult brain and utilized a cross-attention mechanism to integrate age, gender, and brain volume.

Blind source separation (BSS) [17, 18] is a fundamental problem in many image processing applications that aim at revealing latent structures within multivariate data through matrix decomposition techniques. Although the process is completely unaware of how the signals were mixed, it operates under certain assumptions about the source signals (e.g., the source distribution) or the mixing process (e.g., linear mixing). In terms of fMRI data, intrinsic connectivity networks (ICNs) are traditionally extracted using independent component analysis (ICA) [19, 20], a technique that decomposes the multivariate fMRI signals into spatially independent components that represent distinct brain networks. While ICA has been successfully applied in numerous studies, it remains challenging to manually select the relevant components and differentiate neurophysiological signals from noise. Other traditional BSS methods that have proven effective for neuroimaging data include sparse principal component analysis (sPCA) [21], sparse canonical correlation analysis (sCCA) [22], sparse partial least squares (sPLS)[23], and sparse two-dimensional canonical correlation analysis (s2DCCA) [24]. However, these traditional methods [25] are constrained by their linearity and inability to capture complex, non-linear relationships that may exist within the brain’s connectivity patterns.

Meanwhile, the success of deep learning across various neuroimaging applications has led researchers to explore its use in analyzing rs-fMRI data [26]. Previous research applied restricted Boltzmann machines (RBMs) [27] to fMRI volumes to uncover intrinsic functional brain networks [28], resulting in a competitive performance compared to ICA. Another study [29] employed the RBM model on fMRI time courses to extract latent sources that distinguish internal neural activity within a single brain region and reveal functional interactions across multiple brain regions. This approach demonstrated higher correlations compared to ICA in identifying task-related components, though it required empirical parameter tuning. Additionally, researchers have proposed a framework that uses autoencoders and spatiotemporal sparsity constraints for blind source separation (BSS) from fMRI datasets [30]. While the sparse spatiotemporal BSS method shows promise with its computational simplicity and effectiveness, it can experience fluctuations during convergence and also faces model selection challenges. Therefore, despite these advancements, there remains a need for methods that are capable of generating high-quality, diverse, and efficient samples of ICNs from fMRI data.

In terms of application, prior studies have explored several generative and deep learning approaches, demonstrating promising results in classifying patients with schizophrenia from healthy controls using rs-fMRI data. As shown in Table. 1, these studies employed diverse datasets and model architectures, achieving varying levels of classification accuracy. One of the earlier works utilized 3D ICA maps from the COBRE dataset with a 3D-CNN architecture, achieving very high classification accuracy. However, a major limitation of this study was the absence of an independent test dataset, which restricts the model’s ability to generalize beyond the training data. Another study incorporated multiple datasets, including those from COBRE and UCLA, and utilized a discriminant autoencoder for functional connectivity, achieving an accuracy of 85%. Two other models that utilize GANs only achieve moderate accuracy, despite their substantial computational cost. The model that uses GAN architecture with ROI mean time series from fMRI data faces limitations in its ROI-based connectivity analysis. This approach is highly sensitive to prior knowledge and the choice of brain regions, making it challenging to define a robust and consistent set of voxels for analysis. A more recent study combined datasets from fBIRN, COBRE, and MPRC, applying a Vision Transformer (ViT) to functional connectivity, but only achieved a classification accuracy of 75% in the unimodal setting. Meanwhile, recent studies have utilized self-attention networks and anatomy-guided brain network transformers, achieving improved accuracies of 82.3% and 80.73%, respectively. While these prior approaches have shown promise, their limitations underscore the need for more robust and flexible generative frameworks, such as diffusion models, that can better capture the complexity of rs-fMRI data for schizophrenia classification.

Table 1:

Comparative overview of prior models for schizophrenia classification using rsfMRI

Article	Dataset	Features	Model Architecture	Evaluation Metrics	Classification Accuracy
[32]	COBRE	3D ICA maps	3D-CNN	Accuarcy, AUC, specificity, sensitivity	98.09%
[33]	Subdatasets from Xijing hospital, AMU, Xiangya, COBRE, UCLA, and WUSTL	Functional connectivity	Discriminanat autoencoder	Sensitivity, specificity, accuarcy	85%
[34]	Schizophrenia Chinese dataset from 7 hospitals	Functional connectivity	GAN with feature mapping	Accuracy, sensitivity, specificity, f1 score, AUC	80%
[35]	COBRE	ROI’s mean time series from fMRI	GAN	Accuracy, sensitivity, specificity	76.5%
[36]	fBIRN + COBRE + MPRC	Functional connectivity	ViT	Accuracy, AUC, f1 score	75%
[37]	COBRE + Huaxi dataset + Nottingham dataset + Taiwan dataset + Xiangya dataset	Functional connectivity	AGBN transformer	accuracy, sensitivity, specificity, balanced accuracy, positive predictive value, negative predictive value	80.73%
[38]	Subset of fBIRN	Functional connectivity	Connectome SANet (self attention networks)	Sensitivity, specificity, f1 score, accuracy	82.3%

Diffusion models, particularly DDPMs, have recently emerged as a powerful generative framework within neuroimaging, offering notable advantages over conventional approaches. They are known for their ability to iteratively refine noisy data into high-quality outputs, providing greater flexibility and precision in modeling complex connectivity patterns that traditional methods often fail to capture. This framework also addresses the challenge of noise and artifacts commonly present in fMRI data, improving the accuracy of ICN identification. The incorporation of transformer-based architectures further enhances this framework, allowing the model to capture long-range dependencies and detailed patterns within the data. In this study, the proposed method establishes a foundational framework by leveraging unconditional DDPMs to generate high-quality 2D and 3D ICNs, utilizing the iterative refinement capabilities inherent in DDPMs. A preliminary version of this work was presented in [31], where the initial concept of employing DDPMs for ICN generation was introduced. This manuscript builds upon that foundation by integrating conditional DDPMs, enhanced with a specialized transformer architecture, to enable the generation of subject-specific ICNs. Furthermore, by conditioning class labels, this approach facilitates data augmentation and enhances schizophrenia classification performance. Conditioning on an individual’s rs-fMRI data ensures that the generated ICNs are specifically tailored to the unique connectivity patterns of that subject. Additionally, conditioning on class labels allows the generation of ICNs that are not only subject-specific but also reflective of specific brain states or conditions, further improving the model’s ability to capture relevant variations for tasks such as classification. This methodology effectively captures both within-subject and between-subject variability, providing more nuanced insights into brain connectivity patterns. Additionally, transfer learning strategies are employed for pretraining and fine-tuning, which further reduce training time and improve model performance. The main contributions to this work are as follows:

• This work introduces TransUNET-DDPM, a new architecture specifically designed for generating high-quality 3D ICNs. By combining the strengths of transformer networks and DDPMs, this architecture overcomes challenges posed by traditional methods in handling complex neuroimaging data.

• We employ transfer learning to enhance training efficiency. The model is pretrained on unconditional TransUNET-DDPMs using a large dataset to capture general features and patterns, followed by fine-tuning on conditional DDPMs with a smaller, task-specific dataset. This approach not only adapts the model to the unique characteristics of the target task but also significantly reduces training time.

• Generate subject-specific ICNs by integrating a spatial-temporal encoder into conditional TransUNET-DDPMs. The model analyzes resting-state functional MRI (rs-fMRI) data conditioned on corresponding ICNs, allowing it to learn unique brain connectivity patterns for individual subjects. During inference, the model accurately generates ICNs tailored to the specific rs-fMRI data provided, capturing both within-subject and between-subject variability.

• A class-conditioned TransUNET-DDPM model is also introduced for data augmentation, specifically aimed at improving schizophrenia classification. By conditioning disease labels, the model generates synthetic ICNs that are tailored to each class, thereby boosting classification performance, especially with small datasets such as FBIRN.

2. Materials and preprocessing

The neuroimaging dataset for this analysis utilizes three fMRI datasets: UK Biobank [39], Human Connectome Project (HCP) [40, 41], and fBIRN [42]. Here, the UK Biobank serves as the primary source for pretraining across all tasks. It includes a large population sample comprising older adults aged 53 to 87 years, with an average age of 69.75 ± 7.43 years. For both unconditional and conditional ICN generation, the model is trained on UK Biobank and evaluated on UK Biobank and HCP data. The HCP dataset includes 1,200 young adults (ages 22–35) and serves as an external test set. For schizophrenia classification, the model is pre-trained on the UK Biobank and fine-tuned on FBIRN, which includes 310 subjects (150 patients with schizophrenia and 160 healthy controls) aged 18–65.

Resting-state fMRI data from each dataset were first preprocessed, followed by computation of ICNs using NeuroMark [43], an automated spatially constrained ICA method. In the case of UKBiobank, the rs-fMRI scans were performed using 3 Tesla (3T) Siemens Skyra scanners equipped with 32-channel head coils. The imaging protocol employed a gradient-echo echo planar imaging (GE-EPI) technique without iPAT, fat saturation, a flip angle (FA) of 52°, spatial resolution of $2.4\times 2.4\times 2.4\text{mm}$, field of view (FOV) of $88\times 88\times 64$ matrix, repetition time (TR) of 0.735 s, echo time (TE) of 39 ms, and acquisition of 490 volumes. The scan duration was 6 minutes and 10 seconds, during which participants focused on a crosshair and stayed relaxed. Additionally, a multiband sequence with an acceleration factor of eight acquired eight slices simultaneously. Further to ensure high data quality, several preprocessing steps were applied to the UK Biobank data. Firstly, intra-modal motion correction was applied using the MCFLIRT tool [44]. Next, to enable comparison across participants, grand-mean intensity normalization was applied to scale the entire 4D dataset. High-pass temporal filtering addressed residual temporal drifts, and geometric distortions were corrected using FSL’s Topup tool [45]. Subsequently, EPI unwarping and gradient distortion correction (GDC) were conducted, and artifacts were removed using independent component analysis (ICA) combined with FMRIB’s ICA-based X-noiseifier [46]. The data were standardized to an MNI EPI template using FLIRT and SPM12, followed by the application of Gaussian smoothing with a full width at half maximum (FWHM) of 6 mm.

ICN computation was guided by the NeuroMark_fMRI_1.0 template, which includes 53 ICNs derived from a 100-component ICA decomposition of two large healthy control datasets (HCP and GSP). This template was used in an adaptive ICA framework to estimate subject-specific networks and their time courses. The ICNs are grouped into seven functional domains: subcortical (SC: 5), auditory (AUD: 2), sensorimotor (SM: 9), visual (VIS: 9), cognitive control (CC: 17), default mode (DM: 7), and cerebellar (CB: 4). Here, functional brain networks are a broader term that encompasses all network-level organizations inferred from fMRI data, including task-based and resting-state networks. ICNs, in contrast, specifically refer to the spatially distinct, temporally coherent networks that are extracted using data-driven methods like ICA. Thus, ICNs are a specific subset of functional brain networks, capturing the brain’s default connectivity structure, whereas the broader category includes both resting-state and task-evoked networks.

3. Methods

3.1. Denoising Diffusion Probabilistic Models

DDPMs form the foundation of current diffusion models in generative AI. They utilize a diffusion-derived approach to learn the underlying distribution of the data. These models have gained popularity due to their ability to generate high-fidelity samples and their stability during training. They generate these complex data distributions by iteratively refining random noise. The central idea behind DDPMs is that they are based on a diffusion process that systematically adds noise to data in a forward process and learns to reverse this process during generation. DDPMs mainly operate in two distinct modes. In the unconditional mode, images are generated from noise without any external input. In the case of conditional DDPMs, additional information is incorporated to guide the image generation process. This additional information involves class labels, textual descriptions, or other forms of structured data, allowing for the generation of data adhering to specific conditions.

The forward process in DDPMs is a Markov chain where Gaussian noise is added to the data sample ${\mathbf{\text{x}}}_0$ over a fixed number of timesteps $T$. This process continues till the data becomes indistinguishable and becomes pure Gaussian noise by time $T$. The forward process is also parameterized by a noise variance scheduler $\beta$ that controls the noise variance at each timestep. Mathematically, the forward process is defined by a noising function, which is a conditional distribution $q\left({\mathbf{\text{x}}}_t\mid {\mathbf{\text{x}}}_{(t-1)}\right)$. This function determines how noise is added at each step based on the previous time step’s image.

$$q\left({\mathbf{\text{x}}}_t\mid {\mathbf{\text{x}}}_{t-1}\right)=𝒩\left({\mathbf{\text{x}}}_t;\sqrt{1-{\beta }_t}{\mathbf{\text{x}}}_{t-1},{\beta }_t\mathbf{\text{I}}\right)$$ (1)

where $\sqrt{1-{\beta }_t}{\mathbf{\text{x}}}_{t-1}$ represents the mean ${\mu }_t$, and ${\beta }_t\mathbf{\text{I}}$ denotes the variance ${\mathbf{\Sigma }}_t$. As $t$ approaches $T,{\mathbf{\text{x}}}_t$ increasingly resembles a sample from a Gaussian distribution $𝒩(0,\mathbf{\text{I}})$. Moreover, the forward process can also be expressed in a closed form using the reparameterization trick as follows:

$$q\left({\mathbf{\text{x}}}_t\mid {\mathbf{\text{x}}}_0\right)=𝒩\left({\mathbf{\text{x}}}_t;\sqrt{{\bar{\alpha }}_t}{\mathbf{\text{x}}}_0,\left(1-{\bar{\alpha }}_t\right)\mathbf{\text{I}}\right)$$ (2)

where we define ${\bar{\alpha }}_t=\prod _{s=1}^t{\alpha }_s$ and ${\alpha }_t=1-{\beta }_t$. This allows for efficient sampling from any intermediate step $t$ given the original data ${\mathbf{\text{x}}}_0$. The variance parameter ${\beta }_t$ can be constant or vary over $T$ timesteps. The original DDPM used a linear schedule, increasing $\beta$ from 10⁻⁴ to 0.02 at the end.

Following this, during the reverse process, an image-generating model is used to predict the noise added to an image at a specific timestamp. Here, the original data ${\mathbf{\text{x}}}_0$ is recovered from the noisy observation ${\mathbf{\text{x}}}_T$ by sequentially denoising it over the same $T$ steps. The reverse process is modeled using the conditional distribution function $\left.p\left({\mathbf{\text{x}}}_{t-1}\right)\mid {\mathbf{\text{x}}}_t\right)$. Since directly modeling this function is infeasible due to the numerous possible values of ${\text{x}}_{t-1}$ for a given ${\mathbf{\text{x}}}_t$, neural networks are employed for this estimation. The function is modified to $\left.p_{\theta }\left({\mathbf{\text{x}}}_{t-1}\right)\mid {\mathbf{\text{x}}}_t\right)$ where $\theta$ represents the network parameters.

$$p_{\theta }\left({\mathbf{\text{x}}}_{t-1}\mid {\mathbf{\text{x}}}_t\right)=𝒩\left({\mathbf{\text{x}}}_{t-1};{\mu }_{\theta }\left({\mathbf{\text{x}}}_t,t\right),{\Sigma }_{\theta }\left({\mathbf{\text{x}}}_t,t\right)\right)$$ (3)

The training of this network is done by optimizing the variational bound on the negative log-likelihood of the data.

$$L_{\text{vlb}}\left(\theta \right)={\mathbb{E}}_q\left[-\text{log}{p}_{\theta }\left({\mathbf{\text{x}}}_0\right)\right]\approx \sum _{t=1}^T{\mathbb{E}}_q\left[\frac{{\beta }_t}{2{\sigma }_t^2}{‖ϵ-{ϵ}_{\theta }\left({\mathbf{\text{x}}}_t,t\right)‖}^2\right]$$ (4)

The simplified training objective for DDPMs is expressed as follows:

$$L_{\text{simple}}={\mathbb{E}}_{{\mathbf{\text{x}}}_0,ϵ,t}\left[{‖ϵ-{ϵ}_{\theta }\left({\mathbf{\text{x}}}_t,t\right)‖}^2\right]$$ (5)

where $ϵ$ is the Gaussian noise added at each step, and ${ϵ}_{\theta }$ is the neural network’s estimate of this noise.

3.2. TransUNET-DDPM framework

The TransUNET-DDPM architecture is a hybrid diffusion model specifically designed for 3D neuroimaging data s shown in Fig. 1a. It integrates a traditional 3D UNet-style encoder-decoder with a transformer-based bottleneck, all within the DDPM framework. The main goal of this architecture is to combine the strengths of convolutional neural networks for local 3D spatial inference with transformers for global contextual awareness, improving the model’s ability to capture complex patterns within volumetric brain data. During the forward process, noise is added to the input ICNs at each timestep, and during the reverse process, the image is denoised using the proposed hybrid TransUNET architecture. This network begins with a sinusoidal time embedding that encodes the current timestep of diffusion. This embedding is not trained but is initialized using a fixed sinusoidal function, ensuring that each timestep receives a unique and smooth representation. These embeddings are then projected and added to the input of each encoder and decoder block, enabling the network to modulate its behavior based on the stage of the diffusion process. The encoder part of the proposed model consists of several stages of convolutional blocks, each followed by 3D convolutions for downsampling. Each block contains a pair of Conv3D layers, SiLU activation, and layer normalization, making it capable of learning fine spatial features in 3D data. The progressive downsampling helps the model build a hierarchical understanding of the input, capturing features at multiple spatial scales. At the bottleneck of the architecture lies the transformer module, which differentiates this model from standard 3D DDPMs. The Gated Residual Transformer with Internal Token projection (GRIT) block is a transformer layer built specifically to process 3D data embeddings in the bottleneck of the TransUNet-DDPM. It extends the baseline transformer with several architectural enhancements that improve the model’s ability to learn global dependencies across 3D space while also promoting stable training. The GRIT block mainly consists of layer normalization, multi-head self-attention (MHSA), learnable residual gating, and layer scaling for enhanced stability and performance.

Figure 1:

(a) The TransUNET-DDPM framework and (b) the conditional TransUNET-DDPM architecture.

The input to GRIT is a tensor $f\in {\mathbb{R}}^{B\times N\times D}$, where $B$ is the batch size, $N$ is the number of flattened 3D tokens, and $D$ is the embedding dimension. The first layer in GRIT is layer normalization since it improves gradient flow, makes training more stable, and helps with convergence speed.

$${\tilde{f}}_1=\text{LayerNorm}\left(f\right)$$ (6)

This normalized input is then passed through the MHSA mechanism. In the MHSA block, the input ${\tilde{f}}_1$ is linearly projected into three components: queries $Q$, keys $K$, and values $V$, each of shape ${\mathbb{R}}^{B\times N\times d_k}$, where $d_k=D/h$, and $h$ is the number of attention heads. Each attention head computes the scaled dot-product attention. This operation allows each position in the sequence to attend to every other position, capturing interactions and dependencies in the spatial structure. All heads are run in parallel, and their outputs are concatenated and linearly transformed. This attention output is then optionally passed through an additional linear projection $W_h\in {\mathbb{R}}^{D\times D}$ in GRIT, called the internal token projection.

$$f_{\text{attn}}={\tilde{f}}_1+\gamma \cdot \left(\text{Attn}\left({\tilde{f}}_1\right)W_h\right)$$ (7)

This projection adds a learnable transformation after attention has been computed by allowing the model to reweigh the attention output across dimensions before it is added back to the residual path. Here $\gamma \in {\mathbb{R}}^D$ is a learnable layer scale parameter which allows fine control over the flow of gradients and prevents instability, especially when stacking many transformer layers during training.

The second half of the GRIT block is a gated MLP-based feedforward layer. The attention output $f_{\text{attn}}$ is passed through another layer norm:

$${\tilde{f}}_2=\text{LayerNorm}\left(f_{\text{attn}}\right)$$ (8)

Then it is passed through a two-layer MLP with a GELU nonlinearity and dropout:

$$f_{\text{mlp}}=\text{MLP}\left({\tilde{f}}_2\right)=W_2\cdot \text{GELU}\left(W_1\cdot {\tilde{f}}_2\right)$$ (9)

Rather than simply adding this to the residual as in standard transformers, GRIT introduces a learnable gating vector $g\in {\mathbb{R}}^{1\times D}$:

$$f_{\text{out}}=f_{\text{attn}}+g\cdot f_{\text{mlp}}$$ (10)

This residual gate $g$ learns to scale the contribution of the MLP path independently per channel, allowing the model to adaptively suppress or enhance specific dimensions in the learned features. Gating mechanisms like this have shown improvements in convergence speed and generalization, particularly in generative settings like diffusion models.

The output from the GRIT blocks is further passed into the decoder block, which uses transposed convolutions for up sampling and combines the up sampled features with corresponding encoder outputs through skip connections. Each decoder stage is again modulated by time embedding to ensure temporal consistency during the denoising process. The decoder helps reconstruct the original resolution of the 3D ICNs, using the rich hierarchical and contextual information learned during encoding and through the transformer bottleneck.

3.3. Conditional TransUNET-DDPM framework

In conditional DDPMs, the reverse process is extended to incorporate conditioning information $\mathbf{\text{y}}$, which could represent any additional data such as labels or other modalities. The conditioning allows the model to generate samples that align with the given context, thereby enabling a controlled and targeted generation of outputs. The conditional reverse process is thus defined as:

$$p_{\theta }\left({\mathbf{\text{x}}}_{t-1}\mid {\mathbf{\text{x}}}_t,\mathbf{\text{y}}\right)=𝒩\left({\mathbf{\text{x}}}_{t-1};{\mu }_{\theta }\left({\mathbf{\text{x}}}_t,t,\mathbf{\text{y}}\right),{\Sigma }_{\theta }\left({\mathbf{\text{x}}}_t,t,\mathbf{\text{y}}\right)\right)$$ (11)

The mean and variance depend on the conditioning vector $\mathbf{\text{y}}$. Here, the conditioning vector $\mathbf{\text{y}}$ is derived from 4D rs-fMRI data $\mathbf{\text{f}}\in {\mathbb{R}}^{T_f\times H\times W\times D}$, where $T_f$ is the number of timepoints. To capture both spatial and temporal dynamics of brain activity, we use a dual-branch spatiotemporal encoder, as illustrated in Fig. 1b. The spatial encoding branch is formally defined as follows:

$${ℰ}_s:{\mathbb{R}}^{H_f\times W_f\times D_f}\to {\mathbb{R}}^{d_s}$$ (12)

Here $d_s$ is the feature embedding dimension of the CNN. Then, the spatial feature vector at each timepoint $t_1$ is shown below.

$${\mathbf{\text{s}}}_{t_1}={ℰ}_s\left({\mathbf{\text{f}}}_{t_1}\right)={\text{CNN}}_{3D}\left({\mathbf{\text{f}}}_{t_1}\right),{\mathbf{\text{s}}}_{t_1}\in {\mathbb{R}}^{d_s}$$ (13)

Hence, ${ℰ}_s$ is a sequence of 3D convolutional layers followed by normalization and pooling, and finally, it results in a set of spatial embeddings over time.

$$\mathbf{\text{S}}=\left[{\mathbf{\text{s}}}_1,{\mathbf{\text{s}}}_2,\dots ,{\mathbf{\text{s}}}_{T_f}\right]\in {\mathbb{R}}^{T_f\times d_s}$$ (14)

The sequence $\mathbf{\text{S}}\in {\mathbb{R}}^{T_f\times d_s}$ now contains compressed spatial features from each rs-fMRI timepoint, and this is then fed into a transformer to model the temporal dynamics of brain activity. Since transformers lack inherent awareness of sequence order, the positional information $PE\in {\mathbb{R}}^{T_f\times d_s}$ is injected into the temporal sequence. Using sinusoidal positional encodings, it is added elementwise to the sequence.

$$S_{\text{in}}=S+PE\in {\mathbb{R}}^{T_f\times d_s}$$ (15)

The input $S_{\text{in}}$ is processed by a multi-layer transformer encoder composed of self-attention and feed-forward blocks, producing a temporal output $Y\in {\mathbb{R}}^{T_f\times d_s}$. This output is then compressed into a fixed-length vector $y_f\in {\mathbb{R}}^{d_s}$, and these final embeddings are concatenated into the bottleneck of the model to form the conditional context $\stackrel{\mathit{ˆ}}{\mathit{ϵ}}={ϵ}_{\theta }\left({\mathbf{\text{x}}}_t,t,{\mathbf{\text{y}}}_f\right)$. This allows the model to guide generation based on dynamic brain states captured in the rs-fMRI time series. The objective function for training this conditional model is similarly modified to include the conditional variable:

$$L_{\text{cond}}\left(\theta \right)={\mathbb{E}}_{t,{\mathbf{\text{x}}}_0,\mathbf{\text{y}},ϵ}\left[{‖ϵ-{ϵ}_{\theta }\left({\mathbf{\text{x}}}_t,t,\mathbf{\text{y}}\right)‖}^2\right]$$ (16)

3.4. Transfer learning strategy for conditional TransUNet-DDPM

To leverage prior knowledge, reduce training time, and address data scarcity in the conditional generation task, we adopt a transfer learning strategy. Initially, a pretrained 3D TransUNet-DDPM model is used to generate ICNs without any conditioning. This model, trained on a large dataset, learns robust spatial representations of ICN patterns through its UNet backbone, enhanced by transformer layers in the bottleneck. The learned weights from this unconditional model are then transferred to a conditional TransUNet-DDPM model, which is fine-tuned on a smaller dataset.

There are two conditional use cases in the second stage, as shown in Fig. 2. The first involves using rs-fMRI data as conditional input for subject-specific ICN generation. In this setup, the model is extended with a temporal transformer encoder that processes rs-fMRI sequences into a fixed-length conditioning vector. This vector is concatenated with the bottleneck feature representation in the TransUNet-DDPM architecture, allowing the model to incorporate subject-specific temporal dynamics during ICN generation. During inference, the model takes only rs-fMRI data as input and generates the corresponding ICNs based on the learned temporal-spatial relationships.

Figure 2:

(a) Pipeline of the unconditional TransUNET-DDPM with pre-training, followed by fine-tuning using the conditional TransUNET-DDPM for subject-specific ICN generation; (b) Pipeline for class-conditioned TransUNET-DDPM with transfer learning for generating synthetic 3D-ICNs for schizophrenia classification.

The second use case focuses on schizophrenia classification using disease-specific labels. Here, the pretrained TransUNET-DDPM model, having already learned a general distribution of the 3D ICNs, is further fine-tuned on a smaller, label-specific dataset. Class label information is integrated via a learnable embedding layer, which maps discrete labels to dense vectors matching the dimensionality of the time embeddings. During the forward pass, the label embedding corresponding to a given class is added directly to the time embedding. This combined embedding, which carries both temporal and categorical context, is propagated through each major block of the TransUNET-DDPM architecture. In the inference phase, the fine-tuned model can conditionally generate synthetic samples based on a provided class label. These synthetic samples can then be used to augment training data and enhance the performance of the classification task.

4. Experiments and Results

This section outlines the experimental setup and metrics, compares ICN generation with benchmark models, presents external dataset validation, and evaluates subject-specific ICNs and class-conditioned TransUNET-DDPM for schizophrenia classification.

4.1. Experimental setup and evaluation metrics

The proposed TransUNET-DDPM model is initially trained using neuroimaging data from 10,000 subjects sourced from the UK Biobank database. Prior to training, preprocessing steps such as normalization and background removal are performed to improve input data quality and consistency. Normalization standardizes the data by scaling image intensities to a common range, thereby reducing variability across samples. Background removal eliminates extraneous elements like noise and artifacts, enabling the model to concentrate on the brain’s functional networks.

For unconditional 2D and 3D ICN generation, 53 ICNs are produced, each corresponding to distinct functional domains of the brain. During training, data is organized into batches of 32 images to optimize memory efficiency, with training conducted over more than 20 epochs. The TransUNET-DDPM framework employs 1,000 discrete time steps, progressively adding noise to the data. A beta schedule ranging from 10⁻⁴ to 0.02 regulates the noise variance across these steps. The model is trained by minimizing the Mean Squared Error (MSE) loss, comparing the predicted noise to the actual noise added. Weight updates are performed using the Adam optimizer with a learning rate of 0.001. The performance of both the 2D and 3D models is evaluated using the HCP dataset, which serves as an independent test set. The best-performing model, selected during the training process, is subsequently validated by assessing the Multi-Scale Structural Similarity Index Measure (MS-SSIM), MSE, and Peak Signal-to-Noise Ratio (PSNR), ensuring robust generalization across different datasets.

In the case of conditional TransUNet-DDPM, the model is first pretrained on a large set of unconditional UK Biobank ICNs. Once pretrained, it is fine-tuned on a smaller conditional dataset from the same source, where rs-fMRI data serves as the conditioning input. During the pretraining phase, we used a learning rate of 0.0001, a batch size of 4, and trained for 20 epochs using the Adam optimizer with MSE as the loss function. For fine-tuning, we employed a learning rate of 0.00001, a batch size of 4, and trained for 10 epochs, again using the Adam optimizer and MSE loss. All experiments were conducted on a single NVIDIA V100 GPU with 25 GB of memory using mixed precision training. The total runtime was approximately 1 hour 32 minutes for pretraining and 6 hour 40 minutes for fine-tuning. Although pretraining was conducted on a larger dataset, the total training time was shorter compared to fine-tuning due to the significantly lower data complexity. Specifically, pretraining used unconditional 3D volumes of size $53\times 63\times 52$, whereas fine-tuning involved conditional data, where each of the subjects was associated with both a 3D volume of the same size and a high-dimensional conditional input of size $53\times 63\times 52\times 490$. The higher input dimensionality in the fine-tuning phase allowed the model to utilize more detailed conditional information, enhancing its learning capacity but also increasing the computational load per sample. Moreover, only the rs-fMRI data is provided as input during inference for subject-specific generation, enabling the model to synthesize the corresponding ICNs. In the case of class-conditional generation for a classification task, a similar setup is employed, but fine-tuning is done on a smaller schizophrenia dataset (fBIRN). During inference, class labels are provided, allowing the model to generate ICNs corresponding to either the schizophrenia or control groups. These generated ICNs are then used for data augmentation to improve classification accuracy. The classification task is evaluated using standard metrics such as accuracy, F1-score, precision, and recall. The realism of the generated ICNs is evaluated using a combination of visual qualitative analysis and quantitative metric analysis to ensure a comprehensive evaluation of the quality of the generated ICNs.

4.2. 2D ICN generation: Comparative analysis with benchmark models

The performance of 2D ICN generation was evaluated using various models, including spatially constrained ICA, VQ-VAE, progressive GAN, LDM, DDPM, and TransUNET-DDPM. This comparison revealed notable differences among the models, both in terms of output quality and efficiency, as shown in Fig. 3. The evaluation was conducted on seven distinct ICNs belonging to subcortical, auditory, sensorimotor, visual, cognitive control, default mode, and cerebellar domains.

Figure 3:

Visual comparison of 2D ICN generation across various models. This figure demonstrates the comparative performance of different models in generating 2D ICNs belonging to 7 different functional domains of the brain. Notable differences in output quality are observed, emphasizing the advantages and limitations of each model in capturing the complex structure of brain networks.

Although ICA is not a generative model, unlike the rest of the models, it performed well in capturing most of the structural elements of ICNs. It was particularly effective at identifying and capturing linear relationships in the data, resulting in images that accurately reflected simple and uniform patterns present in the ICNs. However, ICA struggled with more complex, non-linear relationships found in intricate brain networks. This limitation arises from ICA’s design, which focuses on independent components and linear separation, making it less effective in modeling complex neural connections. Next, the ICNs generated by VQ-VAE exhibited low resolution and artifacts, with noticeable blurring and spatial discontinuities. The model highlighted regions of interest but struggled to maintain semantic segmentation, resulting in a merged, indistinct representation of the brain regions. This lack of sharp differentiation reduced the clarity and precision of the generated networks. Meanwhile, progressive GANs, although known for generating high-resolution images, performed poorly when tasked with generating 2D ICNs. The visual representation of the generated ICNs highlighted a significant number of artifacts and random noise. It also exhibited mode collapse, producing repetitive ICNs that lacked the complexity and diversity of the training data.

In contrast, LDMs generated smoother images and outperformed both VQ-VAE and progressive GANs in terms of perceptual quality, global coherence, and structural consistency. Notably, for one specific ICN from the cerebellar domain, LDMs capture fine-grained details more effectively than all other models. However, in other ICNs, they exhibited over-smoothing, loss of fine structural details, and minor blurring in regions with complex spatial features. Finally, the ICNs generated by the DDPM and TransUNET-DDPM models were notably superior, effectively capturing both linear and non-linear dependencies within the data. The models’ iterative, step-by-step denoising process led to highly detailed and distinct ICNs. Compared to baseline DDPMs, TransUNET-DDPMs demonstrated a clearer representation of non-linear dependencies, particularly in the cognitive control domain, and produced more visually coherent and sharper ICNs for the default mode and subcortical domains. The transformer block in the TransUNET-DDPMs contributed to this improvement by enhancing global contextual understanding and allowing for better integration of spatially distant features during the generation process.

To demonstrate the generative behavior of the proposed TransUNET-DDPM model, Fig. 5 shows examples of partially denoised ICNs at different stages of the diffusion process, specifically at 0%, 25%, 50%, 75%, and 100% of the total steps. In the early stages (0–25%), the outputs are mostly random patterns and lack any clear anatomical structure. As denoising progresses to intermediate stages (50–75%), emerging spatial patterns begin to resemble functional brain regions, with increasingly sharper boundaries and reduced noise artifacts. By the final stage (100%), the model produces well-formed ICNs that closely resemble functional brain regions. This step-by-step refinement highlights how effectively the diffusion-based model learns to capture complex spatial relationships in fMRI data.

Figure 5:

Progressive denoising of ICNs generated by the TransUNET-DDPM model at different stages of the diffusion process. The visualization highlights the gradual refinement of spatial features as noise is incrementally removed. This demonstrates how the model reconstructs structured and meaningful ICNs from initial noise.

A comprehensive visualization of the reverse diffusion process, focusing on pixel intensity distributions over time, is shown in Fig. 4. An overlapping plot captures the denoising trajectory from a Gaussian prior to the generated data distributions, with the $y$-axis representing timesteps from $T=1000$ down to 0. These visualizations reveal that, during denoising, the model generates images that reflect the statistical properties of the original data. Although not exact replicas, the generated images capture the diversity inherent in the learned distribution. In the plot, the left side shows the initial Gaussian prior, while the right side presents the generated distributions from four different 2D ICNs. The final probability density plot highlights four distinct peaks, and each peak represents the unique attributes of an individual ICN.

Figure 4:

Visualization of the reverse diffusion process modeled as a stochastic differential equation (SDE). The process begins with a noisy prior distribution (left), typically a Gaussian, and iteratively denoises to produce a generated data distribution (right) that approximates the true data distribution.

Table. 2 presents a comparative analysis of various generative models, evaluating them based on parameter count, computational cost, training and inference times, as well as reconstruction quality metrics including PSNR, MSE, and MS-SSIM. These metrics revealed significant differences that affect their practical applicability for 2D ICN generation. In the case of VQ-VAE, it demonstrates high efficiency with minimal parameters and inference time, but falls short in reconstruction quality. Progressive GAN, though computationally intensive, offers lower performance across various image quality metrics. LDM achieves a balance between resource usage and output quality, while DDPM shows strong performance in PSNR and MS-SSIM with moderate resource requirements. Notably, TransUNET-DDPM outperforms all other models in reconstruction quality, achieving the highest PSNR of 24.55, the lowest MSE of 0.004, and the best MS-SSIM of 0.64, while maintaining competitive efficiency. Hence it has demonstrated its effectiveness as a high-performance, diffusion-based model for image generation tasks.

Table 2:

Comparison of generative models on 2D-ICN generation performance and efficiency.

Model	Params (M)	FLOPS (MFLOPS)	Training time (minutes)	Inference time (s)	PSNR	MSE	MS-SSIM
VQ-VAE	0.056	29.79	47.27	0.0014	12.61	0.03	0.31
Progressive GAN	12.13	8631.47	178	0.05	9.6	0.11	0.10
LDM	0.75	0.72	50.67	2.27	14.94	0.03	0.35
DDPM	1.03	239.24	34.32	4.75	20.77	0.008	0.58
TransUNET-DDPM	1.06	247.78	36.23	5.91	24.55	0.004	0.64

4.3. 3D ICN generation and validation on an external dataset

The ICN images generated using 3D TransUNET-DDPMs are illustrated in Fig. 6. A single-subject rs-fMRI data generates 53 ICNs, with each of them belonging to seven different functional domains, such as subcortical, auditory, sensorimotor, visual, cognitive control, default mode, and cerebellar domains. These DDPMs are advanced generative models used to iteratively learn complex distributions by gradually denoising data from a noise process. In this case, the TransUNET-DDPMs generate spatially meaningful ICNs from rs-fMRI data by capturing the spatiotemporal correlations in brain networks. The primary evaluation was performed on the UK Biobank test set, resulting in a higher mean MS-SSIM of 0.80, a lower MSE of 0.001, and a higher PSNR of 31.20. Subsequently, the ICNs were validated on the HCP dataset by comparing them with ICNs derived using spatially constrained ICA. Here, we utilized 500 generated images from each of the models for this comparative analysis. Quantitative metrics yielded a mean MS-SSIM of 0.72, MSE of 0.002, and PSNR of 27 across the generated ICNs, as shown in Fig. 7. Since ICA-derived ICNs represent linear ground truths modeling spatial independence, the observed mean MS-SSIM of 0.72 confirms that the DDPM model captures meaningful nonlinear relationships and hierarchical structures. It also indicates that the model captures higher-order interactions between brain networks, which are not fully modeled by traditional ICA. Moreover, the superior performance observed on the UKBiobank test set is attributable to the distributional alignment between the training and testing data, as the model was trained on UKBiobank samples.

Figure 6:

3D-ICN generation from single-subject rs-fMRI data using TransUNET-DDPM 3D model. The model produces 53 ICNs, grouped into 7 distinct functional domains.

Figure 7:

Validation of the TransUNET-DDPM 3D model on an external dataset. While TransUNET-DDPMs produce ICNs that closely resemble ICA-derived networks, maintaining moderate similarity values ensures that the models capture meaningful variability rather than overfitting to training patterns.

While quantitative performance metrics such as SSIM, MSE, and PSNR were evaluated in the 2D unconditional generation setting, where TransUNET-DDPM outperformed other models, including DDPM, this section highlights training efficiency in the more complex 3D conditional generation task. For 3D TransUNET-DDPM, the model is first pretrained on a large set of unconditional ICNs, followed by fine-tuning on a smaller conditional dataset, where the conditioning input is the rs-fMRI data. In Fig. 8, we compare the total pretraining and fine-tuning times of several 3D generative models, including GANs, LDMs, DDPMs, and TransUNET-DDPM, to assess scalability and efficiency. GANs exhibit the highest computational cost, requiring over 7 hours for pretraining and more than 33 hours for fine-tuning, due to their instability and slower convergence in high-dimensional data generation. LDMs offer moderate improvements but still require more time than other diffusion-based models. In contrast, both DDPM and TransUNET-DDPM complete pretraining in under 1 hour and 35 minutes, with fine-tuning times remaining below 7 hours. These results underscore the training efficiency and scalability advantages of diffusion-based architectures, particularly TransUNET-DDPM, for 3D conditional medical image generation.

Figure 8:

Comparison of pretraining and fine-tuning times for various conditioned 3D generative models.

4.4. Convergence and sensitivity analysis

In order to further validate the robustness and stability of the proposed TransUNET-DDPM architecture, we conducted an extensive convergence and sensitivity analysis using the external HCP dataset. The convergence behavior was assessed by monitoring both training and validation loss curves, specifically the MSE, averaged across multiple independent runs. As illustrated in Fig. 9, the model exhibits smooth and consistent convergence across all metrics, indicating a stable optimization process with no unstable fluctuations observed, even when training on high-dimensional fMRI volumes. These results indicate that the model effectively learns the underlying distribution of the data and maintains training stability despite the known challenges associated with diffusion models.

Figure 9:

Training and validation convergence curves of the 3D TransUNET-DDPM model. The plot shows the MSE averaged across multiple runs, illustrating smooth and stable convergence for both training and validation phases.

In addition to convergence, a sensitivity analysis was performed across the key hyperparameters, including the number of diffusion steps, learning rate, transformer depth, and noise schedule. The results in Fig. 10 show that model performance varies with these configurations but remains within a stable range. Increasing diffusion steps from 250 to 1000 progressively improved MS-SSIM and PSNR scores, with the best performance observed at 1000 steps. This indicates that a finer denoising process enables better preservation of spatial detail. A learning rate of 0.0001 consistently achieved optimal results, outperforming both faster 0.001 and slower 0.00001 learning rates in terms of convergence and reconstruction quality. Similarly, a transformer depth of 8 layers provided the best trade-off between representation capacity and training efficiency. Regarding the noise schedule, a linear configuration led to the highest overall performance, outperforming cosine and sigmoid variants, due to its stable and predictable noise reduction pattern. Based on these observations, the ideal hyperparameter configuration for the proposed TransUNET-DDPM was determined to be 1000 diffusion steps, a learning rate of 0.0001, transformer depth of 8, and a linear noise schedule, which collectively ensured both high-fidelity ICN generation and training stability.

Figure 10:

Sensitivity analysis of the 3D TransUNET-DDPM model across different hyperparameters such as diffusion steps, learning rate, transformer depth, and noise schedule. Performance evaluation is done using metrics such as MS-SSIM, PSNR, and MSE. Optimal performance was observed with 1000 diffusion steps, a learning rate of 0.0001, transformer depth of 8, and a linear noise schedule.

4.5. Quantitative analysis of subject-specific ICN generation

The correlations of the ICNs generated by the TransUNET-DDPM model provide important information into how effectively the model captures intra- and inter-subject variability. Intra-subject correlation measures the consistency of ICNs derived from the same individual under varying temporal conditions. Specifically, each subject’s rs-fMRI data, consisting of 490 time points, is split into two equal temporal segments: the first 245 and the last 245 time points. Independent ICNs are extracted from each segment, and a cross-correlation coefficient is computed between the two sets for each subject. The blue bars in Fig. 11 illustrate these intra-subject correlations, averaged across multiple model runs for four subjects. The resulting values range from 0.70 to 0.75, indicating that the TransUNET-DDPM model robustly captures temporally stable, subject-specific network features.

Figure 11:

Comparison of intra-subject and inter-subject correlations for TransUNET-DDPM-generated ICNs across four subjects.

In contrast, inter-subject correlations assess the similarity of ICNs across different individuals. This is computed by correlating the ICNs of one subject with those of the remaining subjects. These between-subject correlation values were lower, falling within the range of 0.30 to 0.32. The relatively low inter-subject similarity indicates that the model is sensitive to individual-level functional network characteristics. It was also observed that the intra-subject correlations exceed inter-subject correlations for all the subjects. This also confirms that the model is capturing individualized ICN features and effectively modeling subject-specific variability.

Furthermore, Fig. 12 compares ICNs generated by ICA and TransUNET-DDPM for a single subject. Similar to the correlation analysis, the visual comparison shows that TransUNET-DDPM captures more subject-specific variability, while ICA tends to reflect population-level consistency. Fig. 13 extends this comparison across subjects, revealing that TransUNET-DDPM produces more distinct ICNs for different individuals, indicating sensitivity to individual differences. In contrast, ICA-generated ICNs appear more uniform across subjects, with only minor variations. This distinction is further supported by a quantitative analysis of fractal dimension (FD), a measure of structural complexity that captures how the complexity of a pattern evolves across different spatial scales. The average FD of ICNs generated by TransUNET-DDPM is 3.15, significantly higher than the 1.75 average FD observed in ICA-derived ICNs. This indicates that the ICNs produced by TransUNET-DDPM exhibit greater spatial complexity and finer-grained subject-specific features.

Figure 12:

Comparison of ICNs generated using ICA and TransUNET-DDPMs for the same subject. ICA yields more stable and generalized ICNs, while DDPM produces ICNs that better reflect the subject’s unique connectivity pattern.

Figure 13:

Comparison of ICNs generated for multiple subjects using ICA and TransUNET-DDPM. ICA-generated ICNs show less variation across subjects, indicating a focus on global, population-level features. TransUNET-DDPM generated ICNs show more individual variation, capturing finer-grained connectivity differences unique to each subject.

4.6. Evaluation of class conditioned TransUNET-DDPM for schizophrenia classification

The performance of class-conditioned TransUNET-DDPMs was evaluated by analyzing their impact on schizophrenia classification metrics. Initially, classification accuracy, F1 score, recall, and precision were computed using a 3D CNN on the smaller fBIRN dataset, which consisted of ICA-derived ICNs from 311 subjects. As summarized in Table 3, the baseline 3D-CNN model trained solely on real ICN features achieved the highest classification performance, with an accuracy of 75.3%, an F1 score of 76.0%, a precision of 76.1%, and a recall of 76.6%. In contrast, alternative architectures such as the 3D-ResNet and Vision Transformer (ViT) demonstrated notably lower performance, indicating that the 3D-CNN is more suitable for this task and data representation. To assess the impact of synthetic data augmentation, the experiments were extended across three augmentation ratios: 0x (real-only), 0.5x (real + 0.5× synthetic), and 1x (real + 1× synthetic), using high-fidelity diffusion-generated synthetic ICNs. The results, averaging across multiple cross-validation folds and depicted in Fig. 14, reveal consistent improvements across all evaluated metrics with increasing synthetic data augmentation. Here, accuracy improved from 75.0% to 87.0%, F1 score from 76.0% to 84.0%, precision from 76.0% to 80.0%, and recall from 77.0% to 91.0%, demonstrating the efficacy of controlled synthetic augmentation in enhancing classification robustness. Furthermore, the reduced standard deviations observed at higher augmentation levels indicate that synthetic data not only boosts average performance but also stabilizes training outcomes.

Table 3:

Comparison of classification performance for schizophrenia detection using different deep learning models trained on real ICN-based features. The performance metrics include mean ± standard deviation across cross-validation folds.

Data type	Classifier	Accuracy (%)	F1 Score (%)	Precision (%)	Recall (%)
Baseline (Real ICNs only)	3D-CNN	0.7525±0.0287	0.7596±0.0308	0.7614±0.0492	0.7656±0.0760
	3D-ResNet	0.5817±0.0674	0.5130±0.2738	0.5045±0.2666	0.6058±0.3806
	ViT	0.6238±0.1027	0.6589±0.0945	0.6172±0.0854	0.7096±0.1169

Figure 14:

Performance metrics over multiple runs for three augmentation conditions: real-only (0x), partial augmentation (0.5x), and full augmentation (1x) using diffusion-generated synthetic data.

Fig. 15 presents a comparative analysis of image quality metrics between the control and schizophrenia groups using MS-SSIM, MSE, and PSNR. This comparison highlights variations in image reconstruction quality between the two groups, offering insights into distinct patterns of image representation. The first panel, corresponding to MS-SSIM, reveals that both groups achieved relatively high similarity values, ranging between 0.74 and 0.89. Notably, the schizophrenia group shows a slightly higher median MS-SSIM compared to the control group, and a broader range of values. This indicates that while structural similarity between reconstructed and reference images is generally preserved in both groups, the schizophrenia group exhibits more variability. The second panel presents the MSE results, where the control group demonstrates consistently low error values, mostly below 0.03, reflecting stable reconstructions. The schizophrenia group shows a broader distribution, with some higher MSE values reaching up to 0.19. This wider spread indicates greater variability in reconstruction patterns, which reflects a more diverse range of image features being captured. Finally, the third panel displays PSNR, where the control group shows higher values, generally between 30 and 36 dB, indicating high-quality, precise reconstructions. In contrast, the schizophrenia group presents a slightly wider range, from 26 to 34 dB, indicating more variability in image quality. This variation points to distinctive features and greater contrast in the schizophrenia group compared to the control group. Further, a comparison of the seven default mode network (DMN) ICNs generated using the class-conditioned TransUNET-DDPM for both controls and individuals with schizophrenia is shown in Fig. 16. This revealed significant visual differences between the two groups, with the schizophrenia group exhibiting more pronounced variability and less coherence in the network patterns compared to the control group.

Figure 15:

Comparison of image quality metrics (MS-SSIM, MSE, PSNR) between control and schizophrenia groups. Boxplots show the distribution of metric values across subjects for each group. The schizophrenia group exhibits generally higher MS-SSIM values compared to controls, indicating better structural similarity. However, MSE values are higher and PSNR values are lower for the control group.

Figure 16:

Comparison of default mode domain ICNs generated using conditional TransUNET-DDPM for controls (left) and schizophrenia (right).

To enhance model interpretability, a saliency mapping technique was employed to compute the gradient of the model’s prediction score with respect to the input ICNs. For each generated ICN and a specific diagnostic label, the gradient of the model’s output probability was calculated with respect to each voxel in the input. This gradient indicates the saliency of each voxel in contributing to the model’s prediction. To visualize model attention, 3D saliency volumes were projected to 2D by averaging over the width dimension, producing axial-plane representations. The resulting saliency maps in Fig. 17 revealed distinct activation patterns between individuals with schizophrenia and healthy controls. Notably, regions commonly affected in schizophrenia, such as the anterior cingulate cortex, posterior cingulate cortex, and insula, consistently exhibited higher saliency in the schizophrenia group. These regions align with key functional networks, including the default mode network (DMN; ICNs 42–48) and the cognitive control domain (CC; ICN 26), both of which are consistently associated with disrupted connectivity in schizophrenia. Overall, this analysis supports the utility of the proposed generative model, demonstrating that it not only produces realistic-looking ICNs but also captures diagnostically relevant information.

Figure 17:

Gradient-based saliency maps highlighting the most influential brain regions used by the model during classification. These 2D visualizations are derived from 3D ICN volumes and provide insight into whether the model focuses on spatial patterns consistent with known neurobiological features of schizophrenia.

5. Discussion

This study highlights the effectiveness of the proposed TransUNET-DDPM framework in generating spatially coherent and functionally meaningful ICNs from resting-state fMRI data. The integration of transformer-based architectures with diffusion probabilistic models demonstrates substantial improvements qualitatively and quantitatively over existing state-of-the-art approaches. In this study, ICA was included as a baseline due to its established and widespread use as a standard method for deriving ICNs from resting-state fMRI data. While ICA does not generate synthetic data, it serves as a conventional reference point for evaluating the anatomical and functional validity of ICNs, especially in neuroimaging studies. The comparison allowed us to demonstrate the advantages of the proposed TransUNET-DDPM model in capturing complex, nonlinear spatiotemporal patterns that ICA cannot model due to its linear assumptions. Furthermore, comparing against ICA provides meaningful context for the neuroscientific relevance and interpretability of the ICNs generated by our model. Many prior studies, including those on schizophrenia and other psychiatric disorders, rely on ICA-derived ICNs for classification and functional connectivity analyses [32, 47]. By demonstrating that TransUNET-DDPM generates ICNs with comparable or superior anatomical and functional quality relative to conventional approaches, we underscore the practical utility of adopting advanced generative models in clinical and cognitive neuroscience applications. Overall, compared to benchmark models, TransUNET-DDPM significantly enhances image fidelity and domain-specific realism, and its selection was motivated by both its architectural compatibility with 3D neuroimaging data and its ability to model the complex characteristics of ICNs in resting-state fMRI. TransUNET combines a UNet-based encoder-decoder with a transformer-based bottleneck. This architecture is especially well-suited for 3D fMRI data, where both local anatomical precision and global contextual understanding are essential. Unlike ViT or Swin Transformer, TransUNET-DDPM uses the GRIT block, a transformer module explicitly tailored for 2D and 3D embeddings. GRIT introduces architectural elements such as learnable residual gating, internal token projection, and layer scaling. These features improve stability and training dynamics in diffusion-based generative models.

Prior work has shown that CNN-based denoising diffusion models struggle to capture global spatial dependencies in brain images, a challenge addressed by incorporating transformer backbones [10, 48]. This is particularly evident in the cognitive control and default mode networks, where the global context is crucial for accurate representation. In contrast, VAE- and GAN-based models have been found to suffer from mode collapse and spatial blurring in medical imaging applications [49, 50], issues that the iterative denoising process in DDPMs helps to mitigate. To assess architectural choices more comprehensively, preliminary experiments compared TransUNET-DDPM against ViT- and Swin-based DDPMs. ViT-DDPM achieved an MS-SSIM of 0.59, PSNR of 23.16, and MSE of 0.004, while Swin-DDPM resulted in an MS-SSIM of 0.60, PSNR of 22.62, and MSE of 0.006. Although these results slightly outperformed a baseline DDPM, both were consistently outperformed by TransUNET-DDPM across all evaluation metrics. Moreover, the inference times were substantially longer for ViT (9.1 seconds) and Swin (6.38 seconds) compared to TransUNET, indicating higher computational costs and less efficiency. Therefore, even when ViT or Swin Transformers were applied only at the bottleneck, their attention mechanisms, particularly Swin’s windowed self-attention and hierarchical design, introduced greater computational overhead and disrupted spatial continuity. In contrast, GRIT-enhanced TransUNET achieved spatially consistent learning and better fidelity in generated ICNs.

Another comparative analysis of various generative models for 2D-ICN generation (Table 2) highlighted clear trade-offs between computational efficiency and reconstruction quality. VQ-VAE, with just 0.056M parameters, 29.79 MFLOPS, and an inference time of 0.0014s, is highly efficient and suitable for real-time, resource-limited scenarios but suffers from low image quality. Progressive GAN, despite its historical use in image synthesis, exhibits the poorest trade-off, demanding the highest computational cost (8631.47 MFLOPS) while delivering the lowest performance across all quality metrics. LDM achieves better efficiency by operating in a latent space (0.72 MFLOPS, 2.27s), but with only moderate reconstruction fidelity. DDPM showed a stronger balance, offering good reconstruction quality with reasonable cost (239.24 MFLOPS, 4.75s), making it a solid diffusion-based baseline. In contrast, TransUNET-DDPM achieves the highest overall performance while maintaining competitive efficiency (1.06M parameters, 247.78 MFLOPS, 5.91s inference). Its effectiveness is due to its hybrid UNet and transformer architecture, enhanced by an optimized noise schedule. Compared to DDPM, the best-performing model, TransUNET-DDPM improves PSNR by 18.2%, reduces MSE by 50%, and increases MS-SSIM by 10.3%, with only a 24% increase in inference time and a negligible 3.6% increase in FLOPS. This strong balance between quality and efficiency makes TransUNET-DDPM a highly practical solution for high-fidelity 2D-ICN generation, particularly in scenarios where high accuracy is critical and moderate computational resources are available.

The quantitative assessment of the realism and stability of the generated 3D-ICNs was conducted using standard generative evaluation metrics [51] and the MS-SSIM scores achieved values of 0.80 on the UK Biobank test set and 0.72 on the external HCP dataset. These results further confirm that TransUNET-DDPM effectively preserves structural integrity and ensures high signal-to-noise quality in the synthetic outputs. In terms of subject-specific ICN generation and intra-subject consistency, the observed intra-subject correlations (0.70–0.75) versus inter-subject correlations (0.30–0.32) indicate that the model effectively captures subject-specific functional signatures. Prior studies [52] have also demonstrated the uniqueness of individual functional connectomes, and our results extend this by showing that generative models can reproduce this variability reliably. While ICA-generated ICNs often reflect group-level patterns, as noted in previous studies [53, 54], TransUNET DDPM produces outputs that better preserve individual-specific features. Although ICA is widely used in neuroimaging, it is limited by its dependence on linear decomposition assumptions about independence and non-Gaussianity. This restricts its ability to extract the complex, non-linear relationships present in brain networks [55, 56]. On the contrary, the proposed model effectively captures non-linear relationships and generates ICNs with their adherent variability. This is supported by fractal dimension analysis, where TransUNET-DDPM ICNs achieved an average value of approximately 3.15, compared to 1.75 for ICA-derived ICNs, indicating that the former captures a much more intricate and multi-scale representation of brain network dynamics.

Additionally, the effect of synthetic data augmentation on schizophrenia classification performance was further evaluated, highlighting the benefits of employing class-conditioned TransUNET-DDPMs to improve classification accuracy and robustness. The performance of the baseline model without data augmentation indicates that it had high recall but struggled with false positives and overprediction of the schizophrenia class. Following dataset augmentation with synthetic ICNs, all evaluation metrics exhibited improvement, indicating enhanced model reliability and class balance. Furthermore, in the visual analysis of the seven ICNs within the DMN domain, clear differences were observed between schizophrenia and control groups. Specifically, the schizophrenia group exhibited more spatially diffuse and irregular ICN patterns, with increased focal activation in key DMN regions such as the precuneus, anterior cingulate cortex (ACC), and posterior cingulate cortex (PCC). These regions play a crucial role in supporting internally directed cognitive functions [57]. The ACC, in particular plays a key role in cognitive control and emotional regulation. In individuals with schizophrenia, decreased ACC activity and connectivity have been associated with deficits in error processing, decision-making, and executive functioning [58]. These findings align with previous literature indicating disrupted DMN connectivity in schizophrenia, which has been associated with impairments in self-referential processing, memory retrieval, and the integration of emotional and cognitive information [59]. Moreover, decreased DMN connectivity has been linked to more severe positive psychotic symptoms and poorer clinical outcomes in individuals with schizophrenia [60]. Our model’s ability to generate and distinguish these specific spatial patterns not only validates its fidelity but also demonstrates its potential to reveal clinically meaningful biomarkers through synthetic ICN analysis.

6. Conclusion and Future Work

In conclusion, our work presents a novel framework that leverages the generative and nonlinear nature of TransUNET-DDPMs to synthesize high-quality 3D ICNs by training on a large and diverse dataset. By integrating the strengths of diffusion probabilistic models with the global contextual modeling capabilities of transformers, the model outperforms traditional generative methods in both unconditional and conditional settings. It not only preserves structural and functional characteristics of ICNs across multiple domains but also captures subject-specific features with a high degree of intra-subject consistency. Its strong performance in clinically relevant tasks, such as schizophrenia classification, highlights its potential for both research and diagnostic applications. Furthermore, the modular design of TransUNET-DDPM allows for direct adaptation to other medical imaging modalities beyond rs-fMRI. The core components, such as the transformer-based UNet backbone and the diffusion-based generative process, are modality-agnostic and can generalize to a wide range of image types with appropriate data preprocessing and conditioning strategies. This flexibility enables applications in structural MRI, CT, and PET imaging by modifying the input channels and adding various conditioning mechanisms such as anatomical labels, segmentation masks, and clinical metadata. Additionally, the framework’s ability to handle high-dimensional, noisy data makes it well-suited for tasks like lesion synthesis, organ segmentation, disease progression modeling, and modality translation.

Despite its strong performance, the TransUNET-DDPM model does have several limitations. Firstly, it relies on large-scale pretraining data, which may not always be available in clinical and low-resource settings. Additionally, the model’s inference time is relatively high due to the iterative denoising process involved in diffusion models. Our future work aims to address these limitations through several promising directions. First is accelerating inference time by exploring techniques such as reducing the number of denoising steps and utilizing more efficient sampling strategies, such as classifier-free guidance and Laplacian pyramid sampling. Secondly, future work may also explore the previously mentioned applications by fine-tuning the pretrained TransUNET-DDPM on alternative datasets and extending it to multi-modal fusion scenarios. This flexibility highlights the potential of the proposed framework as a general-purpose generative model for diverse medical imaging tasks.

Highlights.

• TransUNET-DDPM combines transformers with diffusion models to generate anatomically and functionally accurate 3D brain connectivity networks.

• A transfer learning strategy enables efficient training and domain adaptation from large-scale unsupervised data to condition-specific neuroimaging tasks.

• Class-conditioned ICN synthesis improves schizophrenia classification by augmenting scarce datasets with diagnostically meaningful synthetic samples.