Skip to main content
NCBI home page

This is a preprint.

It has not yet been peer reviewed by a journal.

The National Library of Medicine is running a pilot to include preprints that result from research funded by NIH in PMC and PubMed.

Research Square logoLink to Research Square
[Preprint]. 2026 Sep 9:rs.3.rs-10851563. [Version 1] doi: 10.21203/rs.3.rs-10851563/v1

Comparative Study of Anatomical and Learned Features in AI Models for Structural Brain MRI

Boyang Yu 1,†, Miquel Lopez Escoriza 1,†, Long Chen 1,2, Arjun V Masurkar 2, Narges Razavian 2, Carlos Fernandez-Granda 1,3,*
PMCID: PMC13622820  PMID: 42818499

Abstract

In this work, we comprehensively evaluate three popular feature-extraction paradigms in AI-based neuroimaging modeling: (1) computation of anatomical surfaces and volumes, (2) supervised learning with convolutional neural networks (CNNs), and (3) unsupervised pretraining of vision transformer (ViT) foundation models, followed by supervised finetuning. Our study is based on 18 publicly available datasets containing 3D structural T1-weighted MRI scans from approximately 80,000 participants across seven distinct clinical tasks. We observe that a linear model based on anatomical features matches the diagnostic performance of complex nonlinear features learned by sophisticated AI frameworks, including foundation models trained on thousands of scans. Conversely, CNNs and pretrained ViTs learn features that implicitly capture relevant anatomical information, bypassing the need for explicit feature extraction. Building upon these insights, we propose Anatomy Segmentation Pretraining (ASP), a novel method to incorporate anatomical information during foundation-model pretraining, which outperforms existing models in biological age estimation.

Keywords: Deep learning, pretraining, neuroimaging, brain anatomy, segmentation

Introduction

Advances in machine learning and computer vision have exerted a major influence on neuroimaging over the past two decades. Initially, these models relied on hand-engineered features, such as hippocampal volume or cortical thickness, combined with classical statistical models (e.g. logistic regression or support vector machines) [1-4]. Subsequently, modern convolutional neural networks (CNNs) enabled automatic learning of complex features, which are challenging to interpret anatomically, but achieved strong performance for a variety of neuroimaging tasks [5-8]. More recently, foundation models based on vision transformers (ViTs) [9] are able to learn features from vast unlabeled datasets [10, 11]. They can then be finetuned for different downstream tasks. Several recent works adapt this approach to neuroimaging data [12-15], for example to develop multimodal models.

In this work, we perform a comparative study of AI models for neuroimaging, with a focus on 3D structural T1-weighted MRI scans. T1-weighted MRI is a standard imaging modality that enables visualization of anatomical detail at a high resolution and is a routine part of most clinical protocols. Figure 1 provides an overview of the study. We consider seven different clinical tasks: diagnostics of Alzheimer’s disease, epilepsy, Parkinson’s disease, identification of tumor IDH mutations, brain age prediction and segmentation of brain tumors and stroke lesions. Overall, we use 18 different publicly-available datasets with more than 96,000 MRI scans from approximately 80,000 participants for training and evaluation.

Fig. 1. Overview of the study.

Fig. 1

Open in a new tab

(a) Cohorts used in the study. Cohorts used for pretraining, downstream training, internal testing, and external evaluation. Reported n values denote MRI scans; all splits were performed at the participant level, such that all scans from the same participant were assigned to the same split. External cohorts were drawn from independent data sources. (b) AI models included in the study. These represent the main feature-extraction approaches for neuroimaging data: linear models based on anatomical features, convolutional neural networks (CNNs) and vision transformers (ViTs) pretrained on large general-population cohort. For the ViTs, in addition to standard pretraining based on masked autoencoding, we propose a novel scheme called Anatomical Segmentation Pretraining (ASP).

Our overarching goal is to compare the three main strategies to obtain features for AI-based modeling in neuroimaging: (1) extraction of preselected anatomical characteristics such as volumes and surfaces of brain regions, (2) supervised learning with convolutional neural networks, and (3) unsupervised pretraining of foundation vision transformers on large general-population cohorts, followed by supervised finetuning. As reported in the Results section, the three strategies yield similar performance across the different tasks. Our results indicate that preselected anatomical features match the diagnostic performance of complex nonlinear features learned by sophisticated AI frameworks, including foundation models trained on thousands of scans. Conversely, convolutional neural networks and pretrained vision transformers learn features that implicitly capture relevant anatomical information, bypassing the need for computationally intensive feature extraction.

In the case of ViTs pretrained on a large general-population cohort, we show that anatomical information is learned already during the pretraining stage, when the learned features become increasingly predictive of different brain structures. Inspired by this observation, we propose a novel pretraining strategy (Anatomy Segmentation Pretraining or ASP), which aims to incorporate explicit anatomical grounding in the training of neuroimaging foundation models. As described in the Methods section, this is achieved by forcing the model to perform segmentation of relevant brain structures during pretraining. While this approach performs on par with the other strategies across the clinical tasks, it achieves superior performance in estimating biological age, a task for which a larger number of training data is available.

In summary, this comparative study demonstrates that explicit anatomical features remain competitive with representations learned by advanced deep learning models, particularly across clinical cohorts ranging from dozens to thousands of training examples. Furthermore, we introduce a novel, anatomy-guided pretraining framework that explicitly leverages anatomical information to enhance the pretraining of neuroimaging foundation models.

Results

Figure 1 provides an overview of the study. We compare the following AI models, adapted to process 3D MRI scans, across different tasks. The models are chosen to represent the main current approaches for image classification and segmentation in data-driven computer vision.

  • Linear logistic-regression models based on anatomical features consisting of 178 volumes and surfaces of brain regions (denoted Anatomical in the figures).

  • Convolutional neural networks, which are either ResNets [16] for classification tasks (denoted CNN) or U-Nets [17] for segmentation tasks.

  • Vision-transformer models [9] pretrained on a large pretraining cohort via masked autoencoding (denoted Pretrained ViT).

  • ViTs pretrained using a novel strategy (Anatomy Segmentation Pretraining, denoted ASP ViT) that leverages anatomical information extracted via segmentation from the scans in the pretraining cohort.

The pretraining cohort includes 69,878 MRI scans from 65,249 subjects, primarily from UK Biobank[18]. More details about the pretraining data are provided in the appendix (Section A.2). In addition, we also include comparisons with BrainIAC [14], a ViT pretrained using contrastive learning on a different pretraining cohort of 32,000 MRI scans, and UNETR [12], a medical image segmentation architecture that combines a ViT encoder with a U-Net–style decoder.

The models are trained to perform the following seven tasks. The results are shown in Figure 2. Additional details about the datasets used to build the training and evaluation cohorts for each task, and the pretraining cohort, are provided in Table 1 and in the appendix (Section A).

Table 1.

Datasets used to build the pretraining cohort, and the training and evaluation cohorts for each task in the study.

Dataset Number of scans Number of subjects Age (years) (mean ± std) Gender (female %) Brain volume in mL (mean ± std)
UKBB 67294 62665 64.93 ± 7.69 52.81% 1156 ± 114
HCP 1350 1350 38.66 ± 25.76 54.97% 1179 ± 137
CamCAN 653 653 54.2 ± 18.5 49.35% 1156 ± 128
IXI 581 581 48.65 ± 16.47 53.87% 1137 ± 131
NACC 6956 4926 71.41 ± 11.25 53.34% 1034 ± 185
ADNI 6471 1817 74.68 ± 7.64 47.45% 1072 ± 116
AIBL 1167 664 74.56 ± 6.79 52.87% 1078 ± 110
OASIS1 436 416 51.35 ± 25.27 61.47% 1089 ± 153
OASIS2 373 150 77.01 ± 7.64 57.10% 1045 ± 124
OASIS3 2275 1044 70.69 ± 9.18 48.39% 1061 ± 113
OASIS4 181 180 71.09 ± 9.60 46.96% 1060 ± 116
MIRIAD 705 69 69.65 ± 6.88 55.51% 1039 ± 115
PPMI 2032 1306 63.29 ± 9.58 40.59% 1119 ± 157
OpenNeuro 163 163 6.67 ± 2.47 46.01% 1195 ± 144
EPISURG 268 268 NA NA 1102 ± 133
UCSF-PDGM 1980 501 56.87 ± 15.02 40.32% NA
BraTS 2502 1251 NA NA 1103 ± 136
ATLAS 655 33 NA NA 1445 ± 124
Open in a new tab

  • Diagnostics of Alzheimer’s disease: The goal is to distinguish between healthy controls, patients with mild cognitive impairment and patients with Alzheimer’s disease. The training cohort consists of n=1,453 subjects from ADNI [19]. Evaluation was carried out on an internal test cohort extracted from ADNI (n=364), and seven external cohorts, extracted from NACC [20] (n=4,926), AIBL [21] (n=664), MIRIAD [22] (n=69), and OASIS-1 (n=416), OASIS-2 (n=150), OASIS-3 (n=1,044) and OASIS-4 (n=180) [23-25]. For the internal test cohort and six external cohorts, ASP ViT has the highest macro-AUC, but the difference with respect to the CNN and the anatomical logistic regression model is only statistically significant for OASIS 2. We didn’t apply BrainIAC to this task because its pretraining cohort includes several of our external test cohorts.

  • Diagnostics of Parkinson’s disease: The goal is to distinguish between patients with Parkinson’s and healthy controls. A training cohort (n=1,572) and an internal test cohort (n=460) were extracted from PPMI [26]. The Pretrained ViT achieves the highest AUC, but the difference is only statistically significant with respect to BrainIAC.

  • Detection of tumor IDH mutations: The goal is to identify brain tumors with Isocitrate Dehydrogenase (IDH) mutation. A training cohort (n=1,874) and an internal test cohort (n=106) were extracted from UCSF-PDGM [27], which contains labels obtained via genetic sequencing of tissue acquired during biopsy or resection. ASP ViT achieves the highest AUC, but the difference with the remaining models is not statistically significant.

  • Diagnostics of epilepsy: The goal is to distinguish between patients with epilepsy and healthy controls. A training cohort (n=82) and an internal test cohort (n=81) were extracted from OpenNeuro [28]. The models were also applied to Episurg [29], an independent cohort containing only epilepsy cases (n=268). The Pretrained ViT achieves the highest AUC, but the difference with the remaining models is not statistically significant. We could not apply the CNN to this task due to the limited training data.

  • Whole tumor segmentation: The goal is delineation of adult brain gliomas. A training cohort (n=1,000) and an internal test cohort (n=251) were extracted from the BraTS 2023 Adult Glioma challenge [30-32]. The U-Net achieved the highest Dice score (slightly above ASP ViT), but the difference is only statistically significant with respect to UNETR. If the training data is reduced to 10%, ASP and Pretrained ViT outperform U-Net and UNETR.

  • Segmentation of stroke lesions: The goal is segmentation of brain lesions associated with stroke. A training cohort (n=355) and an internal test cohort (n=300) were extracted from ATLAS R2.0 [33]. The Pretrained ViT achieved the highest Dice score (slightly above ASP ViT), but the difference is only statistically significant with respect to UNETR.

Overall, we observed that there was no significant difference between the pretrained ViTs, the CNNs and linear models based on anatomical features across the different tasks.

We conjecture that the inability of the more complex deep-learning models to outperform the simpler linear model is due to the limited training data available for each task. In order to investigate this further, we trained the models to estimate biological age using a much larger training cohort drawn from UK Biobank, HCP, CamCAN and IXI (n=62,890). Evaluation was carried out on a held-out subcohort from UK Biobank (n=6,950), and on two external cohorts of healthy subjects extracted from ADNI (n=2,791) and NACC (n=1,136).

Figure 3 shows the results on age estimation. The ASP ViT model outperforms the rest, achieving a mean absolute error (MAE) of 2.54 years in the internal test cohort (a) and of 4.19 years in the NACC test cohort (b) and the ADNI test cohort (c). The Pretrained ViT is close in performance on the internal test cohort (0.35 years greater), but the gap widens for the external cohorts (more than 1 year greater). The logistic regression model performs worse than the ViT approaches by a significant margin (between 1 and 2 years), but clearly outperforms the CNN. The performance differences are maintained when stratifying the data by sex. Stratifying by age reveals that the differences between the models are mainly driven by the superior performance of the ViT models (particularly ASP ViT) on subjects with advanced ages (above 80), which are less represented in the training and pretraining cohorts, especially with respect to the CNN.

The best-performing age-estimation model was also applied to mildly-cognitively impaired (MCI) and dementia patients from the NACC and ADNI datasets, as shown in Figure 3(b,c). Age estimates for dementia patients were consistently above their biological age, in most age intervals for both cohorts. The effect also exists to some extent for MCI subjects, but is less pronounced.

Methods

In this section, we first present a new technique for pretraining vision transformers (ViTs) using anatomical information. Then, we describe the remaining methodology in our study, including masked autoencoder pretraining of ViTs, downstream application of pretrained ViTs, linear models based on anatomical features, convolutional neural networks, and data preprocessing.

Anatomy Segmentation Pretraining of Vision Transformer

We propose a novel approach to pretrain ViTs on MRI data. The goal is to inject anatomical information into the model during pretraining. The motivation is that simple models that rely only on anatomical information, such as the logistic-regression model with brain-region volume and surface features used in this study, can be very effective (see Results section).

We build upon a key observation: when pretraining ViTs via masked autoencoding (MAE), the model automatically learns embeddings that reflect anatomical information, but only to some extent. The green region in the graph from Figure 4(b) shows the accuracy of predicting the volumes of different brain regions from MAE embeddings via linear probing during training. The accuracy increases, indicating that the embeddings gradually capture anatomical information. In order to further boost the anatomical information learned by the model, we propose to incorporate brain-region segmentation as an auxiliary task during pretraining.

Our procedure to perform anatomy segmentation pretraining (ASP) is illustrated in Figure 4(a). A 3D ViT produces the embedding used for downstream tasks. After MAE pretraining (green region in Figure 4(b)), we train a lightweight mask decoder to perform segmentation of different brain regions in two phases. First, the ViT parameters are frozen to ensure that the decoder learns how to use the existing embedding for segmentation (yellow region in Figure 4(b)). Next, the encoder and decoder are trained jointly. In this ASP phase, segmentation and MAE pretraining are performed in alternating epochs to prevent the model from forgetting useful features learned via MAE. This strategy is common in continual-learning methods that balance new-task adaptation with the retention of prior knowledge [34-36].

Regions for ASP were selected based on their disease relevance and anatomical scale. We discarded regions that lacked a strong disease association and contributed less than 1% of the total brain volume. To further improve efficiency, symmetrical structures (such as the left and right hippocampus) and anatomically related sub-units (such as segments of the corpus callosum) were consolidated into single labels. This resulted in 39 distinct regions, listed in Figure 4(c). More details are provided in the Appendix Section A.7.

The segmentation architecture used to perform ASP is tailored specifically to the pretraining task. Deep-learning segmentation models typically rely heavily on skip connections, which emphasize local feature fusion and reduce the importance of global representations [17], [12]. In contrast, the goal of ASP is to concentrate the anatomical information in the ViT embedding. To address this, we use a bottleneck decoder architecture(see Figure C1), where the segmentation prediction is generated directly from the ViT embedding, without any skip connections. The decoder is lightweight, in order to further encourage the model to concentrate rich anatomical information in the ViT transformer. More details are provided in the Appendix Section C.2.

A key challenge is applying ASP to a relatively large number of anatomical regions (39, as explained above). A naive implementation based on a standard softmax output would be too memory intensive, as it would require generating 39 different 3D logit maps (39×128×128×128). Instead, we adopt a prompt-based architecture, inspired by [37], where a prompt is used to adapt the Segment Anything Model [38] to different segmentation tasks. A brain-region prompt indicates which brain region should be segmented by the lightweight decoder, and the decoder output is compared to a binary segmentation task indicating the appropriate region (see Figure 4(a)). The brain region is selected at random during pretraining.

We trained the ASP model for 400 epochs with a batch size of 16 per GPU. We used binary cross entropy and the AdamW optimizer (β1 = 0.9, β2 = 0.95, weight decay = 0.04), with a base learning rate of 5 × 10−4. We applied learning rate scheduling via cosine annealing with linear warm-up over the first 5 epochs. Convergence is observed at around 200 epochs, requiring approximately one month of training on a single 42GB NVIDIA RTX8000 GPU.

Figure 4(c) reports the performance of the decoder for the different regions before and after the ASP phase, showcasing that the model is indeed able to learn how to segment the different regions reasonably accurately (although its performance for smaller or more intricate region is relatively modest). Figure 4(d) shows some examples of segmentation outputs, compared to the ground truth. Figure 4(e) provides an ablation study, which establishes that each element in the proposed framework (prior MAE pretraining, initial freezing of the encoder to train the decoder, and alternating with MAE to avoid forgetting) all provide a non-negligible increase in performance for the downstream task of age estimation. Further information about the ablation study is provided in C.3.

Masked Autoencoder Pretraining of ViTs

Masked Autoencoding (MAE) is a self-supervised learning framework that trains ViTs by reconstructing masked images, which are missing a random subset of their pixels [11]. We adapted this framework to process 3D structural MRI volumes. We used a ViT-Base encoder adapted to 3D inputs, consisting of 12 transformer blocks with 12 attention heads and a 768-dimensional embedding, using 16×16×16-voxel patches. During pretraining, 75% of the 3D patches in each volume were randomly masked, and the ViT was trained to reconstruct the original volumes from the unmasked patches. Volumes were represented as single-channel inputs with intensities scaled to [0, 1]; images were padded to 256 × 256 × 256, randomly cropped to 128 × 128 × 128, and augmented with random flipping and intensity shifting. Pretraining used mean squared error (MSE) reconstruction loss for 1000 epochs with batch size 32 per GPU, AdamW [39] (β1 = 0.9, β2 = 0.95, weight decay = 0.005), and a cosine learning rate schedule [40] with base learning rate 0.0015 and linear warm-up during the first 2.5% of training steps. Convergence is observed at around 900 epochs, which took around 10 days on two 80GB NVIDIA A100 GPUs.

Downstream Application of Pretrained ViTs

To adapt the pretrained ViTs for downstream applications, we feed the resulting embeddings into task-specific output modules: a binary-classification head for diagnostic tasks, a linear-regression head for regression tasks (e.g. estimation of age or of anatomical features) and a U-Net–style decoder for voxel-level segmentation, following the UNETR architecture [12]. See Section C.1 for additional details. The pretrained ViTs were adapted to the different downstream tasks using the following strategies.

  • Linear probing: The linear regression and classification heads were trained to generate predictions from the ViT embedding, using the sklearn implementations of logistic regression and ordinary least squares. This approach was used as a diagnostic tool during pretraining to monitor representation quality using anatomical and demographic targets, as illustrated by Figure 4(b). It was also used for diagnostics of Parkinson’s disease and detection of tumor IDH Mutation, due to the small size of the training datasets.

  • Low-Rank Adaptation (LoRA): Both ViT encoders (ASP and pretrained) were adapted using LoRA [41, 42]: the original model parameters were kept frozen, and trainable low-rank matrices were introduced into the attention layers. These low-rank parameters were optimized jointly with the task-specific heads, substantially reducing the number of trainable parameters compared with updating the entire encoder. Random cropping was employed for regularization during training. At inference time, we used center cropping for classification tasks and a sliding window approach for full-volume segmentation tasks. Training was performed with the AdamW optimizer for 150 epochs.

  • Full Finetuning: For ablation experiments, we updated all parameters from the ASP ViT encoder jointly with the downstream head using the same optimization and augmentation settings as LoRA. See Appendix Section C.

Linear Models with Anatomical Features

For each of our downstream diagnostic tasks, as well as age estimation, we trained linear models based on 178 anatomical features extracted via FreeSurfer [43] (v7.4.1), using the recon-all pipeline. The features include regional volume measurements, cortical surface areas, and cortical thickness values. A complete list is provided in Appendix Table A3. The resulting models are referred to as Anatomical in the Results section. These linear models achieved similar performance as nonlinear alternatives; details are provided in Appendix Section B.1. For Alzheimer’s diagnosis, we encoded healthy as 0, Alzheimer’s as 1 and mild cognitive impairment as 0.5, as this produced better results than treating diagnosis as a multiclass classification task [44], see Appendix Section B.2.

Convolutional Neural Networks

We used the ResNet-18 architecture [16] for diagnostic classification tasks and the U-Net architecture [17] for segmentation tasks, based on preliminary experiments. For classification, the models were trained for up to 60 epochs using a batch size of 16, stochastic gradient descent (SGD) [45], an initial learning rate of 0.01, and early stopping based on validation loss. For Alzheimer’s diagnosis, we used the same label encoding as in the logistic regression model, and optimized a binary cross-entropy objective during training. Additional analyses on baseline CNN selection are reported in Appendix Sections B.3.

Data Preprocessing

To harmonize data across cohorts, we implemented a standardized preprocessing pipeline for all MRIs. While datasets for segmentation tasks (BraTS and ATLAS R2.0) were already standardized, all other raw data—initially in DICOM or NIFTI format—underwent a uniform processing sequence. DICOM series were first converted to NIFTI using the dcm2nii tool.

Except for the pre-standardized segmentation cohorts, scans were processed using FreeSurfer’s recon-all workflow, which consists of three sequential phases (initial processing, surface reconstruction and refinement). This workflow also served as our primary quality control mechanism; any images that failed the initial processing phase were excluded from the study. A detailed description of the recon-all execution time and workflow is provided in Appendix Section A.5.

Finally, to ensure spatial alignment, all images and tissue masks were coregistered to the MNI152 2009a Nonlinear Symmetric template using the mri vol2vol command. The resulting preprocessed scans have a standardized volume of 197 × 233 × 189 voxels with an isotropic resolution of one cubic millimeter.

Statistical analysis

In each experiment, we report the overall metric and 95% confidence interval (CI), calculated by bootstrapping the held-out test set 1000 times. Specifically, we generate 1000 bootstrap samples by resampling the test set with replacement, compute the metric on each sample, and estimate the 95% CI from the 2.5th and 97.5th percentiles of the resulting metric distribution.

Ethics Approval and Consent to Participate

This study used de-identified, secondary data from publicly available neuroimaging repositories (UKBB, HCP, CamCAN, IXI, NACC, ADNI, OASIS, AIBL, MIRIAD, PPMI, OpenNeuro, EPISURG, UCSF-PDGM, BraTS, ATLAS). Each dataset was collected under institutional ethics approval and informed consent obtained by the original data-collecting institutions, as described in their respective publications. No new human data were collected for this study.

Discussion

Our study demonstrates that across a wide range of publicly available structural MRI datasets, linear models trained on preselected anatomical features remain highly competitive with deep neural networks. While traditional segmentation approaches are often computationally expensive for feature extraction, modern deep learningbased segmentation frameworks (e.g., FastSurfer [46]) offer an efficient alternative. Our results suggest that leveraging these deep-learning tools to extract structured anatomical features for linear models provides a computationally efficient, interpretable, and robust baseline for various diagnostic tasks.

Furthermore, our results indicate that diagnostic performance using structural MRI may have reached a saturation point within existing public datasets that contain only hundreds to a few thousand scans. Further progress will likely require the curation of substantially larger cohorts. Our age-estimation experiments on a cohort of 60,000 subjects indicate that scaling public datasets could unlock further methodological innovation.

From the methodological perspective, we show that vision transformers learn relevant anatomical structure during unsupervised pretraining, but at a slow rate that can be greatly accelerated using anatomical segmentation as an auxiliary pretraining task. This suggests that leveraging clinically-relevant tasks could mitigate the small size of medical pretraining datasets when compared to general computer-vision pretraining databases with millions of examples.

Supplementary Material

Supplement 1

Acknowledgements

This work was supported by Alzheimer’s Association grant AARG-NTF-21-848627 and NSF grant 2404476, as well as by the NYU IT High Performance Computing. The funders played no role in study design, data collection, analysis, interpretation, or writing of this manuscript. Dataset-specific acknowledgements are provided in Appendix Section A.1.

Footnotes

Competing interests

The authors declare no competing interests.

Data availability

Data from UK Biobank, ADNI, NACC, AIBL, OASIS 1–4, MIRIAD, PPMI, HCP, CamCAN, IXI, OpenNeuro, EPISURG, UCSF-PDGM, BraTS, and ATLAS are available to qualified researchers through application to the respective consortia (see Appendix A for links and access details). No new data were generated for this study.

Code availability

The code used to train and evaluate the models described in this study is publicly available at https://github.com/MIQUEL-LOPEZ-ESCORIZA/ai4brainmri.

References

  • [1].Klöppel S., Stonnington C.M., Chu C., Draganski B., Scahill R.I., Rohrer J.D., Fox N.C., Jack C.R. Jr, Ashburner J., Frackowiak R.S.: Automatic classification of mr scans in alzheimer’s disease. Brain 131(3), 681–689 (2008) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [2].Wee C.-Y., Yap P.-T., Zhang D., Denny K., Browndyke J.N., Potter G.G., Welsh-Bohmer K.A., Wang L., Shen D.: Identification of mci individuals using structural and functional connectivity networks. Neuroimage 59(3), 2045–2056 (2012) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [3].Popuri K., Ma D., Wang L., Beg M.F.: Using machine learning to quantify structural mri neurodegeneration patterns of alzheimer’s disease into dementia score: Independent validation on 8,834 images from adni, aibl, oasis, and miriad databases. Human Brain Mapping 41(14), 4127–4147 (2020) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [4].Hwang G., Abdulkadir A., Erus G., Habes M., Pomponio R., Shou H., Doshi J., Mamourian E., Rashid T., Bilgel M., et al. : Disentangling alzheimer’s disease neurodegeneration from typical brain ageing using machine learning. Brain Communications 4(3), 117 (2022) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [5].Payan A., Montana G.: Predicting Alzheimer’s disease: a neuroimaging study with 3D convolutional neural networks (2015). https://arxiv.org/abs/1502.02506 [Google Scholar]
  • [6].Basaia S., Agosta F., Wagner L., Canu E., Magnani G., Santangelo R., Filippi M., Initiative A.D.N., et al. : Automated classification of alzheimer’s disease and mild cognitive impairment using a single mri and deep neural networks. NeuroImage: Clinical 21, 101645 (2019) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [7].Chang P., Grinband J., Weinberg B., Bardis M., Khy M., Cadena G., Su M.-Y., Cha S., Filippi C., Bota D., et al. : Deep-learning convolutional neural networks accurately classify genetic mutations in gliomas. American Journal of Neuroradiology 39(7), 1201–1207 (2018) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [8].Havaei M., Davy A., Warde-Farley D., Biard A., Courville A., Bengio Y., Pal C., Jodoin P.-M., Larochelle H.: Brain tumor segmentation with deep neural networks. Medical image analysis 35, 18–31 (2017) [DOI] [PubMed] [Google Scholar]
  • [9].Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D., Zhai X., Unterthiner T., Dehghani M., Minderer M., Heigold G., Gelly S., et al. : An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020) [Google Scholar]
  • [10].Caron M., Touvron H., Misra I., Jégou H., Mairal J., Bojanowski P., Joulin A.: Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9650–9660 (2021) [Google Scholar]
  • [11].He K., Chen X., Xie S., Li Y., Dollár P., Girshick R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022) [Google Scholar]
  • [12].Hatamizadeh A., Tang Y., Nath V., Yang D., Myronenko A., Landman B., Roth H.R., Xu D.: Unetr: Transformers for 3d medical image segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 574–584 (2022) [Google Scholar]
  • [13].Cox J., Liu P., Stolte S.E., Yang Y., Liu K., See K.B., Ju H., Fang R.: Brainsegfounder: towards 3d foundation models for neuroimage segmentation. Medical Image Analysis 97, 103301 (2024) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [14].Tak D., Garomsa B.A., Chaunzwa T.L., Zapaishchykova A., Pardo J.C.C., Ye Z., Zielke J., Ravipati Y., Vajapeyam S., Mahootiha M., et al. : A foundation model for generalized brain mri analysis. medRxiv (2024) [Google Scholar]
  • [15].Zhu W., Huang H., Tang H., Musthyala R., Yu B., Chen L., Vega E., O’Donnell T., Dehkharghani S., Frontera J.A., et al. : 3d foundation ai model for generalizable disease detection in head computed tomography. arXiv preprint arXiv:2502.02779 (2025) [DOI] [PubMed] [Google Scholar]
  • [16].He K., Zhang X., Ren S., Sun J.: Deep Residual Learning for Image Recognition (2015). https://arxiv.org/abs/1512.03385 [Google Scholar]
  • [17].Ronneberger O., Fischer P., Brox T.: U-net: Convolutional networks for biomedical image segmentation (2015) [Google Scholar]
  • [18].Sudlow C., Gallacher J., Allen N., Beral V., Burton P., Danesh J., Downey P., Elliott P., Green J., Landray M., et al. : Uk biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS medicine 12(3), 1001779 (2015) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [19].Petersen R.C., Aisen P.S., Beckett L.A., Donohue M.C., Gamst A.C., Harvey D.J., Jack C. Jr, Jagust W.J., Shaw L.M., Toga A.W., et al. : Alzheimer’s disease neuroimaging initiative (adni) clinical characterization. Neurology 74(3), 201–209 (2010) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [20].Beekly D.L., Ramos E.M., Lee W.W., Deitrich W.D., Jacka M.E., Wu J., Hubbard J.L., Koepsell T.D., Morris J.C., Kukull W.A., et al. : The national alzheimer’s coordinating center (nacc) database: the uniform data set. Alzheimer Disease & Associated Disorders 21(3), 249–258 (2007) [DOI] [PubMed] [Google Scholar]
  • [21].Ellis K., Ames D., Martins R., Hudson P., Masters C.: The australian biomarkers lifestyle and imaging flagship study of ageing. Acta Neuropsychiatrica 18(6), 285–285 (2006) [DOI] [PubMed] [Google Scholar]
  • [22].Malone I.B., Cash D., Ridgway G.R., MacManus D.G., Ourselin S., Fox N.C., Schott J.M.: Miriad—public release of a multiple time point alzheimer’s mr imaging dataset. NeuroImage 70, 33–36 (2013) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [23].LaMontagne P.J., Benzinger T.L., Morris J.C., Keefe S., Hornbeck R., Xiong C., Grant E., Hassenstab J., Moulder K., Vlassenko A.G., et al. : Oasis-3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and alzheimer disease. medrxiv, 2019–12 (2019) [Google Scholar]
  • [24].Marcus D.S., Fotenos A.F., Csernansky J.G., Morris J.C., Buckner R.L.: Open access series of imaging studies: longitudinal mri data in nondemented and demented older adults. Journal of cognitive neuroscience 22(12), 2677–2684 (2010) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [25].Marcus D.S., Wang T.H., Parker J., Csernansky J.G., Morris J.C., Buckner R.L.: Open access series of imaging studies (oasis): cross-sectional mri data in young, middle aged, nondemented, and demented older adults. Journal of cognitive neuroscience 19(9), 1498–1507 (2007) [DOI] [PubMed] [Google Scholar]
  • [26].Marek K., Jennings D., Lasch S., Siderowf A., Tanner C., Simuni T., Coffey C., Kieburtz K., Flagg E., Chowdhury S., et al. : The parkinson progression marker initiative (ppmi). Progress in neurobiology 95(4), 629–635 (2011) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [27].Calabrese E., Villanueva-Meyer J.E., Rudie J.D., Rauschecker A.M., Baid U., Bakas S., Cha S., Mongan J.T., Hess C.P.: The university of california san francisco preoperative diffuse glioma mri dataset. Radiology: Artificial Intelligence 4(6), 220058 (2022) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [28].Schuch F., Walger L., Schmitz M., David B., Bauer T., Harms A., Fischbach L., Schulte F., Schidlowski M., Reiter J., et al. : An open presurgery mri dataset of people with epilepsy and focal cortical dysplasia type ii. Scientific Data 10(1), 475 (2023) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [29].Pérez-García F., Dorent R., Rizzi M., Cardinale F., Frazzini V., Navarro V., Essert C., Ollivier I., Vercauteren T., Sparks R., et al. : A self-supervised learning strategy for postoperative brain cavity segmentation simulating resections. International Journal of Computer Assisted Radiology and Surgery 16, 1653–1661 (2021) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [30].Baid U., Ghodasara S., Mohan S., Bilello M., Calabrese E., Colak E., Farahani K., Kalpathy-Cramer J., Kitamura F.C., Pati S., et al. : The rsna-asnrmiccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107.02314 (2021) [Google Scholar]
  • [31].Menze B.H., Jakab A., Bauer S., Kalpathy-Cramer J., Farahani K., Kirby J., Burren Y., Porz N., Slotboom J., Wiest R., et al. : The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging 34(10), 1993–2024 (2014) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [32].Bakas S., Akbari H., Sotiras A., Bilello M., Rozycki M., Kirby J.S., Freymann J.B., Farahani K., Davatzikos C.: Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientific data 4(1), 1–13 (2017) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [33].Liew S.-L., Lo B.P., Donnelly M.R., Zavaliangos-Petropulu A., Jeong J.N., Barisano G., Hutton A., Simon J.P., Juliano J.M., Suri A., et al. : A large, curated, open-source stroke neuroimaging dataset to improve lesion segmentation algorithms. Scientific data 9(1), 320 (2022) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [34].Kirkpatrick J., Pascanu R., Rabinowitz N., Veness J., Desjardins G., Rusu A.A., Milan K., Quan J., Ramalho T., Grabska-Barwinska A., Hassabis D., Clopath C., Kumaran D., Hadsell R.: Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114(13), 3521–3526 (2017) 10.1073/pnas.1611835114 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [35].Li Z., Hoiem D.: Learning without Forgetting (2017). https://arxiv.org/abs/1606.09282 [DOI] [PubMed] [Google Scholar]
  • [36].Rebuffi S.-A., Kolesnikov A., Sperl G., Lampert C.H.: iCaRL: Incremental Classifier and Representation Learning (2017). https://arxiv.org/abs/1611.07725 [Google Scholar]
  • [37].Liu K., Price B., Kuen J., Fan Y., Wei Z., Figueroa L., Geras K.J., Fernandez-Granda C.: Uncertainty-aware fine-tuning of segmentation foundation models. Advances in Neural Information Processing Systems 37, 53317–53389 (2024) [Google Scholar]
  • [38].Kirillov A., Mintun E., Ravi N., Mao H., Rolland C., Gustafson L., Xiao T., Whitehead S., Berg A.C., Lo W.-Y., et al. : Segment anything. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4015–4026 (2023) [Google Scholar]
  • [39].Loshchilov I., Hutter F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017) [Google Scholar]
  • [40].Loshchilov I., Hutter F.: Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016) [Google Scholar]
  • [41].Zhu Y., Shen Z., Zhao Z., Wang S., Wang X., Zhao X., Shen D., Wang Q.: Melo: Low-rank adaptation is better than fine-tuning for medical image diagnosis. In: 2024 IEEE International Symposium on Biomedical Imaging (ISBI), pp. 1–5 (2024). IEEE [Google Scholar]
  • [42].Hu E.J., Shen Y., Wallis P., Allen-Zhu Z., Li Y., Wang S., Wang L., Chen W.: LoRA: Low-Rank Adaptation of Large Language Models (2021). https://arxiv.org/abs/2106.09685 [Google Scholar]
  • [43].Fischl B.: Freesurfer. Neuroimage 62(2), 774–781 (2012) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [44].Liu S., Masurkar A.V., Rusinek H., Chen J., Zhang B., Zhu W., Fernandez-Granda C., Razavian N.: Generalizable deep learning model for early alzheimer’s disease detection from structural mris. Scientific reports 12(1), 17106 (2022) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [45].Ruder S.: An overview of gradient descent optimization algorithms (2017). https://arxiv.org/abs/1609.04747 [Google Scholar]
  • [46].Henschel L., Conjeti S., Estrada S., Diers K., Fischl B., Reuter M.: Fastsurfer-a fast and accurate deep learning based neuroimaging pipeline. NeuroImage 219, 117012 (2020) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [47].Harms M.P., Somerville L.H., Ances B.M., Andersson J., Barch D.M., Bastiani M., Bookheimer S.Y., Brown T.B., Buckner R.L., Burgess G.C., et al. : Extending the human connectome project across ages: Imaging protocols for the lifespan development and aging projects. Neuroimage 183, 972–984 (2018) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [48].Bookheimer S.Y., Salat D.H., Terpstra M., Ances B.M., Barch D.M., Buckner R.L., Burgess G.C., Curtiss S.W., Diaz-Santos M., Elam J.S., et al. : The lifespan human connectome project in aging: an overview. Neuroimage 185, 335–348 (2019) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [49].Taylor J.R., Williams N., Cusack R., Auer T., Shafto M.A., Dixon M., Tyler L.K., Henson R.N., et al. : The cambridge centre for ageing and neuroscience (cam-can) data repository: Structural and functional mri, meg, and cognitive data from a cross-sectional adult lifespan sample. neuroimage 144, 262–269 (2017) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [50].Rowland A., Burns M., Hartkens T., Hajnal J., Rueckert D., Hill D.L.: Information extraction from images (ixi): Image processing workflows using a grid enabled image database. Proceedings of DiDaMIC 4, 55–64 (2004) [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement 1
NIHPPrs10851563v1-supplement-1.pdf (3.6MB, pdf)

Data Availability Statement

Data from UK Biobank, ADNI, NACC, AIBL, OASIS 1–4, MIRIAD, PPMI, HCP, CamCAN, IXI, OpenNeuro, EPISURG, UCSF-PDGM, BraTS, and ATLAS are available to qualified researchers through application to the respective consortia (see Appendix A for links and access details). No new data were generated for this study.

The code used to train and evaluate the models described in this study is publicly available at https://github.com/MIQUEL-LOPEZ-ESCORIZA/ai4brainmri.

Articles from Research Square are provided here courtesy of American Journal Experts

RESOURCES