Generative AI Models for Developing Neuroimaging Markers of Psychiatric Disorders

The field of biological psychiatry grapples with the formidable challenge of elucidating the intricate neurobiological underpinnings of psychiatric disorders. A central pursuit within this field is the identification of robust and reproducible biomarkers, which hold the promise of transforming diagnosis, prognosis, and treatment strategies. However, progress has been hampered by several factors, notably the inherent complexity of the brain and the limitations of current research methodologies (1).

Brain imaging techniques, such as functional magnetic resonance imaging (fMRI), have emerged as powerful tools for investigating brain structure and function in vivo. These brain imaging tools have yielded unprecedented insights into how brain structure and function are involved in neuropsychiatric disorders. For example, the Enhancing Neuro Imaging Genetics through Meta-Analysis (ENIGMA) consortium provided significant evidence of biomarkers across numerous studies (2). Other studies provided deep insight into the pathophysiology of neuropsychiatric disorders (e.g., (3)). Yet, the application of these techniques in psychiatric research has encountered significant obstacles in the quest for reliable biomarkers with effect sizes sufficient to be implemented in clinical care.

In our view, a major challenge lies in the fact that the complexity of the brain is unlikely to be fully modeled using simple models developed from small sample sizes. Empirical evidence suggests that fMRI studies may necessitate sample sizes in the thousands to achieve reproducible brain-behavior associations. For instance, a seminal study demonstrated that commonly observed effect sizes in fMRI studies are often inflated due to underpowered designs, and that achieving adequate statistical power requires substantially larger sample sizes than typically employed (4). This requirement for large-scale data acquisition poses a substantial logistical and financial burden, often rendering such studies prohibitively expensive and frequently infeasible within the constraints of clinical trials.

Limitations of Task-Specific Brain Imaging Studies. Contemporary brain imaging studies in psychiatry are often task-specific in their design (i.e., diagnostic, associative or predictive task). This entails the construction of brain models tailored to elucidate a particular behavior or clinical phenomenon. For example, resting-state fMRI scans acquired at baseline in a cohort of 200 individuals diagnosed with major depressive disorder (MDD) might be utilized to predict treatment response to a specific antidepressant medication. While such an approach can yield statistically significant models that fit the observed data, the generalizability and replicability of these models have proven challenging (4).

The fundamental limitation stems from the unfavorable ratio of features to observations. fMRI scans generate an immense volume of data, encompassing tens or hundreds of thousands of individual data points. In contrast, typical clinical studies often involve sample sizes in the range of hundreds, rather than thousands, of participants. Consequently, statistical models are constructed with a relatively small number of observations in relation to the high dimensionality of the data. This disparity increases the risk of overfitting, whereby the model captures noise and idiosyncrasies within the specific dataset, rather than the underlying biological signal. As a result, the likelihood of replicating these findings in independent datasets is diminished, contributing to the reproducibility crisis that has plagued the field of neuroscience (5). Although the acquisition of tens of thousands of observations could potentially enhance the robustness and predictive accuracy of these models, the practical constraints of cost, time, and participant availability often preclude such large-scale investigations.

Generative AI as a Potential Solution. Recent advances in the field of generative artificial intelligence (AI) offer a promising avenue for addressing some of the challenges of reproducibility and generalizability in biological psychiatry research. The core concept involves leveraging the power of generative models to learn complex, high-dimensional representations of brain features from large, unlabeled datasets. These foundational models, once trained, can then be adapted or fine-tuned to perform specific tasks, such as predicting antidepressant treatment response, using relatively small, task-specific datasets (6).

This approach holds the potential to overcome the limitations of traditional task-specific studies by decoupling the process of model development from the constraints of limited sample sizes in clinical populations. By initially training a model on a massive, heterogeneous dataset, the model can learn generalizable features of brain function that are not tied to any particular task or clinical condition. Subsequently, this pre-trained model can be efficiently adapted to specific clinical questions using smaller, more focused datasets, thereby enhancing the statistical power and reproducibility of the resulting analyses.

This paradigm shift was recently demonstrated with the development of BrainLM, a foundational model of whole-brain activity trained on a large dataset of fMRI scans from the UK Biobank (7). The UK Biobank is a large-scale biomedical database and research resource, containing detailed genetic, lifestyle, and health information from approximately 500,000 participants in the United Kingdom. The availability of a large subset of fMRI data (~70,000 scans) within the UK Biobank provided an unprecedented opportunity to train a generative model on a scale previously unattainable in psychiatric research.

BrainLM: A Foundation Model of Whole-Brain Activity. BrainLM is a deep learning model developed using a transformer-based architecture, a class of neural networks that has achieved remarkable success in natural language processing and computer vision. The model was trained on a large dataset of resting-state and task fMRI scans from the UK Biobank, encompassing data from thousands of individuals (7).

Through the process of self-supervised learning, BrainLM learned to capture complex patterns and relationships within the fMRI data, without relying on explicit labels or annotations. The model was trained to predict masked portions of the fMRI BOLD time series, forcing it to develop a deep understanding of the underlying spatial and temporal dependencies within the brain’s functional networks. This process enabled BrainLM to acquire a generalizable representation of whole-brain activity, capturing fundamental aspects of brain function that are common across individuals.

Following the initial training phase, BrainLM was successfully fine-tuned to predict a variety of demographic and behavioral measures, including age, neuroticism, anxiety severity, and depression severity. This demonstrated the model’s ability to extract meaningful information from brain activity patterns and to generalize to different tasks and clinical variables. Notably, BrainLM achieved these results using smaller task-specific datasets, highlighting the efficiency and power of the transfer learning approach (7).

Future Directions: BrainLM 2.0. Building upon the successes of BrainLM, ongoing developments in BrainLM 2.0 aim to address key challenges in neuroimaging research through a multimodal foundation model. Leveraging powerful, pretrained large language models (LLMs), which already exhibit sophisticated reasoning and interpretive capabilities from natural language processing tasks, BrainLM 2.0 will utilize advanced multimodal vision-language architectures (e.g., LLaMA 3.2 or Qwen 2.5). These models will encode brain activity as visual data—such as static images or dynamic videos—and integrate behavioral and clinical metadata as textual information. This strategy will enable intuitive natural language queries about brain recordings, significantly enhancing interpretability, accessibility, and usability for researchers and clinicians. By harnessing the intelligence of pretrained LLMs, BrainLM 2.0 promises transformative capabilities for psychiatric research and personalized clinical care.

Another promising avenue is the integration of behavioral large language models (behavioral LLMs), which leverage vast behavioral datasets to generate structured and semantically rich representations of patient behavior and clinical characteristics. Integrating behavioral LLMs with neuroimaging generative models like BrainLM could lead to more robust and comprehensive brain-behavior associations, substantially enhancing predictive accuracy and clinical utility.

Generative AI models, particularly advancements like BrainLM 2.0 and integrated behavioral LLMs, have great potential for immediate clinical impact in disorders such as major depressive disorder (MDD) and post-traumatic stress disorder (PTSD). Realizing this potential requires interdisciplinary collaboration among psychiatrists, neuroscientists, data scientists, ethicists, and regulatory authorities to ensure effective and ethical integration of these technologies into clinical practice.

Regulatory and Practical Challenges. Despite their promise, AI-driven neuroimaging markers face significant practical and regulatory challenges, including approval from governing bodies, standardization across clinical settings, and clinicians training. Furthermore, the potential of using generative AI to model brain imaging still needs validation. For instance, it is currently unclear whether developing a robust foundation model of the brain would necessitate tens of thousands or even millions of scans to fully capture the intricate nature of brain activity and structure. Moreover, creating novel methods to fine-tune these large foundation models using smaller datasets would be crucial for their effective application in drug development and clinical practice. Addressing these challenges will be essential for successful clinical translation. Other general challenges are related to data availability and sharing, computational resources, and better interpretability of the models which often considered “black boxes” requiring further development of tools to better investigate the brain foundation models.

Generative AI offers a transformative opportunity to address the challenges of reproducibility and biomarker discovery in biological psychiatry. By leveraging large, unlabeled datasets to learn generalizable representations of brain activity, generative models can enhance the robustness, generalizability, and statistical power of brain imaging studies. While significant challenges remain, the potential of this technology to advance our understanding of psychiatric disorders and improve clinical outcomes is substantial. Future research should focus on addressing the challenges related to data availability, computational resources, model interpretability, ethical considerations, and clinical translation (8).