Psychiatric neuroimaging at a crossroads: Insights from psychiatric genetics

Abstract

Thanks to methodological advances, large-scale data collections, and longitudinal designs, psychiatric neuroimaging is better equipped than ever to identify the neurobiological underpinnings of youth mental health problems. However, the complexity of such endeavors has become increasingly evident, as the field has been confronted by limited clinical relevance, inconsistent results, and small effect sizes. Some of these challenges parallel those historically encountered by psychiatric genetics. In past genetic research, robust findings were historically undermined by oversimplified biological hypotheses, mistaken assumptions about expectable effect sizes, replication problems, confounding by population structure, and shared biological patterns across disorders. Overcoming these challenges has contributed to current successes in the field. Drawing parallels across psychiatric genetics and neuroimaging, we identify key shared challenges as well as pinpoint relevant insights that could be gained in psychiatric neuroimaging from the transition that occurred from the candidate gene to (post) genome-wide “eras” of psychiatric genetics. Finally, we discuss the prominent developmental component of psychiatric neuroimaging and how that might be informed by epidemiological and omics approaches. The evolution of psychiatric genetic research offers valuable insights that may expedite the resolution of key challenges in psychiatric neuroimaging, thus potentially moving our understanding of psychiatric pathophysiology forward.

Highlights

• •There are several challenges shared by psychiatric genetics and neuroimaging.
• •Challenges include distributed, small effects, replicability, and oversimplified hypotheses.
• •Insights from psychiatric genetics’ history can inform psychiatric neuroimaging.

1. Introduction

The introduction of in vivo neuroimaging in the 1980s brought unprecedented opportunities to explore the neurobiological basis of psychiatric illnesses (Raichle, 2009). In the years that followed, researchers have reported numerous neuroanatomical and functional correlates of mental health problems, in youth and adulthood. Substantial efforts were made to try and uncover biomarkers for the prediction and subtyping of mental illness (Brucar et al., 2023). Intriguing results were reported. Findings were, however, preliminary. They have been predominantly based on small cross-sectional samples, which are more prone to sampling variability, poorer replicability, and other biases (Klapwijk et al., 2021, Marek et al., 2022). Over the past decade, large neuroimaging samples and dense longitudinal data have become increasingly available at an unprecedented scale (Paus, 2010). This involves key studies like the Adolescent Brain Cognitive Development (ABCD) (Casey et al., 2018) Study, Human Connectome Project (HCP) (Elam et al., 2021), Healthy Brain Network (Alexander et al., 2017), and UK Biobank (Littlejohns et al., 2020), as well as collaborative efforts such as the Enhancing Neuro Imaging Genetics through Meta-Analysis (ENIGMA) consortium (Thompson et al., 2020). These have offered tremendous potential for studying the neurobiological basis of mental health and disease, with the prospects of gaining deeper insights into the origins of psychiatric problems, developing biologically-based diagnostic systems (Insel et al., 2010), and improving the accuracy of prognostic predictions.

However, delivering on these potential targets has proven difficult. It has become increasingly evident that a readjustment of expectations on the role of neuroimaging in psychiatric problems is necessary. First, in terms of clinical utility, the highly distributed and small effects of psychiatric neuroimaging have rendered challenging the search for neural markers for disorders’ onset, diagnosis, and prognosis (e.g.,Schmaal et al., 2020). No reliable biomarker with clinical utility has been identified to date in psychiatric neuroimaging (Abi-Dargham et al., 2023, Tervo-Clemmens et al., 2023a). Yet, there are examples of successful integration of neuroimaging into clinical practice, e.g., advanced imaging techniques can guide neurostimulation and neuromodulation in the treatment of depression and obsessive compulsive disorder (Cole et al., 2020, Acevedo et al., 2021). Second, in terms of associational findings, while numerous differences between cases and controls have been reported, their replicability remains contentious, especially when it comes to regional convergence (i.e., location of associations). For instance, a recent meta-analysis found no statistically significant convergent structural or functional alterations for attention-deficit/hyperactivity disorder (ADHD) in children and adolescents (Samea et al., 2019), while a mega-analysis in ENIGMA found cortical surface area differences, although these were restricted to childhood (Hoogman et al., 2019). Other examples include bipolar disorder (Harrison et al., 2020) and autism spectrum disorder, for which consistent neuroanatomical alterations have not been identified to date (Hiremath et al., 2021). Third, effect sizes in psychiatric neuroimaging findings are smaller than previously expected (Marek et al., 2022b). For example, a large-scale study of youths found that the magnitude of associations between imaging-derived and psychiatric phenotypes, at a univariate and multivariate level, corresponded to effect sizes below 0.1 (Makowski et al., 2023, Marek et al., 2022).

In this review, we discuss how psychiatric neuroimaging may find itself at a familiar crossroads, having the potential to draw insights from other fields that have encountered and addressed similar challenges. Psychiatric (molecular) genetics stands out as a prime example of a discipline that has navigated through significant methodological issues and is now steadily generating robust insights on the etiology of psychopathology (Smoller, 2019). Over the past two decades, it has substantially changed, moving from candidate gene approaches in small samples to large-scale studies across the genome (Smoller, 2019). We first identify challenges that are common in both psychiatric genetics and neuroimaging, to then explore how solutions developed in genetics might also benefit, or not, the field of neuroimaging. Notably, one goal of this review is to encourage discussion of strategies from genetics that could move developmental psychiatric neuroimaging forward. It is not our intent to capture all parallels across the two fields. We also recognize that genetics and neuroimaging are distinct and have strengths on different aspects of research (e.g., replicability for genetics, developmental approach for neuroimaging). This offers learning opportunities for each field from the other. Another goal of this review is to navigate potential venues for designing, analyzing, and reporting brain imaging studies, with the aim of improving the validity, reliability, and relevance of neuroimaging research. This can be facilitated with population neuroscience, an emerging field of neuroimaging that combines insights from neurodevelopmental science, epidemiology, and genetics (Paus, 2010, Tiemeier and Muetzel, 2020).

2. How the challenges of psychiatric genetics can inform psychiatric neuroimaging

We discuss five key challenges that have been faced by the psychiatric genetic and neuroimaging fields: (i) oversimplified biological hypotheses, (ii) distributed small effects, (iii) replicability issues, (iv) confounding by population structure, and (v) shared biological signal across disorders. To address the prominent developmental patterns of neuroimaging, we additionally cover how epidemiological and other developmentally-dependent fields (e.g., epigenetics) could benefit imaging. Table 1 illustrates how developmental psychiatric neuroimaging studies fare on the listed challenges. We summarized the characteristics of the most recent ten articles (start year: 2023) in child and adolescent psychiatry and magnetic resonance imaging research, sourced from three leading developmental psychiatry journals: Developmental Cognitive Neuroscience (Son et al., 2023; Guldner et al., 2022; Petrican and Fornito, 2023; Ladouceur et al., 2023; Hardi et al., 2023a; Dimanova et al., 2023; Colich et al., 2023; Voldsbekk et al., 2023; Wiglesworth et al., 2023; Sullivan-Toole et al., 2023), the Journal of Child Psychology and Psychiatry (Bu et al., 2023; Hardi et al., 2023b; Pagliaccio et al., 2023; Graziano et al., 2022; Kirshenbaum et al., 2022; Mewton et al., 2022; Okada et al., 2022; Peterson et al., 2022; Yoon et al., 2022; Postema et al., 2021), and the Journal of the American Academy of Child and Adolescent Psychiatry (Auerbach et al., 2022; Geisler et al., 2022; Wang et al., 2022; Vulser et al., 2023; Fortea et al., 2023; Romer et al., 2023; Weeland et al., 2022; Chen et al., 2022; Bahnsen et al., 2022; Dall’Aglio et al., 2023).

Table 1.

Characteristics of the ten most recent (until end of 2023) developmental psychiatric neuroimaging studies at three key developmental journals.

Journal	Biological hypotheses		Replicability challenges		Population stratification*	Cross-Disorder	Longitudinal Design
	n studies with candidate region analyses	n studies with data-driven analyses	n studies with an independent replication sample, or a meta/mega-analysis	Sample size	n studies accounting for ethnicity, race, or ancestry	n studies examining multiple disorders	n with longitudinal (vs. cross-sectional) design, with (vs. no) adjustments for baseline levels, with repeated (vs. single) measurements of exposure and outcome
DCN (2023)	5/10	6/10	1/10 replication 0/10 meta-analysis 0/10 mega-analysis	Mean: 1359 Median: 170 Range: (39; 9985)	5/10	4/10	Longitudinal: 4/10 Longitudinal with adjustments for baseline levels: 3/10 Longitudinal with repeated exposure and outcome measurements: 2/10
JCPP (2021–2023)	5/10	5/10	0/10 replication 0/10 meta-analysis 1/10 mega-analysis	Mean: 1497 Median: 158 Range: (106; 11,875)	4/10	2/10	Longitudinal: 4/10 Longitudinal with adjustments for baseline levels: 2/10 Longitudinal with repeated exposure and outcome measurements: 0/10
JAACAP (2022–2023)	5/10	7/10	4/10 replication 1/10 meta-analysis 1/10 mega-analysis	Mean: 2264 Median: 256 Range: (128, 11,875)	3/10	2/10	Longitudinal: 7/10 Longitudinal with adjustments for baseline levels: 7/10 Longitudinal with repeated exposure and outcome measurements: 4/10

Note. DCN = Developmental Cognitive Neuroscience; JAACAP = Journal of the American Academy of Child and Adolescent Psychiatry; JCPP = Journal of Child Psychology and Psychiatry.

^*Controlling for race, ethnicity, top principal components for ancestry, or restricting to one ancestry.

2.1. Overcoming oversimplified biological hypotheses

2.1.1. The challenge of limited knowledge of the underlying pathophysiology

Given the limited mechanistic knowledge of the underlying pathophysiology of psychiatric phenotypes, candidate gene selection in psychiatric genetics was often based on hypotheses relying on the known effects of psychiatric medications on biological pathways (Psychiatric GWAS Consortium Coordinating Committee, 2009). For example, medications for depression rely on serotonin, so researchers focused on genes related to serotonin prediction and uptake (Psychiatric GWAS Consortium Coordinating Committee, 2009). Yet, these hypotheses proved to be inadequate in the search for causal genetic variants (Psychiatric GWAS Consortium Coordinating Committee, 2009). Genetic risk for psychiatric problems is driven by many variants that were not hypothesized to be associated with psychiatric disorders in candidate gene research (Lander, 1996). Classical candidate gene studies were unequipped to detect most of such risk variants, as they tested few single nucleotide polymorphisms (SNPs), based on prior hypotheses.

In psychiatric neuroimaging, while there is extensive knowledge of the function of many brain regions and networks, there remains a limited understanding of the neuro-pathophysiology of psychiatric disorders, especially in terms of regional specificity. To date, more traditional psychiatric neuroimaging studies have tested regions of interest (ROIs) based on hypotheses with biological plausibility for a given psychiatric disorder, and findings from animal work. For instance, the volume of the amygdala, which is central to processing fearful and threatening stimuli (Baxter and Croxson, 2012), has been analyzed in relation to disorders like anxiety and depression (Hamilton et al., 2008, Qin et al., 2014, Tye et al., 2011). Such a region of interest approach parallels genetics’ candidate gene research. The prior knowledge and the translation to any given biological measure tested may however not support a specific hypothesis. While some antidepressants may exert their effect primarily via the serotonergic systems, they may not necessarily help identify genetic variants or brain structural and functional patterns that can guide molecular and neurobiological research in depression (Border et al., 2019, Moncrieff et al., 2023, Arnone et al., 2024). Overall, similar to psychiatric genetics, the a priori approach in psychiatric imaging relying on ROIs may be overly simplistic if psychiatric disorders are caused by widespread and unknown structural and functional variations.

2.1.2. Navigating potential ways forward

In psychiatric genetics, the literature became saturated with candidate gene studies based on assumptions or prior beliefs about psychopathology (Hirschhorn et al., 2002). This led to the transition to “unbiased” genome-wide approaches (Sullivan, 2007, Sullivan, 2010). When technological advances permitted it, Genome-Wide Association Studies (GWASs) became routinely used, allowing for the normalization of hypothesis-free approaches. This evolved into the testing of an increasing number of hypotheses when better genome coverage and more detailed imputations became available. Such genome-wide approaches revealed that prior candidate genes were generally not among the salient genes identified by GWAS, and were often not even identified as associated with the disorder of interest. For instance, several candidate genes for major depression and schizophrenia have not been replicated in GWASs (Border et al., 2019, Liu et al., 2019). Importantly, while initial GWAS efforts in psychiatry were characterized by null results or the identification of genes with no known biological relevance (Sullivan, 2010, Beauchamp et al., 2011, Visscher et al., 2012), GWASs have now robustly detected hundreds of loci associated with various psychiatric phenotypes. Characterizing such widespread signals across the genome has been paramount to understanding biological pathways of clinical relevance. Based on the results of hypothesis-free analyses, scientists have delineated the most relevant signals (e.g., with fine-mapping of potentially causal variants (Schaid et al., 2018)) and their associated functional patterns (e.g., gene-set enrichment (Leeuw et al., 2015), transcriptome-wide analysis (Gusev et al., 2016)) to gain greater insights into disorder mechanisms. Overall, facilitated by technological advances, psychiatric genetics moved beyond oversimplified biological hypotheses by leveraging hypothesis-free approaches. These could permit the identification of novel associations and their biological mechanisms, which can eventually lead to the formulation of new or revised, detailed hypotheses.

In psychiatric neuroimaging, similar hypothesis-free approaches proved useful to gain a more holistic overview of disorder mechanisms and etiology. Brain-wide analyses of ROIs, vertices, voxel- and vertex-based analyses, machine learning, and other data-driven methods are common (Table 1). Candidate region or network analyses remain often used (Table 1). This is exemplified by a recent editorial summarizing functional ROI studies as such that disruptions in the default mode network underlay the increase in youth mental health problems (Nasrallah, 2023). This reliance on candidate region analyses is likely because hypothesis-free approaches in imaging are intrinsically tied to larger sample sizes and collaborative efforts (discussed in 2.3. Addressing replicability challenges), which have been challenging to obtain until recently. Hypothesis-free approaches in large neurodevelopmental cohorts, such as the Generation R and ABCD studies, have revealed novel or different regions of association, similar to GWASs. For instance, for ADHD, frontal and temporal areas of associations that did not belong to usual candidate regions were observed (Dall’Aglio et al., 2022). Like in modern genetic approaches, such associational findings could be further characterized to gain insights into the pathophysiology of a psychiatric disorder and develop data-driven biological hypotheses. First, atlases can be inputted into existing software to elucidate associational findings in terms of other brain characteristics, with multi-modal parcellation (Glasser et al., 2016), and brain functional maps based on resting-state functional MRI (rsfMRI) and task fMRI data (Schaefer et al., 2018, King et al., 2023). Second, other biological data can be coupled with neuroimaging findings. These can include histological information from the recently-developed NextBrain (Casamitjana et al., 2024), cytoarchitecture from post-mortem tissue from the BigBrain Atlas (Amunts et al., 2013), molecular (gene expression) (Hawrylycz et al., 2012), metabolic (glucose and oxygen) (Vaishnavi et al., 2010), neurophysiological (e.g., delta, theta power) (Tadel et al., 2011), as well as evolutionary (Hill et al., 2010) and developmental (Reardon et al., 2018) expansion maps (Markello et al., 2022). Such resources can be combined with the neuromaps tool, which finds a common space across different brain maps (Markello et al., 2022). More broadly, any type of spatial overlap between brain maps can be evaluated with tools such as SPIN (Alexander-Bloch et al., 2018), SPICE (Weinstein et al., 2021), and BrainSMASH (BrainSMASH, 2024). Moreover, methods have been recently developed to further interpret multivariate findings through enrichment approaches (Li et al., 2024). Overall, the use of hypothesis-free approaches and linkage to other types of neural and omics data could be a way forward for neuroimaging, as it was for genetics, to better understand the underlying pathophysiology of mental illness.

2.2. Embracing widespread effects of small size

2.2.1. The challenge of small effect sizes in complex disorders

Throughout the course of psychiatric genetics, the search for genes conferring disorder liability was guided by the substantial heritability estimates reported in family-based studies of psychopathology, in which about 30% (e.g., depression) to 80% (e.g., schizophrenia) of the variance in psychiatric disorders was attributable to genetic variation (Pettersson et al., 2019). Early efforts to map psychiatric risk genes were often conducted with the expectation of relatively large, overestimated effect sizes for specific genetic variants, as was the case for Mendelian diseases like cystic fibrosis (e.g., CFTR gene) (NIH, 2023). Yet, for complex common disorders like psychiatric problems, for evolutionary purposes, risk must be predominantly polygenic (Lander, 1996). Genetic liability is thus, in large part, driven by the cumulation of small effects from many contributing variants (Lander, 1996). In fact, most replicated psychiatric GWASs show variant effects will allelic odds ratios of 1.05–1.15 or less (Smoller, 2019, Psychiatric GWAS Consortium Coordinating Committee, 2009). This required the field to grapple with fundamental questions related to estimating and understanding a complex constellation of individually small genetic effects and how they work in concert.

In psychiatric neuroimaging, less guidance on the explanatory potential of brain features on mental health problems is available. While large effect sizes were initially expected for morphological characteristics of many brain regions of interest, growing evidence suggests that neuroanatomical signal for psychopathology arises from distributed effects that are individually small in size. For instance, large meta-analyses, like those conducted by the ENIGMA consortium, find that psychiatric disorders are generally associated with multiple brain regions that span different lobes (Sripada et al., 2021, Kelly et al., 2018), as opposed to a more discrete set of neuroanatomical features. Moreover, a recent study by Marek and colleagues used data from the ABCD, HCP, and UK Biobank datasets to conclude that regional brain-psychopathology relationships are small in nature (Marek et al., 2022b). This has been debated (Spisak et al., 2023). Despite this, it remains evident that effect sizes are generally larger in smaller samples, suggesting that the psychiatric neuroimaging field might have encountered effect overinflation (further discussed in 2.3 Addressing replicability challenges) (Tervo-Clemmens et al., 2023b). This realization parallels historical developments in psychiatric genetics, where GWAS results demonstrated that effect sizes were substantially smaller than those traditionally reported in the candidate gene literature.

2.2.2. Navigating potential ways forward

In psychiatric genetics, several methodological advances have allowed the field to embrace the polygenicity of psychopathology, and evaluate the small molecular effects in aggregate. For instance, genome-wide complex trait analysis (GCTA-GREML) was developed to estimate the heritability of phenotypes like psychiatric disorders using variant-level genetic data (Yang et al., 2011). Additionally, polygenic risk scores (PGSs) aggregate genetic risk for a given trait or disorder into individual-level indices representing how high the genetic liability of a participant is (Choi et al., 2020). These approaches opened doors for investigations into how cumulative genetic liability to a given disorder would reflect onto other traits, such as other psychiatric problems, the exposome, or individual treatment responses (Lewis and Vassos, 2020, Torkamani et al., 2018). Although these methods have advanced the field, they did not capture the whole extent of polygenicity in psychiatric disorders, i.e., all distributed small effects. A substantial portion of variance explained remained “missing”, with parts of it having been increasingly “found” through augmented statistical power, better coverage of the genome from technological advances, and more sophisticated designs which included non-additive models (Génin, 2020). Overall, to embrace the small and distributed effect sizes in genetics, approaches to aggregate such effects have been developed, and facilitated by technological advances and modelling techniques.

In psychiatric neuroimaging, methods to capture distributed effects have been available for decades (e.g., SPM99 (Papadopoulos, 2009)), although with likely limited statistical power until recently (see 2.3. Addressing replicability challenges). The field has tried to quantify the explanatory power of imaging phenotypes in psychiatric problems with methods like the zero-inflated variance estimator (Ren et al., 2023) or morphometricity (Sabuncu et al., 2024). For instance, in the ABCD study, it was revealed that each imaging modality explains up to 5% of psychiatric symptoms in children (Ren et al., 2023). Bayesian approaches are also being developed to quantify explained variance in imaging (BRAINIAC) (Zablocki et al., 2023). Moreover, cumulative scores akin to PGSs are being constructed, such as the neuroimaging association scores (NASs) (Axelrud et al., 2021), the poly-vertex score (Zhao et al., 2021), brain-wide risk scores, and genetically-driven neuroimaging scores (PIDS) (Schleifer, 2024). These explain low variance in cognitive and psychiatric phenotypes, suggesting that their utility for prediction research is currently limited (Axelrud et al., 2021, Zhao et al., 2021). At an etiological level, their associations with psychiatric phenotypes may be non-causal due to genetic and environmental confounding including reverse causality (Axelrud et al., 2021). The usefulness of such scores is thus yet to be established. Nonetheless, PGSs’ clinical utility in psychiatry also remains limited, although such an approach is showing promise for risk prediction in breast cancer for example (Lewis and Vassos, 2020). Yet, we cannot expect methods that were developed in genetics to directly translate to imaging. The additional features of neuroimaging data (e.g., vary across time, multiple modalities) must be explored and could be capitalized to find other approaches to capture cumulative risk (e.g., sum of regions showing sustained neurodevelopmental delay). Notably, multivariate methods have been widely used to jointly model small effects in imaging with pattern classification. Nevertheless, these approaches present many challenges previously discussed (e.g., low external generalizability) (Brucar et al., 2023). Multivariate methods also have a high barrier to implementation (e.g., high computational skills required), which may give rise to methods misapplication. The involvement of methodological experts and the curation of resources and tutorials for methods’ application are paramount. Openly available machine learning algorithms are also becoming increasingly prevalent, allowing to draw upon existing models and repositories. Importantly, unlike genetics, where family-based designs informed on the upper bounds of variance explained by unmeasured genetic data, neuroimaging solely relies on estimates of the variance explained from existing imaging data. Thus, while the variance explained may appear to be small to date (Ren et al., 2023), it remains unclear whether this reflects a low utility of imaging data in psychiatric problems, or the limits of the commonly used data. As in genetics, where technological and study design, and other advances moved the field forward, methodological change may help neuroimaging research explain more variance. These advances could involve boosting sample size, utilizing novel or less commonly-used phenotypes (e.g., grey-to-white matter ratio, gyrification), more advanced diffusion modelling (Pasternak et al., 2018), MR spectroscopy (Egerton et al., 2018), and other methods, including yet unknown ones. Most importantly, embracing small and distributed effect sizes in neuroimaging has key implications for study design, as sample sizes will need to be increased, replicability embedded, and expectations on clinical utility and effect sizes recalibrated.

2.3. Addressing replicability challenges

2.3.1. The challenge of replicability from low statistical power and publication bias

Replicability was the biggest challenge for psychiatric genetics during the candidate gene era, and twenty years later has become a major strength of the field. Around 600 associations between genetic variants and disorders were reported in the candidate gene literature (Hirschhorn et al., 2002). Yet, most findings were from a single publication. When multiple studies were conducted on the same gene, associations were generally inconsistent: Of 166 candidate genes tested on multiple occasions, 6 were repeatedly identified across studies (Hirschhorn et al., 2002). Inconsistencies may reflect population variability. Studies on the topic however showed that low statistical power and publication bias were the large drivers of such discrepancies in the literature (Beauchamp et al., 2011, Colhoun et al., 2003). Candidate gene studies did not have sufficiently large samples to detect the small genetic effects (median sample size N = 345 (Duncan and Keller, 2011)) (Psychiatric GWAS Consortium Coordinating Committee, 2009). Such an issue was compounded by publication bias (Duncan and Keller, 2011). Larger effect sizes, and thus a greater likelihood of finding statistically significant relationships, were more common in smaller than larger samples (Duncan and Keller, 2011). This pattern should generally be reversed. With no publication bias, associations should have larger effect sizes and be more statistically significant in larger samples (assuming the hypothesis is true) (Duncan and Keller, 2011). This exemplified how publication bias rendered challenging the refutation of most candidate genes, with the role of several candidate genes being reconsidered many years after the original study (Beauchamp et al., 2011).

Similarly, concerns regarding the replicability of psychiatric neuroimaging have emerged (Poldrack et al., 2017, Button et al., 2013, Fletcher and Grafton, 2013). As previously discussed, numerous inconsistencies across findings have been observed. There has been low regional convergence for many psychiatric disorders (e.g., ADHD (Samea et al., 2019), Bipolar disorder (Harrison et al., 2020), ASD (Hiremath et al., 2021)), and low consistency between imaging studies with small vs. large sample sizes (e.g., for anxiety Harrewijn et al., 2021; Xie et al., 2021; Besteher et al., 2020). Analogously to psychiatric genetics, beyond population variability and other factors (e.g., differences in samples), low power and publication bias may substantially underlie inconsistencies. Evaluations into whether psychiatric neuroimaging studies are sufficiently large, and thus adequately powered to yield robust results, are taking place. Experts disagree (Brucar et al., 2023, Makowski et al., 2023). Reports of low power in neuroimaging (∼8% on average) have been shared around a decade ago (Button et al., 2013). Small samples of a couple of hundreds of participants are still common (Table 1), suggesting that such statistical power concerns may still be applicable. However, more recent studies offer much larger samples, in the thousands for both population-based (e.g., Generation R (Kooijman et al., 2016)) and clinical samples (e.g., ENIGMA (Thompson et al., 2022)). These have been shown to be both sufficiently and insufficiently powered for psychopathology measures with brain structure and connectivity (Makowski et al., 2023, Marek et al., 2022). Yet, evaluating statistical power is difficult if publication bias occurs, and if the “true” magnitude of effect sizes remains unknown. Nonetheless, patterns of greater statistical significance and larger (likely overestimated) effects with smaller samples have been shown in psychiatric neuroimaging (Feng et al., 2022, Marek et al., 2022). These reflect some of the concerning trends observed previously in psychiatric genetics, and which were indicative of publication bias (Klapwijk et al., 2021). Reports of an excess of positive results have been made for studies relating brain volume and functional differences to neurocognitive outcomes. The literature comprised twice the number of statistically significant findings than what would be expected based on the effect sizes (David et al., 2013, Ioannidis, 2011).

2.3.2. Navigating potential ways forward

These challenges highlight the need for making efforts to replicate the rule rather than the exception, and for creating a context where publication bias is reduced. One approach is to emphasize the need for statistical power, which typically comes with large sample sizes. In psychiatric genetics, samples sizes were scaled up to hundreds of thousands of individuals to attain greater statistical power. These large numbers could be obtained only through the collaboration of many sites (Sullivan, 2010), with consortia like the Psychiatric Genetics Consortium (PGC) (Sullivan, 2010), collaborative efforts from consumer-based industries (e.g., 23andMe (Zettler et al., 2014)), and electronic health records (e.g., PsychEMERGE (Zheutlin et al., 2019)). Further supporting this, journal publishing policies and, importantly, researchers themselves made well-powered replication studies or meta-analyses formal or informal requirements (Hewitt, 2012). Notably, only strict and precise definitions of replication (e.g., same SNP, direction of effect, statistical test) offered suitable assessments, while broader ones may have propagated false positives in the literature (Sullivan, 2007). This and the increased power gained from pooling data across samples made meta-analyses standard in the field of psychiatric genetics. Importantly, beyond capitalizing on data availability, psychiatric genetics has moved forward by reusing the results of such large, time-intensive collaborative efforts. Post-GWAS analyses have been extensive, as summary statistics are publicly shared and methods have been developed for their reuse to investigate other research questions (e.g., Mendelian randomization (Emdin et al., 2017)). These efforts to promote replicability required social changes in the scientific community. A team science approach has been pivotal. Individual or group contributions were de-emphasized at an institutional and funding level. New metrics for attributing credit were considered and collaborative funding options grew, thus offering support for data sharing and resource generation across many sites (Lehner et al., 2015).

Great breakthroughs are possible by promoting replicability in psychiatric neuroimaging. Increasing sample size is key to gaining power. However, this is more challenging in imaging than is in genetics, as the costs of the former remain relatively high. Fostering replications, collaborations, meta-, mega-, and federated analyses is another way forward. Studies in the field currently tend to rely on single samples (Table 1). This likely reflects difficulties with data access, computational challenges, and heterogeneity across studies (e.g., in ages, scanners, and protocols), which may render data pooling complex. Leveraging independent replication samples within one study could be pursued. In such cases, it is important to ensure that the replication test is as similar to the original test as possible, to prevent facing the same issues as in genetics. Several open datasets that could serve as replication studies or for joint analysis are available to address common research questions (e.g., ABCD, HCP). Moreover, untapped sites worldwide could join existing consortia or establish new ones. These could openly share data or summary statistics for mega-, meta-, or federated analysis across studies. Specifically, federated analysis offers the advantages of data pooling while also bypassing the challenges of data sharing, as it allows for data to remain at its original location (Rootes-Murdy et al., 2022). This is for example done with Coinstac (COINSTAC, 2023). Moreover, at least 15 open-source neuroimaging studies, each with over 700 individuals, are already available (Horien et al., 2021). OpenNeuro MRI also gathered data on more than 23,000 individuals from 600 samples (OpenNeuro, 2023). It is important to note, however, that the combination of such information could be challenging as data across studies were not necessarily collected with the same investigative purposes. A paradigm shift in routinely using multiple data resources to increase statistical power is, nonetheless, necessary in psychiatric neuroimaging. Furthermore, the potential for utilization of clinical data remains largely unexplored and could hold important value if more protocolized. When other data cannot be leveraged due to time or computational constraints, publicly available resources could be consulted, such as Neurosynth (Neurosynth, 2024, Yarkoni et al., 2011). It meta-analyzes foci from neuroimaging studies regardless of modality, with functional connectivity maps as reference. Such maps could be compared to study results to further contextualize the potential replicability of findings. Automated meta-analysis of publicly available summary statistics in existing software could also be implemented for other neuroimaging measures. Journal publication requirements may facilitate the routine use of meta/mega-analysis and replication as in genetics. Yet, as manuscripts are now promptly shared in open repositories (e.g., medRxiv (medRxiv.org, 2023)), the paradigm shift is perhaps more relevant at a researchers’ level, through personal standards on exploring replicability of findings, and peer review requests. Moreover, funding agencies like the National Institute of Health and the European Commission could further invest in existing collaborative initiatives, as well as in the development and maintenance of infrastructures and software necessary for such collaborations. To further exploit available resources, and to accommodate the need for greater statistical power when data sharing is difficult, psychiatric neuroimagers must routinize the sharing of summary statistics and brain maps in open repositories like NeuroVault (Gorgolewski et al., 2015), reuse results (e.g., through meta-analysis), and develop novel methods to capitalize on such reuse. Overall, it remains crucial to maximize sample size in psychiatric neuroimaging if average (i.e., between person) effects are of interest. This can be achieved through different strategies (e.g., meta-analysis, federated analyses, pooling of summary statistics, increase exposure variability). Importantly, in certain research endeavors, small studies also provide key insights. One scenario includes research where imaging assessments are used to evaluate symptom course from treatment (e.g., functional connectivity changes before, during, and after transcranial magnetic stimulation (Gell et al., 2024)). Due to their flexible research designs, they are optimally positioned to examine intraindividual (i.e., within-person) effects (Gell et al., 2024). They are also particularly advantageous to tailor to individual needs and hard-to-recruit patient populations (Gell et al., 2024). However, these studies, that rely on change across time, must focus on brain characteristics that are dynamic and show meaningful change over time. This requires a careful selection of timing and measurement intervals. Moreover, dense repeated imaging sampling is needed to reliably estimate individual brain topography (Gell et al., 2024). Such type of sampling remains scarce; few examples are currently present in psychiatric neuroimaging (Gell et al., 2024, Lynch et al., 2020).

2.4. Accounting for structures in the population

2.4.1. The challenge of structures in the population

Two key issues in psychiatric genetics that have been responsible for spurious or conflated associations are population stratification and cryptic relatedness (Sullivan, 2010, Hirschhorn et al., 2002). These refer to differences in genetic structures (allele frequencies) and correlations among genetic variants (linkage disequilibrium (LD)), across populations of distinct genetic ancestral groups (e.g., European, Asian genetic ancestries) (Sullivan, 2010). By affecting both genetic patterns and psychiatric problems, such allele frequency differences across genetic ancestries can confound associations of interest. While candidate gene studies sometimes attempted to address this by accounting for participant characteristics (e.g., ethnicity), the full influence of population stratification and cryptic relatedness on results could not be indexed with these course demographic variables, leading to poor control of bias, which resulted in inaccurate inferences (Sullivan, 2010, Hirschhorn et al., 2002). In an attempt to reduce spurious associations reflecting population structure, genetic studies homogenized their samples by focusing on one ancestry at a time. This, however, has had pervasive consequences as it often resulted in the exclusion of individuals who were not of European genetic ancestry. By 2008, 96% of genetic studies had been conducted in individuals of European descent (Need and Goldstein, 2009); by 2017, 88% (Mills and Rahal, 2019). This prevented the evaluation of the generalizability of findings to multiple populations and perpetuated existing health inequities (Bustamante et al., 2011).

Generalizability of findings to all populations is paramount to achieving health equity (Sterling et al., 2022). In psychiatric neuroimaging, studies have been historically conducted in racially and ethnically homogeneous samples, predominantly representing individuals self-identifying as White people (Sterling et al., 2022). This limited the generalizability of findings. Diverse samples, in terms of genetic ancestry, race, and ethnicity, are increasingly ascertained. Race and ethnicity are social determinants of health (Sterling et al., 2022), with potential relevance to psychiatric neuroimaging research. For example, there is evidence that experiencing systemic and interpersonal racism is associated with variation in the brain, and can place at a higher risk for psychiatric problems (Grasser and Jovanovic, 2022, Kalin, 2021, Okeke et al., 2023). Nonetheless, the field relies on the implicit assumption that population differences do not affect (confound or modify) associations between the brain and psychiatric symptoms, as data from different populations are typically pooled or jointly analyzed. If, however, this assumption is not met, both the validity and generalizability of findings would be affected, when not addressed. Such topic thus warrants more investigation.

2.4.2. Navigating potential ways forward

In psychiatric genetics, as information on ancestry could be inferred from genomic sequencing data, studies started to control for population structure with several approaches. Often, this is performed by adjusting for several principal components for the associations, attributable to ancestry (Beauchamp et al., 2011). Moreover, genomic relationship matrices have been used to account for cryptic relatedness (VanRaden, 2008). These represent the covariance of SNP information across individuals, thus capturing how certain individuals may more closely resemble others due to their population of origin. Genomic relationship matrices have been used as random effects to adjust for population stratification, as implemented in most GWAS tools (e.g., BOLT-LMM (Loh et al., 2018)). The approach of homogenizing samples in genetic ancestry has also been adopted, but it limits generalizability and health equity. Instead, methods to analyze data from ancestrally diverse populations have been increasingly generated in genetics. Notable examples are outlined in the literature (Peterson et al., 2019), and include the modeling of population-specific LD patterns, stratified or cross-ancestral meta-analyses, and methods to account for admixture, i.e., when the genome reflects more than one ancestry (Clyde, 2021). Such approaches have also been shown to increase statistical power, as was the case for cross-ancestry GWAS (Ishigaki et al., 2022), which involves the examination of multiple ancestral groups accounting for different LD patterns across populations.

In psychiatric neuroimaging, studies are increasingly more heterogeneous, sampling individuals of different genetic ancestries, races and ethnic backgrounds. Nonetheless, it has been shown that brain-phenotype relationships tend to predominantly reflect certain demographic characteristics (Greene et al., 2022). Further funding the sampling of groups that have been historically underserved in psychiatric neuroimaging is thus crucial (e.g., self-identified Asian, Black, Hispanic persons). One way to address this challenge could be by oversampling underserved populations in existing studies. Notably, the possible influence of sampling bias should also be considered, i.e., the reduced chance of selection of underserved populations within the study sampling frame. Another way forward includes dedicated studies restricted to historically underrepresented populations in global research (e.g., Chinese HCP (Ge et al., 2023), GUSTO (Soh et al. 2014)). Conducting these dedicated studies has for instance revealed the need for face anonymization templates tailored to Asian populations for data protection during imaging data extraction (Gao et al., 2022). In terms of validity, to estimate associations across several populations, various options warrant consideration. Following in genetics’ footsteps, genetic information on ancestry could be used with principal component and genomic relationship matrices approaches to adjust for the potential confounding influences of population structure. Yet, this approach may not directly translate to imaging, as there is no evidence to date that allelic structure differences across populations may influence brain characteristics. Moreover, the brain is sensitive to environmental exposures, meaning that social factors related to genetic ancestry (e.g., systemic racism) could determine morphological and functional changes, as previously found (Grasser and Jovanovic, 2022, Okeke et al., 2023). How to validly estimate associations between imaging and psychiatric phenotypes in diverse samples is being discussed in the field. Some researchers argue that population-specific brain atlases (Xing et al., 2013) are required to accommodate the need for accurate pre-processing (and subsequent analyses) in populations that have not been traditionally captured in imaging templates. Risk stratification and interaction models by population characteristics have also been suggested, to disaggregate information (Ricard et al., 2023). Others are controlling for race, ethnicity, or genetic ancestry (Table 1). Yet some other researchers have advocated not to adjust for race and ethnicity, and to rather focus on indices of systemic and interpersonal discrimination and other structural and social determinants of health (Cardenas-Iniguez et al., 2020). It remains uncertain whether any of these variables would sufficiently capture the broad impact of system disparities and inequities that have historical and current repercussions in racial and ethnic minoritized groups (e.g., redlining, access to education, etc…). Carefully-designed and ethically-responsible studies should be conducted on the topic, to investigate how to validly estimate brain-psychiatric associations across different populations.

2.5. Capitalizing on shared biological patterns across psychiatric disorders

2.5.1. The challenge of shared biological patterns across psychiatric disorders

Psychiatric disorders and traits co-occur frequently, as exemplified by anxiety and depression which are more often comorbid than not (Beesdo-Baum and Knappe, 2012). Some researchers even postulate a general dimension of common psychopathological vulnerability (Caspi et al., 2014). This co-occurrence and the common underlying trait may be due to greater functional problems that, in turn, increase the risk of developing another problem (McElroy et al., 2018). Alternatively, they may reflect shared genetic or environmental risk factors (Sprooten et al., 2022). Against this background, an intriguing discovery from the genome-wide era of psychiatric genetics is that genetic liability transcends diagnostic boundaries (Smoller, 2019). The clinical nosology of psychiatric disorders does not align with the underlying genetic patterns (Smoller, 2019). That is, psychiatric disorders do not present specific genetic risks, but rather many variants of liability are shared across mental health problems (Smoller, 2019). Genetic correlations across psychiatric disorders have been mapped, showing substantial overlaps, as high as 70% of the shared genetic liability for schizophrenia and bipolar disorder (Bulik-Sullivan et al., 2015). Pleiotropy, i.e., the manifestation of multiple traits from the same gene, is thus omnipresent. This poses a challenge when examining the underlying genomics of single disorders.

Similarly, for psychiatric neuroimaging, shared variance in the neurobiology of mental illness has been suggested. For instance, ENIGMA studies have revealed that major depression, schizophrenia, and bipolar disorder share neuroanatomical differences, albeit with varying magnitude of effect sizes (Schmaal et al., 2020). Another notable example is a study of 6 psychiatric phenotypes, which demonstrated neuroimaging-based correlations of up to 0.63 between autism and schizophrenia (Writing Committee for the Attention-Deficit/Hyperactivity Disorder, 2021). Yet, most studies examine one phenotype in isolation (Table 1) or, when examining multiple phenotypes, focus on the differences between individuals with a disorder vs. another (Mitelman, 2019). While this is not a methodological limitation that ought to be addressed like prior issues in the field (e.g., replication), approaches to exploit neurobiological commonalities across psychiatric disorders could benefit imaging.

2.5.2. Navigating potential ways forward

In psychiatric genetics, it is widely accepted that the association of genetic variants with multiple traits is the norm rather than the exception. To account for the shared liability across mental health problems, cross-disorder analyses are becoming increasingly common. These increase the power of the analyses and help understand shared etiology. Key efforts come from the PGC Cross-Disorder workgroup, which set out to explore overlapping genetic risk for psychiatric problems (Psychiatric GWAS Consortium Coordinating Committee, 2009). Several methods have been developed to investigate shared genetic liability to unravel cross-disorder associations and thus possible pleiotropy based on GWAS summary statistics. MiXeR (Frei et al., 2019) and conjFDR (Smeland et al., 2020) are two examples of methods that allow to ask multivariate questions from univariate summary-statistics data, and that can inform not only on genetic correlations but also on the specific causal variants shared across traits. Genomic structural equation modeling achieves a similar goal by also accounting for the structure of associations between the outcomes, e.g., a hierarchical structure as commonly seen in psychiatric traits (Grotzinger et al., 2019). Finally, multi-trait GWAS (Turley et al., 2018) and conditional GWAS (Byrne et al., 2021) use data from multiple GWASs to remove shared effects and consequently boost the statistical power to find variants unique to a given trait. Illustrating the utility of such methods, combining phenotypes has been shown to increase the number of associated loci for depressive symptoms, neuroticism, and subjective well-being (Turley et al., 2018). These methods show how summary statistics from prior GWASs are used to investigate the shared and unique genetic architectures, without the need for all-encompassing study designs.

Some analogous options could be considered for psychiatric neuroimaging. First, to investigate similarities and differences across psychiatric disorders, summary statistics, and brain maps from univariate analyses could also be used to ask multivariate questions. While still relatively uncommon, this has been previously done with summary statistics from the ENIGMA study for structural abnormalities in major depression, bipolar disorder, and schizophrenia, revealing moderate-to-high correlations in adults (r = 0.4–0.8) (Opel et al., 2020). This can be easily achieved within a given study with data on multiple phenotypes. Notably, in genetics, specific approaches accounting for the LD-structure of the genome have been developed for calculating genetic correlations. In imaging, instead, we still rely on classical correlational methods which may inflate effect sizes. Developing an analogous neural correlation method should be considered. Moving beyond statistical correlations, identifying the specific locations and patterns of shared neurobiology is key. This could be done, for example, by overlapping maps from brain-wide analyses of different disorders within a study. It could also be performed across studies, by leveraging openly shared brain maps (e.g., from NeuroVault (Gorgolewski et al., 2015)). This, however, may present several challenges in neuroimaging. In fact, data are substantially influenced by site effects, scanner, acquisition, and other parameters which may be study-specific (Alfaro-Almagro et al., 2021), including confounding patterns (Bayer et al., 2022). Moreover, maps must be moved to a common space, which could be facilitated by tools like neuromaps (Markello et al., 2022). As cross-disorder approaches enhance the statistical power of genetic analyses, it could be insightful to investigate the impact on statistical power in psychiatric neuroimaging studies when data from multiple disorders are combined. Further, similarly to genetics, structural equation modeling and conditional approaches can be and have been used to identify shared vs. specific neurobiological features (e.g., Cardenas-Iniguez et al., 2020). Additionally, neuroimaging has leveraged multivariate approaches to examine multiple disorders concurrently. For instance, it has been found that aggression and rule-breaking symptoms share functional connectivity patterns in the ABCD Study (Xu et al., 2024). Importantly, cross-disorder investigations would be especially valuable in childhood and adolescence when homotypic continuity, i.e., presenting the same psychiatric phenotype over time, is more limited (Blok et al., 2021).

3. The developmental challenges of psychiatric neuroimaging

Neuroimaging presents an array of challenges above those encountered by genetic research, some of which are due to its more prominent developmental aspects. Variable gene expression and age-dependent genetic effects are well described and currently explored (Thapar and Riglin, 2020), but developmental research in neuropsychiatry has a long history. Unlike genetic variation, the time-varying nature of neuroimaging data requires additional considerations, and the collection of repeated imaging measurements. This has been long recognized in the field as many longitudinal studies are currently available (Kooijman et al., 2016, Garavan et al., 2018, Satterthwaite et al., 2014). How to best harness these data remains however unclear. As shown in Table 1, longitudinal designs are not default. When adopted, these often involve adjustments for baseline levels of the outcome, but the use of repeated exposures and outcomes remains rare. Several “how-to” guides to longitudinal modelling in neuroimaging have been recently made available, offering a wealth of information for tackling developmental questions (McCormick et al., 2023, King et al., 2018). Further, parallels with other fields encountering similar developmental challenges (e.g., epidemiology, transcriptomics, epigenomics) can offer guidance.

Both genetics and imaging likely present timing effects, i.e., specific developmental periods at which the effect of genetics or neurobiology on psychiatric disorders is stronger (e.g., puberty as a sensitive period (Vijayakumar et al., 2018)). For instance, it has been shown that the genetic heritability of ADHD can vary over time, which might reflect changes in gene expression (Chang et al., 2013). To explore timing, certain methods have been applied in epigenetics such as the structured life course modelling method (Lussier et al., 2023, Smith et al., 2022), the strategic pooling of meta-analyses across ages (Neumann et al., 2020), and meta-regression of publicly-available meta-analyses to evaluate how epigenetic patterns at different ages would relate to a given psychiatric outcome (Neumann et al., 2024). Moreover, marginal structural and structural nested models could be borrowed from causal epidemiology to investigate timing effects (Hernan and Robins, 2020, Schwartz and Glymour, 2023). Although data collection efforts allow imaging scientists to test the timing of effects on psychiatric problem development using such approaches, they are not often utilized. Another developmental challenge is the time of effects, i.e., how long it takes for certain neuroimaging differences to be manifested in psychiatric problems development. Several developmental theories have been postulated on this (e.g., maturational framework (Johnson, 2001)), but they remain rarely tested in this context and do not provide clear time estimates. As relevant effect duration remains unknown and is likely dependent on imaging modality, the interval at which repeated measurements are leveraged might obfuscate time effects, if too short or long. The testing of multiple options of developmental times is a way forward (e.g., both two- and four-year intervals for ABCD). Yet, it might be that it is not the length of time of a particular neuroimaging difference (e.g., lower prefrontal cortical thickness in childhood) that is involved in the disorder onset, but rather for how long neurodevelopment deviates from the expected growth patterns over time (e.g., a persistent delay in cortical thickness development). In such cases, user-friendly tools like Predicting Clinical Neuroscience and Centile Brain can facilitate the mapping of deviations in neurodevelopmental patterns compared to normative growth curves, offering a way to explore potential time effects (CentileBrain, 2024, Predictive Clinical Neuroscience Portal, 2024). Overall, the psychiatric neuroimaging field offers unprecedented opportunities to further explore developmental patterns, which, while challenging, could hold great promise to advance our understanding of the neurobiological aetiology of psychiatric disorders.

4. Conclusions

In conclusion, the past decade of psychiatric neuroimaging studies has been reminiscent of some developments that occurred in the history of genetic research. We can draw insights from the evolution of psychiatric genetics by (i) revisiting oversimplified biological hypotheses with hypothesis-free approaches, (ii) embracing distributed small effect sizes, (iii) ensuring replicability by making collaborations, meta-analysis, and the sharing of summary-level statistics a rule rather than the exception, (iv) considering population structure for valid and generalizable inferences, and (v) exploiting shared neurobiological patterns with cross-disorder approaches. With knowledge from the past of psychiatric genetics, and the novel tools available in psychiatric neuroimaging (e.g., large samples, multivariate methods), the field is better equipped than ever to understand the etiology, course, prediction, and treatment of psychiatric disorders. To move forward and increasingly approximate the neuroimaging characteristics of psychiatric problems, such lessons and tools need to be routinely applied, and novel methods developed where needed. The field has made great strides, with numerous essential steps having already been taken. This opens the opportunity for more to be made; quickly and responsibly, with a focus on ensuring that the integration of these insights into one’s workflow is seamless and incentivized. We must draw insights from psychiatric genetics now for psychiatric neuroimaging to flourish.

Funding

This work was supported by an NWO-VICI grant (NWO-ZonMW: 016.VICI.170.200) and the Academy Ter Meulen grant of the Academy Medical Sciences Fund of the Royal Netherlands Academy of Arts & Sciences.

CRediT authorship contribution statement

Henning Tiemeier: Writing – review & editing, Writing – original draft, Supervision, Project administration, Investigation, Funding acquisition, Conceptualization. Ryan L. Muetzel: Writing – review & editing, Supervision, Project administration, Conceptualization. Jordan W. Smoller: Writing – review & editing, Conceptualization. Scott Delaney: Writing – review & editing, Conceptualization. Sander Lamballais: Writing – review & editing, Conceptualization. Travis Mallard: Writing – review & editing, Conceptualization. Saúl Urbina Johanson: Writing – review & editing, Data curation. Lorenza Dall'Aglio: Writing – review & editing, Writing – original draft, Project administration, Investigation, Conceptualization.

Declaration of Generative AI and AI-assisted technologies in the writing process

During the preparation of this work the authors used ChatGPT in order to rephrase parts of the text. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data Availability

No data was used for the research described in the article.