Reply to: New meta-analyses and mega-analyses of MRI findings in schizophrenia: do they really increase our knowledge about the nature of the disease process?

To the Editor:

In their letter to the editor, Vita and De Peri question whether new meta-analyses and mega-analyses of magnetic resonance imaging (MRI) findings in schizophrenia increase our knowledge about the nature of the disease process. In general, meta-analyses and mega-analyses provide objective methods to critically summarize a body of evidence regarding a particular question. As there had been no coordinated meta-analysis of cortical thickness and surface area abnormalities in schizophrenia, it is our view that this new, collaboratively conducted meta-analysis (1) contributes to our knowledge on this question and offers information on the cross-site consistency of observed disease effects. Regional effects on cortical thickness and surface area can be difficult to summarize based on the traditional, literature-based, meta-analysis method, given the heterogeneity of analysis methods used in individual studies.

The Enhancing Neuro Imaging Genetics through Meta Analysis (ENIGMA) approach of collaboratively conducting meta-analyses offers additional benefits. First, ENIGMA’s publicly available methods lend themselves well to independent replication of imaging findings (2, 3), which is crucial given the ‘crisis of replication’ in neuroscience (4-6). Second, use of the same quality assurance, image processing, and statistical analysis methods across samples within and across ENIGMA working groups, minimizes method-related heterogeneity and offers the potential for straightforward cross-disorder comparisons (7-11). Third, use of similar meta-analytic methods across worldwide samples has generated imaging and genetics findings with sample sizes beyond the scope of any individual laboratory or consortium studying a single disorder (12-15).

Vita and De Peri repeat one of the study weaknesses already listed in the discussion, namely that possible group differences in lateralization were not examined. This question is under investigation by the ENIGMA Laterality Working Group, which is currently examining healthy and disordered brain laterality (16, 17). ENIGMA coordinates publication efforts across working groups in order to avoid overlap. Moreover, numerous ENIGMA studies make important contributions showing between-disorder brain differences without addressing laterality.

Vita and De Peri also mention that the meta-analysis does not address possible differential longitudinal trajectories between individuals with schizophrenia and healthy volunteers, which is also correct as this cross-sectional meta-analysis did not aim to examine longitudinal trajectories. There are ongoing efforts by the ENIGMA Plasticity Working Group to study genetic influences on individual differences in longitudinal brain changes (18). We agree that further investigation of questions regarding longitudinal trajectories of brain changes across the lifespan, especially prior to illness onset, e.g., in adolescents at clinical high risk for psychosis, as well as after a first psychotic episode, will provide valuable information with regard to schizophrenia pathogenesis and several such analyses are planned or already ongoing.

Vita and De Peri further state that the meta-analysis does not add relevant information about the effects of antipsychotic medication on brain morphology but qualify that the reported findings seem compatible with findings from longitudinal MRI studies that suggest different effects of first versus second-generation antipsychotic treatments on cortical gray matter changes. We point out that prior meta-analyses did not dissociate effects of antipsychotic treatments on cortical surface area versus cortical thickness, whose product constitutes gray matter volume, and that the consistency of findings is important in the light of reports on non-replication in neuroscience.

The comment that “the supposed huge statistical power of mega-analyses of MRI findings in schizophrenia may be undermined by the large variation of data obtained by different centers in disparate conditions” is incorrect. First, Van Erp et al. (2018) is a meta-analysis and not a mega-analysis, which like any other meta-analysis, summarizes within-sample effects. In fact, joint meta-analyses tend to reduce method-related variation when compared to literature-based meta-analyses because similar analysis methods are applied across samples. Second, multiple imaging genetics meta-analyses replicate common genetic variants associated with measures of brain structure and find a greater number of common variants associated with these measures when additional independent samples are added (19-21). These findings suggest increased power as brain imaging data from independent samples are added. Finally, the suggestion that mega-analyses of MRI data are undermined by between site variation is not borne out by the facts. Research from a decade ago showed the feasibility and the additional power gained by pooling legacy structural imaging data (22). More recent studies show that meta-analyses and mega-analyses of structural imaging data, whether from prospective multi-scanner or independent samples, yield significant and very similar findings (23-25). Each analysis method has strengths, weaknesses, and pitfalls. Hence researchers must consider whether to conduct a meta-analysis, a mega-analysis, or both, to answer a particular question.

The suggestion that meta-analyses and mega-analyses are not hypothesis-driven approaches is also incorrect. All published ENIGMA Schizophrenia Working Group meta-analyses list their hypotheses at the end of their introductions (1, 2, 26-28). Of note, nowhere in the manuscript do we state that “meta-analyses provide better evidence than large, well designed, hypothesis-driven, high-quality individual trials”. On the contrary, all findings from meta-analyses depend on the quality of the studies on which they are based. Even so, meta-analyses can offer additional safeguards against false positive findings generated by individual studies with small or highly heterogeneous samples by taking into account each sample’s error terms. We do agree that missing data for known or supposed significant moderators can be an issue. However, this is a criticism of all analyses of scientific data, rather than of our study specifically.

Finally, we respectfully disagree with the statement by Vita and De Peri “that the time has come for applying really new approaches to the study of the nature of the disease process underlying schizophrenia, rather than promoting redundant research on mega-databases which may even dilute or confuse established knowledge”. We believe there is value both in the relatively new approach of large-scale collaborative research on costly, already collected data, as well as applying other innovative approaches and experimentation in adequately powered samples. We believe that most scientists who contribute to ENIGMA or other consortia as well as the funding agencies who promote large-scale data sharing and analysis recognize that both approaches make valuable contributions to the field.