Skip to main content
NCBI home page
JCPP Advances logoLink to JCPP Advances
. 2025 Aug 1;6(1):e70017. doi: 10.1002/jcv2.70017

Guidance for umbrella reviews of observational studies: A scoping review

Carl Zhou 1, Nicholas Fabiano 1, Arnav Gupta 2,3, Stanley Wong 4, Kelly D Cobey 5,6, David Moher 5, Sanam Ebrahimzadeh 7, Jeremy Y Ng 8, Elena Dragioti 9, Jae Il Shin 10,11, Joaquim Radua 12,13,14, Samuele Cortese 15,16,17,18,19, Beverley Shea 5,6, Nicola Veronese 20, Lisa Hartling 21, Michelle Pollock 22, Stefania Papatheodorou 23,24, John P A Ioannidis 25,26,27, Marco Solmi 1,5,7,28,29,✉
PMCID: PMC12973145  PMID: 41815760

Abstract

Background

Umbrella reviews, or overviews of reviews, synthesize information using systematic reviews (SRs) as their unit of analysis. Although a formal guideline exists for reporting umbrella reviews of healthcare interventions (i.e. Preferred Reporting Items for Overviews of Reviews [PRIOR]), no formal guideline exists for conducting and/or reporting umbrella reviews of observational studies that examine epidemiological associations.

Objective

To review the existing guidance on conducting and/or reporting umbrella reviews of observational studies on epidemiological associations, as part of the process of developing a formal reporting guideline.

Methods

We reviewed the scoping review conducted in the context of PRIOR development and identified documents through forward citation search in PubMed, Scopus, and manual search in Google Scholar, Google Search up to December 22, 2024. Documents, regardless of format, were included if they provided guidance for conducting and/or reporting umbrella reviews of observational studies (including meta‐research studies of their features). Title/abstract screening and data extraction were performed independently and in duplicate and summarized narratively by stages of the umbrella review process.

Results

The search retrieved 4491 unique records, with 96 full texts assessed and eight documents included. These documents, published between 2014 and 2023, offered guidance across seven topic areas, but overall guidance on conducting and/or reporting is limited. These areas include the answerable questions, prerequisite considerations, the scope of umbrella reviews, searching for SRs, primary data collection, analysis, presentation, and assessing the certainty/quality of the body of evidence.

Conclusion

There is a need for dedicated, practical, and evidence‐based formal reporting guidelines for umbrella reviews of observational studies on epidemiological associations. This review lays the groundwork for developing the PRIOR‐extension for such studies: the Preferred Reporting Items for Umbrella Reviews of Cross‐sectional, Case‐control, and Cohort Studies.

Keywords: evidence‐based medicine, evidence synthesis, meta review, observational study, overview of reviews, reporting guideline, systematic review, umbrella review

This review examined existing guidance on conducting and reporting umbrella reviews of observational studies on epidemiological associations, aiming to help develop a formal reporting guideline. The search identified 4491 records, with eight documents providing limited guidance across seven topic areas. The findings highlight the need for dedicated guidelines, leading to the development of the Preferred Reporting Items for Umbrella Reviews of Cross‐sectional, Case‐control, and Cohort Studies.

graphic file with name JCV2-6-e70017-g001.jpg

Key points.

  • Umbrella reviews synthesize findings from systematic reviews and are increasingly used to examine epidemiological associations using observational studies.

  • Currently, there is no formal guideline for conducting or reporting umbrella reviews that focus on cross‐sectional, case‐control, and cohort studies.

  • This scoping review identifies and summarizes existing but fragmented guidance across eight publications.

  • Key gaps remain in areas such as protocol development, managing overlapping reviews, and assessing certainty of evidence.

  • Relevance: This work supports the development of the Preferred Reporting Items for Umbrella Reviews of Cross‐sectional, Case‐control, and Cohort Studies reporting guideline, which aims to enhance methodological transparency and rigor in future umbrella reviews, thereby improving the quality of scientific evidence that informs policy and practice.

INTRODUCTION

The proliferation of systematic reviews (SRs) and meta‐analyses (MAs) in the literature has escalated dramatically over the past 2 decades (Arango et al., 2021; Janiaud et al., 2021). This surge has given rise to a novel category of knowledge synthesis whereby a review is conducted on multiple SRs or MAs to consolidate their findings, which is known as “umbrella review,” “overview of (systematic) reviews,” “meta‐review,” or “review of reviews.” (Ioannidis, 2009; Papa and theodorou, 2019; Pollock et al., 2019) These reviews, which can encompass SRs of experimental, observational, or other study designs, aim to provide a comprehensive synthesis of information by using SRs or MAs as their unit of analysis (Pollock et al., 2019). This approach has become increasingly popular, as evidenced by the approximately eighteen‐fold increase in published umbrella reviews per year from 56 in 2010 to 1013 in 2024 (PubMed, [umbrella review]). In comparison, there was an approximately twelve‐fold increase in published SRs and meta‐analyses per year from 2291 in 2010 to 28,186 in 2024 (PubMed, [SR and meta‐analysis]), and a three‐fold increase in published cross‐sectional studies, case‐control studies, and cohort studies per year from 64,462 in 2010 to 190,734 in 2024 (PubMed, [cross‐sectional OR case‐control OR cohort]).

Overviews of (systematic) reviews of observational studies on epidemiological associations specifically have largely been reported as “umbrella reviews” in the literature (Arango et al., 2021; Janiaud et al., 2021). For consistency, we use the term “umbrella review” herein when referring to this study design.

Despite their growing popularity, the methodologies employed in umbrella reviews exhibit considerable variability, complicating the comparability of results (Arango et al., 2021; Solmi et al., 2018a). Although the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) guideline exists as the standard for reporting SRs and MAs (Page et al., 2021), heterogeneity persists, reflecting the diverse methodologies used in individual SRs and MAs (Hailes et al., 2019; Ioannidis, 2016; Solmi et al., 2018b). Further, the quality of evidence synthesis efforts is often fundamentally linked to the quality of the included studies within SRs and MAs. Generally, randomized controlled trials (RCTs) are regarded as higher‐quality sources within the evidence‐based medicine hierarchy. In contrast, observational studies, such as cross‐sectional, case‐control, and cohort studies, are often more susceptible to various biases (Ioannidis and Trikalinos, 2007).

To address these challenges, reporting guidelines have been developed to enhance the transparency and quality of research reporting (Moher et al., 2010). Notable examples include the PRISMA guidelines for SRs and MAs of interventions and the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines for observational studies (Page et al., 2021; von Elm et al., 2007). These guidelines, tailored to different study designs, provide structured checklists of essential information to report in a manuscript, thereby helping researchers standardize their reporting. This standardization facilitates the assessment of quality, reliability, and validity of research findings and enhances the reproducibility of studies. However, while reporting guidelines exist for umbrella reviews focused on SRs of healthcare interventions, that is, the Preferred Reporting Items for Overviews of Reviews (PRIOR) (Gates et al., 2022), guidance is lacking for those focusing on SRs of observational studies on epidemiological associations. Currently, no EQUATOR Network guidelines address the specific challenges of umbrella reviews for cross‐sectional, case‐control, and cohort studies. This gap highlights the need for tailored reporting guidelines to ensure the quality and reliability of such reviews.

In response to this gap, we are developing a reporting guideline for umbrella reviews for three types of observational studies: cross‐sectional, case‐control, and cohort. Following the EQUATOR guidance for developing reporting guidelines (Moher et al., 2010), we must identify existing knowledge to inform the development of our guideline, the Preferred Reporting Items for Umbrella Reviews of Cross‐sectional, Case‐control, and Cohort Studies (PRIUR‐CCC). Therefore, our goal with this study is to conduct a scoping review of existing documents that discuss the conduct and/or reporting of umbrella reviews of observational studies on epidemiological associations.

METHODS

This scoping review was conducted in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews (Peters et al., 2024). We followed the Preferred Reporting Items for SRs and Meta‐Analyses Extension Statement for Reporting of Scoping Reviews (PRISMA‐ScR) to guide our conduct, recognizing that PRISMA‐ScR is a reporting guideline (Tricco et al., 2018). This review is Step 2 of the EQUATOR guidance‐compliant project titled “Preferred Reporting Items for Umbrella Reviews of Cross‐sectional, Case‐control, and Cohort Studies (PRIUR‐CCC),” which aims to develop a reporting guideline for umbrella reviews of observational studies on epidemiological associations. The protocol of this project has been published (Solmi et al., 2024) and is publicly available at medrxiv.org (MEDRXIV/2022/283,572). The Ottawa Health Science Network Research Ethics Board (20220639‐01H) has approved the broader PRIUR‐CCC project, which includes the following components: (1) stakeholder identification, (2) a scoping review on umbrella review guidance, (3) a three‐round Delphi study, and (4) PRIUR‐CCC statement finalization.

Search strategy

We reviewed the scoping review conducted in the context of PRIOR's development (Gates et al., 2020) for available guidance regarding the conduct and/or reporting of umbrella reviews of observational studies. The PRIOR scoping review included documents that met the broad definition of “being about overviews” or “discussing some aspect of overviews” in its title/abstract screening phase. Then, we performed forward citation searching in PubMed and Scopus for each document included in the PRIOR scoping review to identify “citing” references in Scopus or “similar articles” in Pubmed from 2020 (inclusive) (year of last search for the PRIOR scoping review (Gates et al., 2020)) through December 22, 2024. Manual searches were conducted using Google Scholar for academic literature and Google Search for gray literature during this time to supplement the comprehensive manual search conducted during the PRIOR scoping review (i.e., the websites of 59 organizations and major evidence synthesis centers up to February 12, 2019, the proceedings from four international conferences up to February 5, 2020). Manual searches used the search terms “umbrella review,” “overview of reviews,” “overview of SRs,” and “meta‐review” in conjunction with “guidance.” The first 20 results pages were reviewed, limited to articles since 2020.

Inclusion/exclusion criteria

We included documents in any language or year that provided guidance applicable to the context or process of any aspect of conducting and/or reporting umbrella reviews of three types of observational studies (cross‐sectional, case‐control, and cohort studies), with or without empirical meta‐research studies addressing features of umbrella reviews. This can include documents not explicitly related to umbrella reviews of observational studies, such as those primarily focusing on experimental studies, diagnostic test accuracy, and prediction models.

We excluded guidance documents whose guidance was not applicable to the context or process of any aspect of conducting and/or reporting umbrella reviews of the aforementioned three types of observational studies. We excluded documents whose contents were unrelated to discussing methods or reporting guidance. We also excluded protocols, preprints, and conference abstracts for which a full version of the document that provided additional information was not available.

Study screening and data extraction

Documents from the forward citation database search and all documents and abstracts cited in the PRIOR scoping review on guidance for overviews of reviews of healthcare interventions (Gates et al., 2020) were imported into Covidence, a SR management platform (Covidence Systematic Review Software, Veritas Health Innovation). Three reviewers (NF, AG, SW) independently screened titles and full texts and in duplicate, with disagreements resolved by consensus or the involvement of a third reviewer.

Two reviewers (among NF, AG, SW) independently extracted relevant data from the included literature onto a Microsoft Excel extraction form designed a priori. The data extraction form was not revised during the data extraction process. All extractions were done independently and in duplicate, with discrepancies resolved by consensus. The extracted guidance was narratively summarized and organized by stages of the umbrella review process. Specifically, the guidance was divided into the 15 topic areas identified by PRIOR (Gates et al., 2020), if applicable.

RESULTS

The initial searches retrieved 7790 records. 4491 unique records were retrieved after removing 3299 duplicates. We screened these records by title and abstract, excluding 4395. We then assessed the full text of 96 potentially relevant articles, excluding 88, and included eight documents published between 2014 and 2023. Figure 1 contains a flow diagram of the review process.

FIGURE 1.

FIGURE 1

Open in a new tab

Preferred reporting items for systematic reviews and meta‐analyses flow diagram of document selection.

Descriptive summary

The eight included documents comprise seven review articles (Arango et al., 2021; Aromataris et al., 2015; Baker et al., 2014; Belbasis et al., 2022; Bougioukas et al., 2023; Fusar‐Poli and Radua, 2018; Janiaud et al., 2021) and one letter to the editor (Gianfredi et al., 2022). Most review articles were narrative reviews, except for two reviews of umbrella reviews (Arango et al., 2021; Janiaud et al., 2021) and one meta‐research empirical study (Bougioukas et al., 2023). All documents provided guidance applicable to umbrella reviews of observational studies. Table 1 summarizes the characteristics of the eight included guidance documents. In the following sections, we summarize the available guidance for conducting and/or reporting umbrella reviews of observational studies subdivided into seven of the 15 topic areas identified in the PRIOR scoping review on guidance for overviews of healthcare interventions (Gates et al., 2020). We did not find guidance regarding the remaining eight topic areas, namely: (1) choosing between conducting an umbrella review of observational studies and a new SR, (2) author team composition and roles, (3) target audience of the umbrella review of observational studies, (4) developing and registering an umbrella review of observational studies' protocol, (5) collecting and presenting data on descriptive characteristics of included SRs (and their primary studies), (6) consideration and management of overlapping SRs, (7) interpreting outcome data and drawing conclusions, and (8) updating the umbrella review. The summarized guidance is presented in tabular format in Table 2.

TABLE 1.

Characteristics of the included guidance documents (n = 8).

Author team (research group) Year of publication Document format
Bougioukas et al. (2023) 2023 Review article
Aromataris et al. (2015); (Joanna Briggs Institute) 2015 Review article
Baker et al. (2014) 2014 Review article
Belbasis et al. (2022) 2022 Review article
Fusar‐Poli and Radua (2018); (Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, London, UK) 2018 Review article
Gianfredi et al. (2022); (School of Medicine, Vita‐Salute San Raffaele University, Milan, Italy) 2022 Letter
Arango et al. (2021) 2021 Review article
Janiaud et al. (2021); (Meta‐Research Innovation Center at Stanford [METRICS], Stanford, USA) 2021 Review article
Open in a new tab

TABLE 2.

Summary of the available guidance‐related information for conducting and/or reporting umbrella reviews of observational studies.

Topic Reporting authors Summary of reported guidance
What types of questions can be answered using the umbrella review format? Bougioukas et al. (2023) Umbrella reviews of observational studies typically examine the link between exposures and disease development, investigating risk factors and biomarkers using various observational study designs (Bougioukas et al., 2023). In contrast, overviews involving interventions primarily assess the effectiveness and safety of interventions using SRs of RCTs, occasionally incorporating observational studies for rare adverse events
Items to consider before conducting an umbrella review

Aromataris et al. (2015)

Baker et al. (2014)

 There should be a plan to handle pooled estimates from different study designs, ensuring the methodology and study selection align with the umbrella review's primary objective (Aromataris et al., 2015; Baker et al., 2014)
Specifying the eligibility criteria for the umbrella review Belbasis et al. (2022) Set strict and precise criteria to exclude ineligible primary studies or if this is not feasible, examine each study captured by the SR/MA and recalculate the pooled estimates (see section on collecting, analyzing, and presenting outcome data) (Belbasis et al., 2022)
Conducting and reporting searches for systematic reviews Fusar‐Poli and Radua (2018) The literature search process should strictly adhere to PRISMA recommendations to ensure comprehensive and transparent reporting (Fusar‐Poli and Radua, 2018). Incorporate specific guidelines tailored to the types of studies being reviewed, such as applying the MOOSE guidelines for observational studies
Collecting, analyzing, and presenting outcome data

Fusar‐Poli and Radua (2018)

Gianfredi et al. (2022)

 Using various effect size measures for observational studies is beneficial, but employing a common effect size facilitates easier comparisons (Fusar‐Poli and Radua, 2018). To establish temporality and minimize reverse causation, sensitivity analysis including only prospective studies is advised, while acknowledging that MAs of observational studies often face biases affecting sample representativeness (Fusar‐Poli and Radua, 2018; Gianfredi et al., 2022)
Collecting and presenting data on the methodological quality (risk of bias) of primary studies contained within the included systematic reviews

Belbasis et al. (2022)

Gianfredi et al. (2022)

 When some SRs and MAs include risk‐of‐bias assessments using standardized tools, researchers are encouraged to summarize these assessments in a clear tabular format (Belbasis et al., 2022). The inherent heterogeneity in exposure and outcome assessment from original studies often persists in MAs and umbrella reviews, posing challenges for reliable conclusions. To address this, establish and adhere to strict inclusion and exclusion criteria for SRs and MAs (Gianfredi et al., 2022)
Assessing the methodological quality and certainty of the included systematic reviews

Arango et al. (2021)

Janiaud et al. (2021)

 The certainty of evidence from SRs of observational studies should be assessed with quantitative criteria (i.e., GRADE), including sensitivity analyses for temporality, and findings should be discussed with input from content experts. (Arango et al., 2021; Janiaud et al., 2021)
Open in a new tab

What types of questions can be answered using the umbrella review format?

Bougioukas and colleagues (Bougioukas et al., 2023) stated that umbrella reviews of observational studies typically examine the epidemiological link between exposures (both genetic and non‐genetic) and disease development using data from SRs. These reviews often investigate potential risk or protective factors, but can also measure biomarkers, including SRs of different observational study designs (e.g., cross‐sectional, case‐control, and cohort studies). These associations often cannot be examined through experimental studies, and only observational evidence exists. In contrast, overviews of SRs on interventions evaluate the effectiveness and safety of one or more interventions for disease prevention or treatment and primarily use data from SRs of RCTs, while only occasionally using data from observational studies for rare adverse events. For example, some umbrella reviews on the safety of interventions can include SRs of RCTs and cohort or case‐control studies, examining them separately, using observational studies for events that are rare or require long follow‐up to be detected. Other SRs can focus on rates of medical conditions or markers of given conditions or outcomes, and in these cases, SRs of cross‐sectional studies are typically included.

Items to consider before conducting an umbrella review

Two documents reported the importance of determining the inclusion criteria for SRs a priori (Aromataris et al., 2015; Baker et al., 2014). The methodology and study selection should align with the primary objective of the umbrella review, and all the steps of the umbrella review need to account for the type of eligible studies (Aromataris et al., 2015; Baker et al., 2014). Given that umbrella reviews vary in quality and might at times merge results from different study designs, a plan on how to handle pooled estimates from diverse study designs, particularly whether and why observational studies assessing epidemiological associations should or should not be combined, should be reported.

Specifying the eligibility criteria for the umbrella review

Belbasis and colleagues (Belbasis et al., 2022) cautioned that SRs and MAs may have different eligibility criteria than those intended in the umbrella review. SRs might use similar definitions for different constructs. For instance, SRs might claim to focus on mental disorders but might have included reporting on subjects with mental health symptoms but no formal diagnosis (i.e. using the Diagnostic and Statistical Manual of Mental Disorders or International Classification of Diseases). Another example is that SRs might claim to include longitudinal studies, but also include cross‐sectional studies. In such cases, it is crucial to set strict and precise criteria to exclude SRs that report on studies that do not meet the inclusion criteria. Authors can alternatively examine each study captured by the SR/MA and re‐calculate the pooled estimates, focusing on evidence that is eligible for and applicable to the umbrella review (see section on collecting, analyzing, and presenting outcome data).

Conducting and reporting searches for systematic reviews

Fusar‐Poli and colleagues (Fusar‐Poli and Radua, 2018) suggested that the literature search process should strictly adhere to the PRISMA recommendations to ensure comprehensive and transparent reporting. In addition to following the PRISMA guidelines concerning searching, it is also important to incorporate specific guidelines tailored to the types of studies being reviewed. For instance, when dealing with observational studies, the Meta‐analysis of Observational Studies in Epidemiology (MOOSE) guidelines (Stroup et al., 2000) should be applied. These recommendations were formulated to guide reporting in SRs and MAs before the PRIOR checklist was introduced and indirectly called for specific reporting guidance for umbrella reviews of observational studies. There was no guidance specifically on conducting database or gray literature searches.

Collecting, analyzing, and presenting outcome data

Using various effect size measures for observational studies can be beneficial, as each measure is appropriate for specific study types. However, employing a common effect size across all factors allows for easier comparisons (Fusar‐Poli and Radua, 2018). To establish the temporality of association and minimize reverse causation, temporality should be addressed through a sensitivity analysis that includes only prospective studies (Fusar‐Poli and Radua, 2018). Gianfredi and colleagues discussed that MAs of observational studies are often affected by potential selection bias or lost‐to‐follow‐up bias, which can undermine sample representativeness (Gianfredi et al., 2022). Additionally, heterogeneity in confounding variables, small sample sizes, or inadequate power are potential concerns. Prioritizing adjusted estimates over unadjusted ones is important since different adjustments can lead to varying results and inferences. Where possible, standardization of outcomes, and their operationalization and reporting, may be beneficial.

Collecting and presenting data on the methodological quality (risk of bias) of primary studies contained within the included systematic reviews

When some SRs and MAs include risk‐of‐bias assessments using standardized tools, such as the JBI tool for observational studies (Moola et al., 2024) and the Risk Of Bias In Non‐randomized Studies ‐ of Exposures (ROBINS‐E) tool (Higgins et al., 2024), researchers are encouraged to summarize these assessments in a clear and organized tabular format (Belbasis et al., 2022). Often, the inherent heterogeneity in exposure and outcome assessment found in the original studies carries over into the MAs and subsequently into the umbrella reviews. This variability can pose significant challenges in drawing reliable conclusions. To address this issue effectively, it is recommended to establish and adhere to a strict definition of inclusion and exclusion criteria for the SRs and MAs included in the umbrella review (Gianfredi et al., 2022).

Assessing the methodological quality of the included systematic reviews and the certainty of estimates reported by the included systematic reviews

The certainty of evidence from SRs of observational studies should be assessed, ideally with quantitative criteria and with sensitivity analyses accounting for the temporality of putative risk or protective factors. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) tool (Guyatt et al., 2008) can be used to assess and report the certainty of evidence for each clinically important outcome. If possible, authors should extract and report GRADE assessments from included SRs, but they may need to conduct their own assessments if the included SRs lack comprehensive GRADE evaluations or use different tools. A robust set of quantitative criteria has been largely used across umbrella reviews of observational studies (Arango et al., 2021; Janiaud et al., 2021; Solmi et al., 2023). These criteria use information on the amount of evidence (e.g., >1000 cases or more than >20,000 cumulative participants for continuous outcomes), statistical strength of the evidence (e.g., p‐value, using cut‐offs of <10‐6 and <10‐3), consistency (e.g., not large between‐study heterogeneity, 95% predictive interval excluding the null), and no obvious bias in small‐study effect and excess significance tests. The clinical or public health interpretation of umbrella review findings should be discussed in the context of content expertise among the authors of the umbrella review.

DISCUSSION

This scoping review identified eight articles, which revealed that some guidance already exists for the conduct and reporting of umbrella reviews of observational studies, but it is scattered across different articles. There are still some gaps or uncertainties in the recommendations for various stages of conducting and/or reporting an umbrella review, namely: (1) choosing between conducting an umbrella review of observational studies and a new SR, (2) author team composition and roles, (3) target audience of the umbrella review of observational studies, (4) developing and registering an umbrella review of observational studies' protocol, (5) collecting and presenting data on descriptive characteristics of included SRs (and their primary studies), (6) consideration and management of overlapping SRs, (7) interpreting outcome data and drawing conclusions, and (8) updating the umbrella review. At the same time, aspects to account for specifically in umbrella reviews of observational studies have emerged that call for dedicated and specific reporting guidance for this study design. For instance, the variability of temporal sequences of exposure and outcomes across different study designs and the differences in quality and certainty assessments in SRs of observational studies compared to RCTs (Solmi et al., 2018a). Our findings are corroborated by a recent scoping review, which identified inconsistencies in the certainty of evidence assessments among umbrella reviews of observational studies published between 2010 and 2021 (Sadoyu et al., 2022). Our review of existing guidance documents on the conduct and/or reporting of umbrella reviews of observational studies sets the ground for Step 3 of the PRIUR‐CCC project, that is, the Delphi survey, which will include three rounds of expert participant surveys followed by response analysis. These results will inform the creation of the final PRIUR‐CCC statement, which will include formal reporting guidelines for umbrella reviews of observational studies on epidemiological associations (Solmi et al., 2024).

We employed a transparent and rigorous approach to compile information from all available guidance documents applicable to umbrella reviews of observational studies on epidemiological associations. We adapted the search strategies used in the PRIOR scoping review on guidance for overviews of reviews of healthcare interventions (Gates et al., 2020), which included comprehensive database searches and manual searches of various sources. Notably, the guidance summarized in our review may not be directly applicable to other types of umbrella reviews, such as those focusing on diagnostic accuracy, experimental studies, or qualitative research.

The title/abstract and full‐text screening of the records and the extraction of recommendations from them were conducted independently and in duplicate, serving as a strength of the study. The inclusion criterion of records regardless of language avoids potentially overlooking relevant information. Furthermore, we screened a wide range of literature, including journals and gray literature sources, to ensure a thorough review of the recommendations. However, while we explored both publicly available search results and those accessible through our university library systems and interlibrary loans, we acknowledge that there still exists a possibility that not all relevant recommendations for the conduct and reporting of umbrella reviews of observational studies were captured.

In conclusion, the growing popularity of umbrella reviews has resulted in the creation of reporting guidance on such reviews of healthcare interventions (Gates et al., 2022). In the case of umbrella reviews for observational studies on epidemiological associations, specifically, there is scattered guidance within the current literature, leaving several areas of uncertainty. The findings of this scoping review will contribute to the creation of an evidence‐based and consensus‐driven guideline, the PRIOR‐extension for Umbrella Reviews of Cross‐sectional, Case‐control, and Cohort studies (PRIUR‐CCC). The development of this reporting guideline adheres to established EQUATOR principles and builds upon the foundational PRIOR statement (Gates et al., 2022), ultimately providing a standardized framework to improve the quality and consistency of reporting.

AUTHOR CONTRIBUTIONS

Carl Zhou: Conceptualization; formal analysis; writing—original draft; writing—review and editing. Nicholas Fabiano: Data curation; formal analysis; writing—review and editing. Arnav Gupta: Data curation; formal analysis; writing—review and editing. Stanley Wong: Data curation; formal analysis; writing—review and editing. Kelly D. Cobey: Conceptualization; writing—review and editing. David Moher: Conceptualization; writing—review and editing. Sanam Ebrahimzadeh: Conceptualization; writing—review and editing. Jeremy Y. Ng: Conceptualization; writing—review and editing. Elena Dragioti: Conceptualization; writing—review and editing. Jae Il Shin: Conceptualization; writing—review and editing. Joaquim Radua: Conceptualization; writing—review and editing. Samuele Cortese: Conceptualization; writing—review and editing. Beverley Shea: Conceptualization; writing—review and editing. Nicola Veronese: Conceptualization; writing—review and editing. Lisa Hartling: Conceptualization; writing—review and editing. Michelle Pollock: Conceptualization; writing—review and editing. Stefania Papatheodorou: Conceptualization; writing—review and editing. John P. A. Ioannidis: Conceptualization; writing—review and editing. Marco Solmi: Conceptualization; project administration; supervision; writing—original draft; writing—review and editing.

CONFLICT OF INTEREST STATEMENT

Samuele Cortese has declared reimbursement for travel and accommodation expenses from the Association for Child and Adolescent Central Health (ACAMH) in relation to lectures delivered for ACAMH, the Canadian AADHD Alliance Resource, the British Association of Psychopharmacology, Healthcare Convention and CCM Group team for educational activity on ADHD, and has received honoraria from Medice. Marco Solmi received honoraria/has been a consultant for Angelini, AbbVie, Boehringer Ingelheim, Lundbeck, Otsuka.

ETHICAL CONSIDERATIONS

The broader PRIUR‐CCC project, for which this study is a part, has been approved by The Ottawa Health Science Network Research Ethics Board (20220639‐01H).

Supporting information

Supporting Information S1

JCV2-6-e70017-s001.docx (44.6KB, docx)

ACKNOWLEDGMENTS

Samuele Cortese, NIHR Research Professor (NIHR303122) is funded by the NIHR for this research project. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR, NHS or the UK Department of Health and Social Care. Samuele Cortese is also supported by NIHR grants NIHR203684, NIHR203035, NIHR130077, NIHR128472, RP‐PG‐0618‐20003 and by grant 101095568‐HORIZONHLTH‐ 2022‐DISEASE‐07‐03 from the European Research Executive Agency. Lisa Hartling is supported by a Canada Research Chair in Knowledge Synthesis and Translation. Open Access funding enabled and organized by Projekt DEAL.

DATA AVAILABILITY STATEMENT

The data used in this manuscript are directly reported in the manuscript itself or included as a Supporting Information S1.

REFERENCES

  1. Arango, C. , Dragioti, E. , Solmi, M. , Cortese, S. , Domschke, K. , Murray, R. M. , Jones, P. B. , Uher, R. , Carvalho, A. F. , Reichenberg, A. , Shin, J. I. , Andreassen, O. A. , Correll, C. U. , & Fusar‐Poli, P. (2021). Risk and protective factors for mental disorders beyond genetics: An evidence‐based atlas. World Psychiatry, 20(3), 417–436. 10.1002/wps.20894 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aromataris, E. , Fernandez, R. , Godfrey, C. M. , Holly, C. , Khalil, H. , & Tungpunkom, P. (2015). Summarizing systematic reviews: Methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence‐Based Healthcare, 13(3), 132–140. 10.1097/XEB.0000000000000055 [DOI] [PubMed] [Google Scholar]
  3. Baker, P. R. , Costello, J. T. , Dobbins, M. , & Waters, E. B. (2014). The benefits and challenges of conducting an overview of systematic reviews in public health: A focus on physical activity. Journal of Public Health, 36(3), 517–521. Epub 2014 Aug 1. 10.1093/pubmed/fdu050 [DOI] [PubMed] [Google Scholar]
  4. Belbasis, L. , Bellou, V. , & Ioannidis, J. P. A. (2022). Conducting umbrella reviews. BMJ Med, 1(1), e000071. 10.1136/bmjmed-2021-000071 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bougioukas, K. I. , Pamporis, K. , Vounzoulaki, E. , Karagiannis, T. , & Haidich, A. B. (2023). Types and associated methodologies of overviews of reviews in health care: A methodological study with published examples. Journal of Clinical Epidemiology, 153, 13–25. 10.1016/j.jclinepi.2022.11.003 [DOI] [PubMed] [Google Scholar]
  6. Covidence Systematic Review Software, Veritas Health Innovation . www.covidence.org
  7. Fusar‐Poli, P. , & Radua, J. (2018). Ten simple rules for conducting umbrella reviews. Evidence‐Based Mental Health, 21(3), 95–100. 10.1136/ebmental-2018-300014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Gates, M. , Gates, A. , Guitard, S. , Pollock, M. , & Hartling, L. (2020). Guidance for overviews of reviews continues to accumulate, but important challenges remain: A scoping review. Systematic Reviews, 9(1), 254. 10.1186/s13643-020-01509-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Gates, M. , Gates, A. , Pieper, D. , Fernandes, R. M. , Tricco, A. C. , Moher, D. , Brennan, S. E. , Li, T. , Pollock, M. , Lunny, C. , Sepúlveda, D. , McKenzie, J. E. , Scott, S. D. , Robinson, K. A. , Matthias, K. , Bougioukas, K. I. , Fusar‐Poli, P. , Whiting, P. , Moss, S. J. , & Hartling, L. (2022). Reporting guideline for overviews of reviews of healthcare interventions: Development of the PRIOR statement. BMJ, 378, e070849. 10.1136/bmj-2022-070849 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Gianfredi, V. , Nucci, D. , Amerio, A. , Signorelli, C. , Odone, A. , & Dinu, M. (2022). What can we expect from an umbrella review? Advances in Nutrition, 13(2), 684–685. 10.1093/advances/nmab150 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Guyatt, G. H. , Oxman, A. D. , Vist, G. E. , Kunz, R. , Falck‐Ytter, Y. , Alonso‐Coello, P. , & Schünemann, H. J. (2008). Grade: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 336(7650), 924–926. AD. 10.1136/bmj.39489.470347 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Hailes, H. P. , Yu, R. , Danese, A. , & Fazel, S. (2019). Long‐term outcomes of childhood sexual abuse: An umbrella review. The Lancet Psychiatry, 6(10), 830–839. 10.1016/S2215-0366(19)30286-X [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Higgins, J. P. T. , Morgan, R. L. , Rooney, A. A. , Taylor, K. W. , Thayer, K. A. , Silva, R. A. , Lemeris, C. , Akl, E. A. , Bateson, T. F. , Berkman, N. D. , Glenn, B. S. , Hróbjartsson, A. , LaKind, J. S. , McAleenan, A. , Meerpohl, J. J. , Nachman, R. M. , Obbagy, J. E. , O'Connor, A. , Radke, E. G , … Sterne, J. A. C. (2024). A tool to assess risk of bias in non‐randomized follow‐up studies of exposure effects (ROBINS‐E). Environment International, 186, 108602. 10.1016/j.envint.2024.108602 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Ioannidis, J. P. (2009). Integration of evidence from multiple meta‐analyses: A primer on umbrella reviews, treatment networks and multiple treatments meta‐analyses. Canadian Medical Association Journal, 181(8), 488–493. 10.1503/cmaj.081086 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Ioannidis, J. P. (2016). The mass production of redundant, misleading, and conflicted systematic reviews and meta‐analyses. The Milbank Quarterly, 94(3), 485–514. 10.1111/1468-0009.12210 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Ioannidis, J. P. , & Trikalinos, T. A. (2007). An exploratory test for an excess of significant findings. Clinical Trials, 4(3), 245–253. 10.1177/1740774507079441 [DOI] [PubMed] [Google Scholar]
  17. Janiaud, P. , Agarwal, A. , Tzoulaki, I. , Theodoratou, E. , Tsilidis, K. K. , Evangelou, E. , & Ioannidis, J. P. A. (2021). Validity of observational evidence on putative risk and protective factors: Appraisal of 3744 meta‐analyses on 57 topics. BMC Medicine, 19(1), 157. 10.1186/s12916-021-02020-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Moher, D. , Schulz, K. F. , Simera, I. , & Altman, D. G. (2010). Guidance for developers of health research reporting guidelines. PLoS Medicine, 7(2), e1000217. 10.1371/journal.pmed.1000217 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Moola, S. , Munn, Z. , Tufanaru, C. , Aromataris, E. , Sears, K. , Sfetcu, R. , Currie, M. , Lisy, K. , Qureshi, R. , Mattis, P. , & Mu, P. (2024). Systematic reviews of etiology and risk (2020). In Aromataris E., Lockwood C., Porritt K., Pilla B., & Jordan Z. (Eds.), JBI manual for evidence synthesis. JBI. 10.46658/JBIMES-24-06 [DOI] [Google Scholar]
  20. Page, M. J. , McKenzie, J. E. , Bossuyt, P. M. , Boutron, I. , Hoffmann, T. C. , Mulrow, C. D. , Shamseer, L. , Tetzlaff, J. M. , Akl, E. A. , Brennan, S. E. , Chou, R. , Glanville, J. , Grimshaw, J. M. , Hróbjartsson, A. , Lalu, M. M. , Li, T. , Loder, E. W. , Mayo‐Wilson, E. , McDonald, S , … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. 10.1136/bmj.n71 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Papatheodorou, S. (2019). Umbrella reviews: What they are and why we need them. European Journal of Epidemiology, 34(6), 543–546. 10.1007/s10654-019-00505-6 [DOI] [PubMed] [Google Scholar]
  22. Peters, M. D. J. , Godfrey, C. , McInerney, P. , Munn, Z. , Tricco, A. C. , & Khalil, H. (2024). Scoping reviews (2020). In Aromataris E., Lockwood C., Porritt K., Pilla B., & Jordan Z. (Eds.), JBI manual for evidence synthesis. JBI. 10.46658/JBIMES-24-09 [DOI] [Google Scholar]
  23. Pollock, M. , Fernandes, R. M. , Pieper, D. , Tricco, A. C. , Gates, M. , Gates, A. , & Hartling, L. (2019). Preferred reporting Items for overviews of reviews (PRIOR): A protocol for development of a reporting guideline for overviews of reviews of healthcare interventions. Systematic Reviews, 8(1), 335. 10.1186/s13643-019-1252-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Sadoyu, S. , Tanni, K. A. , Punrum, N. , Paengtrai, S. , Kategaew, W. , Promchit, N. , Lai, N. M. , Thakkinstian, A. , Ngorsuraches, S. , Bangpan, M. , Veettil, S. , & Chaiyakunapruk, N. (2022). Methodological approaches for assessing certainty of the evidence in umbrella reviews: A scoping review. PLoS One, 17(6), e0269009. 10.1371/journal.pone.0269009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Solmi, M. , Cobey, K. , Moher, D. , Ebrahimzadeh, S. , Dragioti, E. , Shin, J. I. , Radua, J. , Cortese, S. , Shea, B. , Veronese, N. , Hartling, L. , Pollock, M. , Egger, M. , Papatheodorou, S. , Ioannidis, J. P. , & Carvalho, A. F. (2024). Protocol for the development of a reporting guideline for umbrella reviews on epidemiological associations using cross‐sectional, case‐control and cohort studies: The preferred reporting Items for umbrella reviews of cross‐sectional, case‐control and cohort studies (PRIUR‐CCC). BMJ Open, 14(6), e071136. 10.1136/bmjopen-2022-071136 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Solmi, M. , Correll, C. U. , Carvalho, A. F. , & Ioannidis, J. P. A. (2018a). The role of meta‐analyses and umbrella reviews in assessing the harms of psychotropic medications: Beyond qualitative synthesis. Epidemiology and Psychiatric Sciences, 27(6), 537–542. 10.1017/S204579601800032X [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Solmi, M. , De Toffol, M. , Kim, J. Y. , Choi, M. J. , Stubbs, B. , Thompson, T. , Firth, J. , Miola, A. , Croatto, G. , Baggio, F. , Michelon, S. , Ballan, L. , Gerdle, B. , Monaco, F. , Simonato, P. , Scocco, P. , Ricca, V. , Castellini, G. , Fornaro, M , … Dragioti, E. (2023). Balancing risks and benefits of cannabis use: Umbrella review of meta‐analyses of randomised controlled trials and observational studies. BMJ, 382, e072348. 10.1136/bmj-2022-072348 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Solmi, M. , Köhler, C. A. , Stubbs, B. , Koyanagi, A. , Bortolato, B. , Monaco, F. , Vancampfort, D. , Machado, M. O. , Maes, M. , Tzoulaki, I. , Firth, J. , Ioannidis, J. P. A. , & Carvalho, A. F. (2018b). Environmental risk factors and nonpharmacological and nonsurgical interventions for obesity: An umbrella review of meta‐analyses of cohort studies and randomized controlled trials. European Journal of Clinical Investigation, 48(12), e12982. 10.1111/eci.12982 [DOI] [PubMed] [Google Scholar]
  29. Stroup, D. F. , Berlin, J. A. , Morton, S. C. , Olkin, I. , Williamson, G. D. , Rennie, D. , Moher, D. , Becker, B. J. , Sipe, T. A. , & Thacker, S. B. (2000). Meta‐analysis of observational studies in epidemiology: A proposal for reporting. Meta‐Analysis of observational studies in Epidemiology (MOOSE) group. JAMA, 283(15), 2008–2012. 10.1001/jama.283.15.2008 [DOI] [PubMed] [Google Scholar]
  30. Tricco, A. C. , Lillie, E. , Zarin, W. , O'Brien, K. K. , Colquhoun, H. , Levac, D. , Moher, D. , Peters, M. D. J. , Horsley, T. , Weeks, L. , Hempel, S. , Akl, E. A. , Chang, C. , McGowan, J. , Stewart, L. , Hartling, L. , Aldcroft, A. , Wilson, M. G. , Garritty, C , … Straus, S. E. (2018). PRISMA extension for scoping reviews (PRISMA‐ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–473. 10.7326/M18-0850 [DOI] [PubMed] [Google Scholar]
  31. von Elm, E. , Altman, D. G. , Egger, M. , Pocock, S. J. , Gøtzsche, P. C. , Vandenbroucke, J. P. , & STROBE Initiative (2007). Strengthening the reporting of observational studies in epidemiology (STROBE) statement: Guidelines for reporting observational studies. BMJ, 335(7624), 806–808. 10.1136/bmj.39335.541782.AD [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supporting Information S1

JCV2-6-e70017-s001.docx (44.6KB, docx)

Data Availability Statement

The data used in this manuscript are directly reported in the manuscript itself or included as a Supporting Information S1.

Articles from JCPP Advances are provided here courtesy of The Association for Child and Adolescent Mental Health and John Wiley & Sons Ltd

RESOURCES