Table 2.
Characteristics of stage I descriptive studies
| Steps in the conduct of an overview | ||||||
|---|---|---|---|---|---|---|
| Citation | Type of study | Summary description of the article | Purpose, objectives, scope | Eligibility criteria | Search methods | Data extraction |
|
Baker 2014 [29] The benefits and challenges of conducting an overview of systematic reviews in public health: a focus on physical activity. |
Article describing methods for overviews | • Describes the usefulness of overviews for decision makers and summarises some procedural steps to be undertaken • Provides a case study of an overview on public health interventions for increasing physical activity |
✓✓ | ✓✓ | ✓✓ | |
|
Becker 2008 [1] Overviews of reviews. |
Guidance for undertaking overviews | • Early guidance providing the structure and procedural steps for the production of an overview • Details the different purposes of an overview, providing examples and describes how to present findings through tables and figures |
✓✓ | ✓✓ | ✓✓ | ✓✓ |
|
Bolland 2014 [30] A case study of discordant overlapping meta-analyses: vitamin D supplements and fracture. |
Article describing methods for overviews | • Describes criteria for explaining differences in overlapping M-As with discordant conclusions • Builds on the guide to interpret discordant SRs proposed by Jadad 1997 • Suggests reporting items there are overlapping trials in M-As |
✓ | ✓✓ | ✓✓ | |
|
Caird 2015 [31] Mediating policy-relevant evidence at speed: are systematic reviews of systematic reviews a useful approach? |
Article describing methods for overviews | • Describes the methodological challenges in the production of overviews that mediate existing synthesised knowledge to policy makers • Describes the trade-offs between producing a rapid overview and its comprehensiveness and reliability |
✓✓ | ✓✓ | ✓✓ | ✓✓ |
|
Chen 2014 [2] Scientific hypotheses can be tested by comparing the effects of one treatment over many diseases in a systematic review. |
Study examining methods used in a cohort of overviews | • Identifies possible aims of an overview as being to detect unintended effects, improve the precision of effect estimates, or explore heterogeneity of effect across disease groups • Describes the value and pitfalls of synthesis of M-As using three case studies |
✓✓ | ✓✓ | ||
|
CMIMG 2012 [4] Review type and methodological considerations. |
Guidance for undertaking overviews | • Provides updated Cochrane guidance on the purpose and conduct of overviews • Builds on the Cochrane guidance for overviews by Becker 2008 • Describes the factors in the decision to conduct an overview vs. an SR |
✓✓ | ✓✓ | ✓✓ | ✓✓ |
|
Cooper 2012 [32] The overview of reviews: unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship. |
Article describing methods for overviews | • Describes steps in the conduct of an overview and methods to address challenges (for example dealing with overlap in primary studies) • Describes methods for second order meta-analysis |
✓✓ | ✓✓ | ✓✓ | ✓✓ |
|
Flodgren 2011 [33] Challenges facing reviewers preparing overviews of reviews.a |
Article describing methods for overviews | • Mentions the issue of missing or inadequately reported data • Mentions the challenges in summarising and evaluating large amounts of heterogeneous data |
✓ | |||
|
Foisy 2011 [34] Mixing with the ‘unclean’: Including non-Cochrane reviews alongside Cochrane reviews in overviews of reviews.a |
Article describing methods for overviews | • Describes some challenges inherent in the eligibility criteria process (defining AMSTAR scoring as inclusion criteria, inclusion of non-Cochrane reviews alongside Cochrane reviews) • Develops inclusion criteria to minimise overlap in primary studies |
✓✓ | ✓✓ | ||
|
Hartling 2012 [35] A descriptive analysis of overviews of reviews published between 2000 and 2011. |
Study examining methods used in a cohort of overviews | • Describes steps in the conduct of overviews and methods used • Describes methodological standards for SRs (MECIR) and their applicability to overviews • Describes PRISMA reporting standards and their applicability to overviews |
✓ | ✓✓ | ✓✓ | ✓✓ |
|
Hartling 2013 [36] Generating empirical evidence to support methods for overviews of reviews.a |
Study examining methods used in a cohort of overviews | • Mentions challenges relating to the eligibility criteria process in terms of SR quality, search dates, the strength of the evidence to include, etc. | ✓ | ✓ | ||
|
Hartling 2014 [37] Systematic reviews, overviews of reviews and comparative effectiveness reviews: a discussion of approaches to knowledge synthesis. |
Article describing methods for overviews | • Briefly defines overviews, mentions the purposes in conducting an overview, and discusses some methodological challenges | ✓ | |||
|
Ioannidis 2009 [38] Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. |
Article describing methods for overviews | • Defines umbrella reviews as a pre-step to network meta-analysis • Describes challenges of overviews and a checklist of pitfalls |
✓✓ | |||
|
James 2014 [39] Informing the methods for public health overview reviews: a descriptive analysis of Cochrane and non-Cochrane public health overviews.a |
Study examining methods used in a cohort of overviews | • Briefly describes several steps in the conduct of overviews including determining the eligibility criteria and search methods • Compares Cochrane and non-Cochrane reviews in terms of restrictions on inclusion criteria |
✓ | ✓ | ✓ | |
|
Joanna Briggs Institute (JBI) 2014 [40, 41] Methodology for JBI umbrella reviews. |
Guidance for undertaking overviews | • Provides guidance as to what methods should be used at which step in the conduct of an overview • Provides stylistic conventions for overviews to meet publication and reporting criteria for the JBI Database of Systematic Reviews and Implementation Reports |
✓✓ | ✓✓ | ✓✓ | ✓✓ |
|
Kovacs 2014 [42] Overviews should meet the methodological standards of systematic reviews. |
Commentary or editorial that discuss methods for overviews | • Mentions four methodological shortcomings of one overview on surgical interventions as a letter to the editor | ✓ | |||
|
Kramer 2009 [43] Preparing an overview of reviews: lessons learned.a |
Article describing methods for overviews | • Mentions the challenges encountered when the authors conducted three overviews including missing information when extracting data | ✓ | ✓ | ||
|
Li 2012 [44] Quality and transparency of overviews of systematic reviews. |
Article describing methods for overviews | • Presents a pilot reporting/quality checklist • Evaluates a cohort of overviews using the pilot tool, with the mean number of items but no details of the items |
✓ | ✓ | ✓ | ✓ |
|
Pieper 2012 [6, 45] Overviews of reviews often have limited rigor: a systematic review. |
Study examining methods used in a cohort of overviews | • Describes the methods used in a cohort of overviews • Recommends using validated search filters for retrieval of SRs • Discusses whether to update the overview by including primary studies published after the most recent SR |
✓ | ✓✓ | ✓✓ | ✓ |
|
Pieper 2014 [46] Methodological approaches in conducting overviews: current state in HTA agencies. |
Article describing methods for overviews | • Describes the methods recommended in 8 HTA guideline documents related to overviews • Compares the Cochrane Handbook guidance to guidance produced by HTA agencies |
✓ | ✓ | ||
|
Pieper 2014 [47] Up-to-dateness of reviews is often neglected in overviews: a systematic review. |
Study examining methods used in a cohort of overviews | • Describes the process of searching for primary studies in an overview • Presents decision rules for when to search for primary studies • Outlines search methods in terms of sequential versus parallel searching for SRs and primary studies |
✓✓ | |||
|
Robinson 2016 [48–52] Integrating bodies of evidence: existing systematic reviews and primary studies. |
Article describing methods for overviews | • Describes the steps to undertake a complex review that includes multiple SRs, which is similar to overviews • Discusses challenges inherent in the production of complex reviews that include SRs |
✓✓ | ✓✓ | ✓✓ | ✓✓ |
|
Ryan 2005 [53, 54] Building blocks for meta-synthesis: data integration tables for summarising, mapping, and synthesising evidence on interventions for communicating with health consumers. |
Article describing methods for overviews | • Presents tabular methods to deal with the preparation of overview evidence • Discusses the data extraction process and organisation of data • Presents a table of taxonomy of outcomes from the included SRs, and a data extraction table based on this taxonomy |
✓✓ | ✓✓ | ||
|
Salanti 2011 [3] Evolution of Cochrane intervention reviews and overviews of reviews to better accommodate comparisons among multiple interventions. |
Guidance for undertaking overviews | • Provides Cochrane guidance on the definition of an overviews and as compared to SRs • Suggests broadening the search in an overview to include individual studies • Suggests missing data should be retrieved from original reports |
✓✓ | ✓✓ | ✓✓ | ✓ |
|
Silva 2015 [55] Overview of systematic reviews - a new type of study. |
Study examining methods used in a cohort of overviews | • Examines a cohort of Cochrane reviews for methods used • Documented the sources and types of search strategies conducted |
✓✓ | |||
|
Singh 2012 [56] Development of the Metareview Assessment of Reporting Quality (MARQ) Checklist. |
Article describing methods for overviews | • Presents a pilot reporting/quality checklist • Evaluates four case studies using the pilot tool, with the mean number of items but no details of the items |
✓✓ | ✓✓ | ||
|
Smith 2011 [57] Methodology in conducting a systematic review of systematic reviews of healthcare interventions. |
Article describing methods for overviews | • Describes some steps and challenges in undertaking an overview, namely search methods, study selection, quality assessment, and presentation of results • Presents tabular methods for the preparation of an overview |
✓✓ | ✓✓ | ✓✓ | ✓ |
|
Thomson 2010 [58] The evolution of a new publication type: Steps and challenges of producing overviews of reviews. |
Article describing methods for overviews | • Describes some steps in undertaking an overview and the challenges inherent in production of overviews • Discusses that gaps or lack of currency in included evidence will weaken the overview findings |
✓✓ | ✓✓ | ||
|
Thomson 2013 [59] Overview of reviews in child health: evidence synthesis and the knowledge base for a specific population. |
Study examining methods used in a cohort of overviews | • Describes the process of including trials in overviews ▪ Discusses the challenge of overview topics differing from the topics of the included SRs ▪ Provides potential solutions as to what to do when mixed populations are reported in SRs and how to extract age subgroup data |
✓✓ | ✓✓ | ✓✓ | |
AMSTAR A MeaSurement Tool to Assess systematic Reviews, CMIMG Comparing Multiple Interventions Methods Group, JBI Joanna Briggs Institute, PRISMA Preferred Reporting Items for Systematic reviews and Meta-Analyses, HTA health technology assessment, MECIR Methodological Expectations of Cochrane Intervention Reviews, SR systematic review, M-As meta-analyses
aIndicates a poster presentation
✓✓ Indicates a study describing one or more methods
✓ Indicates a study mentioning one or more methods