To the Editor:
Well-conducted systematic reviews and meta-analyses (SRMAs) can serve as a basis for formulating clinical practice guidelines [1–3], identifying areas where there is insufficient evidence to answer a research or clinical question [2], and minimizing duplication of effort (“research waste”) [3–5]. To ensure that existing systematic reviews in eyes and vision are identifiable and accessible, we constructed a database of SRMAs to inform decision-making and to identify gaps in eye and vision research.
Methods
We searched PubMed/MEDLINE, Embase, and The Cochrane Library (search strategies found in Appendix) annually for SRMAs that met the following eligibility criteria: (1) the publication reported on at least one eye/vision condition and (2) the publication described one or more systematic reviews and/or meta-analyses. Research assistants worked independently, in pairs, to screen titles/abstracts and review full texts for records which were labeled as an SRMA or met the National Academy of Medicine’s definition of an SRMA [1]. We classified the reviews by condition and summarized bibliographic characteristics.
To support the American Academy of Ophthalmology’s updating of Preferred Practice Patterns (methods reported elsewhere), we assessed the reliability of SRMAs that had addressed the effectiveness of therapeutic and preventive interventions for cataract, corneal diseases, glaucoma, refractive error, and retinal diseases [2–4].
Results
We identified 19,964 potentially relevant records as of July 2020; among them, 4786 met our inclusion criteria (Supplementary Fig.). Most (4692/4786, 98%) were published after 2000 (Table 1). Among all journals, The Cochrane Library contributed more systematic reviews than any other journal (325/4786, 7%) (Table 1). The conditions with the most SRMAs were glaucoma (639/4786, 13%), diabetic retinopathy (486/4786, 10%), and age-related macular degeneration (427/4786, 9%). Approximately 30% of all SRMAs in the database are for retinal and vitreous diseases if considering all retinal sub-conditions. Of 692 reviews assessed for reliability (full results reported elsewhere), the proportion of reliable systematic reviews varied by condition from 28% (corneal diseases) to 66% (refractive error) (Table 1).
Table 1.
Characteristics of records included in the database of eyes and vision systematic reviews, N = 4786 (as of July 2020).
| Characteristics | Number of reviews (n, %) | |
|---|---|---|
| Overall database | N = 4786 | |
| Year published | ||
| ≤1990 | 3 | (0.1%) |
| 1991–1995 | 17 | (0.4%) |
| 1996–2000 | 74 | (1.6%) |
| 2001–2005 | 206 | (4.3%) |
| 2006–2010 | 565 | (11.8%) |
| 2011–2015 | 1511 | (31.6%) |
| 2016–July 6, 2020 (n = 2410) | ||
| 2016 | 436 | (9.1%) |
| 2017 | 558 | (11.7%) |
| 2018 | 508 | (10.6%) |
| 2019 | 566 | (11.8%) |
| 2020 (July 6, 2020) | 342 | (7.2%) |
| Location of publication | ||
| Cochrane Database of Systematic Reviews | 325 | (6.8%) |
| Agency Reportsa | 142 | (2.9%) |
| Journals (n = 4319) | ||
| Ophthalmology | 173 | (3.6%) |
| PLoS ONE | 154 | (3.2%) |
| Investigative Ophthalmology and Visual Science | 122 | (2.6%) |
| British Journal of Ophthalmology | 120 | (2.5%) |
| Acta Ophthalmalogica | 96 | (2.0%) |
| BMC Ophthalmology | 79 | (1.7%) |
| International Journal of Ophthalmology | 70 | (1.5%) |
| Eye | 61 | (1.3%) |
| Retina | 58 | (1.2%) |
| Other journalsb | 3386 | (70.7%) |
| Condition or topicc | ||
| Adverse effects of intervention | 359 | (7.5%) |
| Age-related macular degeneration | 427 | (8.9%) |
| Blood vessel occlusion | 105 | (2.2%) |
| Cataract | 414 | (8.7%) |
| Conjunctivitis | 113 | (2.4%) |
| Corneal problems | 354 | (7.4%) |
| Diabetic retinopathy | 486 | (10.2%) |
| Dry eye | 154 | (3.2%) |
| Eyelid problems (including surrounding structures) | 83 | (1.7%) |
| Glaucoma | 639 | (13.4%) |
| Graves’ ophthalmopathy | 67 | (1.4%) |
| Infection/inflammation (including allergy) | 328 | (6.9%) |
| Iris problems | 9 | (0.2%) |
| Neuro-ophthalmologic problems | 315 | (6.6%) |
| Ocular cancer/tumor | 109 | (2.3%) |
| Ocular surgery | 422 | (8.8%) |
| Ocular trauma | 84 | (1.8%) |
| Oculoplastics | 93 | (1.9%) |
| Orbital problems | 132 | (2.8%) |
| Patient-reported outcomes | 101 | (2.0%) |
| Refractive errors/Refractive surgery | 303 | (6.3%) |
| Other retinal problemsd | 543 | (11.3%) |
| Retinoblastoma | 42 | (0.9%) |
| Retinopathy of prematurity | 162 | (3.4%) |
| Strabismus/Amblyopia | 109 | (2.3%) |
| Trachoma | 27 | (0.6%) |
| Uveal melanoma | 33 | (0.7%) |
| Uveitis | 154 | (3.2%) |
| Vision disorders/deficits | 228 | (4.8%) |
| Vision impairments/blindness | 366 | (7.6%) |
| Vision rehabilitation | 94 | (2.0%) |
| Vision and eye disease screening | 128 | (2.7%) |
| Reviews assessed for reliability | N = 692 | |
| Reliabilitye | ||
| SRMAs on interventions for Cataract (n = 99) | 46 | (46.5%) |
| SRMAs on interventions for Corneal diseases (n = 98) | 65 | (66.3%) |
| SRMAs on interventions for Glaucoma (n = 129) | 49 | (38.0%) |
| SRMAs on interventions for Refractive error (n = 39) | 11 | (28.2%) |
| SRMAs on interventions for Retinal diseases (n = 327) | 131 | (40.1%) |
aEight unique Health Technology Assessment publications merged as “Agency Reports”—Health Technology Assessment Database, Agency for Healthcare Research and Quality, US Preventive Services Task Force, Canadian Agency for Drugs and Technologies in Health, Ontario Health Technology Assessment Series, Evidence Report/Technology Assessment, Health Technology Assessment (Winchester, England), VA Evidence-based Synthesis Program Reports.
b“Other journals” include individual journals that had published fewer than 58 systematic reviews (i.e., less than the top 10 including the Cochrane Database of Systematic Reviews).
cReviews can be tagged with multiple conditions.
dExcluding reviews already tagged as related to age-related macular degeneration, diabetic retinopathy, retinoblastoma, or retinopathy of prematurity.
eA reliable SRMA meets five characteristics: (1) defined eligibility criteria for study selection, (2) clear description of the details of the comprehensive search of the literature conducted to identify potentially eligible studies, (3) assessment of risk of bias in included studies, (4) appropriate methods for any meta-analysis performed, and (5) conclusions that are consistent with review findings.
Discussion
To our knowledge, our database is the first database of SRMAs for eyes and vision. Our database provides a comprehensive view of the landscape of eye and vision primary research, secondary analyses, meta-analyses, and an “overview” of reviews. Using our database: investigators can support the scientific premise underlying their research questions or identify “research gaps” that need further investigation; sponsors and reviewers of applications for research funding can evaluate the novelty and significance of proposals; journal editors and peer-reviewers can gauge the scientific value of the research reported in newly submitted manuscripts; guideline developers can identify evidence to support recommendations; and patients can use the plain language summaries available in some SRMAs to improve their understanding of eye conditions [5].
Members of the public can request access to view the database through the Cochrane Eyes and Vision Website (https://eyes.cochrane.org/resources/cev-database-systematic-reviews-eyes-and-vision). Users should note that we included SRMAs regardless of their potential reliability; we have published criteria that users of our database can use to assess the reliability of SRMAs [2–4]. We also are committed to updating the database annually as long as resources permit.
Supplementary information
Supplemental Figure - Study selection flow diagram
Acknowledgements
We thank students and staff in the Cochrane Eyes and Vision US Satellite, as well as members of the European Glaucoma Society, for their assistance with identifying systematic reviews and classifying review conditions. We thank Dr. Kay Dickersin for her critical input on this project.
Funding
This project was supported by the National Eye Institute, National Institutes of Health (UG1EY020522, PI: TL). BSH acknowledges support from Research to Prevent Blindness (unrestricted funds to Wilmer Eye Institute; New York, NY, USA). The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.
Author contributions
Study concept and design: all authors. Acquisition and analysis of data: all authors. Interpretation of data: TL, JTL, and RQ. Drafting of the manuscript: JTL and RQ. Critical revision of the manuscript for important intellectual content: all authors. Final approval of the version to be published: all authors. Study supervision: TL.
Compliance with ethical standards
Conflict of interest
JTL, RQ, BR, CT, LR, KL, BSH, and TL were affiliated with Cochrane Eyes and Vision US Project during conduct of the work related to this article. Some of the systematic reviews examined in this article were produced by Cochrane Eyes and Vision. This article was prepared when KL, BR, and JTL were research methodologists at the Johns Hopkins Bloomberg School of Public Health. The opinions expressed in this article are the authors’ own and do not reflect the view of the National Institutes of Health, the Department of Health and Human Services, or the United States government.
Footnotes
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
These authors contributed equally: Jimmy T. Lê, Riaz Qureshi
Supplementary information
The online version contains supplementary material available at 10.1038/s41433-021-01514-0.
References
- 1.Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington, DC: National Academies Press; 2011. [PubMed]
- 2.Li T, Vedula SS, Scherer R, Dickersin K. What comparative effectiveness research is needed? A framework for using guidelines and systematic reviews to identify evidence gaps and research priorities. Ann Intern Med. 2012;156:367–77. doi: 10.7326/0003-4819-156-5-201203060-00009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Le JT, Qureshi R, Twose C, Rosman L, Han G, Fapohunda K, et al. Evaluation of systematic reviews of interventions for retina and vitreous conditions. JAMA Ophthalmol. 2019;137:1399–405. doi: 10.1001/jamaophthalmol.2019.4016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Qureshi R, Azuara-Blanco A, Michelessi M, Virgili G, Barbosa Breda J, Cutolo CA, et al. What do we really know about the effectiveness of glaucoma interventions: An overview of systematic reviews. Ophthalmol Glaucoma. 2021. [In Press]. [DOI] [PMC free article] [PubMed]
- 5.Pepose JS, Foulks GN, Nelson JD, Erickson S, Lemp MA. Perspective on systematic medical literature reviews and meta-analyses. Am J Ophthalmol. 2020;211:15–21. doi: 10.1016/j.ajo.2019.11.028. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Figure - Study selection flow diagram