Key Points
Question
What is the reliability of the existing systematic reviews addressing interventions for corneal diseases?
Findings
This study identified 98 systematic reviews (33 classified as unreliable and 65 as reliable) addressing 15 corneal diseases. The most frequent reasons for unreliability were that searches were not comprehensive, risk of bias was not assessed, and, when a quantitative synthesis was conducted, inappropriate methods were used.
Meaning
Adherence to well-established best practices regarding systematic review conduct might help make future systematic reviews in eyes and vision more reliable.
This summary of published systematic reviews summarizes their reliability in addressing treatment interventions for corneal diseases.
Abstract
Importance
Patient care should be informed by clinical practice guidelines, which in turn should be informed by evidence from reliable systematic reviews. The American Academy of Ophthalmology is updating its Preferred Practice Patterns (PPPs) for the management of the following 6 corneal diseases: bacterial keratitis, blepharitis, conjunctivitis, corneal ectasia, corneal edema and opacification, and dry eye syndrome.
Objective
To summarize the reliability of the existing systematic reviews addressing interventions for corneal diseases.
Data Source
The Cochrane Eyes and Vision US Satellite database.
Study Selection
In this study of published systematic reviews from 1997 to 2017 (median, 2014), the Cochrane Eyes and Vision US Satellite database was searched for systematic reviews evaluating interventions for the management of any corneal disease, combining eyes and vision keywords and controlled vocabulary terms with a validated search filter.
Data Extraction and Synthesis
The study classified systematic reviews as reliable when each of the following 5 criteria were met: the systematic review specified eligibility criteria for inclusion of studies, conducted a comprehensive literature search for studies, assessed risk of bias of the individual included studies, used appropriate methods for quantitative syntheses (meta-analysis) (only assessed if meta-analysis was performed), and had conclusions that were supported by the results of the systematic review. They were classified as unreliable if at least 1 criterion was not met.
Main Outcomes and Measures
The proportion of systematic reviews that were reliable and the reasons for unreliability.
Results
This study identified 98 systematic reviews that addressed interventions for 15 corneal diseases. Thirty-three of 98 systematic reviews (34%) were classified as unreliable. The most frequent reasons for unreliability were that the systematic review did not conduct a comprehensive literature search for studies (22 of 33 [67%]), did not assess risk of bias of the individual included studies (13 of 33 [39%]), and did not use appropriate methods for quantitative syntheses (meta-analysis) (12 of 17 systematic reviews that conducted a quantitative synthesis [71%]). Sixty-five of 98 systematic reviews (66%) were classified as reliable. Forty-two of the 65 reliable systematic reviews (65%) addressed corneal diseases relevant to the 2018 American Academy of Ophthalmology PPPs; 33 of these 42 systematic reviews (79%) are cited in the 2018 PPPs.
Conclusions and Relevance
One in 3 systematic reviews addressing interventions for corneal diseases are unreliable and thus were not used to inform PPP recommendations. Careful adherence by systematic reviewers and journal editors to well-established best practices regarding systematic review conduct and reporting might help make future systematic reviews in eyes and vision more reliable.
Introduction
The American Academy of Ophthalmology’s (AAO’s) clinical practice guidelines (ie, Preferred Practice Patterns [PPPs]) are “designed to identify characteristics and components of quality eye care.”1(p1) The Institute of Medicine (now called the National Academy of Medicine) explicitly recommended that quality care should be based on guideline recommendations, which in turn should be based on systematic reviews that are of high quality and reliable.2 To inform decision making, a few minimal attributes distinguish reliable systematic reviews from unreliable systematic reviews.3 Reliable systematic reviews (1) use appropriate methods to search for all available evidence, assess the risk of bias in the included evidence, and qualitatively and quantitatively synthesize it in ways that minimize bias; and (2) are reported completely and transparently.2,4
Since 2014, the AAO has partnered with the Cochrane Eyes and Vision US Satellite (CEV@US) to identify reliable systematic reviews that are relevant to updates of the PPPs. Specifically, CEV@US identifies and supplies each PPP panel with relevant reliable systematic reviews that can inform recommendations on the effectiveness of interventions. The 2016 cataract PPP5 and the 2017 refractive error PPP4 are recent examples of the success of this partnership.
Corneal disease is the fourth leading cause of blindness, accounting for approximately 5% of cases of blindness globally.6 To assist the cornea PPP panel in updating their 2018 PPPs, CEV@US identified, assessed, and summarized potentially reliable systematic reviews addressing interventions for the 6 corneal diseases covered in the PPP, including bacterial keratitis, blepharitis, conjunctivitis, corneal ectasia, corneal edema (ie, retention of excess fluid within 1 or multiple corneal layers7) and opacification (ie, presence of additional material [eg, fluid, scar tissue] within 1 or multiple layers of the area that is associated with loss of corneal clarity7), and dry eye syndrome. Our objective in this article is to summarize the reliability of the existing systematic reviews addressing the management of any corneal disease (including the 6 covered in the 2018 PPPs).
Methods
Source of Systematic Reviews
To support our collaboration with the AAO, CEV@US has been maintaining, since 2007, a regularly updated database of systematic reviews related to eyes and vision. This database includes both Cochrane and non-Cochrane systematic reviews. To identify the systematic reviews,8 CEV@US searched MEDLINE and Embase from inception to 2007, with periodic updates conducted in 2009, 2012, 2014, 2016, and 2017. To identify systematic reviews, the search combined eyes and vision keywords and controlled vocabulary terms with a validated search filter.9 Full-text reports that claimed to be systematic reviews or meta-analyses or met the definition of a systematic review or a meta-analysis (as defined by the Institute of Medicine2) were included in the database.
Eligible Systematic Reviews
For the present project, we searched the CEV@US database of systematic reviews on July 14, 2017, and included systematic reviews that evaluated the effectiveness of interventions for the management of at least 1 corneal disease. When a systematic review had been updated since its initial publication, we included its most recent update. Two individuals (from among 2 authors [I.J.S. and K.B.L.], 2 nonauthor research assistants [Omar Mansour, MHS, and Benjamin Rouse, MHS], and Barbara Hawkins, PhD [a senior faculty member of CEV@US]) independently examined each systematic review to determine whether it addressed corneal disease. For systematic reviews that examined corneal diseases, they classified the specific disease addressed. Dr Hawkins confirmed all classifications.
Data Extraction
We adapted a data extraction form used by our team in previous studies.4,5,8,10,11 The form contained 43 items derived from the Critical Appraisal Skills Programme (CASP),12 the Assessment of Multiple Systematic Reviews (AMSTAR),13 and the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA).14 We extracted the following information pertaining to each systematic review: eligibility criteria for including studies in the systematic review (ie, participants, interventions, comparisons, and outcomes of interest); methods used to search for potentially eligible studies; methods used to screen titles, abstracts, and full texts of identified studies for eligibility; whether and how the systematic review investigators assessed risk of bias in the included studies; how data were extracted from the included studies; methods used for quantitative synthesis of the results of the included studies (meta-analysis), when conducted; and whether the conclusions were supported by the results. We also extracted information about any reported financial support for the systematic review and financial relationships for any of the systematic review’s authors. We extracted all information into the Systematic Review Data Repository (SRDR).15,16
Classification of the Reliability of Systematic Reviews
Based on the information extracted, we classified each systematic review as either reliable or unreliable. Reliable systematic reviews met each of the following criteria: (1) the systematic review specified eligibility criteria for inclusion of studies in the systematic review, (2) the systematic review conducted a comprehensive literature search for studies, (3) the systematic review assessed risk of bias of the individual included studies, (4) the systematic review used appropriate methods for quantitative syntheses (meta-analysis) (only assessed if meta-analysis was performed), and (5) the systematic review had conclusions that were supported by the results of the systematic review. Table 1 lists the definitions that we used to apply each of these criteria. If at least 1 criterion was not met, we classified the systematic review as unreliable.
Table 1. Reasons Why Systematic Reviews Were Classified as Unreliable, Definitions of Reasons, and Examples From Unreliable Systematic Reviews.
Reason Number | Reason for Unreliability | No. (%) of Systematic Reviews Classified as Unreliable (n = 33) | Definition of Reason | Examples From Unreliable Systematic Reviews |
---|---|---|---|---|
1 | Did not specify eligibility criteria for inclusion of studies in the systematic review | 3 (9) | Described inclusion and/or exclusion criteria for eligible studies5 | “We performed a systematic review of English- and Portuguese-language articles, restricted to studies published from 2002 through 2015, with a total of 45 articles related to ocular manifestations in FAP and the therapeutic options”17(p2) |
2 | Did not conduct a comprehensive literature search for studies | 22 (67) | (1) Described an electronic search of ≥2 bibliographic databases, (2) used a search strategy comprising a mixture of keywords and controlled vocabulary terms, and (3) reported using ≥1 other method of searching, such as searching for conference abstracts, identifying ongoing studies, complementing electronic searching by hand searches (eg, checking reference lists), and contacting experts or authors of included studies5,18 | “Key words included rose bengal, artificial tears, conjunctiva and dry eye as well as the main ingredients of these types of products (e.g. hypromellose, carbomer and hyaluronic acid)”19(p574) |
3 | Did not assess risk of bias of the individual included studies | 13 (39) | Used any methods (eg, scales, checklists, and domain-based evaluations) to assess methodologic rigor of included studies5,18 | We classified as unreliable systematic reviews that did not mention a risk of bias/quality assessment of included studies in the abstract or text |
4 | Did not use appropriate methods for quantitative syntheses (meta-analysis) | 12 (71)a | Used quantitative methods that (1) were appropriate for the study design analyzed (eg, maintained the randomized nature of trials and used adjusted estimates from observational studies) and (2) correctly computed the weight for included studies5,18,20 | Meta-analyses were conducted in the context of an extremely high value of I2b (>90%)21 |
5 | Had conclusions that were not supported by the results of the systematic review | 0 | Drew conclusions that were consistent with systematic review findings, provided a balanced consideration of benefits and harms, and did not favor a specific intervention if there was lack of evidence5,10 | Not applicable |
Abbreviation: FAP, familial amyloid polyneuropathy.
The denominator is only those 17 systematic reviews that actually conducted a quantitative synthesis.
Two individuals (from among 2 authors [I.J.S. and K.B.L.] and 4 nonauthor research assistants [Jessica Gayleard, MS, Sueko Ng, MHS, Yuanxi Xia, MSc, and Jiajun Wen, MD]) independently extracted from each systematic review all information, including the criteria used to assess reliability. We adjudicated disagreements between extractors through discussion and, whenever agreement was not reached, through discussion with a third extractor.
Two points about our assessment of reliability are worthy of clarification. First, for each systematic review, we extracted information from the report of the completed systematic review; we did not examine the systematic review protocols for additional information. Second, we determined reliability based on the methods that the systematic review authors reported and not on the quality or quantity of the studies included in the systematic reviews. For example, a reliable systematic review may find no evidence at all or may find only low-quality evidence from nonrandomized studies. As such, the reliability of systematic reviews defined by our 5 criteria should not be confused with an assessment of quality of the body of evidence identified in the reviews.
Results
Included Systematic Reviews
As of July 14, 2017, we identified 98 systematic reviews related to interventions for the management of 15 corneal diseases in the CEV@US database (Figure 1). These systematic reviews were published between 1997 and 2017 (median, 2014) (Table 2).
Table 2. Systematic Reviews Addressing the Management of Corneal Diseases, by Reliability Status.
Variable | No. (%) | |
---|---|---|
Systematic Reviews Classified as Unreliable (n = 33) | Systematic Reviews Classified as Reliable (n = 65) | |
Year of publication | ||
1997-1999 | 1 (3) | 1 (2) |
2000-2002 | 0 | 0 |
2003-2005 | 1 (3) | 3 (5) |
2006-2008 | 0 | 4 (6) |
2009-2011 | 3 (9) | 7 (11) |
2012-2014 | 16 (48) | 24 (37) |
2015-2017 | 12 (36) | 26 (40) |
Diseases addressed in the systematic reviewa | ||
Bacterial keratitisb | 0 | 3 (5) |
Blepharitisb | 0 | 4 (6) |
Conjunctivitisb | 4 (12) | 21 (32) |
Corneal ectasiab | 3 (9) | 5 (8) |
Corneal edema and opacificationb | 1 (3) | 3 (5) |
Dry eye syndromeb | 12 (36) | 10 (15) |
Other (nonbacterial) keratitis | 1 (3) | 4 (6) |
Hordeolum | 0 | 2 (3) |
Pterygium | 2 (6) | 4 (6) |
Keratoconus | 5 (15) | 7 (11) |
Chalazion | 1 (3) | 0 (0) |
Blepharospasm | 1 (3) | 0 (0) |
Corneal injury | 3 (9) | 5 (8) |
Corneal abrasion | 0 | 5 (8) |
Patients undergoing keratoplasty, corneal transplant, or LASIK | 0 | 3 (5) |
Types of management addressed in the systematic reviewa | ||
Medication | 13 (39) | 44 (68) |
Device | 4 (12) | 9 (14) |
Surgery | 15 (45) | 18 (28) |
Other procedure | 1 (3) | 0 (0) |
Supplements, including artificial tears | 10 (30) | 9 (14) |
Acupuncture | 1 (3) | 2 (3) |
Other | 1 (3) | 2 (3) |
Types of study designs included in the systematic reviewa | ||
Randomized clinical trials | 22 (67) | 58 (89) |
Non–randomized clinical trials | 5 (15) | 6 (9) |
Cohort studies | 7 (21) | 6 (9) |
Case-control studies | 0 | 1 (2) |
Cross-sectional studies | 0 | 0 |
Case series or case reports | 5 (15) | 2 (3) |
Other | 1 (3) | 2 (3) |
Included a quantitative synthesis (ie, meta-analysis) | ||
Yes | 17 (52) | 41 (63) |
No | 16 (48) | 18 (28) |
Not applicable because the systematic review included <2 studies | 0 | 6 (9) |
Cochrane review | ||
Yes | 0 | 26 (40) |
No | 33 (100) | 39 (60) |
No. of authors | ||
1 | 2 (6) | 3 (5) |
2-4 | 18 (55) | 32 (49) |
5-7 | 9 (27) | 22 (34) |
>7 | 4 (12) | 8 (12) |
Funding sources for systematic reviewa | ||
Government | 6 (18) | 31 (48) |
Pharmaceutical/other industry | 4 (12) | 3 (5) |
Foundation | 4 (12) | 7 (11) |
Academic institution or department | 4 (12) | 6 (9) |
Other funding source | 0 | 1 (2) |
Explicitly stated as none | 5 (15) | 3 (5) |
Not reported | 15 (45) | 24 (37) |
Author financial relationshipsa | ||
Board membership | 2 (6) | 2 (3) |
Consultancy | 4 (12) | 5 (8) |
Employment | 2 (6) | 1 (2) |
Expert testimony | 0 | 0 |
Gifts | 0 | 0 |
Grants/grants pending | 4 (12) | 3 (5) |
Payment for lectures/speaking | 4 (12) | 2 (3) |
Payment for manuscript preparation | 0 | 0 |
Patents (planned, pending, or issued) | 0 | 0 |
Royalties | 3 (9) | 0 |
Presentations | 0 | 0 |
Stock/stock options | 1 (3) | 0 |
Travel/accommodations/meeting expenses | 0 | 0 |
Other | 0 | 2 (3) |
Explicitly stated as none | 15 (45) | 41 (63) |
Not reported | 13 (39) | 14 (22) |
More than 1 category could apply to a systematic review.
Addressed in most recent American Academy of Ophthalmology Preferred Practice Patterns (PPPs) on corneal disease.
Reasons for Classification of Some Systematic Reviews as Unreliable
We classified 33 of 98 systematic reviews (34%) as unreliable (Figure 2).17,19,21,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53 Among the unreliable systematic reviews, the reasons for unreliability (each systematic review could have more than 1 reason) were that the systematic review did not conduct a comprehensive literature search for studies (22 of 33 [67%]), the systematic review did not assess risk of bias in the individual included studies (13 of 33 [39%]), the systematic review did not use appropriate methods for quantitative syntheses (meta-analysis) (12 of 17 systematic reviews that conducted a quantitative synthesis [71%]), and the systematic review did not specify eligibility criteria for inclusion of studies in the systematic review (3 of 33 [9%]). Table 1 lists illustrative examples of each reason for unreliability. The percentage of systematic reviews that were unreliable was 25% (5 of 20 systematic reviews) before 2012 and 36% (28 of 78 systematic reviews) in 2012 onward.
Characteristics of Reliable Systematic Reviews
Table 2 lists the characteristics of the 98 included systematic reviews. The 65 of 98 systematic reviews (66%) classified as reliable54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118 (Figure 1) were published between 1997 and 2017 (median, 2014). The most frequent diseases addressed in these systematic reviews were conjunctivitis (21 of 65 systematic reviews [32%]) and dry eye syndrome (10 of 65 systematic reviews [15%]); both of these diseases were of interest to the AAO Cornea/External Disease PPP Panel. The other 4 diseases of interest to the panel (bacterial keratitis, blepharitis, corneal ectasia, and corneal edema and opacification) were addressed in 3 of 65 reliable systematic reviews (5%) for bacterial keratitis, 4 of 65 (6%) for blepharitis, 5 of 65 (8%) for corneal ectasia, and 3 of 65 (5%) for corneal edema and opacification.
Most of the 65 reliable systematic reviews evaluated medications (44 [68%]) or surgery (18 [28%]) (Table 2). The most frequent outcome domains assessed in the reliable systematic reviews were adverse events (n = 43), symptom improvement (n = 31), visual acuity (n = 19), quality of life (n = 19), costs (n = 14), symptom resolution (n = 10), and tear production (n = 10). Most of the reliable systematic reviews (64 of 65 [98%]) included experimental studies (ie, randomized clinical trials or non–randomized clinical trials). Nine reliable systematic reviews (14%) also included observational studies. Almost two-thirds (41 of 65 [63%]) of the reliable systematic reviews conducted quantitative syntheses.
Forty percent (26 of 65) of the reliable systematic reviews were Cochrane reviews (Table 2). Thirty-two reliable systematic reviews (49%) had 2 to 4 authors, and 22 (34%) had 5 to 7 authors. Almost one-half of the reliable systematic reviews (48% [31 of 65]) were funded by government sources; 37% (24 of 65) did not report a funding source. Almost two-thirds (63% [41 of 65]) of the reliable systematic reviews explicitly stated that none of the authors had any financial relationships relevant to the systematic review’s content. The eAppendix in the Supplement provides the objectives, diseases addressed, interventions and comparisons, outcomes, number of included studies, number of participants in included studies, and conclusions for all 65 reliable systematic reviews.
The 65 reliable systematic reviews included a median of 9 (interquartile range [IQR], 5-23) studies each; 3 systematic reviews were “empty” (ie, they did not include any studies) (Table 3). The 65 reliable systematic reviews analyzed data from a median of 556 (IQR, 227-1795) study participants each. The 3 empty systematic reviews did not include any study participants, and 13 of 65 systematic reviews (20%) did not report how many participants were included.
Table 3. Number of Included Studies and Number of Participants in Included Studies in Systematic Reviews Addressing the Management of Corneal Diseases, by Reliability Status.
Variable | Systematic Reviews Classified as Unreliable (n = 33) | Systematic Reviews Classified as Reliable (n = 65) |
---|---|---|
No. of included studies | ||
Median (IQR) [range] | 15 (7-32) [5-97] | 9 (5-23) [0-142] |
Not reported, No. (%) | 4 (12) | 0 |
No. of participants | ||
Median (IQR) [range] | 827 (538-3829) [118-29 160] | 556 (227-1795) [0-13 631] |
Not reported, No. (%) | 7 (21) | 13 (20) |
Abbreviation: IQR, interquartile range.
Comparison of Reliable and Unreliable Systematic Reviews
There were some notable differences between systematic reviews that were reliable and those that were not (Table 2). For example, while dry eye syndrome was the most frequent disease addressed in the unreliable systematic reviews (12 of 33 [36%]), conjunctivitis was the most frequent disease addressed in the reliable systematic reviews (21 of 65 [32%]). While surgery was the most frequent type of management addressed in the unreliable systematic reviews (15 of 33 [45%]), medication was the most frequent type of management addressed in the reliable systematic reviews (44 of 65 [68%]). Systematic reviews that were unreliable included higher median numbers of studies (15 vs 9) and study participants (827 vs 556) than reliable systematic reviews (Table 3), but somewhat less frequently included quantitative syntheses (52% [17 of 33] vs 63% [41 of 65]) (Table 2).
No unreliable systematic reviews were Cochrane reviews, but 40% (26 of 65) of the reliable systematic reviews were Cochrane reviews (Table 2). Larger proportions of reliable systematic reviews than unreliable systematic reviews were funded by government sources (48% [31 of 65] vs 18% [6 of 33]) or reported explicitly that the authors had no relevant financial relationships (63% [41 of 65] vs 45% [15 of 33]).
Inclusion of Reliable Systematic Reviews in the 2018 AAO Cornea/External Disease PPPs
Forty-two of the 65 reliable systematic reviews (65%) addressed corneal diseases being covered in the 2018 AAO Cornea/External Disease PPPs update. The AAO sent references for the 42 systematic reviews to the respective PPP panels, and 33 of 42 relevant reliable systematic reviews (79%) are cited in the 2018 PPPs. The reasons for noncitation of the other 9 systematic reviews were that the systematic reviews addressed interventions not being discussed in the PPP (4 systematic reviews68,106,108,115), addressed the setting of primary care (1 systematic review81), found an insufficient number of included studies (1 systematic review103), reported methodologic flaws in included studies (1 systematic review97), reported a lack of relevant outcomes in included studies (1 systematic review107), or reported inconsistent outcomes across included studies (1 systematic review104).
Discussion
Over the last decade, there has been a 3-fold surge in the number of systematic reviews on health topics.119 Within ophthalmology, when CEV@US started compiling its database of eyes and vision systematic reviews in 2007, there were 547 systematic reviews.10 In 2017, this number increased almost 7-fold, to 3777 (as reported herein). The present investigation reveals a cause for concern. It contributes to the evidence that only a fraction of systematic reviews in ophthalmology are deemed to be reliable (subspecialty-specific estimates range from 28% to 70%4,5,8). The late renowned statistician Professor Douglas Altman famously wrote that “We need less research, better research, and research done for the right reasons.”120(p284) As with other fields, it indeed is time for those involved in the ecosystem of evidence-based health care in ophthalmology to address the problem of an ever-growing number of poorly conceived and unreliable systematic reviews.
Main Problems With the Existing Systematic Reviews Addressing Corneal Disease
One-third (33 of 98) of the published systematic reviews addressing corneal diseases we assessed were classified as unreliable, constituting a waste of research resources and potentially misinformed health care decisions. A particularly concerning aspect is that the percentage of unreliable systematic reviews has not diminished, but rather has increased somewhat (from 25% [5 of 20] to 36% [28 of 78]) since 2012, the year after the landmark Institute of Medicine2 standards for systematic reviews were published. A large proportion of the unreliable systematic reviews (85% [28 of 33]) were published since 2012. The most frequent reasons why systematic reviews addressing the management of corneal diseases were judged to be unreliable were that the systematic review did not conduct a comprehensive literature search for other studies, did not assess risk of bias in the individual included studies, and did not use appropriate methods for quantitative syntheses (meta-analysis). Two-thirds (22 of 33) of the unreliable systematic reviews did not describe a comprehensive literature search. A comprehensive and reproducible literature search is a key tenet of what makes a systematic review systematic. The search should target (1) published studies using both free text and Medical Subject Headings (MeSH) in more than 1 electronic database (eg, MEDLINE and Embase) and (2) unpublished studies and ongoing studies (eg, ClinicalTrials.gov).18 Failure to search multiple databases or to describe the search terms, Boolean operators, restrictions/filters, and search dates compromises the reproducibility of the search. Systematic reviewers should follow the Institute of Medicine standards and engage an experienced information specialist when designing and documenting their searches.
More than one-third (39% [13 of 33]) of the unreliable systematic reviews did not provide a risk of bias assessment. Assessing risk of bias is crucial because it informs the extent to which the results of included studies are valid.18 When systematic reviewers do not provide risk of bias assessments, decision makers are unsure about the dependability of the systematic review’s findings. It should be noted that the field has moved away from assigning quality scores for studies to domain-based assessments of risk of bias. Systematic reviewers should be trained in state-of-the-art methods of assessing risk of bias, such as the revised Cochrane risk-of-bias tool for randomized trials (RoB 2.0).121
Approximately 70% (12 of 17) of the unreliable systematic reviews that conducted a quantitative synthesis did so using inappropriate methods. When a quantitative synthesis is conducted using inappropriate methods, such as in the context of substantial statistical heterogeneity of results, a synthesis of available data could be inaccurate and thus misleading to decision makers.
Another problem in the systematic reviews we examined was that many (almost one-half [15 of 33] of the unreliable systematic reviews and about one-third [24 of 65] of the reliable systematic reviews) did not report funding sources. To promote transparency, sources of support need to be reported in systematic reviews.13,14 Within ophthalmology, there has been increasing recognition of the influence of author conflicts of interest, either in the form of financial relationships or intellectual beliefs.122,123 Conflicts of interest, financial or intellectual, can alter the interpretation of identified studies, potentially leading to bias.124 Another factor that was inadequately reported was the number of participants in the studies included in the systematic reviews; approximately 1 in 5 (20 of 98) systematic reviews did not report this information.
Our finding that a sizable proportion of systematic reviews addressing corneal diseases (33% [33 of 98]) are unreliable is in keeping with other investigations that have found similar problems with systematic reviews of eye diseases. Based on systematic reviews identified through the same database that we used, Mayo-Wilson et al4 and Lindsley et al8 found as many as 70% and 30% of systematic reviews addressing refractive error and age-related macular degeneration, respectively, to be unreliable. Similarly, Downie and colleagues124 assessed the methodologic rigor of 71 systematic reviews addressing age-related macular degeneration and found adequate adherence to a mean of only 5.8 of 11 AMSTAR items. Suboptimal conduct and reporting of systematic reviews might indicate that previous incorrect approaches to meta-analysis are simply being propagated in the field.125
Implications of the Findings for Guideline Developers
For guideline developers, trusting the information reported in unreliable systematic reviews can be dangerous because it can lead to recommendations and, consequently, health care decisions that are misinformed. However, disentangling reliable from unreliable systematic reviews requires skills and resources. The partnership between the AAO and CEV@US is an ophthalmology success story that has required leadership and commitment from both entities. Such a partnership has been a win-win collaboration: CEV@US has magnified the influence of its work, and the AAO has ensured that its guidelines (ie, PPPs) are informed by reliable systematic review evidence. This model should be emulated in other fields.
Implications of the Findings for Systematic Reviewers
Conducting a systematic review is expensive and time-consuming. According to a recent examination of the international Prospective Register of Systematic Reviews (PROSPERO), systematic reviews take a median of 66 (range, 6-186) weeks from registration to publication.126 When systematic reviews are unreliable, it amounts to a monumental waste of crucial research resources.127 Moreover, to help guard against inappropriate decisions regarding clinical management, clinicians, patients, guideline developers, and other decision makers should be made aware that not all systematic reviews are reliable. Articles like the present one may help spread such awareness, thus positively altering the influence of this and similar research. In addition, systematic reviewers should stay abreast of widely available standards for the conduct of systematic reviews.2,18
Implications of the Findings for Journal Editors
We agree with the recommendations by Mayo-Wilson and colleagues4 that journal editors should have a greater role by enforcing the following 4 author requirements/policies for systematic review submissions: (1) publication/provision of systematic review protocols, (2) correct completion of a PRISMA checklist, (3) clear justification for the systematic review (to avoid redundancy of systematic reviews), and (4) review of submissions by an expert in systematic reviews. Regarding this last recommended policy, the Cochrane Eyes and Vision Group now has 11 systematic review experts who serve as associate editors and handle systematic review submissions received by 11 major eyes and vision journals in the world (https://eyes.cochrane.org/associate-editors-eyes-and-vision-journals). By promoting best practices in systematic review development and reporting in articles in these journals, we are optimistic that the proportion of reliable systematic reviews in eyes and vision will improve over time.
Limitations
Our study has some limitations. One limitation is that we searched for systematic reviews using MEDLINE and Embase only. Although our search in these databases has been updated biennially, it is possible that we missed some systematic reviews indexed in only other databases, such as some not published in English. A second limitation is that our assessment of reliability involved judgment and reflected our affiliation with Cochrane Eyes and Vision.
Conclusions
Two-thirds (65 of 98) of the identified systematic reviews of treatments for corneal diseases are reliable. These reliable systematic reviews may be useful to clinicians, patients, guideline developers (eg, AAO PPP panel members), and other decision makers. Careful adherence by systematic reviewers to best practices (eg, the Institute of Medicine2 standards for systematic reviews) and by journal editors to recommendations regarding reporting14 and editorial review4 can help improve the reliability of future systematic reviews in eyes and vision.
References
- 1.American Academy of Ophthalmology About Preferred Practice Patterns (PPPs). https://www.aao.org/about-preferred-practice-patterns. Published 2018. Accessed February 9, 2019.
- 2.Institute of Medicine Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: National Academies Press; 2011. [PubMed] [Google Scholar]
- 3.Lavis JN. How can we support the use of systematic reviews in policymaking? PLoS Med. 2009;6(11):e1000141. doi: 10.1371/journal.pmed.1000141 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Mayo-Wilson E, Ng SM, Chuck RS, Li T. The quality of systematic reviews about interventions for refractive error can be improved: a review of systematic reviews. BMC Ophthalmol. 2017;17(1):164. doi: 10.1186/s12886-017-0561-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Golozar A, Chen Y, Lindsley K, et al. . Identification and description of reliable evidence for 2016 American Academy of Ophthalmology Preferred Practice Pattern Guidelines for cataract in the adult eye. JAMA Ophthalmol. 2018;136(5):514-523. doi: 10.1001/jamaophthalmol.2018.0786 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.World Health Organization Blindness and vision impairment. https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment. Published October 11, 2018. Accessed February 9, 2019.
- 7.Farid M, Rhee MK, Akpek EK, et al. ; American Academy of Ophthalmology Preferred Practice Pattern Cornea and External Disease Panel . Corneal edema and opacification Preferred Practice Pattern. Ophthalmology. 2019;126(1):216-P285. doi: 10.1016/j.ophtha.2018.10.022 [DOI] [PubMed] [Google Scholar]
- 8.Lindsley K, Li T, Ssemanda E, Virgili G, Dickersin K. Interventions for age-related macular degeneration: are practice guidelines based on systematic reviews? Ophthalmology. 2016;123(4):884-897. doi: 10.1016/j.ophtha.2015.12.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Montori VM, Wilczynski NL, Morgan D, Haynes RB; Hedges Team . Optimal search strategies for retrieving systematic reviews from Medline: analytical survey. BMJ. 2005;330(7482):68. doi: 10.1136/bmj.38336.804167.47 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Li T, Vedula SS, Scherer R, Dickersin K. What comparative effectiveness research is needed? a framework for using guidelines and systematic reviews to identify evidence gaps and research priorities. Ann Intern Med. 2012;156(5):367-377. doi: 10.7326/0003-4819-156-5-201203060-00009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Yu T, Li T, Lee KJ, Friedman DS, Dickersin K, Puhan MA. Setting priorities for comparative effectiveness research on management of primary angle closure: a survey of Asia-Pacific clinicians. J Glaucoma. 2015;24(5):348-355. doi: 10.1097/IJG.0b013e31829e5616 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Critical Appraisal Skills Programme (CASP) CASP checklists. http://www.casp-uk.net/casp-tools-checklists. Accessed February 9, 2019.
- 13.Shea BJ, Grimshaw JM, Wells GA, et al. . Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10. doi: 10.1186/1471-2288-7-10 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group . Preferred Reporting Items for Systematic Reviews and Meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006-1012. doi: 10.1016/j.jclinepi.2009.06.005 [DOI] [PubMed] [Google Scholar]
- 15.Ip S, Hadar N, Keefe S, et al. . A Web-based archive of systematic review data. Syst Rev. 2012;1:15. doi: 10.1186/2046-4053-1-15 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Li T, Vedula SS, Hadar N, Parkin C, Lau J, Dickersin K. Innovations in data collection, management, and archiving for systematic reviews. Ann Intern Med. 2015;162(4):287-294. doi: 10.7326/M14-1603 [DOI] [PubMed] [Google Scholar]
- 17.Martins AC, Rosa AM, Costa E, Tavares C, Quadrado MJ, Murta JN. Ocular manifestations and therapeutic options in patients with familial amyloid polyneuropathy: a systematic review. Biomed Res Int. 2015;2015:282405. doi: 10.1155/2015/282405 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. London, United Kingdom: Cochrane Collaboration. [Google Scholar]
- 19.Doughty MJ, Glavin S. Efficacy of different dry eye treatments with artificial tears or ocular lubricants: a systematic review. Ophthalmic Physiol Opt. 2009;29(6):573-583. doi: 10.1111/j.1475-1313.2009.00683.x [DOI] [PubMed] [Google Scholar]
- 20.Glenny AM, Altman DG, Song F, et al. ; International Stroke Trial Collaborative Group . Indirect comparisons of competing interventions. Health Technol Assess. 2005;9(26):1-134, iii-iv. doi: 10.3310/hta9260 [DOI] [PubMed] [Google Scholar]
- 21.Hong J, Bielory B, Rosenberg JL, Bielory L. Efficacy of intranasal corticosteroids for the ocular symptoms of allergic rhinitis: a systematic review. Allergy Asthma Proc. 2011;32(1):22-35. doi: 10.2500/aap.2011.32.3420 [DOI] [PubMed] [Google Scholar]
- 22.Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539-1558. doi: 10.1002/sim.1186 [DOI] [PubMed] [Google Scholar]
- 23.Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-560. doi: 10.1136/bmj.327.7414.557 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Ahmad S, Mathews PM, Lindsley K, et al. . Boston type 1 keratoprosthesis versus repeat donor keratoplasty for corneal graft failure: a systematic review and meta-analysis. Ophthalmology. 2016;123(1):165-177. doi: 10.1016/j.ophtha.2015.09.028 [DOI] [PubMed] [Google Scholar]
- 25.Alves M, Fonseca EC, Alves MF, et al. . Dry eye disease treatment: a systematic review of published trials and a critical appraisal of therapeutic strategies. Ocul Surf. 2013;11(3):181-192. doi: 10.1016/j.jtos.2013.02.002 [DOI] [PubMed] [Google Scholar]
- 26.Aycinena AR, Achiron A, Paul M, Burgansky-Eliash Z. Incision and curettage versus steroid injection for the treatment of chalazia: a meta-analysis. Ophthalmic Plast Reconstr Surg. 2016;32(3):220-224. doi: 10.1097/IOP.0000000000000483 [DOI] [PubMed] [Google Scholar]
- 27.Ba J, Wu Y, Li Y, Xu D, Zhu W, Yu JG. Updated meta-analysis of acupuncture for treating dry eye. Med Accupuncture. 2013;25(5):317-327. doi: 10.1089/acu.2013.0968 [DOI] [Google Scholar]
- 28.Balderson DE, Cai G, Fries MA, et al. . A systematic review and meta-analysis to compare the efficacy of acyclovir 3% ophthalmic ointment to idoxuridine in curing herpetic keratitis by day 7 of treatment. BMC Ophthalmol. 2015;15:42. doi: 10.1186/s12886-015-0022-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Chunyu T, Xiujun P, Zhengjun F, Xia Z, Feihu Z. Corneal collagen cross-linking in keratoconus: a systematic review and meta-analysis. Sci Rep. 2014;4:5652. doi: 10.1038/srep05652 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Craig JA, Mahon J, Yellowlees A, et al. . Epithelium-off photochemical corneal collagen cross-linkage using riboflavin and ultraviolet A for keratoconus and keratectasia: a systematic review and meta-analysis. Ocul Surf. 2014;12(3):202-214. doi: 10.1016/j.jtos.2014.05.002 [DOI] [PubMed] [Google Scholar]
- 31.Devillier P, Dreyfus JF, Demoly P, Calderón MA. A meta-analysis of sublingual allergen immunotherapy and pharmacotherapy in pollen-induced seasonal allergic rhinoconjunctivitis. BMC Med. 2014;12:71. doi: 10.1186/1741-7015-12-71 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Doughty MJ. Fluorescein-tear breakup time as an assessment of efficacy of tear replacement therapy in dry eye patients: a systematic review and meta-analysis. Ocul Surf. 2014;12(2):100-111. doi: 10.1016/j.jtos.2013.11.002 [DOI] [PubMed] [Google Scholar]
- 33.Feng Y, Yu JG, Shi J, Wang Q. Meta analysis of randomized controlled trial in the effect of hinge location on dry eye syndrome after LASIK. Chin J Exp Ophthalmol. 2012;30(9):847-852. [Google Scholar]
- 34.Godefrooij DA, Soeters N, Imhof SM, Wisse RP. Corneal cross-linking for pediatric keratoconus: long-term results. Cornea. 2016;35(7):954-958. doi: 10.1097/ICO.0000000000000819 [DOI] [PubMed] [Google Scholar]
- 35.Hallett M, Albanese A, Dressler D, et al. . Evidence-based review and assessment of botulinum neurotoxin for the treatment of movement disorders. Toxicon. 2013;67:94-114. doi: 10.1016/j.toxicon.2012.12.004 [DOI] [PubMed] [Google Scholar]
- 36.Keith PK, Scadding GK. Are intranasal corticosteroids all equally consistent in managing ocular symptoms of seasonal allergic rhinitis? Curr Med Res Opin. 2009;25(8):2021-2041. doi: 10.1185/03007990903094106 [DOI] [PubMed] [Google Scholar]
- 37.Kok YO, Tan GF, Loon SC. Review: keratoconus in Asia. Cornea. 2012;31(5):581-593. doi: 10.1097/ICO.0b013e31820cd61d [DOI] [PubMed] [Google Scholar]
- 38.Lee SY, Tong L. Lipid-containing lubricants for dry eye: a systematic review. Optom Vis Sci. 2012;89(11):1654-1661. doi: 10.1097/OPX.0b013e31826f32e0 [DOI] [PubMed] [Google Scholar]
- 39.Li M, Zhu M, Yu Y, Gong L, Zhao N, Robitaille MJ. Comparison of conjunctival autograft transplantation and amniotic membrane transplantation for pterygium: a meta-analysis. Graefes Arch Clin Exp Ophthalmol. 2012;250(3):375-381. doi: 10.1007/s00417-011-1820-8 [DOI] [PubMed] [Google Scholar]
- 40.Liu A, Ji J. Omega-3 essential fatty acids therapy for dry eye syndrome: a meta-analysis of randomized controlled studies. Med Sci Monit. 2014;20:1583-1589. doi: 10.12659/MSM.891364 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Liu H, Chen Y, Wang P, et al. . Efficacy and safety of deep anterior lamellar keratoplasty vs. penetrating keratoplasty for keratoconus: a meta-analysis. PLoS One. 2015;10(1):e0113332. doi: 10.1371/journal.pone.0113332 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Luo H, Li X, Liu J, Andrew F, George L. Chinese herbal medicine in treating primary Sjögren’s syndrome: a systematic review of randomized trials. Evid Based Complement Alternat Med. 2012;2012:640658. doi: 10.1155/2012/640658 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Lynn DD, Zukin LM, Dellavalle R. The safety and efficacy of Diphoterine for ocular and cutaneous burns in humans. Cutan Ocul Toxicol. 2017;36(2):185-192. doi: 10.1080/15569527.2016.1217423 [DOI] [PubMed] [Google Scholar]
- 44.Molina-Leyva I, Molina-Leyva A, Bueno-Cavanillas A. Efficacy of nutritional supplementation with omega-3 and omega-6 fatty acids in dry eye syndrome: a systematic review of randomized clinical trials. Acta Ophthalmol. 2017;95(8):e677-e685. doi: 10.1111/aos.13428 [DOI] [PubMed] [Google Scholar]
- 45.Sánchez-Thorin JC, Rocha G, Yelin JB. Meta-analysis on the recurrence rates after bare sclera resection with and without mitomycin C use and conjunctival autograft placement in surgery for primary pterygium. Br J Ophthalmol. 1998;82(6):661-665. doi: 10.1136/bjo.82.6.661 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Tandon A, Espandar L, Cupp D, Ho S, Johnson V, Ayyala RS. Surgical management for postkeratoplasty glaucoma: a meta-analysis. J Glaucoma. 2014;23(7):424-429. doi: 10.1097/IJG.0b013e31827a0712 [DOI] [PubMed] [Google Scholar]
- 47.Tian C, Peng X, Fan Z, Yin Z. Corneal refractive surgery and phakic intraocular lens for treatment of amblyopia caused by high myopia or anisometropia in children. Chin Med J (Engl). 2014;127(11):2167-2172. [PubMed] [Google Scholar]
- 48.Wan KH, Chen LJ, Young AL. Efficacy and safety of topical 0.05% cyclosporine eye drops in the treatment of dry eye syndrome: a systematic review and meta-analysis. Ocul Surf. 2015;13(3):213-225. doi: 10.1016/j.jtos.2014.12.006 [DOI] [PubMed] [Google Scholar]
- 49.Wan Q, Wang D, Ye H, Tang J, Han Y. A review and meta-analysis of corneal cross-linking for post-laser vision correction ectasia. J Curr Ophthalmol. 2017;29(3):145-153. doi: 10.1016/j.joco.2017.02.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Weaver CS, Terrell KM. Evidence-based emergency medicine: update: do ophthalmic nonsteroidal anti-inflammatory drugs reduce the pain associated with simple corneal abrasion without delaying healing? Ann Emerg Med. 2003;41(1):134-140. doi: 10.1067/mem.2003.38 [DOI] [PubMed] [Google Scholar]
- 51.Zhao Y, Ma L. Systematic review and meta-analysis on transplantation of ex vivo cultivated limbal epithelial stem cell on amniotic membrane in limbal stem cell deficiency. Cornea. 2015;34(5):592-600. doi: 10.1097/ICO.0000000000000398 [DOI] [PubMed] [Google Scholar]
- 52.Zhou XQ, Wei RL. Topical cyclosporine A in the treatment of dry eye: a systematic review and meta-analysis. Cornea. 2014;33(7):760-767. doi: 10.1097/ICO.0000000000000123 [DOI] [PubMed] [Google Scholar]
- 53.Ziaei M, Barsam A, Shamie N, et al. ; ASCRS Cornea Clinical Committee . Reshaping procedures for the surgical management of corneal ectasia. J Cataract Refract Surg. 2015;41(4):842-872. doi: 10.1016/j.jcrs.2015.03.010 [DOI] [PubMed] [Google Scholar]
- 54.Abudou M, Wu T, Evans JR, Chen X. Immunosuppressants for the prophylaxis of corneal graft rejection after penetrating keratoplasty. Cochrane Database Syst Rev. 2015;(8):CD007603. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Akpek EK, Alkharashi M, Hwang FS, Ng SM, Lindsley K. Artificial corneas versus donor corneas for repeat corneal transplants. Cochrane Database Syst Rev. 2014;(11):CD009561. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Akpek EK, Lindsley KB, Adyanthaya RS, Swamy R, Baer AN, McDonnell PJ. Treatment of Sjögren’s syndrome–associated dry eye: an evidence-based review. Ophthalmology. 2011;118(7):1242-1252. [DOI] [PubMed] [Google Scholar]
- 57.Alhassan MB, Rabiu M, Agbabiaka IO. Interventions for Mooren’s ulcer. Cochrane Database Syst Rev. 2014;(1):CD006131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Alkharashi M, Lindsley K, Law HA, Sikder S. Medical interventions for acanthamoeba keratitis. Cochrane Database Syst Rev. 2015;(2):CD010792. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Bali S, Filek R, Si F, Hodge W. Systemic immunosuppression in high-risk penetrating keratoplasty: a systematic review. J Clin Med Res. 2016;8(4):269-276. doi: 10.14740/jocmr2326w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Barker NH. Ocular herpes simplex. BMJ Clin Evid. 2008;2008:0707. [PMC free article] [PubMed] [Google Scholar]
- 61.Olopatadine for the Treatment of Allergic Conjunctivitis: A Review of the Clinical Efficacy, Safety, and Cost-Effectiveness Ottawa: Canadian Agency for Drugs and Technologies in Health; March 2016. CADTH Rapid Response Reports. [PubMed] [Google Scholar]
- 62.Calder LA, Balasubramanian S, Fergusson D. Topical nonsteroidal anti-inflammatory drugs for corneal abrasions: meta-analysis of randomized trials. Acad Emerg Med. 2005;12(5):467-473. doi: 10.1197/j.aem.2004.10.026 [DOI] [PubMed] [Google Scholar]
- 63.Calderon MA, Penagos M, Sheikh A, Canonica GW, Durham S. Sublingual immunotherapy for treating allergic conjunctivitis. Cochrane Database Syst Rev. 2011;(7):CD007685. [DOI] [PubMed] [Google Scholar]
- 64.Castillo M, Scott NW, Mustafa MZ, Mustafa MS, Azuara-Blanco A. Topical antihistamines and mast cell stabilisers for treating seasonal and perennial allergic conjunctivitis. Cochrane Database Syst Rev. 2015;(6):CD009566. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Chen G, Tzekov R, Li W, Jiang F, Mao S, Tong Y. Deep anterior lamellar keratoplasty versus penetrating keratoplasty: a meta-analysis of randomized controlled trials. Cornea. 2016;35(2):169-174. doi: 10.1097/ICO.0000000000000691 [DOI] [PubMed] [Google Scholar]
- 66.Cheng K, Law A, Guo M, Wieland LS, Shen X, Lao L. Acupuncture for acute hordeolum. Cochrane Database Syst Rev. 2017;2:CD011075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Clearfield E, Muthappan V, Wang X, Kuo IC. Conjunctival autograft for pterygium. Cochrane Database Syst Rev. 2016;2:CD011349. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Compalati E, Canonica GW. Efficacy and safety of rupatadine for allergic rhino-conjunctivitis: a systematic review of randomized, double-blind, placebo-controlled studies with meta-analysis. Curr Med Res Opin. 2013;29(11):1539-1551. doi: 10.1185/03007995.2013.822855 [DOI] [PubMed] [Google Scholar]
- 69.Epling J. Bacterial conjunctivitis. BMJ Clin Evid. 2012;2012:0704. [PMC free article] [PubMed] [Google Scholar]
- 70.Erekosima N, Suarez-Cuervo C, Ramanathan M, et al. . Effectiveness of subcutaneous immunotherapy for allergic rhinoconjunctivitis and asthma: a systematic review. Laryngoscope. 2014;124(3):616-627. doi: 10.1002/lary.24295 [DOI] [PubMed] [Google Scholar]
- 71.Ervin AM, Wojciechowski R, Schein O. Punctal occlusion for dry eye syndrome. Cochrane Database Syst Rev. 2010;(9):CD006775. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Feng YF, Yu JG, Wang DD, et al. . The effect of hinge location on corneal sensation and dry eye after LASIK: a systematic review and meta-analysis. Graefes Arch Clin Exp Ophthalmol. 2013;251(1):357-366. doi: 10.1007/s00417-012-2078-5 [DOI] [PubMed] [Google Scholar]
- 73.Fischer S, Zechmeister-Koss I, Charpentier E. Intrastromal Corneal Implants for Ectatic Corneal Disorders. Vienna, Austria: Ludwig Boltzmann Institute for Health Technology Assessment; 2015. Decision Support Document 85. [Google Scholar]
- 74.FlorCruz NV, Evans JR. Medical interventions for fungal keratitis. Cochrane Database Syst Rev. 2015;(4):CD004241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.Flynn CA, D’Amico F, Smith G. Should we patch corneal abrasions? a meta-analysis. J Fam Pract. 1998;47(4):264-270. [PubMed] [Google Scholar]
- 76.Gane J, Buckley R. Leukotriene receptor antagonists in allergic eye disease: a systematic review and meta-analysis. J Allergy Clin Immunol Pract. 2013;1(1):65-74. doi: 10.1016/j.jaip.2012.07.001 [DOI] [PubMed] [Google Scholar]
- 77.González-López JJ, López-Alcalde J, Morcillo Laiz R, Fernández Buenaga R, Rebolleda Fernández G. Topical cyclosporine for atopic keratoconjunctivitis. Cochrane Database Syst Rev. 2012;(9):CD009078. [DOI] [PubMed] [Google Scholar]
- 78.Henein C, Nanavaty MA. Systematic review comparing penetrating keratoplasty and deep anterior lamellar keratoplasty for management of keratoconus. Cont Lens Anterior Eye. 2017;40(1):3-14. doi: 10.1016/j.clae.2016.10.001 [DOI] [PubMed] [Google Scholar]
- 79.Herretes S, Wang X, Reyes JM. Topical corticosteroids as adjunctive therapy for bacterial keratitis. Cochrane Database Syst Rev. 2014;(10):CD005430. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80.Hu Q, Qiao Y, Nie X, Cheng X, Ma Y. Bevacizumab in the treatment of pterygium: a meta-analysis. Cornea. 2014;33(2):154-160. doi: 10.1097/ICO.0000000000000037 [DOI] [PubMed] [Google Scholar]
- 81.Jefferis J, Perera R, Everitt H, et al. . Acute infective conjunctivitis in primary care: who needs antibiotics? An individual patient data meta-analysis. Br J Gen Pract. 2011;61(590):e542-e548. doi: 10.3399/bjgp11X593811 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 82.Kam KW, Chen LJ, Wat N, Young AL. Topical olopatadine in the treatment of allergic conjunctivitis: a systematic review and meta-analysis. Ocul Immunol Inflamm. 2017;25(5):663-677. doi: 10.3109/09273948.2016.1158282 [DOI] [PubMed] [Google Scholar]
- 83.Keane M, Coster D, Ziaei M, Williams K. Deep anterior lamellar keratoplasty versus penetrating keratoplasty for treating keratoconus. Cochrane Database Syst Rev. 2014;(7):CD009700. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Kobashi H, Rong SS. Corneal collagen cross-linking for keratoconus: systematic review. Biomed Res Int. 2017;2017:8145651. doi: 10.1155/2017/8145651 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85.Lee MS, Shin BC, Choi TY, Ernst E. Acupuncture for treating dry eye: a systematic review. Acta Ophthalmol. 2011;89(2):101-106. doi: 10.1111/j.1755-3768.2009.01855.x [DOI] [PubMed] [Google Scholar]
- 86.Lim CH, Turner A, Lim BX. Patching for corneal abrasion. Cochrane Database Syst Rev. 2016;7:CD004764. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 87.Lin SY, Erekosima N, Kim JM, et al. . Sublingual immunotherapy for the treatment of allergic rhinoconjunctivitis and asthma: a systematic review. JAMA. 2013;309(12):1278-1288. doi: 10.1001/jama.2013.2049 [DOI] [PubMed] [Google Scholar]
- 88.Lin SY, Erekosima N, Suarez-Cuervo C, et al. . Allergen-Specific Immunotherapy for the Treatment of Allergic Rhinoconjunctivitis and/or Asthma: Comparative Effectiveness Review. Rockville, MD: Agency for Healthcare Research and Quality; March 2013. Report 13-EHC061-EF. AHRQ Comparative Effectiveness Reviews. [PubMed] [Google Scholar]
- 89.Lindsley K, Matsumura S, Hatef E, Akpek EK. Interventions for chronic blepharitis. Cochrane Database Syst Rev. 2012;(5):CD005556. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 90.Lindsley K, Nichols JJ, Dickersin K. Non-surgical interventions for acute internal hordeolum. Cochrane Database Syst Rev. 2017;1:CD007742. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 91.Mantelli F, Santos MS, Petitti T, et al. . Systematic review and meta-analysis of randomised clinical trials on topical treatments for vernal keratoconjunctivitis. Br J Ophthalmol. 2007;91(12):1656-1661. doi: 10.1136/bjo.2007.122044 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 92.McAnena L, Doyle F, O’Keefe M. Cross-linking in children with keratoconus: a systematic review and meta-analysis. Acta Ophthalmol. 2017;95(3):229-239. doi: 10.1111/aos.13224 [DOI] [PubMed] [Google Scholar]
- 93.McDonald EM, Ram FS, Patel DV, McGhee CN. Topical antibiotics for the management of bacterial keratitis: an evidence-based review of high quality randomised controlled trials. Br J Ophthalmol. 2014;98(11):1470-1477. doi: 10.1136/bjophthalmol-2013-304660 [DOI] [PubMed] [Google Scholar]
- 94.Nanavaty MA, Wang X, Shortt AJ. Endothelial keratoplasty versus penetrating keratoplasty for Fuchs endothelial dystrophy. Cochrane Database Syst Rev. 2014;(2):CD008420. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 95.O’Gallagher M, Banteka M, Bunce C, Larkin F, Tuft S, Dahlmann-Noor A. Systemic treatment for blepharokeratoconjunctivitis in children. Cochrane Database Syst Rev. 2016;(5):CD011750. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 96.O’Gallagher M, Bunce C, Hingorani M, Larkin F, Tuft S, Dahlmann-Noor A. Topical treatments for blepharokeratoconjunctivitis in children. Cochrane Database Syst Rev. 2017;2:CD011965. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 97.Medical Advisory Secretariat Intrastromal corneal ring implants for corneal thinning disorders: an evidence-based analysis. Ont Health Technol Assess Ser. 2009;9(1):1-90. [PMC free article] [PubMed] [Google Scholar]
- 98.Owen CG, Shah A, Henshaw K, Smeeth L, Sheikh A. Topical treatments for seasonal allergic conjunctivitis: systematic review and meta-analysis of efficacy and effectiveness. Br J Gen Pract. 2004;54(503):451-456. [PMC free article] [PubMed] [Google Scholar]
- 99.Pan Q, Angelina A, Marrone M, Stark WJ, Akpek EK. Autologous serum eye drops for dry eye. Cochrane Database Syst Rev. 2017;2:CD009327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 100.Papaioannou L, Miligkos M, Papathanassiou M. Corneal collagen cross-linking for infectious keratitis: a systematic review and meta-analysis. Cornea. 2016;35(1):62-71. doi: 10.1097/ICO.0000000000000644 [DOI] [PubMed] [Google Scholar]
- 101.Pucker AD, Ng SM, Nichols JJ. Over the counter (OTC) artificial tear drops for dry eye syndrome. Cochrane Database Syst Rev. 2016;2:CD009729. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 102.Ramos-Casals M, Tzioufas AG, Stone JH, Sisó A, Bosch X. Treatment of primary Sjögren syndrome: a systematic review. JAMA. 2010;304(4):452-460. doi: 10.1001/jama.2010.1014 [DOI] [PubMed] [Google Scholar]
- 103.Röder E, Berger MY, de Groot H, van Wijk RG. Immunotherapy in children and adolescents with allergic rhinoconjunctivitis: a systematic review. Pediatr Allergy Immunol. 2008;19(3):197-207. doi: 10.1111/j.1399-3038.2007.00648.x [DOI] [PubMed] [Google Scholar]
- 104.Sacchetti M, Mantelli F, Lambiase A, Mastropasqua A, Merlo D, Bonini S. Systematic review of randomised clinical trials on topical ciclosporin A for the treatment of dry eye disease. Br J Ophthalmol. 2014;98(8):1016-1022. doi: 10.1136/bjophthalmol-2013-304072 [DOI] [PubMed] [Google Scholar]
- 105.Sheikh A, Hurwitz B, van Schayck CP, McLean S, Nurmatov U. Antibiotics versus placebo for acute bacterial conjunctivitis. Cochrane Database Syst Rev. 2012;(9):CD001211. [DOI] [PubMed] [Google Scholar]
- 106.Song JK, Lee K, Park HY, et al. . Efficacy of carboxymethylcellulose and hyaluronate in dry eye disease: a systematic review and meta-analysis. Korean J Fam Med. 2017;38(1):2-7. doi: 10.4082/kjfm.2017.38.1.2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 107.Sopo SM, Macchiaiolo M, Zorzi G, Tripodi S. Sublingual immunotherapy in asthma and rhinoconjunctivitis: systematic review of paediatric literature. Arch Dis Child. 2004;89(7):620-624. doi: 10.1136/adc.2003.030411 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 108.Swamy BN, Chilov M, McClellan K, Petsoglou C. Topical non-steroidal anti-inflammatory drugs in allergic conjunctivitis: meta-analysis of randomized trial data. Ophthalmic Epidemiol. 2007;14(5):311-319. doi: 10.1080/09286580701299411 [DOI] [PubMed] [Google Scholar]
- 109.Sykakis E, Karim R, Evans JR, et al. . Corneal collagen cross-linking for treating keratoconus. Cochrane Database Syst Rev. 2015;(3):CD010621. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 110.van Zuuren EJ, Fedorowicz Z, Carter B, van der Linden MM, Charland L. Interventions for rosacea. Cochrane Database Syst Rev. 2015;(4):CD003262. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 111.Wakai A, Lawrenson JG, Lawrenson AL, et al. . Topical non-steroidal anti-inflammatory drugs for analgesia in traumatic corneal abrasions. Cochrane Database Syst Rev. 2017;5:CD009781. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 112.Wan KH, Chen LJ, Rong SS, Pang CP, Young AL. Topical cyclosporine in the treatment of allergic conjunctivitis: a meta-analysis. Ophthalmology. 2013;120(11):2197-2203. doi: 10.1016/j.ophtha.2013.03.044 [DOI] [PubMed] [Google Scholar]
- 113.Watson SL, Lee MH, Barker NH. Interventions for recurrent corneal erosions. Cochrane Database Syst Rev. 2012;(9):CD001861. [DOI] [PubMed] [Google Scholar]
- 114.Wilhelmus KR. Antiviral treatment and other therapeutic interventions for herpes simplex virus epithelial keratitis. Cochrane Database Syst Rev. 2015;1:CD002898. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 115.Wu D, Chen WQ, Li R, Wang Y. Efficacy and safety of topical diquafosol ophthalmic solution for treatment of dry eye: a systematic review of randomized clinical trials. Cornea. 2015;34(6):644-650. doi: 10.1097/ICO.0000000000000429 [DOI] [PubMed] [Google Scholar]
- 116.Xu Y, Zhou HM, Li J, Ke BL, Xu X. Efficacy of treatment for pterygium by autologous conjunctival transplantation and mitomycin C. Chin Med J (Engl). 2012;125(20):3730-3734. [PubMed] [Google Scholar]
- 117.Zheng K, Cai J, Jhanji V, Chen H. Comparison of pterygium recurrence rates after limbal conjunctival autograft transplantation and other techniques: meta-analysis. Cornea. 2012;31(12):1422-1427. doi: 10.1097/ICO.0b013e31823cbecb [DOI] [PubMed] [Google Scholar]
- 118.Zhu W, Wu Y, Li G, Wang J, Li X. Efficacy of polyunsaturated fatty acids for dry eye syndrome: a meta-analysis of randomized controlled trials. Nutr Rev. 2014;72(10):662-671. doi: 10.1111/nure.12145 [DOI] [PubMed] [Google Scholar]
- 119.Page MJ, Shamseer L, Altman DG, et al. . Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13(5):e1002028. doi: 10.1371/journal.pmed.1002028 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 120.Altman DG. The scandal of poor medical research. BMJ. 1994;308(6924):283-284. doi: 10.1136/bmj.308.6924.283 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 121.Current version of RoB 2. https://www.riskofbias.info/welcome/rob-2-0-tool/current-version-of-rob-2. Published October 9, 2018. Accessed November 5, 2018.
- 122.Liesegang TJ, Bartley GB. Toward transparency of financial disclosure. Am J Ophthalmol. 2014;158(5):855-857. doi: 10.1016/j.ajo.2014.09.014 [DOI] [PubMed] [Google Scholar]
- 123.Saldanha IJ, Scherer RW, Rodriguez-Barraquer I, Jampel HD, Dickersin K. Dependability of results in conference abstracts of randomized controlled trials in ophthalmology and author financial conflicts of interest as a factor associated with full publication. Trials. 2016;17(1):213. doi: 10.1186/s13063-016-1343-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 124.Downie LE, Makrai E, Bonggotgetsakul Y, et al. . Appraising the quality of systematic reviews for age-related macular degeneration interventions: a systematic review. JAMA Ophthalmol. 2018;136(9):1051-1061. doi: 10.1001/jamaophthalmol.2018.2620 [DOI] [PubMed] [Google Scholar]
- 125.Li T, Dickersin K. Citation of previous meta-analyses on the same topic: a clue to perpetuation of incorrect methods? Ophthalmology. 2013;120(6):1113-1119. doi: 10.1016/j.ophtha.2012.11.038 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 126.Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7(2):e012545. doi: 10.1136/bmjopen-2016-012545 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 127.Chalmers I, Bracken MB, Djulbegovic B, et al. . How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156-165. doi: 10.1016/S0140-6736(13)62229-1 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.