Abstract
Background: Randomized controlled trials (RCTs) are situated at the top of hierarchy of evidence-based medicine, where its number and quality are important in the assessment of quality of evidence in a medical field. In this study, we aim to assess the status of RCTs in Ophthalmology.
Methods: On 15 th of May 2019, we performed a PubMed search for randomized controlled trials published in the field of ophthalmology using relevant filters and search terms. We categorized the results into specific topics in ophthalmology according to Medical Subject Heading (MeSH) database classification system. We used Altmetric explorer to identify journals and articles with the highest number of RCTs and highest citations.
Results: We found a total of 540,427 publications in the field of ophthalmology, of which only 11,634 (2.15%) of them were RCTs. ‘Retinal diseases’ was the topic with the highest number of RCTs, followed by ‘glaucoma’ and ‘conjunctival diseases’. The trial with highest number of citations was on retinal diseases. Only around 18% of all ophthalmology RCTs are published in the top 10 ophthalmology journals, with a maximum percentage of RCTs was (5.53%) published in Ophthalmology.
Conclusion: RCTs in ophthalmology primarily concern the retina, glaucoma, and a few other sub-topics, with little focus on sclera, orbit, and the eyelids. Most of the high impact RCTs are published in non-ophthalmology journals.
Keywords: Ophthalmology; Randomized Controlled Trials; PubMed; Retina; Journals; Bibliometrics.
Introduction
Since the conception of the term “evidence-based medicine” in clinical practice in 1992 1, where well-conducted randomized controlled trials (RCTs) are situated at the top of hierarchy of evidence, there has been an emphasis on accepting high quality evidence in terms of RCTs in clinical practice. Moreover, previous reports showed that RCTs have generally higher methodological rigor than observational studies 2. However, despite the rapid growth in ophthalmology literature in the recent years, this growth has not been paralleled by a growth in the quality of evidence 3. This is evident by the number of Cochrane reviews that don’t include any RCTs (i.e. empty review), which were estimated to be half of the total reviews on Cochrane Eyes and Vision in 2013 4. In this study, we aim to assess the status of RCTs in ophthalmology, and will focus on publishing trends for RCTs in ophthalmology in the recent years with regards to different ophthalmology topics.
Methods
PubMed search strategy
On 15 th of May 2019, we performed a PubMed search for randomized controlled trials published in the field of ophthalmology. We used the following search filters:
Ophthalmology studies: eye diseases [MeSH Terms]
RCT: Randomized Controlled Trial [Publication Type]
To categorize the results into specific topics in ophthalmology, we used the Medical Subject Heading (MeSH) database to identify the topics within ophthalmology, where the following were included:
Orbital Diseases
Conjunctival Diseases
Corneal Diseases
Eyelid Diseases
Lacrimal Apparatus Diseases
Lens Diseases
Glaucoma
Refractive Errors
Scleral Diseases
Uveal Diseases
Retinal Diseases
For each topic, we added the query as a MeSH term to the search to identify relevant articles (e.g. Orbital diseases[Mesh Terms]. It is worth noting that trials might be categorized in more than one topic.
To identify journals with the highest number of RCTs and top articles with highest citations, we used Altmetric database, where we inputted the PubMed query we used in the PubMed search in the search field; the database yielded citation information about searched articles along with information about the journals these articles published previously 5.
Variables
For each RCT, we extracted data regarding the topic of the study and categorized them into the following: RCTs per year, percentage of each sub-specialty, Articles per sub-specialty per year, Top 10 journals with their respective data, Top 10 articles with highest dimensions citations
Results
Ophthalmology RCTs
A total of 540,427 publications in the field of ophthalmology were identified, of which only 11,634 (2.15%) of them were RCTs. There was a total of 482,791 RCT identified in all disciplines, of which only 2.4% are in the field of ophthalmology. Of these trials, 124 were phase 1 trials, 270 were phase 2 trials, 380 were phase 3 trials, and 42 phase 4 trials; all others did not have phases. Number of RCTs peaked in 2015 with a total of 583 trials. Figure 1 shows the trend in number of RCTs in the field of ophthalmology.
Figure 1. Number of randomized controlled trials published per year in the field of ophthalmology from 1961 to May 2019.
RCTs in each ophthalmology topic
‘Retinal diseases’ is the topic with the highest number of RCTs, with a total of 2915 trials, followed by ‘glaucoma’, with 2118 trials, and ‘conjunctival diseases’, with 1230 trials. Figure 2 details the number of trials for each topic.
Figure 2. The number of randomized controlled trials (RCTs) for each topic in ophthalmology.
Top RCTs and ophthalmology journals publishing RCTs
The trial with highest number of citations discussed retinal complications of diabetes mellitus entitled “The Effect of Intensive Treatment of Diabetes on the Development and Progression of Long-Term Complications in Insulin-Dependent Diabetes Mellitus”, published in The New England Journal of Medicine 6. Table 1 details the top 10 RCTs with highest citations. A total of 2090 (18%) of the RCTs were published in 10 journals, with “Ophthalmology” being the top journal with highest number of RCT published in it (643 RCTs). Table 2 details the top 10 journals with highest number of RCTs published in them.
Table 1. The 10 randomized controlled trials with the highest number of citations.
| Number | Citations | Title | Journal | Publication
Date |
Reference | OA Status |
|---|---|---|---|---|---|---|
| 1 | 16741 | The Effect of Intensive Treatment of Diabetes on
the Development and Progression of Long-Term Complications in Insulin-Dependent Diabetes Mellitus |
New England Journal
of Medicine |
1993 | 6 | FALSE |
| 2 | 5177 | Tight blood pressure control and risk
of macrovascular and microvascular complications in type 2†diabetes: UKPDS 38 12 |
British Medical
Journal |
1998 | 12 | TRUE |
| 3 | 3623 | Ranibizumab for Neovascular Age-Related
Macular Degeneration |
New England Journal
of Medicine |
2006 | 13 | FALSE |
| 4 | 2343 | Ranibizumab versus Verteporfin for
Neovascular Age-Related Macular Degeneration |
New England Journal
of Medicine |
2006 | 14 | FALSE |
| 5 | 2239 | The Ocular Hypertension Treatment Study | Archives of
Ophthalmology |
2002 | 15 | TRUE |
| 6 | 1736 | Pegaptanib for Neovascular Age-Related
Macular Degeneration |
New England Journal
of Medicine |
2004 | 16 | TRUE |
| 7 | 1683 | The advanced glaucoma intervention study
(AGIS): 7. the relationship between control of intraocular pressure and visual field deterioration |
American Journal of
Ophthalmology |
2000 | 17 | FALSE |
| 8 | 1641 | The Ocular Hypertension Treatment Study | Archives of
Ophthalmology |
2002 | 18 | TRUE |
| 9 | 1640 | Whole-Body Hypothermia for Neonates with
Hypoxic-Ischemic Encephalopathy |
New England Journal
of Medicine |
2005 | 19 | FALSE |
| 10 | 1526 | Grading Diabetic Retinopathy from
Stereoscopic Color Fundus Photographs - An Extension of the Modified Airlie House Classification |
Ophthalmology | 1991 | 20 | FALSE |
OA, open access.
Table 2. The 10 ophthalmology journals with the highest number of randomized controlled trials (RCTs) and proportion of RCTs of total ophthalmology RCTs published in each.
| Journal | Number of RCTs | Percentage from total
ophthalmology RCT |
|---|---|---|
| Ophthalmology | 643 | 5.53% |
| American Journal of Ophthalmology | 333 | 2.86% |
| British Journal of Ophthalmology | 246 | 2.11% |
| Investigative Ophthalmology & Visual Science | 163 | 1.40% |
| Archives of Ophthalmology | 157 | 1.35% |
| Journal of Cataract & Refractive Surgery | 149 | 1.28% |
| Retina | 106 | 0.91% |
| JAMA Ophthalmology | 103 | 0.89% |
| Optometry and Vision Science | 97 | 0.83% |
| Journal of Glaucoma | 93 | 0.80% |
| Total | 2090 | 17.96% |
Discussion
In the current study, we observed a peak in the annual number of RCTs on 2015, after which a steady decrease observed till 2018. Retinal diseases is the topic with the highest number of RCTs, followed by glaucoma and conjunctival diseases. The trial with highest citation was on retinal diseases and was published in The New England Journal of Medicine, where also other top cited trials were published in general non-ophthalmology journals. The total RCTs published in top 100 ophthalmology journals was only 2090 (17.96%).
In general, there has been an increase in the number of RCTs in ophthalmology since the late 1990s. In a study assessing the frequency of prospective studies published in the American Journal of Ophthalmology and British Journal of Ophthalmology, they found an increase from 1% to 12% during the years 1980 to 1999 7. We observed a low number of RCTs among the ophthalmology literature, a percentage that didn’t exceed 2.5% of the overall ophthalmology literature. In a previous study assessing the frequency of RCTs published in the major four ophthalmology journals, they found that only around 3.5% of their annual publications are RCTS 8. Moreover, we found that only around 18% of all ophthalmology RCTs are published in the top 10 ophthalmology journals, with the most RCTs (5.53%) published in Ophthalmology. In a study that reviewed risk of bias in RCTs published in major ophthalmology journals found that a risk of bias was observed in 29.4% of published RCTs 9. In another study that assessed fragility of RCT’s that included the comparison between two groups found a high proportion of fragile results in ophthalmology RCTs 10. In a study that assessed types of articles published in core pediatric journals, they found that only 0.3% were RCTs 11, which supports our findings that a large proportion of RCTs were published in high-impact general medical journals.
One of the main limitations in this study is that it didn’t assess the quality of RCTs, so we included RCTs from our PubMed search regardless of their quality. Recent studies have stated that ophthalmology literature is of questionable methodological robustness, where RCTs become the center of the scope when methodological robustness is assessed, as they are the source of the highest level of evidence 10, 21. Future studies should focus on assessing quality of RCTs rather than the quantity (which was the scope of this study), where the Cochrane Eyes and Vision library criteria for RCT robustness can be utilized 22.
Data availability
Underlying data
Harvard Dataverse: Ophthalmology randomized controlled trials. https://doi.org/10.7910/DVN/TXEYDX 23.
This project contains the articles identified during this study.
Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).
Funding Statement
The author(s) declared that no grants were involved in supporting this work.
[version 1; peer review: 2 approved]
References
- 1. Evidence-Based Medicine Working Group: Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420–5. 10.1001/jama.1992.03490170092032 [DOI] [PubMed] [Google Scholar]
- 2. Lai TY, Leung GM, Wong VW, et al. : How evidence-based are publications in clinical ophthalmic journals?. Invest Ophthalmol Vis Sci. 2006;47(5):1831–8. 10.1167/iovs.05-0915 [DOI] [PubMed] [Google Scholar]
- 3. Boudry C, Baudouin C, Mouriaux F: International publication trends in dry eye disease research: A bibliometric analysis. Ocul Surf. 2018;16(1):173–9. 10.1016/j.jtos.2017.10.002 [DOI] [PubMed] [Google Scholar]
- 4. Wormald R, Dickersin K, Cochrane E, et al. : Evidence-Based Ophthalmology. Ophthalmology. 2013;120(12):2361–2363.e1. 10.1016/j.ophtha.2013.08.032 [DOI] [PubMed] [Google Scholar]
- 5. Priem J, Groth P, Taraborelli D: The altmetrics collection. PLoS One. 2012;7(11):e48753. 10.1371/journal.pone.0048753 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Diabetes Control and Complications Trial Research Group, Nathan DM, Genuth S, et al. : Diabetes Control and Complications Trial Research Group: The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993;329(14):977–86. 10.1056/NEJM199309303291401 [DOI] [PubMed] [Google Scholar]
- 7. Ang A, Tong L, Bhan A: Analysis of publication trends in two internationally renowned ophthalmology journals. Br J Ophthalmol. 2001;85(12):1497–8. 10.1136/bjo.85.12.1497 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Bojikian KD, Gupta D, Dettori JM, et al. : Evidence in ophthalmology: Are we doing better? Ophthalmology. 2015;122(12):2584–6. 10.1016/j.ophtha.2015.06.012 [DOI] [PubMed] [Google Scholar]
- 9. Joksimovic L, Koucheki R, Popovic M, et al. : Risk of bias assessment of randomised controlled trials in high-impact ophthalmology journals and general medical journals: a systematic review. Br J Ophthalmol. 2017;101(10):1309–14. 10.1136/bjophthalmol-2017-310313 [DOI] [PubMed] [Google Scholar]
- 10. Shen C, Shamsudeen I, Farrokhyar F, et al. : Fragility of results in ophthalmology randomized controlled trials: a systematic review. Ophthalmology. 2018;125(5):642–8. 10.1016/j.ophtha.2017.11.015 [DOI] [PubMed] [Google Scholar]
- 11. Hardin WD, Jr, Stylianos S, Lally KP: Evidence-based practice in pediatric surgery. J Pediatr Surg. 1999;34(5):908–13; discussion 912-3. 10.1016/s0022-3468(99)90396-2 [DOI] [PubMed] [Google Scholar]
- 12. UK Prospective Diabetes Study Group: Tight blood pressure control and risk of macrovascular and microvascular complications in type 2 diabetes: UKPDS 38. UK Prospective Diabetes Study Group. BMJ 1998;317(7160):703–13. [PMC free article] [PubMed] [Google Scholar]
- 13. Rosenfeld PJ, Brown DM, Heier JS, et al. : Ranibizumab for neovascular age-related macular degeneration. N Engl J Med. 2006;355(14):1419–31. 10.1056/NEJMoa054481 [DOI] [PubMed] [Google Scholar]
- 14. Brown DM, Kaiser PK, Michels M, et al. : Ranibizumab versus verteporfin for neovascular age-related macular degeneration. N Engl J Med. 2006;355(14):1432–44. 10.1056/NEJMoa062655 [DOI] [PubMed] [Google Scholar]
- 15. Kass MA, Heuer DK, Higginbotham EJ, et al. : The Ocular Hypertension Treatment Study: a randomized trial determines that topical ocular hypotensive medication delays or prevents the onset of primary open-angle glaucoma. Arch Ophthalmol. 2002;120(6):701–13. 10.1001/archopht.120.6.701 [DOI] [PubMed] [Google Scholar]
- 16. Gragoudas ES, Adamis AP, Cunningham ET, Jr, et al. : Pegaptanib for neovascular age-related macular degeneration. N Engl J Med. 2004;351(27):2805–16. 10.1056/NEJMoa042760 [DOI] [PubMed] [Google Scholar]
- 17. Agis Investigators: The Advanced Glaucoma Intervention Study (AGIS): 7. The relationship between control of intraocular pressure and visual field deterioration.The AGIS Investigators. Am J Ophthalmol. 2000;130(4):429–40. 10.1016/s0002-9394(00)00538-9 [DOI] [PubMed] [Google Scholar]
- 18. Gordon MO, Beiser JA, Brandt JD, et al. : The Ocular Hypertension Treatment Study: baseline factors that predict the onset of primary open-angle glaucoma. Arch Ophthalmol. 2002;120(6):714–20. 10.1001/archopht.120.6.714 [DOI] [PubMed] [Google Scholar]
- 19. Shankaran S, Laptook AR, Ehrenkranz RA, et al. : Whole-body hypothermia for neonates with hypoxic-ischemic encephalopathy. N Engl J Med. 2005;353(15):1574–84. 10.1056/NEJMcps050929 [DOI] [PubMed] [Google Scholar]
- 20. Early Treatment Diabetic Retinopathy Study Research Group: Grading diabetic retinopathy from stereoscopic color fundus photographs--an extension of the modified Airlie House classification. ETDRS report number 10. Early Treatment Diabetic Retinopathy Study Research Group. Ophthalmology. 1991;98(5 Suppl):786–806. 10.1016/S0161-6420(13)38012-9 [DOI] [PubMed] [Google Scholar]
- 21. Sanfilippo PG, Casson RJ, Yazar S, et al. : Review of null hypothesis significance testing in the ophthalmic literature: are most 'significant' P values false positives? Clin Exp Ophthalmol. 2016;44(1):52–61. 10.1111/ceo.12570 [DOI] [PubMed] [Google Scholar]
- 22. Higgins JP, Green S, editors: Cochrane handbook for systematic reviews of interventions. John Wiley & Sons;2011. Reference Source [Google Scholar]
- 23. AlRyalat SA: "Ophthalmology randomized controlled trials". Harvard Dataverse, V1.2019. 10.7910/DVN/TXEYDX [DOI]