Standard pairwise meta-analysis compares 2 interventions at a time, for example, treatment A vs treatment B. By contrast, network meta-analysis (NMA) compares 3 or more interventions in a single analysis by combining direct evidence within studies and indirect evidence based on a common comparator across a network of studies. In addition to estimates of the relative effects between any pair of interventions in the network, treatment ranking is a very informative output of NMA. However, inappropriate interpretation and overinterpretation of ranking statistics can misinform clinical decision-making.
In this issue of JAMA Ophthalmology, Cho and colleagues1 present a systematic review and NMA of the comparative effectiveness of various types of multifocal intraocular lenses (IOLs) and monofocal IOLs in patients undergoing bilateral cataract extraction. The authors concluded that “for patients considering a multifocal IOL due to presbyopia, bilateral implantation of a trifocal IOL might be an optimal option for patients without compromising distant vision.”1 A more balanced and nuanced conclusion could be drawn when the ranking statistics are interpreted jointly with (1) the size of treatment effect; (2) safety; (3) certainty in evidence; and (4) clinical experience, patient preference, and cost considerations.2,3
Above all, even when appropriate ranking statistics like surface under the cumulative ranking (SUCRA) values are used, higher-ranking treatments may still have modest or insignificant clinical effects relative to lower-ranking treatments. The ranking statistics, therefore, must be presented and interpreted in the context of the size of treatment effect. In the NMA by Cho and colleagues, the SUCRA method produced a distinct rank for each treatment for each visual acuity outcome (Figure 2 in the Cho et al article). However, among the top 3 ranked IOLs—trifocal diffractive, bifocal diffractive old generation, and bifocal diffractive new generation—for uncorrected near visual acuity (Figure 2A in the Cho et al article), there is no evidence of difference in treatment effect among these 3 types of IOLs (eFigure 3A in the Cho et al article).1 The mean difference in uncorrected near visual acuity comparing trifocal diffractive with bifocal diffractive old generation and bifocal diffractive new generation is 0.00 logMAR (95% credible interval [CrI], 0.19-0.19) and −0.01 logMAR (95% CrI, −0.12 to 0.09), respectively, indicating that factors other than effectiveness, such as safety, cost, and patient preferences, may be more important in selecting the types of multifocal IOLs.
Evidence on some of the IOLs might be less “trustworthy” than for others. In addition to the estimates of treatment effects and ranking statistics, uncertainty, clinical and methodological characteristics, and potential biases within included studies must be conveyed. A common approach for evaluating the certainty of evidence from a systematic review is the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. When applying the GRADE approach to NMA, the evidence is evaluated based on 6 domains: within-study limitations (ie, classic risk of bias items), indirectness, imprecision, heterogeneity, inconsistency, and publication bias.4,5 We will elaborate on 2 domains: indirectness and imprecision.
Indirectness refers to the applicability of the body of evidence to the question of interest in terms of population, interventions, and outcomes. Of the 27 trials included in the review, 10 were published over 10 years ago and only 7 in the last 5 years. In fact, most of the lenses listed are no longer used on patients. Because the technology continues to evolve at a rapid rate, it is challenging to conclude the usefulness of lenses that may be 10 or 15 years old, especially when the latest technology may not be fully evaluated in randomized trials or analyzed in the article.
With regard to imprecision, the clinical utility of various IOLs could be better discerned if the authors provided a more nuanced deliberation of limitations of effectiveness and safety outcomes of included studies. Specifically, data were sparse for uncorrected and distance-corrected intermediate visual acuity, as well as for contrast sensitivity. None of these networks include a closed loop and the resulting estimates are imprecise and noninformative. Furthermore, few studies provided data on glare, halos, and spectacle independence—outcomes that are of main concerns for patients regarding the use of multifocal IOLs. The authors reported in the abstract that “there were no statistical differences between multifocal and monofocal IOLs regarding contrast sensitivity, glare, or halos,” but there was simply not enough evidence to detect a difference.
Randomized clinical trials are only as credible as their outcomes because the effectiveness and safety of interventions are judged by contrasting outcomes observed in participants in the intervention group with those in the comparison group. Research by our group and others has shown that trials in vision science, including trials of multifocal lenses in cataract surgery, have reported numerous different outcomes.6 These outcomes are often incompletely defined, inconsistent across trials, and sometimes unimportant to patients.6 A core outcome set is an agreed-upon minimum set of outcomes (usually 5-7), typically agreed by a community of stakeholders, that will be measured and reported in research in a given disease area. The existence of an agreed-upon core outcome set is important because it represents the community’s recognition of certain outcomes as important, valid, measurable, and relevant to the disease area; ensures consistency across trial findings; and facilitates synthesis of critical outcomes from all pertinent trials in meta-analyses and systematic reviews. To improve comparisons of IOLs, investigators in this field should adopt a core outcome set that is informed by patients’ perspectives.
Choosing an IOL can be a very complex and personal decision for both the patient and the surgeon. While all US Food and Drug Administration–approved presbyopia-correcting lenses are well made and high quality, no IOL performs well in every situation. Things that need to be considered when making this decision include the evidence on the comparative effectiveness of various IOLs, the health of the eye, the presence of any ocular comorbidities, ocular surgical history, and the visual needs of the patient. Finally, patients face growing cost-sharing pressure when choosing a presbyopia-correcting lens. Because multifocal and trifocal lenses are not covered by insurance in the United States, the out-of-pocket expenses can be significant. Even though most patients find the money to be well spent, the financial aspects act as a barrier to entry for many patients, which could exacerbate socioeconomic disparities in cataract surgery and cataract surgery outcomes. For these reasons, there is no “one-size-fits-all” option for every patient when it comes to choosing an IOL.
In summary, Cho and colleagues have made an effort to assess the visual outcomes of various types of multiple and monofocal IOLs in presbyopia-correcting cataract surgery. The treatment rankings must be interpreted cautiously as the treatment at the top of the ranking may not reflect the “best clinical choice.” Rankings must be considered together with relative treatment effects, quality of the evidence, and patient preference.
Conflict of Interest Disclosures:
Dr Li reported grant UG1EY020522 from the National Eye Institute. Dr Davidson reported consulting fees from Alcon, Allergan/AbbVie, Avellino Labs, Centricity, Dompe, EyePoint, Eyevance, Glaukos, Johnson & Johnson, Kala, Novartis, Orasis, Oyster Point, Sun Pharma, Tarsus, and Zeiss. No other disclosures were reported.
References
- 1.Cho JY, Won YK, Park J, et al. Visual outcomes and optical quality of accommodative, multifocal, extended depth-of-focus, and monofocal intraocular lenses in presbyopia-correcting cataract surgery: a systematic review and bayesian network meta-analysis. JAMA Ophthalmol. Published online September 22, 2022. doi: 10.1001/jamaophthalmol.2022.3667 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Chaimani A, Porcher R, Sbidian É, Mavridis D. A Markov chain approach for ranking treatments in network meta-analysis. Stat Med. 2021;40(2):451–464. doi: 10.1002/sim.8784 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Chiocchia V, White IR, Salanti G. The complexity underlying treatment rankings: how to use them and what to look at. BMJ Evid Based Med. 2022;bmjebm-2021-111904. doi: 10.1136/bmjebm-2021-111904 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Salanti G, Del Giovane C, Chaimani A, Caldwell DM, Higgins JP. Evaluating the quality of evidence from a network meta-analysis. PLoS One. 2014;9(7):e99682. doi: 10.1371/journal.pone.0099682 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Puhan MA, Schünemann HJ, Murad MH, et al. ; GRADE Working Group. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ. 2014;349:g5630. doi: 10.1136/bmj.g5630 [DOI] [PubMed] [Google Scholar]
- 6.Evans JR, de Silva SR, Ziaei M, Kirthi V, Leyland MD. Outcomes in randomised controlled trials of multifocal lenses in cataract surgery: the case for development of a core outcome set. Br J Ophthalmol. 2020;104(10):1345–1349. doi: 10.1136/bjophthalmol-2019-315410 [DOI] [PubMed] [Google Scholar]
