Abstract
Background
Network meta-analysis compares multiple treatment options for the same condition and may be useful for developing clinical practice guidelines.
Purpose
To compare treatment recommendations for first-line medical therapy for primary open angle-glaucoma (POAG) from major updates of American Academy of Ophthalmology (AAO) guidelines with the evidence available at the time, using network meta-analysis.
Data Sources
MEDLINE, Embase, and the Cochrane Library were searched on 11 March 2014 for randomized, controlled trials (RCTs) of glaucoma monotherapies compared with placebo, vehicle, or no treatment or other monotherapies. The AAO Web site was searched in August 2014 to identify AAO POAG guidelines.
Study Selection
Eligible RCTs were selected by 2 independent reviewers, and guidelines were selected by 1 person.
Data Extraction
One person abstracted recommendations from guidelines and a second person verified. Two people independently abstracted data from included RCTs.
Data Synthesis
Guidelines were grouped together on the basis of literature search dates, and RCTs that existed at 1991, 1995, 1999, 2004, and 2009 were analyzed. The outcome of interest was intraocular pressure (IOP) at 3 months. Only the latest guideline made a specific recommendation: prostaglandins. Network meta-analyses showed that all treatments were superior to placebo in decreasing IOP at 3 months. The mean reductions (95% credible intervals [CrIs]) for the highest-ranking class compared with placebo were as follows: 1991: β-blockers, 4.01 (CrI, 0.48 to 7.43); 1995: α2-adrenergic agonists, 5.64 (CrI, 1.73 to 9.50); 1999: prostaglandins, 5.43 (CrI, 3.38 to 7.38); 2004: prostaglandins, 4.75 (CrI, 3.11 to 6.44); 2009: prostaglandins, 4.58 (CrI, 2.94 to 6.24).
Limitation
When comparisons are informed by a small number of studies, the treatment effects and rankings may not be stable.
Conclusion
For timely recommendations when multiple treatment options are available, guidelines developers should consider network meta-analysis.
Primary Funding Source
National Eye Institute, National Institutes of Health.
In 2011, the Institute of Medicine defined clinical practice guidelines as “statements that include recommendations intended to optimize patient care, that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options” (1). Historically, guidelines primarily represented the opinions of individual authors or the consensus of experts (2). With the advent of evidence-based health care, guidelines have increasingly used systematic reviews and meta-analyses of randomized, controlled trials (RCTs) to form the basis of recommendations (2–4). Standard meta-analytic techniques can be used if the guideline addresses pairwise comparisons–for example, treatment A versus treatment B. If a guideline is attempting to address the question of which treatment is best among multiple options, however, standard meta-analysis may not be adequate. By contrast, network meta-analysis–a method that uses information from both direct and indirect comparisons and makes inferences about the comparative effectiveness of all the treatments of interest in a single analysis (5, 6)–is particularly suited in such situations.
Clinical conditions for which guidelines could benefit from network meta-analysis the most are those with numerous treatment options, such as first-line medical treatment of primary open-angle glaucoma (POAG). In this condition, which is highly prevalent in the United States and worldwide, optic nerve damage leads to gradual and painless visual field reduction over time (7, 8). Because optic nerve damage is difficult to measure and changes in visual field take years to develop, treatment effectiveness is generally determined by reduction in intraocular pressure (IOP), a modifiable risk factor for POAG over a period of a few months (7, 9).
The American Academy of Ophthalmology (AAO) POAG Preferred Practice Pattern (PPP) has been particularly influential in the United States (7, 10–17). The first version of this guideline was published in 1989, and major revisions have since been published approximately every 3 to 5 years. When the AAO PPP guideline was first developed by AAO's Glaucoma Panel, evidence was gathered on the basis of the panel members' knowledge: Members submitted what they considered seminal works, and these works were distributed among the rest of the panel (18). Since 1996, the panel has been using a more systematic approach, carrying out a formal search of the relevant scientific literature and rating the strength of evidence for recommendations (7, 13–17).
The objective of this study is to compare the evidence base for first-line medical treatments of POAG with the recommendations for each major revision of the AAO PPP by using cumulative network meta-analysis (that is, conducting a series of network meta-analyses on a systematically assembled set of RCTs published up to several distinct periods). Previously, Antman and Lau demonstrated, by comparing the results from cumulative pairwise meta-analyses with recommendations given by experts, that meta-analysis can improve the timeliness of guidance (19, 20). Using this previous work as a model, we evaluated whether network meta-analysis can provide additional benefit in developing clinical practice guidelines. The data for our cumulative network meta-analysis are from a systematic review and network meta-analysis we previously published (21). This study is not intended as criticism of guideline developers for not using statistical methods that were undeveloped at times in the past but as an example to show how network meta-analysis may be able to benefit future guideline recommendations.
Methods
Guideline Identification and Extraction
We searched the AAO Web site (www.aao.org) and contacted the AAO's librarian to identify all versions of AAO's PPPs in August 2014. One member of the team (B.R.) reviewed each version of the guideline, identified statements concerning first-line POAG medical treatment (that is, as topical monotherapy for decreasing IOP [22]), and identified among them the recommendations. We defined recommendations as statements that used the words “recommend,” “should,” “appropriate,” “necessary,” “must,” or other words that suggested a particular practice, such as prescribing a medication. A second author (T.L.) verified the abstraction and the classification of whether a statement was really a “recommendation.” We then categorized recommendations by drug name and class of medical treatment (for example, latanoprost, prostaglandins) and extracted the quantitative estimates of effect (for example, reduction of IOP) when provided. We also extracted the ratings of strength of evidence for each recommendation (for example, level I indicates that the basis is “a high-quality large RCT” or a “systematic review,” and level III indicates that the basis is “consensus of experts”) (7, 13–17).
When 2 or more consecutive guideline versions reported the same literature search years or, in absence of reporting search years, they presented identical recommendations regarding medical treatment, we grouped them together. This was done to facilitate the comparison of the guidelines recommendations with the results from our cumulative network meta-analyses.
Systematic Review and Cumulative Network Meta-analysis
We identified all available RCTs from a systematic review our group recently conducted (21). In this review, we searched Cochrane Register of Controlled Trials (CENTRAL) in The Cochrane Library, MEDLINE, and Embase, on 11 March 2014 (Supplement 1, available at www.annals.org, shows the full search strategy), and included RCTs evaluating first-line topical monotherapies for POAG or ocular hypertension in comparison with no treatment, placebo, or other topical monotherapies. The process of title and abstract screening, full-text screening, data abstraction, and risk of bias evaluation has been described previously (21). All data were extracted into the Systematic Review Data Repository (23, 24).
For this article, we used the latest guideline in each set to define eligible studies for each network meta-analysis. Eligible studies are those published up to the stopping year for the literature search reported in the guidelines or, if such a point was not reported, the year before the guideline were published, to allow for lag time between publication and inclusion of evidence in the guideline. The primary outcome was the mean IOP at 3 months as a continuous variable in units of mm Hg, which corresponds to the primary effectiveness end point on which guideline recommendations were made (7). We prioritized using mean change in IOP from baseline values, but we also accepted mean IOP at 3 months when the change score was not reported (25).
Our analysis did not distinguish between drug concentrations, and comparisons were based on the active ingredient and class of that ingredient. We first examined direct comparisons using random-effects model meta-analysis assuming comparison-specific heterogeneity and a common heterogeneity across all comparisons at both the drug and class level. To assess the statistical heterogeneity, we examined the I2 and τ2 values for these models. Analyses for direct comparisons were conducted with Stata software, version 13, using the metan command.
We fitted Bayesian random-effects network meta-analysis models using WinBUGS 1.4.3 (26–28). We used a 3-level hierarchical model with components at the following levels: study, individual drug, and drug class. This model accounts for the within-study correlation of multigroup trials and also incorporates class effect (26, 27, 29). A valid network analysis requires the assumption of transitivity (that is, there are no systematic differences among the trials other than the treatments being compared) (5). This assumption can be tested by assessing inconsistency, the statistical disagreement between direct and indirect comparisons (5) (Supplement 2, available at www.annals.org).
We examined mean differences in IOP (and 95% credible intervals [CrIs]) between pairs of individual drugs and drug classes (21). We also ranked each drug or class (for example, the probability of a drug being the most effective treatment or the second best). We examined the hierarchy of treatment rankings by using the surface below the cumulative ranking curve (SUCRA) (30, 31). A SUCRA value (or percentage) gives the probability that a treatment is among the most effective treatments, with a value of 1 (or 100%) meaning that a treatment is certain to be the most effective of treatments in the network and a value of 0 (or 0%) meaning that a treatment is certain to be the least effective. Rankings based on SUCRA values are considered to take into account uncertainty in estimated treatment effects better than do general ranking probabilities (30, 31).
Guidelines and Network Meta-analysis Comparison
We compared information extracted from each guideline set to the results of the corresponding network meta-analysis. We assessed whether the recommended drugs or drug classes and effectiveness estimates in the guideline match with the highest-ranking drug or drug class, determined by SUCRA values, from the network meta-analysis.
Comparison With Published Pairwise Meta-analyses
To determine whether network meta-analysis gives incremental information to guideline developers that cannot be gained from pairwise meta-analyses, we examined the results of published, high-quality systematic reviews and pairwise meta-analyses identified previously (32). We matched the pairwise results to the network meta-analysis results on the basis of which interventions were compared and the year of publication. We examined agreement of the findings from the 2 approaches qualitatively. For example, when the 95% confidence interval (CI) covers the null value, we concluded that one drug is not superior to another drug.
Role of the Funding Source
The project was funded by the National Eye Institute, National Institutes of Health. The sponsor had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript.
Results
Guideline Identification and Extraction
We identified 9 versions of the AAO's POAG PPP: 1989, 1990, 1992, 1996, 2000, 2003, 2005, 2006, and 2010 (7, 10–17). According to literature search years and recommendations relevant to POAG medical therapies, we grouped the guidelines together into 5 sets: 1989-1992, 1996, 2000-2003, 2005-2006, and 2010. Of these guideline sets, only 2010 made recommendations about a specific first-line medical therapy (Appendix Table 1, available at www.annals.org). The 2010 guideline stated, on the basis of a meta-analysis of 11 trials (33), that “Prostaglandin analogs are the most effective drugs at lowering IOP and can be considered as initial medical therapy.” No other guideline made specific recommendations at a class or drug level for POAG, although all guidelines recommended medical treatment, in general, as initial therapy. For example, the 2005-2006 guidelines, without citing any literature, stated, “in many instances, topical medications constitute effective initial therapy.” Related recommendations were about considerations in choosing initial care or for monitoring the effect of medical interventions on IOP.
The 1989-1992 and 1996 guideline sets did not report search years; the 2000-2003, 2005-2006; and 2010 sets did: The stopping years were 1999, 2004, and 2009, respectively. Accordingly, 5 separate network meta-analyses were conducted by using studies published up to 1991, 1995, 1999, 2004, and 2009. From here on, we refer to the network meta-analyses based on the last trial publication date. For example, we refer to the network meta-analysis of trials published up to 1995 as the 1995 network meta-analysis.
Network Meta-analysis
Search Result, General Study Characteristics, and Risk of Bias
Of the 10 936 unique records previously identified (21), 91 RCTs met the eligibility criteria for the current study (Figure 1 of the Supplement and Supplement 3, available at www.annals.org). The first trial was published in 1983 and the latest in 2009 (Appendix Table 2, available at www.annals.org, for the characteristics of the trial networks for each analysis year). Later networks include trials that were generally larger, more often multicenter, and of shorter duration than trials included in earlier networks. The proportion of trials categorized with unclear risk of bias seems to decrease in later networks, indicating that reporting of trial quality may have improved over time. The risk of bias of included studies is reported in Appendix Table 2.
Interventions
Overall, included trials studied 12 drugs from 4 classes (α2-adrenergic agonists, β-blockers, prostaglandins, and carbonic anhydrase inhibitors) as well as placebo, vehicle, or no treatment (Figure 2 of the Supplement). Up to 1991, 3 active drugs (betaxolol, levobunolol, and timolol) from 1 class (β-blockers) and placebo were studied in RCTs. By 1995, an additional 5 drugs (apraclonidine, carteolol, dorzolamide, latanoprost, and unoprostone), with at least 1 drug from each class, were studied in trials. The 1999 network includes a total of 10 active drugs with the addition of brimonidine and brinzolamide. Both the 2004 and 2009 networks include all active drugs. Many direct comparisons between drugs, such as latanoprost versus placebo, have not been made in trials, even by 2009, and for the direct comparisons that have been made, there are often only 1 or 2 trials (Figure 3 of the Supplement).
Network Meta-analysis Outcomes
The results of our network meta-analyses indicate that all interventions were superior to placebo, vehicle, or no treatment in decreasing 3-month IOP for all analysis years (Figure 1 and Figure 4 of the Supplement). The mean reduction (95% CrI) for the class with the highest-ranking SUCRA value compared with placebo, vehicle, or no treatment at each analysis year was as follows: 1991: β-blockers, 4.01 (CrI, 0.48 to 7.43); 1995: α2-adrenergic agonists, 5.64 (CrI, 1.73 to 9.50); 1999: prostaglandins, 5.43 (CrI, 3.38 to 7.38); 2004: prostaglandins, 4.75 (CrI, 3.11 to 6.44); 2009: prostaglandins, 4.58 (CrI, 2.94 to 6.24). The drug with the highest ranking at each analysis year was as follows: 1991: levobunolol, 4.53 (CrI, 3.31 to 5.79); 1995: levobunolol, 5.36 (CrI, 4.30 to 6.41); 1999: latanoprost, 5.89 (CrI, 4.66 to 7.14); 2004: bimatoprost, 5.87 (CrI, 4.67 to 7.06); 2009: bimatoprost, 5.87 (CrI, 4.96 to 6.77). Point estimates for drug and class effects seem to attenuate, and the CrIs become narrower over time (Figure 1).
Figure 1.
Network meta-analysis treatment effect estimates relative to placebo for each analysis year.
Rankings based on cumulative ranking probabilities from SUCRA plots were generally consistent with effect estimates (Figure 2 and Figure 5 of the Supplement). The only time at which the highest cumulative rank did not match treatment effect was in the 1995 network, in which apraclonidine had the highest mean effect but levobunolol had the highest cumulative ranking. In the 2004 and 2009 networks, rankings remained stable for both drugs and classes. Sometimes, when 2 drugs were included at the same time point, they crossed in cumulative rank at subsequent points (Figure 2).
Figure 2.
Relative ranking over time based on SUCRA value.
Guideline and Network Meta-analysis Comparison
The Table summarizes the comparison between AAO clinical practice guideline recommendations and network meta-analytic findings. On the basis of our cumulative network-meta-analyses, quantitative evidence of treatment effect could have informed recommendations on specific treatments at both the drug and class level for all guideline sets if network meta-analysis methods had been available. Although the 2010 guideline recommended prostaglandins as first-line treatment for POAG, the AAO PPP might have made this recommendation earlier on the basis of network meta-analysis results (Table). The first prostaglandin drug was approved by the U.S. Food and Drug Administration in 1996 and was first mentioned in the 1996 PPP as a treatment option (13).
Table.
Comparison Between the American Academy of Ophthalmology Preferred Practice Pattern Guidelines Recommendations and Network Meta-analysis Results
| Guideline Sets | Guideline First-Line Therapy Recommendation (Estimated IOP Reduction) | Best-Ranking* Drug Class and Drug IOP Reduction Relative to Placebo, Vehicle, or No Treatment (95% Credible Interval), mm Hg |
|
|---|---|---|---|
| Class | Drug | ||
| 1989, 1990, 1992 | NR | β-Blockers: 4.01 (0.48–7.43) | Levobunolol: 4.53 (3.31–5.79) |
| 1996 | NR | α2-Agonists: 5.64 (1.73–9.50) | Levobunolol: 5.36 (4.30–6:41) |
| 2000, 2003 | NR | Prostaglandins: 5.43 (3.38–7.38) | Latanoprost: 5.89 (4.66–7.14) |
| 2005, 2006 | NR | Prostaglandins: 4.75 (3.11–6.44) | Bimatoprost: 5.87 (4.67–7.06) |
| 2010 | Prostaglandins: 25%–33% | Prostaglandins: 4.58 (2.94–6.24) | Bimatoprost: 5.87 (4.96–6.77) |
IOP = intraocular pressure; NR = no recommendation.
Ranking based on surface under the cumulative ranking curve values.
Comparison With Published Pairwise Meta-analyses
Compared with the 78 drug comparisons that we were able to make in our network meta-analysis, only 6 pairwise comparisons from 4 published high-quality systematic reviews and meta-analyses were identified (34–37) (Figure 3 and Appendix Table 3, available at www.annals.org, which shows the characteristics of comparisons from identified systematic reviews). Because all identified systematic reviews had literature searches conducted in 2005 or 2006, we compared their results with those from our 2009 network meta-analysis. Of these 6 comparisons, the network meta-analysis findings were different for 2. In both cases (latanoprost vs. brimonidine and bimatoprost vs. travoprost), 2 pairwise systematic reviews were on the same topic, with 1 arriving at the same conclusion as our network meta-analysis and 1 arriving at a different conclusion (34, 35). For example, the mean IOP reductions (95% CrI or CI) for latanoprost versus brimonidine from each source were as follows: our analysis: 1.22 (0.56 to 1.88) (latanoprost superior); Hodge and colleagues: 1.10 (0.57 to 1.63) (latanoprost superior) (37); and Li and colleagues: 1.04 (−0.91 to 3.01) (no superiority shown) (34).
Figure 3.
Comparison between network meta-analysis and published pairwise meta-analysis results.
Discussion
We identified 5 sets of guidelines from AAO's POAG PPPs. Specific treatment recommendations were made only in the last update (2010 guideline). Using cumulative network meta-analyses of the RCTs available at the time, we determined which drug and drug class had the greatest IOP-decreasing effect at the time of each major revision. Both the final 2010 guideline and the corresponding network meta-analysis indicate that prostaglandins should be considered first-line treatment in terms of IOP reduction. It is worth noting that, had network meta-analysis been available to guideline developers, prostaglandins, which are now the standard treatment, may have been recommended as early as the 2000 update.
The AAO's POAG PPPs up to 2010 did not give recommendations at the drug level. This may be because the guideline producers did not want to appear to favor a particular drug manufacturer, since some glaucoma drugs, such as bimatoprost, are still under patent. The cultures of other clinical areas or different glaucoma guideline groups may lead them to have different approaches to making treatment recommendations (for example, at the drug level rather than at the class level) (9). Our results indicate that drugs within a class generally have similar effects on IOP. A notable exception is unoprostone, which was the least effective drug in the 2004 and 2009 network meta-analyses despite the high ranking of all other prostaglandins. Indeed, there is uncertainty as to whether unoprostone should be classified as a prostaglandin analogue (38, 39).
Systematic reviews underpin trustworthy clinical practice guidelines (1); however, with the increasing number of competing alternatives available for a given condition, traditional pairwise meta-analysis techniques do not meet the need. This is because pairwise meta-analysis compares only interventions that have been directly evaluated in individual trials. As illustrated in Figure 2 of the Supplement, direct evidence obtained from RCTs was available for only one third of all possible pairwise comparisons of first-line medical treatments for POAG, which limited the potential of evidence synthesis and treatment recommendations. Additionally, as shown in our example, fewer than 10% of the 78 possible pairwise comparisons in 2009 were evaluated in published, high-quality, pairwise systematic reviews, and the conclusions were discordant for 2 of the 6 comparisons.
Without information for all possible direct comparisons, it would be difficult for guideline developers to compare all interventions with one another and form coherent recommendations on the basis of pairwise meta-analysis alone. Since Antman and Lau's landmark cumulative meta-analysis more than 20 years ago (19, 20), the statistical methods for systematic reviews have evolved to allow us to extend their methods to the comparison of multiple treatments in a single analysis to facilitate timely recommendations. Network meta-analysis allows all treatments to be compared to one another and ranked, facilitating the selection of a preferred treatment for a specific condition. Even older interventions, now rarely used, can be included in the analyses to assess whether their disuse was well-founded or whether their use might be resurrected in certain circumstances.
There are several considerations before findings from network meta-analysis are adopted in the making of guideline recommendations. First, it may not be appropriate to make a recommendation on the basis of only the initial studies of a new treatment because their estimates may be less certain and may overestimate the true treatment effect. In our example, the estimated treatment effects fade over time, which is consistent among different drugs and classes (Figure 1). This phenomenon of diminishing effects has been noted in previous studies (40, 41), with potential explanations being time-lag bias, publication bias, small study effects, change in study quality, and heterogeneity in the clinical population (41, 42).
Second, a network meta-analysis conducted as part of guideline development should consider both effectiveness and safety outcomes because both are important to patients, their caregivers, and their physicians (6, 43, 44). Our analysis, conducted mainly to explore the potential utility of the method of network meta-analysis, considered only 1 factor, intraocular pressure, an effectiveness outcome relevant to the recommendation process.
Finally, guidelines producers and clinicians should be cautious in applying the findings of any network meta-analysis, related to the potential limitations of the statistical methods. For example, the validity of the results of a network meta-analysis depends on the validity of the assumptions being made, the results depend on the network definitions applied, and the treatment rankings are associated with uncertainty. Although these issues have been discussed extensively in the network meta-analysis literature (5, 6, 45, 46), they may not be familiar to guidelines producers and clinicians.
In recent years, network meta-analysis has begun to be recognized as a useful tool for guideline developers. The Endocrine Society commissioned a network meta-analysis to inform recommendations of treatment for its 2012 clinical practice guideline for osteoporosis in men (47, 48). The National Institute for Health and Clinical Excellence in the United Kingdom also used network meta-analysis for developing recommendations for its 2013 neuropathic pain treatment guideline (49) and 2014 bipolar disorder guideline (50). On the basis of these examples and our experience, we provide recommendations for guideline developers who seek to conduct or use a network meta-analysis.
First, always work with statisticians and methodologists who understand the methods for network analysis from the outset. Network meta-analysis methods can be complex and include many nuances. The validity of the finding relies on a careful assessment of the transitivity assumption when forming the network and later in the analysis. Whether conducting or commissioning a network meta-analysis, developers should ensure that the review team includes experts in network meta-analysis methods.
Second, define treatment networks explicitly. The nodes in the networks should represent available treatment options in clinical practice. In network meta-analyses of drug interventions, for example, nodes may be defined as each dose of each drug, any dose of each drug with all doses merged, or each class with all drugs merged, depending on the question of interest and the biological or clinical appropriateness of merging the different treatments. In addition, so that the analysis is most clinically informative, all potential treatment options that are suitable or indicated for patients with a given condition should be considered. Finally, even if an intervention is not of direct clinical interest (for example, placebo), it may still be included in the analysis to inform the comparisons. Similar to a pairwise meta-analysis, depending on how interventions are defined and which studies are included, the findings from network meta-analysis may vary.
Third, analyze outcomes that matter to patients. Patients may value treatment safety as much as or more than treatment effectiveness. Analyses that consider both effectiveness and safety outcomes allow for better understanding of treatment applicability in clinical settings. In addition, clinical outcomes and patient-reported outcomes usually are more important to patients than surrogate outcome. For example, the effectiveness outcome in our analysis, IOP, is understood as a surrogate outcome for visual function with conflicting evidence supporting their relationship (51–53). Despite this, IOP served as the basis for recommendations in the AAO PPPs and was the primary determinant of effectiveness in trials, whereas more relevant outcomes, such as visual field, were often not even measured in these primary studies.
Fourth, use the ranking statistics that account for the uncertainty in ranking and interpret ranking to gether with the size of treatment effect. Ranking based on SUCRA values is preferred to crude ranking because it summarizes the estimated probabilities for all possible ranks (30, 31). Even when an appropriate measure is used, however, the highest-ranking treatment may have a modest or insignificant clinical effect, and therefore rankings need to be interpreted in the context of the size of treatment effect.
Fifth, interpret the findings carefully in the case of insufficient data, a large amount of heterogeneity or inconsistency, or data of poor quality. When comparisons are informed by a small number of studies, the treatment effects and rankings may not be stable. The guideline developers should consider the potential for clinical or methodological heterogeneity or inconsistency and risk of bias that may affect the results, and apply appropriate caution in the interpretation of the findings.
In conclusion, our example of IOP in POAG showed that, had network meta-analysis been available, the AAO POAG PPP may have recommended prostaglandins, the current first-line treatment, earlier. When many different treatment options are available, guideline developers may wish to go beyond pairwise meta-analyses because pairwise meta-analyses are limited by treatments that have been compared directly in individual studies. Guideline developers should consider working with trained methodologists to conduct network meta-analysis of all relevant outcomes and using the results of network meta-analyses to inform future clinical recommendations.
Supplementary Material
Acknowledgment
The authors thank Laurie Bagley for providing the 1989-2006 versions of the American Academy of Ophthalmology's Primary Open Angle Glaucoma Preferred Practice Patterns; Hwanhee Hong for her help in developing the models used in this study; Alfred Sommer for providing information about the history of the Primary Open-Angle Glaucoma Preferred Practice Pattern guidelines; Roberta Scherer for her feedback on previous versions of the manuscript; and Georgia Salanti, Huseyin Naci, Deborah Caldwell, Stefan K. Leucht, and Toshiaki A. Furukawa for contributing insights to the design of the project.
Grant Support: By grant 1 RC1 EY020140 and grant 1 U01 EY020522, National Eye Institute, National Institutes of Health (principal investigator: Dr. Kay Dickersin). Dr. Andrea Cipriani is supported by the National Institute for Health Research Oxford Cognitive Health Clinical Research Facility.
Appendix Table 1.
Recommendations From the American Academy of Ophthalmology Preferred Practice Pattern Guidelines for Primary Open-Angle Glaucoma
| Guideline | Years of Literature Searched | Recommendations* Relevant to First-Line Topical Medical Treatment | Does Recommendation Concern a Specific Medical Treatment? | Interpretation of Recommendation | Guideline Rating for Strength of Evidence |
|---|---|---|---|---|---|
| 1989, 1990, 1992 | None specified | “While the choice of initial therapy depends on numerous considerations, in most instances one begins with topical medications.” | No | Recommendation for medical treatment as initial therapy | None |
| 1989, 1990, 1992 | None specified | “To determine the effectiveness of topical therapy, it is necessary to distinguish between the therapeutic impact of an agent on lOP and ordinary background fluctuations of lOP.” | No | Recommendation for monitoring the effects of intervention on IOP | None |
| 1996 | “Since 1985” | “The choice of initial therapy depends on numerous considerations, and discussion of treatment should include all options.” | No | Recommendation for considerations in choosing initial care | III |
| 1996 | “Since 1985” | “In most instances, topical medications are initial therapy.” | No | Recommendation for medical treatment as initial therapy | III |
| 1996 | “Since 1985” | “To determine the effectiveness of topical therapy, it is necessary to distinguish between the therapeutic impact of an agent on lOP and ordinary background fluctuations of lOP.” | No | Recommendation for monitoring the effects of intervention on IOP | None |
| 2000, 2003 | 1995–1999 | “The choice of initial therapy depends on numerous considerations, and discussion of treatment should include all options.” | No | Recommendation for considerations in choosing initial care | III |
| 2000, 2003 | 1995–1999 | “In most instances, topical medications constitute initial therapy.” | No | Recommendation for medical treatment as initial therapy | III |
| 2000, 2003 | 1995–1999 | “To determine the effectiveness of topical therapy, it is necessary to distinguish between the therapeutic impact of an agent on lOP and ordinary background fluctuations of lOP.” | No | Recommendation for monitoring the effects of intervention on IOP | None |
| 2005, 2006 | 1999–2004 | “The choice of initial therapy depends on numerous considerations, and discussion of treatment with the patient should include appropriate options.” | No | Recommendation for considerations in choosing initial care | III |
| 2005, 2006 | 1999–2004 | “In many instances, topical medication constitute effective initial therapy.” | No | Recommendation for medical treatment as initial therapy | None |
| 2005, 2006 | 1999–2004 | “To determine the effectiveness of topical therapy, it is necessary to distinguish between the therapeutic impact of an agent on IOP and ordinary background fluctuations of IOP.” | No | Recommendation for monitoring the effects of intervention on IOP | None |
| 2010 | 2004–2009 | “The choice of initial therapy depends on numerous considerations, and discussion of treatment with the patient should include the relative risks and benefits of the three options.” | No | Recommendation for considerations in choosing initial care | III |
| 2010 | 2004–2009 | To determine the effectiveness of topical therapy, it is necessary to distinguish between the therapeutic impact of an agent on IOP and ordinary background fluctuations of IOP.” | No | Recommendation for monitoring the effects of intervention on IOP | None |
| 2010 | 2004–2009 | “Prostaglandin analogs are the most effective drugs at lowering IOP and can be considered as initial therapy unless other considerations such as cost, side effects, intolerance, or patient refusal preclude this.” | Yes | Recommendation for prostaglandin class as initial medical therapy | I† |
IOP = intraocular pressure.
Any statement that uses “recommend”; “should”; “appropriate”; “necessary”; “must”; or words suggesting a particular practice, such as prescribing a medication, is considered a recommendation.
Appendix Table 2.
Characteristics and Risk of Bias of Networks
| Variable | Analysis Year |
||||
|---|---|---|---|---|---|
| 1991 | 1995 | 1999 | 2004 | 2009 | |
| Characteristics of the trial network | |||||
| Trials, n | 18 | 29 | 48 | 76 | 91 |
| Total participants, n | 1,161 | 2,641 | 5,960 | 10,717 | 13,870 |
| Median trial sample size (IQR), n | 69 (28 to 85) | 72 (42 to 137) | 76 (41 to 159) | 91 (43 to 195) | 90 (47 to 213) |
| Reported as multicenter, n (%) | 7 (39) | 16 (55) | 33 (69) | 49 (64) | 56 (62) |
| Median trial length (IQR), mo | 6 (3 to 15) | 6 (3 to 12) | 3 (3 to 12) | 3 (3 to 6) | 3 (3 to 6) |
| Reported region of recruitment, n (%) | |||||
| Yes | 8 (44) | 14 (48) | 30 (63) | 51 (67) | 62 (68) |
| North America*† | 5 (63) | 8 (57) | 17 (57) | 27 (53) | 32 (52) |
| Latin America*† | 0 (0) | 1 (7) | 1 (3) | 2 (4) | 3 (5) |
| Europe*† | 0 (0) | 2 (14) | 7 (23) | 14 (27) | 15 (24) |
| Africa*† | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) |
| Asia*† | 1 (13) | 4 (29) | 7 (23) | 11 (22) | 14 (23) |
| Oceania*† | 0 (0) | 1 (7) | 1 (3) | 1 (2) | 2 (3) |
| No | 10 (56) | 15 (52) | 18 (38) | 25 (33) | 29 (32) |
| Risk of bias | |||||
| Random sequence generation, n (%) | |||||
| Low | 2 (11) | 6 (21) | 14 (29) | 28 (37) | 37 (41) |
| Unclear | 16 (89) | 23 (79) | 34 (71) | 48 (63) | 54 (59) |
| High | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) |
| Allocation concealment, n (%) | |||||
| Low | 3 (17) | 5 (17) | 8 (17) | 17 (22) | 24 (26) |
| Unclear | 15 (83) | 24 (83) | 40 (83) | 59 (78) | 67 (74) |
| High | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) |
| Masking of participants, n (%) | |||||
| Low | 6 (33) | 12 (41) | 20 (42) | 30 (39) | 36 (40) |
| Unclear | 10 (56) | 14 (48) | 24 (50) | 31 (41) | 39 (43) |
| High | 2 (11) | 3 (10) | 4 (8) | 15 (20) | 16 (18) |
| Masking of outcome assessor for IOP, n (%) | |||||
| Low | 3 (17) | 4 (14) | 6 (13) | 14 (18) | 19 (21) |
| Unclear | 15 (83) | 25 (86) | 42 (88) | 57 (75) | 65 (71) |
| High | 0 (0) | 0 (0) | 0 (0) | 5 (7) | 7 (8) |
| Reported funding source, n (%) | |||||
| Yes | 6 (33) | 11 (38) | 21 (44) | 42 (55) | 50 (55) |
| Industry funding*‡ | 6 (100) | 11 (100) | 21 (100) | 41 (98) | 48 (96) |
| Government funding*‡ | 2 (33) | 3 (27) | 3 (27) | 5 (12) | 6 (12) |
| No | 12 (67) | 18 (62) | 27 (56) | 34 (45) | 41 (45) |
| Reported author financial conflicts of interest, n (%) | |||||
| Yes | 6 (33) | 12 (41) | 22 (56) | 33 (44) | 38 (42) |
| Conflict of interest for at least 1 author* | 6 (100) | 12 (100) | 18 (82) | 24 (73) | 28 (74) |
| No conflicts of interest* | 0 (0) | 0 (0) | 4 (18) | 9 (27) | 10 (26) |
| No | 12 (67) | 17 (59) | 26 (54) | 43 (56) | 53 (58) |
IOP = intraocular pressure; IQR = interquartile range.
Percent denominator is n for “Yes.”
Trials could report more than 1 region of recruitment.
Trials could report more than 1 funding source.
Appendix Table 3.
Characteristics of Comparisons Identified in Published Pairwise Systematic Reviews
| Pairwise Systematic Review, Year (Reference) |
Literature Search Year |
Participants | Intervention | Comparison | Outcome | Time Point | Narrative Findings | Mean Difference (95% CI), mm Hg |
Studies Used in Comparison, n |
Participants, n |
|---|---|---|---|---|---|---|---|---|---|---|
| Li et al, 2006 (34) | 2005 | Patients with OHT or POAG | Travoprost | Timolol | IOP value | Pooled over treatment visits (value at last visit if pooled data not available) | Travoprost is more effective than timolol in lowering IOP | −0.81 (−1.16 to −0.45) | 4 | 1354 |
| 2005 | Patients with OHT or POAG | Travoprost | Bimatoprost | IOP value | Pooled over treatment visits (value at last visit if pooled data not available) | Travoprost seems equivalent to bimatoprost in lowering IOP | −0.08 (−0.62 to 0.79) | 5 | 402 | |
| 2005 | Patients with OHT or POAG | Travoprost | Latanoprost | IOP value | Pooled over treatment visits (value at last visit if pooled data not available) | Travoprost seems equivalent to latanoprost in lowering IOP | −0.57 (−1.18 to 0.04) | 6 | 912 | |
| Fung et al, 2007 (35) | 2006 | Patients with OHT, POAG, or NTG | Latanoprost | Brimonidine | IOP reduction | Trial end point | Latanoprost is more effective than brimonidine in lowering IOP | −1.10 (−1.63 to −0.57) | 14 | 1725 |
| Aptel et al, 2008 (36) | 2006 | Patients with OHT or POAG | Bimatoprost | Latanoprost | IOP reduction | 3 months (or between 1 and 6 months if not available) (Morning IOP) | Bimatoprost is more effective than latanoprost in lowering IOP | −0.50 (−0.99 to −0.01) | 5 | 893 |
| 2006 | Patients with OHT or POAG | Bimatoprost | Travoprost | IOP reduction | 3 months (or between 1 and 6 months if not available) (Morning IOP) | Bimatoprost is more effective than travoprost in lowering IOP | −1.02 (−1.72 to −0.32) | 3 | 458 | |
| 2006 | Patients with OHT or POAG | Latanoprost | Travoprost | IOP reduction | 3 months (or between 1 and 6 months if not available) (Morning IOP) | Latanoprost seems equivalent to travoprost in lowering IOP | −0.40 (−1.29 to 0.40) | 3 | 458 | |
| Hodge et al, 2008 (37) | 2006 | Patients with OHT or glaucoma | Latanoprost | Brimonidine | IOP reduction | 3 months | Latanoprost seems equivalent to brimonidine in lowering IOP | −1.04 (−3.01 to 0.91) | 3 | 451 |
| 2006 | Patients with OHT or glaucoma | Latanoprost | Dorzolamide | IOP reduction | 3 months | Latanoprost is more effective than dorzolamide in lowering IOP | −2.64 (−3.25 to −2.04) | 3 | 328 |
IOP = intraocular pressure; NTG = normal-tension glaucoma; OHT = ocular hypertension; POAG = primary open-angle glaucoma.
Footnotes
Disclosures: Mr. Rouse and Mr. Shi report grants from the National Eye Institute, National Institutes of Health, during the conduct of the study. Dr. Dickersin reports grants from National Eye Institute during the conduct of the study. Authors not named here have disclosed no conflicts of interest. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M15-2367.
Current author addresses and author contributions are available at www.annals.org.
Author Contributions: Conception and design: B. Rouse, A. Cipriani, T. Li.
Analysis and interpretation of the data: B. Rouse, A. Cipriani, Q. Shi, A.L. Coleman, K. Dickersin, T. Li.
Drafting of the article: B. Rouse, A. Cipriani, T. Li.
Critical revision of the article for important intellectual content: B. Rouse, A. Cipriani, A.L. Coleman, K. Dickersin, T. Li.
Final approval of the article: A. Cipriani, A.L. Coleman, K. Dickersin, T. Li.
Provision of study materials or patients: T. Li.
Statistical expertise: T. Li.
Obtaining of funding: K. Dickersin.
Administrative, technical, or logistic support: T. Li.
Collection and assembly of data: B. Rouse, Q. Shi, T. Li.
References
- 1.Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, editors. Clinical Practice Guidelines We Can Trust. National Academies Pr; Washington, DC: 2011. [PubMed] [Google Scholar]
- 2.Eddy DM. Evidence-based medicine: a unified approach. Health Aff (Millwood) 2005;24:9–17. doi: 10.1377/hlthaff.24.1.9. PMID: 15647211. [DOI] [PubMed] [Google Scholar]
- 3.Eden J, Wheatley B, McNeil B, Sox H, editors. Knowing What Works in Health Care: A Roadmap for the Nation. National Academies Pr; Washington, DC: 2008. [Google Scholar]
- 4.Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC., Jr Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA. 2009;301:831–41. doi: 10.1001/jama.2009.205. [PMID: 19244190] doi:10.1001/jama.2009.205. [DOI] [PubMed] [Google Scholar]
- 5.Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods. 2012;3:80–97. doi: 10.1002/jrsm.1037. [PMID: 26062083] doi:10.1002/jrsm.1037. [DOI] [PubMed] [Google Scholar]
- 6.Cipriani A, Higgins JP, Geddes JR, Salanti G. Conceptual and technical challenges in network meta-analysis. Ann Intern Med. 2013;159:130–7. doi: 10.7326/0003-4819-159-2-201307160-00008. [PMID: 23856683] doi:10.7326/0003-4819-159-2-201307160-00008. [DOI] [PubMed] [Google Scholar]
- 7.American Academy of Ophthalmology Glaucoma Panel . Preferred Practice Pattern Guidelines. Primary Open-Angle Glaucoma. American Academy of Ophthalmology; San Francisco: 2010. [DOI] [PubMed] [Google Scholar]
- 8.Quigley HA, Broman AT. The number of people with glaucoma worldwide in 2010 and 2020. Br J Ophthalmol. 2006;90:262–7. doi: 10.1136/bjo.2005.081224. PMID: 16488940. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.National Institute for Health and Care Excellence . Glaucoma: Diagnosis and Management of Chronic Open Angle Glaucoma and Ocular Hypertension. National Institute for Health and Care Excellence; London: 2009. [Google Scholar]
- 10.American Academy of Ophthalmology Glaucoma Panel . Preferred Practice Pattern Guidelines. Primary Open-Angle Glaucoma. American Academy of Ophthalmology; San Francisco: 1989. [DOI] [PubMed] [Google Scholar]
- 11.American Academy of Ophthalmology Glaucoma Panel . Preferred Practice Pattern Guidelines. Primary Open-Angle Glaucoma. American Academy of Ophthalmology; San Francisco: 1990. [DOI] [PubMed] [Google Scholar]
- 12.American Academy of Ophthalmology Glaucoma Panel . Preferred Practice Pattern Guidelines. Primary Open-Angle Glaucoma. American Academy of Ophthalmology; San Francisco: 1992. [DOI] [PubMed] [Google Scholar]
- 13.American Academy of Ophthalmology Glaucoma Panel . Preferred Practice Pattern Guidelines. Primary Open-Angle Glaucoma. American Academy of Ophthalmology; San Francisco: 1996. [DOI] [PubMed] [Google Scholar]
- 14.American Academy of Ophthalmology Glaucoma Panel . Preferred Practice Pattern Guidelines. Primary Open-Angle Glaucoma. American Academy of Ophthalmology; San Francisco: 2000. [DOI] [PubMed] [Google Scholar]
- 15.American Academy of Ophthalmology Glaucoma Panel . Preferred Practice Pattern Guidelines. Primary Open-Angle Glaucoma. American Academy of Ophthalmology; San Francisco: 2003. [DOI] [PubMed] [Google Scholar]
- 16.American Academy of Ophthalmology Glaucoma Panel . Preferred Practice Pattern Guidelines. Primary Open-Angle Glaucoma. American Academy of Ophthalmology; San Francisco: 2005. [DOI] [PubMed] [Google Scholar]
- 17.American Academy of Ophthalmology Glaucoma Panel . Preferred Practice Pattern Guidelines. Primary Open-Angle Glaucoma. American Academy of Ophthalmology; San Francisco: 2006. [DOI] [PubMed] [Google Scholar]
- 18.Sommer A, Weiner JP, Gamble L. Developing specialty-wide standards of practice: the experience of ophthalmology. QRB Qual Rev Bull. 1990;16:65–70. doi: 10.1016/s0097-5990(16)30341-4. PMID: 2110356. [DOI] [PubMed] [Google Scholar]
- 19.Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA. 1992;268:240–8. PMID: 1535110. [PubMed] [Google Scholar]
- 20.Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992;327:248–54. doi: 10.1056/NEJM199207233270406. PMID: 1614465. [DOI] [PubMed] [Google Scholar]
- 21.Li T, Lindsley K, Rouse B, Hong H, Shi Q, Friedman DS, et al. Comparative effectiveness of first-line medications for primary open-angle glaucoma: a systematic review and network meta-analysis. Ophthalmology. 2016;123:129–40. doi: 10.1016/j.ophtha.2015.09.005. [PMID: 26526633] doi:10.1016/j.ophtha.2015.09.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Hommer A. Glaucoma: managing primary open-angle glaucoma–ocular tolerability, compliance, persistence and patient outcomes. EuropeanOphthalmicReview. 2009:19–22. doi:10.17925/EOR.2009.03.01.19. [Google Scholar]
- 23.Ip S, Hadar N, Keefe S, Parkin C, Iovin R, Balk EM, et al. A Web-based archive of systematic review data. Syst Rev. 2012;1:15. doi: 10.1186/2046-4053-1-15. [PMID: 22588052] doi:10.1186/2046-4053-1-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Li T, Vedula SS, Hadar N, Parkin C, Lau J, Dickersin K. Innovations in data collection, management, and archiving for systematic reviews. Ann Intern Med. 2015;162:287–94. doi: 10.7326/M14-1603. [PMID: 25686168] doi:10.7326/M14-1603. [DOI] [PubMed] [Google Scholar]
- 25.Deeks JJ, Higgins JP, Altman DG. Analysing data and undertaking meta-analyses. In: Deeks JJ, Higgins JP, Altman DG, editors. Cochrane Handbook for Systematic Reviews of Interventions 5.1.0. The Cochrane Collaboration; London: 2011. [Google Scholar]
- 26.Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med. 2004;23:3105–24. doi: 10.1002/sim.1875. PMID: 15449338. [DOI] [PubMed] [Google Scholar]
- 27.Lu G, Ades A. Assessing evidence inconsistency in mixed treatment comparisons. Journal of the American Statistical Association. 2006;101:447–59. doi:10.1198/016214505000001302. [Google Scholar]
- 28.Lunn DJ, Thomas A, Best N, Spiegelhalter D. WinBUGS–a Bayesian modelling framework: concepts, structure, and extensibility. Stat Comput. 2000;10:325–37. doi:10.1023/A:1008929526011. [Google Scholar]
- 29.Owen RK, Tincello DG, Keith RA. Network meta-analysis: development of a three-level hierarchical modeling approach incorporating dose-related constraints. Value Health. 2015;18:116–26. doi: 10.1016/j.jval.2014.10.006. [PMID: 25595242] doi:10.1016/j.jval.2014.10.006. [DOI] [PubMed] [Google Scholar]
- 30.Chaimani A, Higgins JP, Mavridis D, Spyridonos P, Salanti G. Graphical tools for network meta-analysis in STATA. PLoS One. 2013;8:e76654. doi: 10.1371/journal.pone.0076654. [PMID: 24098547] doi:10.1371/journal.pone.0076654. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Salanti G, Ades AE, Ioannidis JP. Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol. 2011;64:163–71. doi: 10.1016/j.jclinepi.2010.03.016. [PMID: 20688472] doi:10.1016/j.jclinepi.2010.03.016. [DOI] [PubMed] [Google Scholar]
- 32.Li T, Vedula SS, Scherer R, Dickersin K. What comparative effectiveness research is needed? A framework for using guidelines and systematic reviews to identify evidence gaps and research priorities. Ann Intern Med. 2012;156:367–77. doi: 10.7326/0003-4819-156-5-201203060-00009. [PMID: 22393132] doi:10.7326/0003-4819-156-5-201203060-00009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Stewart WC, Konstas AG, Nelson LA, Kruft B. Meta-analysis of 24-hour intraocular pressure studies evaluating the efficacy of glaucoma medicines. Ophthalmology. 2008;115:1117–1122. doi: 10.1016/j.ophtha.2007.10.004. PMID: 18082886. [DOI] [PubMed] [Google Scholar]
- 34.Li N, Chen XM, Zhou Y, Wei ML, Yao X. Travoprost compared with other prostaglandin analogues or timolol in patients with open-angle glaucoma or ocular hypertension: meta-analysis of randomized controlled trials. Clin Experiment Ophthalmol. 2006;34:755–64. doi: 10.1111/j.1442-9071.2006.01237.x. PMID: 17073898. [DOI] [PubMed] [Google Scholar]
- 35.Fung AT, Reid SE, Jones MP, Healey PR, McCluskey PJ, Craig JC. Meta-analysis of randomised controlled trials comparing latanoprost with brimonidine in the treatment of open-angle glaucoma, ocular hypertension or normal-tension glaucoma. Br J Ophthalmol. 2007;91:62–8. doi: 10.1136/bjo.2006.096693. PMID: 16956912. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Aptel F, Cucherat M, Denis P. Efficacy and tolerability of prostaglandin analogs: a meta-analysis of randomized controlled clinical trials. J Glaucoma. 2008;17:667–73. doi: 10.1097/IJG.0b013e3181666557. [PMID: 19092464] doi:10.1097/IJG.0b013e3181666557. [DOI] [PubMed] [Google Scholar]
- 37.Hodge WG, Lachaine J, Steffensen I, Murray C, Barnes D, Foerster V, et al. The efficacy and harm of prostaglandin analogues for IOP reduction in glaucoma patients compared to dorzolamide and brimonidine: a systematic review. Br J Ophthalmol. 2008;92:7–12. doi: 10.1136/bjo.2007.123737. PMID: 18156371. [DOI] [PubMed] [Google Scholar]
- 38.Harms NV, Toris CB. Current status of unoprostone for the management of glaucoma and the future of its use in the treatment of retinal disease. Expert Opin Pharmacother. 2013;14:105–13. doi: 10.1517/14656566.2013.748038. [PMID: 23199345] doi:10.1517/14656566.2013.748038. [DOI] [PubMed] [Google Scholar]
- 39.Fung DS, Whitson JT. An evidence-based review of unoprostone isopropyl ophthalmic solution 0.15% for glaucoma: place in therapy. Clin Ophthalmol. 2014;8:543–54. doi: 10.2147/OPTH.S41562. [PMID: 24648719] doi:10.2147/OPTH.S41562. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Trikalinos TA, Churchill R, Ferri M, Leucht S, Tuunainen A, Wahlbeck K, et al. EU-PSI project. Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time. J Clin Epidemiol. 2004;57:1124–30. doi: 10.1016/j.jclinepi.2004.02.018. PMID: 15612138. [DOI] [PubMed] [Google Scholar]
- 41.Gehr BT, Weiss C, Porzsolt F. The fading of reported effectiveness. A meta-analysis of randomised controlled trials. BMC Med Res Methodol. 2006;6:25. doi: 10.1186/1471-2288-6-25. PMID: 16689990. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Trikalinos TA, Ioannidis JPA. Assessing the evolution of effect sizes over time. In: Rothstein HR, Sutton AJ, Borenstein M, editors. Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments. J Wiley; Chichester, United Kingdom: 2005. pp. 241–59. doi:10.1002/0470870168.ch13. [Google Scholar]
- 43.Cipriani A, Barbui C, Salanti G, Rendell J, Brown R, Stockton S, et al. Comparative efficacy and acceptability of antimanic drugs in acute mania: a multiple-treatments meta-analysis. Lancet. 2011;378:1306–15. doi: 10.1016/S0140-6736(11)60873-8. [PMID: 21851976] doi:10.1016/S0140-6736(11)60873-8. [DOI] [PubMed] [Google Scholar]
- 44.Castellucci LA, Cameron C, Le Gal G, Rodger MA, Coyle D, Wells PS, et al. Efficacy and safety outcomes of oral anticoagulants and antiplatelet drugs in the secondary prevention of venous thromboembolism: systematic review and network meta-analysis. BMJ. 2013;347:f5133. doi: 10.1136/bmj.f5133. [PMID: 23996149] doi:10.1136/bmj.f5133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Li T, Dickersin K. Citation of previous meta-analyses on the same topic: a clue to perpetuation of incorrect methods? Ophthalmology. 2013;120:1113–9. doi: 10.1016/j.ophtha.2012.11.038. [PMID: 23522971] doi:10.1016/j.ophtha.2012.11.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, D'Amico R, et al. International Stroke Trial Collaborative Group. Indirect comparisons of competing interventions. Health Technol Assess. 2005;9:1–134. doi: 10.3310/hta9260. PMID: 16014203. [DOI] [PubMed] [Google Scholar]
- 47.Watts NB, Adler RA, Bilezikian JP, Drake MT, Eastell R, Orwoll ES, et al. Endocrine Society. Osteoporosis in men: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2012;97:1802–22. doi: 10.1210/jc.2011-3045. [PMID: 22675062] doi:10.1210/jc.2011-3045. [DOI] [PubMed] [Google Scholar]
- 48.Murad MH, Drake MT, Mullan RJ, Mauck KF, Stuart LM, Lane MA, et al. Clinical review. Comparative effectiveness of drug treatments to prevent fragility fractures: a systematic review and network meta-analysis. J Clin Endocrinol Metab. 2012;97:1871–80. doi: 10.1210/jc.2011-3060. [PMID: 22466336] doi:10.1210/jc.2011-3060. [DOI] [PubMed] [Google Scholar]
- 49.National Institute for Health and Care Excellence . Neuropathic Pain–Pharmacological Management: The Pharmacological Management of Neuropathic Pain in Adults in Non-specialist Settings. National Institute for Health and Care Excellence; London: 2013. [PubMed] [Google Scholar]
- 50.Kendall T, Morriss R, Mayo-Wilson E, Marcus E, Guideline Development Group of the National Institute for Health and Care Excellence Assessment and management of bipolar disorder: summary of updated NICE guidance. BMJ. 2014;349:g5673. doi: 10.1136/bmj.g5673. [PMID: 25258392] doi:10.1136/bmj.g5673. [DOI] [PubMed] [Google Scholar]
- 51.Medeiros FA. Biomarkers and surrogate endpoints in glaucoma clinical trials. Br J Ophthalmol. 2015;99:599–603. doi: 10.1136/bjophthalmol-2014-305550. [PMID: 25034049] doi:10.1136/bjophthalmol-2014-305550. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Garway-Heath DF, Crabb DP, Bunce C, Lascaratos G, Amalfitano F, Anand N, et al. Latanoprost for open-angle glaucoma (UKGTS): a randomised, multicentre, placebo-controlled trial. Lancet. 2015;385:1295–304. doi: 10.1016/S0140-6736(14)62111-5. [PMID: 25533656] doi:10.1016/S0140-6736(14)62111-5. [DOI] [PubMed] [Google Scholar]
- 53.Krupin T, Liebmann JM, Greenfield DS, Ritch R, Gardiner S, Low-Pressure Glaucoma Study Group A randomized trial of brimonidine versus timolol in preserving visual function: results from the Low-Pressure Glaucoma Treatment Study. Am J Ophthalmol. 2011;151:671–81. doi: 10.1016/j.ajo.2010.09.026. [PMID: 21257146] doi:10.1016/j.ajo.2010.09.026. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.