Choosing Core Outcomes for Use in Clinical Trials in Ophthalmology: Perspectives from Three Ophthalmology Outcomes Working Groups

Randomized controlled trials (RCTs) and systematic reviews of RCTs, when designed and conducted rigorously, provide valuable evidence for healthcare decision making.¹ Rigorous conduct of RCTs and systematic reviews requires, among other things, assessment of outcomes meaningful to all stakeholders. In RCTs, an outcome is a measurement or event used to assess the effects of treatment.² Outcomes are intended to measure the biological or functional mechanism that the treatment targets (e.g., intraocular pressure), disease progression (e.g., peripheral vision loss), health status (e.g., quality of life), or other aspects (e.g., costs). Patient-important outcomes are outcomes patients value, typically how a patient feels, functions, or survives.³ Patient-reported outcomes are outcomes reported directly by patients without interpretation by anyone else.⁴

For RCT results to be actionable (i.e., results included in systematic reviews and considered for decision making), the outcomes must: (1) be defined consistently across RCTs and (2) address the needs of stakeholders, including patients. However, these conditions are not often being met, causing 2 key problems.

1. Inconsistent outcomes across RCTs. When outcomes across RCTs are inconsistent, a meta-analysis may include only a few, or even no, RCT results, a waste of research effort and money. For example, in a 2016 Cochrane systematic review of over-the-counter tear drops for dry eye, only 4 of 43 eligible RCTs could be included in the meta-analysis for dry eye symptom frequency and severity, the review’s primary outcome.⁵ In large part, these inconsistencies may be the result of the independence of each group of trial investigators; we found as many as 105 unique outcomes in 138 RCTs for dry eye⁶ and 107 unique outcomes in 79 RCTs for age-related macular degeneration (AMD).⁷ Similarly, a plethora of outcomes have been reported for cataract,⁷ diabetic retinopathy,⁷ glaucoma,^7,8 uveitis,⁹ allergic conjunctivitis,¹⁰ and intermittent exotropia.¹¹

2. Outcomes in RCTs are not relevant to stakeholders. The ability of RCTs to inform decision making is compromised when they do not examine relevant outcomes.^12,13 For example, we demonstrated that 10 of 26 outcomes (39%) identified as important by patients with dry eye were rarely reported in existing research, with fewer than 10% of studies reporting these outcomes.⁶

Failure to examine relevant outcomes also has ramifications beyond clinicians and patients. From a health economics standpoint, the value of a treatment is defined by the health outcomes achieved per dollar spent.¹⁴ Failure to measure outcomes meaningful to patients hampers reimbursement decisions.¹⁵ Furthermore, the World Health Organization (WHO) recognizes that “choosing the most important outcome is critical to producing a useful guideline.”¹⁶

Core outcome sets (COSs) can help solve these problems and improve health. A COS is a minimum set of outcomes (usually 5–7), agreed on by various stakeholders, that will be measured and reported in research addressing a given disease or treatment.¹⁷ Widely recognized as integral to solving the current problems with outcomes in RCTs, COSs typically are developed through sequential steps: searching the literature for outcomes that have been assessed, identifying additional outcomes important to stakeholders (e.g., through a survey), and building consensus on outcomes to be included in the COS.¹⁸

Ophthalmology lags behind other fields in terms of COS development: more than 20 COSs each have been developed for cancer, rheumatology, neurology, and cardiology.¹⁹ As of June 2018, we found COSs developed or under development for only 3 eye diseases: glaucoma, cataract, and uveitis.²⁰

To begin work toward standardizing outcomes in ophthalmology research, we have chosen 3 prevalent conditions: AMD (United States prevalence, 1.5%^21,22), refractive error in children (United States prevalence 1%e26%^23–26), and dry eye (United States prevalence, 7%e33%^27,28). We convened 3 Outcomes Working Groups, each focusing on 1 of these conditions. In this editorial, we summarize the composition of the groups, the major opportunities that COS development presents, and the methodologic, practical, and disease-specific considerations that the groups identified for development of COSs (Supplemental Material, available at www.aaojournal.org).

The 3 working groups held their kickoff meetings during the American Academy of Ophthalmology Annual Meeting in November 2017. One clinician (S.D.S. for AMD, M.X.R. for refractive error in children, and E.K.A. for dry eye) and 3 methodologists (T.L., I.J.S., and J.T.L.) led each group. The AMD, refractive error in children, and dry eye outcomes working groups included 27, 24, and 39 individuals, respectively. Some participated in more than 1 group. The 62 unique participants mostly were ophthalmologists and optometrists. Representatives from the National Eye Institute, the Food and Drug Administration (FDA), and a patient group also participated. Participants were based in the United States, Europe, and Asia (84%, 10%, and 6%, respectively).

Core outcome set development presents major opportunities. First, engaging various stakeholders encourages collaboration and mutual exchange of ideas. Second, COS development offers funders the opportunity to influence the outcomes measured in the research they support. Third, COSs help to streamline the design of pivotal RCTs to bring new products to market. A participant pointed out that COS development can help “educate both our colleagues and the regulators as to outcomes that are important.” By promoting COSs developed through stakeholder engagement, regulatory bodies such as the FDA can expedite innovation and product approval. For example, the proportion of RCTs reporting all outcomes in a COS for rheumatoid arthritis spiked after the FDA and the European Medicines Agency endorsed the COS.²⁹ Fourth, COSs help standardize outcomes and enable more RCTs to influence decision making through their inclusion in meta-analyses.

1. Methodologic considerations. First, COS developers should define outcomes according to 5 elements: (1) domain (e.g., best-corrected visual acuity), (2) specific measurement (e.g., Snellen chart), (3) specific metric (e.g., value at a time point, change from baseline), (4) method of aggregation (e.g., mean for continuous data, percent for categorical data), and (5) time points (e.g., 8 weeks).30,31 It is evident that, for a single domain, an RCT may report many defined outcomes because different measurements, metrics, methods of aggregation, and time points can be used.32 Second, COS developers should consider the goals for each outcome. An outcome (e.g., visual acuity) may be intended to measure improvement in health (e.g., a gain of ≥ 15 letters on a Snellen chart), disease progression (e.g., loss of ≥ 15 letters), or avoidance of disease progression (e.g., loss of <5 letters).

2. Practical considerations. First, when defining the scope of the COS, developers should consider disease severity (e.g., mild, moderate, or severe dry eye) and disease subtypes (e.g., dry or wet AMD). Second, preferred measurement instruments could change over time. This is particularly true for outcomes, for example, axial length, measured through fast-evolving technology. Third, the feasibility of measuring each outcome in busy clinics should be considered. Fourth, different decision makers, for example, clinicians and patients, may gauge different outcomes to be important. Given the typically short duration of trials, it is tempting to focus on interim outcomes such as tear volume. Measuring patient-important outcomes has been described as “a new frontier for many physicians.”33 At least 2 challenges exist when considering patient-important outcomes for COSs. First, COS developers need to decide on the scope of patient-important outcomes (e.g., quality of life) that encompass multiple components (e.g., vision-related quality of life, psychological status). Second, in early-stage disease, patient-important outcomes may not be sensitive enough to capture improvement in health because patients often are asymptomatic.

3. Disease-specific considerations. For AMD, participants noted an urgent need for a COS for geographic atrophy. Because no effective treatment for geographic atrophy exists, a COS addressing this subtype would facilitate the testing of new treatments. Participants noted that outcomes relevant to geographic atrophy likely also would be relevant to another form of late AMD, neovascular AMD.

   For refractive error in children, COS developers should consider that PRO instruments for refractive error are of variable quality, specifically in terms of rigor of design and analysis. Additionally, patient age can challenge outcome assessment in this context; patient-reported results for very young children can be assessed only in proxy form.

   For dry eye, first, only limited correlation exists between dry eye symptoms (e.g., grittiness) and clinical signs (e.g., corneal staining).^34–36 In keep-ing with the FDA’s requirement for approval of treatments for dry eye that there be demonstrated evidence of improvement in at least 1 symptom and in at least 1 sign in RCTs,^37–39 COSs for dry eye should include both subjective and objective outcomes. Second, dry eye can arise from various underlying diseases (e.g., Sjögren’s syndrome, blepharitis), some having systemic symptoms and signs. Core outcome set developers should consider whether separate COSs are needed for each underlying disease or for groups of diseases.

Actionable outcomes require consistency across RCTs and must address the needs of stakeholders, including patients. The existence of an agreed-on COS eventually enables treatment guidelines to be informed by comprehensive and accurate estimates of effectiveness and safety. Although COSs promote standardization, they do not stifle innovation; clinical trialists are not constrained to measuring solely the outcomes contained in the COS.

Harmonizing outcomes across RCTs through the development of COSs will require a concerted and sustained effort by many stakeholders. The outcomes working groups we convened include various stakeholders: clinicians who need outcome data from RCTs to treat patients effectively; clinical trialists who choose outcomes in the face of budgetary and other realities; systematic reviewers and guideline developers who are frustrated when inconsistent outcomes among RCTs means that meta-analyses combine data from only a select few RCTs; editors of major ophthalmology journals; the FDA; and the NEI. Three participants also were panel members of a 2016 FDA workshop aimed at building consensus around outcomes for trials of devices for myopia.⁴⁰

One limitation with the current groups is that we did not have funding to bring patients to the groups’ kickoff meetings; only 1 patient participated. In addition, most participants are based in the United States. During the actual development of the COSs, we will need broader stakeholder and geographic representation.

We agree with Dr. Susan Hockfied, President of the American Academy for the Advancement of Science (AAAS), when she writes, “Achieving success in science is a team sport.”⁴¹ We urge the field of ophthalmology to join other fields and embrace COSs. We have begun engaging the field through 3 outcomes working groups; they have laid the groundwork, built momentum, and outlined important considerations for developing COSs. This is an important first step. We expect most of these considerations also to be relevant during COS development for other eye conditions in the future. The vision health of patients stands to gain a tremendous amount from this collective success.