Abstract
Purpose:
To assess the impact of a masked Endpoint Committee on estimates of the incidence of primary open angle glaucoma (POAG), treatment efficacy and statistical power of the Ocular Hypertension Treatment Study-Phase 1, 1994-2002 (OHTS-1).
Design:
Retrospective inter-rater reliability analysis of endpoint attribution by the Endpoint Committee
Methods:
After study closeout, we re-calculated estimates of endpoint incidence, treatment efficacy and statistical power using all-cause endpoints and POAG endpoints. To avoid bias, only the first endpoint per participant is included in this report.
Results:
The Endpoint Committee reviewed 267 first endpoints from 1,636 participants. The Endpoint Committee attributed 58% (155 of 267) of the endpoints to POAG. The incidence of all-cause endpoints vs. POAG endpoints was 19.5% and 13.2% respectively in the observation group and 13.1% and 5.8% respectively in the medication group. Treatment effect for all-cause endpoints was a 33% reduction in risk (relative risk = 0.67, 95% CI of 0.54-0.84) and a 56% reduction in risk for POAG endpoints (relative risk = 0.44, 95% CI of 0.31-0.61). Post-hoc statistical power for detecting treatment effect was 0.94 for all-cause endpoints and 0.99 for POAG endpoints.
Conclusion:
Endpoint Committee adjudication of endpoints improved POAG incidence estimates, increased statistical power, and increased calculated treatment effect by 23%. An Endpoint Committee should be considered in therapeutic trials when common ocular and systemic co-morbidities, other than the target condition, could compromise study results.
Introduction
Clinically, the diagnosis of glaucoma is based on assessment of visual field abnormality and optic nerve deterioration.1-3 However, when considering clinical trials both visual field and optic nerves may not be both evaluated. A systematic review of 13 Cochrane reviews and 9 protocols of glaucoma intervention studies indicates no clear consensus regarding how best to assess outcomes, making it difficult to compare studies when there is significant variation. Of the glaucoma studies reviewed, only 12% used visual fields and 13% a structural assessment, 62% of which were optic nerve head change.4 Almost all National Eye Institute sponsored clinical trials of ocular hypertension and glaucoma have used visual fields (VF) as a primary endpoint, including the Advanced Glaucoma Intervention Study5, Early Manifest Glaucoma Trial6, (EMGT) Collaborative Initial Glaucoma Treatment Study7 (CIGTS) and The Ocular Hypertension Treatment Study8 (OHTS). The EMGT6, CIGTS9 and OHTS8 assessed the optic nerve as well however only EMGT6 and OHTS8 relied on the optic nerve in addition to the visual field as primary endpoints. The importance of evaluating both the visual field and the optic nerve has been noted by others, particularly when evaluating patients with ocular hypertension.2, 10 Moreover, using both functional and structural metrics is consistent with clinical practice.1
However, the assessment of visual field abnormality and optic nerve deterioration is complicated. It is well known that other eye conditions such as cataracts, age-related macular degeneration and diabetic retinopathy, which may be more common than glaucoma in some groups of older adults, can cause visual field loss.11 The task of distinguishing visual field loss attributable to glaucoma from non-glaucomatous causes can be daunting. Similarly, optic disc damage can result from non-glaucomatous causes such as ischemic optic neuropathy, demyelinating disease, and compressive optic nerve lesions. Co-mingling of glaucomatous and non-glaucomatous endpoints in a clinical trial can cause overestimation of the incidence or progression of glaucoma, underestimation of treatment effect and/or a loss of statistical power.
The Ocular Hypertension Treatment Study-Phase 1, 1994-2002 (OHTS-1)8 is an unmasked, randomized trial of the safety and efficacy of ocular hypotensive medication in preventing or delaying the onset of primary open angle glaucoma (POAG). The US Food and Drug Administration and the European Medicines Agency recommend centralized adjudication when endpoints are subjective, require the application of a complex definition and/or when treatment assignment is not masked.12,13 Thus, to achieve greater diagnostic specificity of endpoints, the OHTS used a masked Endpoint Committee to adjudicate whether the cause of reproducible visual field abnormality and reproducible optic disc deterioration were due to POAG. This manuscript assesses the impact of a masked Endpoint Committee adjudication on estimates of POAG conversion, treatment effect and statistical power.
Methods
OHTS Phase 1 (NCT0000125) was undertaken in accordance with good clinical practice guidelines and adhered to the Declaration of Helsinki. The institutional review board of each clinic approved the study protocol and consent form. Participants gave written informed consent to be screened and, if eligible, to be randomized.
Between February 1994 to October 1996, 1,636 individuals with an intraocular pressure (IOP) between 24 mmHg and 32 mmHg in one eye and 21 to 32 mm Hg in the fellow eye and no evidence of glaucomatous damage by standard clinical measures were randomized to either observation or to treatment with commercially available topical ocular hypotensive medications. Neither clinician nor participant was masked to the randomization assignment. Reading Centers determined the occurrence of visual field and optic disc endpoints without regard to cause, “all-cause” endpoints. The masked Endpoint Committee determined whether these endpoints were POAG endpoints. The primary hypothesis compared the incidence of POAG endpoints in the medication and observation groups. A secondary analysis compared incidence of all-cause endpoints in the randomization groups. In 2002, the OHTS primary outcome paper13 reported that ocular hypotensive medication was safe and effective in preventing glaucoma, after which participants in the observation group were offered ocular hypotensive treatment. This report includes data to June 2002 when participants were followed according to their original randomization assignment.
Figure 1 illustrates the flowchart for assessment of endpoints after reading centers determined the occurrence of a reproducible endpoint and the Coordinating Center prepared an endpoint chart for Endpoint Committee adjudication.14-16 Criteria for Reading Center determination of reproducible visual field abnormality and optic disc deterioration are described briefly below.
A reproducible visual field abnormality was defined as a set of three consecutive, reliable visual fields with a corrected pattern standard deviation (CPSD) of P < 5% or a glaucoma hemifield test outside normal limits by STATPAC 2 criteria (Humphrey 30-2) as judged by masked readers at the Visual Field Reading Center, University of California-Davis, Sacramento, California, were sent to the Coordinating Center to initiate the endpoint review process.17,14 Eighty-six percent of the initial visual field abnormalities were not confirmed on retest and were not sent to Endpoint Committee for review. Additional details about the process of reviewing visual fields have been previously published.15 A reproducible disc deterioration was defined as two consecutive sets of optic disc photographs judged by masked certified readers at the Optic Disc Reading Center, Bascom Palmer Eye Institute, Miami, Florida to show generalized or localized thinning of the optic disc neuroretinal rim compared to baseline stereoscopic optic disc photographs. About 18% of the eyes that initially showed disc deterioration from baseline were not confirmed by retake optic disc photographs and were not sent to the Endpoint Committee for review (Internal report, August, 2018). When two consecutive sets of optic disc photographs were judged to show deterioration, the case was sent to the Coordinating Center to initiate the endpoint review process. Additional details about the process of reviewing optic disc photographs have been previously published.16
The Coordinating Center prepared a separate masked endpoint chart for each reproducible visual field abnormality and each reproducible optic disc deterioration. Each member of the Endpoint Committee received a masked endpoint chart for review. The endpoint chart included medical and ocular history (all visual fields, all optic disc photographs, macular and red reflex photographs of both eyes) to date. The endpoint chart was masked as to randomization assignment, hypotensive medication use, IOP and glaucoma surgery.
The masked Endpoint Committee was established to serve as an unbiased clinical surrogate to adjudicate whether reproducible changes identified by either Reading Center were due to POAG and to determine whether optic disc deterioration was clinically significant. Visual field abnormality was deemed a POAG endpoint based on a consistent depression in sensitivity in the Pattern Deviation analysis, compatible with either a nerve fiber bundle defect, paracentral scotoma, or nasal step. Defects occurring only at the margins of the visual field were typically considered artifacts unless there were optic disc changes to corroborate the clinical diagnosis of POAG.
To reflect clinician decision-making, the Data and Safety Monitoring Committee mandated that the Endpoint Committee determine if optic disc deterioration detected by the Optic Disc Reading Center met a clinical threshold of conversion to POAG to justify initiation of treatment. If the clinical threshold was not met, the Endpoint committee did not adjudicate cause. Optic disc deterioration was deemed a POAG endpoint based on clinically significant changes in the following: optic disc rim width, the contour of the disc, or shift of the vessels across the disc rim. Disc hemorrhage alone was not considered an endpoint, unless associated with a change in contour. To mimic clinical practice, the Endpoint Committee reviewed both the disc and visual field of both eyes in their adjudication of every endpoint. If either the optic nerve or visual field endpoint did not meet these criteria, then the endpoint was rejected and additional testing was requested. Ocular media changes or other ocular diagnoses were considered in the deliberation. Examples of a visual field and optic nerve change are illustrated at https://www.vrcc.wustl.edu/ohts/disc/ and https://www.vrcc.wustl.edu/ohts/fields/.
The Endpoint Committee process was consensus-driven. The members of the Endpoint Committee (DH, EH and RP), all practicing clinicians, remained active throughout the study. Each Endpoint Committee member independently determined whether an endpoint was “most probably due to POAG” or “most probably not due to POAG” and, for optic disc deterioration, whether the “change was clinically significant.” The “Endpoint Form” is posted on https://ohts.wustl.edu/wp-content/uploads/2017/02/A_EN.pdf. If the first review did not achieve unanimity a second, independent review was conducted (Figure 1). If the second review did not achieve unanimity, a conference call was held. The target time for reaching unanimity was 31 days from notification of the Coordinating Center by the Reading Center of a reproducible change. The Endpoint Committee also determined if discontinuation of assessment was necessary when visual fields or optic disc stereo-photographs became unascertainable for POAG (e.g. after retinal vein occlusion or stroke).
The first endpoint per participant was selected for inclusion in this report to avoid bias from subsequent endpoint reviews. Participants were classified as converting to POAG if one eye developed a POAG endpoint. If a POAG endpoint was detected by both visual fields and by optic disc deterioration on the same visit, one endpoint was selected randomly. Participants who developed an endpoint due to causes other than POAG were classified as developing a non-POAG endpoint. We evaluated the Endpoint Committee’s performance and impact as follows:
Intergrader agreement among Endpoint Committee members is reported as the proportion of endpoint reviews in which unanimity was achieved in the first and second rounds of independent review.
The all-cause and POAG endpoints are compared by randomization group and separately by visual field endpoints and optic disc endpoints.
Treatment effect was estimated using a post-hoc Mantel-Haenszel test to calculate the relative risk of all-cause endpoints and POAG endpoints in the observation and medication groups.
Post-hoc statistical power of the OHTS was estimated using all-cause endpoints and POAG endpoints assuming a two-sided alpha=0.05 for two-sample proportions test for a fixed sample of 1,636 participants with no adjustment for censoring.
Results
Data were re-analyzed for this report in 2012. Over a median follow-up of 7.5 years of OHTS, the Endpoint Committee reviewed 467 endpoints from 267 participants. The first endpoint per participant (n=267) was selected for inclusion in this report to avoid bias from subsequent reviews. If both visual field and optic disc endpoints were detected on the same visit for 5.2% (14 of 267) of the participants, the endpoint was selected randomly for inclusion in this report.
Inter-grader Agreement
The 3 members of the Endpoint Committee independently reached unanimity in the first round of review for 61.0% (163 of 267) endpoints, in the second round for 32.2% (86 of 267) endpoints and in a consensus conference call for the remaining 6.7% (18 of 267). A slightly higher rate of optic disc endpoints required a third round of review 9.6% (13 of 135 compared to 3.8% (5 of 132) of the visual field endpoints. Mean time to notification of clinics of Endpoint Committee decision in the 1st, 2nd and 3rd rounds of review were 11.0 ± 7.1 SD days, 20.8 ± 14.3 SD days and 31.4 ± 15.6 SD days respectively.
All-Cause and POAG Endpoints, Treatment Effect and Statistical Power
Overall, 16.3% (267 of 1,636) of the participants developed an endpoint and 9.5% (155 of 1,636) developed a POAG endpoint. In the observation group, 19.5% (160 of 819) of the participants developed an endpoint compared to 13.1% (107 of 817) in the medication group (relative risk = 0.67, 95% CI of 0.54-0.84) – a reduction of 33%. In the observation group 13.2% (108 of 819) developed a POAG endpoint compared to 5.8% (47 of 817) in the medication group (relative risk = 0.44, 95% CI of 0.31-0.61) – a reduction of 56%. Post-hoc statistical power for detecting a treatment effect for all endpoints was 0.94 compared to >0.99 for detecting a treatment effect for POAG endpoints with the OHTS sample of 1,636 participants (Table 3).
Table 3.
Reproducible Endpoints |
Endpoints in Observation Group N=819 |
Endpoints in Medication Group n=817 |
Relative Risk of Endpoint 95% Confidence Interval |
Post- hoc Statistical Power |
---|---|---|---|---|
All-cause Endpoints | 19.5% (160/819) | 13.1% (107/817) | 0.67 (0.54,0.84) | 0.941 |
POAG Endpoints | 13.2% (108/819) | 5.8% (47/817) | 0.44 (0.31,0.61) | >0.99 |
All-cause VF Endpoints | 9.2% (75/819) | 7.0% (57/817) | 0.76 (0.55,1.06) | 0.367 |
POAG VF Endpoints | 5.5% (45/819) | 2.4% (20/817)) | 0.44 (0.26,0.75) | 0.883 |
All-cause Disc Endpointsb | 10.4% (85/819) | 6.1% (50/817) | 0.59 (0.42,0.82) | 0.886 |
POAG Disc Endpoints | 7.7% (63/819) | 3.3% (27/817) | 0.43 (0.28,0.67) | 0.974 |
Sample includes the first endpoint per participant. Incidence estimates are not adjusted for variable follow-up time. Median follow-up is 7.5 years.
Optic disc endpoints that were judged “not clinically significant” were not adjudicated as due to POAG or not.
We found no evidence of bias in the Endpoint Committee’s attribution of endpoints to “non-POAG” causes by randomization group. Non-POAG endpoints (n=112) were approximately equally distributed in both randomization groups, 6.3% (52 of 819) in the observation group and 7.3% (60 of 817) in the medication group.
Visual Field POAG Endpoints
Of the 132 visual field (VF) endpoints, the Endpoint Committee classified 49% (65 of 132) as “probably due to POAG” and 51% (66 of 132) as “probably not due to POAG”. One VF endpoint was classified as “no change” (Table 1). Other causes were recorded in notes by Endpoint Committee members and are reported in Table 2. In the observation group, 9.2% (75/819) of the participants developed a VF endpoint compared to 7.0% (57/817) in the medication group (relative risk of 0.76, 95% CI of 0.55-1.06) – a risk reduction of 24%. In the observation group, 5.5% (45/819) of the participants developed a POAG VF endpoint compared to 2.4% (20/817) in the medication group (relative risk = 0.44, 95% CI of 0.26-0.75) – a risk reduction of 56%. Post-hoc statistical power for detecting a treatment effect for all-cause VF endpoints was 0.37 compared to 0.88 for detecting a treatment effect for POAG VF endpoints with a fixed sample of 1,636 participants (Table 3).
Table 1.
Attribution of Cause of Change | Type of Endpoint | |||
---|---|---|---|---|
Visual Field | Optic Disc | |||
N | Percent | N | Percent | |
Probably due to POAG | 65 | 49.2% | 90 | 66.7% |
Probably NOT due to POAG | 66 | 50.0% | 4 | 3.0% |
No Change/Not Clinically Significant | 1 | 0.8% | 41 | 30.4% |
Total | 132 | 100% | 135 | 100% |
If two endpoints occurred on the same date for a participant, one was randomly selected for inclusion in the analysis dataset.
Table 2.
VF | ||
---|---|---|
N | Percent | |
Lid abnormalities/external disease | 23 | 24.5% |
Retinal vein occlusion | 14 | 14.9% |
Cataract/other media opacity | 11 | 11.7% |
Diabetic retinopathy | 6 | 6.4% |
Age-related macular degeneration | 6 | 6.4% |
Neurological | 6 | 6.4% |
Other** | 28 | 29.7% |
All | 94 | 100% |
“Other” causes included e.g. glaucoma other than POAG, optic neuropathy, dry eyes were recorded in Endpoint Committee notes from medical histories completed at each OHTS visit.
Total number of non-POAG causes (n=94) is greater than the number of non-POAG visual field endpoints (n=66) because an endpoint can be attributable to more than one cause.
Optic Disc POAG Endpoints
Of the 135 optic disc endpoints, the Endpoint Committee classified 30.4% (41 of 135) as “not clinically significant and did not adjudicate cause, 66.7% (90 of 135) were classified as “clinically significant and due to POAG, 3% (4 of 135) were clinically significant and not due to POAG (Table 1). In the observation group, 10.4% (85/819) of the participants developed an optic disc endpoint compared to 6.1% (50/817) in the medication group (relative risk of 0.59, 95% CI of 0.42-0.82) – a reduction of 41%. In the observation group 7.7% (63/819) of the participants developed a POAG optic disc endpoint compared to 3.3% (27/817) in the medication group (relative risk = 0.43, 95% CI of 0.28-0.67) – a reduction of 57%. Post-hoc statistical power for detecting a treatment effect for all optic disc endpoints was 0.89 and 0.97 for POAG optic disc endpoints with a fixed sample of 1,636 participants (Table 3).
Discussion
The Endpoint Committee had a clear impact on the accuracy of estimates of POAG conversion and treatment effect. Overall, 16.3% of the participants developed an endpoint, but only 9.5% of the participants had an endpoint attributed to POAG by the Endpoint Committee. The impact of the Endpoint Committee’s determination of POAG from non-POAG endpoints was most pronounced for VF endpoints. Eight percent of the participants developed a VF endpoint, but only 4% developed a VF endpoint attributed to POAG. Non-POAG causes of visual field abnormalities such as cataract, lid abnormalities and neurological disorders (See Table 2) would be expected in a longterm study in older adults.
The US Food and Drug Administration and the European Medicines Agency recommend centralized adjudication when endpoints are subjective, require the application of a complex definition and/or when treatment assignment is not masked. To our knowledge, OHTS is the only NIH-sponsored trial in glaucoma to use a masked Endpoint Committee to adjudicate whether reproducible visual field abnormalities and/or optic disc deterioration, were due to POAG. Beyond the field of ophthalmology, the number of therapeutic trials using Endpoint Committees has increased and best practice guidelines for their operation have been developed.18-21
Four key components of the Endpoint Committee are: 1. Masking 2. Standardization of outcome determination over time and across clinics 3. Integrating both functional and structural tests 4. Requirement of unanimity. The benefits of the Endpoint Committee are clear, particularly when considering the removal of non-POAG causes that dilute the treatment effect. The calculated treatment effect was greater for POAG endpoints than for all-cause endpoints because medication has little or no effect on non-POAG endpoints. Treatment reduced the estimate of all-cause endpoints by 33% (relative risk of 0.67) and the estimate of POAG endpoints by 56% (relative risk of 0.44).
In order to make OHTS results more generalizable to clinical practice, the Data and Safety Monitoring Committee instructed the Endpoint Committee to require optic disc endpoints to be “clinically significant” in magnitude to justify initiation of treatment. The Endpoint Committee classified nearly a third of all optic disc endpoints as “not clinically significant”, although these endpoints had been documented as reproducible by trained readers at the Optic Disc Reading Center. Optic disc endpoints that were judged “not clinically significant” were not adjudicated as to cause. Of the “clinically significant” optic disc endpoints, 97% were attributed to POAG. This high level of specificity should not be interpreted as evidence that optic disc endpoints are generally more specific for POAG than visual field endpoints since adjudication was performed only for the subset of clinically significant optic disc endpoints.
There are some important considerations in evaluating the benefit and limitations of an Endpoint Committee. Not all treatment trials benefit from endpoint adjudication. In a review of 10 trials with over 9,000 events from more than 95,000 participants, Pogue found no difference between adjudicated composite outcomes (cardiovascular death, MI or stroke) and non-adjudicated outcomes.22 Adjudication may be less critical if study duration is short, patient sample is homogeneous, outcomes are quantitative and there are few competing causes of endpoints.2, 3 The use of artificial intelligence and statistical techniques e.g. PROGRESSOR could increase the certainty of disease presence/progression in Reading Centers and thereby could increase the certainty of causal attribution.23 However, given the uncertainty of causal attribution of visual field abnormality and optic disc deterioration, glaucoma intervention studies can benefit from a centralized process that drives consensus. Positioning the Endpoint Committee review following the Reading Center review mitigated bias and affirmed clinically relevant endpoints.
A limitation of the OHTS Endpoint review process is related to turnover in Endpoint Committee membership which could have, but did not, occur during this study. Thus, training and certification of Endpoint Committee members should be considered in future studies. Moreover, there was no attempt to externally validate the Endpoint Committee’s attribution of endpoints to POAG in OHTS. When competing ocular or systemic conditions co-occurred with a possible POAG cause, there is a risk that the attribution process may favor diagnostic specificity and result in an under count of POAG events.
We conclude that the masked adjudication of endpoints by the Endpoint Committee increased statistical power and increased the accuracy of estimates of POAG conversion and treatment effect. Given the strong effect of treatment in this trial, the increased power was not critical. However, in studies where the therapeutic intervention has a less robust effect, the increased power could be critical. An Endpoint Committee should be considered in therapeutic trials when common ocular and systemic co-morbidities, other than the target condition, could compromise study results.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Prum BE Jr., Lim MC, Mansberger SL, et al. Primary Open-Angle Glaucoma Suspect Preferred Practice Pattern Guidelines. Ophthalmology. January 2016;123(1):P112–151. [DOI] [PubMed] [Google Scholar]
- 2.Hood DC, De Moraes CG. Challenges to the Common Clinical Paradigm for Diagnosis of Glaucomatous Damage With OCT and Visual Fields. Invest Ophthalmol Vis Sci February 1 2018;59(2):788–791. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Garway-Heath DF, Crabb DP, Bunce C, et al. Latanoprost for open-angle glaucoma (UKGTS): a randomised, multicentre, placebo-controlled trial. Lancet April 4 2015;385(9975):1295–1304. [DOI] [PubMed] [Google Scholar]
- 4.Ismail R, Azuara-Blanco A, Ramsay CR. Outcome Measures in Glaucoma: A Systematic Review of Cochrane Reviews and Protocols. J Glaucoma September 2015;24(7):533–538. [DOI] [PubMed] [Google Scholar]
- 5.Brown RH, Lynch M, Leef D, et al. The Advanced Glaucoma Intervention Study (Agis) .1. Study Design and Methods and Base-Line Characteristics of Study Patients. Controlled Clinical Trials August 1994;15(4):299–325. [DOI] [PubMed] [Google Scholar]
- 6.Heijl A, Leske MC, Bengtsson B, Hyman L, Bengtsson B, Hussein M. Reduction of intraocular pressure and glaucoma progression: results from the Early Manifest Glaucoma Trial. Arch Ophthalmol October 2002;120(10):1268–1279. [DOI] [PubMed] [Google Scholar]
- 7.Musch DC, Lichter PR, Guire KE, Standardi CL, Grp CS. The Collaborative Initial Glaucoma Treatment Study - Study design, methods, and baseline characteristics of enrolled patients. Ophthalmology April 1999;106(4):653–662. [DOI] [PubMed] [Google Scholar]
- 8.Gordon MO, Kass MA. The Ocular Hypertension Treatment Study: design and baseline description of the participants. Arch Ophthalmol May 1999;117(5):573–583. [DOI] [PubMed] [Google Scholar]
- 9.Parrish RK 2nd, Feuer WJ, Schiffman JC, Lichter PR, Musch DC, Group CODS. Five-year follow-up optic disc findings of the Collaborative Initial Glaucoma Treatment Study. Am J Ophthalmol April 2009;147(4):717–724 e711. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Strouthidis NG, Scott A, Peter NM, Garway-Heath DF. Optic disc and visual field progression in ocular hypertensive subjects: detection rates, specificity, and agreement. Invest Ophthalmol Vis Sci July 2006;47(7):2904–2910. [DOI] [PubMed] [Google Scholar]
- 11.Leibowitz HM, Krueger DE, Maunder LR, et al. The Framingham Eye Study monograph: An ophthalmological and epidemiological study of cataract, glaucoma, diabetic retinopathy, macular degeneration, and visual acuity in a general population of 2631 adults, 1973-1975. Surv Ophthalmol May-Jun 1980;24(Suppl):335–610. [PubMed] [Google Scholar]
- 12.US Department of Health and Human Servides FDA. Guidance for Clinical Trial Sponsors: Establishment and operation of clinical trial data monitoring committees. http://www.fda.gov/downloads/Regulatgory%20information/Guidances/ucm127073.pdf.
- 13.European Medicines Agency. Committee for medicinal products for human use (CHMP): guideline on data monitoring committees. [DOI] [PubMed]
- 14.Keltner JL, Johnson CA, Quigg JM, Cello KE, Kass MA, Gordon MO. Confirmation of visual field abnormalities in the Ocular Hypertension Treatment Study. Ocular Hypertension Treatment Study Group. Arch Ophthalmol September 2000; 118(9): 1187–1194. [DOI] [PubMed] [Google Scholar]
- 15.Ca Johnson, Keltner JL Cello KE, et al. Baseline visual field characteristics in the ocular hypertension treatment study. Ophthalmology March 2002;109(3):432–437. [DOI] [PubMed] [Google Scholar]
- 16.Feuer WJ, Parrish RK 2nd, Schiffman JC, et al. The Ocular Hypertension Treatment Study: reproducibility of cup/disk ratio measurements over time at an optic disc reading center. Am J Ophthalmol. January 2002;133(1):19–28. [DOI] [PubMed] [Google Scholar]
- 17.Kass MA, Heuer DK, Higginbotham EJ, et al. The Ocular Hypertension Treatment Study: a randomized trial determines that topical ocular hypotensive medication delays or prevents the onset of primary open-angle glaucoma. Arch Ophthalmol June 2002;120(6):701–713; discussion 829-730. [DOI] [PubMed] [Google Scholar]
- 18.Vannabouathong C, Saccone M, Sprague S, Schemitsch EH, Bhandari M. Adjudicating outcomes: fundamentals. J Bone Joint Surg Am July 18 2012;94 Suppl 1:70–74. [DOI] [PubMed] [Google Scholar]
- 19.Kahan BC, Feagan B, Jairath V. A comparison of approaches for adjudicating outcomes in clinical trials. Trials. June 8 2017;18(1):266. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Seltzer JH, Heise T, Carson P, et al. Use of endpoint adjudication to improve the quality and validity of endpoint assessment for medical device development and post marketing evaluation: Rationale and best practices. A report from the cardiac safety research consortium. American Heart Journal August 2017; 190:76–85. [DOI] [PubMed] [Google Scholar]
- 21.Zhao W, Pauls K. Architecture design of a generic centralized adjudication module integrated in a web-based clinical trial management system. Clin Trials. April 2016; 13(2):223–233. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Pogue J, Walter SD, Yusuf S. Evaluating the benefit of event adjudication of cardiovascular outcomes in large simple RCTs. Clin Trials June 2009;6(3):239–251. [DOI] [PubMed] [Google Scholar]
- 23.Viswanathan AC, Crabb DP, McNaught AI, et al. Interobserver agreement on visual field progression in glaucoma: a comparison of methods. Br J Ophthalmol June 2003;87(6):726–730. [DOI] [PMC free article] [PubMed] [Google Scholar]