Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Sep 1.
Published in final edited form as: Am J Ophthalmol. 2020 Mar 25;217:131–139. doi: 10.1016/j.ajo.2020.03.024

Gradient Boosting Classifiers Combining Vessel Density and Tissue Thickness Measurements for Classifying Early to Moderate Glaucoma

Christopher Bowd 1, Akram Belghith 1, James A Proudfoot 1, Linda M Zangwill 1, Mark Christopher 1, Michael H Goldbaum 1, Huiyuan Hou 1, Rafaella C Penteado 1, Sasan Moghimi 1, Robert N Weinreb 1
PMCID: PMC7492367  NIHMSID: NIHMS1605107  PMID: 32222368

Abstract

Purpose

To compare gradient boosting classifier (GBC) analysis of optical coherence tomography angiography (OCTA)-measured vessel density (VD) and OCT-measured tissue thickness to standard OCTA VD and OCT thickness parameters for classifying healthy eyes and eyes with early to moderate glaucoma.

Design

Comparison of diagnostic tools.

Methods

One-hundred-eight healthy eyes and 193 glaucomatous eyes with OCTA and OCT imaging of the macula and optic nerve head (ONH) were studied. Four GBCs were evaluated that combined 1) all macula VD and thickness measurements (Macula GBC), 2) all ONH VD and thickness measurements (ONH GBC), 3) all VD measurements from the macula and ONH (Vessel Density GBC), and 4) all thickness measurements from the macula and ONH (Thickness GBC). ROC curve (AUROC) analyses compared the diagnostic accuracy of GBCs to standard instrument provided parameters. A fifth GBC that combined all parameters (Full GBC ) also was investigated.

Results

GBCs had better diagnostic accuracy than standard OCTA and OCT parameters with AUROCs ranging from 0.90 to 0.93 and 0.64 to 0.91, respectively. The Full GBC (AUROC=0.93) performed significantly better than the ONH GBC (AUROC=0.91, p=0.036) and the Vessel Density GBC (AUROC=0.90, p=0.010). All other GBCs performed similarly. The mean relative influence of each parameter included in the Full GBC identified a combination of macular thickness and ONH VD measurements as the greatest contributors.

Conclusion

GBCs that combine OCTA and OCT macula and ONH measurements can improve diagnostic accuracy for glaucoma detection compared to most, but not all, instrument provided parameters.

INTRODUCTION

Optical coherence tomography angiography (OCTA) is a relatively new optical imaging technology that allows non-invasive measurement of vessel density using two subsequent aligned OCT images to detect between-image changes in relative voxel position that indicate the presence of flowing blood. OCTA vessel density measurements (percentage of OCTA image occupied by regions of flowing blood) have been shown to be lower in glaucoma suspect and glaucoma eyes compared to healthy eyes, to decrease with increasing glaucoma severity as described by decreasing standard automated perimetry visual field (VF) mean deviation (MD) and to be lower in the corresponding perimetrically unaffected hemiretina of eyes with hemifield-specific VF defects.eg. 1–10 In addition, baseline parapapillary and macular vessel density measurements are reportedly predictive of more rapid glaucoma-related thinning of the retinal nerve fiber layer (RNFL)11 and a significant decrease in superficial macular vessel density has been reported over time in glaucoma eyes in conjunction with a thinning of the ganglion cell complex (GCC).12

Current OCTA instruments provide limited analytics beyond global and local vessel density measurements obtained from the perifoveal and peripapillary regions. Available numeric vessel density measurements are difficult to assess in part because comparisons to standard normative measurements from healthy eyes used to determine outside of normal limits measurements are not available. Therefore, vessel density measurements obtained using OCTA currently are difficult to interpret beyond subjective visual inspection of en face images and comparison of relative regional vessel densities.

We and others have shown over the last decades that machine learning (ML) models developed from information available in optic nerve head photographs, optical imaging measurements and visual field measurements, alone and combined, can improve classification of healthy and glaucoma patient eyes compared to expert assessment, raw measurements and commercially available instrument manufacturer-provided measurement parameters.13–35 Results provided by ML models allow relatively easy assessment because they can provide output in the form of probability of disease based on information learned from training data. The purpose of the current study was to determine if ML models trained and tested on OCTA vessel density measurements can improve disease classification compared to vessels density measurements alone. We also compared and combined localized OCTA vessel density measurements with localized OCT tissue thickness measurements to determine if combining available regional measurements would improve classification, to evaluate whether combining macular and optic nerve head measurements could contribute to clinical decision making.

MATERIALS AND METHODS

This was a cross-sectional comparison of diagnostic tools involving a group of patients with primary open-angle glaucoma (POAG, as defined below) and a group of healthy control participants from the Diagnostic Innovations in Glaucoma Study (DIGS) (ClinicalTrials.gov identifier: NCT00221897). The DIGS is an ongoing prospective, longitudinal study at the Hamilton Glaucoma Center, University of California, San Diego, designed to evaluate anatomical structure in glaucoma. Details of the DIGS protocol have been described elsewhere.36 All methods adhered to the tenets of the Declaration of Helsinki and the Health Insurance Portability and Accountability Act and were approved by the Institutional Review Board of the University of California, San Diego.

Participants

Eligible participants had best corrected visual acuity of 20/40 or better and open angles on gonioscopy at study entry. All participants were older than 40 years. Participants were excluded if they had a history of intraocular surgery (except for uncomplicated cataract or uncomplicated glaucoma surgery). Eyes with coexisting retinal disease, uveitis, or non-glaucomatous optic neuropathy also were excluded. Diabetic participants with no evidence of retinal involvement were included.

Eyes of healthy participants had healthy appearing optic discs and RNFLs OU based on masked assessment of digital stereoscopic photographs with no history of repeatable abnormal VF results (HFA II with 24–2 testing using the Swedish Interactive Thresholding Algorithm, Carl Zeiss Meditec Inc., Dublin, CA) and no history of elevated intraocular pressure (IOP) (all IOP ≤ 21 mm Hg) in either eye. Normal VFs were defined as those with MD and pattern standard deviation (PSD) within 95% confidence limits (95% CI) and a Glaucoma Hemifield Test (GHT) result within normal limits.

Glaucoma patient eyes had at least 2 consecutive and reliable (defined below) VF examinations with either PSD ≥ 5% or a GHT result outside of the 99% normal limits with similar patterns of glaucoma-related defects in consecutive exams. Severity of glaucoma was restricted from early to moderate with MD ≥ −12 dB.

Visual Field Testing

All VFs were evaluated by UC San Diego Visual Field Assessment Center (VisFACT) personnel based on a standardized protocol.36 Visual fields with more than 33% fixation losses or false-negative errors or more than 15% false-positive errors were automatically excluded. Visual fields exhibiting a learning effect (i.e. initial tests with reduced sensitivity followed by consistent improvement in a series of tests) also were excluded. Visual fields were further reviewed for lid and rim artifacts, fatigue effects, evidence that the visual field results were due to a disease other than glaucoma (e.g. homonymous hemianopia), and inattention. Test results indicating such characteristics were excluded.

Optical Imaging

The commercially available Avanti Angiovue (Optovue Inc. Fremont, CA) combines OCTA and OCT imaging in a single system. Avanti Angiovue OCTA (software version 2017.1.0.151) and OCT (software version 5.6.3.0) images of the macula and optic nerve head were obtained using the Avanti on the same day by the same instrument operator. The Avanti system for measuring vessel density and tissue thickness has been described previously.37

For OCTA vessel density measurements, each volume is composed of 2 consecutive B-scans captured at each fixed position. The instrument-specific split-spectrum amplitude-decorrelation angiography (SSADA) method is used to capture the dynamic motion of the red blood cells between consecutive B-scans to provide a high-resolution 3-dimensional visualization of the perfused retinal vasculature.37 Vessel density is calculated automatically as the proportion of measured area occupied by flowing blood vessels defined as pixels having decorrelation values acquired by the SSADA algorithm above the threshold level.

Avanti high density 6 × 6 mm2 field of view scans centered on the macula and high density 4.5 × 4.5 mm2 field of view scans centered on the optic nerve head (ONH) were acquired and included in the analysis. Macula scan parafoveal measures are obtained within an annular region with an inner diameter of 1 mm and outer diameter of 6 mm. To obtain OCTA vessel density measurements each scan is automatically segmented by the Angiovue software to visualize the instrument defined macular superficial capillary plexus and deep capillary plexus. The ONH circumpapillary measures are obtained from within a 10 pixel–wide annulus with an inner diameter of 3.45 mm.

Ganglion cell complex (GCC, composed of retinal nerve fiber layer, ganglion cell layer and inner plexiform layer) thickness and circumpapillary retinal nerve fiber layer (cpRNFL) thickness measurements derived from the OCTA scans also were included in the analyses. All described measurements are included in standard clinical printouts and can be automatically exported from the Avanti instrument.

To construct parsimonious ML models, we selected 18 macula OCTA parameters from the superficial and deep capillary plexuses, 8 ONH OCTA parameters including parameters with and without large vessels removal, 9 macula OCT GCC thickness parameters and 3 ONH OCT cpRNFL thickness parameters. Chosen parameters are all listed along the x-axis of Figure 1 (see Results) and are shown in Supplemental Tables 1 and 2.

Figure 1:

Figure 1:

Open in a new tab

Mean relative influence over each cross-validation fold in the Full Gradient Boosting Classifier (Full GBC). The upper bar represents the maximum relative influence observed.

OCTA and OCT image quality review was completed according to the UC San Diego Imaging Data Evaluation and Analysis (IDEA) Reading Center standard protocol. Images with a quality index (QI) < 4, poor clarity, residual motion artifacts visible as irregular vessel patterns or disc boundaries on the enface angiogram, image cropping or local weak signal due to vitreous opacity, or segmentation errors that could not be corrected were excluded.

Gradient Boosting Classifier (GBC)

The GBC is an ensemble classifier that attempts to decrease error by resampling and varying the weights for individual weak learners in order to increase classification accuracy.38 In an empirical comparison study of supervised learning algorithms comparing random forests and boosted decision trees, the GBC had the best overall performance.39 Motivated by this success, we chose to use gradient boosting as our classifier. An advantage of GBC is that it provides output in the form of probability of glaucoma that may be more useful for clinical interpretation than global or local percent vessel density and tissue thickness measurements alone. GBCs also provide a metric to assess the relative influence of each parameter included in the classifier.40

Up to five images from each healthy and each patient eye obtained longitudinally were included to increase the sample size and generate more generalizable models resulting in 169 images of healthy eyes and 364 images of glaucoma eyes. Because multiple images were included from a single studied eye, images were initially weighted by the number of images available for each eye (e.g. for an eye providing five images, each would be included in the model with initial weight 0.2). Separate from the ten-fold cross-validation used to assess model performance (described below), a five-fold cross validation was used within each of the ten training sets to select the optimal number of boosting iterations and prevent overfitting the training data. We report the mean and range of the relative influence of each variable included in the Full GBC across all cross-validation folds.

Statistical Analyses

We developed 4 GBCs combining 1) all macula vessel density and thickness measurements (Macula GBC), 2) all ONH vessel density and thickness measurements (ONH GBC), 3) all vessel density measurements from the macula and ONH (Vessel Density GBC), 4) all tissue thickness measurements from the macula and ONH (Thickness GBC) to compare models based on different location specific measurements to each other and to compare models based on different structural measurements to each other. We also developed a fifth Full GBC combining all Avanti parameters included in the above GBCs to determine if combining macula and optic nerve measurements would improve classification overall. An improvement in classification here would suggest that imaging both the macula and optic nerve could improve diagnostic accuracy in patients with early to moderate glaucoma.

Areas under receiver operating characteristic curves (AUROCs) and sensitivities at fixed specificities of 80%, 85%, 90% and 95% were used to describe and compare the ability of individual vessel density and tissue thickness parameters and GBCs to discriminate glaucomatous eyes from healthy eyes. Estimates of the probability of glaucoma given by GBCs were calculated using logistic regression.

Ten-fold cross-validation was used to provide out-of-sample predictions for each logistic and GBC to avoid over optimistic estimates of classification accuracy. First, the full complement of healthy participants and glaucoma patients (and each of their contributed images) was randomly divided into 10 approximately equal, exhaustive and mutually exclusive subsets. Next, each model was trained on nine subsets and subsequently used to generate out-of-sample predictions of the probability of glaucoma on the tenth subset. This sequence was repeated 10 times, with each subset serving as the test set one time, so that each tested subject was never part of its training set and was tested only once.

Bias-corrected confidence intervals and hypothesis tests for AUCs were conducted via a clustered bootstrap with 2,000 resamples. Due to the exploratory nature of this analysis, no type I error correction for multiple comparisons was applied (as recommended by Bender & Lange41). GBC modeling and all statistical analyses were performed using the R statistical programming software (version 3.5.2). The R programming language used is an open-source statistical software that is available free as is the package ‘gbm’ used for GBC modelling herein. Please see42,43 for R GBM availability and usage information.

RESULTS

The study included 108 eyes of 60 healthy study participants and 193 eyes of 143 glaucoma patients. Summary demographic variables and measurements are shown in Table 1. Glaucoma patients were significantly older than healthy participants (mean age = 72.3 years [95% CI = 70.6, 73.9 years] vs. 61.5 years [95% CI = 58.9, 64.1 years], respectively; p < 0.001) at the time of first examination. In addition, glaucoma patients had a significantly higher rate of systemic hypertension (55.9% vs. 40%, respectively; p = 0.046) and diabetes (16.8% vs. 5.0%, respectively; p = 0.024) than healthy participants. As expected, glaucoma eyes had a lower VF mean deviation (MD) than healthy eyes (−5.70 dB vs. 0.05 dB, respectively, p < 0.001; values presented in Table 1 are MD at the time of first imaging visit if multiple imaging visits were included). Glaucoma patients also had a slightly longer mean axial length (24.4 mm [95% CI = 24.2 mm, 24.6 mm] for glaucoma eyes and 23.9 mm [95% CI = 23.5 mm, 24.2mm] for healthy eyes; p = 0.014).

Table 1:

Patient and eye characteristics by diagnosis. Mean values and 95% confidence intervals are shown for continuous variables. Statistical significance of differences in continuous and categorical variables are determined by two-sample t-tests and Fisher’s exact test for patient level variables (respectively) and linear mixed effects models for eye level variables.

Diagnosis
Healthy (n = 60, 108 Eyes) Glaucoma (n = 143, 193 Eyes) p-value
Age (years) 61.5 (58.9, 64.1) 72.3 (70.6, 73.9) <0.001
Sex (% Female) 75.0% 50.3% 0.001
Race (%)
 Non-White 45.0% 35.0% 0.073
 White 53.3% 65.0%
 Unknown 1.7% 0.0%
Hypertension (%) 40.0% 55.9% 0.046
Diabetes (%) 3 (5.0%) 24 (16.8%) 0.024
MD (dB) 0.05 (−1.17, 1.27) −5.70 (−6.50, −4.90) <0.001
IOP (mmHg)a 15.0 (13.9, 16.1) 14.5 (13.8, 15.2) 0.451
AL (mm)b 23.9 (23.5, 24.2) 24.4 (24.2, 24.6) 0.014
CCT (μm)c 547.7 (536.5, 558.8) 536.8 (529.6, 543.9) 0.109
Disc or BMO? Area (mm2) 1.9 (1.8, 2.1) 1.9 (1.8, 2.0) 0.672
Open in a new tab

Missing a29, b7, and c11 values.

The standard OCTA and OCT parameters with the highest AUROC were whole image GCC thickness (AUROC = 0.91, 95% CI = 0.86, 0.94) and whole image vessel density (AUROC = 0.91, 95% CI = 0.86, 0.94), followed by cpRNFL thickness (AUROC = 0.89, 95% CI = 0.84, 0.93) and perifoveal superficial vessel density measured within an annulus between 3 mm and 6 mm from the fovea (AUROC = 0.84, 95% CI = 0.78, 0.89). AUROCs and sensitivities at fixed specificities of 80%, 85%, 90% and 95% for all investigated OCTA and OCT parameters are available in Supplemental Table 1.

AUROCs for classifying eyes as healthy or glaucomatous for all 5 evaluated GBCs and sensitivities at fixed specificities are presented in Table 2 (see also Figure 2 which also includes AUROC results for cpRNFL thickness). The diagnostic accuracy of the Full GBC (AUROC = 0.93, 95% CI = 0.89, 0.96) was significantly better than that of both the ONH GBC (AUROC = 0.91, 95% CI = 0.86, 0.94; p = 0.036) and Vessel Density GBC (AUROC = 0.90, 95% CI = 0.86, 0.93; p = 0.010). Sensitivities at each of the chosen fixed specificities were similar across GBC models. See Supplementary Tables 1 and 2 for more detail.

Table 2:

Predictive performance metrics (areas under the receiver operating characteristic curves, AUROC; and sensitivities at fixed specificities) for each gradient boosting classifier (GBC). Significance is determined using a paired bootstrap test.

Sensitivity At
AUROC (95% CI) 80% Spec. 85% Spec. 90% Spec. 95% Spec. AUROC p-value vs. Full GBC
Full GBC 0.93 (0.89, 0.96) 87.1% 86.0% 79.1% 64.6% -
Macula GBC 0.91 (0.87, 0.95) 87.6% 80.8% 78.0% 65.7% 0.105
ONH GBC 0.91 (0.86, 0.94) 86.3% 81.3% 73.9% 66.2% 0.036
Vessel Density GBC 0.90 (0.86, 0.93) 86.5% 81.0% 76.1% 63.7% 0.010
Thickness GBC 0.91 (0.87, 0.95) 88.5% 87.4% 82.7% 61.5% 0.094
Open in a new tab

Figure 2:

Figure 2:

Open in a new tab

Area under the receiver operating characteristic curve (AUROC) plots for each Gradient Boosting Classifier (GBC).

All GBCs showed significantly better diagnostic accuracy than all superficial and deep layer macula OCTA vessel density measurements (macula vessel density AUROCs ranged from 0.64 to 0.84; all comparisons p ≤ 0.006) and the Full GBC showed significantly better diagnostic accuracy than all ONH cpRNFL thickness measurements investigated (ONH cpRNFL thickness AUROCs ranged from 0.85 to 0.89; all comparisons p ≤ 0.014). The full GBC did not, however, improve classification compared to one of nine GCC thickness measurements (whole image GCC thickness, p = 0.099) and two of eight ONH OCTA vessel density measurements (whole image vessel density, p = 0.113; circumpapillary vessel density, p = 0.095). AUROC comparisons among all GBCs and all investigated OCTA and OCT parameters are available in Supplemental Table 2.

Figure 3 shows the estimated probability of glaucoma distribution for all GBCs and indicates that images from glaucoma eyes were more accurately identified than images from healthy eyes. The distribution of probabilities was most accurate for the Full GBC which estimated more healthy eyes as having between 0% and 10% probability of glaucoma and more glaucoma eyes as having between 90% and 100% probability of glaucoma than other GBCs. Figure 1 shows the mean relative influence of each investigated parameter included in the full GBC and indicates that, for the current study population, the best classifier model is a combination of macular GCC thickness and ONH circumpapillary vessel density measurements. This finding suggests that combining tissue thickness and vessel density measurements obtained from both macula and ONH images may improve the classification of early to moderate glaucoma compared to any of the investigated imaging protocols alone.

Figure 3:

Figure 3:

Open in a new tab

Estimated probability of glaucoma from each Gradient Boosting Classifier (GBC) for healthy and glaucoma images.

DISCUSSION

The current study compared gradient boosting classifiers trained on combinations of whole image and regional OCTA and OCT macula and optic nerve head parameters to individual OCTA and OCT parameters for classifying images from healthy and glaucoma eyes. In the best case, the full GBC (that incorporated all selected OCTA and OCT parameters) showed better diagnostic accuracy than all but 3 individual parameters. Gradient boosting classifiers performed well at the classification task with AUROCs ranging from 0.90 to 0.93. These results show promise for combining structural measurements (i.e. vessel density and tissue thickness) obtained from two Avanti scanning protocols (i.e. macula and ONH) for improved detection of early to moderate glaucoma. To the best of our knowledge, this is the first study to combine both vessel density and tissue thickness in a single classifier model. It is, however, unclear whether the difference in whole AUROCs are of primary clinical importance because high sensitivities at low specificities contribute substantially to AUROC areas. Rather sensitivities at high specificities may be more clinically relevant. Results shown in Supplemental Table 1 indicate that the Full GBC was more sensitive at 90% specificity than all standard parameters investigated. Two macula GCC thickness parameters, six ONH vessel density parameter and one cpRNFL parameter were more sensitive than the least sensitive GBC (ONH GBC).

Other studies have investigated the combination of multiple individual structural parameters for classifying eyes as healthy or glaucomatous.44–46 Blumberg et al.44 compared the diagnostic performance of 19 individual ONH and RNFL Cirrus OCT parameters to a multivariable predictive model using logistic regression. The multivariable model improved the discrimination between glaucomatous and non-glaucomatous eyes (AUROC = 0.892) compared to the best single RNFL parameters (average RNFL, AUROC = 0.855). However, this improvement was not statistically significant. Baskaran et al.45 showed that in early glaucoma (MD ≥ −6.0 dB) the diagnostic performances of linear discriminant analysis (LDA) and Classification And Regression Tree (CART) combining Cirrus OCT ONH and RNFL parameters were similar to those of any single parameter investigated. For comparisons including eyes with early to moderate glaucoma (MD ≥ −12.0 dB) however, both the CART model (AUROC = 0.99) and the LDA (AUROC = 0.94) improved diagnostic accuracy compared to that of any of the single parameter (AUROC range = 0.61–0.89). Yoshida et al.46 used random forest classification to discriminate between glaucomatous and healthy eyes using a total of 151 Topcon OCT peripapillary RNFL, macular RNFL and ganglion cell-inner plexiform layer parameters. Diagnostic performance of the random forest combination of parameters (AUROC = 0.985) was significantly better than any individual parameter. Classifier performance reported in the Yoshida study should be compared to performance in the current study with caution because of their inclusion of several advanced glaucoma eyes (range: MD = 1.8 dB to −23.2) possibly driving the very large AUROCs (although average MD was −5.6 dB, similar to that reported in the current study).

Numerous other studies have attempted to improve diagnostic accuracy in glaucoma by combining both anatomical and functional measurements and have shown these combinations to be superior to single structural or functional measurements. 13,47,48 In particular, several studies have investigated different machine learning algorithms to combine structural and VF measurements to assess diagnostic accuracy in healthy and glaucomatous eyes.47,49–52 Combining OCTA measurements with functional measurements using machine learning classifiers is a future research topic of our group.

Results from the current study confirm that combining structural measurements using MLCs (and LDA) can improve classification of healthy and glaucomatous eyes. The magnitude of the improvement varies among methods based on the parameters used in the models and the characteristics of the population, particularly disease severity. However, combining structural measurements for detection of early to moderate disease, as suggested specifically by the current study, is not without limitations. For instance, clinicians may be limited to a single imaging evaluation at each visit due to time constraints or technician availability. This issue theoretically could be resolved by using a wide-angle imaging protocol with a field of view that includes both the macula and ONH regions.

This study has possible limitations. It may be suggested that including a large number of parameters in each GBC resulted in classifier over-fitting. However, ten-fold cross-validation was used to provide out-of-sample predictions for each logistic and GBC to avoid over optimistic estimates of accuracy. In addition, an increased sample size would allow division of available data into independent training and test sets. We believe that the use of cross-validation in the current study is an acceptable substitute and given the limited sample size, likely is a better method of assessing classifier performance. In addition, significantly more glaucoma patients than healthy participants had hypertension and diabetes (Table 1), both of which can affect blood flow. Following the recommendations of Janes and Pepe53 for adjusted AUROC (adjusting for covariates which impact marker observations among controls), we first performed an exploratory analysis of differences in vessel density measures amongst healthy subjects with and without hypertension. We did not find a significant difference in any macula or ONH vessel density measures (all p > 0.104) and thus did not pursue an adjusted analysis. Due to the low number of diabetic healthy subjects, we are unable to adjust for this covariate in any analysis.

In conclusion, the current results indicate that machine learning gradient boosting classifiers combining OCTA and OCT macula and optic nerve head measurements can improve diagnostic accuracy in early to moderate glaucoma compared to most, but not all, single parameters. Such techniques could be incorporated into instrument software to improve clinical usefulness.

Supplementary Material

1
2

This cross-sectional observational study compares gradient boosting classifier (GBC) analysis of optical coherence tomography angiography (OCTA)-measured vessel density (VD) and OCT-measured tissue thickness to standard OCTA VD and OCT thickness parameters for classifying healthy eyes and eyes with early to moderate glaucoma. 108 healthy eyes and 193 glaucomatous eyes with OCTA and OCT imaging of the macula and optic nerve head (ONH) were studied. Several GBCs were evaluated that combined macula and ONH VD and thickness measurements. Results suggest that GBCs that combine OCTA and OCT macula and ONH measurements can improve diagnostic accuracy for glaucoma detection compared to most, but not all, instrument provided parameters.

ACKNOWLEDGEMENTS/DISCLOSURES

a) Funding/Support:

National Institutes of Health/National Eye Institute, Bethesda MA: R01 EY029058, R21 EY027945, P30 EY022589, R01 EY027510, K12 EY024225.

An unrestricted grant from Research to Prevent Blindness, New York NY.

Footnotes

b) Financial Disclosures:

C. Bowd - None

A. Belghith - None

J.A. Proudfoot - None

L.M. Zangwill - Research Support: Carl Zeiss Meditec, Heidelberg Engineering, Optovue, Topcon Medical Systems

M. Christopher – None

M.H. Goldbaum – None

H. Hou - None

R.C. Penteado - None

S. Moghimi - None

R.N. Weinreb - Consultant: Aerie Pharmaceuticals, Allergan, Bausch&Lomb, Eyenovia, Implantdata, Novartis; Research Support: Bausch and Lomb, Carl Zeiss Meditec, Centervue, Heidelberg Engineering, Konan Medical, Optovue, Research to Prevent Blindness

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Supplemental material available at AJO.COM

REFERENCES

  • 1.Akagi T, Iida Y, Nakanishi H, et al. Microvascular Density in Glaucomatous Eyes With Hemifield Visual Field Defects: An Optical Coherence Tomography Angiography Study. American journal of ophthalmology. 2016;168:237–249. [DOI] [PubMed] [Google Scholar]
  • 2.Jia Y, Bailey ST, Hwang TS, et al. Quantitative optical coherence tomography angiography of vascular abnormalities in the living human eye. Proc Natl Acad Sci U S A. 2015;112(18):E2395–2402. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Jia Y, Morrison JC, Tokayer J, et al. Quantitative OCT angiography of optic nerve head blood flow. Biomed Opt Express. 2012;3(12):3127–3137. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Moghimi S, Bowd C, Zangwill LM, et al. Measurement Floors and Dynamic Ranges of OCT and OCT Angiography in Glaucoma. Ophthalmology. 2019;126(7):980–988. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Rao HL, Pradhan ZS, Weinreb RN, et al. Relationship of Optic Nerve Structure and Function to Peripapillary Vessel Density Measurements of Optical Coherence Tomography Angiography in Glaucoma. J Glaucoma. 2017;March 22. doi: 10.1097/IJG.0000000000000670. [DOI] [PubMed] [Google Scholar]
  • 6.Rao HL, Pradhan ZS, Weinreb RN, et al. Regional Comparisons of Optical Coherence Tomography Angiography Vessel Density in Primary Open-Angle Glaucoma. American journal of ophthalmology. 2016;171:75–83. [DOI] [PubMed] [Google Scholar]
  • 7.Suh MH, Zangwill LM, Manalastas PIC, et al. Deep Retinal Layer Microvasculature Dropout Detected by the Optical Coherence Tomography Angiography in Glaucoma. Ophthalmology. 2016;123(12):2509–2518. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Yarmohammadi A, Zangwill LM, Diniz A, et al. Optical Coherence Tomography Angiography Vessel Density in Healthy, Glaucoma Suspect, and Glaucoma Eyes. Invest Ophth Vis Sci. 2016;57(9):Oct451–Oct459. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Yarmohammadi A, Zangwill LM, Diniz-Filho A, et al. Relationship between Optical Coherence Tomography Angiography Vessel Density and Severity of Visual Field Loss in Glaucoma. Ophthalmology. 2016;123(12):2498–2508. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Wang XL, Jiang CH, Ko T, et al. Correlation between optic disc perfusion and glaucomatous severity in patients with open-angle glaucoma: an optical coherence tomography angiography study. Graef Arch Clin Exp. 2015;253(9):1557–1564. [DOI] [PubMed] [Google Scholar]
  • 11.Moghimi S, Zangwill LM, Penteado RC, et al. Macular and Optic Nerve Head Vessel Density and Progressive Retinal Nerve Fiber Layer Loss in Glaucoma. Ophthalmology. 2018;125(11):1720–1728. [DOI] [PubMed] [Google Scholar]
  • 12.Hou H, Moghimi S, Zangwill LM, et al. Progressive ganglion cell complex thickness and vessel density loss in healthy, pre-perimetric glaucoma and primary open angle glaucoma eyes. Invest Ophthalmol Vis Sci. 2019;59:E-Abstract 3218. [Google Scholar]
  • 13.Bizios D, Heijl A, Bengtsson B. Trained artificial neural network for glaucoma diagnosis using visual field data: a comparison with conventional algorithms. J Glaucoma. 2007;16(1):20–28. [DOI] [PubMed] [Google Scholar]
  • 14.Bowd C, Chan K, Zangwill LM, et al. Comparing neural networks and linear discriminant functions for glaucoma detection using confocal scanning laser ophthalmoscopy of the optic disc. Invest Ophthalmol Vis Sci. 2002;43(11):3444–3454. [PubMed] [Google Scholar]
  • 15.Bowd C, Chan K, Zangwill LM, et al. Comparison of learning neural networks and linear discriminate functions to discriminate between glaucomatous and non-glaucomatous eyes using HRT optic disc topograhy parameters. Invest Ophthalmol Vis Sci. 2001;42:s118. [Google Scholar]
  • 16.Bowd C, Medeiros FA, Zhang Z, et al. Relevance vector machine and support vector machine classifier analysis of scanning laser polarimetry retinal nerve fiber layer measurements. Invest Ophthalmol Vis Sci. 2005;46(4):1322–1329. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Brigatti L, Hoffman D, Caprioli J. Neural networks to identify glaucoma with structural and functional measurements. Am J Ophthalmol. 1996;121(5):511–521. [DOI] [PubMed] [Google Scholar]
  • 18.Burgansky-Eliash Z, Wollstein G, Chu T, et al. Optical coherence tomography machine learning classifiers for glaucoma detection: a preliminary study. Invest Ophthalmol Vis Sci. 2005;46(11):4147–4152. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Chan K, Lee TW, Sample PA, Goldbaum MH, Weinreb RN, Sejnowski TJ. Comparison of machine learning and traditional classifiers in glaucoma diagnosis. IEEE Trans Biomed Eng. 2002;49(9):963–974. [DOI] [PubMed] [Google Scholar]
  • 20.Christopher M, Belghith A, Bowd C, et al. Performance of Deep Learning Architectures and Transfer Learning for Detecting Glaucomatous Optic Neuropathy in Fundus Photographs. Sci Rep. 2018;8(1):16685. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Christopher M, Belghith A, Weinreb RN, et al. Retinal Nerve Fiber Layer Features Identified by Unsupervised Machine Learning on Optical Coherence Tomography Scans Predict Glaucoma Progression. Invest Ophthalmol Vis Sci. 2018;59(7):2748–2756. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Goldbaum MH, Sample PA, Chan K, et al. Comparing machine learning classifiers for diagnosing glaucoma from standard automated perimetry. Invest Ophthalmol Vis Sci. 2002;43(1):162–169. [PubMed] [Google Scholar]
  • 23.Goldbaum MH, Sample PA, White H, et al. Interpretation of automated perimetry for glaucoma by neural network. Invest Ophthalmol Vis Sci. 1994;35(9):3362–3373. [PubMed] [Google Scholar]
  • 24.Hothorn T, Lausen B. Bagging tree classifiers for laser scanning images: a data- and simulation-based strategy. Artif Intell Med. 2003;27(1):65–79. [DOI] [PubMed] [Google Scholar]
  • 25.Li AN, Cheng J, Wong DWK, Liu J. Integrating Holistic and Local Deep Features for Glaucoma Classification. 2016 38th Annual International Conference of the Ieee Engineering in Medicine and Biology Society (Embc) 2016:1328–1331. [DOI] [PubMed] [Google Scholar]
  • 26.Lietman T, Eng J, Katz J, Quigley HA. Neural networks for visual field analysis: how do they compare with other algorithms? J Glaucoma. 1999;8(1):77–80. [PubMed] [Google Scholar]
  • 27.Liu S, Graham S, Shulz A, et al. A Deep Learning-Based Algorithm Identifies Glaucomatous Discs Using Monoscopic Fundus Photographs. Opthalmol Glaucoma. 2018;1(1):15–22. [DOI] [PubMed] [Google Scholar]
  • 28.Madsen EM, Yolton RL. Demonstration of a neural network expert system for recognition of glaucomatous visual field changes. Mil Med. 1994;159(8):553–557. [PubMed] [Google Scholar]
  • 29.Mardin CY, Hothorn T, Peters A, Junemann AG, Nguyen NX, Lausen B. New glaucoma classification method based on standard Heidelberg Retina Tomograph parameters by bagging classification trees. J Glaucoma. 2003;12(4):340–346. [DOI] [PubMed] [Google Scholar]
  • 30.Medeiros FA, Jammal AA, Thompson AC. From Machine to Machine: An OCT-Trained Deep Learning Algorithm for Objective Quantification of Glaucomatous Damage in Fundus Photographs. Ophthalmology. 2019;126(4):513–521. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Mutlukan E, Keating D. Visual field interpretation with a personal computer based neural network. Eye. 1994;8 ( Pt 3):321–323. [DOI] [PubMed] [Google Scholar]
  • 32.Phan S, Satoh S, Yoda Y, et al. Evaluation of deep convolutional neural networks for glaucoma detection. Jpn J Ophthalmol. 2019;63(3):276–283. [DOI] [PubMed] [Google Scholar]
  • 33.Swindale NV, Stjepanovic G, Chin A, Mikelberg FS. Automated analysis of normal and glaucomatous optic nerve head topography images. Invest Ophthalmol Vis Sci. 2000;41(7):1730–1742. [PubMed] [Google Scholar]
  • 34.Wroblewski D, Francis BA, Chopra V, et al. Glaucoma detection and evaluation through pattern recognition in standard automated perimetry data. Graefes Arch Clin Exp Ophthalmol. 2009;247(11):1517–1530. [DOI] [PubMed] [Google Scholar]
  • 35.Zangwill LM, Chan K, Bowd C, et al. Heidelberg retina tomograph measurements of the optic disc and parapapillary retina for detecting glaucoma analyzed by machine learning classifiers. Invest Ophthalmol Vis Sci. 2004;45(9):3144–3151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Sample PA, Girkin CA, Zangwill LM, et al. The African Descent and Glaucoma Evaluation Study (ADAGES): design and baseline data. Arch Ophthalmol. 2009;127(9):1136–1145. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Liu L, Jia Y, Takusagawa HL, et al. Optical Coherence Tomography Angiography of the Peripapillary Retina in Glaucoma. JAMA ophthalmology. 2015;133(9):1045–1052. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Schapire RE, Freund Y, Bartlett P, Lee WS. Boosting the margin: A new explanation for the effectiveness of voting methods. Ann Stat. 1998;26(5):1651–1686. [Google Scholar]
  • 39.Caruana R, Niculescu-Mizil A. An empirical comparison of supervised learning algorithms. Paper presented at: Proceedings of the 23rd international Conference on Machine learning2006; Pittsburgh, PA. [Google Scholar]
  • 40.Friedman JH. Greedy function approximation: A gradient boosting machine. Ann Stat. 2001;29(5):1189–1232. [Google Scholar]
  • 41.Bender R, Lange S. Adjusting for multiple testing - when and how? J Clin Epidemiol. 2001;54(4):343–349. [DOI] [PubMed] [Google Scholar]
  • 42.Ridgeway G Generalized Boosted Models: A Guide to the gbm Package. https://cran.r-project.org/web/packages/gbm/vignettes/gbm.pdf Published 2019.
  • 43.Greenwell B, Boehmke B, Cunningham J. gbm: Generalized Boosted Regression Models. R package version 2.1.5. https://cran.r-project.org/web/packages/gbm/gbm.pdf Published 2019. [Google Scholar]
  • 44.Blumberg DM, Dale E, Pensec N, et al. Discrimination of Glaucoma Patients From Healthy Individuals Using Combined Parameters From Spectral-domain Optical Coherence Tomography in an African American Population. Journal of Glaucoma. 2016;25(3):E196–E203. [DOI] [PubMed] [Google Scholar]
  • 45.Baskaran M, Ong EL, Li JL, et al. Classification Algorithms Enhance the Discrimination of Glaucoma from Normal Eyes Using High-Definition Optical Coherence Tomography. Invest Ophth Vis Sci. 2012;53(4):2314–2320. [DOI] [PubMed] [Google Scholar]
  • 46.Yoshida T, Iwase A, Hirasawa H, et al. Discriminating between Glaucoma and Normal Eyes Using Optical Coherence Tomography and the ‘Random Forests’ Classifier. Plos One. 2014;9(8). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Bowd C, Hao J, Tavares IM, et al. Bayesian machine learning classifiers for combining structural and functional measurements to classify healthy and glaucomatous eyes. Invest Ophthalmol Vis Sci. 2008;49(3):945–953. [DOI] [PubMed] [Google Scholar]
  • 48.Boland MV, Quigley HA. Evaluation of a combined index of optic nerve structure and function for glaucoma diagnosis. Bmc Ophthalmol. 2011;11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Racette L, Chiou CY, Hao J, et al. Combining functional and structural tests improves the diagnostic accuracy of relevance vector machine classifiers. J Glaucoma. 2010;19(3):167–175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Kim SJ, Cho KJ, Oh S. Development of machine learning models for diagnosis of glaucoma. PLoS One. 2017;12(5):e0177726. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Raza AS, Zhang X, De Moraes CGV, et al. Improving Glaucoma Detection Using Spatially Correspondent Clusters of Damage and by Combining Standard Automated Perimetry and Optical Coherence Tomography. Invest Ophth Vis Sci. 2014;55(1):612–624. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Shigueoka LS, Vasconcellos JPC, Schimiti RB, et al. Automated algorithms combining structure and function outperform general ophthalmologists in diagnosing glaucoma. PLoS One. 2018;13(12):e0207784. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Janes H, Pepe MS. Adjusting for covariate effects on classification accuracy using the covariate-adjusted receiver operating characteristic curve. Biometrika. 2009;96(2):371–382. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1605107-supplement-1.docx (18.9KB, docx)
2
NIHMS1605107-supplement-2.docx (19.3KB, docx)

RESOURCES