Evaluation of subjects with a moderate cup to disc ratio using optical coherence tomography and Heidelberg retina tomograph 3: Impact of the disc area

Abstract

Aim:

The aim was to evaluate subjects with a moderate cup to disc ratio using optical coherence tomograph (OCT) and Heidelberg retina tomograph (HRT) 3.

Settings and Design:

We included 80 patients with early glaucoma and 80 nonglaucomatous subjects with moderate cup/disc ratio (range of 0.5–0.8) to this cross-sectional study.

Subjects and Methods:

We compared results of color-coded algorithms of HRT 3 (Moorfields regression analysis [MRA] and Glaucoma probability score [GPS]) and OCT. All outputs are classified into three categories: Within normal limits (WNLs), borderline and outside normal limits (ONLs). Diagnostic accuracies of algorithms were determined using the highest sensitivity criteria.

Results:

The sensitivities of global MRA, GPS and OCT were 0.75, 0.925 and 0.725, respectively, in average disc area group and 0.85, 1.0 and 0.425, respectively, in large disc area group. The specificities of global MRA, GPS and OCT were 0.55, 0.15 and 0.85, respectively, in average disc area group and 0.425, 0.025 and 0.80, respectively, in large disc area group. Area under receiver operating characteristic curve (AUROC) of global MRA, GPS and OCT were 0.667, 0.617 and 0.792, respectively, in average disc area group and 0.746, 0.576 and 0.627, respectively, in large disc area group. AUROC of global MRA and OCT combination in the average and large disc area groups were 0.828 and 0.825, respectively.

Conclusions:

In contrast to GPS and OCT algorithms, diagnostic performance of MRA algorithm increased in large disc area group. Combining MRA and OCT algorithms produced satisfactory diagnostic performance in subjects with an average and large disc area.

Glaucoma is an optic neuropathy characterized by a specific and progressive injury to the optic nerve and retinal nerve fiber layer (RNFL).[1] The vertical cup/disc (c/d) ratio has long been used in the assessment of the suspected glaucoma, although the wide range of c/d ratio values in the normal population, from 0.00 to 0.87, limits its use.[2,3,4]

Among the various glaucoma screening devices, optical coherence tomography (OCT) and confocal scanning laser ophthalmoscopy (CSLO) have gained popularity in clinical practice.[5] The Heidelberg retina tomograph 3 (HRT 3; Heidelberg Engineering GmbH, Heidelberg, Germany) is a CSLO that acquires three dimensional topographic images of the optic nerve head and RNFL. Spectral domain OCT (SD OCT) is extensively used to evaluate retinal, optic nerve and RNFL pathologies.[6]

The incorporation of built-in normative databases is one of the inherent strengths in using automated imaging technologies in the setting of diagnostic or therapeutic monitoring.[7] This data incorporation allows for the direct comparison of the optic disc and RNFL features measured in clinic-based patients to age-matched individuals from the general population.[7] HRT 3 has two and OCT has one automatic classification algorithms. Moorfields regression analysis (MRA) of the HRT 3 requires the placement of a contour line and compares a subject's rim area with the predicted rim area for a given disc area and age from a normative database.[8] The glaucoma probability score (GPS) uses five parameters (cup size, cup depth, rim steepness, and horizontal and vertical RNFL curvatures) for input into a vector machine learning classifier that estimates the probability of the presence of damage consistent with glaucoma.[9] The MRA, GPS and OCT algorithms reveal color-coded classifications that are easy to read and can establish either a global or a sectoral distinction between healthy and diseased subjects.

Objective optic disc analysis systems are particularly important in those cases that pose a challenge to clinicians. However, the performance of automated systems in atypical optic discs (for example, those with small or large discs or those with a moderate c/d ratio) has not yet been satisfactorily addressed. The assessment of optic disc size is an important component of optic nerve examination as the size of the neuroretinal rim and the optic cup vary with disc size.[10,11,12]

If we are aware of the strengths and limitations of optic disc analysis with HRT 3 and OCT in different clinical settings, objective optic disc analyses can support and guide decision-making in clinical practice, especially in clinical challenges such as subjects having a moderate c/d ratio. The aim of this study was to compare the MRA, GPS and OCT algorithms to discriminate early primary open angle glaucoma (POAG) patients and nonglaucomatous subjects with moderate c/d ratios (range of 0.5–0.8) and to evaluate the influence of disc size on diagnostic accuracy.

Subjects and Methods

This cross-sectional study included 80 nonglaucomatous subjects and 80 early POAG patients with moderate vertical c/d ratios between January 2013 and March 2014. The study was performed in adherence with the tenets of the Declaration of Helsinki and was approved by the ethics committee of Abant Izzet Baysal University of Medicine (Number: 2012/265). Informed consent was obtained from all study participants. According to the normative database of HRT 3, the subjects were divided into average (1.63–2.43 mm²) and large (>2.43 mm²) disc area groups.

To be included, the subjects had to have at least 1-year of follow-up and three reliable visual field analyses performed at our clinic, a best-corrected visual acuity (BCVA) of 20/30 or better, refraction within ± three-dimensional of spherical and ± one-dimensional cylindrical values, open angles on gonioscopy and the absence of ocular disease that would prevent the examination of corneal and retinal states or the presence of prior ocular intervention including surgery, laser, or injection other than uncomplicated cataract surgery. Those participants with an abnormal disc appearance including tilted disc, Bergmeister papilla or disc coloboma, were excluded from the study. If both eyes met the inclusion criteria, one eye was randomly selected for the study.

Nonglaucomatous subjects had an intraocular pressure (IOP) of 21 mmHg or less, no past history of raised IOP, a vertical c/d ratio ranging from 0.5 to 0.8 on optic disc photographs by two masked graders (i.e. no focal thinning or notching, c/d ratio asymmetry <0.2 between the two eyes and no optic disc hemorrhage or RNFL defects). The visual field confirmation of the disease status was used as the analysis reference by determining the mean defect (MD) and loss variance (LV) within the 95% confidence interval (CI) of the subjects with reliable visual field analysis of the subjects (false positive and false negative errors were <20%). Subjects were considered normal in the presence of at least three normal reproducible visual fields (MD loss < 2 dB with LV < 6 dB²), a normal clinical examination and no family history of glaucoma. Glaucoma patients had a medically controlled IOP and at least one IOP measurement reading of >21 mmHg using a Goldmann applanation tonometer and they were classified as having early glaucoma by the existence of at least three reproducible visual field losses (MD loss of 2–6 dB with LV >6 dB², fewer than 18 points depressed below the 5% probability level, fewer than 10 points below the 1% level and no point with a sensitivity of <15 dB in the central 5° of fixation) and visual field loss consistent with optic nerve damage.[9] We presented data of the last visual field analysis of the subjects that was taken on the same day as the HRT 3 and OCT imaging.

Each participant underwent a complete ophthalmologic examination including BCVA, slit lamp biomicroscopy, IOP measurement using Goldmann applanation tonometry, gonioscopy using the Goldmann three-mirror lens, dilated fundus and optic disc examination using super 66 lens (Volk Optical Inc., Ohio, USA), standard automated perimetry using program G2 of Octopus 101 (Haag–Streit AG, Koeniz, Switzerland), CSLO using HRT 3 software version 1.5.1 (Heidelberg Engineering, Heidelberg, Germany) and SD OCT imaging using Spectralis OCT software version 5.3 (Heidelberg Engineering, Heidelberg, Germany). We did not perform any contact ocular measurements before HRT and OCT imaging. Evaluation of vertical c/d ratio was based on the assessment of the optic disc photographs (Canon CF-60dsi, Canon Inc., Tokyo, Japan). The photographs were evaluated by two experienced graders (FU and ÜD), and each was masked to the subject's diagnosis. All participants underwent an optic disc examination with the standard scanning modes of the HRT 3 and SD OCT instruments by the same operator. The criteria for determining the scan quality were the following: Image quality score <30 μm with a good imaging quality score for HRT 3 and image quality score > 20 for SD OCT. The same trained operator outlined the optic disc margin (the inner margin of Elschnig's rim) on the mean topographic image of HRT 3 by positioning 6–8 points. The data were analyzed by the HRT 3 software with Caucasian adjustment for ethnicity.

All automatically generated data points from both CSLO and OCT were classified as being within normal limits (WNLs), borderline (BL) and outside normal limits (ONLs). We also converted the MRA into a continuous variable by subtracting the predicted MRA from the actual MRA.[13] GPS output gives a probability score that is automatically classified into three categories: ONL for scores of 65–100%, BL for scores of 28–64% and WNL for scores of 0–27%.

The subjects were classified into four groups depending on the disc area cut-offs provided by the manufacturer (normal limit ranging from 1.63 to 2.43 mm²). Group 1 included nonglaucomatous subjects with an average disc area. Group 2 included nonglaucomatous subjects with a large disc area. Group 3 included POAG patients with an average disc area. Group 4 included POAG patients with a large disc area.

The data were analyzed using SPSS statistical package version 21.0 (SPSS Inc, Chicago, IL). For general statistical reporting, the mean values from each data set were calculated with the standard deviation. The outcome parameters of WNL, BL, and ONL were treated as ordinal data.[13] Differences among groups were assessed by a one-way analysis of variance for continuous parameters, Chi-square test for categorical parameters, and Kruskal–Wallis and Mann–Whitney U-tests for ordinal parameters. The level of statistical significance was chosen as P < 0.05. Measurements of c/d ratio were compared using a paired-samples t-test, and their relationships were assessed using Pearson's coefficient test. The sensitivity and specificity of the MRA, GPS, and OCT algorithms were compared for both global and sectoral results. The BL category of these tests aroused suspicion for the early diagnosis of disease. Therefore, the BL values were considered ONL for the estimation of sensitivity and specificity to evaluate this group of subjects with the highest level of sensitivity.[13,14] The agreement between the global classification algorithms was analyzed by the kappa coefficient (kappa, 0.00–0.29 = weak agreement, 0.30–0.59 = moderate agreement, 0.60–0.89 = good agreement, 0.90–1.00 = optimal agreement). Receiver operating characteristic (ROC) curves were used with ordinal categories to determine the discrimination capabilities between healthy and glaucomatous eyes. The area under the ROC curves (AUROC) was calculated for the three-level variables (WNL, BL, ONL). We also calculated the AUROC of the global MRA and global GPS as continuous variables.

Results

The demographic and clinical characteristics of the study sample and their statistical significances are shown in Table 1. The range of the disc area for healthy subjects with an average and large disc areas was 1.66–2.37 and 2.56–3.56 mm², respectively. The range of the disc area for glaucoma patients with average and large disc areas was 1.55–2.32 and 2.57–3.56 mm², respectively. There was statistically significant difference between the c/d measurement of graders (P < 0.001). However, correlation of c/d measurement between the graders was statistically significant (r = 0.79, P < 0.001). We included subjects, whose c/d ratio was measured between 0.5 and 0.8 by both graders (FU and ÜD). We excluded three subjects in the nonglaucomatous group who had visual field loss progression due to lens opacity. We excluded two glaucoma patients who had visual field loss progression due to moderate stage glaucoma.

Table 1.

Demographic and clinical characteristics of the groups

Moorfields regression analysis had better sensitivity, especially in patients with a large disc area, and OCT had better specificity in subjects with average and large disc areas [Table 2]. GPS had a very high sensitivity with a too low specificity, which limited its diagnostic accuracy [Table 2]. Interestingly, although the sensitivity of the HRT 3 algorithms increased with the large disc areas, the sensitivity of OCT decreased in the large disc area patients [Table 2].

Table 2.

Sensitivity and specificity of the MRA, GPS and OCT test results

In the average disc area group, the AUROC of the global OCT sector was higher (0.792), and in the large disc area group, the AUROC of the global MRA sector was higher (0.746) [Table 3]. In the average disc area group, the AUROC for the global sector of the MRA and OCT together revealed a score of 0.828 (P < 0.001 and CI of 0.735–0.921) [Fig. 1]. In the large disc area group, the AUROC for the global sector of the MRA and OCT together revealed a score of 0.825 (P < 0.001 and CI of 0.734–0.916) [Fig. 1]. Combining the MRA and OCT algorithms produced a satisfactory diagnostic performance increment in both the average and large disc area groups but the combination with GPS did not produce such an effect (AUROC of 0.851 and 0.828, respectively).

Table 3.

ROC curve analyses for MRA, GPS and OCT test results

Figure 1.

The receiver operating curve (ROC) plots constructed for parameters of global Moorfields regression analysis (MRA), global optical coherence tomography (OCT) and combination of MRA and OCT. The dotted line represents diagnostic ability of MRA alone, dashed line represents diagnostic ability of OCT alone and solid line represents diagnostic ability of MRA and OCT combination. (a) ROC plot of subjects with average disc area, (b) ROC plot of subjects with large disc area

In the average disc area group, the AUROC of the MRA as ordinal and continuous variables was 0.674 (P = 0.007) and 0.707 (P = 0.001), respectively, and the AUROC of the GPS as ordinal and continuous variables was 0.617 (P = 0.072) and 0.636 (P = 0.036), respectively. In the large disc area group, the AUROC of the MRA as ordinal and continuous variables was 0.746 (P < 0.001) and 0.784 (P < 0.001), respectively, and the AUROC of the GPS as ordinal and continuous variables was 0.576 (P = 0.051) and 0.689 (P = 0.004), respectively.

In the average disc area group, the sectoral kappa values of the MRA and GPS results, the MRA and OCT results, and the GPS and OCT results ranged from 0.057–0.225, 0.051–0.373 and 0.009–0.142, respectively. In the large disc area group, the sectoral kappa values of the MRA and GPS results, the MRA and OCT results, and the GPS and OCT results ranged from 0.012–0.184, 0.016–0.220 and 0.001–0.019, respectively.

In the global sector of Group 1, the MRA and OCT algorithms revealed ONL together for only 1 subject and BL or ONL for two subjects. In the global sector of Group 2, there was no subject that these algorithms revealed ONL together, and there were four subjects who were scored as BL or ONL. In the global sector of Group 1, there were five subjects that both algorithms revealed as WNL together. In the global sector of the Group 4, there were four subjects that both algorithms revealed as WNL together. Positive likelihood ratio and negative likelihood ratio of the MRA, GPS, and OCT algorithms for both global and sectoral results are presented in Table 4.

Table 4.

Effect of disc size on likelihood ratio of MRA, GPS and OCT test results

Discussion

We assessed the diagnostic accuracy of HRT 3 and OCT in a Turkish population including early stage glaucoma and nonglaucomatous subjects having a cup to disc (c/d) ratio ranging from 0.5 to 0.8 with average and large disc areas. The major strength of this study was that all subjects had at least three reliable and reproducible visual field analyses and at least 1-year of follow-up.

There are several reports about the HRT results in glaucoma and healthy subjects, but the different study designs, studied populations, inclusion and exclusion criteria and glaucoma stages make it difficult to compare the results among these studies. The severity of the glaucoma is important because various studies have reported that the sensitivities of algorithms decrease in eyes with early glaucoma.[7,13,14,15] It is well-established that clinicians must take disc area into account when interpreting the MRA and GPS classifications.[16,17] Our results also showed that the MRA, GPS and OCT algorithms were affected by the disc area.

Using the highest sensitivity criteria, Ferreras et al.[18] reported the sensitivities of global MRA and GPS to be 84.4% and 93.3%, respectively, and the specificities to be 83.8% and 58.0%, respectively, in a group of subjects with similar disc areas (2.04 in the control group and 2.16 in the glaucoma patients). Using the highest specificity criteria, Zangwill et al.[13] determined the sensitivities of global MRA and GPS to be 67.7% and 71.7%, respectively, and the specificities to be 88.7% and 82.3%, respectively, in a group with an average disc area of 1.76 mm² for healthy subjects and 1.97 mm² for POAG patients with early stage visual field defects but they did not report the c/d ratios. They reported AUROC scores of 0.74 and 0.70 for the color-coded classifications of global MRA and GPS, respectively.[13] Using the highest sensitivity criteria, Coops et al.[19] reported the sensitivities of global MRA and GPS to be 78% and 78%, respectively, and the specificities to be 66% and 63%, respectively, in a group with an average disc area of 1.9 mm² for healthy subjects and 2.0 mm² for POAG patients with early stage visual field defects. They also reported AUROC scores for MRA and GPS to be 0.77 and 0.78, respectively, but did not report the c/d ratios and kappa values.[19] The higher sensitivity and much lower specificity levels of the current study compared with those of the previous reports, especially for the GPS algorithm, are most likely due to the higher c/d ratios of subjects with average disc area and larger disc area and higher c/d ratios of subjects with large disc area.

In a Turkish population, according to the highest sensitivity criteria, Bozkurt et al.[20] reported a global MRA sensitivity and specificity of 81.0% and 75.0%, respectively, and a global GPS sensitivity and specificity of 89.2% and 57.6%, respectively, in a group with a c/d ratio of 0.23 for healthy subjects and 0.44 for POAG patients. The sensitivities of MRA and GPS are similar to the values in this study, but the specificity is higher than that in this study, which might be due to the differences in the disc area and c/d ratios of the included subjects.

The published data on the comparison of HRT and SD OCT have favored OCT over HRT. Leaney et al.[21] reported that Spectralis OCT measurements did not vary greatly across the disease severity groups in glaucoma patients.[21] Moreno-Montañés et al.[22] reported the sensitivity of RNFL damage detection using HRT 3 was lower compared with the stratus time domain OCT, especially in early glaucoma. Sato et al.[23] reported that there was a poor agreement between HRT 3 and Cirrus SD OCT. Shpak et al.[24] reported that Cirrus SD OCT might be better than HRT 3 for monitoring the patients with early glaucomatous visual field defects. Lisboa et al.[25] reported that the RNFL assessment with Spectralis OCT performed well in detecting preperimetric glaucomatous damage in a cohort of suspected glaucoma patients and had a better performance than HRT 3. We determined the dependence of the MRA, GPS and OCT algorithms on to the disc size in subjects with moderate c/d ratio. Both MRA and GPS showed a higher sensitivity and lesser specificity in large optic discs, but the situation was different for the OCT algorithm. OCT had a better sensitivity and specificity in average optic discs most likely due to the normative database of OCT, which includes only 201 subjects with different age groups or the standard manually adjusted circumpapillary ring used by OCT. Automated algorithms such as GPS and O OCT need to have a better and larger normative databases that are representative of all clinical variable, including disc size and c/d ratio. Defining a contour line might be a disadvantage for standard subjects, especially for inexperienced operators, but we believe that defining the disc contour manually becomes an advantage for MRA compared with GPS and OCT, especially in subjects with a larger disc area. The dependence of the MRA, GPS and OCT algorithms on to the disc size is known and according to results of this study we propose another factor is c/d ratio.

We could not eliminate the probability of preperimetric normal-tension glaucoma in the nonglaucomatous subjects included in this study was a limitation of the current study.

Disc size, disease severity and other covariates likely have more of an effect on the diagnostic accuracy of these devices. The results of this study revealed that in addition to disc area, the c/d ratio of a subject might affect the diagnostic accuracy of the HRT 3 and OCT algorithms. In the large disc area group the diagnostic performance of the automated algorithms (GPS and OCT) were decreased, but the diagnostic performance of the MRA algorithm increased. Combining the results of the MRA and OCT algorithms produced a satisfactory diagnostic performance in such a challenging group. Further studies are needed to determine the clinical applicability of the normative database of the HRT 3 and OCT algorithms in different clinical cases.