Agreement in Assessing Cup-to-Disc Ratio Measurement Among Stereoscopic Optic Nerve Head Photographs, HRT II, and Stratus OCT

Abstract

Purpose

To compare the level of agreement between subjective and objective methods in estimating horizontal and vertical cupto-disc ratios (HCDR and VCDR, respectively) to determine if objective techniques may be used as surrogates for subjective cup-to-disc (CDR) estimation.

Methods

Fifty-one glaucoma patients and 49 control subjects underwent full ophthalmic examination, stereoscopic optic nerve head photographs (ONHPs), confocal scanning laser ophthalmoscopy (Heidelberg Retina Tomography II [HRT II]), and optical coherence tomography (Stratus OCT). The intra-class correlation coefficient (ICC) and Bland Altman plots were used to assess the agreement across the three methods. Repeated measures analysis of variance (ANOVA), Tukey adjustment, and pairwise P values were used to compare the HCDR and VCDR estimates between three clinicians who reviewed photos, Stratus OCT, and HRT II.

Results

For the clinicians, the agreement in subjectively assessed HCDR and VCDR was substantial (ICC = 0.84 and 0.85, respectively), and for all three methods, overall agreement was good (ICC = 0.75 and 0.77 for the HCDR and VCDR, respectively). Stratus OCT provided the largest overall mean ± SD HCDR (0.68 ± 0.14) and VCDR (0.62 ± 0.13). The smallest overall mean ± SD HCDR was provided by ONHP (0.32 ± 0.16), and the smallest overall mean ± SD VCDR was provided by HRT II (0.26 ± 0.20). Repeated measures ANOVA test demonstrated significant differences across the three methods for glaucomatous (P = 0.0017 and 0.0016, HCDR and VCDR, respectively) and normal (P = 0.0001 for both HCDR and VCDR) eyes. Tukey adjustment demonstrated specific statistical differences between pairs of methods.

Conclusions

Although the overall agreement between various methods was good, the mean estimates were statistically different. Additional studies are needed to evaluate the sources of variability, their level of significance, and longitudinal agreement between various methods of the CDR estimation.

Subjective methods of estimation of the cup-to-disc ratio (CDR) remain frequent clinical techniques of documentation of the optic nerve head status in monitoring glaucoma.¹ However, even when estimated using optic nerve head photography (ONHP), it is still limited due to high inter- and intraobserver variability and the wide range of variability in normal optic disc structure.^2–4

Quantitative, objective methods of CDR estimates include confocal scanning laser ophthalmoscopy (Heidelberg Retinal Tomography [HRT]; Heidelberg Engineering, Dossenheim, Germany) and optical coherence tomography (Stratus OCT; Humphrey Systems, Dublin, CA). HRT provides reproducible measurements of the optic nerve head topography. Stratus OCT is a noninvasive, noncontact imaging technique that produces cross-sectional images of the optic nerve, macula, and retina.^5–9 The agreement of glaucoma diagnosis, CDR measurements between the earlier versions of both HRT and OCT, and between objective and subjective methods has been evaluated previously.^1,10–13 However, the data on the agreement between subjective assessment of the CDR and the newer versions of these instruments are limited.¹⁴ The purpose of this study is to assess the degree of agreement between subjective and objective methods of the CDR estimation using commonly available clinical tests (HRT II and Stratus OCT) to determine if these more reproducible techniques may be used as surrogates for subjective CDR estimation.

PATIENTS AND METHODS

This prospective observational study was approved by the Institutional Review Board of the University of Alabama at Birmingham (UAB). All participants signed an informed consent. The study protocol adhered to the tenets of the Declaration of Helsinki. Subjects with early glaucoma and controls were identified from the UAB Optic Disc Imaging Center's database, consisting of subjects who had optic disc imaging and visual field testing between 1999 and 2004 as part of a longitudinal glaucoma study.

All participants underwent dilated biomicroscopy, gonioscopy, intraocular pressure measurement, stereoscopic disc photos, HRT II, and Stratus OCT examinations between February 2003 and December 2003. Normal participants had clinically healthy appearing optic discs as demonstrated by indirect ophthalmoscopy, intraocular pressure of ≤21 mm Hg, and normal visual fields on standard automated perimetry with Swedish Interactive Thresholding Algorithm (SAP-SITA; Carl Zeiss-Meditech, Dublin, CA). Glaucoma patients had signs of glaucomatous optic neuropathy, including excavation, undermining of the cup, thinning, or notching of the neuroretinal rim, nerve fiber layer defects, disc hemorrhages, or CDR asymmetry of more than 0.2 diagnosed with indirect ophthalmoscopy; and repeatable (two consecutive) reliable (less than 33% fixation losses, false-positive and -negative responses) glaucomatous (pattern standard deviation or cluster of three points on the total deviation probability plot outside the 95% confidence interval [CI], or Glaucoma Hemifield Test outside normal limits) visual field defect on SAP-SITA. All participants had best corrected visual acuity 20/40 or better, open angle on gonioscopy, and refraction within ± 5.0 D sphere and ± 3.0 D cylinder. Those with significant media opacities and posterior pole pathology other than glaucoma were excluded.

Following maximal pupil dilation, stereoscopic photographs of all eyes were taken with the Nidek 3 Dx camera (Nidek Co. Ltd., Tokyo, Japan). Photographs were assessed with a stereoscopic viewer (Pentax II; Asahi Optical Co., Tokyo, Japan) and evaluated for photo clarity, adequacy of stereo, centering of disc, and exposure on a scale of excellent, good, fair, and unacceptable. Photos that scored excellent and good were included in the analysis. In total, 200 photos were evaluated. Photos were graded subjectively as either normal or glaucomatous based on signs of glaucomatous optic neuropathy. In case of disagreement, the most senior grader made the final decision. Three masked glaucoma fellowship trained stereo-photo graders evaluated the photos and recorded HCDR and VCDR in decimal fashion from 0 to 1 in 0.1 increments. The mean of the three observers’ estimations was used for analysis.

Three images of each eye were obtained utilizing the HRT II. This technique has been previously described in detail.⁸ In brief, HRT II, a fully automated version of the scanning laser ophthalmoscope, acquires and analyzes three-dimensional topographic images of the optic disc, obtained by measuring the intensity of reflected light from the retinal surface. Sixteen images per millimeter are acquired, which consist of 384 × 384 pixels with the resolution of 10 μm/pixel, each being a measurement of height at its corresponding location. Following pupil dilation, images of the eyes were obtained at 15° field of view. The disc margin contour line was drawn at the inner edge of the scleral ring using the ONHP as a reference. The reference plane was set 50 μm below the surface of the temporal retina to separate the disc cup from the neuroretinal rim. A mean composite image was obtained from the three images. The HCDR and VCDR (calculated by dividing the cup diameter by the disc diameter along a line through the center of the disc horizontally or vertically, respectively) obtained from the mean composite image were used in statistical analysis. Quality criteria for HRT II images included scan sensitivity less than 90%, centered optic nerve (less than one-fourth of disc outside of target circle), appropriate brightness of image, and clear appearance of blood vessels, optic disc, and retina.¹⁵ Six hundred images conformed to these criteria.

Details of the Stratus OCT technique are described elsewhere.^9,16 In brief, Stratus OCT uses low coherence light from a super luminescent diode source to obtain images. It produces direct measurements of retinal structures based on interference between reference and measurement beams. The disc reference points are located at the level of the retinal pigment epithelium on the edges of the optic nerve head. A straight line between the two disc reference points measures disc diameter. The line parallel and 150 μm anteriorly to the disc line measures the cup diameter. The cup line posteriorly and lines extended from the disc reference points perpendicular to the disc line and up to the anterior surface of the disc laterally demarcate the rim area (Fig. 1). Following pupil dilation, six 4-mm radial line fast optic disc scans were acquired. Each scan consisted of 512 transverse × 1024 axial data points per image acquired in 1.25 seconds. Obtained HCDR and VCDR values (ratios of the longest line across the cup to the longest line across the disc horizontally or vertically, respectively) from the six scans were averaged and used for data analysis. Quality criteria for Stratus OCT fast optic disc scans included sharp scan beam and definition of vessels, scan beam centered on optic disc, optic disc centered on screen, even illumination, and signal strength ≥8. Twelve hundred scans conformed to the above quality criteria.

FIGURE 1.

Optic nerve head analysis conducted by Stratus OCT. Reference points and black arrows correspond to the outer and inner RPE edges. The solid straight blue line between the two disc reference points represents the disc diameter. Cup diameter is measured along the dashed red line, offset 150 μm anteriorly from the reference line. The straight yellow line connecting each disc reference point with the nearest point on the anterior surface represents the average of the nerve bundle widths at the disc margin bilaterally. Rim area is demarcated by the cup line posteriorly (dashed red line) and average nerve width at the disc margin (solid yellow lines).

We analyzed one randomly selected eye from each of the study participants. The intraclass correlation coefficient (ICC)^2,17 and Bland and Altman plots^14,18 were used to assess the agreement between clinicians and methods. We used ICC rather than Pearson correlation coefficient because Pearson correlation coefficient assesses correlation between variables and can be significantly high even in the presence of poor agreement.^14,18 Similar to kappa statistics, which is a chance-corrected ratio between observed agreement and perfect agreement used for categorical data analysis, the ICC is also a measurement of agreement and provides equivalent information. However, it represents a ratio of the variance between graders over the total variance and used for the continuous data analysis. An ICC ranges from 0 to 1 where 0 and 1 indicate perfect disagreement and agreement, respectively.^2,17

Bland Altman^14,18 plots are combinations of graphical and statistical interpretation of agreement evaluation, where the absolute difference between two methods (method 1 – method 2) along axis y is plotted against the average of the two (method 1+method 2)/2 along axis x. The horizontal line at the level of 0 represents the mean difference, the lines above and below it represent 95% CIs: upper (mean+2 SD) and lower (mean – 2 SD), respectively. The good agreement is present when most of the data points are situated between the 95% CIs. We also conducted repeated measures ANOVA to analyze the overall mean HCDR and VCDR of the three techniques. In addition, we used Tukey adjustment to control for the overall type 1 error for the three pairwise comparison among the three techniques.

RESULTS

The demographic data and the SAP-SITA results of the study participants are summarized in Table 1. The study population predominantly consisted of African Americans and included more females than males. Glaucoma patients were older than controls. The majority of glaucoma patients had early to moderate glaucoma.

TABLE 1.

Demographic Data of the Study Participants

Participants	Glaucoma	Controls	P
No. study eyes	51	49	0.84
Gender: females (%)	32 (48.48)	34 (51.52)	0.53
Race: blacks (%)	35 (53.85)	30 (46.15)	0.53
Mean age ± SD	53.64 ± 13.25	40.20 ± 12.06	0.51
Mean IOP ± SD	16.56 ± 3.88	14.71 ± 2.2	0.0001
Mean deviation ± SD (range)	–1.51 ± 3.78 (–15.52 to 0.08)	0.04+1.12 (–2.94 to 0.02)	0.0001
Pattern standard deviationt ± SD	2.87 ± 2.95	1.51 ± 0.38	0.0001

Measurements of HCDR and VCDR differed significantly across the three methods for both glaucomatous and normal eyes. Stratus OCT provided the largest mean HCDR and VCDR, the smallest mean HCDR was provided by ONHP, and the smallest mean VCDR was provided by HRT II, for both glaucomatous and normal eyes (Table 2). Tukey adjustment demonstrated specific statistically significant differences between pairs of methods (Table 3).

TABLE 2.

Mean, SD, and Repeated ANOVA P Values of the Horizontal Cup-to-Disc Ratio (HCDR) and Vertical Cup-to-Disc Ratio (VCDR) for the Glaucomatous and Normal Estimates Among ONHP, HRT II, and OCT 3

	Glaucoma				Normal
	HRT II	Stratus OCT	ONHP	P	HRT II	Stratus OCT	ONHP	P
HCDR	0.61 ± 0.15	0.68 ± 0.14	0.58 ± 0.17	0.0017	0.43 ± 0.19	0.50 ± 0.16	0.32 ± 0.16	0.0001
VCDR	0.52 ± 0.13	0.62 ± 0.13	0.62 ± 0.19	0.0016	0.26 ± 0.20	0.43 ± 0.14	0.33 ± 0.17	0.0001

TABLE 3.

Pairwise P Derived From the Tukey Adjustment of the Horizontal Cup-to-Disc Ratio (HCDR) and Vertical Cup-to-Disc Ratio (VCDR) for the Glaucomatous and Normal Estimates Among Optic Nerve Head Photos (ONHP), HRT II, and Stratus OCT

	Participants
	Glaucoma		Controls
Methods	VCDR P	HCDR P	VCDR P	HCDR P
HRT 11 vs. Stratus OCT	0.0001	0.0263	0.0001	0.1157
HRTI1vs. ONHP	0.0006	0.5425	0.0483	0.0025
Stratus OCT vs. ONHP	0.9016	0.0010	0.0002	0.0001

The agreement between three clinicians ranged from substantial to almost perfect for the glaucomatous and normal eyes. The overall agreement was almost perfect. All three methods, including ONHP, HRT II, and Stratus OCT, had moderate to substantial agreement for glaucomatous and normal eyes. The overall agreement was substantial (Table 4). In addition, we found a substantial overall agreement between Stratus OCT and HRT II, HRT II and ONHP (ICC = 0.68 and 0.79; 0.79 and 0.75, HCDR and VCDR, respectively) and between VCDR Stratus OCT and ONHP (ICC = 0.71). Overall agreement between HCDR OCT3 and ONHP was the highest (ICC = 0.81).

TABLE 4.

Intraclass Correlation Coefficient Among the Clinicians (Interobserver Variation) and Methods (ONHP, HRT II, and Status OCT) of the Horizontal Cup-to-Disc Ratio (HCDR) and Vertical Cup-to-Disc Ratio (VcDR) Evaluation for Glaucoma Patients, Normal Subjects, and Overall Groups

	HCDR		VCDR
Group	Clinicians	Methods	Clinicians	Methods
Glaucoma	0.76	0.61	0.76	0.60
Normal	0.78	0.71	0.77	0.69
Overall	0.84	0.75	0.85	0.77

The overall Bland Altman plots demonstrated that the HCDR and VCDR Stratus OCT measurements tended to be larger than the HCDR and VCDR HRT II values: mean difference and standard deviation were −0.08 ± 0.14 and −0.15 ± 0.15 for the HCDR and VCDR means procedures, respectively (Figs. 2A, B). The HCDR HRT II measurements tended to be larger than the HCDR ONHP, but VCDR HRT II measurements tended to be smaller than VCDR ONHP: Mean difference and standard deviation were 0.06 ± 0.16 and −0.09 ± 0.15 for the HCDR and VCDR means procedures, respectively (Figs. 2C, D). Both lower and higher HCDR and lower VCDR Stratus OCT measurements tended to be larger than the HCDR and VCDR ONHP values; however, higher VCDR Stratus OCT measurements tended to be smaller than the VCDR ONHP values: Mean difference and standard deviation were 0.14 ± 0.12 and 0.06 ± 0.12 for the HCDR and VCDR means procedures, respectively (Figs. 2E, F).

FIGURE 2.

Bland Altman plots. Bland Altman plots are the plots of difference (y axis demonstrates the absolute difference between the two methods) between the methods against their mean (x axis demonstrates the mean of the two methods). The horizontal line at the level of 0 represents the mean difference, the lines above and below it represent 95% confidence intervals (CIs): upper (mean+2 SD) and lower (mean – 2 SD), respectively. The good agreement is present when most of the data points are situated between the 95% CIs. A, The overall plots demonstrate that the horizontal cup-to-disc ratio (HCDR) of the Stratus OCT measurements tended to be larger than the HRT II values (ICC = 0.71, mean ± SD = −0.08 ± 0.14). B, Vertical cup-to-disc ratio (VCDR) of the Stratus OCT measurements also tended to be larger than the HRT II values (ICC = 0.68, mean ± -SD = −0.15 ± 0.15). C, HCDR HRT II measurements tended to be larger than the HCDR ONHP (ICC = 0.75, mean ± SD = 0.06 ± 0.16). D, VCDR HRT II measurements tended to be smaller than VCDR ONHP (ICC = 0.68, mean ± -SD = 0.09 ± 0.15). E, HCDR Stratus OCT measurements tended to be larger than the HCDR ONHP values (ICC = 0.79, mean ± SD = −0.14 ± 0.12). F, Higher VCDR Stratus OCT measurements tended to be smaller, and lower VCDR Stratus OCT measurements tended to be larger than the VCDR ONHP values (ICC = 0.81, mean ± SD = −0.06 ± 0.12).

DISCUSSION

Since early in the disease progression structural changes appear to precede visual field loss,¹⁹ detection of the optic disc damage is important for early diagnosis and follow-up management of glaucoma. This study evaluated the level of agreement between subjective and new version of objective methods in estimating HCDR and VCDR among healthy subjects and patients with early glaucoma. The results indicated a similar level of agreement between subjective and objective estimates.

CDR as an indicator of optic nerve integrity in glaucoma patients was popularized by Armaly.^2,20,21 Subsequent development of a photographic technique facilitated evaluation of the optic disc.^20–23 Further, the agreement in estimating CDR can be optimized under stereo conditions and with rigorous training.^2,24–27 However, this may be difficult to achieve in many clinical practice settings. Additionally, CDR estimation as a single ordinal parameter does not adequately describe the complex architecture of the optic disc and is limited due to high intra- and interobserver variability.^2,4,20 Despite these weaknesses, this method is still widely used in clinical practice.

Development of objective methods of evaluation of the optic disc parameters allowed for elimination of intraand interobserver variability and evaluation of the agreement between objective and subjective methods of CDR estimation. Zangwill and associates¹ demonstrated a moderate to substantial agreement with the VCDR (kappa ranged from 0.57 to 0.72) and a fair to moderate agreement with the HCDR (kappa ranged from 0.21 to 0.55) between ONHP and the original version of HRT. Hatch and associates¹¹ demonstrated a substantial to good agreement between observers using ONHP with ICCs of 0.74 and 0.83 for HCDR and VCDR, respectively, a substantial agreement between observers using planimetry (ICC = 0.79), and a moderate agreement between HRT and planimetry (ICC = 0.57 to 0.65). The authors also noted that the agreement using various methods improved with clinical training. Sung and associates,²⁷ in their evaluation of the interobserver agreement between a digital stereo optic disc camera (Discam) and HRT, found almost perfect interobserver area CDR agreement for HRT (ICC = 0.97) and substantial agreement for Discam (ICC = 0.79). Radial CDR agreement between the two methods ranged from fair to substantial (ICC ranging from 0.12 to 0.74). Schuman and associates¹³ evaluated optic nerve head measurements obtained with HRT I, OCT 2 and 3, and their association with glaucoma status. The overall Pearson correlations between these techniques were high: 0.79 for CDR measurements obtained with HRT and OCT 2 and 0.67 for the HRT and Stratus OCT. The authors also demonstrated similar association between the techniques with glaucoma status. Correnti and associates¹² assessed agreement of CDR measurements among Discam system, HRT, and ONHP. They demonstrated a substantial interobserver agreement of ONHP: ICCs for VCDR and HCDR were 0.66 and 0.67, respectively. Agreement between Discam and HRT was moderate with ICC ranging from 0.43 for the larger optic nerve heads to 0.51 for the medium-sized optic nerve heads. Agreement of VCDR and HCDR between ONHP and Discam was also moderate (0.46 and 0.53, respectively). Recently, Medeiros and associates¹⁴ evaluated agreement of HCDR and VCDR between Stratus OCT and stereo-photographs using Pearson correlation and Bland and Altman plot. Although the correlation between the two methods was high (r = 0.87, P<0.001), analysis of the Bland and Altman plot demonstrated that Stratus OCT CDR measurements were larger than stereoscopic measurements.

The current study is the first simultaneous comparison of horizontal and vertical CDR estimates obtained with the new versions of objective methods, including HRT II and the Stratus OCT, with subjective masked assessment. We found a substantial overall agreement between subjective and objective methods of the HCDR (ICC = 0.75) and VCDR (ICC = 0.77) estimations, a substantial to almost perfect agreement between various groups of methods (ICC ranged from 0.68 to 0.81), and an almost perfect overall agreement between the clinicians’ HCDR (ICC = 0.84) and VCDR (ICC = 0.85) (Table 4), which were higher compared with the majority of previously conducted studies (Table 5). Photo graders were all trained in the same fellowship and underwent similar photo-grading training, which accounted for a high agreement observed in out study. Similar to Medeiros and associates,¹⁴ we found that HCDR and lower VCDR OCT values tended to be larger than ONHP, whereas higher VCDR OCT values tended to be lower than ONHP. Additionally, our study demonstrated statistically significant difference between all mean HCDR and VCDR obtained with the three methods. The Stratus OCT provided the largest estimate of the HCDR (0.68 ± 0.14) and VCDR (0.62 ± 0.13) for both glaucomatous and normal eyes, followed by HRT II and ONHP (Table 2).

TABLE 5.

Comparison of Intraclass Correlation Coefficients and κ Among Observers of Stereoscopic Optic Nerve Head Photographs in Various Studies

Study	No. Graders	HCDR	VCDR
Current study (2004)	3	0.84*	0.85*
Tielsch et al2	2	0.71†	0.74†
Zangwilll et al	3	0.38†	0.65†
Hatch et al11	3	0.74*	0.83*
Correnti et al12	2	0.67*	0.66*

^*Intraclass correlation coefficients.

^†κ.

HCDR indicates horizontal cup-to-disc ratio; VCDR, vertical cup-to-disc ratio.

HRT II and Stratus OCT assess optic disc morphology differently; therefore, CDR measurements obtained with these techniques may differ. HRT produces indirect measurements of retinal structures based on topographic information and the reference plane, which is set 50mm below and parallel to the peripapillary temporal retinal surface and serves as a cutoff to separate the cup from the neuroretinal rim. The HRT II software calculates the CDR by dividing cup area by the global disc area. Thus, the HRT II also includes vascular area within the rim area, which may affect topographic information.²⁸ Further, with the HRT II, the VCDR is measured at the center of the disc, rather than the center of the cup, which may result in its smaller estimate, because it is more proximal to the nasal rim, which is usually thicker than temporal, and includes more of the retinal vasculature. By contrast, Stratus OCT calculates the CDR as a ratio of the line across the cup to the line across the disc.^5,9,10 Despite the differences in the techniques, the measurements obtained with these techniques are well correlated, as demonstrated by the current study and other various investigators.^5,15,29 In addition, previous studies demonstrated that both HRT II and Stratus OCT provide reproducible measurements of the optic nerve head parameters.^8,9

In conclusion, whereas CDR estimates differ between techniques, this study demonstrated substantial agreement between subjective and objective methods of CDR estimation, which was similar to the level of interobserver agreement in the estimation of the CDR. Thus, the use of quantitative instruments may provide an equivalent and more reproducible method^8,9 to estimate CDR. Additional studies are needed to evaluate the sources of variability, their level of significance, and longitudinal agreement between various methods of CDR estimation, as these techniques continue to evolve.