Reproducibility and Agreement Between 2 Spectral Domain Optical Coherence Tomography Devices for Anterior Chamber Angle Measurements

Purpose:

To compare anterior chamber angle parameters based on the location of Schwalbe line (SL) from 2 spectral domain optical coherence tomography (SD-OCT) instruments and to measure their reproducibility.

Methods:

Forty-two eyes from 21 normal, healthy participants underwent imaging of the inferior irido-corneal angle with the Spectralis and Cirrus SD-OCT under tightly controlled low-light conditions. SL-angle opening distance (SL-AOD) and SL-trabecular iris space area (SL-TISA) were measured by masked, certified graders at the Doheny Imaging Reading Center using customized grading software. Interinstrument and intrainstrument, as well as interobserver and intraobserver reproducibility of SL-AOD and SL-TISA measurements were evaluated by intraclass correlation coefficients (ICCs) and Bland-Altman plots with limits of agreement (LoA).

Results:

The mean SL-AOD was 0.662±0.191 mm in Spectralis and 0.677±0.213 mm in Cirrus. The mean SL-TISA was 0.250±0.073 mm² in Spectralis and 0.256±0.082 mm² in Cirrus. The agreement for intrainstrument (ICCs>0.979), intragrader (ICCs>0.992), and intergrader (ICCs>0.929) was excellent. Excellent agreement between the 2 devices was also documented with a mean difference of −0.016 (LoA −0.125 to 0.092) mm for SL-AOD and −0.007 (LoA −0.056 to 0.043) mm² in SL-TISA.

Conclusions:

Both SD-OCTs provided comparable measurements and permitted calculation of SL-based angle metrics. There was excellent interinstrument and intrainstrument and intraobserver and interobserver reproducibility for Spectralis and Cirrus SD-OCTs, suggesting true interchangeability between SD-OCT devices. This has the potential to lead to development of standardized grading assessments and quantification of angle parameters that would be valid across various SD-OCT devices.

Anterior segment optical coherence tomography (AS-OCT) was introduced to provide real-time, noncontact, high-resolution, cross-sectional imaging of the anterior segment of the eye.1 The information gained with imaging instruments provides clinicians with qualitative and quantitative measures of anatomic relationships in the anterior segment. Imaging of the anterior chamber angle (ACA) and measurement of the anatomic landmarks has been suggested to be useful in glaucoma risk assessment, diagnosis, and therapeutic decisions.2–4 As a result, reliable quantitative measurements of ACA have become increasingly important in glaucoma evaluation.2–7

ACA imaging technology has evolved from time domain (TD) OCT devices to spectral domain (SD) systems. The latter make use of Fourier transformation to gather depth data from the spectra of the OCT signal and thus eliminate the need for axial translation of the reference mirror. TD-OCT systems often scan at 1310 nm allowing for greater scan depth, but Cirrus (Carl Zeiss Meditec Inc., Dublin, CA) scans at the 840 nm wavelength and Spectralis (Heidelberg Engineering, Heidelberg, Germany) scans at 870 nm. These SD-OCT systems allow for higher resolution, speed, and sensitivity at the cost of decreased depth of visualization of the inner angle recess.8 As a result, scleral spur-based metrics, which are commonly used in TD-OCT scan analysis are often difficult to compute with SD-OCT. Overall, SD-OCT’s increase in resolution and speed minimizes eye motion artifacts and allows better visualization of Schwalbe line (SL), which can be used to measure the angle-opening distance (AOD), trabecular iris space area (TISA) and subsequent filtration area.9

A reliable measurement of the angle requires reproducible results with each instrument. In addition, agreement between different devices is a prerequisite to directly compare measurements from these instruments. Thus, the objectives of this study were to evaluate the intrainstrument, intragrader, and intergrader reproducibility of ACA measurements with the Cirrus and Spectralis SD-OCT. Furthermore, the scope of this study was to establish the agreement in quantitative measurements of the ACA between the 2 devices.

MATERIALS AND METHODS

Acquisition of Image Data

Twenty-one healthy volunteers were recruited for participation in this study. Participants had no ocular or systemic history and no evidence of ocular disease. None of the participants was using any topical or systemic medication. The presence of bilateral normal eyes with open angles was confirmed by prior ophthalmoscopic examination. The study was conducted in accordance with the ethical standards in the Declaration of Helsinki and was approved by the University of Southern California Institutional Review Board. Written informed consent was obtained from all participants.

All participants underwent nonmydriatic AS-OCT imaging of both eyes using the Cirrus and Spectralis OCT. Scans were obtained in a darkened room, with lighting standardized to 1 cd/m² at the imaging plane, confirmed with a light meter with a reading of ≤0.2 foot candles at the eye (Light Meter FC-840021; Sper Scientific, Scottsdale, AZ). For the Cirrus, 3 mm scan length, 5-line anterior segment raster scans were captured, with a 0.25 mm distance between lines. Scans were performed perpendicularly on the inferior angle at the 6 o’clock position. For Spectralis, a single-line 270-degree scan was obtained. The OCT B-scan images were then exported for subsequent masked grading at the Reading Center.

Measurement of SL-AOD and SL-TISA

Images were graded in a masked fashion by certified AS-OCT graders (X.P., J.M.), using a prespecified standardized grading protocol. In the Cirrus and Spectralis images, the inferior angle SL-AOD and SL-TISA were measured with validated custom grading software (Image J 1.44p; Wayne Rasbands, National Institutes of Health) at the Doheny Image Reading Center. For all eyes, the grader computed 2 parameters: SL-AOD and SL-TISA, using the 6 o’clock line scan of the 5-raster set in the Cirrus (the other 4-line scans were used to refine the position of SL) and single-line scan in the Spectralis. Because adequate quality images were confirmed by the operator at the time of acquisition with visualization of the angle and SL, no case was deemed to be ungradable. The measurements from the first and second acquisition (intrainstrument), from first and second grading of the same image by 1 observer (intraobserver) and from grading of the same images by different observers (interobserver) for both instruments were used for statistical analysis. Images from 21 participants (42 eyes) were used to assess intrainstrument and interinstrument reproducibility. Intraobserver and interobserver reproducibility were evaluated in a subset of 15 patients (30 eyes).

Statistical Analysis

The mean±SD was estimated for each parameter. Intraclass correlation coefficients (ICC) were computed to evaluate absolute agreement and Bland-Altman plots were generated with limits of agreement (LoA), accounting for inclusion of both eyes from each participant in the analysis. Linear regression models were used to establish a correction factor for comparisons between the 2 SD-OCT instruments. All statistical analyses were performed using commercial software (SPSS version 18.0; SPSS Inc., Armonk, NY).

RESULTS

Patients Characteristics

Nine men and 12 women were included in the study. The mean age of the participants was 35±9 years. Thirteen of them were Asian, 3 were Indian, 1 was Hispanic, and 4 of them were White.

Intrainstrument Reproducibility (Variability Between Consecutive Acquisitions)

The mean SL-AOD for the first acquisition on the Cirrus was 0.674±0.213 mm and the second acquisition was 0.681±0.215 mm. The mean difference in SL-AOD between the 2 acquisitions was −0.008 (LoA −0.096 to 0.081) mm (Table 1). The mean SL-AOD for the first acquisition and the second acquisition on the Spectralis was 0.664±0.198 and 0.660±0.185 mm, respectively, with a mean difference of 0.004 (LoA −0.068 to 0.076) mm. The reproducibility was excellent for both the Cirrus and Spectralis for SL-AOD, with an ICC of 0.989 and 0.991, respectively.

TABLE 1.

Intrainstrument Reproducibility With the Cirrus SD-OCT and the Spectralis SD-OCT

The mean SL-TISA on Cirrus on the first acquisition and the second acquisition was 0.255±0.081 and 0.258±0.085 mm², respectively (Table 1). For Spectralis, the mean SL-TISA on the first acquisition and the second acquisition was 0.250±0.076 and 0.250±0.069 mm², respectively. The LoA in SL-TISA were 0.036 to 0.030 mm² on Cirrus and −0.041 to 0.041 mm² on Spectralis between the 2 acquisitions. The ICC was 0.989 for Cirrus and 0.979 for Spectralis.

Variability in Image Analysis; Intraobserver and Interobserver Reproducibility

For SL-AOD, the mean difference between the first and second grading was −0.005 (−0.039 to 0.029) mm with Cirrus and −0.005 (−0.048 to 0.037) mm with Spectralis. For SL-TISA, the mean difference was −0.001 (−0.025 to 0.022) mm for Cirrus and −0.005 (−0.034 to 0.024) mm for Spectralis. The intragrader reproducibility was excellent for both devices when examining SL-AOD and SL-TISA, with ICC values >0.992 (Table 2).

TABLE 2.

Intraobserver Reproducibility With the Cirrus SD-OCT and the Spectralis SD-OCT

High values for ICC were also observed in the interobserver comparisons. The mean difference for SL-TISA was −0.010 (−0.082 to 0.063) and 0.032 (−0.039 to 0.104) mm² for Cirrus and Spectralis, respectively. The ICC values were over 0.929 for all between-graders comparisons (Table 3).

TABLE 3.

Interobserver Reproducibility With the Cirrus SD-OCT and the Spectralis SD-OCT

Interinstrument Reproducibility

The mean SL-AOD measurements were 0.662±0.156 mm for Spectralis and 0.677±0.213 mm for Cirrus (Table 4). This corresponded to a mean difference of −0.016 (LoA −0.125 to 0.092) (Fig. 1A). The SL-TISA differences were also small between the instruments, with a mean difference of −0.007 (LoA −0.056 to 0.043) mm² (Fig. 1B). The ICCs for comparisons between the 2 instruments were 0.961 for SL-AOD and 0.945 for SL-TISA.

TABLE 4.

Interinstrument Reproducibility for SL-AOD and SL-TISA Measurements Acquired With the Spectralis and the Cirrus Instruments in the Same Subjects

FIGURE 1.

A and B, Bland-Altman plots demonstrate interinstrument reproducibility for SL-AOD and SL-TISA. The horizontal lines indicate the mean difference (solid line) and the limits of agreement (dotted lines). AOD indicates angle opening distance; SL, Schwalbe line; TISA, trabecular iris space area.

Correction Factor/Linear Regression Predictions

Based on measurements from 21 OD eyes with both Cirrus and Spectralis, a linear regression line of fit of y=1.1175x–0.054 represents the relationship between the instruments. This line of fit was used to predict the Cirrus measurement values in the 21 OS eyes based on inputting Spectralis measurements into the regression line equation. The average measured SL-AOD with the Cirrus was 0.662±0.208 mm, whereas the predicted value from the Spectralis was 0.677±0.217 mm. The predicted and the measured value had an absolute difference of 0.043±0.035 mm (%difference 6.427±5.427%). The results show that the average measured SL-TISA with Cirrus was 0.250±0.082 mm², whereas the predicted value from Spectralis was 0.254±0.085 mm². The predicted value and measured value had an absolute difference of 0.024±0.018 mm² (0.001 to 0.074 mm², %difference 10.5±8.88%).

DISCUSSION

To the best of our knowledge, this is the first study to compare the ACA measurements between 2 SD-OCT instruments; Cirrus and Spectralis. Measurements were performed in the same eyes in sequential fashion using the same strict lighting conditions. The intrainstrument, intraobserver, and interobserver measurement reproducibility for both OCT machines was also assessed. Importantly, the present study provides direct evidence of consistency of quantitative metrics from both devices, since the effect of consecutive image acquisitions, between and within the observer factors did not produce significant measurement variability. These results may have important implications in clinical practice. First, the excellent reproducibility in the intrainstrument, intraobserver, and interobserver measurements provides confidence in using those measurements for clinical use for screening, management or to follow patients in a longitudinal fashion. Second, the excellent comparability of the results between the 2 SD-OCT devices may allow development of cross-platform standardized measurements that could be correlated clinically to define narrow versus open angles. In this study, no significant differences were found in the quantitative ACA values derived from the 2 SD-OCTs, thus making them practically interchangeable.

The reproducibility of the newer Fourier domain OCT systems has only been assessed in a few studies in the literature. Qin et al10 investigated the interobserver and intraobserver reproducibility of the SL-AOD and SL-TISA metrics for images acquired with the RTVue SD-OCT. Eyes with narrow angles were included in that study and the visibility of SL was as high as 97% to 100%. The authors also reported a high reproducibility with that device. In a previous study, we also reported excellent reproducibility values for Cirrus SL-based quantitative angle metrics in open angle eyes.11 In addition, the swept-source OCT, with a light source of 1310 nm, was also found to exhibit high reproducibility in angle measurements, with ICC values for interobserver and intraobserver reproducibility higher than 0.83.12 Noteworthy, the reproducibility values were found to be better for experts, compared with nonexperts, since the measurement process remains subjective and involves identification of landmarks in the ACA.13

The reliability metrics, including the high ICC achieved in our study are better, compared with previously published reproducibility results using the Visante AS-OCT and the slit-lamp OCT.14–18 Using TD-OCT, the scleral spur has been an important anatomic landmark to define quantitative angle parameters used to assess angle opening.5 Based on initial studies, these instruments have been shown to be variable in respects to repeatability and reproducibility in scleral spur-based AOD₅₀₀, AOD₇₅₀, TISA₅₀₀, and TISA₇₅₀ measurements.16,19 Kim et al17 evaluated the interobserver reproducibility of AOD₅₀₀, AOD₇₅₀, TISA₅₀₀, and TISA₇₅₀ and reported low intersession and interoperator reproducibility with ICCs between 0.662 and 0.892 using the Visante TD-OCT. On the other hand, Tan et al16 obtained ICCs with the Visante between 0.978 and 0.988. Using the slit lamp-OCT, Muller et al18 reported that the interobserver ICC for the ACA was 0.96.

One reason for the differences in reproducibility in these studies could be based on the lack of visualization in some images and variable placement of the scleral spur.17,20 This necessity introduces an important human factor in the analysis of a TD-OCT image, possibly generating non-negligible intraobserver and interobserver variance. Studies investigating the visibility of the scleral spur with AS-OCT showed visualization between 70% and 78.9%.20,21 In SD-OCT, location of SL allows calculation of novel angle metrics based on location of SL and the end of the trabecular meshwork. Despite the fact that the analysis of AS-OCT images is not automated and still requires subjective identification of SL from the observer, our results did indicate that this variability does not introduce a high degree of error.

Disparities between reproducibility levels in the various studies on ACA metrics could also result from differences in the study populations, acquisition protocols, quadrants, and the light levels used for assessment. Pupil size variation is an important factor to obtain reproducible angle measurements.22 Because of the importance of standardizing the lighting conditions for repeatable angle measurements, in the present study, the imaging was performed in tightly controlled lighting conditions that were tested with a light meter. A darkened room with a light meter reading below ≤0.2 foot candles at the eye allows for optimal measuring conditions based on pupil dilation and its effect on angle opening. Studies have also shown significant regional differences in different quadrants.12 Since the objective of this study was to examine the variability of the SD-OCT in measuring the ACA, we focused only on the inferior angle for both instruments.

The present study provides evidence of an excellent agreement between Cirrus and Spectralis angle measurements. This also shows that the instruments are interchangeable and a correction factor is not necessary. Recent studies that included narrow angle patients had a different study design and have shown only moderate agreement between SD-OCT instruments in terms of detecting angle closure. Quek et al23 compared the iVue and Cirrus SD-OCT devices in a cohort with a high percentage of narrow angles based on gonioscopy. They reported only moderate agreement between the 2 instruments and low interobserver reproducibility values for both devices in detecting angle closure. Owing to the apparent differences in study design, sample characteristics and instruments used, a direct comparison with the present study where quantitative measurements of the angle were assessed in open angle eyes cannot be done. In another study comparing Cirrus, Visante, and gonioscopy, the authors found only moderate agreement in detecting angle closure between the 2 OCTs and slight-to-fair agreement with gonioscopy.24 Finally, in another study assessing the performance of the RTVue SD-OCT in the diagnosis of angle closure, the authors reported a decreased ability of this device to image the various ACA landmarks, including the SL and the scleral spur, hence impeding the ability to diagnose a closed angle.25 It is thus unclear whether the performance of the 2 instruments investigated in the present study will be different when imaging eyes with angle closure. Future studies may need to evaluate the performance of both OCTs in those patients. Furthermore, the high level of reproducibility reported in this study is often achievable by using experienced, certified anterior segment reading center graders. It is uncertain whether similar results could be achieved with less intensively trained individuals.

Another possible limitation of our study was the relatively small sample size. However, since the results were strong and the SDs were small, a larger population size may not be necessary for this type of study.

Our study also has several strengths including the use of standardized acquisition and grading protocols. Importantly, our study was the first, to our knowledge, to compare SL-based metrics from different anterior segment SD-OCT instruments.

Consistency in measurements and interchangeability across various SD-OCT platforms from different manufacturers is important to define standardized criteria for screening for narrow angles and assessment of longitudinal change in angle parameters. Exploration and validation of SD-OCT devices for imaging anterior segment is an essential first step in achieving this goal for standardization.