Agreement between spectral-domain and time-domain OCT for measuring RNFL thickness

Abstract

Background/aims

To evaluate spectral-domain (SD) optical coherence tomography (OCT) reproducibility and assess the agreement between SD-OCT and Time-Domain (TD) OCT retinal nerve fibre layer (RNFL) measurements.

Methods

Three Cirrus-SD-OCT scans and one Stratus-TD-OCT scan were obtained from Diagnostic Innovations in Glaucoma Study (DIGS) healthy participants and glaucoma patients on the same day. Repeatability was evaluated using Sw (within-subject standard deviation), CV (coefficient of variation) and ICC (intraclass correlation coefficient). Agreement was assessed using correlation and Bland–Altman plots.

Results

16 healthy participants (32 eyes) and 39 patients (78 eyes) were included. SD-OCT reproducibility was excellent in both groups. The CV and ICC for Average RNFL thickness were 1.5% and 0.96, respectively, in healthy eyes and 1.6% and 0.98, respectively, in patient eyes. Correlations between RNFL parameters were strong, particularly for average RNFL thickness (R² = 0.92 in patient eyes). Bland–Altman plots showed good agreement between instruments, with better agreement for average RNFL thickness than for sectoral RNFL parameters (for example, at 90 μm average RNFL thickness, 95% limits of agreement were −13.1 to 0.9 for healthy eyes and −16.2 to −0.3 μm for patient eyes).

Conclusions

SD-OCT measurements were highly repeatable in healthy and patient eyes. Although the agreement between instruments was good, TD-OCT provided thicker RNFL measurements than SD-OCT. Measurements with these instruments should not be considered interchangeable.

Spectral domain (SD) optical coherence tomography (OCT) technology provides better scan resolution and a greater number of scans acquired at a faster rate than time domain (TD) OCT technology.^{1 2} These improvements have the potential to provide clinicians with enhanced tools for the diagnosis and follow-up of glaucoma. However, before a new diagnostic instrument can be introduced for use in clinical practice, good repeatability of measurements must be demonstrated in healthy and glaucomatous eyes. In addition, it is important to determine if measurements from previous generation OCT technologies are compatible with new generation technologies, both designed to identify the retinal nerve fibre layer (RNFL) and quantify its thickness.^3–5 This study was designed to evaluate the reproducibility of SD-OCT measurements and to measure the agreement between SD-OCT and TD-OCT RNFL measurements in healthy and patient eyes.

MATERIALS AND METHODS

For this prospective longitudinal study, participants were enrolled from the UCSD Diagnostic Innovations in Glaucoma Study (DIGS), an ongoing study designed to evaluate optic nerve structure and visual function in glaucoma. Informed consent was obtained from each participant, and the University of California San Diego Institutional Review Board approved all methodology. Study methods adhered to the provisions of the Declaration of Helsinki guidelines for research involving human participants.

To be included in the study, the eyes of all participants required good-quality stereo-photography (TRC-SS, Topcon Instruments, Paramus, New Jersey) of the optic disc and reliable (false positives, fixation losses and false negatives ≤25% with no observable testing artefacts) standard automated perimetry (SAP, Humphrey Field Analyzer II with Swedish Interactive Thresholding Algorithm, Carl Zeiss Meditec, Dublin, California) testing within 6 months of testing using Stratus TD-OCT and Cirrus SD-OCT protocols (see below).

In addition to the testing described above, each study participant underwent a comprehensive ophthalmological evaluation including a review of medical history, best-corrected visual acuity testing, slit-lamp biomicroscopy, intraocular pressure (IOP) measurement with Goldmann applanation tonometry, gonioscopy and dilated slit-lamp fundus examination with a 78 D lens. To be included in the study, participants had to have a best-corrected acuity better than or equal to 20/40, spherical refraction within ±5.0 D and cylinder correction within ±3.0 D, and open angles on gonioscopy. Eyes with coexisting retinal disease, uveitis or non-glaucomatous optic neuropathy were excluded.

Because this study was designed to evaluate measurement repeatability in a typical clinical patient population to which the instrument should apply, study participants were classified as either healthy controls or patients with glaucomatous or suspicious appearing optic discs by stereo-photography and/or repeatable abnormal SAP results and/or history of elevated IOP in at least one eye prior to study entry. Each stereophotograph was graded as glaucomatous or normal by two experienced observers based on the presence or absence of neuroretinal rim thinning, RNFL thinning (focal or diffuse), or excavation and/or undermining of the cup characteristic of glaucoma. In cases of disagreement between observers, a third observer adjudicated the decision. Abnormal SAP results required pattern standard deviation (PSD) with p≤5% and/or Glaucoma Hemifield Test outside normal limits by STATPAC analysis on two consecutive tests. History of elevated IOP was defined as two or more measurements ≥23 mm Hg using Goldmann applanation tonometry. Healthy controls were normal in both eyes for the above tests.

INSTRUMENTATION

The Cirrus SD-OCT and the Stratus TD-OCT are non-invasive imaging devices that obtain cross-sectional images of ocular microstructures.⁶

Cirrus SD-OCT (software version 3.0; Carl Zeiss Meditec, Dublin, California) utilises spectral domain optical coherence tomography technology to acquire OCT data with better resolution (5 μm compared with approximately 10 μm axial resolution in tissue) about 70 times faster (27 000 vs 400 A-scans/s) than time-domain OCT technology.¹ For the Cirrus SD-OCT, the Optic Disk Cube 200×200 protocol was used for acquisition and analysis. This protocol generates a cube of data through a 6 mm square grid by acquiring a series of 200 horizontal scan lines each composed of 200 A-scans. For analysis, Cirrus algorithms identify the centre of the optic disc and automatically place a calculation circle of 3.46 mm diameter around it. The anterior and posterior margins of the RNFL are delineated, and after extracting from the data cube 256 A-scan samples along the path of the calculation circle, the system calculates the RNFL thickness at each point on the circle. To investigate the repeatability of RNFL thickness measurements, three consecutive Optic Disk Cube scans were acquired and analysed for each eye.

For the Stratus TD-OCT, the Fast RNFL thickness acquisition protocol (software version 4.0.7; Carl Zeiss Meditec) was used. Three scans were automatically acquired consecutively using a circle with a standardised diameter of 3.4 mm. An automated computer algorithm delineates the anterior and posterior margins of the RNFL.⁷

Both the Cirrus SD-OCT and the Stratus TD-OCT provide average RNFL thickness measurements overall and by quadrants and clock hours on a similar printout (see fig 1 for a healthy eye and a glaucomatous eye (patient)). For the purpose of this study, average RNFL thickness and quadrant and clock hour thicknesses were exported from the Stratus TD-OCT and the Cirrus SD-OCT.

Figure 1.

Examples showing the similarities between Cirrus spectral domain (SD) optical coherence tomography (OCT) and time-domain OCT printouts in one eye of a healthy participant and a patient. RNFL, retinal nerve fibre layer.

All measurements in this study were performed on the same day within a 2 h time period with Stratus TD-OCT taken first, without pupil dilation and by the same operator (technician), experienced with taking OCT images and with centring the scans on the optic disc using currently suggested techniques for the Stratus TD-OCT (ie, with the help of a landmark feature).⁸ Internal fixation was used for all scans. For both instruments, scans with signal strength <6 were excluded, as per manufacturers' instructions. In addition, criteria for determining scan quality as reported in previous studies were followed:⁷ the optic disc and scan circle were visible on the fundus image, colour saturation was even and dense throughout the entire scan with red colour visible in the retinal pigment epithelium, and RNFL was visible with no missing or blank areas within the scan pattern.

STATISTICAL ANALYSIS

Cirrus SD-OCT repeatability was assessed using the within-subject standard deviation (Sw), coefficient of variation (CV) and intraclass correlation coefficient (ICC). Because the Stratus TD-OCT Fast RNFL acquisition protocol consists of three single circle scans acquired consecutively and subsequently averaged, the average of the measurements taken from the three Cirrus SD-OCT scans was used for comparison with the Stratus TD-OCT measurements. Agreement between Cirrus SD-OCT and Stratus TD-OCT measurements was evaluated using correlations (R²) and Bland–Altman plots. The method of generalised estimating equations (GEE) was applied to adjust for the correlation between the two eyes of each subject.⁹

For the Bland–Altman plots, the reported 95% limits of agreement were calculated based on a regression approach for non-uniform differences.¹⁰

In addition, because the standard deviations of the measurements from the two devices differed, modified Bland–Altman plots for the average RNFL thickness were displayed by plotting the difference versus the weighted average of the two measurements.¹¹

Analyses were performed using JMP version 6.1 software (SAS Institute, Cary, North Carolina) and SPSS version 15.0 software. A p value less than 0.05 was considered statistically significant.

RESULTS

All available DIGS participants who consented to the study and were imaged with SD-OCT and TD-OCT were selected for the analysis. Sixteen healthy participants (32 eyes) and 39 patients (78 eyes) were included. All subjects had at least three good-quality scans from SD-OCT and one from TD-OCT. Four eyes of two patients were excluded for signal strength <6 (two eyes using SD-OCT, four eyes using TD-OCT). Demographic and ocular characteristics of healthy and patient eyes/participants are shown in table 1.

Table 1.

Demographic data from 110 eyes of 55 individuals included in the analysis

	Healthy participants (n = 16, 32 eyes)	Patients (n = 39, 78 eyes)	p(t)
Female (%)	10 (62.5)	21 (53.8)	0.49 (χ2)
Mean age (years) (SD)	65.4 (9.56)	66.9 (10.2)	0.41
Mean deviation (dB) (SD)	0.07 (0.75)	−1.87 (2.8)	<0.001
Pattern standard deviation (dB) (SD)	1.52 (0.22)	3.71 (3.23)	<0.001
Intraocular pressure (mm Hg) (SD)	15.1 (2.16)	15.9 (5.32)	0.56
Disc area (mm2)* (SD)	1.64 (0.36)	1.91 (0.51)	<0.05

^*Measured using the Heidelberg Retina Tomograph (Heidelberg Engineering, Heidelberg, Germany).

Statistically significant differences in SAP MD and SAP PSD at the time of testing were observed between study groups. Table 2 shows the mean RNFL thickness, Sw, CV and ICC for average RNFL, quadrant and clock hour thicknesses calculated from the three Optic Disk Cube 200×200 scans in the two groups.

Table 2.

Spectral-domain (SD) optical coherence tomography retinal nerve fibre layer (RNFL) thickness reproducibility in healthy participants and patients

	Healthy participants				Patients
	Mean RNFL thickness (μm) (95% Cl)	Sw (1.96 SE)	CV%	ICC (95% Cl)	Mean RNFL thickness (μm) (95% Cl)	Sw (1.96 SE)	CV%	ICC (95% Cl)
Temporal	65.8 (60.7 to 70.9)	1.5 (0.33)	2.3	0.98 (0.97 to 0.99)	58.3 (55.8 to 60.8)	1.6 (0.51)	2.7	0.94 (0.91 to 0.96)
Superior	110.9 (106.1 to 115.6)	2.8 (0.71)	2.6	0.93 (0.89 to 0.96)	93.1 (89.1 to 97)	2.2 (0.33)	2.5	0.98 (0.97 to 0.99)
Nasal	70.2 (66.2 to 74.3)	2.9 (0.79)	4.4	0.90 (0.83 to 0.94)	66.1 (63.8 to 68.4)	2.4 (0.4)	3.7	0.92 (0.89 to 0.95)
Inferior	112.4 (107 to 117.7)	2.6 (0.69)	2.3	0.95 (0.92 to 0.98)	96.4 (91.8 to 101.1)	2.8 (0.58)	3.1	0.97 (0.95 to 0.98)
Average	89.8 (86.6 to 93)	1.3 (0.38)	1.5	0.96 (0.94 to 0.98)	78.4 (76 to 81)	1.2 (0.19)	1.6	0.98 (0.97 to 0.99)
1-o'clock	98.8 (93 to 104.6)	4.9 (1.97)	5	0.81 (0.69 to 0.89)	84.1 (80.2 to 88.1)	3.6 (0.51)	4.3	0.94 (0.92 to 0.96)
2-o'clock	82.7 (75.9 to 89.4)	5 (1.57)	6.6	0.88 (0.80 to 0.94)	78.6 (74.6 to 82.6)	3.5 (0.6)	4.8	0.94 (0.92 to 0.96)
3-o'clock	61.2 (57.6 to 64.8)	2.4 (0.76)	4.2	0.90 (0.83 to 0.94)	59.4 (57 to 61.9)	3 (0.49)	5.1	0.89 (0.85 to 0.93)
4-o'clock	66.8 (61.6 to 72)	3.3 (0.61)	4.9	0.94 (0.89 to 0.97)	60.2 (57.8 to 62.6)	3 (0.52)	5	0.88 (0.83 to 0.92)
5-o'clock	92.4 (83.8 to 100.9)	4 (0.81)	4.1	0.97 (0.94 to 0.98)	82.4 (78.5 to 86.3)	3.9 (0.73)	4.9	0.92 (0.88 to 0.94)
6-o'clock	121.5 (114.8 to 128.2)	4.3 (1.34)	3.5	0.91 (0.84 to 0.95)	103.3 (97.4 to 109)	4.9 (1.06)	5.2	0.94 (0.91 to 0.96)
7-o'clock	123.2 (112.2 to 134.3)	4.4 (0.94)	3.6	0.97 (0.95 to 0.99)	103.5 (96 to 111.1)	4.1 (0.85)	4.5	0.97 (0.96 to 0.98)
8-o'clock	70.3 (62.4 to 78.3)	2.7 (0.78)	3.6	0.97 (0.95 to 0.99)	58.1 (55.3 to 60.9)	2.4 (0.65)	4	0.91 (0.88 to 0.94)
9-o'clock	50.6 (47.3 to 54)	1.1 (0.24)	2.3	0.98 (0.96 to 0.99)	48.1 (45.6 to 50.5)	1.6 (0.94)	2.9	0.84 (0.78 to 0.89)
10-o'clock	76.4 (70.1 to 82.6)	2.3 (0.76)	3.1	0.97 (0.94 to 0.98)	68.6 (64.9 to 72.3)	2.2 (0.47)	3.3	0.97 (0.95 to 0.98)
11-o'clock	120.6 (111.8 to 129.4)	4.3 (0.85)	3.7	0.96 (0.93 to 0.98)	104.2 (99 to 109.4)	4.4 (0.78)	4.6	0.94 (0.92 to 0.96)
12-o'clock	113.2 (105.6 to 120.8)	4.4 (1.01)	3.9	0.94 (0.90 to 0.97)	90.7 (84.8 to 96.6)	4.1 (0.86)	5	0.95 (0.93 to 0.97)

CV, coefficient of variation; ICC, intraclass correlation coefficient; Sw, within-subjects SD.

In general, the reproducibility was excellent in both groups, especially for average RNFL thickness, with quadrants performing better than clock hours. The nasal quadrant provided a higher Sw and CV and lower ICC, suggesting that measurements were less repeatable for this sector (and corresponding clock hours) than for other sectors. For example, in patients, Sws were 2.4 and 2.2, CVs were 3.7 and 2.5, and ICCs were 0.92 and 0.98 for the nasal and superior quadrants, respectively.

Table 3 shows the 95% limits of agreement and R² values for quadrants and average RNFL thickness in the two groups.

Table 3.

Mean with 95% limits of agreement* (mean values shown are spectral-domain optical coherence tomography (SD-OCT) minus time-domain optical coherence tomography (TD-OCT)) and correlation (R²) between SD-OCT and TD-OCT measurements in healthy participants and patients

	Healthy participants		Patients
	Mean (μm) (95% limits of agreement)	R2	Mean (μm) (95% limits of agreement)	R2
TEMP RNFL	−6.3 (3.4 to 0.14×Temp RNFL thickness ± 18.0)	0.76*	−7.8 (3.3 to 0.18×Temp RNFL thickness ± 10.0)	0.85*
SUP RNFL	−9.6 (3.4 to 0.11×Sup RNFL thickness ± 24.9)	0.47*	−8.7 (14.6 to 0.24×Sup RNFL thickness ± 18.4)	0.75*
NAS RNFL	−7 (20.1 to 0.36×Nas RNFL thickness ± 20.0)	0.55*	−1.5 (34 to 0.52×Nas RNFL thickness ± 17.4)	0.54*
INF RNFL	−9 (9.2 to 0.15×Inf RNFL thickness ± 20.4)	0.69*	−4.5 (4.5 to 0.09×Inf RNFL thickness ± 14.5)	0.86*
AVG RNFL	−8 (11.0 to 0.19×Avg RNFL thickness ± 7.0)	0.87*	−5.7 (12.4 to 0.23×Avg RNFL thickness ± 7.9)	0.92*

^*95% limits of agreement are expresses as b_o+ b₁×value of retinal nerve fibre layer (RNFL) thickness ±1.96SD_residuals.

AVG RNFL, average RNFL thickness; INF RNFL, inferior quadrant RNFL thickness; NAS RNFL, nasal quadrant RNFL thickness; RNFL, retinal nerve fibre layer; SUP RNFL, superior quadrant RNFL thickness; TEMP RNFL, temporal quadrant RNFL thickness.

Figure 2 describes the correlation between average RNFL thickness measurements from SD-OCT and TD-OCT in one randomly selected eye (ie, left or right) of healthy participants and patients. In general, all RNFL thickness measurements were highly correlated in both groups, with the association being particularly strong for the average RNFL thickness (R² was 0.87 and 0.92 in healthy participants and patients, respectively).

Figure 2.

Scatter plot showing the correlation between spectral-domain optical coherence tomography (SD-OCT) and time-domain optical coherence tomography (TD-OCT) average retinal nerve fibre layer (RNFL) thickness in one randomly selected eye of healthy participants and patients.

The mean of the differences between SD-OCT and TD-OCT and 95% limits of agreement in table 3 show that TD-OCT consistently provided thicker RNFL measurements than SD-OCT for both global and sectoral parameters. Based on the 95% limits of agreement reported in table 3 for the parameter average RNFL thickness, if an arbitrary value of 90 μm is assumed (ie, “Avg RNFL thickness” in the formula is replaced with 90 μm), the calculated 95% limits of agreement are −13.1 to 0.9 and −16.2 to −0.3 μm in healthy and patient eyes, respectively.

Figure 3 shows the modified Bland–Altman plots for the agreement between SD-OCT and TD-OCT for the average RNFL thickness in healthy participants and patients. Based on the data available from this study, the coefficient used to obtain the weighted average for SD-OCT and TD-OCT was 0.86. The modified Bland–Altman plots show that TD-OCT provides thicker RNFL measurements than SD-OCT, particularly in eyes with thick RNFL measurements demonstrated by calculating the slopes of the average RNFL plots in fig 3 (r = −0.36; p<0.05 and r = –0.48; p<0.05 in healthy participants and patients, respectively). A similar effect was observed in the Bland–Altman plots for the four quadrants (data not shown).

Figure 3.

Modified Bland–Altman plots for agreement between spectral-domain optical coherence tomography (SD-OCT) and time-domain optical coherence tomography (TD-OCT) average retinal nerve fibre layer thickness in healthy participants (A) and patients (B). To obtain the modified Bland–Altman plots, the difference was plotted against the weighted average of SD-OCT and TD-OCT measurements. The regression line is shown along with the 95% limits of agreement in dotted lines displayed as a function of the weighted average.

DISCUSSION

Assessing an instrument's repeatability is an important first step in the evaluation of a new technology for use in clinical practice. Current results suggest that SD-OCT RNFL thickness measurements are highly repeatable and that reproducibility is excellent in both healthy participants and patients. In addition, strong correlations and a good agreement were found between RNFL thickness measurements performed using the SD-OCT and the TD-OCT, suggesting that the two instruments may provide similar estimates of RNFL thickness. This information is important because TD-OCT has already demonstrated the ability to detect glaucomatous structural damage.^12–19 These results show that this technology is promising for measuring RNFL thickness in healthy and glaucomatous eyes.

Several earlier studies have shown that TD-OCT measurements are reproducible, with RNFL thickness in glaucomatous eyes being slightly more variable than in normal eyes.^{8 20–22} However, results from the current study showed that SD-OCT measurements were similarly repeatable in both healthy and patient eyes. This is despite the fact that differences in disc area between healthy and patient eyes favoured a lower repeatability in patient eyes (due to measurements obtained closer to the disc margin in patient eyes that had larger optic discs on average).²² It is possible that SD-OCT provides more robust algorithms than TD-OCT for RNFL delineation, producing less variability in diseased eyes. Alternately, results may be due to differences in the study populations and the criteria applied for classifying healthy and glaucoma. In this study, most of the glaucomatous eyes had early to moderate disease based on the Glaucoma Staging System (GSS) classification for visual-field abnormalities;²³ of the 78 patient's eyes, 24 (30.8%) were classified as GSS Stage 0, 35 (44.9%) were classified as Stage 1, 17 (21.8%) were classified as Stage 2, and two (2.5%) were classified as Stage 3. It is possible that repeatability may be worse in eyes with more advanced glaucoma.

One of the advantages of the Cirrus SD-OCT is that it does not require manual centring of the scan on the optic disc as long as the peripapillary region is included in the Optic Disk Cube 200×200 scan. The scan registration process performed by the Cirrus algorithms is fully automated, reducing the likelihood of operator error. Previous studies with TD-OCT have shown that misalignment of the circle scan by the operator can significantly affect RNFL measurements, particularly in quadrants and clock hours.²⁴ However, in the absence of an eye-tracking system, Cirrus SD-OCT scan acquisition might still be affected by eye movements. In this study, Cirrus Optic Disk Cube 200×200 maps were checked for the presence of eye-movement artefacts such as vessel misalignment and/or image distortion. Nevertheless, as previously found using the TD-OCT, the nasal quadrant RNFL thickness was the most variable sector. As previously suggested for TD-OCT, it is possible that the angle of incidence of the illuminating beam is such that the RNFL is dimmer nasally, thus limiting the ability of the detection algorithm to consistently identify the RNFL at the same location over time.^{21 25}

Our results showed that SD-OCT and TD-OCT thickness measurements are highly correlated. As expected, the correlation was particularly strong for average RNFL thickness compared with quadrant thickness. These results were confirmed in multivariate analysis using GEE adjusting for the correlation between two eyes of the same subject (data not shown). Although a good agreement was found for all parameters, Bland–Altman plots showed that TD-OCT RNFL measurements were consistently thicker than SD-OCT measurements, particularly in eyes with thick RNFL measurements. The reason for this finding is unclear. Because signal strength has been shown to affect RNFL thickness measurements using TD-OCT,^26–31 it is possible that differences in signal strength between the two instruments may be one of the causes. However, in this study, only scans with adequate signal strength were included, and no differences in signal strength were observed between scans taken using the two devices (average signal strengths for SD-OCT and TD-OCT were 8.4 (1.9) and 8.7 (1.1), respectively; t test, p = 0.73). Nevertheless, in healthy participants, the average RNFL thickness measured with SD-OCT was not associated with signal strength (R² = 0.02, p = 0.11), while TD-OCT RNFL thickness showed a positive, although weak association with signal strength (ie, the greater the signal strength, the greater the thickness, R² = 0.11, p<0.05). Alternatively, the discrepancy in the measurements could be related to an intrinsic difference between the instruments and software edge-detection algorithms for measuring the RNFL, and the precise location of the RNFL measured. It is likely that SD-OCT, by providing scans at a higher resolution than TD-OCT, may also provide measurements that reflect a more accurate delineation of the RNFL margins. Future studies might clarify which instrument offers greater accuracy in estimating the “true” in vivo RNFL thickness.

This study evaluated the reproducibility of RNFL thickness measurements using the commercially available Cirrus SD-OCT in healthy eyes and glaucoma patient eyes. A previous report on SD-OCT measurement variability was obtained using a prototype device on a limited sample of healthy eyes.³²

In conclusion, this study showed that SD-OCT provides excellent reproducibility suggesting that this technology is promising for detecting small structural changes over time in most eyes. As RNFL measurements with TD-OCT are thicker than with SD-OCT, measurements with these instruments should not be considered interchangeable. Further studies with longer follow-up are needed in order to assess intervisit variability and to better evaluate variance components that could affect measurements repeatability.