Effect of Signal Strength and Improper Alignment on the Variability of Stratus Optical Coherence Tomography Retinal Nerve Fiber Layer Thickness Measurements

Abstract

PURPOSE

To evaluate the effect of signal strength and improper scan alignment on retinal nerve fiber layer (RNFL) thickness measurement variability.

DESIGN

Retrospective, longitudinal clinical study.

METHODS

All eyes of healthy subjects with at least 2 fast RNFL scan sessions were selected from the Diagnostic Innovations in Glaucoma Study. The chronological first scan was considered to be the baseline. Absolute differences in signal strength and RNFL thickness measurements between baseline and subsequent scans were calculated. Regression analysis was conducted to assess whether signal strength and scan shifts along the horizontal (nasal–temporal) but not the vertical (superior–inferior) axis affect average RNFL thickness measurements.

RESULTS

Ninety-four eyes of 94 subjects were included. All eyes were tested twice or more on the same visit, whereas 30 eyes were followed up longitudinally for 32.4 ± 13.3 months (1 scan per annual follow-up). For quadrants, absolute differences from baseline were greater than for average RNFL thickness and were significantly larger for scans acquired on separate visits. Average RNFL thickness increased only when the difference between the nasal and temporal quadrants increased (R² = 0.16; P < .0001), suggesting it may be affected by horizontal but not vertical scan shifts. Differences in signal strength were associated with differences in average RNFL thickness (R² = 0.19; P < .0001).

CONCLUSIONS

Even under optimal testing conditions, scan quality can adversely effect the ability to detect change over time. Therefore, caution is warranted when detecting glaucomatous progression using scan series of different quality. Careful overall assessment of quadrants and average RNFL thickness measurements is suggested to help identify scan misalignment.

CROSS-SECTIONAL STUDIES HAVE SHOWN THAT Stratus optical coherence tomography [OCT] (Carl Zeiss Meditec, Dublin, California, USA) is capable of detecting glaucomatous structural damage.^1–13 Also, the reproducibility of the Stratus OCT measurements has been evaluated in normal and glaucomatous eyes.^14–17 However, evidence that OCT is capable of detecting disease progression over time is lacking.^18,19 A few studies have evaluated the effect of factors such as pupil size or signal strength on the variability of Stratus OCT measurements.^20,21 However, variability in the scan position over time also may be one possible component affecting intervisit reproducibility, but this has not been evaluated longitudinally.^22,23

Stratus OCT lacks a scan registration feature to ensure that the scan and subsequent measurements are obtained at the same location during follow-up examinations. Rather, the operator consistently must position the scan during each follow-up visit with the help of a landmark feature. In addition, the fundus image is not recorded during actual OCT scan acquisition, but rather only after the OCT scan has been obtained; this implies that, if eye movements have occurred, the scan visible in the fundus image may not correspond to the actual location of the acquired scan. In a recent study, it was demonstrated that peripapillary scan misplacement produces significant changes in retinal nerve fiber layer (RNFL) assessment characterized by an increase in measured RNFL thickness in the quadrant in which the scan is closer to the optic disc and by a significant decrease in RNFL thickness in the quadrant in which the scan is displaced further from the disc.²³ Moreover, it was found that average RNFL thickness is greater when scans are displaced temporally along the horizontal axis.²³ The purpose of this study was to evaluate the effect of signal strength and scan misalignment on the long-term and short-term variability of Stratus OCT-RNFL thickness measurements.

METHODS

ALL HEALTHY SUBJECTS WITH AT LEAST 2 OR MORE STRAtus OCT scans were selected from the Diagnostic Innovations in Glaucoma Study database. All participants in this retrospective clinical study were evaluated at the Hamilton Glaucoma Center, University of California, San Diego.

All subjects underwent a complete ophthalmic examination including review of medical history, slit-lamp biomicroscopy, intraocular pressure (IOP) measurement, dilated stereoscopic fundus examination using a 78-diopter (D) lens, stereoscopic optic disc photography, and standard automated perimetry using the 24-2 Swedish interactive threshold algorithm (Humphrey Visual Field Analyzer; Carl Zeiss Meditec) as part of the Diagnostic Innovations in Glaucoma Study, a prospective, longitudinal study designed to evaluate optic nerve structure and visual function in glaucoma. All eyes had best-corrected visual acuity of 20/40 or better, sphere within ± 5.0 D, cylinder within ± 3.0 D, normal fundus examination results with a healthy appearance of the optic disc and RNFL, open-angle at gonioscopy with IOPs of 22 mm Hg or less, no history of increased IOP, and normal visual field (VF) results. A normal VF was defined as a mean deviation and a pattern standard deviation within the 95% normal confidence limits and glaucoma hemifield test results within normal limits.

INSTRUMENTATION

The Stratus OCT obtains cross-sectional images of ocular microstructures.²⁴ For this study, the fast RNFL thickness protocol (Stratus OCT software version 4.0.7; Carl Zeiss Meditec) was used. All recruited subjects had pupils dilated at the time of imaging, and OCT images of adequate quality (see following section) were obtained.

For the fast RNFL, a total of 3 scans, composed of 256 A scans each, are acquired consecutively using a circle with a standardized diameter of 3.4 mm. An automated computer algorithm delineates the anterior and posterior margins of the nerve fiber layer (NFL). The NFL thickness parameters are measured by the Stratus OCT by assessing a total of 768 data points between the anterior and posterior NFL borders.

All scans included in this study had signal strength of at least 7. In addition, criteria for determining scan quality as reported in previous studies were followed¹⁴: the fundus image was clear enough for the optic disc and the scan circle to be seen before and during imaging, color saturation was even and dense throughout all retinal layers with red color visible in the retinal pigment epithelium, and RNFL was visible with no missing or blank areas within the scan pattern (ie, no algorithm failure). Internal fixation was used at all times.

Scans were performed by experienced operators (technicians), all University of California, San Diego Imaging Data Evaluation and Analysis center-certified for taking OCT images. All scans were centered on the optic disc by the operator with the help of a landmark feature (ie, placing a landmark on a branching vessel or at the disc margin to be used as reference for follow-up scans), as suggested by the manufacturers. The operator assessed each scan for quality and verified that the optic disc was adequately centered or focused within the circular scan. Scans were taken multiple times by the same operator at baseline visit within a 15-minute period. In addition, eyes that also were followed longitudinally had 1 scan obtained per each year of follow-up during office hours. Average and quadrant (superior, inferior, nasal, and temporal) RNFL thickness measurements were recorded for analysis.

STATISTICAL ANALYSIS

The first scan in time was considered to be the baseline scan and was used for comparison with subsequent scan measurements. The absolute difference in signal strength and quadrant and average RNFL thickness measurements between baseline scans and subsequent scans was calculated separately for eyes tested on the same visit and eyes with longitudinal follow-up tested on separate visits. Results from the 2 groups were compared using the Mann–Whitney U test.

A previous study²³ showed that misalignment of the scan circle along the horizontal axis (nasal minus temporal) but not along the vertical axis (superior minus inferior) can lead to erroneous average RNFL thickness estimates. In particular, nasal scan shifts from the baseline scan position may result in a decrease in average RNFL thickness measurements, falsely indicating the presence of disease progression. In addition, it was demonstrated that scan shifts along both the horizontal and the vertical axes result in an increase in RNFL thickness in 1 quadrant and a decrease by a similar magnitude in the opposite quadrant.²³ Therefore, to determine whether horizontal scan shifts but not vertical scan shifts can influence the average RNFL thickness measurements, the difference in RNFL thickness between opposite sectors (nasal minus temporal for horizontal scan shifts and superior minus inferior for vertical scan shifts) between baseline and subsequent scans was calculated and plotted against corresponding differences in average RNFL thickness in linear regression analysis. Also, linear and multiple regression analyses were applied to evaluate the effect of signal strength alone and of both signal strength and the presence of scan shifts on the average RNFL thickness measurements. All analyses were performed using JMP software version 6.1 (SAS Institute, Cary, North Carolina, USA) and SPSS software version 15.0 (SPSS Inc, Chicago, Illinois, USA). A P value less than .05 was considered to be statistically significant.

RESULTS

NINETY-FOUR EYES OF 94 HEALTHY SUBJECTS (MEAN AGE ± standard deviation [SD], 59.3 ± 13.7 years; 52 females) were evaluated, for a total of 254 OCT scans. The demographic of the selected individuals along with baseline average and quadrant RNFL thickness measurements is shown in Table 1. All eyes had scans obtained by the same operator at baseline visit (median, 2 scans; range, 2 to 4 scans). In addition, a subset of 30 eyes was followed up longitudinally for a mean ± SD 32.4 ± 13.3 months, with 1 scan obtained per each year of follow-up (median, 2 scans; range, 2 to 4 scans).

TABLE 1.

Demographic Data, Signal Strengths, and Retinal Nerve Fiber Layer Thickness Measurements (in μm) at Baseline

	No. of Subjects (eyes) Evaluated (n = 94)	No. of Subjects (eyes) with Longitudinal Follow-up (n = 30)	P value
Gender (% of females)a	55.3	60	.6
Mean age (95% CI), yearsb	59.3 (56.5 to 62.1)	60.6 (55.9 to 65.4)	.62
Mean no. of scans (range)	2 (2 to 4)	2 (2 to 4)	—
Follow-up (months)	—	32.4 (27.4 to 37.3)	—
Mean baseline MD (95% CI)b	0.12 (−0.2 to 0.3)	0.16 (−0.16 to 0.34)	.84
Mean baseline PSD (95% CI)b	0.92 (0.74 to 1.15)	0.81 (0.68 to 1.11)	.75
Mean RNFL thickness (95% CI)b	99.4 (97.2 to 101.6)	98.9 (95.1 to 102.6)	.83
Mean superior RNFL thickness (95% CI)b	121.7 (118.1 to 125.3)	123.1 (116.7 to 129.5)	.69
Mean inferior RNFL thickness (95% CI)b	126.7 (122.9 to 130.4)	127.3 (120.8 to 133.9)	.87
Mean nasal RNFL thickness (95% CI)b	77.0 (73.5 to 80.4)	75.8 (69.9 to 81.7)	.8
Mean temporal RNFL thickness (95% CI)b	72.2 (68.8 to 75.6)	69.3 (63.6 to 74.9)	.34
Mean signal strength (95% CI)c	9.2 (9.0 to 9.4)	9.5 (9.2 to 9.8)	.2

CI = confidence interval; MD = mean deviation; PSD = pattern standard deviation; RNFL = retinal nerve fiber layer.

^aChi-square test.

^bt test.

^cWilcoxon test.

The absolute differences in average and quadrant RNFL thickness and signal strength measurements are shown in Table 2 for scans obtained on the same visit and scans obtained on separate visits. For average RNFL thickness, although the absolute differences were higher for separate visits than for the same visit, this was not statistically significant (3.7 vs 2.9 μm, respectively; P = .1). However, a statistically significant difference in signal strength was found between scans obtained on the same visit and scans obtained on separate visits. A statistically significant difference in RNFL thickness also was found for all quadrants. The absolute differences in RNFL thickness measurements were greater for scans obtained on separate visits than for those obtained on the same day (for example, for the inferior quadrant the absolute difference was 10.5 μm on separate visits vs 6.2 μm on the same visit; P = .001).

TABLE 2.

Absolute Differences in Signal Strength and Retinal Nerve Fiber Layer Thickness Measurements (in μm) between Baseline Scan and Each Follow-up Scan (mean with 95% confidence interval)

	Same Visit (n = 94)	Separate Visits (n = 30)	p value
Mean RNFL thickness (95% CI)	2.9 (2.4 to 3.5)	3.7 (3.0 to 4.5)	.1
Mean superior RNFL thickness (95% CI)a	7.8 (6.3 to 9.3)	10.4 (8.4 to 12.3)	.04
Mean inferior RNFL thickness (95% CI)a	6.2 (4.9 to 7.5)	10.5 (8.8 to 12.3)	.001
Mean nasal RNFL thickness (95% CI)a	7.7 (6.4 to 9.0)	10.0 (8.2 to 11.7)	.03
Mean temporal RNFL thickness (95% CI)a	5.3 (4.2 to 6.4)	7.8 (6.3 to 9.2)	.003
Mean signal strength (95% CI)a	0.6 (0.4 to 0.7)	0.8 (0.6 to 1)	.003

CI = confidence interval; RNFL = retinal nerve fiber layer.

^aP < .05, Mann-Whitney U test.

Differences in signal strength from baseline were significantly associated with differences in average RNFL thickness from baseline for scans obtained on the same visit and scans obtained on separate visits combined (R² = 0.19; F = 36.5; P < .0001), suggesting the presence of a positive linear relationship between signal strength and average RNFL thickness (ie, if the signal strength increases from baseline scan, the average RNFL thickness increases and vice versa). A similar significant association was found for scans obtained on the same visit and scans obtained on separate visits examined separately (Table 3).

TABLE 3.

Results from Linear Regression Analysis of Scans Obtained on the Same Visit and of Scans Obtained at Separate Visits

	Same Visit		Separate Visits
	β	SE β	β	SE β
Difference in signal strengtha	1.98	0.39	2.88	0.80
Difference in nasal minus temporal RNFL thicknessb	0.09	0.02	0.15	0.03
Difference in superior minus inferior RNFL thickness	0.02	0.02	−0.05	0.04

β = unstandardized β coefficient; RNFL = retinal nerve fiber layer; SE β = standard error β.

^aSame visit: R² = 0.21; F = 26.4; P < .05; separate visits: R² = 0.22; F = 12.3; P < .05.

^bSame visit: R² = 0.11; F = 11.9; P < .05; separate visits: R² = 0.27; F = 16.6; P < .05.

Linear regression analysis also showed that horizontal scan shifts, but not vertical scan shifts, were associated significantly with the variability in average RNFL thickness (R² = 0.16; F = 29; P < .0001 vs R² < 0.0001; F = 0.008; P = .93, respectively; Figures 1 and 2). In particular, for all scans combined, average RNFL thickness decreased with nasal shifts and increased with temporal shifts (Figure 1), according to the following equation:

$$\text{Avg. RNFL thickness}=0.011+0.102\left(\text{nasal minus temporal RNFL thickness}\right).$$

FIGURE 1.

Scatterplot showing the significant positive association between differences in average retinal nerve fiber layer (RNFL) thickness measurements from baseline and differences in nasal minus temporal quadrants.

FIGURE 2.

Scatterplot showing no association between differences in average RNFL thickness measurements from baseline and differences in superior minus inferior quadrants.

When the 2 groups (scans obtained on the same visit and scans obtained on separate visits) were examined separately, the effect was somewhat more pronounced for scans obtained at separate visits than for scans obtained on the same visit (R² = 0.27 vs R² = 0.11, respectively), as shown in Table 3.

These results were confirmed in multiple regression analysis. Based on previous studies, a known predictor of average RNFL thickness variability such as the signal strength was entered first, followed by the difference between nasal and temporal quadrants and the difference between superior and inferior quadrants. Results from the multiple regression analysis showed that signal strength was the strongest predictor, followed by the difference between nasal and temporal quadrants (for the overall model, R² = 0.27; F = 14.4; P < .05; Table 4).

TABLE 4.

Results from Multiple Regression Analysis of all Scans Combined

	β	SE β	P value
Difference in signal strength	1.55	0.33	< .0001
Difference in nasal minus temporal RNFL thickness	0.08	0.02	< .0001
Difference in superior minus inferior RNFL thickness	0.00	0.02	.62
Visit (same vs separate)	−0.2	0.3	.53

β = unstandardized β coefficient; RNFL = retinal nerve fiber layer; SE β = standard error β.

DISCUSSION

IN THE CURRENT STUDY, RESULTS SHOWED THAT AVERAGE RNFL thickness generally was robust, with small differences between scans obtained on the same visit and between scans obtained at separate visits. However, greater changes occurred to quadrant RNFL thickness, particularly for scans obtained at separate visits. In addition, differences in signal strength from baseline and differences between nasal and temporal quadrants (indicating the occurrence of horizontal scan shifts from baseline) were associated with greater variability in average RNFL thickness. Specifically, a positive association between signal strength and RNFL thickness was found.

Previous studies have shown that Stratus OCT measurements generally are more reproducible when scans are obtained within the same session rather than on separate visits.^15,16 In this study, greater changes in signal strength from baseline occurred when eyes were tested over time on separate visits. In addition, reproducibility studies have reported that quadrants and clock hours always perform worse than the average RNFL thickness.^14–16 Not surprisingly, similar results were found in this study, with the superior and the nasal quadrants reporting on average a greater absolute difference in RNFL thickness from baseline average RNFL thickness. Variability in the location of the scan around the optic disc may explain these findings in part, because it may be easier for the operator to center the scan consistently in a similar position while obtaining scans in sequence during the same visit.

A few studies also have suggested that differences in signal strength between scans can account for some of the variability found in RNFL thickness measurements obtained by OCT.^21,25 Wu and associates showed that the impact of low signal strength scores can be relevant and that larger signal strength changes (> 1 unit) may lead to a greater RNFL thickness difference from baseline.²¹ However, the above-mentioned study failed to demonstrate a significant correlation between signal strength and change in RNFL thickness in scans that met or exceeded the manufacturers' signal strength recommendations. In clinical practice, it may not always be possible to obtain high-quality scans on all patients. However in most cases, the manufacturers' recommendations are at least met or exceeded. In addition, to establish appropriate cut-offs for which a decrease in average RNFL thickness can be attributed to disease progression and not to signal strength variability, it is important to determine the effect of signal strength and other variables on OCT measurements when such criteria are followed. Budenz and associates suggested that an 8-μm decrease in thickness may be accepted as within normal limits of test–retest variability with 95% tolerance.¹⁶

Our study demonstrates that a significant association exists between differences in signal strength and the variability of average RNFL thickness measurements even in the presence of higher signal strengths (> 7 units). Based on our model, for each unit of decrease in signal strength compared with that of the baseline scan, the average RNFL thickness also decreases by approximately 2 μm, suggesting that this effect alone may be responsible for the test–retest variability reported by some studies in a population of healthy subjects.

However, one of the major limitations of the Stratus OCT that has not been studied extensively is the lack of a scan registration feature to ensure that the scan and subsequent measurements are obtained at the same location during follow-up examinations. Nor is there an automated feature that ensures consistent placement of the scan circle around the optic disc. Rather, the operator attempts to center the circle scan consistently on the optic disc with the assistance of marking a branch vessel using the landmark feature. Clinicians reviewing OCT scans also are challenged by the fact that the fundus image appearing on the printout and showing the location of the circle scan is obtained after the scan already has been acquired. In a recent study, it was shown that scan misalignment can produce significant changes in sectoral RNFL thickness.²³ It was also shown that, possibly because of the presence of the papillomacular bundle on the horizontal meridian, horizontal scan shifts can produce significant changes in the average RNFL thickness.²³ This study confirms the previous finding, suggesting that when the temporal quadrant RNFL thickness is increased and the nasal quadrant is decreased compared with the baseline scan (ie, the scan may have shifted nasally at the follow-up scan), the average RNFL thickness is significantly decreased. Without careful overall assessment of quadrant RNFL thickness, in glaucoma a nasal scan shift at follow-up with decreased average RNFL thickness could be mistaken for disease progression. These results also show that a change in average RNFL thickness, although significant, is small and may become relevant only for differences between nasal and temporal quadrants of more than 20 μm, which would correspond to a more than 2-μm change in average RNFL thickness. However, it should be mentioned that all scans included in the study were reviewed for quality by experienced operators who followed the manufacturers' suggested method (the landmark feature) to center the scans. With less skilled operators, in a busy clinical setting and without proper methods to center the scans, more variability because of improper scan alignment should be expected.

In this study, only healthy eyes were included in the analysis. However, because studies have shown that measurement variability may be higher in glaucomatous eyes, similar or even greater effects because of signal strength changes and improper alignment are expected. In addition, because of the lack of a gold standard to establish disease progression and to identify truly stable glaucomatous eyes, healthy eyes followed up over time may be more suitable to study the effect of scan misalignment. Second, this study examined the effect of horizontal and vertical scan shifts only. Shifts in other directions, as well as the influence of other factors other than the signal strength, may be responsible for some of the variability in average RNFL thickness measurements not explained by the model presented in this study.

In conclusion, these results show that in healthy eyes, when manufacturers' suggestions are followed, signal strength and horizontal scan shifts still may have a significant, although weak, effect on the average RNFL thickness measurements. Even when obtaining high-quality scans, it is suggested that signal strength always be recorded and used when interpreting Stratus OCT RNFL thickness results. Based on this study and the suggested cut-offs for no change,¹⁶ an eye with a decrease in RNFL thickness resulting from a decrease in signal strength of 4 units or more from baseline (from 10 to 6, for example) could be considered mistakenly as a progressing eye. To minimize the effect of horizontal scan shifts on the average RNFL thickness measurements, proper methods to center the scan (such as the use of the landmark feature) always should be followed. Particularly when average RNFL thickness decreases over time, careful overall assessment of average and quadrants RNFL thickness is important to help clinicians rule out that the decrease in RNFL thickness can be explained by a difference in signal strength or scan misalignment.

Biosketch

Gianmarco Vizzeri, MD, is a Glaucoma Specialist and a Fellow at the Hamilton Glaucoma Center, University of California San Diego. Dr Vizzeri's research interests include imaging of the optic disc and retinal nerve fiber layer for glaucoma diagnosis and for detecting glaucomatous progression, and other diagnostic testing in glaucoma.