Agreement and Reproducibility of Retinal Pigment Epithelial Detachment Volumetric Measurements Through Optical Coherence Tomography

Abstract

Purpose

To assess the agreement and reproducibility of retinal pigment epithelial detachment (RPED) volumetric measurements using a commercially available optical coherence tomography software available for the Zeiss Cirrus HD-OCT.

Methods

Twelve eyes of 10 patients with a diagnosis of neovascular age-related macular degeneration with RPED, seen at the New England Eye Center between October 2012 and December 2012, were enrolled in the study. Three separate scans per affected eye were obtained using the “Macular Cube 512 × 128” protocol. “Retinal pigment epithelial (RPE) elevation analysis” software was used to measure RPED volumes in the central 3-mm and 5-mm circles by calculating the volume between the “RPE fit” and “true RPE” lines. All 128 raster scans for each eye were exported into the AMIRA software for manual segmentation of RPED volumes in the central 3-mm and 5-mm circles. Interscan reproducibility and manual-to-automated agreement were assessed by intraclass correlation coefficient. Incidence of automated segmentation line error for both RPE fit and true RPE lines in the central 1 mm region was calculated.

Results

Average RPED volumes through automated segmentation software were 0.14 mm³ and 0.21 mm³ in the central 3-mm and 5-mm circles, respectively. Manual segmentation yielded average RPED volumes of 0.50 mm³ in the 3-mm circles and 0.92 mm³ in the 5-mm circles. Manual segmentation yielded significantly greater RPED volumes compared with automated measurements (P < 0.05). Intraclass correlation coefficients across the 3 automated measurements were 0.954 and 0.983 for volume in the 3-mm and 5-mm circles, respectively. Intraclass correlation coefficients between the manual and automatic volumes were 0.296 and 0.337 for the 3-mm and 5-mm circles, respectively. In the central 1 mm region, 11 of the 12 scans had breakdown in RPE fit line, whereas 8 of the 12 scans showed true RPE line breakdown.

Conclusion

Automated “RPED elevation” software demonstrated high interscan reproducibility. However, it showed low agreement with manual measurements from high rates of segmentation line breakdown, especially at the level of the RPE fit line (91.7%). Manual measurements resulted in greater volumes compared with automated measurements.

Optical coherence tomography (OCT) is a technology that is designed to perform noninvasive structural imaging of the eye.¹ Optical coherence tomography has revolutionized the way that ophthalmologists diagnose and monitor many eye diseases. Currently, it is used for imaging in disorders of the retina,^{2, 3} glaucoma,^{4, 5} and anterior segment.^6–8 Specifically, in the area of retinal disorders, OCT has become instrumental in the evaluation of patients with conditions such as age-related macular degeneration.^9–11 This can be performed on a qualitative basis, for example, to visualize the presence or absence of intraretinal, subretinal, and/or sub-retinal pigment epithelial (RPE) fluid and to analyze the type of pathology present.¹² Additionally, OCT can assess fluid in a quantitative way, for example, to measure micron scale changes in macular thickness in response to treatment.¹³

The advent of spectral domain OCT enables high speed, high resolution of the retina, RPE, and choroid and allows for the generation of high-density and detailed raster scans for total retinal thickness. However, accurate quantitative volumetric measurements of subretinal pathology are difficult, given the difficulty in accurately and reproducibly segmenting the superior and inferior borders of deeper lesions. Thus, total macular thickness has been used as a proxy to measure the degree of macular pathology and their changes over time, including retinal fluid, fibrosis, and neovascular membranes. This method is not exact because it can miss significant degrees of subretinal pathology, including retinal pigment epithelial detachments (RPEDs), as the outer retinal boundary is set at the location of the “true RPE,” not where the RPE is ideally at the location of the “RPE fit” line (Figure 1B, pink line). Several groups, including our own, have attempted to accurately measure the volume of RPEDs using a manual or semimanual technique and third-party software.^{13, 14} Although these measurements are considered accurate and reproducible, this is also a very time-consuming process, taking more than 30 minutes to segment a complete volumetric set of data consisting of more than 100 raster scans, thereby rendering it impractical for use in a busy clinical setting for routine patient monitoring.

Fig. 1.

Automated RPED measurement. A. Elevation map showing the region of RPED in the fundus enface image. B. Placement of automated segmentation lines in the OCT image (blue: true RPE and pink: RPE fit). C. Print out of automated analysis, displaying volumes measured in the central 3 mm and 5 mm regions.

Recently, advancements in segmentation technology have made segmentation of subretinal pathologies possible. In 2009, Ahlers et al¹⁴ used proprietary investigational software and demonstrated the ability to segment RPED volume from exported OCT scans. However, this software was not a commercially available tool. Several commercially available software to accomplish RPED volume measurements are available now. One example is the Cirrus HD-OCT (Carl Zeiss Meditec, Dublin, CA) “RPE elevation analysis” software. This software generates the area and volume that the RPE has been elevated (true RPE) over the ideal location of the RPE (RPE fit line). Nittala et al¹⁵ demonstrated a high level of agreement measuring drusen volume using this software compared with manual measurements. However, the utility of this software in pathologies such as serous or fibrovascular RPED is less well studied. Thus, the purpose of this study is to assess the agreement and reproducibility of volumetric RPED measurements using this commercially available technique in the setting of patients with age-related macular degeneration.

Methods

Subjects and Optical Coherence Tomography Scan Protocol

We performed a prospective study of a cohort of patients with history of neovascular age-related macular degeneration and clinical evidence of RPED, examined at the New England Eye Center, Tufts Medical Center between October and December 2012. Patients' medical records were reviewed to exclude patients with other retinal or choroidal pathologies besides neovascular age-related macular degeneration. This study was approved by the Tufts Medical Center Institutional Review Board and was conducted in accordance with the ethical standards stated in the 1964 Declaration of Helsinki.

All patients were imaged using the macular cube protocol on the Cirrus HD-OCT, which was composed of one hundred and twenty-eight 5-mm raster scans, each consisting of 512 B-scans. The study eye selected was the one with the presence RPED. Both eyes were included into the study if there was bilateral serous or fibrovascular RPED verified on fundus examination and fluorescein angiography. Images were taken in the standard fashion, centered at the fovea, with the choroid farther away from the zero-delay line. Patients were imaged 3 times using the same protocol, with 5 minutes between each image acquisition. Patients were instructed to blink at their normal rate and relax in between scans. Only scans with signal strengths ≥6 were included in the study.

Automated and Manual RPED Volumetric Measurements

The scan with the highest signal strength of the three was selected per study eye for automated volumetric analysis. If multiple scans had equal signal strength, the one taken the earliest was chosen. “RPE elevation analysis software” is available on the Cirrus HD-OCT software (version 6.0), which calculates the volume of subretinal pathology by automatically creating segmentation lines in the retina at the level of the true RPE (the level where the RPE actually is) and the RPE fit (where the RPE should ideally be in its natural position), then measuring the area in each B-scan over the central 3-mm and 5-mm circles from the fovea to obtain the RPED volume (Figure 1).

The same scans used for automated volumetric analysis were exported from the Cirrus HD-OCT and imported into the AMIRA software (Visualization Sciences Group SAS, Burlington, MA) for manual segmentation. Every 5 scans within the central 5 mm region were manually segmented at the true RPE and the RPE fit in the location of the RPEDs using a tablet personal computer, and then automatic interpolation was conducted in between the scans. Every scan was then analyzed to eliminate incidence of software breakdown, and the segmentation lines in the interpolated scans were adjusted as necessary. Then, volumes of the central 3-mm and 5-mm circles were generated using the AMIRA software (Figure 2).

Fig. 2.

Manual RPED measurement. A. Three-dimensional reconstruction of the macular region to be manually segmented. B. Cut-away 3-dimentional view of the macula with B-scan highlighting the area of RPED segmentation in blue. C. Three dimensional rendering of RPED volume manually segmented. D. Three dimensional rendering of the RPED volume overlaid onto the macular region.

Paired T-tests were conducted between the manual and automatic volumetric measurements in both the central 3-mm and 5-mm circles. Agreement between manual and automated RPED volumes in the central 3-mm and 5-mm circles was assessed through intra-class correlation coefficients (ICCs).

Reproducibility of Automated RPED Volumetric Measurements

The reproducibility of RPED volumetric measurements was calculated through ICC between the three sets of measurements for each study eye. All statistical analyses were conducted through Microsoft Excel (version 12.1; Microsoft Corporation, Redmond, WA) and SPSS Statistics (version 19; IBM Corporation, Chicago, IL).

Error Analysis

Error analysis was performed by examining the incidence of automated segmentation line breakdown for both the RPE fit and the true RPE lines in the central 1 mm region of the macula, defined as visible misidentification of the retinal boundary by >1 μm (as defined by a previous study from our group).¹⁶

Results

Of the 10 patients recruited, 2 were men and 8 were women. Eight right eyes and 4 left eyes were used. The average age of the patients was 80 ± 9 years. Nine patients were white and 1 patient was Asian.

Automated Versus Manual RPED Volumetric Measurements

Using the automated RPE elevation software, average RPED volume was 0.14 ± 0.07 mm³ in the central 3-mm circle and 0.21 ± 0.16 mm³ in the 5-mm circle (Table 1). Manual segmentation yielded an average RPED volume of 0.50 ± 0.32 mm³ in the central 3-mm circle and 0.92 ± 0.91 mm³ in the 5-mm circle. Retinal pigment epithelial detachment volume was significantly greater by manual measurements compared with automated measurements in both the central 3-mm (P < 0.01) and 5-mm circles (P < 0.05).

Table 1. Summary of Automated Versus Manual Measurements.

Measurement	Area, mm	Automated	Manual	P
Volume	3	0.14 ± 0.07 mm3	0.50 ± 0.32 mm3	<0.01
	5	0.21 ± 0.16 mm3	0.92 ± 0.91 mm3	<0.05
ICC	3	0.954	NA	<0.05
	5	0.983	NA	<0.05
ICC	3	0.296		NS
	5	0.337		NS
RPE fit	1	91.7%		NA
True RPE	1	66.7%		NA

NA, not applicable; NS, not significant.

Intraclass correlation coefficients were obtained to assess the agreement between the two methods. Intraclass correlation coefficients for manual versus automated RPED volumes were 0.296 in the central 3-mm circle and 0.337 3-mm in the central 5-mm circle.

Reproducibility of Automated RPED Volumetric Measurements

Reproducibility was assessed by analyzing the ICCs from automated RPED volumetric measurements across the three sets of “macular cube” performed for each study eye. The ICCs were 0.954 and 0.983 for the 3 measurements in the central 3-mm and 5-mm circles, respectively.

Error Analysis

Analyzing the central 1 mm region of automated segmentation (central 26 B-scans) for incidence of segmentation breakdown, 11 of the 12 scans (91.7%) demonstrated RPE fit line breakdown, whereas 8 of the 12 scans (66.7%) showed true RPE line breakdown (Figure 3).

Fig. 3.

Placement of automated segmentation lines. A. Ideal location of the RPE fit line in dashed red line and true RPE line in blue. B. Actual placement of RPE fit line in pink.

Discussion

This study evaluated the reproducibility of a commercially available proprietary automated RPE analysis software to calculate RPED volumes and to test its agreement with manual measurements. Chiu et al¹⁷ previously demonstrated good reproducibility of drusen measurements using a prototype OCT and segmentation software. Nittala et al¹⁵ also demonstrated the reproducibility of volumetric drusen measurements using the Cirrus HD-OCT and the RPE analysis software. This latter study also found that the RPE analysis software allowed for the reproducible calculation of drusen volumes, with high ICCs between manual and automated measurements in both the 3-mm and the 5-mm circles. More recently, Penha et al¹⁸ demonstrated high reproducibility of RPED measurements, with ICC > 0.99 using a combination of Cirrus HD-OCT software for RPE measurement and a proprietary algorithm developed by one of the authors. Our study also found high interscan reproducibility in the setting of RPEDs, where ICC was 0.954 for the volume in the 3-mm circle and 0.983 in the 5-mm circle across the repeated measurements.

Agreement between manual and automated measurements, however, was poor in this study. Intraclass correlation coefficients between automated and manual measurements were 0.296 in the 3-mm circle and 0.337 in the 5-mm circle. This low agreement was from a high incidence of segmentation line breakdown, particularly of the RPE fit line. In this study, 66.7% and 91.7% of the eyes demonstrated breakdown in the true RPE line and RPE fit line, respectively. This was in contrast to previous findings of “excellent agreement” in drusen volume between manual and automated measurements by Nittala et al and “fair agreement” between manual and automated drusen area by Yehoshua et al.^{15, 19}

Penha et al compared two commercially available SD-OCT devices—Cirrus HD-OCT and Spectralis SD-OCT—in the detection of retinal boundaries at the internal limiting membrane and true RPE in patients with RPEDs.²⁰ They found that central retinal thicknesses were significantly greater for the Spectralis than the Cirrus. Additionally, although Cirrus SD-OCT identified correct segmentation of boundaries in all eyes at the internal limiting membrane and true RPE, Spectralis yielded unreliable segmentation results with breakdown seen in the vast majority (92.6%) of scanned eyes. Our study, in contrast, examined the detection of RPED volume (rather than retinal thickness), with segmentation lines placed at the RPE fit and true RPE lines. This study found a relatively lower rate of breakdown at the true RPE line compared with the RPE fit line. Thus, it was not surprising that rates in segmentation error were lower in the study of Penha et al, where retinal thickness (between internal limiting membrane and true RPE) were measured rather than RPED volume.

We found a high level of segmentation line error in the Cirrus HD-OCT imaging algorithm, particularly at the RPE fit line in the setting of RPED. This was most likely the explanation for the low level of agreement observed between manual and automated RPED measurements. This high incidence of RPE fit line breakdown was also shown in previous studies for drusen. Nittala et al¹⁵ noted that nearly all discrepancies between manual and automated measurements came from the RPE fit line. However, in the study of Nittala et al, the errors did not make a statistical difference, probably given the relative small size of drusen. In this study, errors in the RPE fit line did lead to a significant change in volumes, likely given the larger size of RPEDs. Additionally, RPEDs tend to be less homogenous than drusen, with multiple hyperreflective and hyporeflective interfaces, which may lead to errors in segmentation. However, although drusen are small, they are also typically more numerous in comparison with RPEDs. In some instances, total drusen volume may be equal or larger than that of RPEDs. Thus, the small sample size in this study may contribute to the differences between the automated and manual RPED measurements. Improvements in automated segmentation algorithm, especially in the RPE fit line, will be needed for more accurate volumetric RPED analysis. Clinicians must take time to carefully assess scans for the presence of segmentation line errors before extrapolating the data for clinical use.

The automated volumes for the central 3-mm and the 5-mm circles were both significantly smaller than their respective manual measurements. We suspect that the smaller sizes may be from the systemic underestimation of the RPE fit line. The current Cirrus software generates RPE fit lines consistently above the level of the Bruch membrane (Figure 3). This is likely secondary to the underlying algorithm used to generate this boundary.

This study used a relatively small sample size, but the differences between the manual and automated segmentation were apparent from the incidence of segmentation line breakdown in the RPE fit line and the significant differences between manual and automated measurements. This study analyzed RPED measurements in the central 3-mm and 5-mm circles as per the Cirrus algorithm. Thus, RPEDs outside this region were missed. Future updates to the software allowing for measurement of the full macular region may be helpful. Additionally, although it may be interesting to examine potential gender or racial differences in RPED volume measurements, these subgroup analyses are limited given the small sample size.

Given the high rates of segmentation software breakdown, especially at the RPE fit line, further development of the software module to better accommodate the RPED retinal morphology, especially at the level of the RPE fit line, is needed to ensure accurate quantitation of RPEDs. This study illustrated that improvements to software algorithm will still need to be made to ensure accurate volumetric measurements of subretinal pathologies. These adjustments should be made before its use clinically.