Detection of Cystoid Macular Edema with Three-Dimensional Optical Coherence Tomography versus Fluorescein Angiography

This study was a comparison of the sensitivity and reproducibility of three-dimensional optical coherence tomography (3D-OCT) and fluorescein angiography (FA) for the detection of cystoid macular edema (CME).

Abstract

Purpose.

To compare the sensitivity and reproducibility of three-dimensional optical coherence tomography (3D-OCT) and fluorescein angiography (FA) for the detection of cystoid macular edema (CME).

Methods.

Data were retrospectively collected from all patients who underwent digital FA and 512 × 128 horizontal raster 3D-OCT scans on the same day in a retina subspecialty clinic. Images were reviewed independently by four reading center graders and adjudicated as a group to render a single result for each eye and each imaging modality. The κ statistic was used to determine the level of agreement between graders for each modality. The sensitivity of each imaging modality for CME detection was calculated by using the presence of CME on either modality as the ground truth; subgroup analysis was performed according to disease diagnosis and lens status.

Results.

Four hundred thirteen eyes of 207 patients were included in the analysis. Intergrader agreement was higher for 3D-OCT than for FA both before (κ_OCT = 0.61, κ_FA = 0.43) and after adjudication (κ_OCT = 0.74, κ_FA = 0.58).The sensitivity for detection of definite CME was higher for 3D-OCT (95%, 144/151 cases) than for FA (44%, 67/151 cases). Definite FA (+) 3D-OCT (−) CME was identified in 1 eye (0.2%), whereas definite FA (−) 3D-OCT (+) CME was identified in 40 eyes (10%). No significant associations between CME detection and lens examination or disease diagnosis were observed.

Conclusions.

In this study, 3D-OCT was more sensitive and had better intergrader agreement than did FA for the detection of CME.

Cystoid macular edema (CME) is a cause of severe visual loss that occurs in a variety of pathologic conditions, such as age-related macular degeneration (AMD),^1,2 diabetic retinopathy (DR),^2–4 branch or central retinal vein occlusion (BRVO/CRVO),² and epiretinal membrane or vitreomacular traction (ERM/VMT),^2,5 and as a complication of intraocular surgery.² In CME, two primary pathologic events occur: abnormal fluid accumulation and cystoid degeneration.⁶

On fluorescein angiography (FA), the appearance of CME is relatively well-defined as a petaloid, or honeycomb-like, pattern of hyperfluorescence as a result of dye pooling in the cystoid spaces.⁷ Although FA has been the traditional gold standard for the detection of CME, it is somewhat invasive, has potential adverse effects, and is largely a subjective tool with demonstrated interpretation variability in diseases such as age-related macular degeneration.^8,9 Moreover, several disease entities produce a slit lamp biomicroscopic picture typical of CME, yet FA reveals no abnormal vascular permeability or accumulation of dye.^10–12

Optical coherence tomography (OCT), first described by Huang et al.¹³ in 1991, is an imaging modality capable of providing high-resolution cross-sectional images of the neurosensory retina. As OCT is noninvasive, it was quickly adopted by clinicians for the assessment of patients with CME. In part because of the relative ease with which they can detect CME, many clinicians now prefer OCT over FA for its detection.¹⁴ It has been reported that certain conditions may demonstrate significant intraretinal cystoid spaces on OCT without leakage on FA.^5,14–18 Conversely, investigators have also described cases of CME that appear only on FA and not on OCT.^3,19 Most of these studies, however, were performed with conventional time-domain OCT, with at most six radially oriented, cross-sectional scans with limited axial resolution (∼8–10 μm). More recently, the advent of high-speed, spectral-domain OCT (3D-OCT) has allowed more complete coverage of the macular area via dense raster scanning and with higher axial resolution (∼6 μm) that may provide higher sensitivity for the detection of macular abnormalities.²⁰

In this study, we compared the sensitivity of FA and 3D-OCT in the detection of CME and determined the level of agreement of these assessments between certified graders at an image-reading center. We chose to focus only on CME for this study because of the relatively clear definitions of CME on FA and OCT. Therefore, we remind the reader that results from this study should not be extrapolated to comparisons of detection of noncystoid causes of leakage by FA with that of edema by OCT.

Methods

Data Collection

A retrospective database review was performed of all patients who underwent digital FA (model 50IX; Topcon Corp., Tokyo, Japan) and optical coherence tomography imaging in both eyes, on the same day, over a 25-month period at a satellite retina subspecialty clinic of the Doheny Eye Institute. Eyes were excluded if they did not undergo 3D-OCT imaging (3D-OCT-1000; Topcon Corp.) or if they had ungradable images from such causes as media opacities. Approval for data collection and analysis was obtained from the institutional review board of the University of Southern California. The research adhered to the tenets set forth in the Declaration of Helsinki.

Medical records of all eligible patients were reviewed to determine age, sex, best-corrected Snellen visual acuity (BCVA), lens examination, and disease diagnosis. FAs were performed with a digital retinal camera system (model 50IX; Topcon Corp.), with image resolution ranging from 1.3 to 2.7 megapixels in accordance with the standard FA protocol of the Doheny Eye Institute Ophthalmic Imaging Unit. In each case, one eye (as defined by the ordering physician) was defined as the transited eye, in which multiple stereoscopic frames of the macula (field 2) were obtained from the point of dye entry into the eye until the late venous phase (typically, 45 seconds after the start of fluorescein injection). Additional mid-phase (typically, between 60 and 90 seconds) and late-phase (∼5 minutes) images were captured in both eyes. Late-phase frames are recognized by some investigators to be most crucial for identification of CME on FA.²¹

Only patients scanned with 3D-OCT were included in the study. All 3D-OCT image sets were obtained by using a raster scan protocol of 128 × 512 A-scans on a prototype instrument. This system acquires 18,000 A-scans per second with an axial resolution of 6 μm. Using the 3D-OCT raster scan protocol, the complete data set is acquired in fewer than 3.7 seconds.

Raw FA and OCT data were exported from the imaging instruments for review at the reading center.

Grading Methodology

Four graders (YOY, PAK, SRS, ACW), certified for assessing CME using both FA and OCT images at the Doheny Image Reading Center (DIRC) evaluated each set of FA images for each eye independently. The grading protocol used for identification of CME from FA was the same as that of the Early Treatment Diabetic Retinopathy Study,²² in which CME was identified by the accumulation of dye in petaloid or honeycomb-like spaces within the retina in the late phases of the angiogram. Thus, cases with leakage in the late phases of an FA but without a petaloid or honeycomb-like appearance were considered to be non-CME in this study. Using these criteria, we assessed CME as present (Y), questionable (Q), absent (N), or cannot grade (CG) for each case.

3D-OCT scans were assessed with a previously described DIRC OCT grading tool (termed the OCTOR and publicly available at www.diesel.la/ provided in the public domain by the Doheny Eye Institute) that was specially adapted (as the 3D-OCTOR) for this study to load and display the dense 3D-OCT data sets (128 B-scans per case). We defined cystoid spaces on OCT B-scans as circular or ovoid intraretinal hyporeflective spaces present at the same approximate transverse location on two adjacent B-scans (Fig. 1). The requirement that cystoid spaces must exist on two adjacent B-scans was implemented since OCT speckle noise may produce hyporeflective pockets that mimic small cystoid spaces. These noise pockets, however, should not exist in adjacent B-scans that are separated by 47 μm spatially and several milliseconds of time. Therefore, any space that appeared cystoid and was present on two adjacent scans was deemed to be cystoid space. As for FA, CME on 3D OCT was assessed as Y, Q, N, or CG for each case.

Figure 1.

Cas examples: (A) FA (+), 3D-OCT (+). (B) FA (+), 3D-OCT (−). (C) FA (−), 3D-OCT (+). (D) FA (−), 3D-OCT (−).

Case examples are shown in Figure 1. The case in Figure 1A was graded as FA (+), 3D-OCT (+) because of late petaloid and honeycomb leakage on FA and circular or ovoid intraretinal hyporeflective spaces on adjacent OCT B-scans. The case in Figure 1B was graded as FA (+), 3D-OCT (−) because of late petaloid leakage on FA, but no definite circular or ovoid intraretinal hyporeflective spaces on OCT. It is notable that the graders agreed that this case demonstrated noncystoid edema but did not meet the criteria for CME as defined in this study. The case in Figure 1C was graded as FA (−), 3D-OCT (+) because of diffuse, nonpetaloid, nonhoneycomb late leakage on FA, yet multiple circular or ovoid intraretinal hyporeflective spaces on OCT. The case in Figure 1D was graded as FA (−), 3D-OCT (−) because no evidence of late petaloid or honeycomb leakage was found on FA, and no circular or ovoid hyporeflective spaces were seen on OCT.

A single consensus grade for each case was achieved with a standard reading center adjudication process. Briefly, for all cases in which there was any disagreement, all four graders reviewed the case together and discussed findings in an attempt to reach agreement. After this discussion, each grader was given the opportunity to revise the original assessment. These new individual grades were the basis of the postadjudication κ agreement score calculations discussed in the next section. If, after this group discussion and opportunity to revise grades, a split decision (50:50) remained, the grade provided by the most senior grader and reading center principal investigator (SRS) was chosen as the final adjudicated answer. This adjudication process was performed separately for the FA and OCT data, and the OCT and FA adjudications were conducted several weeks apart to avoid any potential grading bias.

Statistical Analysis

The level of agreement between multiple graders, both before and after adjudication, was determined using Fleiss' κ coefficient. Fleiss' κ is a statistical measure between a fixed number of raters when assigning categorical ratings to several items or classifying items. It contrasts with other κ statistics, such as the commonly used Cohen's κ, which is used when assessing the agreement between two raters. Although there is no universal agreement on measure of significance, we used previously suggested guidelines to assess our results, with 0.21 to 0.40 considered to be fair agreement; 0.41 to 0.60, moderate agreement, 0.61 to 0.80, substantial agreement; and 0.81 to 1.00, almost perfect agreement.²³ Given the potential confusion of questionable grades, each κ value was measured with and without the questionable grades (Intercooled Stata for Windows, ver. 9, Statacorp LP, College Station, TX).

The single final adjudicated answer for each modality for each eye was used in sensitivity calculations. The ground truth for each case was determined by combining positive findings from FA and 3D-OCT to generate the maximum possible level of detection for CME. As mentioned previously, eyes assessed as CG in either FA or 3D-OCT were excluded. Sensitivity calculations were performed both with and without the inclusion of cases with questionable grades. When including questionable grades, multiple strategies were considered. In one scenario, questionable grades were given only partial credit (Table 1). For example, in a case in which the ground truth is Y but the adjudicated grade is Q, a value of 0.5 is added to the true-positive column (implying that the modality is half correct), and a value of 0.5 is added to the false-negative column (implying that the modality is half incorrect). In a case where the ground truth is Q and the adjudicated grade is N, the modality receives partial credit for a negative answer (true negative = 0.5), since the ground truth grade is uncertain; however, the modality also receives partial credit for a false negative, since the ground truth indicates that there is a chance that CME is present. If, in this case, the modality grade is Q (which agrees with the ground truth), the modality is given partial credit both for a true positive and a true negative, since both grades show equal uncertainty about the existence of CME. Sensitivities were also separately calculated counting questionable grades as if they were Y and then again as if they were N. The ground truth for each case was determined by combining positive findings from FA and 3D-OCT to generate the maximum possible level of detection for CME. The sensitivity of each method for the detection of CME was calculated as the proportion of true positives correctly identified as such [sensitivity = true positives/(true positives + false negatives)].

Table 1.

Partial Credit Values for Sensitivity Calculation

Ground Truth	Test	True Positive	False Negative	True Negative	False Positive
Y	Y	1	0	0	0
Y	Q	0.5	0.5	0	0
Y	N	0	1	0	0
Q	N	0	0.5	0.5	0
N	N	0	0	1	0

Questionable grades were incorporated into sensitivity calculations for each imaging test versus the ground truth. Y, present; N, absent; Q, questionable.

Results

Characteristics of the Study Population

Two hundred forty-nine patients underwent both angiography and OCT at the same visit in the specified period. Forty-two were excluded due to missing medical records data or lack of 3D-OCT data. Two hundred seven met the eligibility criteria for this study. Fifteen eyes of 15 patients were excluded because of the inability to locate both 3D-OCT and FA images. Three eyes (two FA, one 3D-OCT) had insufficient image quality, as assessed by reading center criteria, to be included in the study, leaving 413 visits for 399 unique eyes of 207 patients (107 women and 100 men) to be included in the study. Fourteen eyes of seven patients were also studied in follow-up visits, meaning that these patients' eyes were included twice in the study.

The mean age of the patients was 68.32 years, with a range of 19 to 95 years. The range of BCVA was 20/20 to hand motion with a mean visual acuity of 20/60 (Table 2). Among all eyes, 146 (35%) had AMD, 20 (5%) had BRVO/CRVO, 77 (19%) had diabetes mellitus (DM), 49 (12%) had ERM/VMT, and 161 (39%) had other retinal diseases (Table 2). In total, 272 (66%) eyes were phakic and 141 (34%) were pseudophakic or aphakic (Table 2). All images included in the study met reading center criteria for sufficient image quality, including the absence of significant image artifacts or generalized reductions in signal strength.

Table 2.

Demographics of Patients in the Study

Patients, n	207
Eyes, n	413
Sex (men/women), % (n)	48%:52% (100:107)
Mean age, y (range)	68.32 (19–95)
Mean vision (range)	20/60 (20/20, hand motion)
Distribution of eyes, n (%)
Macular disease
Age-related macular degeneration	146 (35)
Branch/central retinal vein occlusion	20 (5)
Diabetes mellitus	77 (19)
Epiretinal membrane/vitreomacular traction	49 (12)
Other retinal diseases	161 (39)
Lens status
Phakic eyes	272 (66)
Aphakic/pseudophakic eyes	141 (34)

Sensitivities for the Detection of CME

Grading results are shown in Table 3. Inclusion of questionable (Q) grades as positively detected CME resulted in a sensitivity for detection of CME of 74% for FA (156/210) and 83% for 3D-OCT (175/210). If questionable grades were interpreted as non-CME cases, the sensitivity for detection of CME was 44% for FA (67/151) and 95% for 3D-OCT (144/151). If questionable grades were given partial credit (see Table 1), the sensitivity for detection of CME was 62% (111.5/180.5) for FA and 88% (159.5/180.5) for 3D-OCT. If questionable grades were excluded from all the calculations, sensitivities for detection of CME were 63% for FA (67/107) and 99% for 3D-OCT (144/145).

Table 3.

Distribution of CME Grades for Fluorescein Angiography (FA) and 3D-OCT

FA Grade	OCT Grade
	N	Q	Y
All Eyes
N	203 (49)	14 (3)	40 (11)
Q	34 (8)	11 (3)	44 (11)
Y	1 (0.2)	6 (1)	60 (15)
Transited Eyes Only
N	70 (33)	5 (2)	22 (10)
Q	23 (11)	9 (4)	32 (15)
Y	0 (0)	5 (2)	44 (21)

Data are shown as n (%).

Since the larger number of FA frames present in transited eyes could increase the sensitivity for detection of CME by FA, the subgroup of transited eyes were also analyzed independently, to determine whether the sensitivity for detection of CME was different. These results (Table 3) were not significantly different from inclusion of both eyes, when analyzed by the Kolmogorov-Smirnov test (P = 0.675).

Subgroup analyses were also performed based on diagnostic categories for each eye. No significant differences in the sensitivity for CME detection by FA and 3D-OCT were discovered for different disease categories (Table 4). Table 4A depicts the number of Y, N, and Q grades for FA and 3D-OCT image sets. A higher sensitivity was observed for 3D-OCT compared with FA for all disease etiologies, regardless of which method of reporting questionable cases was chosen. Sensitivities calculated after giving questionable cases partial credit are shown in Table 4B.

Table 4.

Detection of Cases of CME by FA and 3D-OCT

A. Detection According to Disease Etiology
Clinical Diagnosis	Grade
	N		Q		Y
	FA (n)	3D-OCT (n)	FA (n)	3D-OCT (n)	FA (n)	3D-OCT (n)
Age-related macular degeneration	89	84	44	13	13	49
Branch/central retinal vein occlusion	10	7	6	0	4	13
Diabetes mellitus	27	23	20	9	30	45
Epiretinal membrane/vitreomacular traction	26	22	10	6	13	21
Other retinal diseases	109	103	29	10	23	48
Phakic eyes	193	173	50	18	29	81
Aphakic/pseudophakic eyes	64	65	39	13	38	63

B. Diagnosis-Specific Sensitivities for FA and 3D-OCT Using Partial Credit*
Clinical Diagnosis	FA (%)	3D-OCT (%)
Age-related macular degeneration	54	85
Branch/central retinal vein occlusion	54	100
Diabetes mellitus	74	92
Epiretinal membrane/vitreomacular traction	68	91
Other retinal diseases	61	87
Phakic eyes	53	90
Aphakic/pseudophakic eyes	72	87

^*See Table 1 for description of partial credit.

Intergrader Agreement

If questionable grades were included in the analysis, Fleiss' κ coefficient for multiples graders was 0.43 for FA and 0.61 for 3D-OCT before adjudication. It increased to 0.58 for FA and 0.74 for 3D-OCT after adjudication. When questionable grades were excluded from the analysis, Fleiss' κ coefficient for multiples graders was 0.66 for FA and 0.87 for 3D-OCT. After adjudication, the κ statistics increased to 0.99 for FA and 1.00 for 3D-OCT (Fig. 2).

Figure 2.

Fleiss' κ scores for fluorescein angiography (FA) and 3-D optical coherence tomography (3D-OCT) before adjudication when Q grades were included and excluded.

Discussion

In this study, we found that 3D-OCT was more sensitive and reproducible than FA for the detection of CME by trained expert human graders. These results appear to corroborate previous reports that CME can be present in the absence of leakage into cystoid spaces on FA.^5,14–18 Since this study dealt only with CME, these results should not be extrapolated to compare detection of noncystoid leakage by FA with that of noncystoid edema by 3D-OCT.

CME is a pathologic definition with two components: abnormal collection of extracellular fluid and cystoid space formation.⁶ With FA, the presence of excess intraretinal fluid is often suggested by the progressive leakage of dye, sometimes with accumulation in apparently well-demarcated spaces. The presence of leakage on FA, however, is not always consistent with accumulation of intraretinal fluid. In the normal eye, the volume and composition of the extracellular compartment of the neurosensory retina is regulated by retinal capillary endothelial cell tight junctions (inner blood–retinal barrier), retinal pigment epithelial (RPE) cell tight junctions (outer blood–retinal barrier), and by the pumping function of RPE cells. Thus, intraretinal fluid can accumulate when there is loss of functional integrity in either of these fluid barriers and/or loss of an effective RPE pump. Although leakage of fluid (and progressive hyperfluorescence on FA) can result in fluid accumulation, leakage may also occur without fluid accumulation if the fluid that leaks into the retina or subretinal space is pumped out by the RPE cells at the same or a greater rate than the leakage. In this situation, fluorescein leakage would be visible on FA without corresponding fluid accumulation.

The opposite scenario is also possible. For example, fluid accumulation may occur without obvious hyperfluorescence if the source of leakage is very small, and the fluorescein molecules leak slowly and disperse quickly into the space. In this situation, these tissues may hyperfluoresce very late on the FA and appear only faintly hyperfluorescent on FA images captured within the typical time scale (up to 10 minutes). For this reason, FA may fail to demonstrate CME. In addition, in complex diseases such as CME in the setting of choroidal neovascularization (CNV), the accumulation of dye in intraretinal cystoid spaces may be difficult to distinguish from leakage from the underlying CNV or RPE alterations.

OCT has been touted to provide superior morphologic information compared with color photography and angiography. Numerous articles support this perspective by describing the presence of CME on OCT in a variety of macular diseases.^5,14,16–18 However, in most of these previous studies time-domain OCT instruments such as the Stratus OCT (Carl Zeiss Meditec, Dublin, CA) were used and a standardized protocol for CME interpretation was not implemented. The limitations of the sparse scanning density of time-domain OCT coupled with unstandardized evaluation methods may reduce the power of the conclusions in these previous studies and may limit their applicability in an era increasingly dominated by spectral domain OCT.

In a study of patients with diabetes mellitus, Ozdek et al.,³ detected CME with Stratus OCT in 30 (15.4%) eyes but could not confirm the diagnosis with FA in 63.3% of these cases. These results suggest that OCT is an important tool for detecting foveal changes that are not evident in diabetic eyes angiographically. However, this discrepancy, which is much larger than in our analysis, could also be the result of masking of the cystoid staining pattern in eyes with severe focal and diffuse leakage. Furthermore, it also could be the result of using different assessment methods for CME on OCT. If, for example, the authors used a lower threshold for determining the presence of a cystoid space on OCT versus FA, they could have discovered a larger disparity between FA and OCT for CME detection. In contrast, in the present study, a cystoid space was deemed to be present only if it was visible on two adjacent scans. Since the space between two B-scans in a 3D-OCT volume scan consisting of 128 B-scans across 6 mm is 47 μm, cystoid spaces would have to be at least 50 μm in diameter to fulfill our criteria.

We have proposed a simple and objective method of reporting of CME by using 3D-OCT. It is our hope that applying this criterion will lead to better grading reproducibility for cystoid spaces by the reading center graders. In this study, we did, in fact, observe significantly better intergrader agreement for OCT grading compared with FA grading. Specifically, the κ scores for multiple graders assessing 3D-OCT scans were higher than those obtained with FA, both before and after adjudication and regardless of inclusion or exclusion of questionable grades. Before adjudication, which better simulates a real clinical setting where only one clinician assesses diagnostic studies, intergrader agreement for 3D-OCT including questionable grades showed substantial agreement among different graders (Fleiss' κ = 0.61), whereas the agreement for FA was only moderate (Fleiss' κ = 0.43).

Regardless of how questionable grades were treated, 3D-OCT was found in this study to be more sensitive than FA for identifying intraretinal cystoid spaces (ranging from 83% to 99% for 3D-OCT and 44% to 74% for FA). This superior sensitivity of 3D-OCT was consistently observed regardless of underlying disease etiology. These results were somewhat unexpected, as we suspected that the disparity in sensitivity would be greater in eyes with choroidal vascular disease where underlying fluorescence could potentially make detection of cystoid spaces on FA more difficult. It should be noted that interpretation of the sensitivity results according to disease etiology are somewhat confounded in our analysis by the fact that several patients had more than one underlying disease (e.g., patients with both diabetic retinopathy and an ERM) and the disease subgroups were relatively small (n = 20–161; mean, 90.6).

In this study, we found no relationship between the inclusion of additional FA frames from the transited eye and improved detection of CME by FA when compared with 3D-OCT. As noted in the Methods section, the Doheny Imaging Unit FA protocol requires photographers to obtain mid- and late-phase images of both eyes. Since the cystoid spaces are most clearly evident in the late phases after there has been time for dye to pool in the space, this observation is not surprising.

This study has several limitations. The data were collected retrospectively, and thus there is a potential ascertainment bias. In addition, although the photographers at the Doheny Ophthalmic Imaging Unit use standard protocols for FA and 3D-OCT imaging, there may be variability due to physician and patient factors. These variables include different fields of view (50° vs. 35°), different late frame times (10 vs. 15 minutes), inconsistent venous access, and different amounts of stereopsis in stereo image pairs. Since only one eye was transited, the amount of FA image data differed for approximately half of the eyes in this study, although, as mentioned, no relationship was observed between sensitivity results and the transited eye. Differential effects of media opacities on FA and 3D-OCT image quality may have played a role in the graders' ability to recognize accumulation of dye in cystoid spaces on FA. In addition, the threshold used to identify cystoid spaces on 3D-OCT may have been too high, since it excludes potential cystoid spaces less than 50 μm in diameter. However, if this were the case, we would have measured an even greater sensitivity for 3D-OCT which would not have substantially changed the study's conclusions.

Lack of a definite ground truth is also a limitation. Ideally, FA and OCT findings should be compared to histology. Since this is obviously not possible, we chose to use the combined findings of both modalities as the best available surrogate. This combined gold standard approach removes the possibility of having false positives and therefore results in perfect specificity. In turn, this method could enable unrealistic estimations of sensitivity if, for instance, the threshold for detection of CME by OCT were too low.

The handling of questionable grades is another potential limitation. Because the optimal method for handling these indeterminate grades is uncertain, we evaluated multiple strategies for excluding or including these cases. As a result, we are only able to provide a range of sensitivities. We were reassured to see a consistent relationship regardless of the method chosen.

Conclusion

The findings from our study suggest that cystoid spaces are more frequently identified on 3D-OCT compared with FA, regardless of disease etiology. Furthermore, the reproducibility of CME detection with 3D-OCT was superior to FA. These findings may be useful in the design of future studies and clinical trials of treatments for diseases associated with cystoid macular edema.