Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Aug 1.
Published in final edited form as: Ophthalmology. 2015 May 28;122(8):1601–1608. doi: 10.1016/j.ophtha.2015.04.023

Influence of Fluorescein Angiography on the Diagnosis and Management of Retinopathy of Prematurity

Michael A Klufas 1,*, Samir N Patel 1,*, Michael C Ryan 2, Mrinali Patel Gupta 1, Karyn E Jonas 1, Susan Ostmo 2, Maria Ana Martinez-Castellanos 3, Audina M Berrocal 4, Michael F Chiang 2, RV Paul Chan 1
PMCID: PMC4800809  NIHMSID: NIHMS763678  PMID: 26028345

Abstract

Purpose

To examine the influence of fluorescein angiography (FA) on the diagnosis and management of retinopathy of prematurity (ROP).

Design

Prospective cohort study.

Participants

Nine recognized ROP experts (3 pediatric ophthalmologists; 6 retina specialists) interpreted 32 sets (16 color fundus photographs; 16 color fundus photographs paired with the corresponding FAs) of wide-angle retinal images from infants with ROP.

Methods

All experts independently reviewed the 32 image sets on a secure web site and provided a diagnosis and management plan for the case presented, first based on color fundus photographs alone, and then by color fundus photographs and corresponding FA.

Main Outcome Measures

Sensitivity and specificity of the ROP diagnosis (zone, stage, plus disease, and category – i.e. no ROP, mild ROP, type-2 ROP, and treatment-requiring ROP) was calculated using a consensus reference standard diagnosis, determined from the diagnosis of the color fundus photographs by three experienced readers in combination with the clinical diagnosis based on ophthalmoscopic examination. The kappa statistic was used to analyze the average intergrader agreement among experts for the diagnosis of zone, stage, plus disease, and category.

Results

Addition of FA to color fundus photographs resulted in a significant improvement in sensitivity for diagnosis of stage 3 or worse disease (39.8% vs. 74.1%, P = 0.008), type-2 or worse ROP (69.4% vs. 86.8%, P = 0.013), and pre-plus or worse disease (50.5 vs. 62.6%, P = 0.031). There was a nonsignificant trend towards improved sensitivity for diagnosis of treatment-requiring ROP (22.2% vs. 40.3%, P = 0.063). Using the kappa statistic, addition of FA to color fundus photographs significantly improved intergrader agreement for diagnosis of treatment-requiring ROP. Addition of FA to color fundus photographs did not significantly affect intergrader agreement for the diagnosis of stage, zone, or plus disease.

Conclusions

Compared to color fundus photographs alone, fluorescein angiography may improve the sensitivity of diagnosis of ROP by experts, particularly for stage 3 disease. In addition, intergrader agreement for diagnosis of treatment-requiring ROP may improve with FA interpretation.

Clinical examination by indirect ophthalmoscopy has long been the standard modality for the diagnosis of retinopathy of prematurity (ROP). On the basis of large, well-designed clinical trials including the Cryotherapy for ROP and Early Treatment for ROP trials,1, 2 a consensus policy statement was established in the United States for the screening and management of ROP.3 This policy statement recommended that examinations be performed “using binocular indirect ophthalmoscopy.”3 The policy statement also acknowledged a growing role for digital imaging in ROP but emphasized the need for further studies to parse out the utility of these imaging modalities in the diagnosis and management of ROP.

Fluorescein angiography (FA) has been shown to be critical for assessing the retinal vasculature in vasoproliferative disorders such as diabetic retinopathy4 and exudative age-related macular degeneration in adults.5 Additionally, FA has an important role in the evaluation and management of pediatric vascular disorders including Coats’ disease,6 choroidal neovascular membranes,7 sickle cell retinopathy,8 ocular tumors,9 and other conditions.10, 11 FA appears to be safe in children including neonates with ROP, with no adverse effects reported in several series.12–15 Flynn and colleagues introduced FA as a method to study retrolental fibroplasia in the late 1960s,16–18 and other investigators in this era noted the benefit of FA in evaluating the peripheral retina in the acute stages of ROP as well as for identifying late complications.19 These early investigators noted the presence of changes seen on FA that were not visible on clinical exam. Flynn and colleagues used a Zeiss fundus camera (Dublin, CA) to obtain the angiograms, but due to limitations in obtaining fundus images in neonates with this device, there were limited reports on FA in ROP for many subsequent years.

With the introduction of newer digital wide-angle and ultra-widefield imaging systems including those designed for pediatric use (e.g. RetCam, Clarity Medical Systems, Inc., Pleasanton, CA),10, 11, 20 it is now becoming more common to perform bedside fundus imaging21, 22 and FA12, 23 in the pediatric population. Given that bedside FA is now more accessible and may provide useful information regarding the developing retinal vasculature, there has been renewed interest in utilizing this diagnostic modality in the evaluation of ROP. Moreover, the shortage of trained ROP experts worldwide has prompted an interest in the role of telemedicine for this disease.24, 25 In turn, there has been particular interest in the utility of digital imaging for ROP.

The role of FA in the diagnosis of ROP by digital imaging still remains unclear. FA has recently been implemented to evaluate retinal vascular morphology in eyes injected with intravitreal anti-vascular endothelial growth factor (VEGF) therapy26–28; however, current studies on FA in ROP have been predominately descriptive, limiting their clinical impact and conclusions.12, 14, 15, 26, 27, 29 As such, a gap exists in understanding the utility of FA on the accuracy of diagnosis and management of ROP by pediatric ophthalmologists and retina specialists. The purpose of this study was to evaluate the influence of FA on the diagnosis and management of ROP by ROP experts using wide-angle fundus images.

Methods

This study was approved as a prospective study by the Institutional Review Board at Weill Cornell Medical College. Informed consent was obtained from all study participants prior to participation, and waiver of consent was obtained for use of de-identified retinal images. This study was conducted in accordance with Health Insurance Portability and Accountability Act guidelines and adhered to the tenets of the Declaration of Helsinki.

Image Acquisition

Wide-angle images of the posterior retina and corresponding FAs were captured bilaterally from 8 infants with ROP (16 eyes) using the RetCam-II (Clarity Medical Systems, Pleasanton, CA). Images were taken of infants between 33 and 44 weeks postmenstrual age. For acquisition of FAs, 4/8 (50%) infants were imaged in the neonatal intensive care unit without intubation or sedation, while the remaining 4/8 (50%) infants were imaged in the operating room under sedation.

Consensus Reference Standard Diagnosis

For each image set, a consensus reference standard ROP diagnosis was established. This was done by combining the clinical diagnosis as determined by indirect ophthalmoscopy and the image-based diagnosis from multiple experienced readers, as previously described.30 This consensus reference standard was then used for the purposes of this current study.

Study Experts

Eligible participants for this study were defined as board certified practicing pediatric ophthalmologists or retina specialists who routinely evaluate infants for ROP and met at least one of the following criteria: having been a principal investigator or certified investigator for the Cryotherapy for ROP study or Early Treatment for ROP study or having published at least two peer-reviewed ROP articles. These participants are further referred to as “experts” in this study.

Study Design

Study experts were directed to a secure website developed by the authors (MAK, SNP, MFC, RVPC). Initial baseline demographic data was collected from each expert including what fellowship training had been completed (pediatric ophthalmology, medical retina, surgical retina), years since completion of fellowship, and level of comfort with reading color fundus photographs and FAs in ROP (not comfortable, somewhat comfortable, comfortable). Experts were also asked, in their clinical practice, what percentage of patients with type-2 or worse ROP did they obtain a FA (0%, 1–25%, 26–50%, 51–75%, 75–99%, 100%), and finally, whether they believe FA is safe in infants and neonates (yes, no). Type-2 ROP is defined as (a) zone I, stage 1 or 2 without plus disease; or (b) zone II, stage 3 without plus disease. Treatment-requiring ROP is defined as (a) zone I, any stage with plus disease; (b) zone I, stage 3 with or without plus disease; or (c) zone II, stage 2 or 3 with plus disease.

Experts were presented with a series of eight ROP cases. Each case consisted of baseline demographic information (birth weight, gestational age, and postmenstrual age at the time of imaging) and an image set of color fundus photographs (Figure 1 available at http://www.aaojournal.org). Color fundus photographs were displayed as a set of three retinal images of each eye (temporal, posterior, nasal). For each image set, experts were asked to choose the zone (I, II, II-Posterior, III), stage (1, 2, 3, 4, 5), plus (no, pre-plus, plus), category (mild, type-2 ROP, treatment-requiring ROP), management (observation, laser only, anti-VEGF only, laser with anti-VEGF, surgery), presence of aggressive posterior ROP (AP-ROP; yes/no), and recommended clinical follow-up (less than 1 week, 1 week, 2 weeks, greater than 2 weeks). Experts were asked their level of confidence (confident, somewhat confident, not confident) in determining the clinical diagnosis based on color fundus photographs provided and whether they would obtain a FA based on the color fundus photographs (yes/no). This was performed in sequential order for each of the 8 cases.

Figure 1. Study Design Presented to Retinopathy of Prematurity (ROP) Experts to Determine ROP Diagnosis and Management Plan using Color Fundus Photography and Fluorescein Angiography (FA).

Figure 1

Open in a new tab

In the first part of this study, each expert completed 8 clinical cases in which he or she provided a diagnosis of zone, stage, plus, category, management, advanced posterior-ROP, and recommended clinical follow-up. In the second part of the study, experts were presented with the same 8 clinical cases, but were now provided with the corresponding FAs. For each clinical case, the experts were asked to provide a clinical diagnosis and management plan, but were not able to see their previous responses from the first part of the study.

Next, a second image set comprised of the same color fundus photographs accompanied by their corresponding FAs, was presented for each of the 8 cases in the same sequential order. For each image set, the expert was again asked to determine zone, stage, plus, category, management, presence of AP-ROP, and recommended clinical follow-up. Experts were also asked to gauge their level of confidence in determining the diagnosis based on color fundus photographs and FAs (confident, somewhat confident, not confident) and whether they felt that the FAs had provided clinically useful information for management purposes (yes/no).

Data Analysis

All data were analyzed using statistical software (Stata/SE 12.0; StataCorp LP, College Station, TX). A Wilcoxon signed-ranked test was run to determine if there were differences in image quality or confidence in identifying zone, stage, and category of disease using the two imaging modalities.

Using the consensus reference standard diagnosis, the performance of individual experts was evaluated for each modality (color fundus photographs and color fundus photographs + FA) by comparing areas under the receiver operating characteristic curves. These results were averaged among all experts to determine the sensitivity and specificity of each modality for detecting stage 1 or worse disease, stage 2 or worse disease, stage 3 or worse disease, disease in zone I, disease in either zone I or zone II, pre-plus or worse disease, plus or worse disease, mild or worse ROP, type-2 ROP or worse, and treatment-requiring ROP. The non-parametric sign test was performed to determine if the mean difference in sensitivity/specificity between paired color fundus photographs and FAs was significantly different from zero.31

To evaluate intergrader reliability, each International Classification of Retinopathy of Prematurity (ICROP) criterion was dichotomized, forming a two-level categorization: ≥stage 3 disease or not, zone I disease or not, plus disease or not, ≥type-2 ROP or not, and treatment-requiring ROP or not. The unweighted kappa (κ) statistic was calculated to measure chance-adjusted agreement for each head-to-head paring of readers. These results were averaged to determine the mean unweighted κ for each reader in each category. A widely-accepted scale was used to interpret the results: 0 to 0.20 indicated slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, substantial agreement; and 0.81 to 1.00, almost perfect agreement.22

Results

Study Experts

Nine ROP experts (based on the study definition) consented to participate. Among these experts, 6/9 (67%) were retina specialists and 3/9 (33%) were pediatric ophthalmologists. The experts have been practicing ophthalmology for an average of 18.9 years (range: 10 – 33 years).

Each expert evaluated 32 image sets (16 color fundus, 16 color fundus + FA) from 16 eyes, for a total of 288 readings. 8/9 (89%) experts reported that they were “comfortable” with reading color fundus photographs, while 1/9 (11%) expert was “somewhat comfortable”. When experts were asked about comfort with reading fluorescein angiograms, 6/9 (67%) were “comfortable”, 2/9 (22%) were “somewhat comfortable”, and 1/9 (11%) was “not comfortable”.

Distribution of Expert Responses based on Imaging Modality

Figure 2 shows the diagnostic responses of the nine experts when the consensus reference standard diagnosis was type-2 ROP (Figure 2A), treatment-requiring ROP (Figure 2B), stage 3 (Figure 2C), stage 2 (Figure 2D), and zone I (Figure 2E).

Figure 2. Mean Distribution of Retinopathy of Prematurity (ROP) Diagnosis by 9 ROP Experts using Color Fundus Photographs Only versus Color Fundus Photographs with Corresponding Fluorescein Angiograms (FA).

Figure 2

Open in a new tab

Gray bars indicate diagnosis using color fundus photographs only and black bars indicate diagnosis using color fundus photographs and corresponding FAs (black bars). A, Distribution of responses for the 8 eyes with a consensus diagnosis of type 2-ROP. B, Distribution of responses for the 8 eyes with a consensus diagnosis of treatment-requiring ROP. C, Distribution of responses for the 12 eyes with a consensus diagnosis of stage 3. D, Distribution of responses for the 4 eyes with a consensus diagnosis of stage 2. E, Distribution of responses for the 8 eyes with a consensus diagnosis of zone I.

ROP = Retinopathy of Prematurity; FA = Fluorescein Angiogram; TR-ROP = Treatment requiring ROP

Notably, among the 12 eyes with the consensus reference standard diagnosis of stage 3, experts diagnosed stage 3 or worse in 43/108 (40%) responses using color fundus photographs alone versus 80/108 (74%) responses using color fundus photographs and FAs (Figure 2C). Among the 4 eyes with the consensus reference standard diagnosis of stage 2, experts diagnosed stage 2 in 19/36 (53%) responses using color fundus photographs alone versus 15/36 (42%) responses using color fundus photographs and FAs (Figure 2D). Furthermore, among these 4 eyes with a consensus reference standard diagnosis of stage 2, experts diagnosed stage 3 or worse in 13/36 (36%) responses using color fundus photographs alone versus 21/36 (58%) responses using color fundus photographs and FAs.

When viewing the corresponding color fundus photographs and FA, experts altered their choice of category (none, mild, type-2 ROP, treatment-requiring ROP) in 66/144 (46%) responses. Pediatric ophthalmologists had 19/48 (40%) changes in category, while retina specialists had 47/96 (49%) changes in category after viewing the corresponding FA. Of the 66 changes in category after viewing the color fundus photographs and FA, 31/66 (47%) were changes to a diagnosis consistent with the consensus reference standard diagnosis, 15/66 (23%) were changes from one consistent with the consensus reference standard diagnosis to a diagnosis inconsistent with the consensus reference standard diagnosis, and 20/66 (30%) were changes from a diagnosis inconsistent with the consensus reference standard diagnosis to another diagnosis also inconsistent with the consensus reference standard diagnosis.

After viewing the FAs corresponding to the color fundus photographs, experts altered their choice of management in 37/144 (26%) responses. Pediatric ophthalmologists had 11/48 (23%) changes in management after viewing the corresponding FA and, specifically, 2/11 (18%) responses changed from observation to laser treatment, while 5/11 (45%) responses changed from observation to anti-VEGF therapy. Retina specialists had 26/96 (27%) changes in management after viewing the corresponding FA and, specifically, 13/26 (50%) responses changed from observation to laser treatment, while 3/26 (12%) changed from observation to anti-VEGF therapy.

Sensitivity and Specificity of ROP Diagnosis

Table 1 shows the mean (95% CI) sensitivity and specificity of stage, zone, plus disease, and category for study experts as compared to the consensus reference standard diagnosis. With the supplementation of FAs to color fundus photographs, there were statistically significant improvements in the sensitivity for diagnosing stage 2 or worse (P = 0.016), stage 3 or worse (P = 0.008), pre-plus or worse (P = 0.031), and type-2 ROP or worse (P = 0.013) (Table 1). No significant changes in specificity were seen with the supplementation of FAs to color fundus photographs.

Table 1.

Sensitivity and Specificity of Retinopathy Of Prematurity (ROP) Diagnosis By ROP Experts Based on a Consensus Reference Standard Diagnosis.

Sensitivity
(95% CI)		 P value* Specificity
(95% CI)		 P value*
Color Fundus
Only		 Color Fundus
+ FA		 Color Fundus
Only		 Color Fundus
+ FA
Stage
  Stage 1 or worse 92.4
(86.7 – 96.1)		 100.0
(97.5 – 100)		 0.063 - - -
  Stage 2 or worse 81.3
(73.9 – 87.3)		 99.3
(96.2 – 99.9)		 0.016 - - -
  Stage 3 or worse 39.8
(30.5 – 49.7)		 74.1
(64.8 – 82.0)		 0.008 63.9
(46.2 – 79.2)		 41.7
(25.5 – 59.2)		 0.453
Zone
  Zone I 47.2
(35.3 – 59.3)		 61.1
(48.9 – 72.4)		 0.219 100
(95 – 100)		 100
(95 – 100)		 1.000
  Zone II 97.2
(93 – 99.2)		 97.2
(93 – 99.2)		 1.000 - - -
Plus
  Pre-Plus or worse 50.5
(40.3 – 60.7)		 62.6
(52.3 – 72.1)		 0.031 73.3
(58.1 – 85.4)		 55.6
(40 – 70.4)		 0.125
  Plus 13.3
(5.1 – 26.8)		 20.0
(9.6 – 34.6)		 1.000 87.9
(79.8 – 93.6)		 84.8
(76.2 – 91.3)		 1.000
Category
  Mild or worse 91.7
(85.9 – 95.6)		 100
(97.5 – 100)		 0.0625 - - -
  Type-2 or worse 69.4
(61.2 – 76.8)		 86.8
(80.2 – 91.9)		 0.013 - - -
  Treatment-requiring 22.2
(13.3 – 33.6)		 40.3
(28.9 – 52.5)		 0.063 86.1
(75.9 – 93.1)		 76.4
(64.9 – 85.6)		 0.375
Open in a new tab

FA, Fluorescein angiogram; CI, Confidence Interval.

*

Non-parametric sign test.

Intergrader Agreement of Zone, Stage, Plus and Category of Disease

Figure 3 displays a strip-plot for the mean intergrader agreement for each expert as compared to all other experts using unweighted κ. When stage was dichotomized into stage 3 ROP or not, experts had a mean (range) kappa of 0.06 (−0.05 to 0.20) with color fundus photographs versus 0.09 (−0.06 to 0.21) with color fundus photographs and FAs, P = 1.00. When zone was dichotomized into zone I or not, experts had a mean (range) kappa of 0.34 (0 to 0.47) with color fundus photographs versus 0.48 (0 to 0.64) with color fundus photographs and FAs, P = 0.07. When plus was dichotomized into plus disease or not, experts had a mean (range) kappa of 0.07 (−0.05 to 0.23) with color fundus photographs versus 0.04 (0 to 0.13) with color fundus photographs and FAs, P = 0.38. When category was dichotomized into whether or not there was type-2 ROP or worse, experts had a mean (range) kappa of 0.11 (−0.02 to 0.22) with color fundus photographs versus 0.01 (−0.11 to 0.08) with color fundus photographs and FAs, P = 0.29. When category was dichotomized into treatment-requiring ROP or not, experts had a mean (range) kappa of 0.003 (−0.06 to 0.05) with color fundus photographs versus 0.089 (0 to 0.18) with color fundus photographs and FAs, P = 0.02.

Figure 3. Intergrader Agreement of Retinopathy of Prematurity (ROP) Diagnosis by 9 ROP Experts using Color Fundus Photographs only versus Color Fundus Photographs with Corresponding Fluorescein Angiograms (FA).

Figure 3

Open in a new tab

Intergrader agreement is measured as the unweighted kappa (κ) statistic for each expert as compared to all other experts. Each open circle represents the mean unweighted κ for a single expert. The closed circle is the value for the unweighted κ averaged across all experts, and the whiskers off the closed circles denote the 95% confidence interval for this mean. Significance is calculated using a nonparametric sign test that analyzes mean κ differences using color fundus photographs versus color fundus photographs with corresponding FAs.

FA = Fluorescein Angiogram

Confidence of Clinical Diagnosis and Image Quality

For the 144 color fundus photographs, image quality was scored as “adequate” in 85/144 (59%) responses, “somewhat adequate” in 37/144 (26%) responses, and “not adequate” in 22/144 (15%) responses. For the 144 FAs, image quality was scored as “adequate” 72/144 (50%) responses, “somewhat adequate” in 58/144 (40%) responses, and “not adequate” in 14/144 (10%) responses. Overall, there was no difference in image quality scoring between color fundus photographs and FAs (P = 0.517).

For the color fundus photographs, confidence in diagnosis was scored as “confident” in 20/72 (28%) responses, “somewhat confident” in 36/72 (50%) responses, and “not confident” in 16/72 (22%) responses. For the combined color fundus photograph and FA image set, confidence in diagnosis was scored as “confident” in 46/72 (64%) responses, “somewhat confident” in 24/72 (33%) responses, and “not confident” in 2/72 (3%) responses. There was a statistically significant increase in confidence with the diagnosis when experts used color fundus photographs and FAs compared to the color fundus photographs only (P < 0.001).

After viewing the color fundus photographs for each case, 9/72 (13%) responses stated they would obtain a FA. After viewing the color fundus photographs and FA for each case, 58/72 (81%) responses stated they believed the FA provided clinically useful information for management purposes.

Discussion

The key findings of this study are: (1) the addition of FAs to color fundus photographs resulted in a statistically significant increase in sensitivity for diagnosis of stage 2 or worse, stage 3 or worse, pre-plus or worse, and type-2 ROP or worse as compared to a consensus reference standard diagnosis; (2) the addition of FAs to color fundus photographs resulted in a statistically significant improvement in intergrader agreement for diagnosis of treatment-requiring ROP.

The first key finding is that compared to color fundus photographs alone, the addition of FA resulted in an increased sensitivity of diagnosis of stage 2 or worse, stage 3 or worse, pre-plus or worse, and type-2 ROP or worse. Perhaps the most notable effect of FA was upon the sensitivity of diagnosis of stage 3 or worse. In eyes with a consensus reference standard diagnosis of stage 3, the sensitivity for diagnosis of stage 3 or worse by experts increased from 43/108 (40%) using color photographs alone to 80/108 (74%) with color fundus photographs and FA. In turn, there was an increase in sensitivity of diagnosis of type-2 ROP or worse (P = 0.013) and a trend toward an increased sensitivity for diagnosis of treatment-requiring ROP (P = 0.063). Concomitant with the improvement in sensitivity of stage 3 or worse diagnosis was a trend toward reduction in specificity, which did not reach significance (P = 0.375). Indeed, even in patients with a consensus reference standard diagnosis of stage 2, FA resulted in a shift towards less stage 2 and more stage 3 diagnoses (Figure 2D). In contrast, this study also demonstrated that fluorescein angiography may offer less of an advantage over color fundus photographs alone in the diagnosis of zone or plus disease.

The impact of FA on stage 3 diagnosis likely reflects this imaging modality’s ability to highlight retinal vessels and in turn, stage 3 (extraretinal fibrovascular proliferation) may be more easily identified on fluorescein angiogram. The ability of FA to demonstrate retinal neovascularization not apparent on clinical examination has long and extensively been demonstrated in other vascular retinal disorders.32 The ability of FA to highlight retinal vascular structures may also underlie the increased diagnosis of stage 3 by the ROP experts in this study after incorporation of FA in cases with a consensus reference standard diagnosis of stage 2. The characteristic clinical finding in stage 2 is the presence of a ridge without extraretinal fibrovascular proliferation. This ridge is comprised of endothelial cells that may form vascular channels and shunts.17 As previously demonstrated by Flynn et al,17 these vascular channels and shunts in stage 2 may exhibit leakage on FA. Misinterpretation of this leakage as neovascularization may have contributed to the shift in diagnosis from stage 2 to stage 3 with addition of FA in cases with a consensus reference standard diagnosis of stage 2.

Our findings therefore emphasize the potential need to develop standardized criteria for FA findings for each ROP stage. Because there are currently no standardized criteria for classifying FA findings in stage 2 versus stage 3, it is unclear what metrics were implemented by the study experts in the evaluation of the FAs. Although ROP classification using color fundus photographs is standardized according to the criteria outlined by ICROP,33 there is no defined role or consensus classification of fluorescein angiograms in ROP diagnosis. If more physicians begin to incorporate FA into their management of ROP, it would be useful that a standardized classification system be devised as has occurred with other ophthalmic disorders using FA like diabetic retinopathy and age-related macular degeneration.4, 5 Nevertheless, if FA and other imaging modalities are incorporated including OCT, there is an inherent cost to this algorithm.

Previous reports have suggested that FA might provide a more objective assessment of zone due in part to providing high contrast images of the peripheral retina.12, 15 However, our study did not find a statistically significant improvement in sensitivity or specificity of a zone I or II diagnosis when color fundus photographs were supplemented with fluorescein angiograms (Table 1). We did observe that a greater proportion of experts did trend toward selection of zone I disease when presented with FAs (Figure 2E), and that for the diagnosis of zone I or not, intergrader agreement trended from “fair” to “moderate” agreement (P = 0.07) (Figure 3).

The second key finding in this study is that supplementing FA to color fundus photographs resulted in an improvement in intergrader agreement among experts for the diagnosis of treatment-requiring ROP, with a mean kappa improvement from 0.003 to 0.089 (Figure 3). Although this is a statistically significant improvement, the interpretation of the mean kappa remains at “slight agreement” with the addition of FAs based on a widely-accepted scale for kappa interpretation.22 One previous study demonstrated that agreement between expert and non expert interpretations of retinal photomontages was greater for fluorescein angiograms than for color fundus photographs.34 Furthermore, prior studies have revealed that variability exists in intergrader agreement for diagnosis of ROP: The Cryotherapy for ROP trial noted disagreement between two unmasked certified examiners as to whether threshold disease was present in 12% of eyes.1 Disagreement in the diagnosis of severe ROP has also been noted among ROP experts even when examining the exact same image sets.22, 35 Given prior reports of poor intergrader agreement in ROP diagnosis using color fundus photographs only, our findings in this study of improved intergrader agreement for the diagnosis of treatment-requiring ROP suggest that FA may be a useful adjunct in improving diagnostic agreement among ROP examiners.

Several previous studies have investigated expert versus non-expert diagnosis of ROP with wide-angle colors fundus photographs only,36, 37 and one other has introduced fluorescein angiograms.34 This area deserves special attention given the increasing need for digital or telemedicine systems to undertake ROP screening with a limited supply of expert ophthalmologists available to perform screenings.38 Previous studies on ROP screening among trainees have noted that variability exists among pediatric ophthalmology and retinal fellows in the diagnostic accuracy of clinically significant ROP.39, 40 If FA is able to improve the ability of imaging to detect treatment-requiring disease among non-experts,34 and potentially among experts, this could have important implications for ROP screening protocols. Indeed, we found poor sensitivity for diagnosis with color fundus photographs – especially for more advanced ROP (i.e. zone I, stage 3, plus, and treatment-requiring ROP). Addition of FA significantly improved sensitivity of stage 3 diagnosis, resulting in a statistically significant increase in sensitivity of type-2 ROP or worse diagnosis and a trend towards increased sensitivity for the diagnosis of treatment-requiring ROP. This study may demonstrate the potential for FA to offset some of the shortcomings of using color fundus photographs alone for ROP diagnosis. Also, because our consensus reference standard diagnosis strongly incorporated the clinical diagnosis using indirect ophthalmoscopy, our findings suggest that FA may add a dimension to digital image analysis that is more consistent with examination findings from indirect ophthalmoscopy.

Limitations of this study include (1) the lack of multiple early, middle and late fluorescein angiograms. Some exams were performed under anesthesia, and the need to limit anesthesia time prompted the acquisition of only a finite number of FA images; (2) Limitations also exist with color fundus photographs, which may have peripheral distortion, glare and compromised contrast depending on the skill of the user obtaining the photographs and other optical factors at the time of image acquisition.41 Indeed, the eight clinical cases for the study were selected to be of excellent quality and represent a range of ROP severity; (3) This study did not assess the impact of FA on ROP diagnosis in the clinical setting as other similar studies have done for FA in other retinal conditions,42, 43 but rather, the impact of FA on ROP diagnosis based on digital imaging alone. However, we integrated the clinical diagnosis based on color fundus photographs into the consensus reference standard diagnosis in order to account for the findings noted on clinical exam. (4) This study did not incorporate other multimodal imaging such as OCT nor did it perform a cost analysis if fluorescein angiography was incorporated into all standard telemedicine screening programs. (5) Additionally, there exists variability between the experts’ comfort with reading color fundus photos and fluorescein angiograms which may be a function of years of experience, current practice patterns, among others which may introduce bias and influence the results of this study.

Overall, this study contributes to the body of ROP knowledge by showing that compared to color fundus photographs alone, fluorescein angiography may improve the sensitivity of diagnosis by ROP experts, particularly for stage 3 disease and in turn, may improve accuracy of diagnosis of type-2 ROP or worse and treatment-requiring ROP. Fluorescein angiography continues to have an evolving role in the screening, diagnosis, and management of ROP in the era of multimodal imaging and telemedicine. Larger studies utilizing standardized reading criteria for fluorescein angiograms in ROP are needed to further elucidate the optimal use of FA in ROP.

Acknowledgments

Financial Support: This investigation was supported by National Center for Advancing Translational Sciences (NCATS) grant # UL1TR00457 of the Clinical and Translational Science Center at Weill Cornell Medical College (New York, New York), grant EY19474 from the National Institutes of Health (Bethesda, Maryland), unrestricted departmental funding from Research to Prevent Blindness (New York, New York), The St. Giles Foundation (New York, New York) and The iNsight Foundation (New York, New York). The sponsors and funding organizations had no role in the design or conduct of this research.

Abbreviations and Acronyms

FA

Fluorescein angiography

ROP

retinopathy of prematurity

VEGF

vascular endothelial growth factor

AP-ROP

aggressive posterior retinopathy of prematurity

ICROP

International Classification of Retinopathy of Prematurity

CI

confidence interval

Footnotes

Meeting Presentation: Portions of this manuscript have been presented at the XXIXth Meeting of the Club Jules Gonin, September 3–6, 2014, Zurich, Switzerland and the Annual Meeting of the Retina Society, September 11–14, 2014, Philadelphia, PA.

Conflict of Interest: MFC is an unpaid member of the Scientific Advisory Board for Clarity Medical Systems (Pleasanton, CA).

This article contains additional online-only material. The following should appear online-only: Figure 1.

References

  • 1.Cryotherapy for Retinopathy of Prematurity Cooperative Group. Multicenter Trial of Cryotherapy for Retinopathy of Prematurity: preliminary Results. Pediatrics. 1988;81(5):697–706. [PubMed] [Google Scholar]
  • 2.Early Treatment For Retinopathy Of Prematurity Cooperative Group. Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial. Arch Ophthalmol. 2003;121(12):1684–1694. doi: 10.1001/archopht.121.12.1684. [DOI] [PubMed] [Google Scholar]
  • 3.American Academy of Pediatrics Section on Ophthalmology, American Academy of Ophthalmology, American Association for Pediatric Ophthalmology and Strabismus, American Association of Certified Orthoptists. Screening Examination of Premature Infants for Retinopathy of Prematurity. Pediatrics. 2013;131(1):189–195. doi: 10.1542/peds.2012-2996. [DOI] [PubMed] [Google Scholar]
  • 4.Early Treatment Diabetic Retinopathy Study Research Group. Classification of Diabetic Retinopathy from Fluorescein Angiograms: ETDRS Report Number 11. Ophthalmology. 1991;98(suppl 5):807–822. [PubMed] [Google Scholar]
  • 5.Mokwa NF, Ristau T, Keane PA, et al. Grading of Age-Related Macular Degeneration: comparison between Color Fundus Photography, Fluorescein Angiography, and Spectral Domain Optical Coherence Tomography. J Ophthalmol. 2013;2013:385915. doi: 10.1155/2013/385915. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Koozekanani DD, Connor TB, Jr, Wirostko WJ. RetCam II Fluorescein Angiography to Guide Treatment and Diagnosis of Coats Disease. Ophthalmic Surg Lasers Imaging. 2010:1–3. doi: 10.3928/15428877-20100215-86. [DOI] [PubMed] [Google Scholar]
  • 7.Giansanti F, Virgili G, Varano M, et al. Photodynamic therapy for choroidal neovascularization in pediatric patients. Retina. 2005;25(5):590–596. doi: 10.1097/00006982-200507000-00009. [DOI] [PubMed] [Google Scholar]
  • 8.Hero M, Harding SP, Riva CE, et al. Photographic and angiographic characterization of the retina of Kenyan children with severe malaria. Arch Ophthalmol. 1997;115(8):997–1003. doi: 10.1001/archopht.1997.01100160167005. [DOI] [PubMed] [Google Scholar]
  • 9.Shields JA, Reichstein D, Mashayekhi A, Shields CL. Retinal vasoproliferative tumors in ocular conditions of childhood. J AAPOS. 2012;16(1):6–9. doi: 10.1016/j.jaapos.2011.10.010. [DOI] [PubMed] [Google Scholar]
  • 10.Kang KB, Wessel MM, Tong J, et al. Ultra-widefield imaging for the management of pediatric retinal diseases. J Pediatr Ophthalmol Strabismus. 2013;50(5):282–288. doi: 10.3928/01913913-20130528-04. [DOI] [PubMed] [Google Scholar]
  • 11.Tsui I, Franco-Cardenas V, Hubschman JP, Schwartz SD. Pediatric retinal conditions imaged by ultra wide field fluorescein angiography. Ophthalmic Surg Lasers Imaging Retina. 2013;44(1):59–67. doi: 10.3928/23258160-20121221-14. [DOI] [PubMed] [Google Scholar]
  • 12.Ng EY, Lanigan B, O'Keefe M. Fundus fluorescein angiography in the screening for and management of retinopathy of prematurity. J Pediatr Ophthalmol Strabismus. 2006;43(2):85–90. doi: 10.3928/0191-3913-20060301-07. [DOI] [PubMed] [Google Scholar]
  • 13.Azad R, Chandra P, Khan MA, Darswal A. Role of intravenous fluorescein angiography in early detection and regression of retinopathy of prematurity. J Pediatr Ophthalmol Strabismus. 2008;45(1):36–39. doi: 10.3928/01913913-20080101-03. [DOI] [PubMed] [Google Scholar]
  • 14.Zepeda-Romero LC, Oregon-Miranda AA, Lizarraga-Barron DS, et al. Early retinopathy of prematurity findings identified with fluorescein angiography. Graefes Arch Clin Exp Ophthalmol. 2013;251(9):2093–2097. doi: 10.1007/s00417-013-2321-8. [DOI] [PubMed] [Google Scholar]
  • 15.Purcaro V, Baldascino A, Papacci P, et al. Fluorescein angiography and retinal vascular development in premature infants. J Matern Fetal Neonatal Med. 2012;25(suppl 3):53–56. doi: 10.3109/14767058.2012.712313. [DOI] [PubMed] [Google Scholar]
  • 16.Cantolino SJ, O'Grady GE, Herrera JA, et al. Ophthalmoscopic monitoring of oxygen therapy in premature infants. Fluorescein angiography in acute retrolental fibroplasia. Am J Ophthalmol. 1971;72(2):322–331. doi: 10.1016/0002-9394(71)91301-8. [DOI] [PubMed] [Google Scholar]
  • 17.Flynn JT, Cassady J, Essner D, et al. Fluorescein angiography in retrolental fibroplasia: experience from 1969–1977. Ophthalmology. 1979;86(10):1700–1723. doi: 10.1016/s0161-6420(79)35329-5. [DOI] [PubMed] [Google Scholar]
  • 18.O'Grady GE, Flynn JT, Clarkson J, Clark RD. Retrolental fibroplasia: clinical, fluorescein angiographic and pathological correlation. Mod Probl Ophthalmol. 1974;12(0):144–151. [PubMed] [Google Scholar]
  • 19.Payne JW, Patz A. Fluorescein angiography in retrolental fibroplasia. Int Ophthalmol Clin. 1977;17(2):121–135. doi: 10.1097/00004397-197701720-00011. [DOI] [PubMed] [Google Scholar]
  • 20.Fung TH, Muqit MM, Mordant DJ, et al. Noncontact high-resolution ultra-wide-field oral fluorescein angiography in premature infants with retinopathy of prematurity. JAMA Ophthalmol. 2014;132(1):108–110. doi: 10.1001/jamaophthalmol.2013.6102. [DOI] [PubMed] [Google Scholar]
  • 21.Ells AL, Holmes JM, Astle WF, et al. Telemedicine approach to screening for severe retinopathy of prematurity: a pilot study. Ophthalmology. 2003;110(11):2113–2117. doi: 10.1016/S0161-6420(03)00831-5. [DOI] [PubMed] [Google Scholar]
  • 22.Chiang MF, Jiang L, Gelman R, et al. Interexpert agreement of plus disease diagnosis in retinopathy of prematurity. Arch Ophthalmol. 2007;125(7):875–880. doi: 10.1001/archopht.125.7.875. [DOI] [PubMed] [Google Scholar]
  • 23.Wagner RS. Fundus fluorescein angiography in retinopathy of prematurity. J Pediatr Ophthalmol Strabismus. 2006;43(2):78. doi: 10.3928/0191-3913-20060301-04. [DOI] [PubMed] [Google Scholar]
  • 24.Gilbert C. Retinopathy of prematurity: a global perspective of the epidemics, population of babies at risk and implications for control. Early Hum Dev. 2008;84(2):77–82. doi: 10.1016/j.earlhumdev.2007.11.009. [DOI] [PubMed] [Google Scholar]
  • 25.Richter GM, Williams SL, Starren J, et al. Telemedicine for retinopathy of prematurity diagnosis: evaluation and challenges. Surv Ophthalmol. 2009;54(6):671–685. doi: 10.1016/j.survophthal.2009.02.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Henaine-Berra A, Garcia-Aguirre G, Quiroz-Mercado H, Martinez-Castellanos MA. Retinal fluorescein angiographic changes following intravitreal anti-VEGF therapy. J AAPOS. 2014;18(2):120–123. doi: 10.1016/j.jaapos.2013.12.009. [DOI] [PubMed] [Google Scholar]
  • 27.Tahija SG, Hersetyati R, Lam GC, et al. Fluorescein angiographic observations of peripheral retinal vessel growth in infants after intravitreal injection of bevacizumab as sole therapy for zone I and posterior zone II retinopathy of prematurity. Br J Ophthalmol. 2014;98(4):507–512. doi: 10.1136/bjophthalmol-2013-304109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Lepore D, Quinn GE, Molle F, et al. Intravitreal Bevacizumab versus Laser Treatment in Type 1 Retinopathy of Prematurity: report on Fluorescein Angiographic Findings. Ophthalmology. 2014;121(11):2212–2219. doi: 10.1016/j.ophtha.2014.05.015. [DOI] [PubMed] [Google Scholar]
  • 29.Yokoi T, Hiraoka M, Miyamoto M, et al. Vascular abnormalities in aggressive posterior retinopathy of prematurity detected by fluorescein angiography. Ophthalmology. 2009;116(7):1377–1382. doi: 10.1016/j.ophtha.2009.01.038. [DOI] [PubMed] [Google Scholar]
  • 30.Ryan MC, Ostmo S, Jonas K, et al. Development and evaluation of reference standards for image-based telemedicine diagnosis and clinical research studies in ophthalmology. AMIA Annu Sym Proc. 2014:1902–1910. [PMC free article] [PubMed] [Google Scholar]
  • 31.Paulson EK, Harris JP, Jaffe TA, et al. Acute Appendicitis: Added Diagnostic Value of Coronal Reformations from Isotropic Voxels at Multi–Detector Row CT. Radiology. 2005;235(3):879–885. doi: 10.1148/radiol.2353041231. [DOI] [PubMed] [Google Scholar]
  • 32.Color photography vs fluorescein angiography in the detection of diabetic retinopathy in the diabetes control and complications trial. The Diabetes Control and Complications Trial Research Group. Arch Ophthalmol. 1987;105(10):1344–1351. doi: 10.1001/archopht.1987.01060100046022. [DOI] [PubMed] [Google Scholar]
  • 33.International Committee for the Classification of Retinopathy of Prematurity. The International Classification of Retinopathy of Prematurity revisited. Arch Ophthalmol. 2005;123(7):991–999. doi: 10.1001/archopht.123.7.991. [DOI] [PubMed] [Google Scholar]
  • 34.Guagliano R, Barilla D, Bertone C, et al. Fluorescein angiography-based diagnosis for retinopathy of prematurity: expert-non expert comparison. Eur J Ophthalmol. 2013;23(6):881–886. doi: 10.5301/ejo.5000319. [DOI] [PubMed] [Google Scholar]
  • 35.Chiang MF, Thyparampil PJ, Rabinowitz D. Interexpert agreement in the identification of macular location in infants at risk for retinopathy of prematurity. Arch Ophthalmol. 2010;128(9):1153–1159. doi: 10.1001/archophthalmol.2010.199. [DOI] [PubMed] [Google Scholar]
  • 36.Williams SL, Wang L, Kane SA, et al. Telemedical diagnosis of retinopathy of prematurity: accuracy of expert versus non-expert graders. Br J Ophthalmol. 2010;94(3):351–356. doi: 10.1136/bjo.2009.166348. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Kang KB, Orlin A, Lee TC, et al. The use of digital imaging in the identification of skip areas after laser treatment for retinopathy of prematurity and its implications for education and patient care. Retina. 2013;33(10):2162–2169. doi: 10.1097/IAE.0b013e31828e6969. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Kemper AR, Freedman SF, Wallace DK. Retinopathy of prematurity care: patterns of care and workforce analysis. J AAPOS. 2008;12(4):344–348. doi: 10.1016/j.jaapos.2008.02.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Myung JS, Chan RVP, Espiritu MJ, et al. Accuracy of retinopathy of prematurity image-based diagnosis by pediatric ophthalmology fellows: Implications for training. J AAPOS. 2011;15(6):573–578. doi: 10.1016/j.jaapos.2011.06.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Chan RVP, Williams SL, Yonekawa Y, et al. Accuracy of Retinopathy of Prematurity Diagnosis by Retinal Fellows. Retina. 2010;30(6):958–965. doi: 10.1097/IAE.0b013e3181c9696a. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Chiang MF. Image analysis for retinopathy of prematurity: where are we headed? J AAPOS. 2012;16(5):411–412. doi: 10.1016/j.jaapos.2012.08.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Leder HA, Campbell JP, Sepah YJ, et al. Ultra-wide-field retinal imaging in the management of non-infectious retinal vasculitis. Journal of ophthalmic inflammation and infection. 2013;3(1):30. doi: 10.1186/1869-5760-3-30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Campbell JP, Leder HA, Sepah YJ, et al. Wide-field Retinal Imaging in the Management of Noninfectious Posterior Uveitis. Am J Ophthalmol. 2012;154(5):908–911. doi: 10.1016/j.ajo.2012.05.019. [DOI] [PubMed] [Google Scholar]

RESOURCES