Skip to main content
PMC home page
JAMA Network logoLink to JAMA Network
. 2019 Jul 3;137(9):1029–1036. doi: 10.1001/jamaophthalmol.2019.2442

A Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning to Monitor Disease Regression After Treatment

Kishan Gupta 1, J Peter Campbell 1, Stanford Taylor 1, James M Brown 2, Susan Ostmo 1, R V Paul Chan 3, Jennifer Dy 4, Deniz Erdogmus 4, Stratis Ioannidis 4, Jayashree Kalpathy-Cramer 2,5, Sang J Kim 1,6, Michael F Chiang 1,7,✉, for the Imaging and Informatics in Retinopathy of Prematurity Consortium
PMCID: PMC6613298  PMID: 31268499

Key Points

Question

Can a quantitative measurement of retinopathy of prematurity severity be used over time to monitor disease regression after treatment?

Findings

In this cohort study of at-risk infants using data collected for the Imaging and Informatics in Retinopathy of Prematurity study, the quantitative retinopathy of prematurity vascular severity score developed using an automated deep learning–based plus disease classifier consistently reflected clinical disease posttreatment regression in 46 included eyes with laser or bevacizumab treatment.

Meaning

Tracking quantitative measurements of retinopathy of prematurity severity may be an effective method of following disease regression and identifying patients at risk for recurrence after retinopathy of prematurity treatment.

Abstract

Importance

Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but treatment failure and disease recurrence are important causes of adverse outcomes in patients with treatment-requiring ROP (TR-ROP).

Objectives

To apply an automated ROP vascular severity score obtained using a deep learning algorithm and to assess its utility for objectively monitoring ROP regression after treatment.

Design, Setting, and Participants

This retrospective cohort study used data from the Imaging and Informatics in ROP consortium, which comprises 9 tertiary referral centers in North America that screen high volumes of at-risk infants for ROP. Images of 5255 clinical eye examinations from 871 infants performed between July 2011 and December 2016 were assessed for eligibility in the present study. The disease course was assessed with time across the numerous examinations for patients with TR-ROP. Infants born prematurely meeting screening criteria for ROP who developed TR-ROP and who had images captured within 4 weeks before and after treatment as well as at the time of treatment were included.

Main Outcomes and Measures

The primary outcome was mean (SD) ROP vascular severity score before, at time of, and after treatment. A deep learning classifier was used to assign a continuous ROP vascular severity score, which ranged from 1 (normal) to 9 (most severe), at each examination. A secondary outcome was the difference in ROP vascular severity score among eyes treated with laser or the vascular endothelial growth factor antagonist bevacizumab. Differences between groups for both outcomes were assessed using unpaired 2-tailed t tests with Bonferroni correction.

Results

Of 5255 examined eyes, 91 developed TR-ROP, of which 46 eyes met the inclusion criteria based on the available images. The mean (SD) birth weight of those patients was 653 (185) g, with a mean (SD) gestational age of 24.9 (1.3) weeks. The mean (SD) ROP vascular severity scores significantly increased 2 weeks prior to treatment (4.19 [1.75]), peaked at treatment (7.43 [1.89]), and decreased for at least 2 weeks after treatment (4.00 [1.88]) (all P < .001). Eyes requiring retreatment with laser had higher ROP vascular severity scores at the time of initial treatment compared with eyes receiving a single treatment (P < .001).

Conclusions and Relevance

This quantitative ROP vascular severity score appears to consistently reflect clinical disease progression and posttreatment regression in eyes with TR-ROP. These study results may have implications for the monitoring of patients with ROP for treatment failure and disease recurrence and for determining the appropriate level of disease severity for primary treatment in eyes with aggressive disease.

This cohort study uses clinical images collected for the Imaging and Informatics in Retinopathy of Prematurity consortium to assess whether an automated retinopathy of prematurity vascular severity score obtained from a deep learning–based algorithm plus disease classifier can objectively monitor disease progression and regression after treatment among infants with treatment-requiring disease.

Introduction

Retinopathy of prematurity (ROP) remains a growing cause of visual impairment worldwide although most cases of blindness can be prevented with appropriate screening and timely treatment.1,2 Two international classification of ROP committees have established modern classification schema,3,4 and several National Institutes of Health–funded trials1,5,6 have defined the standard of care for ROP management. Owing to differences in primary prevention, screening guidelines vary by region,7,8 but treatment guidelines are consistent worldwide. More recently, treatment with vascular endothelial growth factor antagonists (anti-VEGF agents) has increased.6,9,10 However, there is wide variability in application of current treatment guidelines. An ROP diagnosis is subjective and qualitative, with high rates of interobserver diagnostic variability even in clinical trials.8,11,12,13,14,15,16,17,18,19 In particular, plus disease is the most important clinical feature defining treatment-requiring ROP and has been shown to be a highly variable diagnosis among examiners.8,11,12,13,14,16,17 Differences in clinical outcomes and in rates of disease recurrence may be partly caused by differences in ROP severity at the time of treatment.

Furthermore, although the natural history of ROP progression has been well described by the Cryotherapy for ROP (CRYO-ROP) and Early Treatment for ROP (ETROP) trials,1,5 to date, there is no standard nomenclature for classifying treated ROP or for describing disease regression patterns. There are no evidence-based guidelines for disease retreatment in patients with persistent, recurrent, or reactivated ROP. The lack of those standards and guidelines has become increasingly problematic as ROP treatment with anti-VEGF agents has become more common with the high rate of recurrence and the importance of long-term monitoring after anti-VEGF treatment.6,9 Finally, there appear to be significant differences in disease regression patterns after treatment with laser photocoagulation vs treatment with anti-VEGF agents.6,20,21

A quantitative approach to ROP classification and management would provide an important tool for studying posttreatment changes. Computer-based image analysis has been used to quantitatively evaluate plus disease in ROP.13,14,22,23,24,25 Members of our group have previously developed a fully automated algorithm for plus disease diagnosis on a 3-level scale (normal, preplus, plus) and have shown that it can classify disease with comparable or better accuracy than clinical experts.25 In a parallel study, our group has described a quantitative vascular severity scale on a continuous scale from 1 (normal) to 9 (severe) and has shown its utility for monitoring disease progression.26 In the present study, we assessed whether the quantitative ROP vascular severity scale could be used to track disease regression after treatment and to evaluate differences between treatments using laser photocoagulation vs an anti-VEGF agent.

Methods

This study was performed using data from the multicenter Imaging and Informatics in ROP (i-ROP) cohort. The i-ROP consortium comprises 9 tertiary referral centers that screen high volumes of at-risk infants for ROP. The present study was approved by the Institutional Review Board at the coordinating center (Oregon Health & Science University) and by each of the 9 participating institutions (Columbia University, University of Illinois at Chicago, William Beaumont Hospital, Children’s Hospital Los Angeles, Cedars-Sinai Medical Center, University of Miami, Asociacion para Evitar la Ceguera en Mexico). All institutions abided by the tenets of the Declaration of Helsinki,27 and written informed consent was obtained from the parents of all infants enrolled.

Data Set

Deidentified images from clinical examinations performed between July 2011 and December 2016 were assessed. All images were obtained using a commercially available camera (RetCam; Natus Medical Incorporated). Each study eye examination was classified (no ROP, mild ROP, type 2 ROP or preplus, or type 1 ROP) using a reference standard diagnosis that harmonized findings from clinical examination and image-based examination by multiple experts using previously described methods.28

Image Analysis

The i-ROP deep learning system was used to classify the probability of plus disease on a 3-level scale (normal, preplus, plus) for each image in the data set. An automated ROP vascular severity score was then assigned to each image, from 1 (normal retinal vasculature) to 9 (worst retinal vasculature, eg, severe plus disease) using methods previously published29 based on the probabilities of each disease category: (1 × probability of normal) + (5 × probability of preplus disease) + (9 × probability of plus disease).

All patients with treatment-requiring ROP in the i-ROP cohort study were eligible for inclusion in the present study. Infants were excluded if they did not have at least an imaging examination in the 4 weeks prior to treatment, at the time of treatment, and within 4 weeks after treatment.

Statistical Analysis

For all infants who developed treatment-requiring ROP, the vascular severity scores were analyzed in weekly cohorts in the 4 weeks prior to and following treatment, adjusted for time of treatment. The distribution of scores at each point before and after treatment was compared against the scores at the time of treatment using the Wilcoxon rank sum test.

Two separate subgroup analyses were performed. First, the primary analysis was conducted in 2 groups: eyes that were treated only with laser and eyes that were treated only with an anti-VEGF agent. Treatment with laser or anti-VEGF in the i-ROP cohort study was performed solely at investigator discretion. Second, eyes that required retreatment within 6 weeks were analyzed, and multivariable logistic regression was performed to identify the factors that were associated with the need for subsequent treatment (ROP vascular severity score at time of initial treatment, rate of disease progression, gestational age, birth weight, and postmenstrual age [PMA] at the time of initial treatment). Statistical analyses were performed using MatLab, version 2014A (MathWorks). Descriptive statistics, the t test, the Kruskal-Wallis test, the Wilcoxon rank sum test with Bonferroni correction, and 2-way analysis of variance (ANOVA) were used to assess an association between the ROP vascular severity score and progression. An α level of .05 was used with P < .05 denoting statistical significance.

Results

Description of Study Cohort

In total, 5255 eye examinations from 871 infants were assessed for eligibility in this study, and 91 eyes received treatment (5.4%). Of those 91 eyes, 46 met study eligibility criteria based on the availability of images before treatment, at the time of treatment, and after treatment. The mean (SD) birth weight of those patients was 653 (185) g, with a mean (SD) gestational age of 24.9 (1.3) weeks. The mean (SD) PMA at time of treatment was 37.4 (3.8) weeks. No patients included in the present study later required vitrectomy.

Figure 1 and Figure 2 present representative images from eyes that required treatment. Among the patients, 38 eyes (25 patients) were initially treated with laser, and 8 eyes (5 patients) were initially treated with bevacizumab. Of 46 eyes, 45 (98%) were diagnosed as having type 1 ROP by the examining clinician. One eye was diagnosed as having type 2 ROP but treated outside of published guidelines because the infant was undergoing general anesthesia for treatment of the fellow eye.30

Figure 1. Serial Wide-Angle Retinal Images and Corresponding Clinical Diagnosis From Deep Learning in an Infant Treated With Laser.

Figure 1.

Open in a new tab

The infant was born at 25 weeks 3 days gestational age and developed treatment-requiring disease. The automated retinopathy of prematurity severity scores are 5.91 (A), 8.01 (B), and 6.99 (C).

Figure 2. Serial Wide-Angle Retinal Images and Corresponding Clinical Diagnosis From Deep Learning in an Infant Treated With Intravitreal Bevacizumab.

Figure 2.

Open in a new tab

The infant was born at 24 weeks 5 days gestational age and developed treatment-requiring disease. The automated retinopathy of prematurity severity scores are 6.21 (A), 8.94 (B), and 6.09 (C).

ROP Vascular Severity Score Before and After Treatment

Figure 3 shows the mean (95% CI) ROP vascular severity scores for each point before and after treatment of 46 eyes. The mean (SD) ROP vascular severity score at the time of treatment was 7.43 (1.89). The mean (SD) ROP vascular severity scores significantly increased 2 weeks prior to treatment, peaked at treatment, and decreased for at least 2 weeks after treatment (all 3 P < .001 assessed using unpaired 2-tailed t tests with Bonferroni correction).

Figure 3. Quantitative Retinopathy of Prematurity (ROP) Vascular Severity Score Progression and Regression in a Cohort of Infants Who Developed Treatment-Requiring Disease.

Figure 3.

Open in a new tab

Mean scores are displayed relative to time of treatment (which is time 0). Number of clinical examinations (n) evaluated at each time is displayed. Error bars indicate 95% CIs.

Figure 4 shows the same data points by treatment modality (anti-VEGF vs laser). Patients receiving anti-VEGF therapy had higher mean (SD) ROP severity scores (8.79 [0.39]) compared with laser-treated eyes (7.15 [1.96]) at the time of treatment (P = .02). The mean change in ROP vascular severity score in the first week after treatment was higher in the bevacizumab-treated group, decreasing at a mean (SD) rate of −3.28 (1.51) vs −1.91 (1.74) in the laser-treated group (P = .04).

Figure 4. Quantitative Retinopathy of Prematurity (ROP) Vascular Severity Score Distribution in a Cohort of Infants Who Were Treated With Laser Photocoagulation vs Antivascular Endothelial Growth Factor (Anti-VEGF) Agents.

Figure 4.

Open in a new tab

Mean scores are displayed 4 weeks prior to, before, and after treatment. Error bars indicate 95% CIs.

Characteristics of Retreated Eyes

Of 8 eyes, 3 (38%) initially treated with bevacizumab were retreated with bevacizumab because of poor initial response and were deemed treatment failures with mean (SD) time to retreatment of 9 (7) days. No delayed recurrence (after initial disease regression) was seen within 6 weeks for the bevacizumab group. Of 38 eyes, 14 (37%) initially treated with laser required retreatment within 6 weeks. The mean (SD) time to retreatment in this group was 2.5 (2.7) weeks. The PMA at time of treatment did not vary between eyes that required retreatment (median [interquartile range] PMA, 37.0 [1.7] weeks; mean [SD] PMA, 38.9 [5.8] weeks) and those who were treated a single time (median [interquartile range] PMA, 37.0 [3.3] weeks; mean [SD] PMA, 36.4 [2.2] weeks; P = .47, Wilcoxon rank sum test).

Figure 5 displays the mean (95% CI) ROP severity scores for patients who required retreatment with laser within 6 weeks compared with those who did not. Multivariable logistic regression showed that the ROP vascular severity score at time of initial treatment (odds ratio [OR], 1.2; 95% CI, 1.04-1.41; P = .01), the birth weight (OR, 1.01, 95% CI, 1.00-1.01; P < .001), and the ROP vascular severity score progression rate after initial screening (OR, 2.84; 95% CI, 1.87-4.32; P < .001) were associated with requiring retreatment. Gestational age (OR, 1.11; 95% CI, 0.72-1.73; P = .63) and PMA at time of treatment (OR, 1.11; 95% CI, 0.96-1.30; P = .16) were not statistically significant. Eyes that eventually required retreatment had a mean (SD) ROP vascular severity score at the initial treatment of 8.47 (1.08) compared with a score of 6.48 (2.08) in eyes that did not eventually require retreatment (P < .001). The retreated eyes had significantly higher mean (SD) disease severity from treatment through 3 weeks (week 0, 8.47 [1.08], P < .01; week 1, 7.15 [2.43], P < .01; week 2, 6.16 [1.87], P = .02; week 3, 7.05 [2.22], P = .03) after initial treatment compared with mean (SD) for single-treated eyes (week 0, 6.49 [2.08]; week 1, 3.92 [2.49]; week 2, 3.23 [1.91]; week 3, 2.94 [2.72]). Based on the regression analysis results, each point increase in the ROP vascular severity score at the time of initial treatment increased the odds for retreatment 1.2 times. Each point per week increase in the ROP vascular severity score rate of progression at initial screening increased the odds for retreatment 2.8 times.

Figure 5. Quantitative Retinopathy of Prematurity (ROP) Vascular Severity Score Distribution in a Cohort of Infants With Treatment-Requiring Disease Who Received Laser Photocoagulation and Who Required Retreatment With Laser vs Those Who Required No Retreatment.

Figure 5.

Open in a new tab

Mean scores are displayed 4 weeks before and after treatment, and the number of clinical examinations is shown at each time. At treatment and 1, 2, and 3 weeks after treatment, retreated eyes had significantly higher ROP vascular severity scores than eyes that never required retreatment. Error bars indicate 95% CIs.

aP < .05.

Discussion

In the present study, we showed the feasibility of using a quantitative ROP severity score derived from a deep learning system to monitor disease regression in a cohort of infants with treatment-requiring disease. Our key study findings were as follows: (1) ROP quantitative vascular severity score trends over time were associated with clinically observed disease progression (before treatment) and regression (after treatment). (2) Infants treated with bevacizumab had a higher severity score at the time of treatment and more rapid disease regression than infants treated with laser photocoagulation. (3) Among patients treated with laser, those who eventually required retreatment had a higher ROP vascular severity score at the time of initial treatment and a more aggressive pace of disease than patients who did not eventually require retreatment.

The first key finding was that an ROP vascular severity score derived from a deep learning classifier mirrored the clinically observed disease progression that led to the decision to treat and the clinically observed disease regression after treatment. The natural history of ROP progression has been well described by the CRYO-ROP and ETROP studies.1,5 However, the terms recurrence, retreatment, and reactivation are often used interchangeably or inconsistently in the literature.10,20 Moreover, with 2 different types of treatments, it is becoming more apparent that regression patterns, disease recurrence risk, and long-term management needs are different between laser-treated and anti-VEGF–treated eyes.10,20,21,31,32,33 Many existing studies are confounded by differences in disease severity between infants treated with laser vs an anti-VEGF agent, the latter of which tends to be used for more posterior and aggressive disease.31,32,33 In addition, differences in lengths of follow-up time after treatment of anti-VEGF–treated eyes can recur much later, and variable published recurrence rates after laser treatment create difficulty interpreting the literature.10,20,31,32,33 The findings in the present study suggest that this quantitative ROP vascular severity score could be used to identify typical and atypical disease regression patterns and to potentially guide retreatment decision for treatment failure (lack of initial response to treatment) or recurrence (reprogression of disease after initial disease regression following treatment). Further validation in larger cohorts and in prospective studies will provide additional information about the clinical utility of this quantitative score.

Systematic differences in the level of severity at which treatment is initiated have been well reported by clinicians.11,12,14,15,16,17,18,28 These differences may be attributable to the subjectivity of plus disease or zone I diagnosis or to bias in the clinical factors relevant for the decision to treat.34 That these biases existed in all previous ROP clinical trials, and that, to date, there are no evidence-based objective criteria for identifying treatment-requiring ROP, leads to real-world differences among physicians regarding level of disease severity at treatment.11 Prospective evaluation of this automated ROP vascular severity score may be able to address these challenges in the future.

To our knowledge, although there is growing literature on disease retreatment and recurrence, little published work has systematically examined the typical pace of disease regression following ROP treatment.6,10,20,21,31,32,33,35,36 The second key finding of the present study was that the ROP vascular severity score showed promise as a tool to investigate the differences between therapeutic laser-induced and anti-VEGF treatment–induced regression. In our data, eyes treated with bevacizumab had more aggressive progression of disease before treatment and had higher ROP vascular severity scores at the time of treatment. Of course, this may be partially explained by the tendency of clinicians to select bevacizumab, rather than laser photocoagulation, as treatment for more aggressive disease, such as aggressive posterior ROP or zone I ROP.31,32,33 Furthermore, we showed more rapid disease regression after treatment with an anti-VEGF agent than after laser treatment (Figure 4). This has important implications for retreatment due to primary treatment failure as well as for monitoring disease recurrence after anti-VEGF treatment.

Early retreatment in anti-VEGF–treated eyes is often caused by treatment failure. The reasons this occurs are unknown but may be because of egress of medication out of the eye following intravitreal injection or differences in the VEGF concentration in eyes with more severe disease. Indeed, 3 of 8 eyes (38%) in this study were treated a second time with bevacizumab in the week following primary treatment. Delayed retreatment with an anti-VEGF agent is typically because of disease recurrence, that is, after initial disease regression, with subsequent recurrent neovascularization or development of plus disease.6,10,20 This late disease recurrence has been reported as long as 100 weeks’ PMA, and clinically significant sequelae of anti-VEGF treatment have been reported as long as 3 years after treatment.37,38 Following the BEAT-ROP (Bevacizumab Eliminates the Angiogenic Threat for Retinopathy of Prematurity) study that reported lower rates of retreatment in eyes treated with anti-VEGF agents through 54 weeks, most subsequent studies have found higher retreatment rates with anti-VEGF agents.10,20 These studies are potentially difficult to compare owing to differences in underlying disease severity (eg, some studies may have had more severe disease, on average, than BEAT-ROP) because early reports have highlighted higher retreatment rates in eyes with more aggressive and posterior disease.21,32,33 The higher severity score in eyes treated with bevacizumab, presumably associated with more aggressive posterior disease, leads to the question of whether earlier treatment of aggressive-posterior ROP (at a lower severity score) might lead to improved outcomes. This has not been tested. However, because a parallel study by some in our group has shown that eyes that eventually progress to treatment-requiring disease might be identified earlier using this quantitative ROP severity score, it may be possible to design a study to answer this question in the future.26,35

The third key finding is that among eyes treated with laser, those that required retreatment had a higher ROP vascular severity score and more aggressive pace of disease at the initial treatment. This was true even after adjusting for birth weight, gestational age, PMA, and rate of initial disease progression. There are 3 potential explanations for this finding: (1) those infants had more aggressive disease at baseline, which was more likely to recur; (2) those patients were observed more conservatively until progressing to a more severe level of disease before treatment was initiated, either intentionally or because of diagnostic differences among examiners;11,12 or (3) there was variability in treatment efficacy among different physicians.39 Although the present study was not powered to detect differences in retreatment rates among physicians, the retreatments were distributed among study sites, and all of the i-ROP centers relied on experienced ROP clinicians. We note that the laser retreatment rate in the present cohort (14 of 38 [37%]) was higher than rates typically reported in the literature. Because of the inclusion criteria that patients in this study required posttreatment imaging, there may be a selection bias toward more severe disease in this subcohort of i-ROP patients. Further prospective analysis using the ROP severity score may identify the optimal severity for intervention to minimize disease recurrence and treatment failure.

Limitations

There are a number of study limitations. First, the effect of excluding patients for lack of available imaging is unclear, but there are several potential reasons imaging may not have been obtained. For example, there was no requirement for capturing images at every examination although this was the standard practice of most clinicians in the study. In addition, images were generally obtained following the ophthalmoscopic examination, and therefore in cases of clinical instability, photography may have been deferred. Infants with mild ROP originally may have been examined less frequently than weekly. Moreover, outborn infants who were referred in for treatment without prereferral images were excluded. Except as noted above with regard to retreatment, we do not feel that excluding those infants introduced any systematic bias into the study, but analysis using larger prospective data sets will be valuable. Second, in terms of comparing treatment modalities, because there was no randomization to treatments and the number of patients treated with the anti-VEGF agent was small, we would caution generalizing these conclusions on regression pattern differences between treatments until larger data sets can be analyzed. Third, this study was not designed to analyze retreatments after 6 weeks, and thus we are unable to evaluate this technology for monitoring disease recurrence following anti-VEGF treatment. This will be an important area for future analysis. Fourth, this deep learning technology is currently available only for research studies; thus, the real-world effect of these findings is currently limited. Further work must be done to demonstrate the association of image quality, field of view, and camera system with ROP vascular severity score. Fifth, it will be necessary to perform similar analysis in cohorts outside of North America and among infants treated with anti-VEGF agents other than bevacizumab to assess the generalizability of the present findings. Such studies will be particularly important in low- and middle-income countries, where ROP is epidemic and oxygen regulation produces a more aggressive disease phenotype.

Conclusions

The present results suggested that an automated quantitative ROP vascular severity score obtained using a deep learning classifier may be used to monitor disease regression after treatment. This technology exhibits promise for monitoring typical and atypical disease regression patterns, identifying disease recurrence, and providing an objective metric to guide treatment and posttreatment decision making in the future.40 These findings may be generalizable to other ophthalmologic and medical diseases that are currently managed on the basis of subjective clinical characteristics.41

References

  • 1.Cryotherapy for Retinopathy of Prematurity Cooperative Group Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results. Arch Ophthalmol. 1988;106(4):471-479. doi: 10.1001/archopht.1988.01060130517027 [DOI] [PubMed] [Google Scholar]
  • 2.Gilbert C. Retinopathy of prematurity: a global perspective of the epidemics, population of babies at risk and implications for control. Early Hum Dev. 2008;84(2):77-82. doi: 10.1016/j.earlhumdev.2007.11.009 [DOI] [PubMed] [Google Scholar]
  • 3.The Committee for the Classification of Retinopathy of Prematurity An international classification of retinopathy of prematurity. Arch Ophthalmol. 1984;102(8):1130-1134. doi: 10.1001/archopht.1984.01040030908011 [DOI] [PubMed] [Google Scholar]
  • 4.International Committee for the Classification of Retinopathy of Prematurity The international classification of retinopathy of prematurity revisited. Arch Ophthalmol. 2005;123(7):991-999. doi: 10.1001/archopht.123.7.991 [DOI] [PubMed] [Google Scholar]
  • 5.Early Treatment For Retinopathy Of Prematurity Cooperative Group Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial. Arch Ophthalmol. 2003;121(12):1684-1694. doi: 10.1001/archopht.121.12.1684 [DOI] [PubMed] [Google Scholar]
  • 6.Mintz-Hittner HA, Kennedy KA, Chuang AZ; BEAT-ROP Cooperative Group . Efficacy of intravitreal bevacizumab for stage 3+ retinopathy of prematurity. N Engl J Med. 2011;364(7):603-615. doi: 10.1056/NEJMoa1007374 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Fierson WM; American Academy of Pediatrics Section on Ophthalmology; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus; American Association of Certified Orthoptists . Screening examination of premature infants for retinopathy of prematurity. Pediatrics. 2018;142(6):e20183061. doi: 10.1542/peds.2018-3061 [DOI] [PubMed] [Google Scholar]
  • 8.Reynolds JD, Dobson V, Quinn GE, et al. ; CRYO-ROP and LIGHT-ROP Cooperative Study Groups . Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies. Arch Ophthalmol. 2002;120(11):1470-1476. doi: 10.1001/archopht.120.11.1470 [DOI] [PubMed] [Google Scholar]
  • 9.Mintz-Hittner HA. Retinopathy of prematurity: intravitreal injections of bevacizumab: timing, technique, and outcomes. J AAPOS. 2016;20(6):478-480. doi: 10.1016/j.jaapos.2016.10.002 [DOI] [PubMed] [Google Scholar]
  • 10.Darwish D, Chee R-I, Patel SN, et al. . Anti-Vascular endothelial growth factor and the evolving management paradigm for retinopathy of prematurity. Asia Pac J Ophthalmol (Phila). 2018;7(3):136-144. doi: 10.22608/APO.201850 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Fleck BW, Williams C, Juszczak E, et al. ; BOOST II Retinal Image Digital Analysis (RIDA) Group . An international comparison of retinopathy of prematurity grading performance within the Benefits of Oxygen Saturation Targeting II trials. Eye (Lond). 2018;32(1):74-80. doi: 10.1038/eye.2017.150 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Campbell JP, Kalpathy-Cramer J, Erdogmus D, et al. ; Imaging and Informatics in Retinopathy of Prematurity Research Consortium . Plus disease in retinopathy of prematurity: a continuous spectrum of vascular abnormality as a basis of diagnostic variability. Ophthalmology. 2016;123(11):2338-2344. doi: 10.1016/j.ophtha.2016.07.026 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Kalpathy-Cramer J, Campbell JP, Erdogmus D, et al. ; Imaging and Informatics in Retinopathy of Prematurity Research Consortium . Plus disease in retinopathy of prematurity: improving diagnosis by ranking disease severity and using quantitative image analysis. Ophthalmology. 2016;123(11):2345-2351. doi: 10.1016/j.ophtha.2016.07.020 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Campbell JP, Ataer-Cansizoglu E, Bolon-Canedo V, et al. ; Imaging and Informatics in ROP (i-ROP) Research Consortium . Expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis. JAMA Ophthalmol. 2016;134(6):651-657. doi: 10.1001/jamaophthalmol.2016.0611 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Chiang MF, Thyparampil PJ, Rabinowitz D. Interexpert agreement in the identification of macular location in infants at risk for retinopathy of prematurity. Arch Ophthalmol. 2010;128(9):1153-1159. doi: 10.1001/archophthalmol.2010.199 [DOI] [PubMed] [Google Scholar]
  • 16.Chiang MF, Jiang L, Gelman R, Du YE, Flynn JT. Interexpert agreement of plus disease diagnosis in retinopathy of prematurity. Arch Ophthalmol. 2007;125(7):875-880. doi: 10.1001/archopht.125.7.875 [DOI] [PubMed] [Google Scholar]
  • 17.Campbell JP, Ryan MC, Lore E, et al. ; Imaging & Informatics in Retinopathy of Prematurity Research Consortium . Diagnostic discrepancies in retinopathy of prematurity classification. Ophthalmology. 2016;123(8):1795-1801. doi: 10.1016/j.ophtha.2016.04.035 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Quinn GE, Ells A, Capone A Jr, et al. ; e-ROP (Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity) Cooperative Group . Analysis of discrepancy between diagnostic clinical examination findings and corresponding evaluation of digital images in the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity study. JAMA Ophthalmol. 2016;134(11):1263-1270. doi: 10.1001/jamaophthalmol.2016.3502 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Scott KE, Kim DY, Wang L, et al. . Telemedical diagnosis of retinopathy of prematurity intraphysician agreement between ophthalmoscopic examination and image-based interpretation. Ophthalmology. 2008;115(7):1222-1228. doi: 10.1016/j.ophtha.2007.09.006 [DOI] [PubMed] [Google Scholar]
  • 20.Mintz-Hittner HA, Geloneck MM, Chuang AZ. Clinical management of recurrent retinopathy of prematurity after intravitreal bevacizumab monotherapy. Ophthalmology. 2016;123(9):1845-1855. doi: 10.1016/j.ophtha.2016.04.028 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Mueller B, Salchow DJ, Waffenschmidt E, et al. . Treatment of type I ROP with intravitreal bevacizumab or laser photocoagulation according to retinal zone. Br J Ophthalmol. 2017;101(3):365-370. doi: 10.1136/bjophthalmol-2016-308375 [DOI] [PubMed] [Google Scholar]
  • 22.Wittenberg LA, Jonsson NJ, Chan RVP, Chiang MF. Computer-based image analysis for plus disease diagnosis in retinopathy of prematurity. J Pediatr Ophthalmol Strabismus. 2012;49(1):11-19. doi: 10.3928/01913913-20110222-01 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Wallace DK. Computer-assisted quantification of vascular tortuosity in retinopathy of prematurity (an American Ophthalmological Society thesis). Trans Am Ophthalmol Soc. 2007;105:594-615. [PMC free article] [PubMed] [Google Scholar]
  • 24.Ataer-Cansizoglu E, Bolon-Canedo V, Campbell JP, et al. ; i-ROP Research Consortium . Computer-based image analysis for plus disease diagnosis in retinopathy of prematurity: performance of the “i-ROP” system and image features associated with expert diagnosis. Transl Vis Sci Technol. 2015;4(6):5. doi: 10.1167/tvst.4.6.5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Brown JM, Campbell JP, Beers A, et al. ; Imaging and Informatics in Retinopathy of Prematurity (i-ROP) Research Consortium . Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol. 2018;136(7):803-810. doi: 10.1001/jamaophthalmol.2018.1934 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Taylor S, Brown JM, Gupta K, et al. ; Imaging and Informatics in Retinopathy of Prematurity Consortium . Monitoring disease progression with a quantitative severity scale for retinopathy of prematurity using deep learning [published online July 3, 2019]. JAMA Ophthalmol. doi: 10.1001/jamaophthalmol.2019.2433 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.World Medical Association World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191-2194. doi: 10.1001/jama.2013.281053 [DOI] [PubMed] [Google Scholar]
  • 28.Ryan MC, Ostmo S, Jonas K, et al. . Development and evaluation of reference standards for image-based telemedicine diagnosis and clinical research studies in ophthalmology. AMIA Annu Symp Proc. 2014;2014:1902-1910. [PMC free article] [PubMed] [Google Scholar]
  • 29.Redd TK, Campbell JP, Brown JM, et al. ; Imaging and Informatics in Retinopathy of Prematurity (i-ROP) Research Consortium . Evaluation of a deep learning image assessment system for detecting severe retinopathy of prematurity. [published online November 23, 2018]. Br J Ophthalmol. 2018;bjophthalmol-2018-313156. doi: 10.1136/bjophthalmol-2018-313156 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Gupta MP, Chan RVP, Anzures R, Ostmo S, Jonas K, Chiang MF; Imaging & Informatics in ROP Research Consortium . Practice patterns in retinopathy of prematurity treatment for disease milder than recommended by guidelines. Am J Ophthalmol. 2016;163:1-10. doi: 10.1016/j.ajo.2015.12.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Walz JM, Bemme S, Reichl S, et al. ; Retina.net ROP-Register-Studiengruppe . Treated cases of retinopathy of prematurity in Germany: 5-year data from the Retina.net ROP registry [in German]. Ophthalmologe. 2018;115(6):476-488. doi: 10.1007/s00347-018-0701-5 [DOI] [PubMed] [Google Scholar]
  • 32.VanderVeen DK, Melia M, Yang MB, Hutchinson AK, Wilson LB, Lambert SR. Anti-vascular endothelial growth factor therapy for primary treatment of type 1 retinopathy of prematurity: a report by the American Academy of Ophthalmology. Ophthalmology. 2017;124(5):619-633. doi: 10.1016/j.ophtha.2016.12.025 [DOI] [PubMed] [Google Scholar]
  • 33.Sankar MJ, Sankar J, Chandra P. Anti-vascular endothelial growth factor (VEGF) drugs for treatment of retinopathy of prematurity. Cochrane Database Syst Rev. 2018;1(1):CD009734. doi: 10.1002/14651858.CD009734.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Hewing NJ, Kaufman DR, Chan RVP, Chiang MF. Plus disease in retinopathy of prematurity: qualitative analysis of diagnostic process by experts. JAMA Ophthalmol. 2013;131(8):1026-1032. doi: 10.1001/jamaophthalmol.2013.135 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Brown JM, Campbell JP, Beers A, et al. . Fully automated disease severity assessment and treatment monitoring in retinopathy of prematurity using deep learning In: Zhang J, Chen P-H, eds. Proceedings of SPIE 10579, Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. Bellingham, WA: SPIE; 2018, doi: 10.1117/12.2295942. [DOI] [Google Scholar]
  • 36.Hartnett ME. Pathophysiology and mechanisms of severe retinopathy of prematurity. Ophthalmology. 2015;122(1):200-210. doi: 10.1016/j.ophtha.2014.07.050 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Hajrasouliha AR, Garcia-Gonzales JM, Shapiro MJ, Yoon H, Blair MP. Reactivation of retinopathy of prematurity three years after treatment with bevacizumab. Ophthalmic Surg Lasers Imaging Retina. 2017;48(3):255-259. doi: 10.3928/23258160-20170301-10 [DOI] [PubMed] [Google Scholar]
  • 38.Snyder LL, Garcia-Gonzalez JM, Shapiro MJ, Blair MP. Very late reactivation of retinopathy of prematurity after monotherapy with intravitreal bevacizumab. Ophthalmic Surg Lasers Imaging Retina. 2016;47(3):280-283. doi: 10.3928/23258160-20160229-12 [DOI] [PubMed] [Google Scholar]
  • 39.Kang KB, Orlin A, Lee TC, Chiang MF, Chan RVP. The use of digital imaging in the identification of skip areas after laser treatment for retinopathy of prematurity and its implications for education and patient care. Retina. 2013;33(10):2162-2169. doi: 10.1097/IAE.0b013e31828e6969 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Ting DSW, Wu W-C, Toth C. Deep learning for retinopathy of prematurity screening. [published online November 23, 2018]. Br J Ophthalmol. 2018;bjophthalmol-2018-313290. doi: 10.1136/bjophthalmol-2018-313290 [DOI] [PubMed] [Google Scholar]
  • 41.Ting DSW, Pasquale LR, Peng L, et al. . Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol. 2019;103(2):167-175. doi: 10.1136/bjophthalmol-2018-313173 [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from JAMA Ophthalmology are provided here courtesy of American Medical Association

RESOURCES