Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Feb 18.
Published in final edited form as: Curr Opin Ophthalmol. 2020 Sep;31(5):312–317. doi: 10.1097/ICU.0000000000000680

Artificial intelligence for retinopathy of prematurity

Rebekah H Gensure a, Michael F Chiang b, John P Campbell b
PMCID: PMC7891849  NIHMSID: NIHMS1666030  PMID: 32694266

Abstract

Purpose of review

In this article, we review the current state of artificial intelligence applications in retinopathy of prematurity (ROP) and provide insight on challenges as well as strategies for bringing these algorithms to the bedside.

Recent findings

In the past few years, there has been a dramatic shift from machine learning approaches based on feature extraction to ‘deep’ convolutional neural networks for artificial intelligence applications. Several artificial intelligence for ROP approaches have demonstrated adequate proof-of-concept performance in research studies. The next steps are to determine whether these algorithms are robust to variable clinical and technical parameters in practice. Integration of artificial intelligence into ROP screening and treatment is limited by generalizability of the algorithms to maintain performance on unseen data and integration of artificial intelligence technology into new or existing clinical workflows.

Summary

Real-world implementation of artificial intelligence for ROP diagnosis will require massive efforts targeted at developing standards for data acquisition, true external validation, and demonstration of feasibility. We must now focus on ethical, technical, clinical, regulatory, and financial considerations to bring this technology to the infant bedside to realize the promise offered by this technology to reduce preventable blindness from ROP.

Keywords: artificial intelligence, deep learning, machine learning, retinopathy of prematurity

INTRODUCTION

Childhood blindness due to retinopathy of prematurity (ROP) is a major ophthalmologic problem worldwide [1]. As healthcare systems have adapted in their ability to care for preterm infants of younger age and lower birth weight, the number of infants at risk of ROP continues to rise. Determining screening criteria means balancing the inherent tension between the number of babies who need to be examined and the risk of missing severe disease. The burden of screening is especially evident in higher income settings; for example, an average of 55 infants examined for every one infant treated in the United Kingdom [2]. The model of direct bedside examination for every at-risk baby may not be feasible in areas where adequately trained ophthalmologists are in short supply. Another challenge to ROP screening is that clinical diagnosis in ROP is subjective with high rates of interobserver variability [3], and there is evidence that this leads to real-world treatment differences [4].

Increasing use of fundus photography for documentation of ROP and in telemedicine programs has facilitated the implementation of computer-based image analysis (CBIA) in ROP. Computers have the advantage over humans in not being susceptible to fatigue and other biases that may affect assessment of ROP severity. Moreover, shifting toward CBIA methods may allow for technicians, other ancillary healthcare staff, or even neonatologists to perform screening photography rather than requiring ophthalmologist examinations each time. As part of a screening program, computer-assisted diagnosis may optimize the tradeoff between maximizing sensitivity of a screening exam and minimizing the time needed for the ophthalmologist. Recent advances in computer-assisted diagnosis have been made using artificial intelligence in multiple domains [5,6■]. In this article, we review the current state of artificial intelligence applications in ROP and provide insight on challenges as well as strategies for bringing these algorithms to the bedside.

APPROACHES TO COMPUTER-ASSISTED RETINOPATHY OF PREMATURITY DIAGNOSIS

Current diagnostic criteria for ROP were established by the International Classification of ROP published in 1984 and later revised in 2005 [7,8]. Subsequent clinical studies such as the Cryotherapy for ROP and Early Treatment for ROP (ETROP) studies established evidence-based criteria for disease screening and demonstrated the importance of plus disease, a measure of vascular dilation and tortuosity. Plus disease is defined as arterial tortuosity and venous dilation equal to or greater than a standard reference photograph selected by expert consensus in the 1980s [8]. An intermediate category called ‘preplus,’ defined as tortuosity and venous dilatation that is abnormal but less severe than the reference standard plus disease photograph, was added to the classification in 2005 [9].

Features of interest in retinopathy of prematurity

Plus disease is both important for diagnosis of treatment-requiring ROP based on ETROP and subjective with significant interobserver disagreement; thus, early efforts at CBIA in ROP targeted quantification of retinal vascular dilation and tortuosity as indicators of ROP.

Manual and semiautomated machine learning

Traditional machine-learning approaches involve the following steps: first, defining disease-specific features of interest (e.g., dilation, tortuosity, presence of a ridge, etc.); second, computation of features of interest using explicit algorithms (feature extraction); third, training a classifier to optimize performance given ground-truth labels to determine the relationship between features and diagnosis (training); and fourth, testing system performance on previously unseen data. System performance may be affected by any of these steps (e.g., feature that did not correlate with ground-truth, methodology that did not accurately quantify the feature, classifier that imperfectly learned from or overfit training data, or ground-truth labels that were too noisy or did not generalize).

Early CBIA systems for ROP diagnosis used manual and semiautomated machine-learning approaches to compute features from fundus photography based on vessel diameter and tortuosity. A variety of mathematical approaches to quantify vessel tortuosity have been utilized including length-to-chord [10,11], curvature-based [12,13], angle-based [14], or spatial frequency measures [15], but these do not always fit with clinical perception of tortuosity [16]. A major barrier to these early algorithms was accuracy of vessel segmentation, requiring time-consuming manual input to carefully delineate posterior pole vessels or utilizing morphological preprocessing for semiautomated vessel segmentation [10,17–20]. In general, these systems were either too cumbersome or did not have adequate diagnostic performance to be implemented clinically.

Deep learning in retinopathy of prematurity

The past 5 years have seen a dramatic shift from feature extraction machine-learning approaches to ‘deep’ convolutional neural networks (CNNs), aka deep learning. Unlike earlier machine-learning approaches, a deep learning classifier learns without being explicitly told what to focus on (i.e., without user-defined features) [21,22]. The earliest published application of a fully automated ROP detection system using deep learning was in 2016 by Worrall et al. [23]; their CNN-based system performed similarly to human graders in determining diseased ROP cases vs. healthy cases. Mulay et al. [24] built a regional CNN-based model to detect ridges for improved classification of Stage 2 ROP. Wang et al. [25■■] published an automated ROP screening application called DeepROP which involved two CNN classifiers: one to identify presence or absence of ROP features (Id-Net), and a second to grade ROP cases as minor or severe (Gr-Net). For classification of plus disease, the Imaging and Informatics in ROP (i-ROP) consortium built the i-ROP deep learning (i-ROP-DL) system using a two-CNN strategy with the first a U-net CNN for vessel segmentation and the second for classification [26■■]. Both the DeepROP and i-ROP-DL approaches are pretrained using the publicly available ImageNet database and demonstrate strong agreement with expert opinion as well as high performance. Recent work demonstrated the potential use of a deep learning-derived ROP severity score based on the i-ROP-DL classifier for disease screening [27■]. The ROP vascular severity score has demonstrated utility in monitoring disease progression [28■], monitoring disease regression after treatment [29■], and differentiating the pace of disease in aggressive-posterior ROP [30]. Employing the successful U-Net-based segmentation algorithm for better delineation of vessels followed by a traditional feature extraction and classification scheme, the i-ROP consortium showed that a hybrid system, i-ROP ASSIST, could achieve CNN-like performance while also allowing for explainable features [31]. The remainder of this article will focus on potential hurdles for implementation of these technologies into modern day practice.

CHALLENGES AND STRATEGIES FOR DEVELOPING CLINIC-READY ARTIFICIAL INTELLIGENCE

Although it seems apparent that artificial intelligence-assisted ROP screening could solve several existing problems, it is not self-evident when or how artificial intelligence might be incorporated into routine ROP care. Consider the case of optical coherence tomography (OCT). The OCT technology that began in the 1980s as a promising research tool was successfully commercialized and transformed into a critical part of ophthalmology clinical care, spanning across nearly all subspecialties, allowing us to better understand ophthalmic disease [32]. Widespread integration of OCT into daily ophthalmic practice took over a decade from the first demonstration of its clinical utility and reflects tremendous synergy between researchers, industry, regulatory bodies, and clinicians [33]. We can learn from OCT perils and pitfalls and adapt what worked in transitioning artificial intelligence ROP research from desktop bench to infant bedside.

Integration of artificial intelligence into ROP care is limited by several gaps in knowledge: first, do artificial intelligence algorithms work in the ‘real-world’, generalizing to unseen data from different cameras, populations, variations in image quality, and so on, with acceptable performance? Second, how would this technology integrate into new or existing clinical workflows? Ethical, medicolegal, and regulatory issues as well as public perception of artificial intelligence are also critical to consider, though outside the scope of this article, and we would direct the reader to several excellent reviews on these topics [22,34–38]. We discuss challenges for developing clinic-ready artificial intelligence for ROP here along with proposed strategies.

Generalizability

Medical advances that work in research studies often fail to demonstrate real world effectiveness [39]. For artificial intelligence, a troublesome implementation barrier is generalizability. Deep CNNs are highly sensitive to patterns in training data, not only relevant patterns, but also potentially confounding patterns such as image quality, pixel-level image acquisition variability, different fundus pigmentation, and so on. If the real ROP population being tested differs in these ways from the training population (e.g., a manufactured image dataset), the system may not perform as well. Now that several artificial intelligence in ROP approaches have demonstrated adequate proof-of-concept performance in research studies with curated image datasets from specific cameras and acquisition protocols [23,25■■,26■■,27■,28■,30], we need to study whether these algorithms are robust to variable clinical and technical parameters in practice.

There are both theoretical and practical ways images might vary in clinical practice compared with research studies. First, algorithms that perform well on one camera system may fail to perform similarly across different camera systems. With many different camera manufacturers each wanting part of the ROP market, the process of image standardization and regulatory approval becomes quite complicated, since algorithms will be approved only for cameras with available validation data. For diabetic retinopathy, the IDx-DR developers opted to require a specific camera, the Topcon NW400 nonmydriatic camera (Topcon Medical Systems, Oakland, New Jersey, USA), for image acquisition [40]. It remains to be seen how to tackle the issue of multiple camera vendors in ROP. Second, capturing images on a premature infant in the neonatal ICU is more challenging than imaging a cooperative adult in the outpatient clinic setting. Will real world image quality be good enough for use in artificial intelligence systems? Technical training or image quality prescreening may be required to trust the output of these artificial intelligence systems [41]. Third, fundus pigmentation and ROP phenotypes vary across ethnic and geographic boundaries. ROP in low-income and middle-income countries can present differently than in the United States and Europe [42] and will require independent validation. Similar to phase III/IV clinical trials for drug development, we anticipate two validation phases: first, highly regulated and protocolized image acquisition and evaluation designed to obtain regulatory approval for clinical use, and second, postmarketing evaluations to assess real-world effectiveness. These latter studies will be pivotal in demonstrating a model’s generalizability to gain acceptance by clinicians worldwide.

Clinical integration

Practical barriers in ROP care delivery must be addressed for artificial intelligence technology to be utilized widely. First, the artificial intelligence technology discussed here requires photo-documentation; however, currently most ROP screening occurs at the bedside via ophthalmoscopy without image acquisition. Reasons for favoring this screening approach are presumably multifactorial including camera expense, medicolegal concerns, and reimbursement. Thus, a workflow will need to be established where the economic realities of camera expense are factored in. Successful telemedicine models for ROP care delivery exist [43–45], but despite their apparent advantages [46], they remain only a small part of ROP care delivery worldwide.

Second, how might an ‘artificial intelligence for ROP’ device integrate into the clinical decision-making process? Is the technology meant to be a screening system (refer or not refer), diagnostic aid (providing suggested diagnosis of zone, stage, or plus disease), or part of a risk model to guide follow-up and/or treatment? [28■,29■,47,48]. Should the artificial intelligence be autonomous (working without supervision) or assistive (diagnostic aid to a clinician)? If assistive, should it aid the ophthalmologist, a nonphysician grader [49–52], a neonatologist? Will this workflow vary based on the resources of a given region? Each potential condition has different regulatory burdens and requires extensive validation. Finally, the expense of regulatory approval may be problematic for implementation especially in resource-poor areas where the need for screening is high but available funding to purchase additional hardware may be quite low. Thus, although the most useful system might be an autonomous cloud-based artificial intelligence screening program that works across multiple cameras, integration into a single camera as an assistive device would presumably be the lowest hurdle for regulatory approval.

FUTURE DIRECTIONS FOR ARTIFICIAL INTELLIGENCE IN RETINOPATHY OF PREMATURITY

The implementation challenges detailed above are clearly critical next steps for translational artificial intelligence research and should not be trivialized. Reserving those for the moment, we can envision a number of potential clinical workflows for artificial intelligence in ROP.

Artificial intelligence-assisted retinopathy of prematurity screening

Approaches that might work for artificial intelligence-assisted ROP screening include diagnosis of ‘stage’ in ROP images [24], diagnosis of preplus or worse disease [25■■,26■■], or quantitative assessment of vascular severity on a continuous scale [27■–29■,30]. As a screening tool, artificial intelligence assistance would increase the sensitivity of detecting severe ROP and reduce the chances of missed diagnosis. It would also add objectivity to a disease diagnosed on the basis of subjective clinical features that can be difficult to detect and agree upon. Artificial intelligence input could be implemented either at the point-of-care on a camera or in the cloud as part of a telemedicine system. In this model, the artificial intelligence system would provide a preliminary read (like an EKG-automated interpretation) that the clinician reader could accept or overrule.

Autonomous artificial intelligence retinopathy of prematurity screening

Several key considerations make the concept of autonomous real-time artificial intelligence reading of ROP images advantageous. First, since roughly 80% of exams in a typical screening population will have no or mild disease, the workload for clinical review would decrease by 80% [27■]. Second, in any telemedicine program, there is an inevitable delay in providing feedback, and there is inherent value in real-time diagnosis being provided to families and care teams when technicians and other members of the ROP screening team are available to provide education, facilitate referral, and so on.

Artificial intelligence-assisted management decisions

Evidence from multiple clinical trials has shown that clinicians do not always agree on the diagnosis of treatment-requiring ROP [4,53,54]. This disagreement is at least in part due to the subjective assessment of plus disease as well as cognitive biases that affect clinicians’ perceptions of disease severity (e.g., assuming that a high birth weight infant wouldn’t have plus disease). Automated and objective assessment of disease severity using artificial intelligence could provide a framework for new evidence-based guidelines for initiation of treatment, perhaps factoring in pre- and post-test probabilistic risk modeling including demographics, comorbidities, and so on. Moreover, quantitative assessment of ROP severity using deep learning may generate improved risk models that can identify ahead of time which patients are NOT likely to need treatment resulting in fewer recommended screening exams for low-risk babies, optimizing physician time and reducing stress on babies.

Artificial intelligence-assisted image acquisition

Thus far, we have focused on artificial intelligence for interpretation of images; however, there is growing appreciation that graphics-processing units, which are the backbone of deep learning, can also aid in image acquisition, providing real-time identification of landmarks, assisted autofocus, and improved image capture. In addition, artificial intelligence software could be trained to label right vs. left eye, field of view, and so on to aid in documentation and training of new artificial intelligence algorithms. In this article, we have focused primarily on fundus photography; however, the same analysis applies to new and emerging imaging techniques such as OCT angiography [55]. Deep learning is well suited to combining information from multiple imaging modalities and could aid in understanding how ROP severity affects underlying retinal structure.

CONCLUSION

Technological innovations have made artificial intelligence a hot topic in image-based medical specialties for good reason. In theory, artificial intelligence could improve efficiency, accuracy, and objectivity of ROP diagnosis. It could facilitate future clinical trials based on objective disease severity thresholds. It could further aid in improving image acquisition and development of new biometrics. But technology alone will not change clinical practice without careful validation of safe and effective use, both in research and real-world settings. Real-world implementation of artificial intelligence for ROP diagnosis will require massive efforts targeted at developing standards for data acquisition, true external validation, and demonstration of feasibility. Furthermore, transforming care delivery requires systemic changes in clinical practice, with numerous implementation challenges and opportunities including workable business models that justify the added expense of camera systems, information technology infrastructure, and research and development costs for artificial intelligence. In 2020, the novelty that artificial intelligence can diagnose disease in images is wearing off. We must now focus on ethical, technical, clinical, regulatory, and financial considerations to bring this technology to the infant bedside in order to realize the promise offered by this technology in reducing preventable blindness from ROP.

KEY POINTS.

  • Now that several artificial intelligence in ROP approaches have demonstrated adequate proof-of-concept performance in research studies, we need to determine whether these algorithms are robust to variable clinical and technical parameters in practice.

  • In the past few years, there has been a dramatic shift from machine learning approaches based on feature extraction to ‘deep’ convolutional neural networks.

  • Integration of artificial intelligence into ROP screening and treatment is limited by several important gaps in knowledge: first, generalizability of the algorithms to maintain performance on unseen data; and second, integration of artificial intelligence technology into new or existing clinical workflows.

  • Potential clinical workflows for artificial intelligence in ROP may include autonomous ROP screening, artificial intelligence-assisted ROP management decisions, and artificial intelligence-assisted image acquisition.

Financial support and sponsorship

The project was supported by grants R01EY19474, K12EY027720, and P30EY10572 from the National Institutes of Health (Bethesda, MD), by grant SCH-1622679 from the National Science Foundation (Arlington, VA), by unrestricted departmental funding and a Career Development Award (JPC) from Research to Prevent Blindness (New York, NY), and by research support from Genentech.

Footnotes

Conflicts of interest

M.F.C. is a Consultant for Novartis (Basel, Switzerland), and an initial member of InTeleretina (Honolulu, HI). J.P.C. and M.F.C. may receive royalties from related technology licensed to BostonAI (Boston, MA).

REFERENCES AND RECOMMENDED READING

Papers of particular interest, published within the annual period of review, have been highlighted as:

■ of special interest

■■ of outstanding interest

  • 1.Solebo AL, Teoh L, Rahi J. Epidemiology of blindness in children. Arch Dis Child 2017; 102:853–857. [DOI] [PubMed] [Google Scholar]
  • 2.Haines L, Fielder AR, Scrivener R, et al. Retinopathy of prematurity in the UK I: the organisation of services for screening and treatment. Eye (Lond) 2002; 16:33–38. [DOI] [PubMed] [Google Scholar]
  • 3.Campbell JP, Ryan MC, Lore E, et al. Diagnostic discrepancies in retinopathy of prematurity classification. Ophthalmology 2016; 123:1795–1801. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Stenson BJ, Tarnow-Mordi WO, Darlow BA, et al. , BOOST II United Kingdom Collaborative Group; BOOST II Australia Collaborative Group; BOOST II New Zealand Collaborative Group. Oxygen saturation and outcomes in preterm infants. N Engl J Med 2013; 368:2094–2104. [DOI] [PubMed] [Google Scholar]
  • 5.Ting DSW, Cheung CY, Lim G, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA 2017; 318: 2211–2223. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.■.De Fauw J, Ledsam JR, Romera-Paredes B, et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med 2018; 24:1342–1350. [DOI] [PubMed] [Google Scholar]; The authors describe a clinically heterogeneous optical coherence tomography dataset used to develop a deep-learning framework capable of making referral recommendations at or exceeding expert level; they also demonstrate the device-independent properties of their model’s segmentation framework, which shows sustained high performance on data captured from a different device.
  • 7.[No author listed]. An International Classification of Retinopathy of Prematurity. II. The classification of retinal detachment. The International Committee for the Classification of the Late Stages of Retinopathy of Prematurity. Arch Ophthalmol 1987; 105:906–912. [PubMed] [Google Scholar]
  • 8.[No author listed]. An International Classification of Retinopathy of Prematurity. The Committee for the Classification of Retinopathy of Prematurity. Arch Ophthalmol 1984; 102:1130–1134. [DOI] [PubMed] [Google Scholar]
  • 9.International Committee for the Classification of Retinopathy of Prematurity. The International Classification of Retinopathy of Prematurity revisited. Arch Ophthalmol 2005; 123:991–999. [DOI] [PubMed] [Google Scholar]
  • 10.Heneghan C, Flynn J, O’Keefe M, Cahill M. Characterization of changes in blood vessel width and tortuosity in retinopathy of prematurity using image analysis. Med Image Anal 2002; 6:407–429. [DOI] [PubMed] [Google Scholar]
  • 11.Grisan E, Foracchia M, Ruggeri A. A novel method for the automatic grading of retinal vessel tortuosity. IEEE Trans Med Imaging 2008; 27:310–319. [DOI] [PubMed] [Google Scholar]
  • 12.Onkaew D, Turior R, Uyyanonvara B, et al. “Automatic retinal vessel tortuosity measurement using curvature of improved chain code,” in InECCE 2011 – International Conference on Electrical, Control and Computer Engineering, 2011, pp. 183–186. [Google Scholar]
  • 13.Gelman R, Jiang L, Du YE, et al. Plus disease in retinopathy of prematurity: pilot study of computer-based and expert diagnosis. J AAPOS 2007; 11:532–540. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Oloumi F, Rangayyan RM, Casti P, Ells AL. Computer-aided diagnosis of plus disease via measurement of vessel thickness in retinal fundus images of preterm infants. Comput Biol Med 2015; 66:316–329. [DOI] [PubMed] [Google Scholar]
  • 15.Capowski JJ, Kylstra JA, Freedman SF. A numeric index based on spatial frequency for the tortuosity of retinal vessels and its application to plus disease in retinopathy of prematurity. Retina 1995; 15:490–500. [DOI] [PubMed] [Google Scholar]
  • 16.Nisha KL, Sreelekha G, Sathidevi PS, et al. A computer-aided diagnosis system for plus disease in retinopathy of prematurity with structure adaptive segmentation and vessel based features. Comput Med Imaging Graph 2019; 74:72–94. [DOI] [PubMed] [Google Scholar]
  • 17.Poletti E, Ruggeri A. “Segmentation of vessels through supervised classification in wide-field retina images of infants with retinopathy of prematurity,” in Proceedings – IEEE Symposium on Computer-Based Medical Systems, 2012. [Google Scholar]
  • 18.Wallace DK, Jomier J, Aylward SR, Landers MB. Computer-automated quantification of plus disease in retinopathy of prematurity. J AAPOS 2003; 7:126–130. [DOI] [PubMed] [Google Scholar]
  • 19.Wallace DK, Zhao Z, Freedman SF. A pilot study using ‘ROPtool’ to quantify plus disease in retinopathy of prematurity. J AAPOS 2007; 11:381–387. [DOI] [PubMed] [Google Scholar]
  • 20.Swanson C, Cocker KD, Parker KH, et al. Semiautomated computer analysis of vessel growth in preterm infants without and with ROP. Br J Ophthalmol 2003; 87:1474–1477. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Reid JE, Eaton E. Artificial intelligence for pediatric ophthalmology. Curr Opin Ophthamol 2019; 30:337–346. [DOI] [PubMed] [Google Scholar]
  • 22.Scruggs BA, Chan RVP, Kalpathy-Cramer J, et al. Artificial intelligence in retinopathy of prematurity diagnosis. Transl Vis Sci Technol 2020; 9:5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Worrall DE, Wilson CM, Brostow GJ. “Automated Retinopathy of Prematurity Case Detection with Convolutional Neural Networks,” 2016, pp. 68–76. [Google Scholar]
  • 24.Mulay S, Ram K, Sivaprakasam M, Vinekar A. “Early detection of retinopathy of prematurity stage using deep learning approach,” in Medical Imaging 2019: Computer-Aided Diagnosis, 2019, vol. 10950, p. 107. [Google Scholar]
  • 25.■■.Wang J, Ju R, Chen Y, et al. Automated retinopathy of prematurity screening using deep neural networks. EBioMedicine 2018; 35:361–368. [DOI] [PMC free article] [PubMed] [Google Scholar]; The study presents an automated retinopathy of prematurity (ROP) screening application with high accuracy in classifying absent, minor, or severe ROP; the DeepROP model involves two convolutional neural network (CNN) classifiers: one to identify presence or absence of ROP features (Id-Net), and a second to grade ROP cases as minor or severe (Gr-Net).
  • 26.■■.Brown JM, Campbell JP, Beers A, et al. Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol 2018; 136:803–810. [DOI] [PMC free article] [PubMed] [Google Scholar]; For classification of plus disease, the Imaging and Informatics in ROP (i-ROP) deep-learning (i-ROP-DL) system uses a two-CNN strategy with the first a U-net CNN for vessel segmentation and the second for classification. The i-ROP-DL model showed strong agreement with expert opinion as well as high performance.
  • 27.■.Redd TK, Campbell JP, Brown JM, et al. Evaluation of a deep learning image assessment system for detecting severe retinopathy of prematurity. Br J Ophthalmol 2019; 103:580–584. [DOI] [PMC free article] [PubMed] [Google Scholar]; The study evaluates the potential use of a deep-learning-derived ROP severity score based on the i-ROP-DL classifier for disease screening.
  • 28.■.Taylor S, Brown JM, Gupta K, et al. Monitoring disease progression with a quantitative severity scale for retinopathy of prematurity using deep learning. JAMA Ophthalmol 2019; 137:1022–1028. [DOI] [PMC free article] [PubMed] [Google Scholar]; The article demonstrates utility of the deep-learning-derived ROP vascular severity score in monitoring disease progression.
  • 29.■.Gupta K, Campbell JP, Taylor S, et al. A quantitative severity scale for retinopathy of prematurity using deep learning to monitor disease regression after treatment. JAMA Ophthalmol 2019; 137:1029–1036. [DOI] [PMC free article] [PubMed] [Google Scholar]; The article demonstrates further utility of the deep-learning-derived ROP vascular severity score in monitoring disease regression after treatment.
  • 30.Bellsmith KN, Brown J, Kim SJ, et al. Aggressive posterior retinopathy of prematurity: clinical and quantitative imaging features in a large North American Cohort. Ophthalmology 2020. S0161–6420(20)30129–9. doi: 10.1016/j.ophtha.2020.01.052. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Yildiz VM, Tian P, Yildiz I, et al. Plus disease in retinopathy of prematurity: convolutional neural network performance using a combined neural network and feature extraction approach. Transl Vis Sci Technol 2020; 9:10–110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Gabriele ML, Wollstein G, Ishikawa H, et al. Optical coherence tomography: history, current status, and laboratory work. Invest Ophthalmol Vis Sci 2011; 52:2425–2436. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Fujimoto J, Swanson E. The development, commercialization, and impact of optical coherence tomography. Invest Ophthalmol Vis Sci 2016; 57:OCT1–OCT13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Park SH, Do KH, Choi JI, et al. Principles for evaluating the clinical implementation of novel digital healthcare device. J Korean Med Assoc 2018; 61:765–775. [Google Scholar]
  • 35.Laï MC, Brian M, Mamzer MF. Perceptions of artificial intelligence in healthcare: findings from a qualitative survey study among actors in France. J Transl Med 2020; 18:14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Dzobo K, Adotey S, Thomford NE, Dzobo W. Integrating artificial and human intelligence: a partnership for responsible innovation in biomedical engineering and medicine. OMICS 2019; 24:247–263. [DOI] [PubMed] [Google Scholar]
  • 37.The Economist Group. AI, radiology and the future of work. Econ 2018; 2018:2016–2020. [Google Scholar]
  • 38.Castelvecchi D AI pioneer: the dangers of abuse are very real. Nature 2019; 2019:2019–2021. [DOI] [PubMed] [Google Scholar]
  • 39.Kent DM, Steyerberg E, Van Klaveren D. Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects. BMJ 2018; 363:k4245. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Abràmoff MD, Lavin PT, Birch M, et al. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med 2018; 1:39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Coyner AS, Swan R, Campbell JP, et al. Automated fundus image quality assessment in retinopathy of prematurity using deep convolutional neural networks. Ophthalmol Retina 2019; 3:444–450. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Quinn GE. Retinopathy of prematurity blindness worldwide: phenotypes in the third epidemic. Eye Brain 2016; 8:31–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Vinekar A, Gilbert C, Dogra M, et al. The KIDROP model of combining strategies for providing retinopathy of prematurity screening in underserved areas in India using wide-field imaging, tele-medicine, nonphysician graders and smart phone reporting. Indian J Ophthalmol 2014; 62:41–49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Zhang X, Wang Y, Ulrich JN, et al. Evaluation of retinopathy of prematurity incidence with telemedicine confirmation in Gansu, China: a pilot study. Ophthalmic Epidemiol 2017; 25:120–125. [DOI] [PubMed] [Google Scholar]
  • 45.Begley BA, Martin J, Tufty GT, Suh DW. Evaluation of a remote telemedicine screening system for severe retinopathy of prematurity. J Pediatr Ophthalmol Strabismus 2019; 56:157–161. [DOI] [PubMed] [Google Scholar]
  • 46.Brady CJ, D’Amico S, Campbell JP. Telemedicine for retinopathy of prematurity. Telemed J E Health 2020; 26:556–564. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Pivodic A, Hård AL, Löfqvist C, et al. Individual risk prediction for sight-threatening retinopathy of prematurity using birth characteristics. JAMA Ophthalmol 2019; 138:1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Ying GS. A prediction model for retinopathy of prematurity – is it ready for prime time? JAMA Ophthalmol 2020; 138:29–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Graham E Quinn, e-ROP Cooperative Group. Telemedicine approaches to evaluating acute-phase retinopathy of prematurity: study design. Ophthalmic Epidemiol 2014; 21:256–267. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Quinn GE, Ying GS, Repka MX, et al. Timely implementation of a retinopathy of prematurity telemedicine system. J AAPOS 2016; 20: 425–430.e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Cheng QE, Daniel E, Pan W, et al. Plus disease in telemedicine approaches to evaluating acute-phase ROP (e-ROP) study: characteristics, predictors, and accuracy of image grading. Ophthalmology 2019; 126:868–875. [DOI] [PubMed] [Google Scholar]
  • 52.Vinekar A, Mangalesh S, Jayadev C, et al. Impact of expansion of telemedicine screening for retinopathy of prematurity in India. Indian J Ophthalmol 2017; 65:390–395. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Reynolds JD, Dobson V, Quinn GE, et al. Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies. Arch Ophthalmol 2002; 120: 1470–1476. [DOI] [PubMed] [Google Scholar]
  • 54.Quinn GE, Ells A, Capone A, et al. Analysis of discrepancy between diagnostic clinical examination findings and corresponding evaluation of digital images in the telemedicine approaches to evaluating acute-phase retinopathy of prematurity study. JAMA Ophthalmol 2016; 134:1263–1270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Wang J, Hormel TT, Gao L, et al. Automated diagnosis and segmentation of choroidal neovascularization in OCT angiography using deep learning. Biomed Opt Express 2020; 11:927–944. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES