FARFUM-RoP, A dataset for computer-aided detection of Retinopathy of Prematurity

Abstract

Retinopathy of Prematurity (ROP) is a critical eye disorder affecting premature infants, characterized by abnormal blood vessel development in the retina. Plus Disease, indicating severe ROP progression, plays a pivotal role in diagnosis. Recent advancements in Artificial Intelligence (AI) have shown parity with or surpass human experts in ROP detection, especially Plus Disease. However, the success of AI systems depends on high-quality datasets, emphasizing the need for collaboration and data sharing among researchers. To address this challenge, the paper introduces a new public dataset, FARFUM-RoP (Farabi and Ferdowsi University of Mashhad’s ROP dataset), comprising 1533 ROP fundus images from 68 patients, annotated independently by five experienced childhood ophthalmologists as “Normal,” “Pre-Plus,” or “Plus.” Ethical principles and consent were meticulously followed during data collection. The paper presents the dataset structure, patient details, and expert labels.

Background & Summary

Retinopathy of Prematurity (ROP) is a potentially blinding eye disorder that primarily affects premature infants. This condition is characterized by abnormal blood vessel development in the retina of the eye in babies born before 31 weeks of gestation. A critical aspect of ROP is ‘Plus Disease’, a severe manifestation indicating rapid disease progression. This essay delves into the etiology, pathophysiology, clinical features, management, and implications of ROP and Plus Disease, drawing on a range of scholarly sources. ROP is fundamentally linked to prematurity and low birth weight. The underdevelopment of retinal blood vessels in premature infants is a key factor¹. After birth, exposure to high levels of oxygen in neonatal care can lead to the cessation of normal retinal vascular growth and subsequent neovascularization². Plus Disease is a marker of disease severity in ROP. It is characterized by dilatation and tortuosity of the posterior retinal blood vessels, indicating a significant increase in disease activity³. The diagnosis of ROP is clinical, based on retinal examinations. The International Classification of Retinopathy of Prematurity (ICROP) categorizes ROP based on zone, stage, and the presence of Plus Disease (International Committee for the Classification of Retinopathy of Prematurity, 2005). Plus Disease is identified by the comparison of retinal vascular appearance to a standard photograph⁴. Early detection and treatment are crucial. The management of ROP involves regular eye examinations by a pediatric ophthalmologist. Treatment options include laser therapy, cryotherapy, and anti-VEGF injections, primarily aimed at preventing or slowing the progression of retinal detachment⁵. Untreated, severe ROP can lead to complications such as retinal detachment and blindness. Early treatment, especially in cases of Plus Disease, can significantly improve outcomes. However, long-term follow-up is essential as these infants are at risk of developing other visual impairments^6,7.

The application of Artificial Intelligence (AI) in diagnosing ROP and Plus Disease represents a groundbreaking shift in pediatric ophthalmology. Leveraging advanced AI techniques, such as machine learning and deep learning, has the potential to enhance diagnostic accuracy, efficiency, and accessibility. Recent developments in AI involve creating sophisticated algorithms capable of analyzing retinal images to detect ROP and Plus Disease. These algorithms are trained on extensive datasets, enabling them to identify subtle indicators of the disease with high accuracy⁸. Recent studies have shown that AI can achieve parity with or exceed the diagnostic capabilities of human experts in ROP. AI’s ability to detect Plus Disease, with its complex vascular changes, has been notably enhanced, leading to more reliable and early detection⁹.

The advent of AI in the diagnosis of ROP represents a significant leap forward in pediatric ophthalmology. However, the efficacy and accuracy of these AI systems heavily rely on the availability and quality of datasets used in their training and validation. It encourages collaboration between researchers, practitioners, and developers if public datasets exist for this purpose. Sharing data enables the pooling of resources and expertise, leading to more innovative and effective AI solutions. This collaborative environment accelerates the pace of AI development and its subsequent impact on ROP diagnosis. However, it is still a challenging endeavor for researchers that they are unable to precisely evaluate and compare their results with each other. This problem comes from the fact that there is no suitable publicly available dataset for this application. To tackle this problem, some researchers have generated private datasets, which are not shared.

In this paper, we present a public dataset of ROP images, called FARFUM-RoP (standing for Farabi and Ferdowsi University of Mashhad’s ROP dataset), for computer-aided detection with research and education purposes. It includes ROP fundus images of 68 patients with 1533 images. Five professional childhood ophthalmologists independently annotated the images to “Normal”, “Pre-Plus” and “Plus”.

Comparing with other RoP Datasets

Currently, there is no publicly accessible image bank related to ROP (Retinopathy of Prematurity), and therefore FARFUM-RoP is being offered as a public resource. For comparison, here and in Table (1), several ROP-related image banks are presented, all of which are private and the details can only be derived from their respective publications. The only public dataset available in ROP field is the valuable dataset HVDROPDB¹⁰, which is the first public dataset for the segmentation of retinal structure in fundus images of preterm infants.

Table 1.

Comparing FARFUM-RoP with different ROP private datasets.

Name of Dataset	Gestation Age (week)	Birth Weight (g)	Image per Patient	Diseased Case Images	Not Diseased Case Images	Number of Labelers	Labels
DIAROP13		<2000	10	8090	9711	5	Disease/Not Disease
Zhang and et al.14				5731	6499		AP-ROP/Regular ROP/Normal
Lei and et al.15	<36.5	<2000	1~24	11168	11793	5	ROP/Normal
Li and et al.16				3041	8012	6	ROP/Normal
Mao and et al.17	26–36	740–3400		981	5180		Normal/Pre-plus/Plus
KIDROP18	<34	<2000		89	200	3	No-Plus/Plus
Li and et al.19	<37	<2000	6	3869	14437	2	Normal/Stages
FARFUM-RoP	<34	<2000	1~41	2085	915	5	Normal/Pre-plus/Plus, Diagnostic, Stage

Methods

Ethical considerations

The present study strictly adhered to the ethical principles outlined in the Declaration of Helsinki, ensuring the welfare and confidentiality of all participants. Approval for the research was obtained from the institutional review board of the Tehran University of Medical Sciences (Ethical approval number: IR.TUMS.FARABIH.REC.1400.073). Written informed consent was meticulously acquired from the parents of each participant, granting explicit permission for their infants to undergo imaging and participate in the study. Also, parental consent for the open publication of the data has been waived by Farabi Hospital Ethics Committee for the stated reasons below. To safeguard privacy, expert technicians anonymize all images before inclusion in the dataset, mitigating any risk of privacy breach.

1. Nature of Data: The retinal images collected do not include any personally identifiable information beyond the images themselves. All data is anonymized to ensure the privacy and confidentiality of the pediatric patients.

2. Usage and Storage: The images are stored in the image repository of Farabi Hospital and are to be used in ophthalmology networks and research projects. These images will not be associated with any identifiable information of the children.

3. Open Publication Waiver: Given the anonymized nature of the data and the potential significant contributions to scientific research and public health, the Ethics Committee has decided to waive the requirement for parental consent specifically for the open publication of this dataset. This decision is based on the understanding that the open publication will not compromise the privacy or safety of the patients involved.

4. Ethical Considerations: The committee acknowledges that open publication of this dataset is essential for the advancement of ophthalmological research. The ethical considerations have been balanced against the potential risks, and it is determined that the benefits of openly publishing this anonymized dataset far outweigh any potential risks.

5. Compliance with Policies: This decision aligns with the ethical standards and policies of Farabi Hospital and relevant international guidelines on human data research.

Image acquisition and selection

A database was created consisting of wide-angle posterior retinal images obtained from various preterm infants in Iran from April 2016 to May 2019. These infants met the published criteria for retinopathy of prematurity (ROP) screening examination, specifically having a birth weight of less than 2000 grams and a gestational age of less than 34 weeks. The images were acquired as part of routine clinical care in the Retinopathy of Prematurity Ward at Farabi Eye Hospital, affiliated with Tehran University of Medical Sciences in Tehran, Iran. Prior to image capture, a mydriatic eye drop, including diluted Phenylephrine and tropicamide drops, was administered to dilate the pupils. The eye drop was administered three times in each eye with a 10-minute interval between each administration. The baby’s feeding was discontinued following the third droplet. A total of 2 to 12 pictures were obtained for each eye. RetCam (Clarity Medical Systems, Pleasanton, CA) was used to capture posterior, temporal, superior, inferior, and nasal view pictures of each eye of the newborn.

The challenges faced in collecting retinal images of infants are caused by insufficient patient participation and their limited attention span throughout the image acquisition process. These problems all contribute to the creation of low-quality images with artifacts such as focusing issues, contrast inadequacies, motion blurring, and uneven illumination. The database images are high-quality specialist images that have been selected based on expert opinion from among numerous images taken from the patient. In order to solve image quality concerns, an expert methodically removed about 30 low-quality images from the dataset.

The result, a total bank consisting of 1533 images related to 68 patients was prepared to be labeled by expert doctors in the next step. Some examples of Dataset FARFUM-RoP are shown in Fig. 1.

Fig. 1.

Examples of Dataset FARFUM-RoP.

Dataset

Details

The real names of the patients have been anonymized and encoded as Patientnn, where nn is a number ranging from 01 to 68. The database consists of 68 compressed.rar files, each named corresponding to the patient’s code and containing several.jpg images with dimensions of 1200 by 1600 pixels. For each patient, there are between one to 41 images, averaging 23 images per patient. Figure 2 shows the distribution of patient’s images. Additionally, two XLSX files are included. The file named Dataset_ Details.xlsx contains patient information and consists of 68 rows and 4 columns. These columns are: id, which includes codes from Patient01 to Patient68; Patient.BirthWeight, indicating the patient’s birth weight in grams; Patient.GestationAge, showing the gestational age at birth in weeks; and finally, Patient.Gender, indicating the patient’s gender. Figure 3 shows the distribution of patient’s gestation age for our dataset. The other file, Dataset_Labels.xlsx, consists of 1533 rows and eight main columns. These columns are: id, including codes from Patient01 to Patient68; image_name, the names of the image files provided for the patients; five columns for the specialist labels; each of them consists of three sub-columns: retinopathy grade, stage, and diagnostic. Also, the last column presents the collective grade as Label, which specifies the disease label assigned by the five membered Medical Advisory Team. This column contains numbers ranging from 1 to 3, representing the conditions normal, preplus, and plus, respectively.

Fig. 2.

Distribution of patient’s images in FARFUM-RoP dataset.

Fig. 3.

Distribution of patient’s gestation age in FARFUM-RoP dataset.

Data Records

All the data records discussed in this paper are accessible via the Figshare Repository¹¹, providing an organized format for the research community. The dataset is structured, encompassing 68 individual patients, identified as Patient01 to Patient68.

The patient’s dataset is documented in two distinct XLSX files:

1. Dataset Details (Dataset_Details.xlsx): This XLSX file serves as a repository of information regarding the patients and includes birth weight, gestation age, and gender.

2. Dataset Labels Based on Expert Opinions (Dataset_Labels.xlsx): This XLSX file contains labels assigned to each patient based on their ROP fundus images. The images were reviewed by five individual specialists, denoted as A to E, who provided classifications in five different columns:

1. “Grade” with three conditions: “1 as normal,” “2 as preplus,” or “3 as plus”. “0 means that grade isn’t specified”.

2. “Diagnostic” with three conditions: “no treatment,” “revisit,” or “treatment.”

3. “Stage” with assigned stage values ranging from one to five. “?” means that the ophthalmologist was not sure about the diagnostic.

Additionally, all five specialists, comprising a Medical Advisory Team, collectively reviewed the images and assigned the classifications of “1 as normal,” “2 as preplus,” and “3 as plus”. The resulting consensus was recorded in the “Label” column of this file.

Technical Validation

A software was designed for image labelling, and it has been provided to five retina fellowship-trained experts. These experts are tasked with labelling images from a bank. Among them, one has over 15 years of experience, three have between 10 to 15 years of experience, and the fifth has less than 10 years of experience. All are renowned ophthalmology professors at universities in Iran.

The software features a user interface as shown in Fig. 4. Images are assigned to the specialists, along with relevant information about the patient’s weight. For every patient, a set of images, ranging from one to 41 (with an average of 22 images per patient), is curated and prepared. This software allows the specialist to initially review some of a patient’s images which are randomly selected. Therefore, the specialist will only be able to see images of the patient that are randomly selected from all the images and can see all these selected images before diagnosing the disease. So, Annotations were performed on individual images, not on entire examinations. After this comprehensive observation, the specialist is then tasked with conducting evaluations across three distinct areas:

1. Grade Assessment: Here, the expert must categorize the image into one of three possible conditions: normal, preplus, or plus. This classification is vital in understanding the patient’s current ocular state.

2. Diagnostic Decision: In this step, the specialist selects one of three potential courses of action based on their analysis: no treatment required, a recommendation for a revisit for further evaluation, or immediate treatment.

3. Stage Determination: The expert is tasked with assigning a stage to the condition based on International Committee for the Classification of Retinopathy of Prematurity (ICROP)¹², ranging from one to five. This staging helps in understanding the severity and progression of the ocular issue.

Fig. 4.

Labeling software user interface. With the help of this software, the tagger records his idea about the condition, diagnosis, risk of macular edema, and stage of the disease for each patient.

This process ensures a comprehensive evaluation of each image, leveraging the expertise of specialists with varied experiences.

Author contributions

M.A. performed this research as a part of his MSc thesis. H.-R.P. designed the study as the research supervisor. E.K.P., A.D.F., F.B., N.E., M.I.F. and F.R. participated in image analysis and drafted the manuscript.

Code availability

For this study no custom code was used.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41597-024-03897-7.