Abstract
Background/Objectives
To determine the efficacy of the DIGIROP model in detecting treatment-requiring retinopathy of prematurity (TR-ROP) in a Portuguese cohort.
Subjects/Methods
Multicentre, retrospective cohort study of all consecutive preterm infants who underwent ROP screening from April 2012 to May 2019 in two neonatal units. Gestational age (GA), birth weight (BW) and sex were inserted in the DIGIROP platform. The optimal cut-off point to achieve 100% sensitivity was calculated. Area under the receiver operating characteristic curve (AUC) was calculated.
Results
Of the 431 infants who underwent ROP screening, 257 were eligible for DIGIROP analysis and 174 infants were excluded for having a GA outside the range 24–30 weeks imposed by the DIGIROP algorithm. Median GA was 29 weeks (range 24–30) and BW was 1060 g (range 408–2080). Twenty-tree infants (8.9%) developed TR-ROP. The highest risk obtained for TR-ROP was 0.5404 (95% CI 0.4343–0.6616) with a median achieved risk of 0.0938 (range 0.0016–0.5404). The optimal cut-off point to achieve 100% sensitivity on TR-ROP was 0.0016. The number of infants receiving ROP examinations would have been reduced from 257 to 187 infants (−27.2%) if the model was applied.
Conclusions
In our cohort, of 257 infants, the optimal cut-off point to achieve 100% sensitivity for TR-ROP was 0.0016 with moderate accuracy in the AUC (0.70). The number of infants requiring screening would have decreased 27.2% if the model was applied. It is essential that algorithms continue to be tested in different populations, especially in cohorts that include both younger and older GA infants.
Subject terms: Retinal diseases, Predictive markers
Introduction
Retinopathy of prematurity (ROP) affects the immature retina of premature infants and is a leading cause of lifelong visual disability worldwide [1]. Infants at risk for ROP are identified using recommended birth weight (BW) and gestational age at birth (GA) criteria. These BW and GA levels are set high in an attempt to ensure that all infants requiring treatment are examined [2–4]. Although currently recommended guidelines have high sensitivity for the detection of ROP, their specificity is low, as less than 10% of examined infants receive treatment, and only about half of examined infants develop any ROP [1, 5, 6].
Due to the low diagnostic ability of current screening guidelines, various statistical prediction models have been developed to try to identify infants at high risk for ROP allowing better allocation of resources for those at high risk while sparing those at low risk [7].
DIGIROP (https://www.digirop.com) [8] is a recently published online prediction model that uses only birth characteristics data (GA, BW and sex) to estimate the risk for treatment of sight-threatening ROP for infants born at 24–30 weeks’ GA. In the original cohort, which included 7609 Swedish infants, with a mean GA of 28.1 weeks and a mean BW of 1119 g, this algorithm obtained a sensitivity, specificity, positive predictive value and negative predictive value that were numerically non-inferior as those obtained from previously reported algorithms (Children’s Hospital of Philadelphia-ROP (CHOP-ROP), Omaha-ROP (OMA-ROP), weight, insulin-like growth factor 1, neonatal, ROP (WINROP) and Colorado-ROP (CO-ROP)) [8].
The primary aim of this study was to determine the efficacy of the online prediction model DIGIROP in detecting treatment-requiring retinopathy of prematurity (TR-ROP) in a Portuguese cohort.
Materials and methods
Study design
This was a retrospective cohort study of all preterm infants who underwent ROP screening from April 2012 to May 2019, in two neonatal intensive care units from two institutions in Portugal. The study was approved by the Ethical Review Boards of Centro Hospitalar de Lisboa Ocidental and Hospital Beatriz Ângelo, by the National Data Protection Authority (Comissão Nacional de Proteção de Dados) and by the Ethics Committee of the Nova Medical School. It was carried out in compliance with the tenets of the Declaration of Helsinki, in its latest version (Brazil, 2003).
Methods
Following the current Portuguese screening guidelines, all infants with GA ≤ 32 weeks, BW ≤ 1500 g or at higher risk of ROP determined by a neonatologist were eligible to be included. Infants without a known ROP outcome and those with any ocular diseases apart from ROP were excluded. The estimation of GA was based on foetal ultrasonography.
For ROP screening, an initial fundus examination using indirect ophthalmoscopy was performed at 32 weeks of postmenstrual age (PMA) or at 4 weeks of chronological age, whichever came later. The diagnosis of ROP and indication of treatment for ROP followed the International Classification of ROP Revisited [9] and the Early Treatment for ROP Study [10], respectively. The term severe ROP included both ‘Type 1 ROP’ (defined as zone I with plus disease with any stage or zone I stage 3 with no plus disease or zone II stage 2 or 3 with plus disease) and ‘Type 2 ROP’ (defined as zone I stage 1 or 2 without plus disease and zone II stage 3 without plus disease), both defined by the criteria of the Early Treatment for ROP study [10]. All infants diagnosed with Type 1 ROP underwent digital imaging recording and were subsequently treated.
Birth characteristics data, including sex, BW, GA, dates of retinal examinations, ROP stage and zone at every examination and any treatments performed for ROP, were collected from the clinical records.
DIGIROP model
The online prediction model DIGIROP (https://www.digirop.com) provides risk estimation and 95% confidence interval (CI) for sight-threatening ROP requiring treatment using only birth characteristics data for infants with GAs from 24 to 30 weeks.
GA, BW and sex were retrospectively inserted on the online application and the risk estimation with 95% CI was recorded.
Statistical analysis
The algorithm was applied to all infants with GA 24–30 weeks. The optimal cut-off point to achieve 100% sensitivity was calculated. Receiver operating characteristic curves, with calculation of the area under the curve (AUC) and 95% CI, were used to evaluate performance of the algorithm to identify TR-ROP. Data analyses were performed using SPSSv23 software (IBM®, USA).
Results
Baseline characteristics
Of the 431 infants who underwent ROP screening and had information on ROP outcome, 174 infants were excluded for having a GA outside 24–30-week limit imposed by the DIGIROP algorithm criteria. Of the 257 infants eligible for DIGIROP analysis, median GA was 29 weeks (range 24–30), median BW was 1060 g (range 408–2080) and 8.9% (n = 23) developed TR-ROP.
Among the 174 infants excluded from DIGIROP, median GA was 31 weeks (range 23–36), median BW was 1446 g (660–2670) and 1.3% (n = 2) developed TR-ROP, (Table 1). Common risk factors for the development of ROP among the cohorts are listed in Table 2.
Table 1.
Gestational age (GA), birth weight (BW) and gender.
| Median GA | Median BW | Male (%) | |
|---|---|---|---|
| (range) | (range) | ||
| Initial cohort (n = 431) | 30 | 1230 | 226 |
| (23–36) | (408–2670) | (52.4%) | |
| No ROP | 31 | 1350 | 174 |
| (n = 323; 74.9%) | (24–36) | (500–2670) | (53.9%) |
| ROP | 27 | 872 | 52 |
| (n = 108; 25.0%) | (23–31) | (408–1790) | (48.1%) |
| Type 1 ROP | 25 | 760 | 13 |
| (n = 25; 5.8%) | (23–31) | (427–1790) | (52.0%) |
| DIGIROP cohort (n = 257) | 29 | 1061 | 140 |
| (24–30) | (408–2080) | (54.5%) | |
| No ROP | 29 | 1160 | 90 |
| (n = 156; 60.7%) | (24–30) | (500–2080) | (57.6%) |
| ROP | 27 | 845 | 50 |
| (n = 101; 39.3%) | (24–30) | (408–1540) | (49.5%) |
| Type 1 ROP | 25 | 760 | 12 |
| (n = 23; 8.9%) | (24–30) | (427–1444) | (52.2%) |
| Patients excluded from the algorithm (n = 174) | 31 | 1446 | 86 |
| (23–36) | (660–2670) | (49.4%) | |
| No ROP | 31 | 1450 | 84 |
| (n = 167; 96.0%) | (31–36) | (660–2670) | (50.3%) |
| ROP | 31 | 1335 | 2 |
| (n = 7; 4.0%) | (23–31) | (660–1760) | (28.6%) |
| Type 1 ROP | 27 | 1210 | 1 |
| (n = 2; 1.1%) | (23–31) | (660–1760) | (50.0%) |
Table 2.
Risk factors for retinopathy of prematurity.
| BPD (range) |
Days of oxygen (range) |
Days of invasive ventilation (range) |
Early-onset sepsis (%) |
Late-onset sepsis (%) |
PIVH (%) |
PVL (%) |
Small for GA (%) |
NEC (%) |
Thrombocytopenia (<150.000/mml) (%) |
RBC transfusion (%) |
RBC transfusion first week (%) |
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Initial cohort (n = 431) |
144 (33.4) |
10 (0–231) |
0 (0–121) |
182 (42.2) | 174 (43.4) | 148 (34.3) | 62 (14.4) | 86 (19.9) | 23 (5.3) |
181 (42.0) |
187 (43.4) |
95 (22.0) |
|
No ROP (n = 323; 74.9%) |
66 (20.4) |
6 (0–226) |
0 (0–86) |
113 (35.0) | 92 (28.5) | 90 (27.9) | 34 (10.5) | 61 (19.0) | 11 (3.4) |
116 (35.9) |
97 (30.0) |
34 (10.5) |
|
ROP (n = 108; 25.0%) |
77 (71.3) |
46 (0–231) |
11 (0–121) |
69 (63.9) | 82 (76.0) | 58 (53.7) | 28 (26.0) | 25 (23.1) | 12 (11.1) |
65 (60.1) |
90 (83.3) |
61 (56.5) |
|
Type 1 ROP (n = 25; 5.8%) |
22 (88.0) |
86 (8–231) |
34 (0–121) |
17 (68.0) | 17 (68.0) | 17 (68.0) |
9 (36.0) |
7 (28.0) |
2 (8.0) |
16 (64.0) |
20 (80.0) |
18 (72.0) |
| DIGIROP cohort (n = 257) | 134 (52.1) |
30 (0–231) |
3 (0–121) |
143 (55.6) | 140 (54.4) | 115 (44.7) | 51 (19.8) |
45 (17.5) |
16 (6.2) |
127 (49.4) |
161 (62.6) |
91 (35.4) |
|
No ROP (n = 156; 60.7%) |
57 (36.5) |
19 (0–226) |
1 (0–86) |
76 (48.7) | 62 (39.7) | 59 (37.8) | 23 (14.7) |
22 (14.1) |
6 (3.8) |
64 (41.0) |
73 (46.8) |
31 (19.8) |
|
ROP (n = 101; 39.3%) |
77 (76.2) |
49 (0–231) |
12 (0–121) |
67 (66.3) | 78 (77.2) | 56 (55.4) | 28 (27.7) |
23 22.7) |
10 (9.9) |
15 (14.8) |
88 (87.1) |
60 (59.4) |
|
Type 1 ROP (n = 23; 8.9%) |
21 (91.3) |
86 (15–231) |
34 (0–121) |
16 (69.6) | 15 (65.2) | 16 (69.6) |
9 (39.1) |
7 (30.4) |
2 (8.7) |
15 (65.2) |
19 (82.6) |
19 (82.6) |
|
Cohort excluded from algorithm (n = 174) |
11 (6.3) |
3 (0–102) |
0 (0–102) |
39 (22.4) | 34 (19.5) | 33 (18.9) |
11 (6.3) |
41 (23.5) |
7 (4.0) |
54 (31.0) |
26 (14.9) |
4 (2.3) |
|
No ROP (n = 167; 96.0%) |
9 5.3) |
3 (0–56) |
0 (0–16) |
37 (22.1) | 30 (17.9) | 31 (18.5) |
11 (6.5) |
39 (23.4) |
5 (3.0) |
52 (31.1) |
24 (14.4) |
3 (1.8) |
|
ROP (n = 7; 4.0%) |
2 (28.6) |
19 (1–102) |
0 (0–59) |
2 (28.6) | 4 (57.1% | 2 (28.6) |
0 (0.0) |
2 (28.6) | 2 (28.6) |
2 (28.6) |
2 (28.6) |
1 (14.3) |
|
Type 1 ROP (n = 2; 1.1%) |
1 (50.0) |
55 (8–102) |
30 (0–59) |
1 (50.0) |
2 (100.0) | 1 (50.0) |
0 (0.0) |
0 (0.0) |
0 (0.0) |
1 (50.0) |
1 (50.0) |
1 (50.0) |
Algorithm outcome
After applying the DIGIROP model, the highest obtained risk estimation for sight-threatening TR-ROP was 0.5454 (95% CI 0.4343–0.6616) with a median achieved risk of 0.0938 (range 0.0016–0.5404). In the non-TR-ROP group, the highest obtained risk for TR-ROP given by the algorithm was 0.3655 (95% CI 0.2899–0.4519) with a median risk of 0.0039 (range 0.0001–0.3655).
The optimal cut-off point to achieve 100% sensitivity was 0.0016. The number of infants requiring screening for ROP would have decreased from 257 to 187 infants (−27.2%) if the model was applied.
AUC for TR-ROP was 0.70 (95% CI 0.57–0.83) with a sensitivity of 0.52 (95% CI 0.31–0.73) and a specificity of 0.86 (95% CI 0.76–0.93).
Discussion
Failure to detect TR-ROP can lead to irreversible blindness. Therefore, screening criteria have to have high sensitivity. Since only 4.3–30.4% of infants [11] will develop TR-ROP, many who do not require treatment will be screened. Screenings are not free of complications. There are several indicators that examinations are stressful and painful for the infants and pharmacological mydriasis may have potential serious adverse events. Moreover, ROP screening requires a skilled workforce available 52 weeks a year, which may not always be feasible [12, 13].
Given this context, several attempts have been made to improve screening in order to stratify risk and decision making, thus reducing unnecessary examinations and improving resource allocation.
Most of the proposed prediction algorithms use statistical modelling approaches that include BW, GA and other postnatal risk factors, such as oxygen exposure, sepsis or postnatal weight gain [7] that need to be collected over time. For instance, the weight in the WINROP model is inserted up to 36-week PMA [14, 15], whilst for the Growth and Retinopathy of Prematurity (G-ROP) modified screening criteria it is measured up to postnatal day 39 [16]. The need for continuous data input can hamper the use of these models namely when babies are transferred between neonatal units.
Recently, Pivodic et al. [8] developed and validated an individual risk prediction model (DIGIROP) for identifying TR-ROP using solely birth characteristics (BW, GA and sex), without the need of any other postnatal factors. The prediction model was developed based on 7286 Swedish infants and had an AUC of 0.90. After external validation in new cohorts in Sweden (n = 323), United States (n = 1535) and Europe (n = 354), the AUC remained high (0.94, 0.87 and 0.90, respectively) [8]. In addition, DIGIROP obtained a sensitivity and specificity that were numerically non-inferior to those obtained from previously reported algorithms requiring more complex postnatal data (CHOP-ROP, OMA-ROP, WINROP and CO-ROP) [8].
Our goal was to determine the ability of the DIGIROP model in detecting TR-ROP in a Portuguese cohort.
In our cohort, the highest risk obtained in the TR-ROP population sample was 0.5404 vs. 0.3655 in the non-TR-ROP population sample, with a median achieved risk of 0.1174 in TR-ROP vs. 0.0039 in non-TR-ROP. The AUC for TR-ROP was numerically lower than the one reported by the DIGIROP group (0.70 vs. 0.90), indicating only moderate accuracy. There is no information regarding race/ethnicity in the development model. However, the AUC by race/ethnicity, as reported for the US cohort, varied from 0.79 for Hispanics to 0.90 for blacks [17]. It is possible that differences in race/ethnicity might partially explain our results.
In our cohort, the optimal cut-off point to achieve 100% sensitivity was 0.0016. Even though not discussed in detail in their manuscript, the cut-off values for the DIGIROP model selected based on sensitivity obtained applying the published cut-off values from the CHOP-ROP, OMA-ROP, WINROP and CO-ROP models varied between 0.0076 and 0.02 [7]. The apparent different performance of DIGIROP in our cohort might be due to differences between patient populations.
The Total SWEDROP Cohort, Validation US Group and Validation European Group, from which de DIGIROP was initially validated, had a median GA of 28.6, 27.9 and 28.1 and a median BW of 1110, 980 and 990 g, respectively. The present cohort had numerically slightly different BW and GA (median GA was 29 weeks and median BW 1060 g).
Prediction models cannot be used in new settings without validation studies to prove that the model is generalisable and performs well with other patients and settings [18]. Since no prediction model performs well universally, research on developing and validating robust ROP prediction models for clinical use continues to be of interest. Our findings show that even algorithms that were validated in large cohorts (>7000 infants) can perform differently when applied in a different setting.
This is not the first time that an algorithm underperforms when tested on a new cohort. ROP risk profiles can vary widely from country to country, and even from city to city and unit to unit [19]. The WINROP model demonstrated 100% sensitivity (95% CI, 90–100%) for severe ROP in its original cohort but subsequent validation studies demonstrated consistently lower sensitivities ranging from 98.6% (95% CI, 95–99.8%) in a multicentre US and Canadian cohort, to 62% (95% CI, 55.9–69.9%) in Spain [20–26].
One of the proposed explanations for this phenomenon is that the mechanism for ROP development and progression may differ between high-income countries and low-income countries. In highly developed neonatal care settings, low BW and early GA are significant risk factors for ROP, and the addition of slow weight gain, as a surrogate for low serum IGF-1, helps to increase the sensitivity for identifying ROP. In countries with developing neonatal care systems, infants with significantly higher BW and GA can develop severe ROP, primarily due to excessive supplemental oxygen administration [6]. Hyperoxia inhibits VEGF activity and destroys immature retinal vasculature, causing a true ‘oxygen-induced retinopathy’ type of ROP [7]. Moreover, older GA infants have higher levels of endogenous IGF-1 production, suggesting that low serum IGF-1 is not a factor in limiting VEGF activity, and slow weight gain is not a predictive factor for this variant of ROP [7].
Portugal has a well-developed neonatal care, with a survival threshold of 25 weeks [27]. According to the 2013 Portuguese Very Low Birth Weight Infant Registry, the overall survival rate was 89%. When analysed by GA intervals, survival rates were 95.3% for 28 and 31 weeks, 77.6% for 25–27 weeks and 42.1% and 16.7% for 23 and 24 weeks, respectively [27]. Early and late neonatal mortality rates at and after 22 weeks of gestation in 2015 in Portugal, when compared to Sweden, were 2.1 and 1.5 per 1000 live births, respectively [28]. In comparison, the total preterm-related infant mortality rate was 2.01 per 1000 live births in 2016 in the United States [29].
In Portugal, all infants under 32 weeks are screened. Due to the GA limits of DIGIROP, 174 infants were not able to be inserted in the algorithm and, of these, 2 (1.3%) developed TR-ROP. In the recently developed G-ROP model [16], validated in 7483 infants in North America (with BW < 1501 g, GA < 32 weeks or a poor postnatal course), 1440 infants (19.2%) had a GA of 31 weeks or more. This ability to include all screened infants is an advantage comparatively to DIGIROP. Most of the older GA babies will not develop TR-ROP but some do require treatment. It is imperative to develop prediction models that correctly identify this small subset of infants at high risk. Pivodic et al. [8] did not compare DIGIROP with G-ROP in their article.
Another possible explanation for the different performance of DIGIROP in our cohort is that although the sole use of birth characteristics makes this model appealing and easy to use, the exclusion of postnatal factors might also have led to the non-identification of possible TR-ROP. It is widely recognised that postnatal factors play an important role in the development of severe ROP [7].
There are some limitations to consider in this study, namely its retrospective nature, the inclusion of only two institutions from the same geographical area and the relatively small sample size. Also, the collection spanned 2012–2019 as such we cannot fully account for changes in medical practice during this period. However, any prediction model must be validated in novel populations, and testing this new algorithm’s applicability in the Portuguese population is important.
In conclusion, in our cohort, the optimal cut-off point to achieve 100% sensitivity was 0.0016, which would have led to a decrease of 27.2% of screened infants. It is essential that algorithms continue to be tested in different populations, especially in cohorts that include both younger and older GA infants.
Summary
What was known before
Retinopathy of prematurity (ROP) is a leading cause of preventable blindness worldwide.
Prediction models can potentially help to stratify risk/decision, while reducing unnecessary examinations and improving resource allocation.
Prediction models cannot be used in new settings without validation studies.
What this study adds
First time DIGIROP has been applied after its original publication
The optimal cut-off point to achieve 100% sensitivity in our cohort was 0.0016 vs. 0.0076–0.02 in the original publication. AUC for TR-ROP revealed moderate accuracy.
The number of infants requiring screening for ROP would have decreased from 257 to 187 infants (−27.2%) if the model was applied.
It is essential that algorithms continue to be tested in different populations, especially in cohorts that include both younger and older GA infants.
Acknowledgements
The authors would like to express their gratitude to all participants who have agreed to collaborate in this study. The authors would also like to thank Mark Afonso for his expert help with the English language editing.
Compliance with ethical standards
Conflict of interest
The authors declare no competing interests.
Consent to participate
Parents/guardians have given their written informed consent.
Ethics approval
This research complies with the guidelines for human studies and was conducted ethically in accordance with the World Medical Association Declaration of Helsinki. The study protocol was approved by the relevant ethics committees (Ethical Review Board of Centro Hospitalar de Lisboa Ocidental reference number 172/CES – 2015; Ethical Review Board of Hospital Beatriz Ângelo reference number 1944/2017_MJHNO; National Data Protection Authority (Comissão Nacional de Proteção de Dados (CNPD)) reference number 9377/2016 and the Ethics Research Committee NMS|FCM-UNL (CEFCM) reference number 28/2016/CEFCM).
Footnotes
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Blencowe H, Lawn JE, Vazquez T, Fielder A, Gilbert C. Preterm-associated visual impairment and estimates of retinopathy of prematurity at regional and global levels for 2010. Pediatr Res. 2013;74:35–49. doi: 10.1038/pr.2013.205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Mora JS, Waite C, Gilbert CE, Breidenstein B, Sloper JJ. A worldwide survey of retinopathy of prematurity screening. Br J Ophthalmol. 2018;102:9–13. doi: 10.1136/bjophthalmol-2017-310709. [DOI] [PubMed] [Google Scholar]
- 3.Henriques G, Brito C, Teixeira S. Consenso Clínico: Retinopatia da prematuridade. Seção de Neonatologia, Sociedade Portuguesa de Pediatria 2014:1-12.
- 4.Fierson WM. American Academy of Pediatrics Section on Ophthalmology; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus; American Association of Certified Orthoptists. Screening examination of premature infants for retinopathy of prematurity. Pediatrics. 2018;142:e20183061. doi: 10.1542/peds.2018-3061. [DOI] [PubMed] [Google Scholar]
- 5.Norman M, Hellström A, Hallberg B, Wallin A, Gustafson P, Tornqvist K, et al. Prevalence of severe visual disability among preterm children with retinopathy of prematurity and association with adherence to best practice guidelines. JAMA Netw Open. 2019;2:e186801. doi: 10.1001/jamanetworkopen.2018.6801. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Binenbaum G, Tomlinson LA, de Alba Campomanes AG, Bell EF, Donohue P, Morrison D, et al. Validation of the Postnatal Growth and Retinopathy of Prematurity Screening Criteria. JAMA Ophthalmol. 2019;138:31–7. 10.1001/jamaophthalmol.2019.4517. Epub ahead of print. Erratum in: JAMA Ophthalmol. 2020;138:105. [DOI] [PMC free article] [PubMed]
- 7.Lin L, Binenbaum G. Postnatal weight gain and retinopathy of prematurity. Semin Perinatol. 2019;43:352–9. doi: 10.1053/j.semperi.2019.05.008. [DOI] [PubMed] [Google Scholar]
- 8.Pivodic A, Hård AL, Löfqvist C, Smith LEH, Wu C, Bründer MC. Individual Risk Prediction for Sight-Threatening Retinopathy of Prematurity Using Birth Characteristics. JAMA Ophthalmol. 2019;138:1–9. doi: 10.1001/jamaophthalmol.2019.4502. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.International Committee for the Classification of Retinopathy of Prematurity. The international classification of retinopathy of prematurity revisited. Arch Ophthalmol. 2005;123:991–9. doi: 10.1001/archopht.123.7.991. [DOI] [PubMed] [Google Scholar]
- 10.Early Treatment For Retinopathy Of Prematurity Cooperative Group. Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial. Arch Ophthalmol. 2003;121:1684–94. doi: 10.1001/archopht.121.12.1684. [DOI] [PubMed] [Google Scholar]
- 11.Darlow BA, Lui K, Kusuda S, Reichman B, Hakansson S, Bassler D, et al. International variations and trends in the treatment for retinopathy of prematurity. Br J Ophthalmol. 2017;101:1399–404. doi: 10.1136/bjophthalmol-2016-310041. [DOI] [PubMed] [Google Scholar]
- 12.Jiang JB, Zhang ZW, Zhang JW, Wang YL, Nie C, Luo XQ. Systemic changes and adverse effects induced by retinopathy of prematurity screening. Int J Ophthalmol. 2016;9:1148–55. doi: 10.18240/ijo.2016.08.11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Laws DE, Morton C, Weindling M, Clark D. Systemic effects of screening for retinopathy of prematurity. Br J Ophthalmol. 1996;80:425–8. doi: 10.1136/bjo.80.5.425. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lofqvist C, Andersson E, Sigurdsson J, Engstrom E, Hard AL, Niklasson A, et al. Longitudinal postnatal weight and insulin-like growth factor I measurements in the prediction of retinopathy of prematurity. Arch Ophthalmol. 2006;124:1711–8. doi: 10.1001/archopht.124.12.1711. [DOI] [PubMed] [Google Scholar]
- 15.Lofqvist C, Hansen-Pupp I, Andersson E, Holm K, Smith LE, Ley D, et al. Validation of a new retinopathy of prematurity screening method monitoring longitudinal postnatal weight and insulinlike growth factor I. Arch Ophthalmol. 2009;127:622–7. doi: 10.1001/archophthalmol.2009.69. [DOI] [PubMed] [Google Scholar]
- 16.Binenbaum G, Bell EF, Donohue P, Quinn G, Shaffer J, Tomlinson LA, et al. Development of modified screening criteria for retinopathy of prematurity: primary results from the postnatal Growth and Retinopathy of Prematurity study. JAMA Ophthalmol. 2018;136:1034–40. doi: 10.1001/jamaophthalmol.2018.2753. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Zackula RE, Raghuveer TS. Prediction of severe retinopathy of prematurity in 24-30 weeks gestation infants using birth characteristics. J Perinatol. 2021;41:351–5. [DOI] [PubMed]
- 18.Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ. 2009;338:b604. doi: 10.1136/bmj.b604. [DOI] [PubMed] [Google Scholar]
- 19.Zin AA, Moreira ME, Bunce C, Darlow BA, Gilbert CE. Retinopathy of prematurity in 7 neonatal units in Rio de Janeiro: screening criteria and workload implications. Pediatrics. 2010;126:e410–7. doi: 10.1542/peds.2010-0090. [DOI] [PubMed] [Google Scholar]
- 20.Choi JH, Lofqvist C, Hellstrom A, Heo H. Efficacy of the screening algorithm WINROP in a Korean population of preterm infants. JAMA Ophthalmol. 2013;131:62–6. doi: 10.1001/jamaophthalmol.2013.566. [DOI] [PubMed] [Google Scholar]
- 21.Piyasena C, Dhaliwal C, Russell H, Hellstrom A, Lofqvist C, Stenson BJ, et al. Prediction of severe retinopathy of prematurity using the WINROP algorithm in a birth cohort in South East Scotland. Arch Dis Child Fetal Neonatal Ed. 2014;99:F29–33. doi: 10.1136/archdischild-2013-304101. [DOI] [PubMed] [Google Scholar]
- 22.Sun H, Kang W, Cheng X, Chen C, Xiong H, Guo J, et al. The use of the WINROP screening algorithm for the prediction of retinopathy of prematurity in a Chinese population. Neonatology. 2013;104:127–32. doi: 10.1159/000351297. [DOI] [PubMed] [Google Scholar]
- 23.Zepeda-Romero LC, Hard AL, Gomez-Ruiz LM, Gutierrez-Padilla JA, Angulo-Castellanos E, Barrera-de-Leon JC, et al. Prediction of retinopathy of prematurity using the screening algorithm WINROP in a Mexican population of preterm infants. Arch Ophthalmol. 2012;130:720–3. doi: 10.1001/archophthalmol.2012.215. [DOI] [PubMed] [Google Scholar]
- 24.Hard AL, Lofqvist C, Fortes Filho JB, Procianoy RS, Smith L, Hellstrom A. Predicting proliferative retinopathy in a Brazilian population of preterm infants with the screening algorithm WINROP. Arch Ophthalmol. 2010;128:1432–6. doi: 10.1001/archophthalmol.2010.255. [DOI] [PubMed] [Google Scholar]
- 25.Ko CH, Kuo HK, Chen CC, Chen FS, Chen YH, Huang HC, et al. Using WINROP as an adjuvant screening tool for retinopathy of prematurity in Southern Taiwan. Am J Perinatol. 2015;30:149–54. [DOI] [PubMed]
- 26.Chaves-Samaniego MJ, Gómez Cabrera C, Chaves-Samaniego MC, Escudero Gómez J, García Campos JM, Muñoz Hoyos A, et al. Multicenter validation study of the WINROP algorithm as a method for detecting retinopathy of prematurity. J Matern Fetal Neonatal Med. 2020;33:1302–6. [DOI] [PubMed]
- 27.Pinto F, Fernandes E, Virella D, Abrantes A, Neto MT. Born preterm: a public health issue. Port J Public Health. 2019;37:38–49. doi: 10.1159/000497249. [DOI] [Google Scholar]
- 28.Euro-Peristat Project. European Perinatal Health Report. Core indicators of the health and care of pregnant women and babies in Europe in 2015. November 2018. Available at https://www.europeristat.com/.
- 29.Ely DM, Driscoll AK. Infant mortality in the United States, 2017: data from the period linked birth/infant death file. National Vital Statistics Reports. 2019;68. [PubMed]