Validation of the postnatal growth and retinopathy of prematurity screening criteria

Abstract

The objective of this study was to validate the generalizability of the Postnatal Growth and Retinopathy of Prematurity Study screening criteria in a new cohort of infants at risk for retinopathy of prematurity (ROP). This retrospective validation study conducted at a single academic medical center included 484 infants at risk for ROP born between January 14, 2014, and December 21, 2019. The primary outcomes evaluated were sensitivity for both type 1 and type 2 ROP, as defined by the Early Treatment of Retinopathy of Prematurity Study, as well as the reduction in total number of infants requiring ROP examinations. Secondary outcomes included the total number of ROP examinations avoided and the potential cost reduction of eliminating these examinations. In a cohort of 484 infants at risk for ROP, the criteria identified 40 of 40 (100%, 95% confidence interval 91.19%–100%) type 1 ROP cases and 27 of 27 (100%, 95% confidence interval 87.23%–100%) type 2 ROP cases while reducing the total number of infants screened by 35.7%. The Postnatal Growth and Retinopathy of Prematurity Study criteria were found in this study to be generalizable to a cohort of infants at a single teaching institution in central Texas. If applied during the studied interval, these criteria could have significantly reduced the number of infants undergoing ROP examinations and maintained excellent sensitivity for type 1 ROP.

Retinopathy of prematurity (ROP), a disease of the developing retinal vasculature, represents a major cause of preventable childhood vision impairment in the US and worldwide. ¹ Detecting clinically significant ROP is necessary, as multiple prior studies have demonstrated improvement in long-term visual outcomes with timely treatment. ^{2 , 3} Screening guidelines in the United States were updated in December 2018 and recommend examinations for any child with a birth weight <1501 g or a gestational age at birth of ≤30 weeks, as well as those with a birth weight of 1500 to 2000 g with a gestational age >30 weeks who are deemed by a pediatrician or neonatologist to be at risk for ROP. ⁴

The current methodology for ROP screening is highly sensitive but lacks specificity for detecting treatment-requiring ROP. Less than 5% of infants screened in the US require treatment related to ROP. ^{2 , 5–7} Efforts are under way to revise screening criteria to incorporate postnatal weight gain into prediction models. ^{3 , 8–16} Developing retinal vasculature is mediated by retinal vascular endothelial growth factor, and insulin-like growth factor-1 plays a permissive role in activating retinal vascular endothelial growth factor. A slow rate of weight gain in the postnatal period is considered a surrogate marker for a slower than normal increase in serum levels of insulin-like growth factor-1. ^17–20 Hence, it is thought that postnatal weight gain can be used to identify infants at higher risk for ROP.

The Postnatal Growth and Retinopathy of Prematurity Study (G-ROP) model performed well in its initial retrospective model development cohort. Sensitivity for early treatment for type 1 ROP was 100% (95% confidence interval [CI], 99.2%–100%), with a reduction of 30.3% in screened infants. ¹⁴ In a prospective validation study, this model achieved 100% sensitivity (95% CI, 99.4%–100%) and reduced the number of infants screened by 32.5%. ²¹ Of the models currently being investigated, it appears that the G-ROP model is the most promising for future clinical adaptation. In this study, we retrospectively applied the G-ROP criteria to a cohort of infants at Baylor Scott & White Central Texas in an effort to validate the performance of the model.

METHODS

All infants who met current ROP screening criteria born between January 14, 2014, and December 21, 2019, were identified. Gender, birth weight, gestational age, diagnosis of hydrocephalus, daily weights, mortality, and ROP staging were recorded for each patient. All infants were screened by board-certified pediatric ophthalmologists or retina specialists who had training and experience in the diagnosis, management, and treatment of ROP. The combination of the most posterior zone and most advanced stage in the most severe eye was recorded for each infant with a diagnosis of ROP. The ROP was classified as type 1 or type 2, as defined by the Early Treatment of Retinopathy of Prematurity Study, if applicable. Any treatment related to ROP was also recorded.

The G-ROP criteria were then applied to the data set. A child would be identified as needing to undergo screening if he or she met any one of these criteria: gestational age <28 weeks, birth weight <1051 g, weight gain between 10 and 19 days of age <120 g, weight gain between 20 and 29 days of age <180 g, weight gain between 30 and 39 days of age <170 g, or a diagnosis of hydrocephalus. ¹⁴ If no criteria were met, the child would not be screened.

If an infant had missing daily weights, the next daily weight available was used to evaluate the weight gain over a given interval, but the intervals were never extended. This was done in an effort to maintain a more conservative approach if any data were unavailable. For infants discharged from the hospital before day 39, the decision to include or exclude them from screening was based on the data intervals that were available.

The percentage reduction in infants screened and total examinations avoided were calculated. For any examinations that would not have been necessary under the G-ROP model, the total charges billed for services were calculated.

Two primary outcomes were investigated: (1) sensitivity for both type 1 and type 2 ROP, and (2) reduction in total number of infants undergoing ROP examinations. The secondary outcomes investigated were the reduction in total number of ROP examinations and the potential cost savings of eliminating these examinations.

Institutional review board approval was obtained from Baylor Scott & White Health. The data were recorded into Microsoft Excel by two data collectors and analyzed for error using audits and discrepancy checks. Any discrepancy found was referred back to the collecting agent, who resolved the error by reviewing the patient chart. Statistical analysis was performed using SAS version 9.4.

RESULTS

A total of 499 infants born between January 14, 2014, and December 21, 2019, met current ROP screening criteria. There were 246 (49.3%) males and 253 females (50.7%). Their average gestational age was 28.4 weeks (range 22.5–34.2 weeks), and their average birth weight was 1120 g (range 410–3610 g). Of these 499 infants, 484 underwent ROP screening, had a known ROP outcome, and were included in the data analysis.

The G-ROP identified 40 of 40 (100%, 95% CI 91.19%–100%) type 1 ROP cases and 27 of 27 (100%, 95% CI 87.23%–100%) type 2 ROP cases. The G-ROP criteria also identified 51 of 51 (100%, 95% CI 93.02%–100%) treated infants. There were 188 infants identified who would not have been screened if the G-ROP criteria had been applied. Of these 188, 15 infants never underwent screening examinations. These infants were excluded from the data analysis. Thus, 173 of 484 (35.7%) included infants would not have undergone screening examinations using G-ROP criteria.

Twelve children are now deceased. Of these, 11 either had a known negative ROP outcome or met G-ROP inclusion criteria. One infant died at approximately 4 weeks of age having never undergone an exam. This child did not meet any G-ROP inclusion criteria at the time of death, did not have a known ROP outcome, and was therefore excluded.

These 173 infants underwent a total of 767 examinations (4.43 examinations per infant), of which 377 were inpatient and 390 were outpatient. Determining total charges billed for inpatient examinations proved difficult. The professional component billed for inpatient examinations totaled $48,738. Charges billed for both new and established examinations in the outpatient setting totaled $273,909. Total known charges submitted for all examinations of these 173 infants was $322,647 but this value likely underestimates inpatient charges.

DISCUSSION

This study aimed to validate the generalizability of the G-ROP screening criteria in a new cohort of at-risk infants. The criteria applied as is were successful in capturing all type 1 and 2 ROP in our cohort, as well as all infants treated for ROP. The low number of cases in each category did not allow for a narrow 95% confidence interval in our cohort, and these intervals remained wide as expected. This model decreased the total amount of infants screened by 35.7%.

The G-ROP model achieved 100% sensitivity for type 1 ROP in the initial cohort, a subsequent large validation study, and here again in our study. This reproducibility of excellent sensitivity for type 1 ROP is what sets the G-ROP model apart from many other recently developed models that incorporate postnatal weight gain.

The sensitivity for type 2 ROP was also excellent in our study, as it was in both the development study and subsequent prospective validation. ^{14 , 21} While it is not recommended currently that type 2 ROP be treated, ⁷ it is doubtful that a model that misses a significant number of type 2 ROP cases would be adopted, as these cases are at higher risk for progression to type 1 ROP.

Sixteen infants with a diagnosis of ROP would not have been screened under the G-ROP criteria. All 16 vascularized fully without intervention.

The 35.7% reduction in total infants screened has benefits for all parties involved. Infants could avoid the stress and pain of undergoing these examinations. ^{22 , 23} There is also a cost savings to the health care system in reducing these examinations. A larger cost analysis is being planned by the developers of the G-ROP model, but this is yet to be published. ²¹ In our cohort, $322,647 would have not been billed for these 767 examinations. Clinical staff would also benefit, as coordinating these examinations can be time consuming. A recently published review of the process of coordinating ROP care at Bascom Palmer Eye Institute demonstrated how complex this process can be. ²⁴

Clearly, the G-ROP model performed very well in a retrospective review in our cohort of infants. This study alone does not demonstrate the readiness of the model for clinical adaptation. However, the findings contribute to the body of evidence that this model is successful in identifying all treatment-requiring ROP and reducing the total number of infants undergoing screening examinations.

This study is, of course, not without limitations. Fifteen infants who met current screening criteria never underwent screening. These infants had an average gestational age of 31.7 weeks (range 28.6–34.2 weeks) and an average birth weight of 1449 g (range 1120–2580 g). Only one child of this group was born under 1200 g and at <30 weeks’ gestation, and this child unfortunately died at <4 weeks of age. Of the 180 infants diagnosed with any degree of ROP in our cohort, the average gestational age was 26.39 weeks, and the gestational age of those who did not develop any ROP was 29.54 weeks. The average birth weight for those who developed any ROP was 900 g, and the average birth weight for infants who did not develop ROP was 1250 g. These are averages, but nonetheless suggest that these infants that went unscreened would have been at lower risk for the development of ROP.

Not all infants had weights recorded for every day of life. Shortened intervals were used at a low rate for days 10 to 19 (5.6%), days 20 to 29 (7.55%), and days 30 to 39 (1.31%). Many infants were discharged from the hospital prior to day 39; thus, all interval calculations could not be completed for all infants. The collection of daily weights was done in a nonstandardized way, including collection at various times of day. Examinations were also not done by a single ophthalmologist, which could theoretically impact the precise staging of ROP.

In conclusion, the G-ROP model was found in this study to be generalizable to a cohort of infants at a single teaching institution in central Texas. If applied, this model would have significantly reduced the number of infants undergoing ROP examinations and maintained excellent sensitivity for type 1 ROP.