Quantification of Early Neonatal Oxygen Exposure as a Risk Factor for Retinopathy of Prematurity Requiring Treatment

Abstract

Purpose

Retinopathy of prematurity (ROP) is a leading cause of childhood blindness related to oxygen exposure in premature infants. Since oxygen monitoring protocols have reduced the incidence of treatment-requiring ROP (TR-ROP), it remains unclear whether oxygen exposure remains a relevant risk factor for incident TR-ROP and aggressive ROP (A-ROP), a severe, rapidly progressing form of ROP. The purpose of this proof-of-concept study was to use electronic health record (EHR) data to evaluate early oxygen exposure as a predictive variable for developing TR-ROP and A-ROP.

Design

Retrospective cohort study.

Participants

Two hundred forty-four infants screened for ROP at a single academic center.

Methods

For each infant, oxygen saturations and fraction of inspired oxygen (FiO₂) were extracted manually from the EHR until 31 weeks postmenstrual age (PMA). Cumulative minimum, maximum, and mean oxygen saturation and FiO₂ were calculated on a weekly basis. Random forest models were trained with 5-fold cross-validation using gestational age (GA) and cumulative minimum FiO₂ at 30 weeks PMA to identify infants who developed TR-ROP. Secondary receiver operating characteristic (ROC) curve analysis of infants with or without A-ROP was performed without cross-validation because of small numbers.

Main Outcome Measures

For each model, cross-validation performance for incident TR-ROP was assessed using area under the ROC curve (AUC) and area under the precision-recall curve (AUPRC) scores. For A-ROP, we calculated AUC and evaluated sensitivity and specificity at a high-sensitivity operating point.

Results

Of the 244 infants included, 33 developed TR-ROP, of which 5 developed A-ROP. For incident TR-ROP, random forest models trained on GA plus cumulative minimum FiO₂ (AUC = 0.93 ± 0.06; AUPRC = 0.76 ± 0.08) were not significantly better than models trained on GA alone (AUC = 0.92 ± 0.06 [P = 0.59]; AUPRC = 0.74 ± 0.12 [P = 0.32]). Models using oxygen alone showed an AUC of 0.80 ± 0.09. ROC analysis for A-ROP found an AUC of 0.92 (95% confidence interval, 0.87–0.96).

Conclusions

Oxygen exposure can be extracted from the EHR and quantified as a risk factor for incident TR-ROP and A-ROP. Extracting quantifiable clinical features from the EHR may be useful for building risk models for multiple diseases and evaluating the complex relationships among oxygen exposure, ROP, and other sequelae of prematurity.

Retinopathy of prematurity (ROP) is a leading worldwide cause of childhood blindness that is often preventable with timely screening and treatment. The purpose of ROP screening is to identify premature infants at risk of developing treatment-requiring ROP (TR-ROP). The 2 main risk factors for TR-ROP developing are the degree of prematurity (birth weight [BW] and gestational age [GA]) and supplemental oxygen exposure,1, 2, 3, 4, 5 especially in regions without the resources for careful oxygen management.^3,6, 7, 8 However, current North American screening guidelines take into account only the former, mandating screening for babies with BW of less than 1501 g and GA of less than 31 weeks.9, 10, 11 Although these 2 criteria are highly sensitive, only 5% to 10% of a population will require treatment,⁹ resulting in a large screening burden for ophthalmologists.12, 13, 14

Predictive models have emerged as an attempt to optimize screening guidelines by earlier identification of infants at high-risk of developing TR-ROP or aggressive ROP (A-ROP). Aggressive ROP is described by the third edition of the International Classification for Retinopathy of Prematurity as a “severe, rapidly progressing form of ROP . . . that may occur . . . in larger preterm infants, particularly in regions of the world with limited resources,” a subcategory of which was previously described as aggressive posterior (AP) ROP.¹⁵ Existing risk models for TR-ROP have mostly focused on BW or GA,^8,16, 17, 18, 19, 20, 21, 22 weight gain,^16,18,23,24 other comorbidities,16, 17, 18, 19^,25,26 and easily measurable oxygen parameters (i.e., average oxygen saturation at 1 month).^26,27 However, previous work in North America has shown that development of TR-ROP is correlated with more complex oxygen parameters such as high oxygen saturation targets,^28,29 fluctuations in oxygen saturation,^1,30 supplemental oxygen therapy duration,^1,21,31,32 and time on mechanical ventilation.^20,22,33 Although these findings have guided modern oxygen saturation management strategies,^33,34 a gap in knowledge persists regarding whether oxygen exposure remains relevant to, or may be useful to predict, incident TR-ROP or A-ROP in regions with modern neonatal oxygen management.

The availability of data in the electronic health record (EHR) creates opportunities for evaluating the role and interaction of multiple demographic, laboratory, and imaging features on clinical disease and the development of integrated risk models.^35,36 In terms of ROP, EHRs facilitate frequent quantification of oxygen exposure throughout the neonatal period that may potentially be used to evaluate oxygen’s more nuanced role in TR-ROP and A-ROP. However, the sheer volume of high-signal (i.e., frequently collected) EHR data, such as real-time collection of the fraction of inhaled oxygen (FiO₂), pulse oxygen saturation, and daily laboratory values, complicates analysis.³⁷ Although the use of EHR data in machine learning for ophthalmology is well documented,^5,38,39 a second gap in knowledge exists regarding how big data analytics techniques might be leveraged to improve clinical care in ROP through analysis of the wealth of EHR data.

The purposes of this proof-of-concept study were 3-fold: (1) to demonstrate the feasibility of developing quantitative variables for oxygen exposure from the EHR, (2) to evaluate the relationship between these oxygen variables versus incident TR-ROP and A-ROP, and (3) to use machine learning to determine if quantifying oxygen exposure may add predictive value for incident TR-ROP in a population of extremely prematurely born infants in North America.

Methods

Dataset

As part of a multicenter ROP cohort study, infants screened for ROP at Oregon Health and Science University were eligible. The dataset primarily included babies whose parents or guardians consented to their inclusion in the Imaging and Informatics in ROP study from January 2011 through October 2020. However, a few babies born before 2011 were included in the dataset if they met other inclusion criteria for the Imaging and Informatics in ROP study. Infants were included in this study if: (1) they were born at or before 30 weeks’ GA and (2) oxygen data were complete from birth to 30 weeks’ PMA in the EHR. Patients were excluded if they had missing EHR data, were born after 30 weeks’ GA, or were transferred to Oregon Health and Science University after birth. This study was conducted in accordance with the tenets of the Declaration of Helsinki and was approved by the institutional review board at Oregon Health and Science University. Informed consent was obtained from the parents of all infants enrolled. Each infant screened for ROP was examined based on ROP screening guidelines from at least 31 weeks postmenstrual age (PMA) based on ROP screening guidelines. Each examination consisted of a clinical bedside examination and image-based diagnosis by 3 expert ROP graders (including J.P.C., M.F.C., S.O.) using all 5 retinal fundus image views and International Classification for Retinopathy of Prematurity criteria.⁴⁰ Consensus between the clinical examination and the 3 experts resulted in a reference standard diagnosis (none, mild, type II ROP, type I ROP without A-ROP, or type I ROP with A-ROP). Gestational age and BW, as obtained during routine clinical care, were also noted for all infants. The PMA at treatment was also documented for all infants who developed TR-ROP.

Extraction of Oxygen Parameters from the Electronic Health Record

As part of clinical care in the neonatal intensive care unit (NICU), infants’ oxygen saturation, FiO₂ received, and the type of supplemental oxygen device used were documented regularly in the EHR (Epic Systems). Retrospective review of each eligible infant’s chart was performed, and weekly minimum, maximum, and mean FiO₂ and oxygen saturations, as well as all types of supplemental oxygen devices used at any point, regardless of duration, during that week, were extracted from the EHR. Data preprocessing was performed in R software version 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria). The method of oxygen saturation measurement was not included in the oxygen exposure variable and may have varied depending on the infant’s clinical needs. Cumulative minimum, maximum, and mean oxygen variables were calculated by summing each respective variable per week of life up to and including 30 weeks’ PMA. The unit for these cumulative measurements was defined as percent-weeks, or the cumulative percentage dose of oxygen up until the specified week. We chose to model up to 30 weeks PMA to assess whether oxygen exposure could serve as a predictive risk factor that would be known before the first ROP screening. Time on mechanical ventilation was calculated by summing the total number of weeks an infant was intubated on a weekly basis. Statistical significance between demographic factors was assessed using a Mann–Whitney–Wilcoxon test, with P ≤ 0.05 defined as statistically significant.

Risk Model Development for Treatment-Requiring Retinopathy of Prematurity

Modeling was performed using Python version 3.7⁴¹ using the scikit-learn⁴² package. Before modeling, recursive feature selection, a methodology that ranks the most important features by training models on smaller and smaller sets of variables, was performed to narrow all available demographic and oxygen variables to the 4 variables most likely to be predictive of TR-ROP: GA, BW, cumulative minimum FiO₂ (percent-weeks), and cumulative mean FiO₂. Variance inflation factors were calculated to assess the extent of collinearity between all variables. Gestational age and cumulative minimum FiO₂ were found to have the lowest variance inflation factors and were used for model training.

Using 5-fold cross-validation, models using GA alone, FiO₂ alone, and both variables were trained with a random forest algorithm for eventual development of TR-ROP. For each set of models, hyperparameters were tuned with randomized grid search, including: the number of branches, the depth, bootstrapping, minimum samples per leaf, minimum samples per split, and number of estimators. Performance was evaluated using the mean area under the receiver operating characteristic curve (AUC) and area under the precision-recall curve (AUPRC) scores. Statistical significance between model performance was defined at P ≤ 0.05 using a Mann–Whitney–Wilcoxon test.

Subgroup Analysis of Aggressive Retinopathy of Prematurity

Although A-ROP is rare, and thus there were not enough cases to build machine learning models, we evaluated the demographics of the infants who did and did not develop A-ROP without cross-validation. We assessed the predictive value of oxygen exposure for incident A-ROP by calculating the AUC for cumulative minimum FiO2 before 31 weeks PMA and eventual A-ROP. Sensitivity and specificity were calculated at multiple operating points on the receiver operating characteristic (ROC) curve. A threshold for cumulative oxygen exposure was selected to maximize sensitivity (minimize false-negative results) and specificity (minimize false-positive results). The 95% confidence intervals (CI) for ROC and sensitivity and specificity were calculated using DeLong’s method and bootstrapping (n = 1000 replications), respectively.

Results

Dataset Characteristics

Three hundred twenty-eight infants were identified as part of the Imaging and Informatics in ROP cohort study during the study period; however, 31 infants had missing or incomplete EHR data and 53 infants were born at 30 weeks PMA or later (and therefore did not have oxygen data before 30 weeks). Thus, 244 premature infants were included in the analysis (Table 1). Infants who eventually developed TR-ROP with A-ROP (n = 28) and without A-ROP (n = 5) had lower GA and BW than those who did not require treatment (P < 0.001 for both risk factors). In addition, they had higher oxygen exposure by nearly every method of quantification, including mean FiO₂ before 31 weeks PMA (mean ± standard deviation, 35.2 ± 5.2 weeks PMA for A-ROP and 32.7 ± 8.9 weeks PMA for TR-ROP only vs. 29.9 ± 7.4 weeks PMA; P < 0.01 for both comparisons). Consistent with prior reports, we also found that infants who developed TR-ROP were on mechanical ventilation for a longer time^18,25,27 (mean ± standard deviation, 5.6 ± 2.1 weeks for A-ROP and 2.7 ± 2.2 weeks for TR-ROP only vs. 1.2 ± 2.8 weeks; P < 0.001 for both comparisons). No significant differences were found in mean oxygen saturation between the groups.

Table 1.

Cohort Demographics

Variable	No Treatment-Requiring ROP	Treatment-Requiring ROP without Aggressive ROP	Aggressive ROP
Birthweight (g)	996.8 ± 263.5	784.0 ± 210.7∗	618.2 ± 93.5∗,†
Gestational age (wks)	27.4 ± 1.8	25.2 ± 1.1∗	24.0 ± 0.5∗,†
FiO2 (%)
Minimum	21.3 ± 2.0	21.8 ± 1.3∗	22.1 ± 2.5∗
Maximum	53.8 ± 27.2	60.3 ± 23.5	57.8 ± 14.9
Mean	29.9 ± 7.4	32.7 ± 8.9∗	35.2 ± 5.2∗
Cumulative FiO2 (percent-weeks)
Minimum	94.5 ± 53.5	157.7 ± 70.6∗	192.4 ± 27.7∗,†
Mean	119.0 ± 73.4	192.4 ± 84.5∗	237.5 ± 33.7∗
Mean oxygen saturation (%)	86.8 ± 7.1	86.0 ± 6.4	90.2 ± 0.8
Postmenstrual age at treatment (wks)	N/A	38.1 ± 3.5	34.4 ± 0.5†
Time on mechanical ventilation (wks)	1.2 ± 1.8	2.7 ± 2.2∗	5.6 ± 2.1∗,†
Total	211	28	5

FiO₂ = fraction of inspired oxygen; ROP = retinopathy of prematurity.

Data are presented as mean ± standard deviation, unless otherwise indicated. Gestational age and oxygen-based variables were extracted manually from the electronic health record. We used gestational age and cumulative minimum FiO₂ summed on a weekly basis to train models to predict treatment-requiring ROP before ophthalmologist screening.

^∗Statistically significant differences were defined at a cutoff of P ≤ 0.05 for comparisons with infants who did not develop treatment-requiring ROP.

^†Statistically significant differences were defined at a cutoff of P ≤ 0.05 for comparisons between infants who developed treatment-requiring ROP without aggressive ROP versus infants who developed aggressive ROP.

Performance of Treatment-Requiring Retinopathy of Prematurity Risk Models

The ROC and precision-recall curves are shown in Figure 1. On 5-fold cross-validation, models using both GA and cumulative minimum FiO₂ (AUC, 0.93 ± 0.06; AUPRC, 0.76 ± 0.08) were not significantly better than models trained on GA alone (AUC, 0.92 ± 0.06 [P = 0.59]; AUPRC, 0.74 ± 0.12 [P = 0.32]). However, models trained on minimum FiO₂ alone showed moderate predictive power for TR-ROP (AUC, 0.80 ± 0.09; AUPRC, 0.41 ± 0.09).

Figure 1.

Five-fold cross-validation precision-recall curves for combinations of random forest models trained on gestational age (GA), oxygen exposure, or both. Random forest models trained on both variables achieved similar performance compared with the models trained on GA alone when evaluated on (A) area under the receiver operating characteristic curve (AUC) and (B) area under the precision-recall curve (AUPRC) scores. However, oxygen exposure alone showed moderate predictive power for treatment-requiring retinopathy of prematurity. FiO₂ = fraction of inspired oxygen.

Demographics and Oxygen Exposure and Aggressive Retinopathy of Prematurity

In this population, infants with A-ROP (n = 5) tended to be smaller (618 ± 94 g vs. 784 ± 211 g; P = 0.01), born earlier (24.0 ± 0.5 weeks vs. 25.2 ± 1.1 weeks; P = 0.03), and treated earlier (34 ± 1 weeks PMA vs. 38 ± 4 weeks PMA; P < 0.001) compared with those with TR-ROP without A-ROP, as seen in Table 1. In addition, as shown in Figure 2, they showed higher oxygen exposure and oxygen saturation than infants with TR-ROP and infants in whom TR-ROP never developed (P < 0.001 when compared with both groups). Using ROC analysis, a clear association was demonstrated between cumulative minimum FiO₂ exposure and A-ROP (AUC, 0.92; 95% CI, 0.87–0.96). After adjusting for thresholds, cumulative minimum FiO₂ exposure achieved 100% sensitivity (95% CI, 100%–100%) and 85% specificity (95% CI, 80%–89%) for predicting A-ROP at a cutoff of 157 percent-weeks. That is, a threshold for oxygen exposure with 100% negative predictive value for A-ROP seemed to exist in this dataset; however, the outcome was rare enough that we were unable to fully evaluate the added value of this information to knowledge of GA alone, which is collinear with cumulative oxygen exposure. Future external validation is needed.

Figure 2.

Box plot showing cumulative minimum fraction of inspired oxygen (FiO₂) exposure versus ROP severity (no treatment, treatment-requiring ROP, and treatment-requiring ROP with aggressive ROP) demonstrates a strong association with disease severity. ∗Significant statistical difference by Mann–Whitney–Wilcoxon test in oxygen exposure when comparing infants who developed aggressive ROP (A-ROP) with infants who never developed A-ROP, regardless of whether treatment-requiring ROP (TR-ROP) developed (P < 0.001 for both comparisons).

Discussion

The 2 main clinical factors influencing the incidence of severe ROP are the degree of prematurity and oxygen exposure in the NICU; however, typically ophthalmologists do not quantify the latter as a risk factor. In this proof-of-concept study, we extracted EHR-based oxygen exposure variables to evaluate the relationship between oxygen exposure in TR-ROP versus that in A-ROP. Our study has 2 key findings: (1) EHR data can be extracted readily to build quantitative oxygen exposure variables and (2) although a clear dose response was found between oxygen exposure in TR-ROP versus that in A-ROP, we did not find that this information added predictive value to models based only on GA. Several potential explanations exist for this negative finding, including: (1) that other factors are more important than total oxygen exposure, (2) the way we measured oxygen was overly simplistic and failed to model in vivo oxygen saturation accurately, and (3) the numbers here were too small to detect the signal when we split the sample into cross-validation groups. In this dataset, GA was the single most important factor in the predictive model, and cumulative weekly oxygen lost predictive ability when GA was known, which raises a question regarding the pathophysiologic relationship between the degree of retinal immaturity (GA) and future TR-ROP.

Since the early experiments done by Patz et al and others43, 44, 45 demonstrating the relationship between oxygen and ROP, it has been known that oxygen plays a key role in the pathophysiologic features of severe ROP. This has been further established through the preclinical models of oxygen-induced retinopathy,^46,47 which describe a phase I (hyperoxia-induced vasoconstriction and ischemic injury) and phase II (vascular endothelial growth factor–driven vasoproliferation) of the disease. These phases can be seen clearly in cases where premature babies are exposed to extremely high concentrations of oxygen, as is often the case in low- and middle-income countries, where NICUs often lack human and material resources, and therefore the ability to monitor oxygen carefully.^21,48 As a result, babies who can develop TR-ROP and A-ROP at GA would not be at risk for ROP under optimal circumstances.³ At the extreme end of the phenotypic spectrum, some observers have called this a different disease more similar to the animal models of oxygen-induced retinopathy than typical ROP.^2,49 The International Classification for Retinopathy of Prematurity, Third Edition, acknowledges the range of presentations of A-ROP and the fact that worldwide, A-ROP can occur in older, less premature babies and consequently more anteriorly in the retina.¹⁵

The relationship between oxygen and ROP severity in the setting of strict oxygen monitoring protocols is less clear. Aggressive ROP most often occurs in the youngest babies (a recent large cohort study found no A-ROP in babies older than 26 weeks GA) in the United States; however, the precise reasons why A-ROP develops in some infants and not others remains unclear.⁷ In this study, a clear and statistically significant relationship seemed to exist between degree of prematurity, oxygen exposure, and severity of ROP (Fig 2), despite the noisy nature of weekly cumulative oxygen exposure. In fact, a cutoff (157 percent-weeks) achieved 100% sensitivity with 85% specificity for prediction of A-ROP based on knowledge of cumulative minimum oxygen concentration alone. These results suggest that, like in low- and middle-income countries, a relationship exists between oxygen exposure and A-ROP and that at the extremes of viability we can very much see the delicate balance between the degree of oxygen necessary for resuscitation and survival and the sequelae of hyperoxic injury to the retina. Because A-ROP can present earlier and develop more aggressively than non–A-ROP, knowledge of this variable may improve clinicians' judgment of disease risk before examination and may improve detection of eyes with early A-ROP, which often appear benign (Fig 3).

Figure 3.

Fundus photographs showing examples of infants before and after development of advanced aggressive retinopathy of prematurity (A-ROP). A, C, Posterior pole fundus images from 2 infants with A-ROP before diagnosis demonstrating severe vasoconstriction, a phenotype of phase I A-ROP that can be mistaken for normal vessels. B, D, Both of these infants required treatment at 34 weeks postmenstrual age for A-ROP with flat stage 3 extraretinal neovascularization in zone I. Oxygen exposure before retinopathy of prematurity (ROP) screening for infants 1 and 2 was extremely high (208 and 230 percent-weeks, respectively), which may be useful in helping to identify eyes like those in (A) and (C) before the development of proliferative ROP.

Oxygen exposure could be made available at the time of first ROP screening as easily as other demographic risk factors, such as BW and GA, especially if made more efficient than manual review alone by prospective data collection or automated extraction of oxygen data from the EHR. More importantly, however, researchers have only scratched the surface in terms of data potentially available in the NICU that may be relevant to ROP incidence and severity, including continuous data (every second) from birth regarding oxygen exposure, saturation, and time on mechanical ventilation. An analytic approach using machine learning may enable identification of physiologic patterns, such as wide fluctuations in terms of oxygen saturation (i.e., intermittent hypoxia) and exposure, which are more specific than minimum FiO₂.³⁴ These approaches could build on existing models that incorporate other prognostic clinical factors, such as weight gain,^16,23,24 anemia,⁵⁰ thrombocytopenia,⁵¹ sepsis,^52,53 and necrotizing enterocolitis.^51,54 More importantly, both the approach (integrated risk modeling using big data) and the specific line of questioning (exposure to hyperoxic conditions causing changes in multiple organ systems) may generalize to other organ systems and disease processes.

Study Limitations

This study has several limitations. First, because this was a proof-of-concept study from a single academic center, the results are limited in their generalization to other populations, both domestically in North America and internationally. Second, the number of infants with A-ROP was small, which limits the conclusions that can be proven about the role of oxygen in the pathophysiologic features of A-ROP. Further investigation validating the relationship between oxygen exposure and A-ROP is warranted in both developed countries as well as low- and middle-income countries, where A-ROP is more common.⁴ Third, 31 patients had missing oxygen data in the EHR and were excluded. Although this seemed to be random, we cannot exclude the possibility of selection bias if these records were systematically related to the outcomes. Finally, we did not, in this study, attempt to compare the risk model of GA plus oxygen to other existing risk models.^{16,17,26,27,55}

In conclusion, oxygen variables can be extracted readily from the EHR and can be applied to predictive modeling for TR-ROP and A-ROP. This proof-of-concept study demonstrated that oxygen exposure may be correlated with development of severe ROP, which may offer future direction in understanding the physiologic features of A-ROP development and other comorbidities associated with oxygen exposure. Future work may build on this study by examining oxygen concentrations at a more granular level to improve modeling for TR-ROP and other diseases related to oxygen exposure.

Manuscript no. D-21-00094.