Risk Score for Predicting Treatment-Requiring Retinopathy of Prematurity in the e-ROP Study

Abstract

Purpose

To develop a risk score for predicting treatment-requiring ROP (TR-ROP) as determined by study-certified ophthalmologists in the e-ROP study.

Design

Second Analyses of an observational cohort study.

Participants

Infants with birth weight <1251g who had at least one imaging session by 34 weeks of postmenstrual age (PMA) and at least one subsequent ROP examination for determining TR-ROP by study-certified ophthalmologists.

Methods

Non-physician trained readers evaluated wide-field retinal image sets for characteristics of ROP, pre-plus/plus disease and retinal hemorrhage. Risk score points for predicting TR-ROP were derived from the regression coefficients of significant predictors in a multivariate logistic regression model.

Results

Among 771 infants, 85 (11.0%) developed TR-ROP. In a multivariate model, significant predictors [odds ratio (95% confidence interval)] for TR-ROP were: gestational age (GA) [5.7 (1.7 – 18.9) for ≤25 vs. ≥28 weeks], need for respiratory support [7.0 (1.3 – 37.1) for high frequency oscillatory ventilation vs. no respiratory support], slow weight gain [2.4 (1.2 – 4.6) for weight gain ≤12g/day vs. >15g/day], and image findings at the first image session including: number of quadrants with pre-plus [3.8 (1.5 – 9.7) for 4 pre-plus quadrants vs. no pre-plus], stage and zone of ROP [4.7 (2.1 – 11.8) for stage 1–2 zone I, 5.9 (2.1 – 16.6) for stage 3 zone I vs. no ROP], and presence of blot hemorrhage [3.1 (1.4 – 6.7)]. Image findings predicted TR-ROP better than GA (area under ROC curve [AUC] =0.82 vs. 0.75, p=0.03). The 22-point risk score derived from 6 significant predictors had a median of 4 points (range: 0 to 18). The risk of TR-ROP steadily increased with higher risk score (0% for 0 point, 1–8% for 1–5 points, 12–27% for 6–10 points, 43% for 11–13 points and 94% for 14 points or above), and predicted TR-ROP well (AUC=0.88, 95% CI: 0.85–0.92). Risk score ≥3 points for predicting TR-ROP had sensitivity 98.8%, specificity 40.1%, positive and negative predictive values of 17.0% and 99.6% respectively.

Conclusions

Image characteristics at 34 PMA weeks or earlier independently predict TR-ROP. If externally validated in other infants, risk score, calculated from image findings, GA, weight gain and respiratory support, enables early identification of infants in need of increased surveillance for TR-ROP.

INTRODUCTION

Retinopathy of prematurity (ROP) is a serious vaso-proliferative disease of retina of premature infants, and can lead to visual impairment or blindness in children. Severe ROP usually can be effectively treated with retinal ablative surgery if diagnosed within a tight time window.¹ Detection of treatment-requiring ROP (TR-ROP) usually involves subjecting at-risk babies to resource-intensive and thereby costly, serial diagnostic eye exams performed by ophthalmologists, beginning at 32–34 weeks post-menstrual age (PMA) regardless of gestation age (GA). In United States, ROP screening guideline requires examining all infants with birth weight (BW) of 1500 grams or less or GA of 30 weeks or less.² However, this guideline results in many unnecessary examinations because fewer than 10% of infants meet the ROP treatment criteria.^1;3 A potential alternative is to use a telemedicine system for ROP screening, to identify infants who require examinations by ophthalmologists to determine whether treatment is required. Recent studies have shown that telemedicine for ROP screening can be safe and effective.^4–9 Our “Telemedicine approaches to evaluating acute-phase retinopathy of prematurity” (e-ROP) Study demonstrated that a telemedicine system using non-physician imagers and non-physician trained readers for detection of referral-warranted ROP (RW-ROP, defined as Zone I disease, Stage 3 disease or worse, or plus disease) is valid, safe and cost-effective.^5;10–12

The image evaluation of ROP usually involves determining the presence of ROP morphological features including pre-plus or plus disease, location of ROP (zone I or II), and ROP stage. Our previous investigation showed that ROP morphological findings of an eye at the first eye exam performed by an ophthalmologist predicted the subsequent development of referral-warrant ROP (RW-ROP), beyond what is already predicted by the BW, GA, postnatal weight gain, and respiratory support status at exam.¹³ Since ROP telemedicine is based on evaluation of retinal images, identifying earlier image characteristics for developing a risk score that predicts the subsequent development of TR-ROP (TR-ROP) will enhance its functionality and clinical utility. Such risk score could lead to improved ROP telemedicine screening and reduce the burden of diagnostic eye examinations by ROP specialists for detecting TR-ROP.

The purpose of this paper is to determine if image findings at 32–34 PMA weeks predict the development of TR-ROP, and to develop a risk score based on the image findings and other established ROP risk factors for predicting TR-ROP.

METHODS

This is a secondary analysis of the data from the e-ROP Study. The e-ROP Study is a multi-center observational cohort study to evaluate the validity, reliability, feasibility, safety and relative cost-effectiveness of a digital imaging system to identify infants with RW-ROP. The details of the design, methods, and primary results for the e-ROP Study have been previously published.^5;11;12;14 Major features of the image evaluation procedure and the candidate predictors for TR-ROP are described here.

Study Subjects

Infants eligible for enrollment into the e-ROP Study had BW less than 1251g, were likely to survive 28 days and to remain in neonatal intensive care units (NICU) for serial ROP examinations and imaging. The study enrolled 1284 infants in neonatal intensive care units from 13 North American centers from May 25, 2011, through October 31, 2013. Informed consent was obtained from the parent/guardian of eligible infants. The study protocol and informed consent process were approved by institutional review boards of participating centers.

Study Procedure

Infants underwent serial ROP imaging in both eyes using the RetCam Shuttle® (Clarity Medical Systems, Pleasanton, CA), in addition to a standard diagnostic examination by Study-certified ophthalmologists experienced in diagnosing ROP.⁵ The results of ROP (stage, zone, and pre-plus/plus disease) from eye examinations were summarized using the 2005 International Classification of ROP.¹⁵ From each eye, a trained imager obtained an image set that included an external image to assess pupillary dilation, along with five retinal views: disc center and four disc off-centered giving views of the inferior, superior, temporal and nasal retina. The imagers were masked to the results of the examination and the ophthalmologists were masked to the images. Timing of first ROP examinations was determined by local Clinical Center criteria for usual clinical care and imaging sessions began at 32 weeks PMA.

Follow-up diagnostic examinations and concurrent imaging were conducted at least every other week unless medically contraindicated until the ophthalmologist noted one of the following: mature retinal vessels, immature zone III on two occasions at least 7 days apart, ROP regressed or regressing on two visits at least 7 days apart, treatment for severe ROP, or the infant reached 40 weeks PMA with no ROP or only stage 1 or 2 ROP.^5;14 For the infants who were discharged to other non-study clinical centers before meeting these study endpoint, follow up information was obtained to determine t whether the infant was treated for ROP.

Image Evaluation

All image sets were graded by non-physician trained readers in a centralized image reading center in the University of Pennsylvania following the standard grading protocol.¹¹. Each image set was graded independently by two trained readers with discrepancies adjudicated by the Reading Supervisor. Readers were masked to results of diagnostic examinations, previous gradings for either eye of the infant, images of the fellow eye, and demographic data. The readers were allowed to use the functional modalities of the e-ROP grading software to manipulate image contrast and brightness to enhance the appearance of the ROP morphology and attenuating background noise in poor-quality images. ¹¹

All readers determined for each quadrant, whether the posterior pole vessels were normal or sufficiently abnormal to be designated as pre-plus or plus disease. In addition the reader determined zone of vascularization or the zone in which morphologic features consistent with ROP were present. ROP was determined by the presence of a demarcation line (stage 1), a ridge (stage 2), extraretinal neovascularization (stage 3), or retinal detachment (stage 4). When the image was not of sufficient quality to grade specific ROP features, it was recorded as cannot grade (CG).

Presence of retinal hemorrhage was not evaluated in the primary e-ROP image evaluation. As a secondary analysis of e-ROP images, all image sets (n=1705) from eyes ever noted to have hemorrhage either on examination by a study-certified ophthalmologist or on initial e-ROP image grading were selected for detailed hemorrhage grading for the presence, location, and type of retinal hemorrhages (dot, blot, flame, pre-retinal).

Predictive Factors

Demographic characteristics were collected at enrollment including BW, GA, sex, race, ethnicity, singleton or multiple births.

Data on feeding and respiratory support status of an infant and the most recent weight prior to the first image session were collected by the study coordinator. Based on the BW and weight at the first study-related image session, weight gain rate (g/day) was calculated as the weight gain between birth and the first image session divided by days of life at first image session.

The image evaluation findings from the first image session considered as predictors for TR-ROP included: presence and quadrants with pre-plus/plus disease, presence, stage and zone of ROP, and presence and type of retinal hemorrhage (dot, blot, flame, and pre-retinal).

Statistical Analysis

We described the demographic characteristics of infants or morphological findings using percentages for categorical features, and mean (standard deviation) for continuous measures. We first analyzed predictors for TR-ROP using univariate logistic regression models followed by a multivariate logistic regression model that included predictors with p<0.10 in the univariate analyses and their two-way interaction terms. The multivariate model went through backward stepwise selection of predictors, and the final multivariate model retained the predictors with p<0.05. Ocular predictors at the first image session were determined based on the worse eye of a specific morphological feature. Development of TR-ROP was defined as ever having ROP treatment in either eye. The odds ratio (OR) and its 95% confidence intervals (95% CI) of each predictor was calculated from logistic regression models. Risk score points were assigned to each level of statistically significant predictors based on the regression coefficients in the final multivariate logistic regression model, and the total risk score point for each infant was calculated based on the sum of all risk points of predictors. The prediction derived risk score for TR-ROP was evaluated using area under ROC curve (AUC). Sensitivity and specificity, positive predictive value and negative predictive value for TR-ROP using various cut-points of risk score were also calculated.

To determine the optimistic bias in prediction performance from using the same data for prediction model development and prediction performance evaluation, we performed the internal validation using the bootstrap cross-validation method of Harrell et al.¹⁶ This validation is based on 1000 bootstrap replicates, each consisting of 771 total subjects sampled from 771 study subjects with replacement. Using each bootstrap replicate, a prediction model was developed and its performance was evaluated on both bootstrap replicate data and original data by calculating the area under ROC curves for predicting TR-ROP. The “optimism” in AUC was the difference between that from the bootstrap replicate and the original data set. The average “optimism” of AUC and 95% confidence intervals (CIs) were calculated on the basis of 1000 replicates. All the statistical analyses were performed in SAS v9.4 (SAS Institute Inc., Cary, NC).

RESULTS

Among 1284 infants enrolled into the e-ROP study, 771 (60%) infants had an imaging session at 34 PMA weeks or earlier and at least one subsequent ROP examination for determining TR-ROP by study-certified ophthalmologists, thus were included in this TR-ROP prediction analysis (Figure 1, available at http://www.aaojournal.org). The mean BW of these 771 infants was 853g (range: 380 to 1250g), mean GA 26 weeks (range: 22 to 33 weeks), 12% were Hispanic or Latino, 56% were White, and 31% were Black, 30% were multiple births (Table 1, available at http://www.aaojournal.org). 85 (11.0%) infants developed TR-ROP in one eye or both eyes.

The first image session occurred at 32 weeks PMA in 292 (39%) infants, at 33 weeks in 319 (43%) infants and at 34 weeks in 137 (18%) infants. Based on the trained readers evaluation of images taken at the first image session, 154 (20%) infants had pre-plus disease. Among these infants with pre-plus disease at the first image session, pre-plus was present in one quadrant in 53 (34%) infants, 2 quadrants in 39 (25%) infants, 3 quadrants in 31 (20%) infants, and 4 quadrants in 32 (21%) infants. Four infants had plus disease in one quadrant (3 infants) or four quadrants (1 infant). ROP was present in the first image session in 269 (35%) infants, majority of them were stage 2 (61%) or stage 3 (39%), and 61 (23%) infants had ROP in zone I. Retinal hemorrhage was noted in 91 (12%) of infants, including dot hemorrhage in 34 (4.4%) infants, blot hemorrhage in 48 (6.2%) infants, pre-retinal hemorrhage in 30 (3.9%) infants, and flame hemorrhage in 7 (0.9%) infants (Table 2, available at http://www.aaojournal.org).

At the first image session, 106 (13.7%) infants required the controlled mechanical ventilation and 11 (1.4%) needed high frequency oscillatory ventilation. The mean daily weight gain rate was 14.4g/day (SD=4.4), with 217 (28.1%) infants having growth rate of 12g/day or less, and 330 (42.8%) infants having growth rate of 15g/day or more.

In univariate analyses (Tables 3, available at http://www.aaojournal.org), demographic predictors associated with TR-ROP were: low BW (p<0.0001), low GA (p<0.0001), and Hispanic or Latino ethnicity (p=0.03). The image findings (Table 4) from the first image session associated with TR-ROP were: larger number of quadrants with pre-plus or plus disease (p<0.0001), higher stage of ROP (p<0.0001), lower zone of ROP (p<0.0001), and presence of retinal hemorrhage (p<0.0001), including blot hemorrhage (p<0.0001) and pre-retinal hemorrhage (p<0.0001). Feeding status of nil per os (p=0.0001) and need for respiratory support (p<0.0001) at the first image session were associated with the high risk for TR-ROP (Table 5, available at http://www.aaojournal.org). A slower weight gain (12 g/day or less) at the time of the first image session was associated with increased risk of TR-ROP (OR=2.43, p=0.005) when compared to the infants with weight gain of more than 18 g/day (Table 5, available at http://www.aaojournal.org).

Table 4.

Univariate analysis for association of ROP findings from the first session image evaluation with incidence of treatment-requiring ROP

Findings from evaluation of images from the first session at 32–34 weeks PMA in either eye	# of infants	Treatment -requiring ROP (%)	OR (95% CI)	P-value
Posterior pole				<0.0001
Normal	601	37 (6.2%)	1.00
1 quadrant pre-plus	53	8 (15.1%)	2.71 (1.19 – 6.17)
2 quadrants pre-plus	39	9 (23.1%)	4.57 (2.02 – 10.3)
3 quadrant pre-plus	30	8 (26.7%)	5.54 (2.31 – 13.3)
4 quadrants pre-plus or any plus	33	16 (48.5%)	14.3 (6.71 – 30.7)
Unknown	15	7 (46.7%)	13.3 (4.59 – 38.8)
Stage of ROP				<0.0001
No	480	21 (4.4%)	1.00
Stage I or II	164	23 (14.0%)	3.57 (1.92 – 6.63)
Stage III	105	35 (33.3%)	10.9 (6.02 – 19.8)
Unknown	22	6 (27.3%)	8.20 (2.91 – 23.1)
Zone of ROP				<0.0001
No ROP	480	21 (4.4%)	1.00
II	208	29 (13.9%)	3.54 (1.97 – 6.37)
I	61	29 (47.5%)	19.8 (10.2 – 38.6)
Unknown	22	6 (27.3%)	8.20 (2.91 – 23.1)
Stage and zone of ROP				<0.0001
No ROP	480	21 (4.4%)	1.00
Stage 1–2, Zone II	136	12 (8.8%)	2.12 (1.01 – 4.42)
Stage 3, Zone II	72	17 (23.6%)	6.76 (3.36 – 13.6)
Stage 1–2, Zone I	28	11 (39.3%)	14.1 (5.89 – 33.9)
Stage 3, Zone I	33	18 (54.6%)	26.2 (11.6 – 59.1)
Unknown	22	6 (27.3%)	8.19 (2.91 – 23.1)
Any retinal hemorrhage				<0.0001
No	680	60 (8.8%)	1.00
Yes	91	25 (27.5%)	3.91 (2.30 – 6.66)
Dot hemorrhage				0.89
No	737	81 (11.0%)	1.00
Yes	34	4 (11.8%)	1.08 (0.37 – 3.14)
Blot hemorrhage				<0.0001
No	723	67 (9.3%)	1.00
Yes	48	18 (37.5%)	5.86 (3.11 – 11.1)
Flame hemorrhage				0.16
No	764	83 (10.9%)	1.00
Yes	7	2 (28.6%)	3.28 (0.63 – 17.2)
Pre-retinal hemorrhage				<0.0001
No	741	75 (10.1%)	1.00
Yes	30	10 (33.3%)	4.44 (2.00 – 9.84)

In the multivariate analysis (Table 6), significant predictors [OR (95% CI)] for TR-ROP were: low GA [5.7 (1.7 – 18.9) for ≤25 weeks vs. ≥28 weeks], number of quadrants with pre-plus or presence of plus [4.0 (1.4 – 11.4) for 3 pre-plus quadrants, and 3.8 (1.5 – 9.7) for 4 pre-plus quadrants or presence of plus in any quadrant vs. no pre-plus or plus], stage and zone of ROP [4.7 (2.1 – 11.8) for stage 1–2 ROP in zone I, 5.9 (2.1 – 16.6) for stage 3 in zone I vs. no ROP], blot hemorrhage [3.1 (1.4 – 6.7) for presence vs. absence], need for respiratory support [7.0 (1.3 – 37.1) high frequency oscillatory ventilation vs. no respiratory support], and slow weight gain [2.4 (1.2 – 4.6) for weight gain ≤12g/day vs. >15g/day].

Table 6.

The multivariate analysis for predictors of treatment-requiring ROP^§

Predictors	# of infants	Treatment-requiring ROP (%)	Adjusted OR (95% CI)	P-value	Risk points
Gestational age (weeks)				0.01
≤25	280	65 (23.2%)	5.68 (1.71 – 18.9)		4
26	137	9 (6.6%)	2.78 (0.74 – 10.4)		2
27	136	7 (5.2%)	2.68 (0.70 – 10.3)		2
≥28	137	4 (1.8%)	1.00		0
Posterior pole				0.001
Normal	601	37 (6.2%)	1.00		0
1 quadrant pre-plus	53	8 (15.1%)	2.03 (0.77 – 5.36)		1
2 quadrants pre-plus	39	9 (23.1%)	2.41 (0.91 – 6.39)		2
3 quadrant pre-plus	30	8 (26.7%)	4.02 (1.41 – 11.4)		4
4 quadrants pre-plus or plus in any quadrant	33	16 (48.5%)	3.84 (1.51 – 9.73)		4
Unknown	15	7 (46.7%)	8.08 (1.50 – 43.3)		5
Stage and Zone of ROP				0.004
No ROP	480	21 (4.4%)	1.00		0
Stage 1–2, Zone II	136	12 (8.8%)	1.09 (0.47 – 2.54)		0
Stage 3, Zone II	72	17 (23.6%)	2.13 (0.90 – 5.02)		2
Stage 1–2, Zone I	28	11 (39.3%)	4.67 (1.61 – 13.6)		3
Stage 3, Zone I	33	18 (54.6%)	5.91 (2.10 – 16.6)		4
Unknown	22	6 (27.3%)	1.64 (0.30 – 8.84)		1
Blot hemorrhage				0.005
No	723	62 (9.3%)	1.00		0
Yes	48	18 (37.5%)	3.09 (1.42 – 6.71)		2
Respiratory support				0.0004
No respiratory support	173	7 (4.1%)	1.00		0
Nasal canula	212	14 (6.6%)	0.72 (0.26 – 2.05)		0
CPAP/NIPPV/HFNC	269	29 (10.8%)	0.75 (0.28 – 2.02)		0
Controlled mechanical ventilator	106	30 (28.3%)	2.33 (0.85 – 6.38)		2
High frequency oscillatory ventilation	11	5 (45.5%)	7.02 (1.33 – 37.1)		5
Weight growth rate (g/day)				0.04
≤12	217	36 (16.6%)	2.35 (1.20 – 4.59)		2
>12, ≤15	224	24 (10.7%)	1.34 (0.66 – 2.72)		1
>15	330	25 (7.6%)	1.00		0

^§The multivariate model started with all the significant predictors in Tables 1, 2 and 3. The model went through the stepwise selection by only keeping the statistical significant variables in the final multivariate model.

The performance of predictions of TR-ROP is presented in Table 7 (available at http://www.aaojournal.org). Image findings by 34 weeks PMA strongly predicted the development of TR-ROP with AUC of 0.82 (95% CI: 0.77 – 0.87), which is significantly higher than predictions based on GA (AUC=0.75, p=0.03), respiratory status (AUC=0.70, p=0.002), and weight gain rate (AUC=0.60, p<0.0001). Combining image findings of the first image session with GA improved the prediction (AUC=0.86, p=0.052). Adding the respiratory support and weight gain further improved the prediction for TR-ROP (AUC=0.88, p=0.004). The final prediction model of TR-ROP had the predictors image findings, GA, respiratory status, and rate of weight gain, and predicted TR-ROP well (AUC=0.88, 0.84 – 0.91). From 1000 bootstrap replicates for internal cross-validation, the average “optimism” of AUC from the prediction model for TR-ROP was 0.02 (95% CI: 0.01 to 0.05).

Based on the regression coefficients from the final multivariate logistic regression model, risk score points were assigned for each level of the 6 statistical significant predictors as shown in Table 6, including GA (4 points) pre-plus or plus disease (5 points), stage and zone of ROP (4 points), blot hemorrhage (2 points), respiratory support (5 points) and slow weight gain (2 points). A 22-point risk score was calculated from the values assigned to each of these 6 significant predictors. The risk score calculated for each infant had median of 4 points (range: 0 to 18). The percentage of infants developing TR-ROP steadily increased as the risk score increased (Table 8, available at http://www.aaojournal.org), ranging from 0% for risk score 0 point, 1–8% for 1–5 points, 12–27% for 6–10 points, 43% for 11–13 points to 94% for 14 points or above (Figure 2). The risk score predicted TR-ROP well with AUC of 0.88 (95% CI: 0.85–0.92, Figure 3, available at http://www.aaojournal.org). The sensitivity and specificity for predicting TR-ROP at various cut-points of risk score are shown in Table 9. When a risk score of 3 or above was considered as high risk, the sensitivity was 98.8% (95% CI: 93.6 – 99.8%) and specificity was 40.1% (95% CI: 36.5 – 43.8%), positive predictive value was 17.0% (95% CI: 13.9 – 20.5%) and negative predictive value was 99.6% (95% CI: 98.0 – 99.9%).

Figure 2.

The rate of treatment-requiring retinopathy of prematurity by the risk score points. The risk score was calculated based on the image findings at the first image session.

Table 9.

Sensitivity and specificity for predicting treatment-requiring ROP using various cut-points of the risk score

Risk score cut-point	Sensitivity (N=85)	Specificity (N=686)
≥1	85 (100%)	88 (12.8%)
≥2	84 (98.8%)	142 (20.7%)
≥3	84 (98.8%)	275 (40.1%)
≥4	83 (97.7%)	337 (49.1%)
≥5	77 (90.6%)	443 (64.6%)
≥6	73 (84.7%)	497 (72.5%)
≥7	63 (74.1%)	566 (82.5%)
Area under ROC curve (95% CI)	0.88 (0.85 – 0.91)

DISCUSSION

In this secondary analysis of e-ROP data, we evaluated the prediction of the development of TR-ROP using findings of the trained readers’ evaluation of digital wide-field retinal images taken at 34 weeks of PMA or earlier, along with GA and medical status including respiratory support and the rate of weight at the first image session. We found that image evaluation findings of pre-plus disease, stage and zone of ROP, and blot retinal hemorrhage at the first image session strongly predicted the development of TR-ROP (AUC =0.82). The risk score calculated based on image findings, GA, respiratory status, and weight gain rate predicted TR-ROP well (AUC =0.88). Such a risk score could provide a useful tool for identifying highest risk infants in need of increased surveillance for detecting TR-ROP, and the lowest risk infants who might need less intensive surveillance.

In the past decade, many ROP prediction models and algorithms have been developed and demonstrated the potential to reduce the number of ROP examinations by ophthalmologists.^13;17–24 These models varied as they use different predictors for different severity of ROP (RW-ROP, severe ROP, treatment requiring ROP, etc.). In contrast to previous ROP prediction models/algorithms, our prediction model and resulting risk score is unique in that we used the findings of trained readers’ evaluation of images taken at the first image session (34 PMA weeks or earlier) for predicting the future development of TR-ROP. Using image findings for predicting TR-ROP enhances the utility of telemedicine of ROP.²⁵

We found that pre-plus or plus disease, stage and zone of ROP, and the blot hemorrhage noted in retinal images were predictive of development of TR-ROP. The higher number of quadrants with pre-plus disease was associated with a higher likelihood of TR-ROP ranging from 6% in infants with normal posterior pole, to 15%, 23%, 27% and 49% for infants with 1, 2, 3, 4 quadrants of pre-plus respectively. Interestingly, among 15 infants with image quality not sufficient for determining posterior pole abnormality, 7 (49%) infants developed TR-ROP, suggesting that infants with poor image quality should either be re-imaged to obtain acceptable quality images or have a diagnostic eye examination. The finding of ROP stage and zone in the first image session significantly predicts the development of TR-ROP, with the TR-ROP rate ranging from 4% in infants without ROP, to 9% for stage 1–2 ROP in zone II, 24% for stage 3 ROP zone II, 39% for stage 1–2 ROP in zone I, and 55% for stage 3 ROP zone I, supporting the importance of noting ROP stage and zone in the telemedicine of ROP.

In this secondary analysis, we graded the presence of various types of retinal hemorrhage in retinal images (dot, blot, pre-retinal and flame) and evaluated the association of each type of hemorrhage with TR-ROP. Although both blot and pre-retinal hemorrhages were found to be associated with TR-ROP in the univariate analysis, only blot hemorrhage was independently associated with TR-ROP (OR=3.1, p=0.005) in the multivariate analysis, indicating that blot hemorrhages should be an integral part of image evaluation when evaluating ROP characteristics.

Consistent with our previous findings from the assessment of demographic and medical status predictors of RW-ROP, this study also confirmed that low GA, need for respiratory support, and slow weight gain at the first image session were predictive of TR-ROP. Adding these easily available predictors into the prediction model significantly improved the prediction of TR-ROP beyond what was predicted by the gradings at the first image session alone (AUC improved from 0.82 to 0.88, p=0.004).

Current ROP screening guidelines recommend serial ROP examinations on premature infants at risk; however, relatively low percent of examined infants (less than 10%) require treatment.¹ A risk score for TR-ROP could provide a useful tool for stratifying at-risk infants into high risk and low risk, with high risk infants requiring serial eye examinations and low risk infants requiring less frequent eye examination or imaging. Our risk scoring system uses 6 statistically significant predictors, and the derived risk score is a good predictor for the development of TR-ROP. Among 88 infants with a risk score of 0, none developed TR-ROP. Among 276 infant with risk scores of 2 points or less, only one infant developed TR-ROP. The risk of TR-ROP steadily increased with risk score, with 2–8% TR-ROP for risk scores of 3 to 5 points, 12–27% for 6–10 points, 43% TR-ROP for risk scores of 11–13 points, up to 94% for risk scores of 13 points or more. Using risk score of 3 points or more as an “alarm” for TR-ROP, we found that the risk score had a sensitivity of 98.8% (detecting 84 of 85 infants with TR-ROP) and allow reduction for examinations in 36% infants. If this approach is validated by an independent study, risk scores could be used clinically for identifying the high risk infants that are in need of increased surveillance to detect TR-ROP.

Our risk score was developed by using the findings of retinal images taken at the first image session (34 weeks PMA or earlier). Although e-ROP study demonstrated that non-physician trained readers image grading agrees well with ophthalmologists eye examination for detecting referral-warranted ROP,⁵ the risk score developed from image evaluation findings of ROP characteristics may not be directly applicable to the findings from eye examination using indirect ophthalmoscopy. In the setting of telemedicine of ROP, an infant will undergo serial imaging over time, the risk score can be re-calculated as new images become available to monitor the change of risk of TR-ROP with referral of an infant for ROP examination when the risk score is above the threshold (e.g. 3 points). In our study, two infants had low risk score (1 point and 3 points) at the first image session but later developed treatment-requiring ROP. In each of the 2 infants, their risk score at the second image sessions of 33 and 34 PAM weeks respectively (2 days and 7 days after their first image session) was increased to 6 and 8 points respectively due to the development of stage 3, Zone I ROP (without plus disease), thus would have been alarmed indicating the need for an eye examination with detection of TR-ROP.

In summary, findings in digital retinal images (posterior pole vessel abnormalities, stage and zone of ROP, and blot hemorrhage) at 34 PMA weeks or earlier are predictive of TR-ROP. If externally validated in other cohort of infants, a risk score using these image evaluation findings along with other ROP predictors (GA, respiratory support, and weight gain rate) can be used to predict the development of TR-ROP, thus providing a useful tool for identifying high risk infants who require increased surveillance for TR-ROP or lowest risk infants who might need less intensive surveillance.