Usefulness of Children's Hospital of Philadelphia ROP (CHOP ROP) model in the prediction of type 1 ROP

Abstract

Purpose:

Children's Hospital of Philadelphia retinopathy of prematurity (CHOP ROP) model can be used to predict ROP, a leading cause of childhood blindness, using risk factors such as postnatal weight gain, birth weight (BW), and gestation age (GA). The purpose of this study was to determine the usefulness of the CHOP ROP for the prediction of treatable ROP.

Methods:

This was a prospective observational study. Babies <34 weeks of GA, BW <2000 grams, and GA 34–36 weeks with risk factors such as respiratory distress syndrome (RDS) were included; ROP screening, follow-up, and treatment were performed based on national guidelines. The average daily postnatal weight gain was measured, and the CHOP nomogram was plotted. Babies were categorized as high risk or low risk based on the “CHOP” alarm. The sensitivity and specificity of the CHOP ROP for the detection of treatable ROP were determined. In case of poor sensitivity, a new cutoff alarm level was planned using logistic regression analysis.

Results:

Of 62 screened infants, 23 infants did not fulfill the criteria of the CHOP algorithm and were excluded. Thus, in the study on 39 infants, the predictive model with an alarm level of 0.014 had 100% specificity and 20% sensitivity. With the “new” alarm level (cutoff) of 0.0003, the CHOP nomogram could detect all the infants who developed treatable ROP, that is, sensitivity increased to 100% but specificity decreased to 10.5%.

Conclusion:

The CHOP ROP model with a cutoff point (0.014) performed poorly in predicting severe ROP in the study. Thus, there is a need to develop inclusive and more sensitive tailor-made algorithms.

Retinopathy of prematurity (ROP) remains one of the most common preventable causes of childhood blindness worldwide. Over 22% of childhood blindness in India is attributable to retinal etiologies.[1] ROP is reported to occur in 21.7%–51.9% of low birth weight (BW) infants and is the most common preventable cause.[2] The serum insulin-like growth factor (IGF) spike helps in weight gain and the retinal vascular endothelial growth factor (VEGF) spike results in retinal vascular growth and maturation early in postnatal life, thus connecting ROP and weight gain together.[3,4] Indian babies with a weight gain of <21.9 g/day may be at high risk of severe ROP.[5] A well-developed postnatal weight gain-based model could potentially improve the detection of ROP with less subjectivity while greatly reducing unnecessary examinations for lower-risk infants and improving resource allocation, particularly in settings with limited resources and a high burden of ROP in India—currently in its third ROP epidemic.[6]

There are many weight gain-based models for the prediction of ROP. Weight, insulin-like growth factor 1, neonatal retinopathy of prematurity (WINROP) is a Web-based application. The deviations between expected weight (derived from control infants with no or mild ROP) and actual weight are accumulated weekly. It gives an alarm when cumulative deviations exceed a threshold. It is the most widely studied model and recognizes high-risk infants early. However, complex calculation is involved, and low sensitivity is reported in low- to middle-income countries.[7,8,9,10] According to the postnatal growth and retinopathy of prematurity (G-ROP) model, an infant qualifies for a retinal examination if any of the model components (i.e. gestational age <28 weeks; BW < 1051 g; weight gain over 3 growth periods (10–19 days: <120 g; 20–29 days:<180 g; and 30–39 days:<170 g); and hydrocephalus) are present. This algorithm is validated in a large multicenter cohort but is new and more studies are needed to assess generalizability.[11] Premature infants in need of transfusion study (PINT) ROP is a nomogram-based predictive model that considers BW, gestational age, and rate of daily weight gain but is not widely validated.[12] Retinopathy of prematurity score (ROPScore) is based on cumulative risk factors (BW, gestational age, weight gain, oxygen use, and blood transfusion) and establishes cutoff points for different stages of ROP. However, it provides a risk assessment once per child at 6 weeks of age, potentially missing infants with aggressive posterior ROP that often present earlier.[13] Colorado - retinopathy of prematurity (CO-ROP) qualifies an infant for a retinal examination if all three components (gestational age, BW, and weight gain) are present at a single time point; however, it suffers from poor generalizability.[14]

The Children's Hospital of Philadelphia ROP (CHOP ROP) model uses BW, gestational age at birth, and weight gain rate to predict the risk of severe ROP after plotting a nomogram.[15] If found appropriate for our study population, using the CHOP ROP nomogram would result in a large reduction in the number of ROP examinations compared with current screening guidelines.[16] Therefore, we conducted this study to determine the utility of the CHOP ROP model to predict treatable ROP.

Methods

This was a prospective observational study conducted from March 2019 to September 2021. It abided by the Declaration of Helsinki, and approval from the institutional review board was obtained. All infants were screened for ROP according to Rashtriya Bal Swasthya Karyakram (RBSK) guidelines with BW less than 2000 grams, gestation age (GA) less than 34 weeks, and GA between 34 and 36 weeks but with risk factors such as cardiorespiratory support, prolonged oxygen therapy, and respiratory distress syndrome (RDS).[16] Factors that could result in imprecise weight measurement, for example, hemodynamically unstable babies on mechanical ventilation, or infants with any ocular abnormality interfering with screening or treatment of ROP, were excluded. Infants with GA > 34 weeks and BW > 1800 grams cannot be plotted on the CHOP nomogram and were thus excluded.[15] Babies receiving intravitreal anti-VEGF injection were also excluded because of the “induced” delay in completion of ROP screening and the possible effect of anti-VEGF injection on infant weight gain due to systemic absorption.[17] Detailed written informed consent was obtained from the parents. Serial weight measurements were obtained every time the baby came for ROP screening. It was performed by the pediatrics follow-up team, taking all necessary precautions on a well-calibrated electronic infant weighing machine with an accuracy of +/- 5 grams. The postnatal weight gain was measured for all the eligible infants. Then, the average daily weight gain was calculated as follows:

The average daily weight gain was calculated till the baby developed treatable ROP or till complete vascularization of the retina, that is, till completion of ROP screening. Weight was recorded at the time of first contact with the baby or in the second or third week after birth. Weight in the first week was excluded owing to the common weight loss that occurs in low BW infants during this period. ROP disease was classified according to International Classification for ROP (ICROP) guidelines.[18] The CHOP nomogram was plotted for all infants. For each baby, by plotting the CHOP nomogram, the baby was placed into the high-risk or low-risk category.

These babies were followed according to RBSK guidelines and treated if they developed treatable (type 1 and aggressive ROP) ROP.[16] The sensitivity and specificity of the CHOP ROP algorithm were determined for predicting treatable ROP. It was decided before the start of the study that if found inappropriate, we will attempt to deduce a new “cutoff alarm level” point to make the algorithm more sensitive.

Data was described in terms of range, mean ± standard deviation (±SD), frequencies, and relative frequencies (percentages) as appropriate. A comparison of quantitative variables between the study groups was conducted using Analysis of variance (ANOVA). For comparing continuous data, Student's t-test of independent variables was performed. The sensitivity, specificity, and positive and negative predictive values were calculated using standard equations. The “new level” point of probability was determined using a multivariate regression equation. Receiver operating characteristic curves were plotted for conventional and “new” alarm levels.

Results

Of 150 eligible infants, 111 infants were excluded taking into consideration our exclusion criteria as shown in the study flow [Fig. 1]. Thirty-nine newborns were enrolled in the study. It is of note that 23 infants did not fulfill the criteria of being plotted on the CHOP nomogram (i.e. only infants <34 weeks of GA and BW <1800 gram) and thus could not be included [Fig. 1].

Figure 1.

Study flow

There were 16 male and 23 female babies (P > 0.05). Of the 39 enrolled babies, 20 were treated for severe ROP and 19 developed no or mild ROP. The average GA, BW, and rate of weight gain are tabulated in Table 1. GA and the rate of daily weight gain were significantly different between the two groups.

Table 1.

Average gestational age, birth weight, and rate of weight gain in the study cohort and both groups

	Total	Treatable ROP (n=20)	No/mild ROP (n=19)	P
Gestational age (weeks)	31.33±1.92	30.40±5.09	32.18±1.59	<0.001
Birth weight (grams)	1397.69±293.92	1399.03±306.29	1342.05±359.94	0.49
Rate of weight gain (grams/day)	21.23±11.76	19.14±11.34	28.15±10.06	0.01

Using an independent t-test

Of 39 infants, four (10.3%) reached the alarm level, while the remaining 35 (89.7%) did not reach the alarm level. On follow-up, it was observed that as per CHOP ROP eligibility criteria, four of four (100%) infants reaching the alarm level had developed treatable ROP. However, 16 of 35 (45.71%) infants not reaching the alarm level had also developed treatable ROP. The 2 * 2 table for the CHOP ROP nomogram as a prediction tool for treatable ROP is provided in Table 2. Consequently, the sensitivity and specificity of the CHOP algorithm were found to be 20% and 100%, respectively. The positive and negative predictive values were 100% and 54.29%, respectively.

Table 2.

2*2 tables for accuracy of the CHOP algorithm alarm levels (actual and deduced “new”)

Precision of alarm level (0.014) of the CHOP algorithm with actual treatment
	Developed treatable ROP		Total
	No	Yes
CHOP
Did not reach alarm level	19 (True negative)	16 (False negative)	35
Reached alarm level	0 (False positive)	4 (True positive)	4
Total	19	20	39
Precision of deduced CHOP alarm level of 0.0003 with actual treatment
	Developed treatable ROP		Total
	No	Yes
CHOP
Did not reach alarm level	2 (True negative)	0 (False negative)	2
Reached alarm level	17 (False positive)	20 (True positive)	37
Total	19	20	39

We inferred that the CHOP ROP algorithm had missed out on a lot of treatable ROP cases and was not a good screening tool due to poor sensitivity of a mere 20%. So, we deduced a “new alarm cutoff point” on the nomogram so that we could increase the sensitivity. For our set of infants, this cutoff came as 0.0003—this value lay in the middle of 0.001 and 0.0001 on the chop nomogram.[15] For the new alarm level (cutoff) of “0.0003” of the CHOP algorithm, of 39 infants plotted on the CHOP nomogram, 37 infants (94.9%) reached the alarm level, while the remaining two (5.1%) did not. It was observed that as per the new cutoff alarm level, 20 of 37 (54.05%) infants reaching the alarm level had developed treatable ROP. However, 17 of 35 (45.95%) infants reaching the alarm level did not develop treatable ROP. The accuracy of the new CHOP alarm level of “0.0003” with actual treatment is shown in Table 2. Therefore, the sensitivity and specificity of the CHOP algorithm were found to be 100% and 10.53%, respectively. The positive and negative predictive values were 54.05% and 100%, respectively. Thus, it was observed that, when the alarm level was lowered to 0.0003, the CHOP algorithm could detect 100% of infants who are likely to develop treatable ROP but with poor specificity and positive predictive values. The ROC curves with conventional and deduced alarm levels are shown in Fig. 2a and b, respectively. The area under the curve (AUC) with a new alarm level (0.0003) was significantly higher (0.87; 0.84; P = 0.02) than the AUC with a conventional alarm level.

Figure 2.

(a) ROC curve for the CHOP ROP for original alarm level (area under ROC curve: 0.843, 95% CI: 0.718–0.967; P = 0.026). (b) ROC curve for the CHOP ROP for new alarm level (area under ROC curve: 0.872, 95% CI: 0.740–1.000; P = 0.080)

Discussion

In this prospective observational study, we have attempted to validate the CHOP ROP model for the prediction of treatable ROP in 39 infants. We used the CHOP algorithm as it uses an easily plottable nomogram, which provided us an opportunity to “modify a cutoff alarm level,” as was conducted in a few other studies.[15,19,20,21]

The mean gestational age (31.4 weeks) and mean BW (1572.4 grams) were higher in our study than in other studies. In studies evaluating CHOP ROP validity conducted outside India, the mean GA was 27.6 ± 3.5 weeks and the mean BW was 1091.5 ± 221.3 grams. Even in a study conducted in India by Doshi et al., the mean GA was 28.5 weeks and the mean BW was 878 grams.[19] Thus, there was a substantial difference in the included infant population in our study.

The original CHOP ROP nomogram was provided by Binenbaum et al. and showed 98% sensitivity for predicting type 1 or 2 ROP and 100% sensitivity for type 1 ROP.[15] Most studies conducted outside India also showed 100% sensitivity in predicting treatable ROP.[20,21] Sun et al. conducted a retrospective study in 3624 Chinese infants and showed that infants who developed type 1 ROP were correctly predicted by the CHOP ROP model with a sensitivity of 100%, but with a low specificity (21.4%).[20] An Italian retrospective (n = 445) study by Piermarocchi et al. also showed that the CHOP ROP correctly identified all infants having any ROP or type 1 ROP with 100% sensitivity.[21]

A few studies have been conducted in India to verify the validity of weight gain-based ROP prediction models. They consistently showed poor sensitivity as summarized in Table 3. This may be because of the recognized fact that ROP develops in heavier babies in developing countries such as India, which have poor neonatal care.[6]

Table 3.

Sensitivity and specificity of various weight gain-based ROP prediction models in the Indian context

Authors	Algorithm validated	n	Sensitivity	Specificity
Sanghi et al.[9]	WINROP	70	90.32%	38.46%
Sute et al.[10]	WINROP	102	80%	80.6%
Thomas et al.[8]	WINROP	382	85.4%	36.2%
	CHOP ROP	498	54%	71.4%
	ROPScore	370	72.9%	67.3%
Kamath et al.[22]	CHOP score cutoff alarm level (0.014)	65	13.4%	100%
Doshi et al.[19]	CHOP score cutoff alarm level (0.014)	191	66.67%	74.58%
	“Deduced” CHOP score cutoff point (0.010)	191	100%	51.40%
Present	CHOP score cutoff alarm level (0.014)	39	20%	100%
	“Deduced” CHOP score cutoff alarm level (0.0003)	39	100%	10.53%

A meta-analysis performed by Athikarisamy et al. on six studies (on 2135 infants) has shown sensitivity and specificity of 95% and 52% for CHOP ROP in the prediction of treatable ROP.[23] However, they too have pointed out that studies conducted in India and other low- and middle-income countries have shown poor sensitivity. The original cutoff alarm level for the CHOP model is 0.014. Four of six studies have described better (100%) sensitivity with a “new” cutoff alarm level. The original authors, Binenbaum et al., have also described a sensitivity of 100% at an alarm level of 0.0034.[15] Doshi et al. conducted a study in Western India, which showed 67% sensitivity with a cutoff point of 0.014. Yet, when they lowered the CHOP score cutoff point to 0.010, sensitivity increased to 100%.[19] However, the cutoff point we obtained was further lower at 0.0003. At this new alarm cutoff only, we have obtained 100% sensitivity. We do not suggest that ours is the “benchmark” alarm level for the Indian scenario, but given the grave blinding implications of the missed treatment of type 1 ROP, it is appropriate to have no less than 100% sensitivity for the detection of severe ROP.

Thus, we found the CHOP nomogram in its current form to be not very useful. Although it provided a specificity of 100%, it did not make for a good screening tool as it could not detect all severe ROP cases and had a low sensitivity in the Indian infants. Still, as advocated by Binenbaum et al., it may be used to decrease the frequency of ROP screening.[15] Another problem was that the CHOP ROP model could not be applied to all babies in the Indian context because of their higher BW and gestation. In our study, 23 babies (37%) had to be excluded due to GA > 34 weeks and BW > 1800 grams and resulting failure had to be plotted on the CHOP nomogram. In addition to that, there is an absence of a fixed cutoff alarm level for the Indian babies. Even among Indian studies such as ours and Doshi et al., there is a wide variation (approximately 2 log units) between “new” calculated alarm levels.[19]

The limitation of our study is its small sample size and limited power; thus, the CHOP cutoffs cannot be generalized. Also, we have plotted the rate of weight gain on the CHOP nomogram by calculating it after the infant has completed ROP screening; thus, we assumed the weight gain rate to be constant in the first 2–3 months of life. This, however, may not be true in the real scenario. The authors, that is, Binenbaum et al.,[15] have calculated the rate of weight gain and applied the CHOP algorithm on a weekly basis.

Conclusion

We conclude that the CHOP ROP algorithm needs customization so that all babies at risk according to national guidelines can be included. In particular, the alarm point should be lowered, and range of included gestation and BW broadened to include heavier Indian babies who develop ROP. More research is required to make new or tailored algorithms based on national data. This would introduce uniformity, generalizability, and “faith” in not missing severe ROP, especially where a trained ophthalmologist or telemedicine is not available. Much research is required to address the third ROP epidemic in India as we have a unique and varied infant population, even within the country, which is at risk of ROP.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.