Evaluation of the economic impact of modified screening criteria for retinopathy of prematurity from the Postnatal Growth and ROP (G-ROP) Study

Abstract

Importance

The Postnatal Growth and Retinopathy of Prematurity (G-ROP) Study showed that addition of postnatal weight gain to birth weight and gestational age detects similar numbers of infants with ROP, but requires examination of fewer infants.

Objective

To determine the incremental cost-effectiveness of screening with G-ROP compared to Conventional Screening.

Design, Setting and Participants

We built a microsimulation model of a one-year US birth cohort <32 weeks gestation, using data from the G-ROP study. We obtained resource utilization estimates from the G-ROP dataset and from secondary sources, and test characteristics from the G-ROP cohort.

Results

Among 78,281infants nationally, screening with G-ROP detected approximately 25 additional infants with Type 1 ROP. This was accomplished with 36,233 fewer examinations, in 14,073 fewer infants, with annual cost savings of approximately $2,931,980 USD through hospital discharge.

Conclusions

Screening with G-ROP reduced costs while increasing the detection of ROP compared to current screening guidelines.

INTRODUCTION

Retinopathy of prematurity (ROP) is one of the leading causes of severe childhood visual impairment in middle- and high-income countries. Infants who develop a significant severity of disease known as Type 1 ROP have a more than 50% likelihood of unfavorable visual outcome (defined functionally as visual acuity four standard deviations below the mean, or structurally as posterior retinal fold or detachment involving the macula) (1). However, with treatment, this probability can be reduced substantially (2, 3).

ROP is diagnosed almost exclusively in infants who are born at gestational age below 32 weeks or birth weight below 1,500 grams, but does not develop for several weeks after birth. Serial retinal examinations with indirect ophthalmoscopy can reliably detect Type 1 disease in high-risk infants in time for intervention to take place. Current consensus recommendations by professional societies recommend screening of all infants born at less than 1,501 grams, or less than or equal to 30 weeks, or those between 1,500 and 2,000 grams with hypotension or substantial oxygen exposure (4).

Although the currently recommended guidelines have high sensitivity for detection of ROP, their specificity is low, and the great majority of screened infants never develop ROP that requires intervention. Over the last 15 years, several predictive models have suggested that the addition of suboptimal postnatal growth to birth weight and gestational age maintained sensitivity of screening criteria while substantially improving specificity, but these studies were based on relatively small cohorts (5–10). Recently, the Postnatal Growth and Retinopathy of Prematurity (G-ROP) Study examined a cohort of 7,483 preterm infants to develop a predictive model that included such modified screening criteria, with resulting sensitivity of 100% and specificity of 32%, both superior to traditional birthweight and gestational age screening alone in the study cohort (11). Given the substantial reduction of the number of infants requiring serial ophthalmological examinations using these criteria, we undertook a study to determine the economic implications of adopting such a screening strategy.

METHODS

Study Design and Model Specification

We developed a decision analytic microsimulation model to evaluate the proposed G-ROP revision to current screening criteria for ROP, using TreeAge Pro Suite software (TreeAge Software Inc 2017, Williamstown, MA). The model, presented in schematic form in Figure 1, generated estimates of resource utilization and outcomes associated with ROP for a hypothetical one-year national birth cohort under various assumptions of the criteria employed for screening.

Figure 1.

Decision tree model for the base case (Discharge) analysis

We compared two approaches to screening. The first comparator (Conventional Screening, CS) was based on the current guideline of the American Academy of Ophthalmology and the American Academy of Pediatrics, which recommend screening of infants with birth weight 1,500 grams or less, or gestational age at birth 30 weeks or less (4). The second comparator (G-ROP) incorporated the G-ROP criteria, in which infants are screened if they meet any one of the following criteria: gestational age at birth less than 28 weeks; or birth weight less than 1051 grams; or weight gain of less than 120 g, 180 g, or 170 g during ages 10 to 19, 20 to 29, or 30 to 39 days, respectively; or hydrocephalus (11). Of note, a subjective third criterion is presently used clinically, in which infants with birth weight between 1,500 and 2,000 grams may receive retinal examinations if they have had an unstable clinical course in the judgment of the treating neonatologist (4). For a valid comparison to be made among the two screening approaches, a subjective criterion also could be added to the G-ROP criteria. However, because this criterion is subjective and nonspecific, it was not considered as part of either the current CS guidelines or the G-ROP criteria in this analysis.

The model was formulated in two versions. In the first phase (“Discharge”), we sought to minimize model assumptions by limiting model efficacy inputs to those available in the G-ROP database. This version analyzed only short-term outcomes through hospital discharge, and included screening and acute treatment costs, and the number of cases of treatable ROP detected. In the second phase (“Lifetime”), we employed literature sources to extend these results to the lifetime horizon, in order to include severe visual impairment and quality of life, as well as broader societal estimates of costs.

Description of Primary Analysis (Discharge)

Framing

The planned main outcome for the primary analysis was the incremental cost (ΔC) incurred by a proposed G-ROP Screening strategy relative to a Conventional Screening strategy, divided by its incremental effectiveness (ΔE). The resulting incremental cost-effectiveness ratio (ΔC /ΔE) represented the cost-effectiveness of G-ROP screening, expressed in terms of the additional cost per additional infant with Type 1 ROP detected.

Secondary outcomes included the number of infants screened and the number of examinations performed annually in the United States, as well as the annual number of false positive screening results (those infants without Type 1 ROP who were selected for examination by each model) and the number of false negative screening results (those infants with Type 1 ROP who were missed by each model).

The primary analysis was performed from the perspective of a third-party payer, in which only direct medical costs were included, and with a time horizon to first discharge home from hospital.

Institutional review board approval was obtained and a waiver of consent was granted for the collection of deidentified data in the original study cohort at all participating hospitals.

Model Inputs: Effectiveness

Inputs for effectiveness are provided in Table 1. Effectiveness in our model was expressed as the proportion of cases of Type 1 ROP detected, which were defined as those diagnosed in the large, multi-center G-ROP cohort as meeting the Early Treatment of Retinopathy of Prematurity (ETROP) Study Type 1 ROP criteria (2). The gestational-age-specific probability of developing Type 1 ROP was estimated directly from its incidence in the G-ROP study database (12).

Table 1.

Estimates of population, efficacy and resource use

Input	Value	Type	Range or Parameter	Source
Estimates of population, efficacy and resource use
Sensitivity
G-ROP	100.0 %	Beta distribution	n=4590 r=4589	(11)
CS a	99.4 %	Beta distribution	n=459 r=456	(11)
Specificity
G-ROP	32.3 %	Beta distribution	n=7024 r=2269	(11)
CS a	11.8. %	Beta distribution	n=6197 r=827	(11)
Gestational Age (weeks)	22–32	Empirical distribution a		(19)
Birth Weight by GA	400–5000	Empirical distribution a		(19)
ROP Incidence by GA	0 – 0.31	Probability table a		(11)
Time for Examination
MD	14 (SD 7.7)	Normal distribution	mean=14 SD=7.7	Observation d
RN	15.2 (SD 7.9)	Normal distribution	mean=15.2 SD=7.9	Observation d
Coordinator	21.9 (SD 8.0)	Normal distribution	mean=21.9 SD=8.0	Observation d
Number of examinations
With ROP	6.1 (SD 2.9)	Normal distribution	mean=6.1 SD=2.9	(11)
Without ROP	2.6 (SD 1.5)	Normal distribution	mean=2.6 SD=1.5	(11)
Probability of laser photocoagulation
If Type 1 detected	1.0	Point estimate	NA	Assumption
If Type 1 missed	0	Point estimate	NA	Assumption
Probability of retinal surgery
If Type 1 detected	0.064	Point estimate	NA	(2)
If Type 1 missed	0.43	Point estimate	NA	(22)
Discount rate	0.03	Point estimate	0.015 – 0.045	(20)
Utility
Good visual outcome w/o ROP	0.97	Point estimate	NA	(28)
Poor visual outcome with ROP	0.27	Point estimate	0.14 – 0.41	(28)
Good visual outcome with ROP	0.87	Point estimate	0.44 – 1.00	(28)
Probability of poor visual outcome
Without ROP	0	Point estimate	NA	Assumption
With ROP detected and treated	0.143	Point estimate	0.143 – 0.198	(2)
With ROP not detected or treated	0.643	Point estimate	NA	(3)
Estimates of unit cost b
Hourly personnel cost
Ophthalmologist	254	Point estimate	127 – 381	(15)
RN	48	Point estimate	24 – 72	(14)
Coordinator	22	Point estimate	11 – 33	(14)
Retinal surgery	13,343	Point estimate	6,672 – 20,015	(16)
Laser photocoagulation	3,114	Point estimate	2,180 – 4,048	(16)
Lifetime cost of poor visual outcome	791,432	Point estimate	554,002–1,028,862	(29)

^aConventional Screening

^bFull details in Supplemental Tables

^cValues are in 2017 US dollars

^dObservational study ancillary to current project (13); details in methods section above

In the G-ROP clinical study, the screening criteria had sensitivity 100% (95% CI, 99.2%−100%) and specificity 32.3% (95% CI, 31.2%−33.4%) for the detection of Type 1 ROP. In the same population, Conventional Screening criteria had sensitivity 99.4% (95% CI, 98.1%−99.8%) and specificity 11.8% (95% CI, 11.0%−12.6%) (11).

Model Inputs: Resource Use and Costs

Estimates for resource utilization and costs are provided in Table 1. Prior to the start of the G-ROP Study, resource utilization data were collected and assessed on a sample of 100 infants undergoing ROP examinations at three hospitals in Seattle and Philadelphia. No changes in non-ROP-related resource utilization were associated with ROP examinations, including changes in respiratory support, nutrition, radiograph usage, laboratory tests, and non-ophthalmological surgeries and procedures (data not shown). Therefore, we proceeded under the assumption that the only differences in resource utilization prior to discharge for ROP evaluations were those related to screening or treatment of ROP. We estimated the frequency of ophthalmological examinations per patient from G-ROP case report forms, which collected data on all eye examinations.

Because neither professional nor institutional billing reliably reflect the personnel effort, we estimated professional time input in an observational time and motion study of individuals involved in care of infants undergoing ROP diagnostic examinations (13). Ophthalmologists, neonatal nurses, ophthalmic technicians, and ROP coordinators were timed using digital timers and standardized data collection forms for work completed at four neonatal intensive care units and two outpatient ophthalmology clinics in Philadelphia and San Francisco between February and December 2014. Primary outcomes were the overall and subtask times per infant. Seven pediatric ophthalmologists and eight ophthalmic technicians were timed performing 303 inpatient and 37 outpatient ROP exams (Table 1).

Hourly time costs of physicians, registered nurses, and administrative support staff for examinations were calculated from the Doximity physician salary survey and U.S. government data, respectively (14, 15). A 30% fringe rate was added to these values to account for institutional overhead costs. Because infants were assumed to be resident in the NICU and therefore subject to bundled institutional reimbursement, a separate institutional technical fee for these examinations was not included. Costs for laser photocoagulation procedures and retinal procedures were derived from previously published micro-costed estimates by our group (16).

Total costs were calculated as the product of the resources used and the unit prices associated with those resources. Although incurred between 2006 and 2012, all costs are expressed in this report in 2017 US dollars (USD). Where necessary, we converted costs from other dates using the Personal Consumption Expenditure Health price deflator from the Center for Medicare and Medicaid Services (17, 18). Because the time horizon for the primary analysis was less than one year, discounting was not undertaken in the primary analysis.

Model Inputs: Population Characteristics

We used data from the U.S. National Center for Health Statistics to assign the frequency of birth at each week of gestational age (eTable 1), and the distribution of birth weights within each week of gestational age (eTables 2a – 2k) (19).

Uncertainty and Heterogeneity

In order to reflect the heterogeneity of the population to whom the results will be applied, we conducted a computer microsimulation of a one-year United States birth cohort, in which each infant was assigned a birthweight and gestational age based on empirical data for infants between 22 and 32 weeks from the 2017 US census (the most recent year available) (19), followed by a risk of ROP based on these characteristics. Infants then proceeded through the model, one infant at a time, and were screened and accrued costs and outcomes according to their baseline characteristics and the probabilities of detection encountered in the model. At the end of the cohort, the average cost and effectiveness across all infants, and the incremental cost-effectiveness ratio, was calculated.

We repeated this simulation with 1000 cohorts, corresponding to 1000 years. Sampling uncertainty was assessed for sensitivity, specificity, number of examinations, and professional time by entering these into the model as distributions, the details of which are provided in Table 1. At the start of each cohort, the model drew randomly from each of these distributions, and that estimate was employed for all infants in that year.

We also performed deterministic sensitivity analysis, in which a specific input to the model was changed through the upper and lower range of plausible values, and the entire simulation re-run using that value, in order to determine the impact on the resulting costs and effectiveness. Best-case and worst-case scenarios were implemented by varying multiple inputs simultaneously.

Description of Secondary Analysis (Lifetime)

The Lifetime analysis sought to optimize generalizability of the results and the ability to compare them to other health care programs, and thus differed in several ways from the primary model. First, it employed a societal perspective, in which all costs were considered, including family out-of-pocket expenses, productivity (wage) losses to both the patient in later life and to the infant’s family, and educational and other expenditures by non-medical agents. These costs were considered over a lifetime time horizon rather than to first discharge home. Visual outcomes were converted to utilities, or the individuals’ preferences for living with either normal or poor visual outcomes, and the results expressed as quality adjusted life years, defined as the utility multiplied by the average life expectancy in a given state of visual function. Costs were discounted at 3% per annum, and this rate was varied in sensitivity analysis (20). Other aspects of the model were similar to those in the primary analysis.

RESULTS

Each microsimulation run included a one-year cohort of 78,281 infants (19). Screening with G-ROP, compared with Conventional Screening (CS), detected approximately 25 additional infants with Type 1 ROP (Table 2). This improved detection was accomplished despite 36,233 fewer examinations being performed, in 14,073 fewer infants, and at an annual cost savings of approximately 2,931,980 USD in the short run and 12,873,157 USD in the long run. These savings correspond to 9.5% and 2.6%, respectively, of the total annual relevant ROP-related expenditure. The lower costs and higher effectiveness of G-ROP screening indicate that it is a “dominant” option, compared to Conventional Screening (Table 3).

Table 2.

Details of Screening Performance through Discharge in United States Birth Cohort ^{a b}

Parameter	Conventional Screening	G-ROP Screening
Total Cost	$ 30,885,069	$ 27,953,089
Type 1 ROP detected	3,948	3,973
Type 1 ROP missed	26	1
Eligible for serial examination	68,331	54,258
Not eligible for serial examination	9,950	24,023
Number of examinations	189,220	152,987
No Type 1 ROP, Flagged by model	64,383	50,285
No Type 1 ROP, Not flagged	9,924	24,022

^aThrough 32 weeks gestational age, inclusive n = 78,281

^bValues rounded to 0 digits

Table 3.

Calculation of Point Estimate of Cost-effectiveness for Primary Outcome, Dollars per Case of Severe Visual Impairment Prevented Through Discharge^a

Comparator	Cost ($) C	Incremental Cost ΔC	Effectiveness c E	Inc Effectiveness ΔE	ICER d ΔC/ΔE
G-ROP	27,953,089	−2,931,980	3,973	25	DOMINANT
CS	30,885,069		3,948

^aValues rounded to 0 digits

^bConventional Screening

^cCases of Type 1 ROP detected

^dIncremental cost-effectiveness ratio

Results for the probabilistic sensitivity analysis, reflecting sampling uncertainty for all the distributional inputs listed in Table 1, are shown in Figure 2. Each dot on this cost-effectiveness scatterplot represents the incremental cost and effectiveness of one of the 1,000 replications of our one-year cohort of patients. Of these, 97.2% are in the right lower quadrant, with lower costs and higher effectiveness, indicating a 97.2% probability that G-ROP screening is a dominant option. A similar result is depicted in the Cost-Effectiveness Acceptability Curve in eFigure 1, which plots the proportion of replications that would be considered “cost-effective” for a decision maker who had the willingness-to-pay (WTP) for one additional case of ROP detected listed on the horizontal axis (21). In this plot, G-ROP is a preferred option even at a WTP of 0, and rapidly approaches 100% probability of desirability at higher WTP.

Figure 2.

Differences in mean costs and effects between Conventional Screening and G-ROP screening cohorts for the base case (Discharge), cost per case of Type 1 ROP detected. Each point represents one run of the simulation model as described in the text. Ellipse represents 95% confidence bound for joint distribution of cost and effectiveness.

Findings from deterministic sensitivity analyses are presented in eTable 3. Despite varying the cost inputs over a very broad range of 70% to 130% of the baseline value, both individually and as a group, there was minimal change in the proportion of replications in which G-ROP Screening was a dominant option. Similarly, we substituted the progression to unfavorable structural outcome seen with cryotherapy in the original CRYO-ROP trial (22) for the superior efficacy in the intervention arm of the ET-ROP trial (2), as well as for a hypothetical lower progression than ET-ROP, neither of which resulted in a probability of dominance below 90%. Finally, we varied all of the above inputs simultaneously, in best and worst-case scenarios, for which the probabilities of dominance were 97.2% and 90.8%, respectively.

The Lifetime model showed similar results with a lifetime time horizon and societal perspective, with G-ROP simultaneously yielding lower costs and more quality adjusted life years than the conventional screening strategy, at a very high probability of 98.8% in probabilistic sensitivity analysis (eTable 4). The direction of this result was again robust to changes in input assumptions (eTable 3).

DISCUSSION

In this microsimulation cost-effectiveness study of a US birth cohort born between 22 and 32 weeks gestational age, implementing screening criteria using the G-ROP model was an economically dominant strategy. Identifying infants as eligible for ophthalmological screening for ROP using the G-ROP screening criteria, which incorporate postnatal weight gain and hydrocephalus in addition to gestational age and birth weight criteria, increased the number of infants identified with Type 1 ROP while simultaneously reducing the number of examinations and decreasing costs. The results were robust to adjustments in the underlying assumptions about efficacy and costs.

Although several modeling analyses of the economic implications of ROP management have been reported, these compared either interventions (such as treatment at an earlier stage of retinal pathology), telemedicine screening devices, or combinations of screening and intervention (16, 23–27). To our knowledge, no studies have specifically compared the cost-effectiveness of one of the newer, weight-velocity based screening algorithms with conventional screening.

This study has several notable strengths. First, we incorporate the results of the development of a revised screening approach in a large dataset. The analysis considers both the heterogeneity of the population of eligible infants, by assigning birth weight, gestational age and gestational-age-specific ROP incidence to individual infants, as well as sampling uncertainty related to important parameters such as sensitivity and specificity, using distributions that reflect the variance in the studies that generated the estimates. Moreover, the analyses were designed to simultaneously address the cost-effectiveness of G-ROP compared to conventional screening, as well as the national implications in terms of numbers of infants exposed to screening, and the numbers screened who have little to no likelihood of developing serious ROP. Such explicit national estimates of the burden of screening will hopefully be of use in policy discussions of recommendations for screening strategies.

Certain limitations must be noted. First, although the initial G-ROP dataset included a large sample of infants, the approach has not to date been independently validated in a separate clinical sample, which is necessary prior to clinical adoption. There is a risk that overfitting to the current data set may have resulted in an over-estimation of sensitivity and specificity, although the large development cohort minimizes this possibility. The economic model incorporates the sampling uncertainty in both of these test characteristics, but it does not consider the possibility of systematic bias in that distribution. Validation in a prospectively collected U.S. dataset is underway now, and the economic model will be re-run in this sample when available. Even with this revalidation, the generalizability to developing health care systems will remain uncertain until retested in those settings, where the characteristics of infants developing severe ROP differs considerably, and the G-ROP criteria may not be reliable. Second, in the absence of reliable national estimates, we applied the gestational age-specific incidence of ROP from the G-ROP study to the national cohort in our model. Changes in incidence will not alter the sensitivity or specificity of the model, but they could potentially alter the economic implications. Third, while in the simulation the use of the G-ROP criteria increased detection of Type 1 ROP by 25 cases per year compared to conventional BW and GA criteria, these cases are likely being captured in practice by use of the subjective third criterion of an unstable clinical course in the judgment of the neonatologist, which was not included in the model due to the infeasibility of quantification. In contrast, the G-ROP criteria had 100% sensitivity without the need for a subjective criterion. Therefore, even if such a criterion were to be added for both strategies and consequently sensitivity increased to 100% for conventional screening, the study conclusions would not change, as the G-ROP criteria would still demonstrate “weak dominance,” in which costs are lower but outcomes the same. Finally, we note that there are undoubtedly missed appointments and delayed examinations. These are included in the numbers of examinations estimated in the retrospective G-ROP dataset and should be similar in both groups, and variations are included as these were entered into the model as distributions. If the results are applied to a population that has a very different rate of compliance, however, the impact of screening might differ as well.

CONCLUSIONS

The G-ROP modified screening criteria are a dominant, economically desirable strategy when compared to the conventional screening. If validated and subsequently applied in clinical practice, their use would improve detection of treatment-requiring ROP while simultaneously greatly reducing the number of infants receiving retinal examinations and the number of examinations being performed, with resultant cost savings.

ABBREVIATIONS

BW

Birth weight

CI

Confidence interval

CS

Conventional screening

GA

Gestational age

G-ROP

Growth for Retinopathy of Prematurity Study

ROP

Retinopathy of prematurity

PMA

Post-menstrual age

QALY

Quality adjusted life years

USD

United States dollars

WTP

Willingness to pay