Discrimination of GutCheck^NEC, a clinical risk index for necrotizing enterocolitis

Abstract

Background

Better measures are needed to identify infants at risk for developing necrotizing enterocolitis (NEC) and facilitate communication about risk across transitions. Although NEC is multi-factorial, quantification of composite risk for NEC in an individual infant is not clearly defined.

Objective

This study’s objective was to describe the derivation, validation and calibration testing of a novel clinical NEC risk index, GutCheck^NEC. Individual risk factors were weighted to assess composite odds of developing NEC. GutCheck^NEC is designed to improve communication about NEC risk and coordination of care among clinicians across an infant’s clinical course.

Methods

Based on a synthesis of research evidence about NEC risk and an e-Delphi study including 35 neonatal experts, we identified NEC risk factors believed by the experts to be most relevant for a NEC risk index then applied a logistic model building process to derive and validate GutCheck^NEC. De-identified data from the Pediatrix BabySteps Clinical Data Warehouse (discharge date 2007-2011) were split into three samples for derivation, validation and calibration. By comparing infants with medical NEC, surgical NEC, and those who died to infants without NEC, we derived the logistic model using the un-matched derivation set. Discrimination was then tested in a case-control matched validation set and an un-matched calibration set using ROC curves.

Results

Sampled from a cohort of 58 820 infants, the randomly selected derivation set (n= 35 013) revealed 9 independent risk factors (gestational age, history of packed red blood cell transfusion, unit NEC rate, late onset sepsis, multiple infections, hypotension treated with inotropic medications, Black or Hispanic race, outborn status, and metabolic acidosis) and 2 risk reducers (human milk feeding on both days 7 and 14 of life, and probiotics). Unit NEC rate carried the most weight in the summed score. Validation using a 2: 1 matched case-control sample (n=360) demonstrated fair to good discrimination. In the calibration set (n= 23 447), GutCheck^NEC scores (range 0-58) discriminated those infants who developed surgical NEC (AUC=0.84, 95% CI 0.82-0.84) and NEC leading to death (AUC=0.83, 95% CI 0.81-0.85), more accurately than medical NEC (AUC= 0.72, 95% CI 0.70-0.74).

Conclusion

GutCheck^NEC represents weighted composite risk for NEC and discriminated infants who developed NEC from those who did not with very good accuracy. We speculate that targeting modifiable NEC risk factors could reduce national NEC prevalence.

Introduction

Necrotizing enterocolitis (NEC) leads to death in up to 30-50% of the premature infants who develop it,(1, 2) and population level incidence has not decreased appreciably in over a decade.(3, 4) NEC results in extremely long hospitalizations, long-term gut morbidity, prolonged indwelling central line time, and co-morbidity with central-line associated infections. Moreover, NEC accounts for nearly 20% of annual NICU spending in the US.(5) It is estimated that avoiding even a single case of surgical NEC could save up to $250,000.(6) Risk for NEC extends beyond the non-modifiable factors (low gestational age, birth weight and severe illness) to which it is often attributed. Modifiable risk factors likely play a significant role, as the highly variable NEC rates across NICUs suggest. Modifiable risk factors include the proportion and dose of human milk fed,(7-9) unit adoption of a standardized feeding protocols,(10-13) a single dose of antenatal steroids given to the mother prior to delivery,(14) avoiding histamine-blocker antagonists,(15-17) and closing a PDA with ibuprofen instead of indomethacin.(18) Packed red blood cell transfusions too are associated with NEC,(19) and some centers have dramatically reduced their NEC incidence by withholding feedings during transfusion,(20-22) even as the international debate over feeding, not feeding or restrictive feeding concomitantly with neonatal transfusions continues.(23) The causes of NEC are widely agreed to be multi-factorial, but a multi-factorial approach to understanding combined risk for NEC has not been rigorously explored or quantified. The purpose of the current study was to determine the extent to which a composite NEC risk index could accurately discriminate those infants who developed NEC from those who did not.

Our specific goal was to derive and validate a tool (i.e. GutCheck^NEC) to improve interprofessional communication about NEC risk, especially during transitions (e.g. change of shift, change in location, change in caregiver). This approach resembles the use of the Apgar score to communicate delivery room status efficiently among multiple clinicians over time. If successfully validated, predictive and clinically useful, GutCheck^NEC might similarly bring information disparately located within the medical record into a single, efficient score that can heighten clinicians’ awareness of NEC risk to support early recognition and prompt treatment. Infants with a risk score over a specific threshold would be identified as high risk and treatment could be tailored. Further, when nonspecific symptoms of NEC present, the risk score could provide context for the symptoms.

Preliminary work that led to the development of GutCheck^NEC included synthesis of the evidence available about NEC risk (24) and determination of risk factors that neonatal experts believed to be most relevant through an e-Delphi methodology, an approach to support consensus building.(25) In the e-Delphi, 35 neonatal experts (pediatric surgeons, neonatologists, NEC researchers, neonatal nurse practitioners and nurses) were recruited from across the United States and 4 other countries. Experts completed three rounds of surveys in which they were asked about the relevance of 64 proposed risk factors based on an initial evidence synthesis and invited to suggest new risk items. In the e-Delphi, 33 distinct risk items met the criteria for consensus. A thematic analysis of their comments revealed that the experts believed NEC risk to be a product of both individual neonatal vulnerability to the disease (e.g., severity of illness, biodemographic factors, extreme prematurity) and variation in care practices within and across the NICU (e.g., standardized feeding protocols, preferential feeding of human milk, availability of donor milk, antibiotic stewardship, transfusion practices and PDA management, etc.).(25) To further refine the 33 risk items to those most predictive and clinically useful, we subjected them to the logistic model building process reported here.

Methods

Data source

A de-identified data set from the Pediatrix BabySteps Clinical Data Warehouse (CDW) was used to derive the empiric weights, validate GutCheck^NEC’s discrimination of cases (NEC) from non-cases (no-NEC), and then test calibration. Because NEC affects an estimated 7% of the population of US VLBW infants,(26) it is challenging to derive the numbers of cases needed for model building using this approach from single-center studies. The BabySteps CDW is a privately-held database that includes over 925,000 admissions and represents approximately 20% of NICU admissions in the US.(27) The benefits of using the BabySteps CDW include its size, representativeness, granularity, and reliability. The database is derived from clinical notes that control the structure of the data entered through drop-down menus. Once entered, the data from over 560 fields are de-identified and extracted daily into a central repository to yield consistent and accurate data for coding and research. The Pediatrix CDW is certified annually by the Western Institutional Review Board (IRB) as de-identified for research purposes in order to carry out these analyses. A data use agreement was completed by the non-employee first author. The University of Arizona IRB approved an exemption for this study. Data abstracted from the CDW were limited to discharges from January 1, 2007 to December 31, 2011, birthweight < 1500 grams and gestational age < 36 weeks.

Definition of NEC

Clinicians entering a diagnosis of NEC into the Pediatrix database use the following standard definition for the disease. To be diagnosed with NEC, one or more clinical gastrointestinal signs must be present (e.g. bilious emesis or gastric aspirate, blood in stool without a rectal fissure, abdominal distention); and at least one radiographic sign. Radiographic findings must include pneumatosis intestinalis, pneumoperitoneum and/or hepatobiliary gas. The clinician selects either NEC-medical or NEC-surgical for the diagnosis at the time of documentation. It is possible to be coded as both. In the current study, when this occurred the infant was coded as NEC-surgical. To avoid cross-contamination of infants with isolated spontaneous intestinal perforation (SIP), infants with that diagnosis were excluded. Surgical data were evaluated for all NEC patients; those coded as NEC-medical who also had abdominal surgery or bowel perforation were coded as NEC-surgical. Using this definition, Bell’s staging was not identified and unconfirmed “suspected NEC” not included. Additional information about the pathogenesis of the disease for infants who died was not available for this analysis.

Although it would be helpful to understand if these infants died suddenly, i.e. from fulminant NEC, or died slowly, we did not evaluate that as part of the study aims. Cohort analysis of infants who have died from NEC is available in another study,(28) but not included here.

Data Preparation

We matched items from BabySteps CDW to the 33 candidate risk items from the preliminary e-Delphi, identifying proxies when an exact match was unavailable. For example, delivery room resuscitation including chest compressions or epinephrine was unavailable in BabySteps so the 5 minute Apgar was used as a proxy. Because feeding on day of life 7 and 14 were both available, a new variable was constructed denoting when human milk was given on both days to approximate exclusive human milk feeding. No data were available to determine if NICUs had adopted standardized feeding guidelines. NICU unit characteristics including patient volume, AAP level, and unit NEC incidence were used to reflect the NICU contribution to NEC risk. The NICU NEC rate for infants < 1500 grams was calculated and coded as low (<2%= 0), moderate (2-4.99%=1), average (5-7.99%=2), high (8-12%=3) or very high (> 12%=4).

Research design

GutCheck^NEC was developed and tested using a step-wise approach similar to that used to derive and validate other pediatric and neonatal intensive care risk indices (e.g., PRISM;(29) SNAP, SNAPP-II and SNAPPE-II(30)). Non-overlapping samples from the BabySteps CDW were used for the 3 steps of empiric derivation, validation and calibration. To minimize statistical selection bias that can overfit a regression model,(31) we pre-specified the variables to include as potential predictors based on the expert opinion we identified in the previous e-Delphi.(32)

To achieve sufficient statistical power, it was necessary to evaluate cases from many years and multiple centers. Once the data set was received, it was split into three sets. First, the case-control set (used for validation in step two, n=360) was removed from the total set by randomly sampling 120 NEC cases and matching each to two controls. Then, a second random sample of 60% of the remaining set was separated for derivation (n=35 013) for use in step one. The remaining 40% was used for calibration (n=23 447) in step three. Data sets used in step one for derivation and in step 3 for calibration, were not matched samples.

Step One: Empiric Derivation

To accomplish empirical derivation, risk elements from the e-Delphi were analyzed using univariate then multivariate regression. Risk factors that achieved statistical significance in the univariate analysis (p <0.01) were entered into a multivariate regression model using a backward likelihood ratio method. The likelihood ratio approach was used to accommodate the predominantly categorical nature of the data (i.e., the variable was either present or absent). Variables were entered into the model in blocks, with those reaching > 85% agreement among experts in the e-Delphi entered first, 80-85% entered second, 70-80% entered third, and 65-70% entered last. Risk factors retained in the multivariate model were retained in GutCheck^NEC. Empirical weights were derived for each item by multiplying the unstandardized beta value by 10 and rounding to the nearest integer value. Individual risk factor scores were then summed to produce a GutCheck^NEC composite score. Using this statistical approach, weights are derived only in this step and the remaining two steps (i.e. validation and calibration) test the model.(31-33) Re-estimation of the empiric weights in un-related samples in the future can evaluate persistence of the weights.

Step Two: Validation using Known Groups Comparison

A random sample of 120 NEC cases was selected to achieve 80% power to detect a moderate effect. Each case was matched to two controls by birth weight within 100 grams, gestational age within one week, and year of birth within one year. We did not match on race or gender to allow those variables to be identified as risk factors. Both cases and controls were automatically scored using the “compute function” in SPSS which calculated an item score then summed them to total the GutCheck^NEC score. Discrimination accuracy was evaluated via ROC curve analysis for medical NEC, surgical NEC and NEC leading to death. Intra-individual reliability of scoring was accomplished by having one rater score ten cases two weeks apart. This was done to ensure that when manual scoring was done, one rater was consistently yielding the same result.

Step Three: Calibration

Aside from selecting cases and matching to controls, the procedure for calibration mimicked that used for validation. Individual GutCheck^NEC scores were computed for each case in the calibration set then tested for prediction using ROC curves.

Data Analysis

GutCheck^NEC scores for cases and controls were analyzed for a difference in means using the independent samples Student’s t-test. Discrimination was measured by assessing Goodness of Fit (Hosmer-Lemeshow Chi-Square) and Area under the Curve (AUC) using ROC curve analysis.(29) An area under the curve > 0.80 was the goal for very good prediction, with a score > 0.90 the target for excellent discrimination.(33)

Results

Study Population

NICUs were broadly represented, including 284 different neonatal units. Pediatrix administers over 20% of the neonatal practice groups in the US with centers in at least 34 different states. Most infants (69%) were born in high volume NICUs, with the remaining born in low (8%) and medium volume NICUs (23%). The American Academy of Pediatrics (AAP) designates the acuity of a NICU based on the lowest gestational age infant they care for in the center, the availability of surgical services and specialty medical care. (34) In this sample, 72% of infants were cared for in Level IIIb or higher designated NICUs. Such NICUs are capable of caring for the least mature and most resource-demanding ill newborns. The NEC rate was calculated overall, by birth weight group, and by year of discharge. Unit NEC rates varied by birth weight group over the five year period. Infants weighing 401-1000 g at birth experienced NEC most often (8.2% in < 401, 3.6% in 1001-1500). See supplemental table 1 for a description of sample characteristics.

Empirical Derivation and weighting of risk factors

The derivation sample was randomly sampled to include approximately 60% of all cases < 1500 grams after the case-control set was removed, yielding 35,013 infants. Cases with missing data were eliminated listwise. Multicollinearity statistics revealed that birth weight and gestational age were highly related. For items with high collinearity, only one survived the model building process. Univariate analysis was run on the first model with all NEC as the dependent variable and all of the risk items retained in the e-Delphi using forced entry (Table 1). Significance was set at P < .01 for retention. Variables significant in the univariate analysis were entered in blocks into the multivariate logistic model. The final model (Table 2) demonstrated acceptable fit as reflected in the non-significant Hosmer-Lemeshow Chi-Square goodness of fit test (X² =14, P =.080, Nagelkerke R²=.127). Beta-weights for each item were multiplied by 10 to transform them into an integer value, and weighted items were summed for a total score.

TABLE 1.

Univariate Analysis of Risk Factors

Risk Item	β	Std.	t	sig
Gestational age (weeks)	0.01	0	2.52	0.012
Birth weight (grams)	0	0	−9.36	<.0001
Outborn	0.01	0	4.02	<.0001
Male	0.01	0	2.9	0.004
Antenatal Steroids	0	0	−1.07	0.286
African-American	0.01	0	2.89	0.004
Hispanic	0.01	0	2.07	0.039
5 minute Apgar < 7	−0.01	0	−2.2	0.028
10 minute Apgar < 7	−0.01	0.01	−1.91	0.056
Temperature < 36 degrees at one hour of age	−0.02	0	−5.64	<.0001
High frequency ventilation on day of life 7	−0.02	0	−6.41	<.0001
Indomethacin	−0.01	0	−3.85	<.0001
Indomethacin and dexamethasone	−0.02	0.01	−3.8	<.0001
Dopamine, dobutamine or milrinone	0.05	0	10.73	<.0001
Hypotension	0.01	0	1.41	0.157
Metabolic acidosis	0.02	0	6.15	<.0001
Probiotics given (formulation not specified)	0.04	0.01	3.21	0.001
Early sepsis	−0.02	0.01	−2	0.045
Late sepsis	0.03	0	5.69	<.0001
Received H2 blocker	0.07	0.01	8.28	<.0001
Patent Ductus Arteriosus, any treatment	0.01	0	3.35	0.001
Packed Red Blood Cell (PRBC) transfusion	0.04	0	14.07	<.0001
2 or more positive cultures (blood, urine, other)	0.02	0.01	3.75	<.0001
Human milk at day of life 7 and 14	0.02	0.01	3.4	0.001
Human milk at day of life 7,14 and discharge	0	0.01	−0.6	0.546
Human milk at day of life 7	0	0	−0.82	0.411
Human milk at day of life 14	0	0	−0.13	0.899
NICU volume	0	0	2.12	0.034
Unit NEC rate	0.03	0	18.75	<.0001

t depicts difference between NEC and no-NEC infants in the derivation set (n=35 013)

Table 2.

Logistic Model with Weighted Risk Items

Risk Item	β	GutCheckNEC item weight	Sig.	Odds Ratio	95% CI
Human Milk at both day 7 and day 14 of life	−0.34	−3	< 0.001	0.71	0.62	0.82
Late Sepsis	0.40	4	< 0.001	1.49	1.30	1.72
Packed Red Blood Cell Transfusion (PRBCs)	0.81	8	< 0.001	2.26	2.02	2.52
NICU NEC rate (< 2%)		0	< 0.001
NICU NEC rate (2-4.99%)	0.94	9	< 0.001	2.56	1.62	4.03
NICU NEC rate (5-7.99%)	1.61	16	< 0.001	5.00	3.19	7.84
NICU NEC rate (8-11.99%)	1.87	19	< 0.001	6.49	4.12	10.22
NICU NEC rate (>12%)	2.29	23	< 0.001	9.84	6.13	15.79
Gestational Age ≥ 32 weeks		0	< 0.001
Gestational Age 28-32 weeks	0.78	8	< 0.001	2.17	1.68	2.82
Gestational Age < 28 weeks	0.87	9	< 0.001	2.37	1.78	3.16
Black Race	0.20	2	< 0.001	1.22	1.09	1.35
Hispanic	0.17	2	0.007	1.18	1.05	1.34
Probiotics	−0.54	−5	0.005	0.58	0.40	0.85
Outborn	0.27	3	< 0.001	1.31	1.17	1.46
Metabolic Acidosis	0.29	3	< 0.001	1.33	1.18	1.50
Dopamine, Dobutamine or Milrinone combined with Hypotension	0.41	4	< 0.001	1.51	1.36	1.69
Two or more positive blood or urine cultures	0.16	2	0.038	1.18	1.01	1.37

Risk factors that survived the two-phase process and therefore were retained in GutCheck^NEC included: late sepsis, gestational age groups (< 28 and 28-31 weeks), receipt of a transfusion, the NICU-specific NEC rate, Black or Hispanic race, outborn status, multiple infections (>2 positive blood or urine cultures), metabolic acidosis and a history of severe hypotension treated with inotropic medications (dopamine, dobutamine or milrinone). Risk reducers that contributed negative scores (thereby reducing the score) included human milk fed at both day 7 and day 14 (a proxy for proportion of human milk fed) and probiotics. Of note, in earlier models, cold stress on admission was associated with a decreased risk for NEC (admission temperature 34.5-36.0 Celsius, OR 0.81, 95% CI 0.71-0.94); temperature < 34.5; OR 0.65, 95% CI .44-.97). This was an unexpected finding considering that other studies have reported higher mortality in infants that are cold when admitted to the NICU. For that reason and for clinical utility, the risk reduction from cold stress was not included in the final model. In this analysis, cold stress on admission was not associated with higher NICU mortality rates. If a unit has a NEC rate that is considered “average” at 5-7.99%, it contributes 16 points to the GutCheck^NEC score, a unit with a “high” rate of 8-11.99% contributes 19 points and a unit with a very high rate (>12%) contributes 23 points. Unit NEC rates varied widely with a median incidence of 5.2% (10^th and 90^th percentile of 2.3, 9.2%) in the derivation set across 284 NICUs. The majority of NICUs performed between the 25^th and 75^th percentile, with NEC rates ranging from 3.5% to 7.5%.

Validation

In the case-control validation set, discrimination of GutCheckNEC was determined using ROC curves. GutCheck^NEC scores ranged from 0-53 (median 32); and infants represented 129 different NICUs. The most infants that any one NICU contributed to the validation sample was 11 (3%). NEC cases were different from their birthweight, gestational age and discharge year controls. In particular, they were more likely to be outborn, die before discharge, fed human milk at discharge, receive hemodynamic medications after the first day of life (epinephrine, dopamine, dobutamine or milrinone), experience hypotension, and be cared for in units with higher NEC rates. Unit NEC rates for cases averaged 7.4 vs. 5.3 in controls. Discrimination was poor for medical NEC but fair for surgical NEC with wide confidence intervals (Figures 1a-c).

Figure 1.

a. Discrimination of GutCheck^NEC for medical NEC in the case-control validation set

b. Discrimination of GutCheck^NEC for surgical NEC in the case-control validation set

c. Discrimination of GutCheck^NEC for NEC leading to death in the validation set

Calibration

The role of the calibration step is to test the performance of the scoring in a separate non-overlapping sample. Discrimination, measured by AUC > 0.80, met the goal for predictive validity in the surgical NEC and NEC leading to death groups in the calibration set (n= 23 447). GutCheck^NEC scores (range 0-58) best discriminated those infants who developed surgical NEC (AUC=0.84, 95% CI 0.82-0.84) and NEC leading to death (AUC=0.83, 95% CI 0.81-0.85).

Discrimination of medical NEC (AUC= 0.72, 95% CI 0.70-0.74) was fair. A total score of 58 was possible in GutCheck^NEC, and sensitivity and specificity are reported for a cut point of 32 or greater. See Table 3 for a summary of discrimination performance across the validation and calibration sets.

Table 3. Discrimination of GutCheck^NEC.

	AUC	Sensitivity	Specificity	AUC	Sensitivity	Specificity
NEC Subset	Validation Set			Calibration Set Specificity
All NEC	0.67 (0.61-0.73)	0.65	0.60	0.76 (0.75-0.78)	0.62	0.76
NEC-Medical	0.57 (0.50-0.64)	0.55	0.53	0.72 (0.70-0.74)	0.54	0.75
NEC-Surgical	0.73 (0.66-0.80)	0.80	0.57	0.84 (0.82-0.84)	0.79	0.75
NEC leading to death	0.65 (0.55-0.76)	0.72	0.54	0.83 (0.81-0.85)	0.75	0.74

Data are presented as Areas Under the Curve with 95% Confidence Interval

Sensitivity is presented at a cut point score > 32

Specificity is presented at a cut point score > 32

GutCheck^NEC Scoring Consistency

GutCheck^NEC includes 10 items, each reflecting independent predictors retained in the derivation set (see Figure 3). One item reflects two risk factors: late onset sepsis and 2 or more infections after the 3^rd day of life (i.e. late onset sepsis = 4 points and 2 or more infections adds 2 points). Scoring can be done by any clinician at the bedside or automated based on occurrence of risk factors with permission of the first author. Once a risk factor occurs, a positive score is assigned and is not deducted in the future. The one exception is when the infant receives human milk enterally at both day of life 7 and day of life 14 for which 3 points are deducted from the overall score. We evaluated the extent to which a clinician consistently scored the same set of infants at two time points. Ten cases were randomly selected from the derivation set and used to evaluate intra-individual reliability. One rater first scored the cases manually and repeated the process one week later. A blinded assessor computed the intra-class correlation. Near perfect agreement was obtained between the two time points (ICC (19) = 0.97, p < .001). Studies are underway to determine consistency among different raters and variation in scores over time.

Figure 3.

GutCheck^NEC Risk Index

Discussion

In this study a new composite NEC risk index, GutCheck^NEC, was derived and tested. It demonstrated very good discrimination for the most severe NEC using a large and diverse sample. Three controversial NEC risk factors (transfusions, probiotics and institutional risk) were supported by our findings. In the e-Delphi completed prior to this study, experts had agreed that the NICU NEC rate can be used as a proxy to represent multiple practices that impact NEC risk; and a significant number of expert’s comments in that study focused on the impact of unit variation on NEC risk. Depicting NEC risk as related not only to an individual neonate’s vulnerability, but also to unit practices, was apparent in the heavy weighting of unit NEC risk in the composite score. Thus, in this study, an infant’s NEC risk reflected characteristics of NICU care as well as other risk factors (illness severity, prenatal risk, birth weight, very low gestational age, etc.).

We speculate that the pathogenesis of NEC, while yet controversial and likely a result of multiple pathways, is impacted by treatment practices. In a compelling discussion about national will to cut the NEC occurrence in half, Christensen, Gordon, and Besner argued that four key evidence supported practices could reduce NEC in the US today. They include: 1) practicing as a group (i.e. by adopting standard approaches to transfusion management, feeding and PDA management), 2) promoting human milk feeding to increase both the proportion and dose of human milk infants receive, 3) adopting a standardized feeding advance based on gestational age and including a plan for management of intolerance, and 4) liberal use of glycerin to promote motility and reduce intolerance.(13) It is possible that NICUs with low NEC rates have adopted evidence for exclusive human milk feeding, transfusion management, and standardized feeding guidelines in total, to moderate an individual vulnerability risk that infants present. Conversely, failure to adopt such prevention practices with high reliability may relate to higher NEC rates. Part of the contribution of institution to NEC risk may reflect the patient population and obstetric practice as well as neonatal practices, in particular use of antenatal steroids and breastfeeding promotion. Future studies are needed to identify the role of NICU variation on NEC risk. Even so, some NICUs have reduced their rates well below 3% (9, 10, 12, 22) and others are getting near to zero NEC occurrence. (20)

Risk factors that were not retained in the final GutCheck^NEC but may be moderated by medical management included: patent ductus arteriosus, intrauterine growth restriction and use of Histamine-2 blockers. PDA management is controversial but a meta-analysis supports less NEC when ibuprofen instead of indomethacin is used for medical closure (6 studies, n=166; typical RR 0.44, 95% CI 0.23, 0.82; NNTB 7, 95% CI 4 to 25).(18) In the Pediatrix group, the Olsen growth curves are used to classify IUGR. Yet reporting of IUGR was not consistent using the diagnosis field, accounting for missing data and eliminating this risk item from analysis. H₂ blockers were given to very few patients, yielding a statistically insignificant impact on NEC risk, contrary to other study findings.(16, 35) NICU volume was suggested as a risk factor by an expert in the prior e-Delphi and has been shown to impact other neonatal morbidities like Chronic Lung Disease (CLD), severe Intraventricular Hemorrhage (IVH) and Retinopathy of Prematurity (ROP). Although NICU volume did not survive the model building process when the unit’s NEC risk was considered, it was statistically significant before unit NEC risk was considered. Outborn infants were more likely to develop NEC in this sample than inborn. However, given the variables available in the database it is unclear if such infants were transferred to a higher level of care when they developed NEC or were transferred earlier or later in the clinical course.

Probiotics independently predicted approximately 40% reduced risk (OR 0.58, 95% CI 0.40-0.85) even though very few infants received them (n=249). Probiotics have been demonstrated in a recent meta-analysis to reduce NEC incidence and severity by up to 65%, but optimal protection requires bifidobacterium as part of the formulation.(36) In premature infants, establishing normal microbial gut flora is compromised by admission to the NICU, antibiotic exposure, limited bacterial variability due to prematurity,(37) and a lack of human milk.(7, 38, 39) Probiotics support the establishment of bacterial diversity within the premature gut and prevent the overgrowth of pathogenic bacteria; yet their routine use has been limited by a concern for safety of delivering live bacteria into a vulnerable premature infant and the lack of a FDA approved formulation in the US.(40)

Antibiotic exposure was not explicitly evaluated in this analysis, yet the impact of minimal bacterial diversity on the premature gut was reflected by inclusion of late onset sepsis and multiple infections in the final score. Use of antibiotics reduces biodiversity in the microbiota, delays colonization with beneficial flora and promotes overgrowth of pathogenic species.(37, 41-44) (45) At least 3 cohort studies support the role of an initial empiric antibiotic therapy course beyond four days on increased risk for NEC.(44-46) Alexander and colleagues reported a threefold increased odds for NEC when total antibiotic exposure exceeded 10 days.(46) Reducing late onset sepsis associated with central lines is a national priority, and a target for improvement that could also impact NEC. At a minimum, reducing NEC decreases the number of central line days and limits exposure to pathogens causing CLABSI. Similarly, promoting feeding progression through adoption of standardized feeding guidelines also reduces the number of central line days and days to full feedings.(10, 47, 48)

The impact of transfusions and transfusion-related practices on NEC needs further exploration. Anemia, common in this population, compromises mesenteric blood flow to cause intestinal hypoxia and mucosal injury. The relationship between NEC and transfusion is hypothesized to relate to a reperfusion injury in a hypoxemic gut.(49, 50) In studies using near infrared spectroscopy to evaluate perfusion during transfusion, Marin et al. identified infants who developed transfusion associated NEC had wider variation in oxygenation during the transfusion and received larger blood volumes than their no-NEC counterparts.(51) Singh and colleagues found that anemia in the 96 hours preceding NEC was an important risk factor for NEC.(52) Observational studies support withholding feedings during transfusion, especially if the feedings are not human milk.(19, 21, 50, 53) Along that line, quality improvement projects report significant decreases in occurrence of NEC, in particular surgical NEC when feedings are withheld.(20, 22) We did not limit our analysis to transfusion within 48 hours of NEC. However, the odds ratio we report demonstrating an association between NEC and transfusion in 35 013 infants (OR= 2.26, 95% CI 2.02, 2.52) approximates that reported in the meta-analysis of observational studies by Mohamed and Shah (3 studies, pooled OR=2.48, 95% CI 1.97, 3.12, I² = 0 %).¹⁹

While the first study of its kind that identified a novel composite risk score for NEC, limitations exist. Although retrospective data typically may be of low quality, missing or not coded exactly as an investigator would desire; our experience with the BabySteps CDW yielded high quality, mostly granular data that we were able to code to fit the risk items identified in the previous eDelphi study. This study design allows only for assessment of risk factors associated with NEC, not causation; but this is arguably the best approach to study disease risk. Proportion and dose of human milk fed was not evaluated due to the number of infants included in the sample and the way the relevant information was structured in the database. The human milk feeding variable was constructed as a proxy to approximate exclusive human milk feeding. It is arguably an inadequate representation of exclusive feeding of human milk but given the limitations of this study, it was retained and did confer a reduced risk. Future analyses need to consider the impact of feeding practices around transfusion.

Future research is needed to analyze unit variation in NEC rates in depth. Specifically, research is needed to explore the extent to which adoption of standardized feeding practices,(11) proportion and dose of human milk fed,(8, 54) availability of donor milk,(55) varying transfusion practices,(51) management of central catheters to limit bloodstream infections, length of initial course of antibiotics for empiric therapy (length of first course of antibiotics when blood culture is negative),(44) use of histamine blocker therapy(16) and treatment of PDA (56) contribute to unit NEC incidence. Understanding better the interplay of these variables may contribute important information about practices that protect against or contribute to NEC. To enhance the clinical utility of GutCheck^NEC, further refinement is needed and is currently underway.

Conclusion

This study derived, validated and calibrated a novel risk score for NEC, GutCheckNEC, building on prior work including evidence synthesis and qualifying expert perspectives about NEC risk. GutCheckNEC demonstrated the best prediction for NEC leading to death or NEC leading to surgery in the large calibration set. Of the risk items persisting in the score, the influence of the unit NEC rate carried the most weight, reflecting broad variations in NEC rates. For clinical use, refinement of GutCheck^NEC is ongoing. It holds promise to communicate NEC risk status across transitions in care as neonatal abstinence scores and Apgar scores do currently. While NEC prevention and early recognition occurs on an individual patient level, this supports unit level prevention practices to reduce NEC, if organizations change the standard of care for all infants rather than tailoring care case-by-case directed by an individual score. For example, if units understand that their own very high NEC rates are outside the norm and seek to adopt practices shown effective in low risk NEC units (e.g., instituting standardized feeding guidelines, making donor human milk more available and providing human milk as the only acceptable choice), their outcomes may improve. Even so, the GutCheck^NEC risk score may be used by clinicians in its current form to put nonspecific symptoms into context and communicate clearly about early symptoms to support early recognition.

Figure 2.

a. Discrimination of GutCheckNEC for Medical NEC in the calibration set

b. Discrimination of GutCheckNEC for surgical NEC in the calibration set

c. Discrimination of GutCheckNEC for NEC leading to death in the calibration set