A Neonatal Sequential Organ Failure Assessment Score Predicts Mortality to Late-Onset Sepsis in Preterm Very Low Birth Weight Infants

Abstract

Objective

An operational definition of organ dysfunction applicable to neonates that predicts mortality in the setting of infection is lacking. We determined the utility of an objective, electronic health record (EHR)-automated, neonatal sequential organ failure assessment (nSOFA) score to predict mortality from late-onset sepsis (LOS) in premature, very low birth weight (VLBW) infants.

Methods

Retrospective, single-center study of bacteremic preterm VLBW newborns admitted between 2012–2016. nSOFA scores were derived for patients with LOS at multiple timepoints surrounding the sepsis evaluation.

Results

nSOFA scores at evaluation and at all points measured after evaluation were different between survivors and non-survivors. Among patients with an nSOFA score of > 4, mortality was higher at evaluation (13% vs 67%, p<0.001), +6 hours (15% vs 64%, p=0.002), and +12 hours (7% vs 71%, p<0.001) as compared to patients with a score of ≥4. Receiver operating characteristics area under the curve was 0.77 at evaluation (95% CI 0.62–0.92; p=0.001), 0.78 at +6hrs (0.66–0.92; p<0.001) and 0.93 at +12 hrs (0.86–0.997; p<0.001).

Conclusions

The nSOFA scoring system predicted mortality in VLBW infants with LOS and this automated system was integrated into our EHR. Prediction of LOS mortality is a critical step towards improvements in neonatal sepsis outcomes.

Introduction

Neonatal sepsis is a common, deadly problem with global impact including significant morbidity and mortality, even in high-resource countries(1, 2). The greatest incidence and impact of neonatal sepsis is seen in the smallest, most prematurely-born infants. In a cohort of 5,100 extremely preterm infants (born before 29 completed weeks of gestation), 34% experienced late-onset sepsis (LOS, a positive blood culture with a bacterial or fungal organism after 72 hours of life), which was associated with 18% mortality(3). LOS survivors demonstrate significant neurodevelopmental impact, which is superimposed on the impact of premature birth, and extends into the second decade of life(4, 5).

The distinction of infection from sepsis is not widely recognized in the NICU. In contrast to sepsis definitions in adults and children, definitions of sepsis commonly used in neonatology are variable and heavily predicated on the isolation of pathogens from blood and/or the associated length of prescribed antimicrobial treatment(6). However, establishing the definition of sepsis on bacteremia and the accompanying treatment alone will not further advance either research or clinical decisions to reduce mortality(7). The definition of sepsis is “life-threatening organ dysfunction caused by a dysregulated host response to infection”(8). Organ dysfunction indicates a pathobiology more complex than simply infection plus an accompanying inflammatory response. The presence of life-threatening organ dysfunction is demonstrated using a sequential organ failure assessment (SOFA) to determine risk of ICU admission or mortality(8). Because admission to a NICU for preterm infants is not optional, ICU admission is not applicable in this population. Therefore mortality, the most severe outcome, must be chosen as the initial outcome of interest. To define sepsis in neonates therefore requires an operational definition of organ dysfunction applicable specifically to this population (neonatal SOFA; nSOFA) that predicts mortality in the setting of presumed infection. This deficiency specifically prevents the establishment of a consensus definition for neonatal sepsis that is a prerequisite for improved sepsis outcomes. While defining infection is not the objective of this work, identification of patients with infection that have an increased risk of mortality based on objective organ dysfunction is the goal and is the prerequisite to redefining sepsis in this population.

We recently showed the progression of organ failure in neonates with lethal LOS in a large retrospective cohort(9). The need for mechanical ventilation, oxygen requirement, requirement for cardiovascular support in the form of vasoactive drugs, and the prevalence of thrombocytopenia all significantly increased during the progression to mortality with LOS. Guided by those data, we developed and tested an objective, electronic health record (EHR)-automated, nSOFA scoring system to predict mortality from LOS in premature, very low birth weight infants at our center.

Methods

Study population

This retrospective case-control study was approved by the University of Florida (UF) Institutional Review Board (waiver of consent) prior to the collection of any data. Following IRB approval, an integrated data repository of all VLBW newborns born at <33 weeks gestation (by obstetric estimate) admitted to the UF Health, level IV neonatal intensive care unit (NICU) between January 2012 and September 2016 without patient identifiers was created via an honest broker. This repositiory included all data recorded in the EHR and was used for this work. Infants with late-onset sepsis were identified. Infants that were transferred to the UF Health NICU from outside hospitals that developed signs of sepsis prior to transfer were excluded because the variables of interest were not all available in the EHR.

Definitions

Late-onset sepsis was defined as an event that met all of the following criteria before data was extracted to reduce potential bias: 1) the event occurred after the 3rd day of life(10), 2) subjective clinician concern for serious infection (demonstrated by a blood culture being performed, empiric antimicrobial treatment started at evaluation and continued for at least 7 days or until death), and 3) a single bacterial/fungal pathogen isolated from blood or death with necrotizing enterocolitis. All C-reactive protein measurements made during the episode (<7 days from the blood culture or until death) were collected to objectively demonstrate the presence of inflammation. Coagulase-negative Staphylococcus isolates were included only if there was evidence of significant CRP elevation (≥ 45 mg/L) during the episode(11).

Components of the nSOFA score

The organ systems selected and scoring for the nSOFA score were guided by our large retrospective study that chronicled the progressive organ dysfunction that occurred among 679 infants hospitalized in the NICU with lethal LOS(9). The nSOFA utilizes categorical scores (total score range 0–15) to objectively describe dynamic changes in: 1) the need for mechanical ventilation and oxygen requirement (score range 0–8), 2) the need for inotropic support including the use of corticosteroid support (for presumed adrenal insufficiency or catecholamine-resistant shock) (score range 0–4), and 3) the presence and degree of thrombocytopenia (score range 0–3) (Table 1). A description of the nSOFA components is presented in detail in Supplemental Data. nSOFA scores were calculated for each infant at 9 time points surrounding the sepsis evaluation (−48 hours, −24, −12, −6, evaluation, +6, +12, +24, +48).

Table 1.

Neonatal sequential organ failure (nSOFA) components and scoring (0–15 point range).

Respiratory score	0	2	4	6	8
Criteria	Not intubated OR Intubated, SpO2/FiO2 ≥ 300	Intubated, SpO2/FiO2 <300	Intubated, SpO2/FiO2 <200	Intubated, SpO2/FiO2 <150	Intubated, SpO2/FiO2 <100
Cardiovascular score	0	1	2	3	4
Criteria	No inotropes AND No systemic steroids	No inotropes AND Systemic steroid treatment	One inotrope AND No systemic steroids	Two or more inotropes OR One inotrope AND systemic steroid treatment	Two or more inotropes AND Systemic steroid treatment
Hematologic score	0	1	2	3
Criteria	Platelet count ≥ 150 × 103	Platelet count 100–149 × 103	Platelet count <100 × 103	Platelet count <50 × 103

Statistics

Continuous variables were summarized as a median with quartiles (25 and 75^th percentiles). Categorical variables were summarized using percentages. Fisher exact tests were used for categorical variables and the Wilcoxon signed-rank test or Mann-Whitney were used for continuous variables. Areas under the receiver operating characteristic curves (AUCs) were calculated. Graph Pad Prism (version 7) was used for all calculations.

Results

We identified 44 sepsis survivors and 16 non-survivors that met our definition of LOS (Table 2). The groups were similar with respect to birth weight, gestational age, delivery mode, sex, and race. Non-survivors experienced sepsis earlier after birth compared with survivors (median day of life 19 vs 11, p=0.002). Sepsis survivors exhibited a greater C-reactive protein peak compared to non-survivors (152 mg/L vs 91 mg/L, p=0.003), which was likely affected by either the time from evaluation to death for non-survivors (median 54 hours) or that providers may not have repeated C-reactive protein in non-survivors due to a perceived lack of clinical indication. Among LOS survivors, 25 (57%) had blood cultures positive for gram negative bacteria; there were two cases of Candida and the remaining infants (n = 17, 39%) had blood cultures positive for gram positive bacteria (Supplemental Table S1). Among non-survivors, six (38%) patients had gram negative bacteremia, five (31%) had gram positive bacteremia, four (25%) had negative cultures (all had Modified Bell’s stage ≥ 2 necrotizing enterocolitis), and one had a blood culture positive for Candida albicans.

Table 2.

Demographics of study population with late-onset sepsis

	Survivors (n = 44)	Non-survivors (n = 16)	p-value
Birth weight, grams (median, quartiles)	807 (630, 1146)	643, (529, 1123)	0.23a
Gestational age, weeks (median, quartiles)	26 (24, 28)	25 (23, 28)	0.46a
C-section (%)	61 (n=27)	69 (n=11)	0.76b
Female (%)	45 (n=20)	69 (n=11)	0.15b
Race			0.69b
Black (%)	39 (n=17)	38 (n=6)
Caucasian (%)	48 (n=21)	44 (n=7)
Other (%)	14 (n=6)	19 (n=3)
Day of life sepsis event occurred, days (median, quartiles)	19 (11, 38)	11 (5, 14)	0.002
Peak episode C-reactive protein, mg/L (median, quartiles)	152*	91	0.003a

^aMann-Whitney test

^bFisher’s exact test

^*24 of 44 (55%) surviving infants had CRP values measured at least 12 hours after presentation

Median nSOFA scores between survivors and non-survivors prior to the LOS evaluation were different at −48 [0 (quartiles 0, 2) vs 2 (0, 3); p=0.001], −24 [0 (0, 2) vs 2 (0, 3); p=0.042], and −12 hour time points [0 (0, 2) vs 2 (0, 6); p=0.020] but not at −6 hours (Figure 1). nSOFA scores at evaluation and at all points measured after evaluation were different between survivors and non-survivors [evaluation: median 1 (quartiles: 0, 2) vs 6 (2, 9); +6: 1 (0, 4) vs 6 (2, 10), +12 hours: 1 (0, 3) vs 8 (6, 11); +24: 2 (0, 3) vs 11 (10, 13); +48: 2 (0, 3) vs 10 (7, 14); all p<0.001]. Significant changes between survivors and non-survivors within nSOFA score component scores occurred in the respiratory [−48 (p=0.016); −12 (p=0.018); evaluation, +6, +12, +24, and +48 (all p<0.001, Supplemental Figure S1A), cardiovascular (all time points except −48, p<0.05; Supplemental Figure S1B), and platelet component scores (+12 and +24, p<0.05; Supplemental Figure S1C). Examination of all nSOFA scores from survivors at all time points showed 91% of nSOFA scores calculated were ≤ 4. Mortality was higher at evaluation (13% vs 67%, p<0.001), +6 hours (15% vs 64%, p=0.002), +12 hours (7% vs 71%, p<0.001) among patients with an nSOFA score of > 4 as compared to patients with a score of ≤ 4. Mortality correlated with the maximum nSOFA score (nSOFA^max) at evaluation or +6 hours (13% for nSOFA^max <2 to 88% for nSOFA^max >8) (Figure 2). Because mortality among non-survivors reached 25% by 24 hours and 50% by 48 hours, mortality risk was not calculated at these later time points.

Figure 1. Neonatal sequential organ failure (nSOFA) total scores among survivors and non-survivors around sepsis episode.

All data points shown for both groups at all time points. Median values and interquartile ranges shown.

Figure 2. Mortality risk based on the nSOFA maximum at evaluation or +6 hours.

Mortality (%) was determined by using the maximum nSOFA value measured at evaluation or +6 hours among the 44 survivors and 16 non-survivors.

A change in nSOFA score (nSOFA^Δ) over time may also provide useful information on the risk of sepsis mortality (Figure 3). Prior to evaluation, non-survivors experienced a significant median nSOFA^Δ compared to survivors [−48 to evaluation: 4 (0, 5) vs 0 (0, 2); p=0.002; −24 to evaluation: 3 (0, 5) vs. 0 (0, 2), p<0.001] (Figure 3A). No differences in median nSOFA^Δ between survivors and non-survivors were seen in the intervals −48 to −24, −24 to −12, or −12 to −6 hours. However, a difference in median nSOFA^Δ was found from −6 to evaluation for non-survivors as compared to survivors [2 (0, 5) vs 0 (0, 2); p=0.021]. As expected, a significant increase in median nSOFA^Δ occurred in non-survivors compared to survivors over multiple time intervals surrounding evaluation [−6 to +6; 2 (0, 4) vs. 0 (0, 0), −12 to +12; 5 (2, 9) vs 0 (0, 0), −24 to +24; 9 (3, 12) vs 2 (0, 3), −48 to +48; 9 (3, 12) vs 1 (0, 2); all p<0.001] (Figure 3B). Six patients among the non-survivors experienced little change in the nSOFA scores during the 96-hour interval we examined. All six demonstrated clinical decline and died after 48 hours. Three patients had NEC totalis and 1 had pulmonary hypoplasia.

Figure 3. The change in nSOFA scores among sepsis survivors and non-survivors.

Each symbol represents the intra-individual difference in nSOFA scores between the respective time points (nSOFA^Δ). Median values and interquartile ranges shown. If mortality occurred prior to a timed score, then the most proximal nSOFA score was substituted to allow calculations.

To examine the utility of the nSOFA at the individual patient level for mortality, a receiver operating characteristics curve showed an area under the curve of 0.77 at evaluation (95% CI 0.62–0.92; p=0.001), 0.79 at +6 hours (0.66–0.92; p<0.001) and 0.93 at +12 hours (0.86–0.997; p<0.001) (Figure 4). Combined sensitivity and specificity were maximized using an nSOFA score of ≥ 3 [evaluation (75%, 77%); +6 (75%, 68%); +12 (100%, 70%)]. The median time to death was 54 hours (quartiles: 22–188) and all infants that died received parenteral antibiotics from the time of sepsis evaluation until death. No deaths occurred prior to 12 hours.

Figure 4. Receiver operating characteristics curve for nSOFA to predict mortality at the specified timepoints.

Area under the curve (AUC) was 0.77 at evaluation (95% CI 0.62–0.92; p=0.001), 0.79 at +6 hours (0.66–0.92; p<0.001) and 0.93 at +12 hours (0.86–0.997; p<0.001).

Although the nSOFA scores for the nine timepoints we measured highlight the dynamics of sepsis-induced organ dysfunction, these time points only represent a fraction of the possible granularity when nSOFA scores are calculated from data collected as part of routine care and saved in the EHR. As representative examples, longitudinal profiles of nSOFA scores for a survivor (Supplemental Figure S2A) and non-survivor (Supplemental Figure S2B) with LOS caused by Klebsiella pneumoniae demonstrate the individual dynamics and complexity of this disease. We integrated the nSOFA score algorithm into our EHR system (EPIC), so real-time nSOFA score profiles can be added to a patient list for any neonatal care provider (Supplemental Figure S3A). A color-coded designation reflects the severity of the current nSOFA score, and a mouse-hover-over function allows the user to quickly see the individual component scores for each patient. The nSOFA-related data fields (SpO₂, FiO₂, mechanical ventilation, vasoactive drugs, steroids, platelet count) are updated in real-time as any new data is made available and nSOFA scores, including individual component scores, are recorded in the EHR every 15 minutes (Supplemental Figure S3B). nSOFA scores are also plotted in an accordion view to allow graphic representation of the change over a selected interval of time (Supplemental Figure S3C). Thus, as new real-time data is acquired, the nSOFA updates automatically eliminating the need for nursing to chart or calculate nSOFA scores. For geographic areas or health care systems that do not use EPIC, we made the web-app nSOFA calculator available online: (http://www.peds.ufl.edu/apps/nsofa/default.aspx).

Discussion

Our goal was to develop an objective, automated, evidence-based organ dysfunction scoring system dependent on routinely collected data and recorded in an electronic health record (EHR) that can identify NICU patients at a high risk of mortality among those with LOS. The nSOFA is applicable specifically to this patient population and predicts mortality in the setting of LOS. We found comparable trajectories of organ dysfunction for respiratory, cardiovascular, and platelet components in this single-center, retrospective cohort of VLBW preterm infants as we saw in the prior study of progressive organ failure with lethal LOS(9). We were able to successfully integrate and automate the nSOFA scoring system into EPIC, a commonly used EHR platform.

In adults, a rise in the SOFA score of 2 or more points in the setting of presumed infection has an AUC 0.76 for mortality(12). We found similar AUCs for the nSOFA to predict mortality at the time of sepsis evaluation and at +6 hours that rose to 0.93 by +12 hours in this cohort of neonates with LOS. Although validation of the nSOFA in another cohort of neonates with presumed infection is necessary, these data suggest that the nSOFA could perform similarly to the adult SOFA and have utility at the individual patient level. If it is shown that the nSOFA score successfully predicts sepsis mortality in subsequent neonatal cohorts, use of the nSOFA may facilitate a reduction in the number of patients required in interventional randomized clinical trials aimed at reducing sepsis-related mortality.

While the goal is to develop an organ dysfunction system that is: 1) based only on objective criteria that predicts sepsis mortality risk, 2) applicable to both EOS and LOS, and 3) is useful across all gestational ages, this as an initial step may be unrealistic. Significant organ dysfunction is frequently present at birth among VLBW infants that are not infected secondary to physiologic immaturity. Infections that begin in utero, and present immediately after birth (early-onset), are superimposed upon the significant physiologic and molecular alterations that accompany preterm birth and the transition to extrauterine life(11). Machine-based learning approaches on much larger cohorts would be desirable. However, the lack of a consensus definition in this population, in conjunction with the strong emphasis on microbiologic criteria alone rather than life-threatening organ dysfunction, places the validity of the primary exposure (sepsis) for those patients in question. This limitation would be expected to decrease the signal to noise ratio and mask any potential benefit. The majority of microbiologically-confirmed sepsis episodes and mortality in the NICU are due to LOS and occur in the VLBW population, which is why we have focused our initial efforts in these areas.

As we have proposed, the nSOFA score could and likely will expand to include other measures of organ dysfunction as they prove meaningful to sepsis mortality in evidence-based studies(13). For example, in the adult SOFA score, central nervous system, liver, and renal dysfunction are included. Currently, measures of these systems in neonates are more problematic. For example, in adults a total bilirubin of 5 mg/dL represents hepatic dysfunction(8) and yet would be absolutely within normal limits for neonates for several days after birth. Changes in renal function including creatinine and urine output may be likely additions to nSOFA scoring in the future, but these have many possible caveats (fluid intake, diuretics) that require more interpretation than the objective data used in the current iteration of the nSOFA. Central nervous system dysfunction scores in adults and children are based on the Glasgow coma scale system(8, 14). Central nervous system function assessments in neonates and particularly in preterm neonates, are broad and primarily subjective (decreased activity, lethargic). Incorporation of an objective CNS component, for example a change in regional cerebral oxygen saturation, if predictive of mortality in preterm neonates as in preliminary studies in adults with septic shock(15), will be a positive addition to the nSOFA. The addition of platelet transfusion in response to low values and the response to transfusion may also add predictive value(16).

One of the significant achievements of this work was the successful implementation of the nSOFA scoring system into the EHR. The integration of nSOFA scores automatically recorded every 15 minutes into a commonly-used EHR system without burdening the bedside nurse to document should have a significant impact on multiple unmet needs related to neonatal sepsis. The real-time automatically updating nSOFA score has great potential to help covering clinicians and nursing leadership with shift-based unit staffing needs and may improve patient safety outcomes. Notification of clinical providers when a threshold score is reached or a rate of change that is concerning occurs may provide additional utility, especially for large NICUs.

We acknowledge this study has several limitations inherent to a single-center, retrospective case-control study. One advantage of a single center study is the ability to achieve adequate data granularity, which was critical to defend the primary exposure (LOS). Confirmation of the utility of the nSOFA to predict mortality risk will require testing in another cohort of infants. A study of the utility of prospectively recorded nSOFA scores, including the determination of the association between nSOFA scores and mortality risk, as well as with survivor outcomes, is underway at our institution. We did not have adequate numbers of patients to compare the utility of the nSOFA by primary sites of infection (peritonitis, pneumonia, meningitis, UTI, CLABSI, primary bacteremia). The site of origin will be considered in future studies. Surgical peritonitis is a definitive state of infection and could be associated with higher nSOFA scores as compared to non-surgical LOS. To address this potential concern, we identified 11 additional patients with surgical peritonitis (and negative blood cultures) that survived, calculated their nSOFA scores, and re-analyzed the critical timepoints (−6, 0, +6, +12) with these new patients (n = 55 survivors, 16 non-survivors) (Supplemental Figure S4). None of the main results (nSOFA comparisons at −6, 0, +6, +12; AUC ROC) were affected by the inclusion of these 11 survivors with surgical peritonitis. We intentionally excluded examination of early-onset sepsis (EOS) because the degree of organ dysfunction immediately after birth may be high for VLBW preterm infants in the absence of infection. Microbiologically confirmed EOS is very rare, especially in term infants, so to test the utility of the nSOFA for EOS mortality will require a multi-center or large database approach to address other confounding conditions that lead to organ dysfunction and early death (pulmonary hypoplasia, severe intraventricular hemorrhage, hypoxia-ischemic encephalopathy).

The nSOFA scoring system has the potential to align neonatal sepsis investigators globally by ensuring neonatal sepsis-related data derived from cohorts studied in different settings by different investigators can be successfully integrated into the literature and interpreted. The nSOFA, if validated in additional cohorts, would address a critical unmet need for a useful, objective operational definition of organ dysfunction applicable to this unique population that can align neonatal sepsis investigators globally, allow longitudinal disease stratification including prognostication, and facilitate predictive enrichment for future prospective interventional clinical trials(6, 9, 17).