Multicenter validation of the neonatal sequential organ failure assessment score for prognosis of mortality in the neonatal intensive care unit

Abstract

Infants in the neonatal intensive care unit are at risk of life-threatening organ dysfunction, but few objective tools with utility exist. In a multicenter cohort of 20,152 infants, we show the neonatal sequential organ failure assessment (nSOFA) score had good-to-excellent discrimination of mortality across centers, birthweights, and timepoints after admission.

Introduction

Infants in the neonatal intensive care unit (NICU) are at risk of life-threatening organ dysfunction, both from developmental and pathologic causes. Organ dysfunction and mortality risk may be present immediately after birth or occur later in the NICU hospitalization. In adults, the severity of organ dysfunction is quantitatively and objectively described by the sequential organ failure assessment (SOFA) score and applied to different use cases, including diagnosis of sepsis and prognostication of in-hospital mortality for resource allocation prioritization during healthcare crises.^{1, 2} Recently, a pediatric version of the SOFA score, the pSOFA score, was validated for similar purposes.^{3, 4} A neonatal version of the SOFA score, the nSOFA score, has been validated as a means to discriminate mortality risk among preterm infants with late-onset sepsis,^{5, 6} however it has not been validated as a prognostic tool of all-cause mortality in a general NICU population, which has been identified as an important unmet need.^{7, 8} In this study, we aimed to measure the performance of the nSOFA score for all-cause mortality in a large, multicenter cohort of NICU patients.

Methods

We conducted a multicenter observational cohort study of infants admitted to the NICUs of three academic medical centers representing NICUs with different geographic locations (Illinois and Florida) and baseline populations (two centers with deliveries and referrals and one with referrals only). After institutional review board approval, data from 2009 to 2020 for NICU A (Chicago, IL), and 2012 to 2020 for NICUs B and C (Chicago, IL and Gainesville, FL) were extracted from the electronic health records (EHRs). Neonates who were born alive but died in the delivery room were excluded in the two centers where births occurred. The nSOFA score was calculated hourly from admission until day 28, discharge or death, whichever came first, using data extracted from the EHR. The nSOFA score ranges from 0–15 and is calculated using the presence of mechanical ventilation and SpO₂:FiO₂ ratio for respiratory dysfunction, presence of vasoactive medications and/or corticosteroids for cardiovascular dysfunction, and platelet count for hematologic dysfunction.⁶ For missing variables, the most recent value was used. If there was no prior value it was assumed to be normal, as in prior studies.⁴ The primary outcome was all-cause 28-day mortality. The performance of the nSOFA score to discriminate mortality was evaluated using the area under the curve (AUC) calculated. Statistical analyses were performed using R version 3.6.2 (R Foundation for Statistical Computing).

Results

A total of 20,152 infants met inclusion criteria and 603 (3%) died within 28 days (Table 1). Overall, the maximum nSOFA score in the first 24 hours had good discrimination of mortality (AUC range across the three centers: 0.88–0.89) and by day 28 had excellent discrimination (AUC 0.93–0.95). Discrimination of mortality stratified by birth weight is presented in Table 1. Patients with weights <750 grams had AUCs ranging from 0.57 to 0.94; those with weight between 750 and 1499 grams had AUCs ranging from 0.82 to 0.99; those with weight between 1500–2499 had AUCs ranging from 0.88 to 0.95; and patients with weights over 2500 grams had AUCs ≥0.92. Distribution of the maximum nSOFA score by day 28 as well as score-associated mortality is presented in Figure 1. A maximum nSOFA score of zero occurred in 12,368 (61.4%) infants, of whom 0.1% experienced mortality. The nSOFA score has been integrated into the EHR of multiple NICUs and can be calculated using an online calculator: https://peds.ufl.edu/apps/nsofa/default.aspx.^{5, 6}

Table 1.

Clinical characteristics, outcomes, and performance of the nSOFA score.

Variables	NICU A	NICU B	NICU C
Patients (n).	8797	4191	7157
Clinical Characteristics
Birth weight, grams (IQR)	2645 (1815, 3290)	2800 (1790, 3370)	2523 (1789, 3200)
Mechanical ventilation, n (%)	3456 (39.2%)	1743 (41.6%)	1692 (23.6%)
Vasoactive use, n (%)	790 (8.9%)	466 (11.1%)	545 (7.6%)
Outcomes
Length of stay, days (IQR)	9 (4, 27)	11 (4, 36)	10 (4, 29)
28-day mortality, n (%)	213 (2.4%)	146 (3.5%)	244 (3.4%)
nSOFA score, AUC for mortality (95%CI)
Max. score in first 24 hours	0.88 (0.85, 0.90)	0.89 (0.86, 0.91)	0.88 (0.85, 0.90)
Max. score in 28 days	0.93 (0.92, 0.95)	0.93 (0.91, 0.95)	0.95 (0.94, 0.96)
Max. score in 28 days, by birth weight
<500 grams	0.57 (0.42, 0.73)	0.94 (0.83, 1.00)	0.70 (0.53, 0.87)
500–749 grams	0.67 (0.59, 0.74)	0.86 (0.79, 0.93)	0.82 (0.77, 0.88)
750–999 grams	0.82 (0.74, 0.91)	0.88 (0.83, 0.94)	0.90 (0.79, 1.00)
1000–1249 grams	0.92 (0.86, 0.98)	0.97 (0.93, 1.00)	0.98 (0.97, 1.00)
1250–1499 grams	0.99 (0.98, 1.00)	0.95 (0.87, 1.00)	0.97 (0.96, 1.00)
1500–1999 grams	0.92 (0.84, 1.00)	0.88 (0.75, 0.99)	0.95 (0.90, 1.00)
2000–2499 grams	0.94 (0.90, 0.99)	0.91 (0.85, 0.97)	0.94 (0.90, 0.99)
2500–2999 grams	0.92 (0.83, 1.00)	0.93 (0.84, 1.00)	0.95 (0.92, 0.97)
3000–3499 grams	0.98 (0.98, 0.99)	0.95 (0.92, 0.97)	0.92 (0.90, 0.95)
>3499 grams	0.98 (0.97, 0.99)	0.96 (0.93, 0.99)	0.94 (0.92, 0.97)

Abbreviations: nSOFA, neonatal sequential organ failure assessment; NICU, neonatal intensive care unit; AUC, area under the curve; CI, confidence interval; Max., maximum, IQR, interquartile range

Figure 1. Distribution of the maximum nSOFA score and associated 28-day mortality in the three NICUs.

Across the three sites, 61% of patients had a maximum nSOFA of 0 by day 28, and these had an associated 28-day mortality of 0.1% (not pictured). The whiskers in the top panel represent the 95% confidence interval for 28-day mortality. Abbreviations: nSOFA, neonatal sequential organ failure assessment; NICU, neonatal intensive care unit.

Discussion

This is the first report to validate the association the nSOFA score with all-cause mortality in the NICU population. The maximum nSOFA score had good-to-excellent discrimination across birth weights (particularly ≥750 grams), centers with different patient populations, and at different time points after admission. The utility of an organ dysfunction score that discriminates mortality across the hospitalization is important for many use cases, including late-onset sepsis and resource allocation during healthcare crises.^{6, 8} For example, allocation decisions for resources such as mechanical ventilators, renal replacement therapy devices, and extracorporeal life support machines, may be needed during different timepoints in the clinical course and not necessarily just on admission.⁴ This is particularly important in the NICU, given that the existing severity of illness scores are generally validated for use within 24 hours of birth only.⁸

Our study has several strengths and limitations. Although the study was retrospective, the inclusion of a large, multicenter cohort of NICU patients from academic centers with different characteristics strongly supports generalizability of the results. We observed a large variation in performance of the nSOFA score in infants born at <750 grams across centers. This finding may be due to differences in center characteristics (e.g., centers with deliveries vs. referral only) or the reasons for death (e.g., withdrawal of support of neonates with very poor prognosis vs organ dysfunction after physiologic decompensation). In a study of 1,113 newborns delivered between 2008–2012 at 22 and 23 weeks completed gestation, which constitutes many infants <750 grams, mortality during the birth hospitalization was 93% and 68%, respectively.⁹ Infants born at these extremely preterm gestations and birth weights often manifest severe, protracted, life-threatening organ dysfunction. The variation in accuracy of the nSOFA for mortality in this unique subset of patients may be influenced by variations in goals of care, the high risk of mortality as well as the nearly ubiquitous life-threatening organ dysfunction secondary to physiologic immaturity and new pathology. However, the nSOFA score demonstrated good-to-excellent discrimination in the vast majority of cases in our study. Our findings can be useful to clinicians and researchers that require tools to objectively measure mortality risk, as well as health care leaders and policymakers who develop resource allocation protocols for critically ill patients of all ages.^{7, 8}

Abbreviations:

SOFA

sequential organ failure assessment

pSOFA

pediatric sequential organ failure assessment

nSOFA

neonatal sequential organ failure assessment

NICU

neonatal intensive care unit

EHR

electronic health records

AUC

area under the curve