This cohort study assesses the validity of a pediatric version of the Sequential Organ Failure Assessment (SOFA) scoring system compared with the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) to examine the in-hospital mortality of critically ill children with confirmed or suspected infection and sepsis.
Key Points
Questions
Is a pediatric version of the Sequential Organ Failure Assessment score valid, and can it be used to evaluate the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) in critically ill children?
Findings
In this large cohort study of 8711 pediatric intensive care unit encounters, the pediatric Sequential Organ Failure Assessment score demonstrated excellent discrimination for in-hospital mortality, and the Sepsis-3 definitions identified a cohort of patients with high mortality and microbiological characteristics associated with severe sepsis in prior studies.
Meaning
The evaluation of the Sepsis-3 definitions in children using the pediatric Sequential Organ Failure Assessment score shows promising results.
Abstract
Importance
The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) uses the Sequential Organ Failure Assessment (SOFA) score to grade organ dysfunction in adult patients with suspected infection. However, the SOFA score is not adjusted for age and therefore not suitable for children.
Objectives
To adapt and validate a pediatric version of the SOFA score (pSOFA) in critically ill children and to evaluate the Sepsis-3 definitions in patients with confirmed or suspected infection.
Design, Setting, and Participants
This retrospective observational cohort study included all critically ill children 21 years or younger admitted to a 20-bed, multidisciplinary, tertiary pediatric intensive care unit between January 1, 2009 and August 1, 2016. Data on these children were obtained from an electronic health record database. The pSOFA score was developed by adapting the original SOFA score with age-adjusted cutoffs for the cardiovascular and renal systems and by expanding the respiratory criteria to include noninvasive surrogates of lung injury. Daily pSOFA scores were calculated from admission until day 28 of hospitalization, discharge, or death (whichever came first). Three additional pediatric organ dysfunction scores were calculated for comparison.
Exposures
Organ dysfunction measured by the pSOFA score, and sepsis and septic shock according to the Sepsis-3 definitions.
Main Outcomes and Measures
The primary outcome was in-hospital mortality. The daily pSOFA scores and additional pediatric organ dysfunction scores were compared. Performance was evaluated using the area under the curve. The pSOFA score was then used to assess the Sepsis-3 definitions in the subgroup of children with confirmed or suspected infection.
Results
In all, 6303 patients with 8711 encounters met inclusion criteria. Each encounter was treated independently. Of the 8482 survivors of hospital encounters, 4644 (54.7%) were male and the median (interquartile range [IQR]) age was 69 (17-156) months. Among the 229 nonsurvivors, 127 (55.4%) were male with a median (IQR) age of 43 (8-144) months. In-hospital mortality was 2.6%. The maximum pSOFA score had excellent discrimination for in-hospital mortality, with an area under the curve of 0.94 (95% CI, 0.92-0.95). The pSOFA score had a similar or better performance than other pediatric organ dysfunction scores. According to the Sepsis-3 definitions, 1231 patients (14.1%) were classified as having sepsis and had a mortality rate of 12.1%, and 347 (4.0%) had septic shock and a mortality rate of 32.3%. Patients with sepsis were more likely to die than patients with confirmed or suspected infection but no sepsis (odds ratio, 18; 95% CI, 11-28). Of the 229 patients who died during their hospitalization, 149 (65.0%) had sepsis or septic shock during their course.
Conclusions and Relevance
The pSOFA score was adapted and validated with age-adjusted variables in critically ill children. Using the pSOFA score, the Sepsis-3 definitions were assessed in children with confirmed or suspected infection. This study is the first assessment, to date, of the Sepsis-3 definitions in critically ill children. Use of these definitions in children is feasible and shows promising results.
Introduction
The Sequential Organ Failure Assessment (SOFA) score was selected as the scoring system to quantify organ dysfunction in the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).1 The Sepsis-3 Task Force validated the SOFA score in adult patients with suspected infection and found the SOFA system to be either comparable or superior to other scoring systems at discriminating in-hospital mortality.1 The Sepsis-3 definitions are expected to be widely adopted and, by extension, the use of SOFA score in patients with confirmed or suspected infection. One of the major limitations of the SOFA score is that it was developed for adult patients and contains measures that vary significantly with age, which makes it unsuitable for children.2 The Sepsis-3 Task Force recognized this problem and identified it as an area for further development.1
Several pediatric organ dysfunction scores take into account the age dependency of their variables, including the Pediatric Logistic Organ Dysfunction (PELOD) score, the updated PELOD-2 score, and the Pediatric Multiple Organ Dysfunction Score.3,4,5 Use of any of these scores as a measure of organ dysfunction in infected children could be considered for adapting the Sepsis-3 definitions to pediatric patients, but the range, scale, and coverage of these scores are significantly different from those of the SOFA score, which makes their concurrent use problematic. Fundamentally, having different definitions of sepsis for patients above or below the pediatric-adult threshold has no known physiologic justification and should therefore be avoided.
Prior studies have attempted to adapt the SOFA score to pediatric patients, mostly focusing on the cardiovascular subscore.6,7 However, none have taken into account the age-related variability of the renal subscore criteria despite the increasingly recognized detrimental effect of kidney dysfunction in younger patients.8,9,10 In addition, the respiratory subscore criteria—based on the ratio of Pao2 to the fraction of inspired oxygen (Fio2)—have not been modified in previous adaptations of the SOFA score even though the decreased use of arterial blood gases in children is a known limitation.11,12,13 Fortunately, the cardiovascular and renal components of the SOFA score were evaluated and adapted to pediatric patients by the PELOD-2 score investigators, and the ratio of peripheral oxygen saturation (Spo2) to Fio2 has been validated as an alternative to the Pao2:Fio2 ratio in children.4,12
In this study, we sought to adapt and validate a SOFA score for critically ill pediatric patients (pSOFA) using age-adjusted criteria. In addition, we sought to assess the Sepsis-3 definitions for sepsis and septic shock in the subgroup of critically ill children with confirmed or suspected infection using the pSOFA score.
Methods
Patients and Data Collection
We performed a single-center, retrospective cohort study of critically ill children presenting to a multidisciplinary, tertiary pediatric intensive care unit (PICU). This 20-bed PICU serves a mixed population of medical, surgical, and trauma patients. We included all patients 21 years or younger on admission and who received care in the PICU between January 1, 2009, and August 1, 2016. Each hospitalization with a PICU admission was treated independently. Data were extracted from an electronic health record database (Epic; Epic Systems Corporation). The institutional review board of The University of Chicago approved this study and waived patient informed consent because of the observational nature of the study.
Development of the pSOFA Score
The pSOFA score was developed by adapting the original SOFA score through 2 approaches. First, the age-dependent cardiovascular and renal variables of the original SOFA score were modified using validated cutoffs from the PELOD-2 scoring system.4 Second, the respiratory subscore was expanded to include the Spo2:Fio2 ratio as an alternative surrogate of lung injury (Table 1).12
Table 1. Pediatric Sequential Organ Failure Assessment Score.
| Variables | Scorea | ||||
|---|---|---|---|---|---|
| 0 | 1 | 2 | 3 | 4 | |
| Respiratory | |||||
| Pao2:Fio2b
or Spo2:Fio2c |
≥400 | 300-399 | 200-299 | 100-199 With respiratory support | <100 With respiratory support |
| ≥292 | 264-291 | 221-264 | 148-220 With respiratory support | <148 With respiratory support | |
| Coagulation | |||||
| Platelet count, ×103/μL | ≥150 | 100-149 | 50-99 | 20-49 | <20 |
| Hepatic | |||||
| Bilirubin, mg/dL | <1.2 | 1.2-1.9 | 2.0-5.9 | 6.0-11.9 | >12.0 |
| Cardiovascular | |||||
| MAP by age group or vasoactive infusion, mm Hg or µg/kg/mind | |||||
| <1 mo | ≥46 | <46 | Dopamine hydrochloride ≤5 or dobutamine hydrochloride (any) | Dopamine hydrochloride >5 or epinephrine ≤0.1 or norepinephrine bitartrate ≤0.1 | Dopamine hydrochloride >15 or epinephrine >0.1 or norepinephrine bitartrate >0.1 |
| 1-11 mo | ≥55 | <55 | |||
| 12-23 mo | ≥60 | <60 | |||
| 24-59 mo | ≥62 | <62 | |||
| 60-143 mo | ≥65 | <65 | |||
| 144-216 mo | ≥67 | <67 | |||
| >216 moe | ≥70 | <70 | |||
| Neurologic | |||||
| Glasgow Coma Scoref | 15 | 13-14 | 10-12 | 6-9 | <6 |
| Renal | |||||
| Creatinine by age group, mg/dL | |||||
| <1 mo | <0.8 | 0.8-0.9 | 1.0-1.1 | 1.2-1.5 | ≥1.6 |
| 1-11 mo | <0.3 | 0.3-0.4 | 0.5-0.7 | 0.8-1.1 | ≥1.2 |
| 12-23 mo | <0.4 | 0.4-0.5 | 0.6-1.0 | 1.1-1.4 | ≥1.5 |
| 24-59 mo | <0.6 | 0.6-0.8 | 0.9-1.5 | 1.6-2.2 | ≥2.3 |
| 60-143 mo | <0.7 | 0.7-1.0 | 1.1-1.7 | 1.8-2.5 | ≥2.6 |
| 144-216 mo | <1.0 | 1.0-1.6 | 1.7-2.8 | 2.9-4.1 | ≥4.2 |
| >216 moe | <1.2 | 1.2-1.9 | 2.0-3.4 | 3.5-4.9 | ≥5 |
Abbreviations: Fio2, fraction of inspired oxygen; MAP, mean arterial pressure; pSOFA, pediatric Sequential Organ Failure Assessment; Spo2, peripheral oxygen saturation.
SI conversion factors: To convert bilirubin to micromoles per liter, multiply by 17.104; creatinine to micromoles per liter, multiply by 88.4; and platelet count to ×109/L, multiply by 1.
The pSOFA score was calculated for every 24-hour period. The worst value for every variable in each 24-hour period was used to calculate the subscore for each of the 6 organ systems. If a variable was not recorded in a given 24-hour period, it was assumed to be normal and a score of 0 was used. Daily pSOFA score was the sum of the 6 subscores (range, 0-24 points; higher scores indicate a worse outcome).
Pao2 was measured in millimeters of mercury.
Only Spo2 measurements of 97% or lower were used in the calculation.
MAP (measured in millimeters of mercury) was used for scores 0 and 1; vasoactive infusion (measured in micrograms per kiligram per minute), for scores 2 to 4. Maximum continuous vasoactive infusion was administered for at least 1 hour.
Cutoffs for patients older than 18 years (216 months) were identical to the original SOFA score.
Glasgow Coma Scale was calculated using the pediatric scale.
Cardiovascular Subscore
The age-adjusted mean arterial pressure cutoffs for the first score of the PELOD-2 cardiovascular criteria were used to assign a score of 1 in the pSOFA subscore. Scores 2 to 4 were kept identical to the original SOFA criteria.
Renal Subscore
The age-adjusted serum creatinine level cutoffs for the first score of the PELOD-2 renal criteria were used to assign a score of 1 in the pSOFA renal subscore. Scores 2 to 4 were modified by increasing the cutoff values for each score by the same factor as the original SOFA criteria, similar to the approach proposed by other authors.14 Exceptions to this approach were the cutoff values for the first age group (<1 month) owing to the renal physiologic differences of neonates. For this neonatal age group, the cutoff value increase for each score was done by the same amount as the infant group (1-12 months) given the similarity in the glomerular filtration rate in both age groups.15
Respiratory Subscore
The original Pao2:Fio2 ratio cutoffs were kept identical to the original score, but the Spo2:Fio2 ratio was used as an alternative surrogate of lung injury. The adaptation proposed by Khemani and colleagues12 was used to define the Spo2:Fio2 ratio cutoffs.
Coagulation, Hepatic, and Neurologic Subscores
The original coagulation and hepatic criteria, based on platelet count and bilirubin level, were kept identical to the original score. The Glasgow Coma Scale criteria for the neurologic subscore were also kept identical to the original score, but the pediatric version of the scale was used.16
The calculation of the pSOFA score was performed in the same way as the calculation of the original SOFA score.2 The worst variable in each 24-hour period was used to assign a subscore for each system (ranging from 0-4 points). The sum of the 6 subscores in each 24-hour period resulted in a daily pSOFA score (ranging from 0-24 points; higher scores indicate a worse outcome). If a variable was not measured in a 24-hour period, it was considered to be normal, which is consistent with the original criteria.
To compare the performance of the adapted variables in pSOFA with those in the original SOFA score, we calculated the maximum individual subscores for the cardiovascular, renal, and respiratory components using both the original and the adapted criteria.
Comparison of pSOFA With Other Organ Dysfunction Scores
We compared the performance of the pSOFA score with 3 other pediatric organ dysfunction scores—the PELOD score, the updated PELOD-2 score, and the Pediatric Multiple Organ Dysfunction Score.3,4,5 We calculated the daily score of the 4 scoring systems for each 24-hour period from PICU admission until day 28 of hospitalization, discharge, or death, whichever came first. The maximum and mean scores for each scoring system were used to compare the scores and evaluate the clinical validity of pSOFA. To evaluate the clinical utility of pSOFA on admission, we compared it with the Pediatric Risk of Mortality (PRISM) III score, a marker of severity of illness on admission, using information from the first 24 hours.17 To further evaluate the clinical validity and utility of pSOFA in comparison to the other scoring systems, the maximum and mean scores were also calculated at 4 landmarked times—days 2, 4, 7, and 14 after PICU admission. Only patients still alive and hospitalized on landmarked days were used for calculations.
To evaluate the effect of patients who had more than 1 hospitalization with a PICU admission, a sensitivity analysis using only the first PICU admission for each patient was performed.
Sepsis-3 Definitions
We assessed the Sepsis-3 definitions in the subgroup of patients with confirmed or suspected infection by using previously published criteria.18 Patients with suspected infection were defined as those who had negative microbiological study results and initiation of treatment with nonprophylactic antibiotics, antifungals, or antiviral medications within 24 hours.18 Coagulase-negative Staphylococcus was only considered infectious if it was isolated in 2 or more cultures from sterile sites.
Patients with sepsis were defined as those with confirmed or suspected infection who had an acute rise in the pSOFA score of 2 points or more from up to 48 hours before the infection to 24 hours after the infection and who received antimicrobial therapy in the PICU.18 Infection time was defined as the time when the first microbiological study or antimicrobial therapy was ordered by a physician, whichever came first. If the patients were not known to have previous organ dysfunction, the preinfection pSOFA score was assumed to be zero. Septic shock was defined as patients with sepsis who required a vasoactive infusion and had a maximum serum lactate level greater than 2 mmol/L (18 mg/dL).1 The microbiological etiology and infection source of patients with a confirmed or suspected infection were analyzed and compared between those who met sepsis criteria and those who did not.
Statistical Analysis
The primary outcome was in-hospital mortality. Data were analyzed using Stata, version 14 (StataCorp LLC), and R, version 3.2.2 (R Foundation for Statistical Computing). Categorical variables were compared using the χ2 test, and continuous variables were compared using the Mann-Whitney test. A 2-sided P < .05 was considered statistically significant.
The performance of the scores to discriminate in-hospital mortality was evaluated using the area under the curve (AUC). Comparison between scores was performed using the DeLong method19 to compare AUCs and the Integrated Discrimination Improvement Index20 to evaluate the reclassification of predicted probabilities between survivors and nonsurvivors. Youden J statistic21 was used to evaluate the optimal cutoff of the pSOFA score to discriminate mortality.
Reporting of this validation study was performed using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines.22
Results
In all, 6303 patients with 8711 encounters met inclusion criteria. Of the 8482 survivors of hospital encounters, 4644 (54.7%) were male and the median (interquartile range [IQR]) age was 69 (17-156) months. In-hospital mortality was 2.6%. Among the 229 nonsurvivors, 127 (55.4%) were male with a median (IQR) age of 43 (8-144) months. The demographic and clinical characteristics of survivors and nonsurvivors are shown in Table 2. (Note that we treated each of the 8711 encounters independently given that the major risk factors for the outcome [ie, organ dysfunctions] and the outcome [ie, survival or nonsurvival at the end of the hospitalization, or in-hospital mortality] are contained within each encounter.)
Table 2. Demographic and Clinical Characteristics of Survivors and Nonsurvivors.
| Characteristic | Survivorsa (n = 8482) |
Nonsurvivorsa (n = 229) |
P Value |
|---|---|---|---|
| Clinical Variables | |||
| Age, median (IQR), mob | 69 (17-156) | 43 (8-144) | .002 |
| Male, No. (%) | 4644 (54.7) | 127 (55.4) | .90 |
| Race/ethnicity, No. (%) | |||
| Black | 4719 (55.6) | 103 (45) | <.001 |
| White | 2091 (25) | 52 (22.7) | |
| Hispanic | 982 (11.6) | 36 (16) | |
| Asian | 158 (1.9) | 5 (2) | |
| Other/Unknown | 532 (6.3) | 33 (14) | |
| Admission type, No. (%)c | |||
| Respiratory failure | 2425 (28.6) | 54 (24) | .10 |
| Neurologic compromise | 1169 (13.8) | 22 (10) | .07 |
| Cardiovascular compromise | 749 (8.8) | 67 (29) | <.001 |
| Trauma | 759 (8.9) | 24 (10) | .40 |
| Metabolic derangement/poisoning | 742 (8.7) | 6 (3) | <.001 |
| Hematologic derangement | 409 (4.8) | 20 (9) | .007 |
| Postoperative recovery | 1870 (22) | 21 (10) | <.001 |
| Other | 359 (4.2) | 15 (7) | .08 |
| Admission source, No. (%) | |||
| Emergency department | 2907 (34.3) | 72 (31) | .40 |
| Inpatient floor | 910 (10.7) | 50 (22) | <.001 |
| Operating room | 2119 (25) | 22 (10) | <.001 |
| Other hospitalsd | 2546 (30) | 85 (37) | .02 |
| Required mechanical ventilation, No. (%) | 2022 (23.8) | 193 (84) | <.001 |
| Required vasoactive infusions, No. (%) | 461 (5.4) | 173 (76) | <.001 |
| PRISM III score on admission, median (IQR) | 2 (0-6) | 22 (12-31) | <.001 |
| Maximum OD score, median (IQR) | |||
| pSOFA | 2 (1-5) | 13 (10-16) | <.001 |
| PELOD | 10 (1-11) | 40 (31-43) | <.001 |
| PELOD-2 | 2 (2-5) | 13 (10-15) | <.001 |
| P-MODS | 0 (0-2) | 6 (3-9) | <.001 |
| Outcomes, Median (IQR) | |||
| Ventilator-free dayse | 28 (28-28) | 0 (0-3) | <.001 |
| Vasoactive infusion–free dayse | 28 (28-28) | 2 (0-7) | <.001 |
| ICU-free dayse | 26 (25-27) | 0 (0-0) | <.001 |
| ICU LOS, d | 1.3 (0.8-2.9) | 3.1 (1.2-11.2) | <.001 |
| Hospital LOS, d | 4 (2-8) | 6.2 (2-26.1) | <.001 |
Abbreviations: ICU, intensive care unit; IQR, interquartile range; LOS, length of stay; OD, organ dysfunction; PELOD, Pediatric Logistic Organ Dysfunction; P-MODS, Pediatric Multiple Organ Dysfunction Score; PRISM III, Pediatric Risk of Mortality III; pSOFA, pediatric Sequential Organ Failure Assessment.
Each encounter was treated independently given that the major risk factors for the outcome (ie, organ dysfunctions) and the outcome (ie, in-hospital mortality) are contained within each encounter.
Continuous variables are presented as median (IQR) values.
Admission type was determined using the primary diagnosis code on admission of the International Classification of Diseases, Ninth Revision, Clinical Modification.
Other hospitals include emergency departments in other institutions.
Ventilator-free, vasoactive infusion–free, and ICU-free days were calculated using 28 days as a reference.
Performance of pSOFA
Nonsurvivors had a significantly higher median (IQR) maximum pSOFA score than survivors (13 [10-16] vs 2 [1-5]; P < .001; Table 2). The maximum pSOFA score had excellent discrimination for in-hospital mortality, with an AUC of 0.94 (95% CI, 0.92-0.95). The in-hospital mortality rate for patients according to their maximum pSOFA score are shown in the Figure. The performance of the maximum pSOFA score to discriminate in-hospital mortality remained stable across sex, age groups, and admission types (eTable 1 in the Supplement). The optimal pSOFA cutoff to discriminate mortality was a score higher than 8 points.
Figure. In-Hospital Mortality Rate Based on the Maximum Pediatric Sequential Organ Failure Assessment (pSOFA) Score.
Maximum pSOFA score was the highest daily pSOFA score achieved by day 28 after pediatric intensive care unit admission, discharge, or death (whichever came first). Error bars represent 95% CIs.
The performance of the maximum pSOFA cardiovascular, renal, and respiratory subscores at discriminating in-hospital mortality were better than the non–age-adjusted maximum SOFA cardiovascular (AUC, 0.87 vs 0.86; P < .001), renal (AUC, 0.76 vs 0.61; P < .001), and respiratory (AUC, 0.87 vs 0.75; P < .001) subscores. The reclassification of estimated probabilities based on the Integrated Discrimination Improvement Index for the respiratory and renal subscores was significant (total ≥2.6%; P < .001) but negligible for the cardiovascular subscore (eTable 2 in the Supplement).
Comparison of pSOFA With Other Pediatric Organ Dysfunction Scores and PRISM III
The performance of the maximum pSOFA score at discriminating in-hospital mortality (AUC, 0.94; 95% CI, 0.92-0.95) was similar to the performance of PELOD and PELOD-2 (AUC, 0.93 vs 0.94; 95% CI, 0.91-0.95 vs 0.92-0.95; P > .20) and better than the Pediatric Multiple Organ Dysfunction Score (AUC, 0.91; 95% CI, 0.88-0.93; P = .001). The performance of pSOFA on the day of admission at discriminating in-hospital mortality (AUC, 0.88; 95% CI, 0.86-0.91) was better than the other organ dysfunction scores (P ≤ .02) and similar to PRISM III (AUC, 0.88; 95% CI, 0.86-0.91; P = .94) (Table 3). On the landmarked day analyses, pSOFA performed slightly better on days 2 and 4 and performed similarly to the other scores on days 7 and 14 (Table 3). The Integrated Discrimination Improvement Index showed small differences in the reclassification of estimated probabilities (eTable 3 in the Supplement).
Table 3. Comparison of pSOFA With Other Pediatric Organ Dysfunction Scores and PRISM-III at Discriminating In-Hospital Mortality.
| Scoring System | AUC (95% CI) |
P Value for AUC Comparisona |
|---|---|---|
| Comparison of Overall Scores and Admission Scores (n = 8711) | ||
| Maximum scoreb | ||
| pSOFA | 0.94 (0.92-0.95) | |
| PELOD | 0.93 (0.91-0.95) | .24 |
| PELOD-2 | 0.94 (0.92-0.95) | .85 |
| P-MODS | 0.91 (0.88-0.93) | .001 |
| Mean scoreb | ||
| pSOFA | 0.96 (0.94-0.97) | |
| PELOD | 0.95 (0.93-0.97) | .15 |
| PELOD-2 | 0.95 (0.93-0.97) | .004 |
| P-MODS | 0.93 (0.90-0.95) | <.001 |
| Admission scorec | ||
| pSOFA | 0.88 (0.86-0.91) | |
| PELOD | 0.86 (0.83-0.88) | .001 |
| PELOD-2 | 0.87 (0.84-0.89) | .02 |
| P-MODS | 0.86 (0.83-0.89) | .009 |
| PRISM-III | 0.88 (0.86-0.91) | .94 |
| Comparison of Scores on Landmarked Daysd | ||
| Day 2 (n = 7803) | ||
| Maximum score | ||
| pSOFA | 0.89 (0.87-0.92) | |
| PELOD | 0.87 (0.84-0.90) | .03 |
| PELOD-2 | 0.87 (0.84-0.90) | .002 |
| P-MODS | 0.86 (0.83-0.87) | .004 |
| Mean score | ||
| pSOFA | 0.89 (0.87-0.92) | |
| PELOD | 0.87 (0.84-0.90) | .02 |
| PELOD-2 | 0.87 (0.84-0.90) | .002 |
| P-MODS | 0.86 (0.83-0.89) | .005 |
| Day 4 (n = 5252) | ||
| Maximum score | ||
| pSOFA | 0.88 (0.85-0.91) | |
| PELOD | 0.85 (0.82-0.89) | .03 |
| PELOD-2 | 0.85 (0.82-0.89) | .008 |
| P-MODS | 0.85 (0.82-0.89) | .06 |
| Mean score | ||
| pSOFA | 0.89 (0.86-0.92) | |
| PELOD | 0.86 (0.83-0.90) | .03 |
| PELOD-2 | 0.87 (0.84-0.90) | .005 |
| P-MODS | 0.87 (0.83-0.90) | .04 |
| Day 7 (n = 2845) | ||
| Maximum score | ||
| pSOFA | 0.82 (0.78-0.86) | |
| PELOD | 0.81 (0.77-0.85) | .27 |
| PELOD-2 | 0.81 (0.76-0.85) | .16 |
| P-MODS | 0.81 (0.76-0.85) | .38 |
| Mean score | ||
| pSOFA | 0.85 (0.81-0.89) | |
| PELOD | 0.83 (0.79-0.88) | .24 |
| PELOD-2 | 0.83 (0.79-0.88) | .06 |
| P-MODS | 0.83 (0.79-0.88) | .25 |
| Day 14 (n = 1205) | ||
| Maximum score | ||
| pSOFA | 0.77 (0.71-0.84) | |
| PELOD | 0.76 (0.71-0.82) | .37 |
| PELOD-2 | 0.76 (0.71-0.82) | .32 |
| P-MODS | 0.79 (0.73-0.85) | .64 |
| Mean score | ||
| pSOFA | 0.83 (0.77-0.88) | |
| PELOD | 0.83 (0.77-0.87) | .88 |
| PELOD-2 | 0.80 (0.75-0.86) | .03 |
| P-MODS | 0.82 (0.76-0.87) | .50 |
Abbreviations: AUC, area under the curve; PELOD, Pediatric Logistic Organ Dysfunction; PELOD-2, PELOD, updated; PICU, pediatric intensive care unit; P-MODS, Pediatric Multiple Organ Dysfunction Score; PRISM III, Pediatric Risk of Mortality III; pSOFA, pediatric Sequential Organ Failure Assessment.
P value for the AUC comparison was based on the DeLong method19 comparing the pSOFA AUC with the AUC of each of the other scoring systems.
Overall maximum and mean scores for each scoring system were calculated using data from PICU admission until day 28 of hospital stay, discharge, or death, whichever came first.
Scores for the organ dysfunction on admission and PRISM III were obtained using variables from the first 24 hours of PICU admission.
Maximum and mean scores for landmarked days were calculated using data from admission until the landmarked day for patients who were still hospitalized and alive on the landmarked day.
A sensitivity analysis using only the first PICU admission for each patient showed results similar to the complete cohort (eTable 4 in the Supplement).
Assessment of the Sepsis-3 Definitions in Patients With Confirmed or Suspected Infection
Of 8711 patient encounters, 4217 (48.4%) had a confirmed or suspected infection in the PICU. The maximum pSOFA score had excellent discrimination of in-hospital mortality in this subgroup of patients (AUC, 0.92; 95% CI, 0.91- 0.94). When we used the adapted Sepsis-3 definitions with the pSOFA score, 1231 patients (14.1% of the PICU population) met sepsis criteria and had a mortality rate of 12.1%. Of those, 347 patients (4.0% of the PICU population) also met septic shock criteria and had a mortality rate of 32.3%. Among patients with confirmed or suspected infection, being diagnosed with sepsis significantly increased the likelihood of dying in the hospital (odds ratio, 18; 95% CI, 11-28). Of the 229 patients who died during their hospitalization, 149 (65.0%) had sepsis or septic shock during their course.
The demographic, clinical, and microbiological characteristics of patients with a confirmed or suspected infection with or without sepsis are shown in Table 4.
Table 4. Assessment of the Sepsis-3 Definitions in Critically Ill Children With Confirmed or Suspected Infection.
| Variable | Confirmed or Suspected Infection (n = 4217) |
P Value | |
|---|---|---|---|
| No Sepsis (n = 2986) |
Sepsis (n = 1231) |
||
| Clinical Variablesa | |||
| Age, median (IQR), mob | 49 (13-134) | 49 (11-146) | .81 |
| Male, No. (%) | 1638 (54.9) | 680 (55.2) | .82 |
| Race/ethnicity, No. (%) | |||
| Black | 1897 (63.5) | 622 (50.5) | <.001 |
| White | 555 (18.6) | 302 (24.5) | |
| Hispanic | 328 (11) | 198 (16.1) | |
| Asian | 47 (2) | 26 (2) | |
| Other/unknown | 159 (5.3) | 83 (7) | |
| Admission type, No. (%) | |||
| Respiratory failure | 1298 (43.5) | 354 (28.8) | <.001 |
| Neurologic compromise | 424 (14.2) | 140 (11.4) | .01 |
| Cardiovascular compromise | 330 (11.1) | 246 (20) | <.001 |
| Hematologic derangement | 131 (4.4) | 121 (9.8) | <.001 |
| Metabolic derangement/poisoning | 130 (4.4) | 30 (2) | .003 |
| Trauma | 101 (3.4) | 58 (5) | .04 |
| Postoperative recovery | 482 (16.1) | 197 (16) | .91 |
| Other | 90 (3) | 85 (7) | <.001 |
| Admission source, No. (%) | |||
| Emergency department | 1233 (41.3) | 427 (34.7) | <.001 |
| Inpatient floor | 289 (9.7) | 229 (18.6) | <.001 |
| Operating room | 541 (18.1) | 226 (18.4) | .85 |
| Other hospitalsc | 922 (30.9) | 348 (28.3) | .09 |
| PRISM III score on admission, median (IQR) | 2 (0-6) | 9 (5-17) | <.001 |
| pSOFA score on admission, median (IQR) | 2 (1-4) | 7 (4-10) | <.001 |
| Maximum pSOFA score, median (IQR) | 2 (1-4) | 8 (5-12) | <.001 |
| Microbiological analysis results, No. (%)d | |||
| Gram-positive bacteria | 269 (9) | 352 (28.6) | <.001 |
| Gram-negative bacteria | 252 (8.4) | 344 (27.9) | <.001 |
| Viruses | 1343 (45) | 294 (23.9) | <.001 |
| Fungi | 39 (1) | 99 (8) | <.001 |
| No organism identified | 1023 (34.3) | 545 (44.3) | <.001 |
| Infection source, No. (%)d | |||
| Bloodstream | 178 (6) | 244 (19.8) | <.001 |
| Respiratory tract | 1466 (49.1) | 480 (39) | <.001 |
| Cerebral spinal fluid | 84 (3) | 14 (1) | .001 |
| Genitourinary tract | 238 (8) | 53 (4) | <.001 |
| Other | 196 (6.6) | 38 (3) | <.001 |
| Outcomes | |||
| Ventilator-free days, median (IQR)e | 28 (28-28) | 23 (14-28) | <.001 |
| Vasoactive infusion–free days, median (IQR)e | 28 (28-28) | 28 (26-28) | <.001 |
| ICU-free days, median (IQR)e | 26 (25-27) | 20 (8-24) | <.001 |
| Hospital LOS, median (IQR), d | 4.2 (2.6-7.6) | 14.9 (8.8-28.7) | <.001 |
| In-hospital mortality, No. (%) | 23 (1) | 149 (12.1) | <.001 |
Abbreviations: ICU, intensive care unit; IQR, interquartile range; LOS, length of stay; OD, organ dysfunction; PRISM III, Pediatric Risk of Mortality III; pSOFA, pediatric Sequential Organ Failure Assessment.
Clinical variables included demographic, clinical, and microbiological characteristics of patients with confirmed or suspected infection and with or without sepsis according to the Sepsis-3 definitions.
Continuous variables are presented as median (IQR) values.
Other hospitals included other emergency departments.
More than 1 type is possible, and percentages may add to more than 100%.
Ventilator-free, vasoactive infusion–free, and ICU-free days were calculated using 28 days as a reference.
Discussion
We adapted and validated an age-adjusted version of the SOFA score for pediatric patients (pSOFA). The pSOFA score showed excellent discrimination for in-hospital mortality in a general PICU population, which was comparable to or better than the performance of other common pediatric organ dysfunction scores. We then used the pSOFA score to perform the first assessment of Sepsis-3 in critically ill children.
Several motivations were behind the adaptation of the adult SOFA score and the Sepsis-3 definitions to children. Having a harmonized definition of sepsis across age groups while recognizing the importance of the age-based variation of its measures can have many benefits, including better design of clinical trials, improved accuracy of reported outcomes, and better translation of the research and clinical strategies in the management of sepsis. Furthermore, as we focus our efforts on uncovering the pathobiological basis of the different subtypes of sepsis,23,24 we should avoid being limited by differing definitions of the syndrome across artificial constructs, such as the pediatric-adult age threshold. In our study, we found that the optimal pSOFA score cutoff to discriminate mortality was identical to the cutoff found by Ferreira and colleagues25 in adult critically ill patients (ie, a SOFA score >8 points). This observation requires further validation; however, when measured with a comparable scoring system—and despite the differences in baseline mortality—the degree of organ dysfunction in adults and children seems to discriminate outcomes in a similar way.
Another motivation for the adaptation of Sepsis-3 to children is that the current definitions of sepsis in pediatrics are not without problems. Weiss and colleagues26 recently demonstrated a lack of agreement between the 2005 International Pediatric Sepsis Consensus Conference criteria and the treating physician’s diagnosis of severe sepsis in the Sepsis Prevalence, Outcomes, and Therapies (SPROUT) study. This discrepancy implies that the results from research using the 2005 International Pediatric Sepsis Consensus Conference criteria could lack generalizability to almost a third of PICU patients with sepsis. One of the goals of Sepsis-3 is to harmonize the definitions of sepsis and septic shock using readily available objective clinical data,18 and its adaptation to children may help balance the existing diagnostic discrepancies in pediatric patients.
The results of our assessment of the Sepsis-3 definitions in critically ill children with confirmed or suspected infection are encouraging. The Sepsis-3 definitions identified a group of patients among those with confirmed or suspected infection who were 18 times more likely to die in the hospital. Furthermore, patients with sepsis in our cohort had characteristics similar to the critically ill children who were diagnosed with severe sepsis by either the 2005 International Pediatric Sepsis Consensus Conference criteria or physician diagnosis in the SPROUT study.26 Patients with the “broader” diagnosis of severe sepsis in the SPROUT cohort had similar incidence of gram-positive (27% vs 28% in our population) and gram-negative (26% vs 28%) bacterial infections as well as viral (22% vs 24%) and fungal (12% vs 8%) infections. The infection sources of both cohorts also showed similarities in the respiratory tract (41% vs 39%), bloodstream (19% vs 20%), and genitourinary tract (4% vs 4%) infection sites. Furthermore, these similarities were all significantly different from the microbiological characteristics of patients with confirmed or suspected infection but no sepsis. This finding suggests that the Sepsis-3 definitions could help bridge the current diagnostic discrepancies in identifying children with severe sepsis. (Of note, the label severe sepsis was eliminated by the Sepsis-3 definitions in favor of sepsis, which is more commonly used in patient care.)1 One major difference with our cohort is that the incidence of severe sepsis in the SPROUT study was lower (10% vs 14%), and those patients had higher mortality (23% vs 12%).26 This difference suggests that our adaptation of the Sepsis-3 definitions with the pSOFA score likely captures a larger cohort of patients with infection and milder organ dysfunction. However, this group might still be a valuable population to consider in the context of sepsis, especially given the shared microbiological characteristics with the SPROUT cohort and a significant increase in the likelihood of dying when compared with infected patients with no sepsis. The mortality difference between our PICU population and the SPROUT cohort may also be attributable to the differences in the patient populations seen in PICUs outside the United States.26 This difference emphasizes the need for further validation of the Sepsis-3 definitions in critically ill children in other settings and populations.
Limitations
There are several limitations to our study. First, our results were generated using retrospective data from a single center. Validating pSOFA in a larger, multicenter sample of critically ill children is necessary to assess the generalizability of the score. The adaptation of the Sepsis-3 definitions to children with confirmed or suspected infection using the pSOFA score or similar approaches must be validated as well in other settings and populations. Furthermore, the utility of the Sepsis-3 definitions for identifying children with sepsis for enrollment in clinical trials, reporting outcomes, and providing clinical care must be carefully examined by the pediatric community. Second, we did not evaluate the performance of the pSOFA as a longitudinal biomarker. Studies of the longitudinal performance of pSOFA would be helpful in further assessing its clinical utility.27,28 We present what we believe to be promising results for the pSOFA score and the adapted Sepsis-3 definitions in a single cohort of critically ill children, but further research is warranted.
Conclusions
We adapted and validated the pSOFA score, an age-adjusted pediatric version of the adult SOFA score, and used it to assess the Sepsis-3 definitions in critically ill children. The pSOFA score showed excellent discrimination for in-hospital mortality in a general PICU population and in the subgroup of patients with suspected or confirmed infection. Our assessment of the Sepsis-3 definitions in children showed promising results, but further validation in children in different settings and populations is warranted.
eTable 1. Performance of the Maximum pSOFA Score at Discriminating In-Hospital Mortality Across Risk Groups
eTable 2. Integrated Discrimination Index (IDI) for the Respiratory, Cardiovascular and Renal Subscores of SOFA vs pSOFA at Discriminating In-Hospital Mortality
eTable 3. Integrated Discrimination Index (IDI) for pSOFA Against Other Scores at Discriminating In-Hospital Mortality
eTable 4. Sensitivity Analysis: Comparison of pSOFA With Other Pediatric Organ Dysfunction Scores and PRISM-III at Discriminating In-Hospital Mortality for First PICU Admission Only
References
- 1.Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):-. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Vincent JL, Moreno R, Takala J, et al. ; Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine . The SOFA (Sepsis-Related Organ Failure Assessment) score to describe organ dysfunction/failure. Intensive Care Med. 1996;22(7):707-710. [DOI] [PubMed] [Google Scholar]
- 3.Leteurtre S, Martinot A, Duhamel A, et al. Validation of the Paediatric Logistic Organ Dysfunction (PELOD) score: prospective, observational, multicentre study. Lancet. 2003;362(9379):192-197. [DOI] [PubMed] [Google Scholar]
- 4.Leteurtre S, Duhamel A, Salleron J, Grandbastien B, Lacroix J, Leclerc F; Groupe Francophone de Réanimation et d’Urgences Pédiatriques (GFRUP) . PELOD-2: an update of the Pediatric Logistic Organ Dysfunction score. Crit Care Med. 2013;41(7):1761-1773. [DOI] [PubMed] [Google Scholar]
- 5.Graciano AL, Balko JA, Rahn DS, Ahmad N, Giroir BP. The Pediatric Multiple Organ Dysfunction Score (P-MODS): development and validation of an objective scale to measure the severity of multiple organ dysfunction in critically ill children. Crit Care Med. 2005;33(7):1484-1491. [DOI] [PubMed] [Google Scholar]
- 6.Shime N, Kageyama K, Ashida H, Tanaka Y. Application of modified sequential organ failure assessment score in children after cardiac surgery. J Cardiothorac Vasc Anesth. 2001;15(4):463-468. [DOI] [PubMed] [Google Scholar]
- 7.Jhang WK, Kim YA, Ha EJ, et al. Extrarenal sequential organ failure assessment score as an outcome predictor of critically ill children on continuous renal replacement therapy. Pediatr Nephrol. 2014;29(6):1089-1095. [DOI] [PubMed] [Google Scholar]
- 8.Sanchez-Pinto LN, Khemani RG. Development of a prediction model of early acute kidney injury in critically ill children using electronic health record data. Pediatr Crit Care Med. 2016;17(6):508-515. [DOI] [PubMed] [Google Scholar]
- 9.Hassinger AB, Garimella S, Wrotniak BH, Freudenheim JL. The current state of the diagnosis and management of acute kidney injury by pediatric critical care physicians. Pediatr Crit Care Med. 2016;17(8):e362-e370. [DOI] [PubMed] [Google Scholar]
- 10.Sanchez-Pinto LN, Goldstein SL, Schneider JB, Khemani RG. Association between progression and improvement of acute kidney injury and mortality in critically ill children. Pediatr Crit Care Med. 2015;16(8):703-710. [DOI] [PubMed] [Google Scholar]
- 11.Khemani RG, Smith LS, Zimmerman JJ, Erickson S; Pediatric Acute Lung Injury Consensus Conference Group . Pediatric acute respiratory distress syndrome: definition, incidence, and epidemiology: proceedings from the Pediatric Acute Lung Injury Consensus Conference. Pediatr Crit Care Med. 2015;16(5)(suppl 1):S23-S40. [DOI] [PubMed] [Google Scholar]
- 12.Khemani RG, Thomas NJ, Venkatachalam V, et al. ; Pediatric Acute Lung Injury and Sepsis Network Investigators (PALISI) . Comparison of Spo2 to Pao2 based markers of lung disease severity for children with acute lung injury. Crit Care Med. 2012;40(4):1309-1316. [DOI] [PubMed] [Google Scholar]
- 13.Leteurtre S, Dupré M, Dorkenoo A, Lampin ME, Leclerc F. Assessment of the pediatric index of mortality 2 with the Pao2/Fio2 ratio derived from the Spo2/Fio2 ratio: a prospective pilot study in a French pediatric intensive care unit. Pediatr Crit Care Med. 2011;12(4):e184-e186. [DOI] [PubMed] [Google Scholar]
- 14.Shime N, Kawasaki T, Nakagawa S. Proposal of a new pediatric sequential organ failure assessment score for possible validation. Pediatr Crit Care Med. 2017;18(1):98-99. [DOI] [PubMed] [Google Scholar]
- 15.Schwartz GJ, Brion LP, Spitzer A. The use of plasma creatinine concentration for estimating glomerular filtration rate in infants, children, and adolescents. Pediatr Clin North Am. 1987;34(3):571-590. [DOI] [PubMed] [Google Scholar]
- 16.Reilly PL, Simpson DA, Sprod R, Thomas L. Assessing the conscious level in infants and young children: a paediatric version of the Glasgow Coma Scale. Childs Nerv Syst. 1988;4(1):30-33. [DOI] [PubMed] [Google Scholar]
- 17.Pollack MM, Patel KM, Ruttimann UE. PRISM III: an updated Pediatric Risk of Mortality score. Crit Care Med. 1996;24(5):743-752. [DOI] [PubMed] [Google Scholar]
- 18.Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):762-774. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837-845. [PubMed] [Google Scholar]
- 20.Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157-172; discussion 207-212. [DOI] [PubMed] [Google Scholar]
- 21.Fluss R, Faraggi D, Reiser B. Estimation of the Youden Index and its associated cutoff point. Biom J. 2005;47(4):458-472. [DOI] [PubMed] [Google Scholar]
- 22.Moons KG, Altman DG, Reitsma JB, et al. Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015;162(1):W1-W73. [DOI] [PubMed] [Google Scholar]
- 23.Hotchkiss RS, Monneret G, Payen D. Sepsis-induced immunosuppression: from cellular dysfunctions to immunotherapy. Nat Rev Immunol. 2013;13(12):862-874. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Wong HR, Cvijanovich NZ, Anas N, et al. Developing a clinically feasible personalized medicine approach to pediatric septic shock. Am J Respir Crit Care Med. 2015;191(3):309-315. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Ferreira FL, Bota DP, Bross A, Mélot C, Vincent J-L. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA. 2001;286(14):1754-1758. [DOI] [PubMed] [Google Scholar]
- 26.Weiss SL, Fitzgerald JC, Maffei FA, et al. ; SPROUT Study Investigators and Pediatric Acute Lung Injury and Sepsis Investigators Network . Discordant identification of pediatric severe sepsis by research and clinical definitions in the SPROUT international point prevalence study. Crit Care. 2015;19(1):325. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Maziarz M, Heagerty P, Cai T, Zheng Y. On longitudinal prediction with time-to-event outcome: comparison of modeling options. Biometrics. 2017;73(1):83-93. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Liu D, Albert PS. Combination of longitudinal biomarkers in predicting binary events. Biostatistics. 2014;15(4):706-718. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eTable 1. Performance of the Maximum pSOFA Score at Discriminating In-Hospital Mortality Across Risk Groups
eTable 2. Integrated Discrimination Index (IDI) for the Respiratory, Cardiovascular and Renal Subscores of SOFA vs pSOFA at Discriminating In-Hospital Mortality
eTable 3. Integrated Discrimination Index (IDI) for pSOFA Against Other Scores at Discriminating In-Hospital Mortality
eTable 4. Sensitivity Analysis: Comparison of pSOFA With Other Pediatric Organ Dysfunction Scores and PRISM-III at Discriminating In-Hospital Mortality for First PICU Admission Only