Abstract
Purpose
The Sepsis-3 consensus task force defined sepsis as life-threatening organ dysfunction caused by dysregulated host response to infection. However, the clinical criteria for this definition were neither designed for nor validated in children. We validated the performance of SIRS, age-adapted SOFA, quick SOFA and PELOD-2 scores as predictors of outcome in children.
Methods
We performed a multicentre binational cohort study of patients < 18 years admitted with infection to ICUs in Australia and New Zealand. The primary outcome was ICU mortality. SIRS, age-adapted SOFA, quick SOFA and PELOD-2 scores were compared using crude and adjusted area under the receiver operating characteristic curve (AUROC) analysis.
Results
Of 2594 paediatric ICU admissions due to infection, 151 (5.8%) children died, and 949/2594 (36.6%) patients died or experienced an ICU length of stay ≥ 3 days. A ≥ 2-point increase in the individual score was associated with a crude mortality increase from 3.1 to 6.8% for SIRS, from 1.9 to 7.6% for age-adapted SOFA, from 1.7 to 7.3% for PELOD-2, and from 3.9 to 8.1% for qSOFA (p < 0.001). The discrimination of outcomes was significantly higher for SOFA (adjusted AUROC 0.829; 0.791–0.868) and PELOD-2 (0.816; 0.777–0.854) than for qSOFA (0.739; 0.695–0.784) and SIRS (0.710; 0.664–0.756).
Conclusions
SIRS criteria lack specificity to identify children with infection at substantially higher risk of mortality. We demonstrate that adapting Sepsis-3 to age-specific criteria performs better than Sepsis-2-based criteria. Our findings support the translation of Sepsis-3 into paediatric-specific sepsis definitions and highlight the importance of robust paediatric organ dysfunction characterization.
Electronic supplementary material
The online version of this article (10.1007/s00134-017-5021-8) contains supplementary material, which is available to authorized users.
Keywords: Childhood, Critical care, Mortality, Infection, PELOD, Sepsis, SIRS, SOFA, Scores
Take-home message
| The SIRS criteria lack specificity to identify children with infection at substantially higher risk of mortality. Adapting Sepsis-3 criteria using age-specific SOFA scores performs better than Sepsis-2-based criteria. Our findings support the need to translate Sepsis-3 criteria into paediatric-specific sepsis definitions and highlight the importance of robust organ dysfunction characterization in children with infections. |
Introduction
While the prevalence and mortality of paediatric sepsis has become comparable to figures reported in adult ICUs in high-income countries [1–3], defining sepsis in the absence of a gold standard remains a challenge [4]. Following the 2001 consensus statement of the Society of Critical Care Medicine, paediatric sepsis was defined as infection in the presence of at least two out of four criteria of the systemic inflammatory response syndrome (SIRS) [5, 6]. The 2005 Consensus definition for paediatric sepsis maintained the requirement for SIRS, providing further specification on organ failure definitions [6]. The validity of SIRS criteria to identify and risk-stratify patients with sepsis has been challenged in adults, where insufficient sensitivity and specificity were demonstrated [7, 8]. While tachycardia and tachypnoea represent adaptive mechanisms commonly seen in febrile childhood infections, including diseases with near-zero mortality (e.g. bronchiolitis [9]), the face, and construct validity and sensitivity of SIRS criteria have not been studied in large cohorts of critically ill children.
The recent Sepsis-3 consensus definition emphasized that sepsis is differentiated from uncomplicated infection by the presence of life-threatening organ dysfunction as a result of a dysregulated host response to infection [10]. The Delphi process, systematic reviews, and development and validation cohorts leading to Sepsis-3 were based on adult populations and the task force recognized “the need to develop similar updated definitions for pediatric populations” [11]. However, current paediatric sepsis definitions remain essentially based on Sepsis-2, representing a major obstacle towards research, benchmarking, coding, and quality monitoring [11, 12]. The operationalization of clinical criteria to identify individuals meeting outcomes consistent with sepsis in Sepsis-3 is based on the Sequential Organ Failure Assessment (SOFA) score, however neither SOFA nor quick SOFA (qSOFA) were developed for children.
We hypothesized that in children admitted to ICU with infection, the presence of organ dysfunction would better identify patients at substantially higher mortality in comparison to the presence of ≥ 2 SIRS criteria. We compared the performance of SIRS criteria with measures of organ failure to characterize outcomes of children with sepsis.
Methods
Study population
Multicentre binational cohort study of patients < 18 years admitted to ICUs in Australia and New Zealand. The study was approved by the Human Research and Ethics Committee. Patients were eligible if they had presence of suspected or proven infection [7, 8] at admission to an adult or combined adult/paediatric ICU which contributed data to the ANZICS Adult Patient Database. The ANZICS Adult Patient Database captures prospective information on more than 90% of all adult and mixed adult/paediatric ICU admissions in Australia and New Zealand, but does not include the specialized paediatric ICUs which contribute to the separate ANZPIC registry (Supplementary Material). For patients that were transferred to a PICU, cases were followed through the Australian and New Zealand Paediatric Intensive Care Registry [2, 13].
Outcomes and definitions
In-hospital mortality was defined as the primary outcome. The composite secondary outcome was defined as in-hospital mortality or ICU length of stay of 3 days or longer [8, 14].
Physiological parameters on cardiorespiratory, neurologic, hepatic, renal and haematological organ dysfunction were prospectively recorded, capturing the highest and lowest value recorded during the first 24 h of ICU admission. SIRS criteria, pediatric logistic organ dysfunction score-2 (PELOD-2), SOFA, and qSOFA were calculated (Supplementary Table 1). Age-specific cut-offs to define SIRS criteria, and definitions for severe sepsis were applied as per the 2005 Pediatric Sepsis Consensus statement and the correction provided in a subsequent author’s reply [6, 15]; paediatric SIRS was defined as presence of ≥ 2 SIRS criteria, one of which must be abnormal temperature or WCC. All PELOD-2 items, except for pupillary dilatation and serum lactate levels, were available in the database to allow calculation of a PELOD-2 ranging from zero (best) to 22 (worst) [16]. Given the absence of age-specific SOFA definitions, we developed an age-adapted SOFA by defining increasing severity of cardiovascular and renal dysfunction using the PELOD-2 cut-offs for mean arterial blood pressure and serum creatinine increase. qSOFA was defined as a score composed of three binary variables (tachypnoea, altered mentation, hypotension) [10]. Age-specific qSOFA scores were defined by applying age-specific cut-offs for respiratory rate and systolic blood pressure, respectively, as per the corrected 2005 Pediatric Sepsis definitions [6, 15].
Statistics
Data are presented as percentages and numbers or means with standard deviation. We measured the discrimination of each score using the area under the receiver operating characteristic curve (AUROC). The sensitivity, specificity, negative predictive value (NPV) and positive predictive value (PPV) was calculated for each score. A baseline risk model was developed to reflect the underlying risk of a patient developing the primary and secondary outcomes using available information at the time of ICU admission not contained in any of the scores. Univariate mixed effects logistic regression models, with a random effect for each site, were used to assess associations between patient factors and the primary outcome. Variables with associations p < 0.2 where considered for inclusion in a multivariable model. The same model was applied to the secondary outcome.
Sensitivity analyses were performed by using age- and sex-specific systolic blood pressure cut-offs based on the 5th percentile previously validated in children with sepsis [13, 17], and by using systolic blood pressure cut-offs used to define arterial hypotension in the corrected 2005 consensus definition [6, 15]. Analyses were conducted using Stata (version 14.0, Stata Corp, College Station, TX, USA). p values < 0.05 were considered significant.
Results
Study population
Between 2000 and 2016, 2,715 patients aged < 18 years were admitted to an adult or mixed ICU because of infection and recorded in the ANZICS Adult Patient Database. 121 episodes were excluded: one (0.04%) duplicated record, 49 (1.8%) patients > 16 years were transferred alive to another adult ICU with unknown outcome and 71 (2.6%) had missing outcome data. A final cohort of 2594 paediatric ICU admission encounters due to infection with known outcomes were identified, with a median age of 13 years (IQR 1–16). 151 (5.8%) children died in hospital and 949 (36.6%) died in hospital or experienced an ICU length of stay of 3 days or more (Supplementary Table 2). 1510 (58.3%) were classified as severe sepsis. The mortality in this group was 7.4% and the secondary outcome was met in 43.9% of patients with severe sepsis.
SIRS criteria
Of all 2594 episodes, SIRS data was incomplete in 323 (12.4%). 57/2271(2.2%) children did not present with any SIRS criteria during the first 24 h of ICU admission, 356 (15.6%) met one SIRS criterion, 1858/2271 (81.8%) fulfilled at least two SIRS criteria, and 1675 (73.8%) met paediatric SIRS (Table 1, Fig. 1). Mortality increased from 3.1% in presence of < 2 SIRS criteria to 6.8% if ≥ 2 SIRS criteria were present (between-group difference, 3.6%, 95% CI 1.6–5.7, p = 0.005, Fig. 2, Supplementary Figs. 1 and 2). Using patients with < 2 SIRS criteria as a reference, the relative increase in the primary and secondary outcomes was not significant for 2 SIRS criteria, but was significant for 3 SIRS criteria (primary outcome OR 1.94, 95% CI 1.02–3.70); secondary outcome OR 1.46, 95% CI 1.12–1.92), and for 4 SIRS criteria (primary outcome OR 3.31, 95% CI 1.72–6.37; secondary outcome OR 2.97, 95% CI 2.19–4.03).
Table 1.
Distribution of signs meeting SIRS criteria in children admitted to ICU with infection, according to primary outcome (mortality) and secondary outcome (mortality or ICU stay ≥ 3 days)
| SIRS criterion met, N (%) | All patients | Primary outcome (mortality) | Secondary outcome (mortality or ICU LOS ≥ 3 days) | |||
|---|---|---|---|---|---|---|
| N | % | N | % | N | % | |
| Abnormal heart rate | 1295 | 57.0 | 108 | 8.3 | 593 | 45.8 |
| Increased respiratory rate | 1896 | 83.5 | 106 | 5.6 | 700 | 36.9 |
| Abnormal temperature | 1091 | 48.0 | 89 | 8.2 | 509 | 46.7 |
| High temperature | 669 | 29.5 | 47 | 31.1 | 337 | 35.5 |
| Low temperature | 487 | 21.4 | 49 | 32.5 | 211 | 22.2 |
| Abnormal white-cell count | 1342 | 59.1 | 103 | 7.7 | 574 | 42.8 |
| High WCC | 1106 | 48.7 | 57 | 37.8 | 446 | 47 |
| Low WCC | 248 | 10.9 | 46 | 30.5 | 135 | 14.2 |
| 2 or More SIRS criteria | ||||||
| Not present | 413 | 18.2 | 13 | 3.2 | 120 | 29.1 |
| Present | 1858 | 81.8 | 126 | 6.8 | 758 | 40.8 |
| Paediatric SIRS (2 or more SIRS criteria, one of which must be abnormal temperature or white cell count) | ||||||
| Not present | 596 | 26.2 | 21 | 3.5 | 176 | 29.5 |
| Present | 1675 | 73.8 | 118 | 7.0 | 702 | 41.9 |
| Number of SIRS criteria present | All patients | Primary outcome (mortality) | Secondary outcome (mortality or ICU LOS ≥ 3 days) | |||
|---|---|---|---|---|---|---|
| N | % | N | % (95% CI) | N | % (95% CI) | |
| 0 | 57 | 2.5 | 2 | 3.5 (0.04–12.1) | 19 | 33.3 (21.4–47.1) |
| 1 | 356 | 15.6 | 11 | 3.1 (1.6–5.5) | 101 | 28.4 (23.7–33.4) |
| 2 | 704 | 31.0 | 30 | 4.3 (2.9–6.0) | 230 | 32.7 (29.2–36.3) |
| 3 | 756 | 33.3 | 49 | 6.5 (4.8–8.5) | 297 | 39.3 (35.8–42.9) |
| 4 | 398 | 17.5 | 47 | 11.8 (8.8–15.4) | 231 | 58.0 (53.0–62.9) |
LOS intensive care unit length of stay, SIRS systemic inflammatory response syndrome
Fig. 1.
Distribution of patients < 18 years with infection by SIRS criteria, PELOD-2 score, SOFA and qSOFA score measured during the first 24 h of ICU admission
Fig. 2.
Mortality by SIRS criteria, PELOD-2 score, SOFA and qSOFA score measured during the first 24 h of ICU admission in patients < 18 years admitted with infection
Using models adjusted for baseline risk, the relative odds of death increased in the presence of paediatric SIRS (OR 1.83, 95% CI 1.13–2.99, p = 0.015), or the presence of any ≥ 2 SIRS criteria (OR 2.00, 95% CI 1.10–3.64, p = 0.023). The relative odds of the secondary outcome increased in the presence of paediatric SIRS (OR 1.54, 95% CI 1.25–1.90, p < 0.001), and in the presence of ≥ 2 SIRS criteria (OR 1.50, 95% CI 1.21–1.85, p < 0.001). Overall, each additional SIRS criterion was associated with a 50% increase in the relative odds for the primary outcome (OR 1.50, 95% CI 1.25–1.81, p < 0.001) and a 38% increase for the secondary outcome (OR 1.38, 95% CI 1.27–1.52, p < 0.001).
SOFA, PELOD-2 and qSOFA
1690 (74.2%) of infected patients had an age-adapted SOFA score of ≥ 2 (Fig. 1). The mortality increased from 1.9 to 7.6% if the SOFA score was ≥ 2 (between-group difference, 5.7%, 95% CI 4.0–7.4, p < 0.001, Fig. 2). The risk of the secondary outcome increased from 17.6 to 46.1% in those with a SOFA score of ≥ 2 (between-group difference, 28.5%, 95% CI 24.7–32.4, p < 0.001, Supplementary Fig. 2).
When assessing organ dysfunction using PELOD-2, a score of ≥ 8 performed best in identifying patients at higher mortality: There were 374 (14.4%) children with a PELOD-2 score ≥ 8, with a mortality of 22.2% (versus 3.0% in those with scores < 8, p < 0.001) and a secondary outcome incidence of 79.4% (versus 20.6%, p < 0.001). Among children with a PELOD-2 score of ≥ 2, mortality was 7.3% (versus 1.7% in those with scores < 2, p < 0.001) and the secondary outcome occured in 43.8% (versus 17.2%, p < 0.001).
For those who had a qSOFA (altered mentation, arterial hypotension, and tachypnea) score of ≥ 2, mortality was 8.1% (97/1200) in comparison to 3.9% (41/1059) with a qSOFA < 2 (between-group difference, 4.2%; 95% CI 2.3–6.1, p < 0.001).
Comparison of SIRS, severe sepsis, SOFA, PELOD-2 and qSOFA
There were significant differences in discrimination of both primary and secondary outcomes in crude and adjusted analyses (p < 0.001). For the primary outcome, discrimination was highest for SOFA (AUROC = 0.829) which was significantly higher than SIRS (AUROC = 0.727, p < 0.001), severe sepsis (AUROC = 0.711, p < 0.001), and qSOFA (AUROC = 0.739, p < 0.001), though not significantly higher than PELOD-2 (AUROC = 0.816, p = 0.970). For the secondary outcome, discrimination was highest for PELOD-2 (AUROC = 0.771), which was significantly higher than SIRS (AUROC = 0.676, p < 0.001), severe sepsis (AUROC = 0.677, p < 0.001), qSOFA (AUROC = 0.682, p < 0.001), and SOFA (AUROC = 0.751, p < 0.001) (Table 2, Fig. 3, Supplementary Table 3). The best binary performance for PELOD-2 was using a cutpoint score of ≥ 8, resulting in an adjusted AUROC for in-hospital mortality of 0.812 (95% CI 0.774–0.851), and a sensitivity of 88.1% and a specificity of 55.7%.
Table 2.
Crude and adjusted AUROCs for discrimination characteristics of SIRS, SOFA, PELOD-2 and qSOFA on ICU admission in patients < 18 years with suspected or confirmed infection
| Predictor | Definition | Primary outcome: Hospital mortality | Secondary outcome: mortality and/or ICU LOS ≥ 3 days | ||
|---|---|---|---|---|---|
| Area under the curve | Area under the curve | ||||
| Crude | Adjusted | Crude | Adjusted | ||
| SIRS | SIRS N criteria | 0.630 (0.583–0.676) | 0.727 (0.682–0.772) | 0.603 (0.580–0.626) | 0.676 (0.654–0.699) |
| SIRS ≥ 2 criteria (paediatric) | 0.559 (0.528–0.591) | 0.710 (0.664–0.756) | 0.551 (0.533–0.568) | 0.664 (0.641–0.687) | |
| SIRS ≥ 2 criteria (any) | 0.563 (0.532–0.594) | 0.708 (0.664–0.751) | 0.565 (0.548–0.582) | 0.673 (0.652–0.694) | |
| Severe sepsis | 2005 consensus | 0.586 (0.550–0.622) | 0.711 (0.667–0.755) | 0.592 (0.574–0.611) | 0.677 (0.656–0.698) |
| SOFA | SOFA score | 0.782 (0.738–0.827) | 0.829 (0.791–0.868) | 0.731 (0.710–0.752) | 0.751 (0.730–0.772) |
| SOFA ≥ 2 criteria | 0.595 (0.571–0.620) | 0.743 (0.701–0.785) | 0.615 (0.599–0.631) | 0.702 (0.689–0.723) | |
| PELOD | PELOD score | 0.774 (0.731–0.818) | 0.816 (0.777–0.854) | 0.750 (0.730–0.770) | 0.771 (0.752–0.790) |
| PELOD ≥ 2 criteria | 0.601 (0.578–0.625) | 0.726 (0.685–0.767) | 0.613 (0.597–0.629) | 0.694 (0.673–0.715) | |
| PELOD ≥ 8 criteria | 0.719 (0.679–0.759) | 0.812 (0.774–0.851) | 0.633 (0.618–0.649) | 0.744 (0.724–0.764) | |
| qSOFA | qSOFA score | 0.638 (0.588–0.687) | 0.739 (0.695–0.784) | 0.597 (0.575–0.620) | 0.682 (0.659–0.704) |
| qSOFA ≥ 2 criteria | 0.591 (0.552–0.631) | 0.722 (0.677–0.767) | 0.581 (0.560–0.601) | 0.679 (0.656–0.701) | |
| Predictor | Definition | Primary outcome: hospital mortality | |||
|---|---|---|---|---|---|
| Sensitivity (%) | Specificity (%) | Negative predictive value (%) | Positive predictive value (%) | ||
| SIRS | SIRS ≥ 2 criteria (pediatric) | 84.9 | 27.0 | 96.5 | 7.0 |
| SIRS | SIRS ≥ 2 criteria (any) | 83.4 | 29.1 | 96.6 | 6.8 |
| Severe sepsis | 2005 consensus | 74.5 | 42.7 | 96.5 | 7.4 |
| SOFA | SOFA ≥ 2 criteria | 92.1 | 26.9 | 98.1 | 7.6 |
| qSOFA | qSOFA ≥ 2 criteria | 70.3 | 48.0 | 96.1 | 8.1 |
| PELOD | PELOD ≥ 2 criteria | 91.9 | 28.3 | 98.3 | 7.3 |
| PELOD | PELOD ≥ 8 criteria | 88.1 | 55.7 | 97.0 | 22.2 |
Fig. 3.
Comparison of area under the receiver operating characteristic curves (AUROCS) to discriminate in-hospital mortality (primary outcome) and in-hospital mortality or ICU length of stay of 3 days or more (secondary outcome) for SIRS criteria, SOFA, qSOFA, and PELOD scores at ICU admission. AUROCs are shown for primary (a, c) and secondary (b, d) outcomes using crude (a, b) and adjusted (c, d) models
Sensitivity analyses were performed using different adjusted models, and based on different thresholds to define qSOFA, which resulted similar (Online Supplementary Table 3).
Discussion
In this multicentre cohort of 2594 children aged < 18 years admitted to ICU with infection, we externally validated and assessed the discriminatory capacities of SIRS, severe sepsis, SOFA, PELOD-2, and qSOFA. We observed superior prognostic accuracy of SOFA and PELOD-2, both for in-hospital mortality and for the composite outcome of in-hospital mortality or ICU length of stay of ≥ 3 days, in comparison with SIRS, severe sepsis, or qSOFA. SIRS lacked specificity to identify children with infection at substantially higher mortality risk.
Key features underlying the Sepsis-3 consensus definition relate to the differentiation of sepsis from non-life-threatening infection, operationalization of the definition, and establishment through a data-driven process using large cohorts [18]. In contrast, paediatric Sepsis-2 definitions focus on systemic inflammation, applying non-validated criteria commonly seen outside sepsis, and have specific requirements for individual organ dysfunctions, attributing more weight to cardiovascular or respiratory organ dysfunction [19, 20]. The paradigm of SIRS as a feature of paediatric sepsis has been maintained for two decades, but neither SIRS nor the particular organ dysfunction criteria—which overlap with multiorgan dysfunction—to define severe sepsis in children have been externally validated in large ICU cohorts. Previous studies reported that SIRS criteria are met in > 90% of febrile children presenting to ED, of which < 5% require ICU admission [21]. We demonstrate that ≥ 2 SIRS criteria are present in 81.8% of paediatric patients admitted to ICU with infection, resulting in poor specificity and poor positive predictive value to capture patients at risk for adverse outcomes. While each additional SIRS criterion was associated with an increase in the relative odds of the primary, and secondary outcomes, significance was only reached when 3 or 4 SIRS criteria were present. Furthermore, the sensitivity of ≥ 2 SIRS criteria to discriminate in-hospital mortality was inferior to an incremental increase by ≥ 2 points in SOFA or PELOD-2, challenging the common notion that SIRS has excellent sensitivity for sepsis. Our findings are supported by a study in critically ill adults, demonstrating shortcomings in sensitivity, specificity, and validity of ≥ 2 SIRS criteria to define sepsis [7]. The limited utility if SIRS is further illustrated by substantial differences when applying Sepsis-3 versus Sepsis-2 criteria to infected children [11]. In a recent study on children with bloodstream infection, 30-day mortality was 1% in the presence of bacteraemia and SIRS without organ dysfunction, but increased to 17% when organ dysfunction was present [22]. Hence, our findings support abandoning SIRS since operationalising inflammation performs inferior to operationalising organ dysfunction when predicting death or prolonged ICU stay as outcomes.
Both age-adapted SOFA and PELOD-2 were superior to SIRS in identifying patients with infection at greater risk of mortality. Given the limited evidence on optimal blood pressure thresholds [23, 24], and to avoid overfitting of models, we applied the validated PELOD-2 age-specific cut-offs for cardiovascular and renal dysfunction, which may partially account for the similar performance observed between SOFA and PELOD-2 [25]. When simplifying the discrete scores (SIRS, SOFA, PELOD-2, qSOFA) to binary categorizations, PELOD ≥ 8 performed best. Notably, Sepsis-3 defined an increase in SOFA by ≥ 2 points based on the a priori requirement to identify presence of ≥ 1 (new) organ dysfunction to characterize sepsis, and not by post hoc derivation of optimal cut-offs. While a PELOD-2 ≥ 2 will capture patients with ≥ 1 organ dysfunction, PELOD ≥ 8 performed best in our study but such scores will predominantly reflect multiorgan dysfunction. Our findings support the operationalization of clinical criteria to paediatric patients with sepsis [10] and are highly comparable to a large external validation cohort in critically ill adults captured by the same database [8]. Leclerc and colleagues assessed PELOD-2 in 862 children with infection recruited in the original PELOD cohort and reported a high in-sample performance [23]. Our findings are further supported by a recent single centre PICU study including patients up to 21 years which tested a paediatric SOFA adaptation [26] based on the same PELOD-2 cut-offs for arterial hypotension and renal dysfunction but applying a more granular score increase, resulting in excellent performance. In contrast to the paper by Matics et al. [26] which did not report on SIRS or severe sepsis, we analyzed multicentre data of patients < 18 years, using mortality as the primary and mortality and/or PICU length of stay ≥ 3 days as the composite secondary outcome, and applied crude and adjusted analyses using similar methodology to adult Sepsis-3 validation cohorts [8].
qSOFA has been proposed as a screening tool in adults with infection, prompting assessment for evidence of organ dysfunction on hospital floors or in the ED [10]. In our study, the performance of our adapted qSOFA score to identify children who subsequently died or had prolonged length of stay was only moderate, which potentially could reflect the use of ventilation, sedation, and inotropes in ICU, altering respiratory rate, blood pressure, and GCS. In comparison, other PICU sepsis scores and paediatric Early Warning Tools have reported AUROCs of > 0.80 [12, 27]. Specific features of paediatric sepsis and septic shock [28], such as late development of arterial hypotension and a higher proportion of fulminant presentations [12, 22, 29, 30], warrant improved rapid identification of infected paediatric patients with organ dysfunction [30] across emergency department, hospital ward, and ICU settings. Given the high proportion of patients with organ dysfunction (74% with SOFA ≥ 2) in our cohort, future studies are needed to test the discriminatory performance in cohorts of lower average acuity.
The key strengths of this study relate to the application of stringent data-driven validation procedures to allow comparison of the prognostic accuracy of SIRS, severe sepsis, SOFA, PELOD-2 and qSOFA. Moreover, given the limitations of using mortality as an outcome, analyses included the composite outcome of mortality or ICU length of stay of 3 days or longer, aligned with adult Sepsis-3 studies [8, 13]. We deliberately did not restrict the study to admissions coded as sepsis but instead included all children admitted to ICUs with suspected and confirmed infection. Diagnoses were based on assessment by trained ICU specialists and not administrative coding.
Our study carries several limitations. First, it is based on a binational prospective dataset of adult and mixed adult-paediatric ICUs, whereas the main PICUs contribute to a different registry, which may have led to selection bias. In contrast to the paediatric ANZPIC registry, the ANZICS APD has been prospectively collecting data on SIRS, APACHE and SOFA score in patients admitted to the contributing units which ensured consistent practices of organ dysfunction assessment and data monitoring. Due to the high centralization of PICU services in Australia and New Zealand, it is common for critically ill children outside large metropolitan areas to be admitted to mixed ICUs. In the present study, 149 (5.8%) children required secondary interhospital transfer to a PICU, all of which were tracked to capture the primary and secondary outcome. Second, capturing the worst parameters within the first 24 h of ICU admission may not capture peak disease severity, resulting in lower performance of the scores [26]. However, prediction requires early assessment by definition and the passage of time is associated with competing risk bias due to death or discharge from ICU. In a recent study on a different cohort of septic children admitted to specialized PICUs we have demonstrated that a small set of clinical variables available within the first hour of PICU admission allows to establish robust severity stratification for paediatric sepsis mortality [12]. Third, although data collection for this binational ICU registry had been monitored using regular quality controls and mandatory audits, data had not been primarily captured for sepsis studies. Fourth, the SOFA score was modified as detailed information on vasopressor type and dose was not consistently available. Finally, two items of the PELOD score, including pupil size and serum lactate levels, were not available in the database, which may have reduced the performance of PELOD. Several studies have identified lactate as one of the best predictors of paediatric sepsis severity [12, 31, 32].
In conclusion, the two SIRS variables based sepsis criteria had poor specificity and diagnostic performance to discriminate children with infection at substantially higher mortality risk. In contrast, SOFA and PELOD-2 had significantly greater prognostic accuracy for in-hospital mortality. Our findings indicate that age-specific translation of Sepsis-3 definitions to critically ill children using validated measures of organ dysfunction should be considered in the next revision of paediatric sepsis definitions. In contrast, the performance of qSOFA to identify patients with organ dysfunction at risk for worse outcomes was poor, and may not be of sufficient clinical value to be recommended as a screening tool for paediatric age groups within the ICU.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Acknowledgements
The authors would like to thank the medical and nursing teams in the ICUs contributing to the database, and the data teams performing monitoring and validation.
Abbreviations
- PICU
Paediatric intensive care unit
- OR
Odds ratio
- SSC
Surviving Sepsis Campaign
- PELOD
Paediatric logistic organ dysfunction
- SIRS
Systemic inflammatory response syndrome
- SOFA
Sequential (Sepsis-related) Organ Failure Assessment
- qSOFA
Quick Sequential (Sepsis-related) Organ Failure Assessment
Funding source
This work was supported by a Clinical Research Seeding Grant from Mater Research Institute—The University of Queensland, and by a grant from the Norva Dahlia Foundation, Australia. The study was supported by the Australian and New Zealand Intensive Care Research Centre, which is supported by an enabling Grant (0606978) from the Australian National Health and Medical Research Council. The funding source had no involvement in study design, analyses, nor interpretation of the results.
Conflicts of interest
None of the authors have declared a conflict of interest.
References
- 1.Weiss SL, Fitzgerald JC, Pappachan J, Wheeler D, Jaramillo-Bustamante JC, Salloo A, Singhi SC, Erickson S, Roy JA, Bush JL, Nadkarni VM, Thomas NJ, Sepsis Prevalence O. Therapies Study I. Pediatric Acute Lung I. Sepsis Investigators N Global epidemiology of pediatric severe sepsis: the sepsis prevalence, outcomes, and therapies study. Am J Respir Crit Care Med. 2015;191:1147–1157. doi: 10.1164/rccm.201412-2323OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Schlapbach LJ, Straney L, Alexander J, MacLaren G, Festa M, Schibler A, Slater A, Group APS Mortality related to invasive infections, sepsis, and septic shock in critically ill children in Australia and New Zealand, 2002–13: a multicentre retrospective cohort study. Lancet Infect Dis. 2015;15:46–54. doi: 10.1016/S1473-3099(14)71003-5. [DOI] [PubMed] [Google Scholar]
- 3.Hartman ME, Linde-Zwirble WT, Angus DC, Watson RS. Trends in the epidemiology of pediatric severe sepsis. Pediatr Crit Care Med. 2013;14:686–693. doi: 10.1097/PCC.0b013e3182917fad. [DOI] [PubMed] [Google Scholar]
- 4.Angus DC. Opening the debate on the new sepsis definition defining sepsis: a case of bounded rationality and fuzzy thinking? Am J Respir Crit Care Med. 2016;194:14–15. doi: 10.1164/rccm.201604-0879ED. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Carcillo JA, Fields AI, American College of Critical Care Medicine Task Force Committee M Clinical practice parameters for hemodynamic support of pediatric and neonatal patients in septic shock. Crit Care Med. 2002;30:1365–1378. doi: 10.1097/00003246-200206000-00040. [DOI] [PubMed] [Google Scholar]
- 6.Goldstein B, Giroir B, Randolph A. International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med. 2005;6:2–8. doi: 10.1097/01.PCC.0000149131.72248.E6. [DOI] [PubMed] [Google Scholar]
- 7.Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med. 2015;372:1629–1638. doi: 10.1056/NEJMoa1415236. [DOI] [PubMed] [Google Scholar]
- 8.Raith EP, Udy AA, Bailey M, McGloughlin S, MacIsaac C, Bellomo R, Pilcher DV, Australian, New Zealand Intensive Care Society Centre for O, Resource E Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA. 2017;317:290–300. doi: 10.1001/jama.2016.20328. [DOI] [PubMed] [Google Scholar]
- 9.Schlapbach LJ, Straney L, Gelbart B, Alexander J, Franklin D, Beca J, Whitty JA, Ganu S, Wilkins B, Slater A, Croston E, Erickson S, Schibler A, Australian, New Zealand Intensive Care Society Centre for O, Resource E, the A, New Zealand Intensive Care Society Paediatric Study G, (2017) Burden of disease and change in practice in critically ill infants with bronchiolitis. Eur Respir J 49(6). 10.1183/13993003.01648-2016 [DOI] [PubMed]
- 10.Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, Hotchkiss RS, Levy MM, Marshall JC, Martin GS, Opal SM, Rubenfeld GD, van der Poll T, Vincent JL, Angus DC. The third international consensus definitions for sepsis and septic shock (Sepsis-3) JAMA. 2016;315:801–810. doi: 10.1001/jama.2016.0287. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Schlapbach LJ. Time for Sepsis-3 in children? Pediatr Crit Care Med. 2017;18:805–806. doi: 10.1097/PCC.0000000000001203. [DOI] [PubMed] [Google Scholar]
- 12.Schlapbach LJ, Kisson N (2018) Pediatric sepsis definitions - an urgent need for change. JAMA Pediatrics (in press)
- 13.Schlapbach LJ, MacLaren G, Festa M, Alexander J, Erickson S, Beca J, Slater A, Schibler A, Pilcher D, Millar J, Straney L, Australian, New Zealand Intensive Care Society Centre for O, Resource E, Australian, New Zealand Intensive Care Society Paediatric Study G Prediction of pediatric sepsis mortality within 1 h of intensive care admission. Intensive Care Med. 2017;43:1085–1096. doi: 10.1007/s00134-017-4701-8. [DOI] [PubMed] [Google Scholar]
- 14.Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, Rubenfeld G, Kahn JM, Shankar-Hari M, Singer M, Deutschman CS, Escobar GJ, Angus DC. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3) JAMA. 2016;315:762–774. doi: 10.1001/jama.2016.0288. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Gebara BM. Values for systolic blood pressure. Pediatr Crit Care Med. 2005;6:500. doi: 10.1097/01.PCC.0000164344.07588.83. [DOI] [PubMed] [Google Scholar]
- 16.Leteurtre S, Duhamel A, Salleron J, Grandbastien B, Lacroix J, Leclerc F, Groupe Francophone de Reanimation et d’Urgences P PELOD-2: an update of the PEdiatric logistic organ dysfunction score. Crit Care Med. 2013;41:1761–1773. doi: 10.1097/CCM.0b013e31828a2bbd. [DOI] [PubMed] [Google Scholar]
- 17.Haque IU, Zaritsky AL. Analysis of the evidence for the lower limit of systolic and mean arterial pressure in children. Pediatr Crit Care Med. 2007;8:138–144. doi: 10.1097/01.PCC.0000257039.32593.DC. [DOI] [PubMed] [Google Scholar]
- 18.Singer M. The new sepsis consensus definitions (Sepsis-3): the good, the not-so-bad, and the actually-quite-pretty. Intensive Care Med. 2016;42:2027–2029. doi: 10.1007/s00134-016-4600-4. [DOI] [PubMed] [Google Scholar]
- 19.Vincent JL, Mira JP, Antonelli M. Sepsis: older and newer concepts. Lancet Respir Med. 2016;4:237–240. doi: 10.1016/S2213-2600(15)00522-6. [DOI] [PubMed] [Google Scholar]
- 20.Vincent JL, Opal SM, Marshall JC, Tracey KJ. Sepsis definitions: time for change. Lancet. 2013;381:774–775. doi: 10.1016/S0140-6736(12)61815-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Scott HF, Deakyne SJ, Woods JM, Bajaj L. The prevalence and diagnostic utility of systemic inflammatory response syndrome vital signs in a pediatric emergency department. Acad Emerg Med. 2015;22:381–389. doi: 10.1111/acem.12610. [DOI] [PubMed] [Google Scholar]
- 22.Agyeman PKA, Schlapbach LJ, Giannoni E, Stocker M, Posfay-Barbe KM, Heininger U, Schindler M, Korten I, Konetzny G, Niederer-Loher A, Kahlert CR, Donas A, Leone A, Hasters P, Relly C, Baer W, Kuehni CE, Aebi C, Berger C. Epidemiology of blood culture-proven bacterial sepsis in children in Switzerland: a population-based cohort study. Lancet Child Adolesc Health. 2017;1:124–133. doi: 10.1016/S2352-4642(17)30010-X. [DOI] [PubMed] [Google Scholar]
- 23.Marlais M, Lyttle MD, Inwald D. Ten concerns about blood pressure measurement and targets in paediatric sepsis. Intensive Care Med. 2017;43:433–435. doi: 10.1007/s00134-016-4642-7. [DOI] [PubMed] [Google Scholar]
- 24.Ray S, Rogers L, Noren DP, Dhar R, Nadel S, Peters MJ, Inwald DP. Risk of over-diagnosis of hypotension in children: a comparative analysis of over 50,000 blood pressure measurements. Intensive Care Med. 2017;43(10):1540–1541. doi: 10.1007/s00134-017-4843-8. [DOI] [PubMed] [Google Scholar]
- 25.Leclerc F, Duhamel A, Deken V, Grandbastien B, Leteurtre S, Groupe Francophone de Reanimation et Urgences P Can the pediatric logistic organ dysfunction-2 score on day 1 be used in clinical criteria for sepsis in children? Pediatr Crit Care Med. 2017;18:758–763. doi: 10.1097/PCC.0000000000001182. [DOI] [PubMed] [Google Scholar]
- 26.Matics TJ, Sanchez-Pinto LN. Adaptation and validation of a pediatric sequential organ failure assessment score and evaluation of the Sepsis-3 definitions in critically ill children. JAMA Pediatr. 2017;171(10):e172352. doi: 10.1001/jamapediatrics.2017.2352. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Chapman SM, Wray J, Oulton K, Pagel C, Ray S, Peters MJ. ‘The Score Matters’: wide variations in predictive performance of 18 paediatric track and trigger systems. Arch Dis Child. 2017;102:487–495. doi: 10.1136/archdischild-2016-311088. [DOI] [PubMed] [Google Scholar]
- 28.Morin L, Ray S, Wilson C, Remy S, Benissa MR, Jansen NJ, Javouhey E, Peters MJ, Kneyber M, De Luca D, Nadel S, Schlapbach LJ, Maclaren G, Tissieres P, ESPNIC Refractory Septic Shock Definition Taskforce tISISsoE Refractory septic shock in children: a European Society of Paediatric and neonatal intensive care definition. Intensive Care Med. 2016;42:1948–1957. doi: 10.1007/s00134-016-4574-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Cvetkovic M, Lutman D, Ramnarayan P, Pathan N, Inwald DP, Peters MJ. Timing of death in children referred for intensive care with severe sepsis: implications for interventional studies. Pediatr Crit Care Med. 2015;16:410–417. doi: 10.1097/PCC.0000000000000385. [DOI] [PubMed] [Google Scholar]
- 30.Lin JC, Spinella PC, Fitzgerald JC, Tucci M, Bush JL, Nadkarni VM, Thomas NJ, Weiss SL, Sepsis Prevalence O. Therapy Study I New or progressive multiple organ dysfunction syndrome in pediatric severe sepsis: a sepsis phenotype with higher morbidity and mortality. Pediatr Crit Care Med. 2017;18:8–16. doi: 10.1097/PCC.0000000000000978. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Scott HF, Brou L, Deakyne SJ, Kempe A, Fairclough DL, Bajaj L. Association between early lactate levels and 30-day mortality in clinically suspected sepsis in children. JAMA Pediatr. 2017;171:249–255. doi: 10.1001/jamapediatrics.2016.3681. [DOI] [PubMed] [Google Scholar]
- 32.Schlapbach LJ, MacLaren G, Straney L. Venous versus arterial lactate and 30-day mortality in pediatric sepsis. JAMA Pediatr. 2017;171:813. doi: 10.1001/jamapediatrics.2017.1598. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.