Abstract
Background
Sepsis is the leading cause of intensive care unit (ICU) admission. The purpose of this study was to explore the prognostic value of the Sequential Organ Failure Assessment (SOFA) score, the Acute Physiological and Chronic Health Evaluation II (APACHE II) score, and procalcitonin (PCT), albumin (ALB), and lactate (LAC) levels in patients with sepsis.
Methods
Consecutive adult patients with suspected or documented sepsis at ICU admission were recruited. Their basic vital signs and related auxiliary examinations to determine their PCT and ALB levels and APACHE II score were recorded at ICU admission, and their LAC levels and SOFA scores were recorded for one week after admission. The influence of these variables on hospital mortality was evaluated. Logistic regression was used to derive the Sepsis Hospital Mortality Score (SHMS), a prediction equation describing the relationship between predictors and hospital mortality. The median survival time was calculated by the Kaplan–Meier method. In the validation group, the kappa value was calculated to evaluate the stability of the derived formula.
Results
This study included 894 sepsis patients admitted to 18 ICUs in 16 tertiary hospitals. Patients were randomly assigned to an experimental group (626 cases) and validation group (258 cases). In addition, a nonsurvival group (248 patients) of the experimental group was established according to the outcome at the time of discharge. The hospital mortality rate in the experimental group was 39.6% (248/626). Univariate and multivariate regression analyses revealed that the APACHE II score (odds ratio [OR] = 1.178), △SOFA (OR = 1.186), △LAC (OR = 1.157), and SOFA mean score (OR = 1.086) were independently associated with hospital mortality. The SHMS was calculated as logit(p) = 4.715 – (0.164 × APACHE II) – (0.171 × △SOFA) – (0.145 × △LAC) – (0.082 × SOFA mean). A receiver operating characteristic curve was constructed to further investigate the accuracy of the SHMS, with an area under the curve of 0.851 (95% confidence interval [CI] 0.821–0.882; p < 0.001) for hospital mortality. In the low-risk group and high-risk groups, the corresponding median survival times were 15 days and 11 days, respectively.
Conclusion
The APACHE II score, △SOFA, △LAC and SOFA mean score were independently associated with hospital mortality in sepsis patients and accurately predicted the hospital mortality rate and median survival time. Data on the median survival time in sepsis patients could be provided to clinicians to assist in the rational use of limited medical resources by facilitating prudent resource allocation.
Trial registration
ChiCTR-ECH-13003934, retrospectively registered on August 03, 2013.
Keywords: Sepsis, SOFA, APACHE II, Lactate, Mortality, Sequential Organ Failure Assessment, Acute Physiological and Chronic Health Evaluation II
Highlights.
This article is a multicenter study which was performed from January 2014 to August 2015 in 18 ICUs in 16 tertiary hospitals in China.
The APACHE II score, ΔSOFA, ΔLAC and SOFA mean score were independently associated with hospital mortality in sepsis patients and accurately predicted the hospital mortality rate and median survival time.
Data on the median survival time in sepsis patients could be provided to clinicians to assist in the rational use of limited medical resources by facilitating prudent resource allocation.
Background
Sepsis is the leading cause of intensive care unit (ICU) admission [1]. In addition, sepsis is associated with concurrent multiple organ dysfunction syndrome, which is the main cause of death in ICU patients [2]. The social and economic impact of sepsis consumes a considerable proportion of healthcare resources [3, 4]. Despite the decline in mortality in the past decade due to increased sepsis awareness and management, the short-term mortality rate has remained at 20% or higher as the population ages, the number of invasive medical procedures increases, and the incidence of cancer in the elderly population increases [5, 6]. Early stratification and identification of patients with a high risk of death is essential [7]. The early warning score is an alternative tool for risk stratification. This will help guide clinicians in developing different treatment plans in a timely manner for the individual patient with different risk levels.
The latest definition of sepsis emphasizes organ failure [8], referring to two or more changes in the Sequential Organ Failure (SOFA) score. One of the scoring systems used for predicting mortality in septic patients is the initial SOFA score. Harm-Jan de Grooth and colleagues [9] pointed out that the effects of treatment on the change in the SOFA score appear to be reliably and consistently associated with mortality in random controlled trials (RCTs). Fixed-day SOFA was the most frequently reported outcome among the reviewed RCTs, but it was not significantly associated with mortality.
One of the characteristics of a clinical early warning scoring system is that it needs to apply commonly used and easily available predictive indicators. The Acute Physiological and Chronic Health Evaluation II (APACHE II) score, SOFA score, lactate (LAC) and procalcitonin (PCT) levels are all meet this requirement. However, the establishment of the diagnostic criteria was based on a retrospective analysis of the database, and significant data loss inevitably affected the final results. Their limited performance prevents these markers from being applied to individual risk stratifications [10, 11].
Combining several predictors into a single classification rule should help to improve their accuracy and practicability. We investigated the dynamic changes in the SOFA score and LAC levels in patients with sepsis during the first week after admission. The purpose of the present study was to derive a prediction equation using a combination of the APACHE II score, SOFA score, and LAC and PCT levels to predict hospital mortality in sepsis patients; this combined score is called the sepsis hospital mortality score (SHMS). Furthermore, according to the formula, the severity of the conditions of patients was evaluated, and stratification was performed to calculate the median survival time corresponding to each category of patients. Data on the median survival time in sepsis patients could be provided to clinicians to assist in the rational use of limited medical resources by facilitating prudent resource allocation and appropriate classification for research or administrative purposes.
Methods
Patient selection
This prospective observational study was performed from January 2014 to August 2015 in 18 ICUs in 16 tertiary hospitals in China.
The sepsis 1.0 standard was used to establish the database initially [12]. We conducted a retrospective study with information from this prospective cohort database, so database patients were diagnosed according to the new definitions proposed by the Third International Consensus Definitions for Sepsis and Septic Shock [8].
Consecutive adult patients with suspected or documented sepsis at ICU admission were recruited. We excluded patients who met any one of the following conditions: (1) withdrawal of life-sustaining treatment; (2) postoperative cardiopulmonary resuscitation (CPR); or (3) incomplete data.
The protocol used in this study was approved by the local ethics committee (Fuxing Hospital, Capital Medical University, 2013FXHEC-KY018). Eight hundred and ninety-four patients were recruited. García-Gallo et al. reported the grouping ratio as 7:3 [13], approximately 70% of the patients were randomly assigned to the experimental group (626 sepsis cases). It is generally believed that the sample size should be at least 10–20 times higher than the number of independent variables [14]; the relatively stricter requirement is that the number of samples in each category in the two-classification results is at least 10 times higher than the number of independent variables.
Basic vital signs and patient characteristics, such as age, sex, admission category, auxiliary examinations, comorbidities, primary sites of infection, PCT level, albumin (ALB) level, and APACHE II score, were collected at ICU presentation, while the LAC levels and SOFA scores were recorded for one week after admission. The values immediately preceding missing values were used when data values were missing [15].
Definition of predictive indicators
To create a predictive model, biomarker levels were presented in absolute values and measured at different time points; the relative change in biomarker values was analyzed according to each indictor, which was described as follows:
SOFA max = maximum SOFA score over 7 consecutive days, SOFA min = minimum SOFA score over 7 consecutive days, SOFA initial = SOFA score on the first day after admission, SOFA mean = average SOFA score for 7 consecutive days.
△SOFA1 = SOFA max – SOFA min, △SOFA2 = SOFA max – SOFA initial.
LAC max = maximum LAC over 7 consecutive days, LAC min = minimum LAC over 7 consecutive days, LAC initial = LAC on the first day after admission, LAC mean = average LAC for 7 consecutive days.
△LAC1 = LAC max – LAC min, △LAC2 = LAC max – LAC initial.
Statistical analyses
The baseline patient characteristics were summarized in percentages for categorical variables, and data were presented as the mean ± standard deviation for normally distributed variables or median (25th–75th percentile) for nonnormally distributed variables. The Shapiro–Wilk test was used to assess the normality of the data.
Univariate comparisons of variables between the two groups were analyzed using the independent t-test for normally distributed variables and the Mann–Whitney U test for nonnormally distributed variables. Categorical variables were presented as frequencies (percentages) and were compared using a chi-squared test; p < 0.05 was considered statistically significant.
Our sepsis mortality score represents the predicted probability of hospital mortality. To derive this score, we used logistic regression by including all predictors with a univariate significance of p < 0.1 as covariates and hospital mortality as the dependent variable, employing the forward elimination method. In multiplicative models, such as logistic regression [16], the product term is used to indicate the presence or absence of an interaction (p < 0.05). The generated coefficients for each predictor in the final step of the logistic regression were used to create the equation to predict a logit transformation of the probability of hospital mortality: Logit(p) = β0(intercept) + β1 (APACHE II) + β2 (△SOFA2) + β3(△LAC2) + β4(SOFA mean).
The model calibration was evaluated using the Hosmer–Lemeshow goodness-of-fit test. The area under the receiver operating characteristic (AUROC) curve [95% confidence interval (CI)] for the sepsis mortality score was calculated to predict hospital mortality.
Patients were then grouped according to the optimal cut-off value of the sepsis mortality score. A log-rank test was performed to compare the survival curves of the groups. The median survival time was calculated by the Kaplan–Meier method. In the validation group, the kappa value was calculated to evaluate the stability of the derived equation.
All statistical analyses were performed using SPSS version 22.0 (IBM, Armonk, NY, USA).
Results
Patient characteristics and prevalence of sepsis
The patients eligible for inclusion and exclusion are outlined in Fig. 1. A total of 894 patients diagnosed with sepsis were included in this study. According to the ratio of 7:3 [13], approximately 70% of the patients were randomly assigned to the experimental group (626 cases). In addition, the experimental group was further divided into a survivor group (378 cases) and a nonsurvivor group (248 cases) according to the outcome at the time of discharge.
Figure 1.
Inclusion of patients for analysis of the association between different predictors and mortality at hospital discharge between January 2014 to August 2015. CPR Cardiopulmonary Resuscitation
Table 1 shows the demographics and clinical characteristics of all patients. In the experimental group, the hospital mortality was 39.6%. The mean age was 68 years (range 55–80 years). Occurrences of mechanical ventilation and renal replacement therapy correlated with death during the hospital day. ICU admission sources were from medical wards (65.8%) and surgical wards (34.2%). Age, sex, and comorbidity influenced the diagnostic or predictive accuracy of the predictors. However, there was no significant difference in the above factors between the two groups, suggesting good homogeneity. In terms of infection focus, only pulmonary infection, bacteremia and abdominal infection showed significant differences.
Table 1.
Baseline demographics and clinical characteristics in survivors and nonsurvivors at ICU discharge of two groups
| Variables | Experimental group | Validation group | ||||||
|---|---|---|---|---|---|---|---|---|
| All | Survivors | Nonsurvivors | P | All | Survivors | Nonsurvivors | P | |
| n = 626 | n = 378 | n = 248 | n = 268 | n = 190 | n = 78 | |||
| Demographic | ||||||||
| Age (years) | 68.0 (55.0, 80.0) | 68.5 (55.0, 80.0) | 68.0 (55.0, 79.5) | 0.728 | 68.0 (58.0, 80.0) | 64.0 (54.0, 77.0) | 75.0 (65.0, 82.0) | 0.000 |
| Sex (male), n (%) | 394 (62.9) | 229 (60.6) | 163 (65.7) | 0.193 | 169 (63.1) | 125 (65.8) | 44 (56.4) | 0.148 |
| BMI | 22.9 (21.0, 24.9) | 23.0 (21.1, 24.7) | 22.9 (20.8, 25.0) | 0.354 | 22.7 (20.5, 24.5) | 22.8 (20.8, 24.5) | 22.3 (19.6, 24.2) | 0.134 |
| Clinical | ||||||||
| Admission Category, n (%) | ||||||||
| Medical | 412 (65.8) | 237 (62.7) | 175 (70.6) | 0.011 | 225 (84.0) | 152 (80.0) | 73 (93.6) | 0.010 |
| Surgical | 214 (34.2) | 141 (37.3) | 73 (29.4) | 0.011 | 43 (16.0) | 38 (20.0) | 5 (6.4) | 0.010 |
| Severity of illness, n (%) | ||||||||
| MV | 469 (74.9) | 246 (65.1) | 223 (89.9) | 0.000 | 185 (69.0) | 116 (61.1) | 69 (88.5) | 0.000 |
| CRRT | 111 17.7) | 36 (9.5) | 75 (30.2) | 0.000 | 38 (14.2) | 13 (6.8) | 25 (32.1) | 0.000 |
| Sepsis | 325 (51.9) | 227 (60.0) | 98 (39.5) | 0.000 | 128 (47.8) | 105 (55.3) | 23 (29.5) | 0.000 |
| Septic shock | 301 (48.1) | 151 (39.9) | 150 (60.4) | 0.000 | 140 (52.2) | 85 (44.7) | 55 (70.5) | 0.000 |
| CCI | 1 (0, 2) | 1 (0, 2) | 2 (1, 2) | 0.001 | 1 (0, 2) | 1 (0, 2) | 2 (1, 3) | 0.000 |
| Primary sites of infection, n (%) | ||||||||
| Lungs | 333 (53.2) | 180 (47.6) | 153 (61.7) | 0.001 | 144 (53.7) | 94 (49.5) | 50 (64.1) | 0.029 |
| Thoracic cavity | 11 (1.8) | 7 (1.9) | 4 (1.6) | 1.000 | 8 (3.0) | 7 (3.7) | 1 (1.3) | 0.513 |
| Abdomen | 162 (25.9) | 110 (29.1) | 52 (21.0) | 0.023 | 65 (24.3) | 52 (27.4) | 13 (16.7) | 0.063 |
| Urinary tract | 30 (4.8) | 19 (5.0) | 11 (4.4) | 0.157 | 7 (2.6) | 7 (3.7) | 0 (0.0) | 0.195 |
| Bacteremia | 38 (6.1) | 15 (4.0) | 23 (9.3) | 0.007 | 17 (6.3) | 11 (5.8) | 6 (7.7) | 0.562 |
| Catheter | 3 (0.5) | 1 (0.3) | 2 (0.8) | 0.712 | 2 (0.7) | 1 (0.5) | 1 (1.3) | 1.000 |
| Soft tissue | 15 (2.4) | 9 (2.4) | 6 (2.4) | 0.975 | 10 (3.7) | 9 (4.7) | 1 (1.3) | 0.317 |
| Nervous system | 6 (1.0) | 4 (1.1) | 2 (0.8) | 1.000 | 2 (0.7) | 0 (0.0) | 2 (2.6) | 0.152 |
| Hospital days | 16 (9, 27) | 17 (10, 28) | 15 (6, 24) | 0.003 | 19 (11, 30) | 20 (11, 31) | 15 (9, 25) | 0.007 |
BMI body mass index, CCI Charlson Comorbidity Index, CRRT continuous renal replacement therapy, ICU intensive care unit, MV mechanical ventilation. Continuous variables are presented as mean ± standard deviation or median and interquartile range when not normally distributed. Categorical variables are presented as number (%)
Predictor profiles
The medians and interquartile ranges are shown for each predictor for the population as a whole and stratified by the outcome of hospital mortality (Table 2). The APACHE II score, SOFA score, ALB and LAC levels differed significantly among the groups. The median SOFA score was higher in the nonsurvival group than in the survival group (p < 0.001). In contrast, ALB was lower in the nonsurvival group than in the survival group (25.5 vs. 27.0 mmol/L, respectively, p < 0.001). Factors other than PCT were independent predictors of mortality in patients with sepsis. The clinical value of the different predictors composed of original laboratory parameters in the diagnosis of sepsis was evaluated with a receiver operating characteristic (ROC) curve. We determined the AUROC and the cut-off values for each predictor to classify patients with in-hospital mortality as a summary measure of the predictive accuracy (Table 2).
Table 2.
APACHE II, SOFA, LAC, and PCT in survivors and nonsurvivors and their predictive for hospital mortality
| Biomarkers | All (n = 626) | Survivors (n = 378) | Nonsurvivors (n = 248) | P value | AUROC (95%CI) | Cut-off |
|---|---|---|---|---|---|---|
| APACHE II | 18 (14, 25) | 16 (11, 20) | 25 (19, 30) | 0.000 | 0.823 (0.789–0.856) | 21.5 |
| PCT | 1.5 (0.2, 9.9) | 1.3 (0.2, 10.0) | 2.1 (0.2, 10.9) | 0.290 | 0.525 (0.479–0.572) | 0.8 |
| ALB | 26 (22.7, 30.0) | 27.0 (23.0, 31.2) | 25.5 (22.0, 8.5) | 0.001 | 0.419 (0.373–0.464) | 27.7 |
| Temperature | 37.6 (36.7, 38.7) | 37.5 (36.6, 38.5) | 37.6 (36.8, 38.9) | 0.022 | 0.554 (0.508–0.600) | 38.9 |
| SOFA initial | 7.0 (4.0, 10.0) | 6.0 (4.0, 8.0) | 9.0 (6.0, 13.0) | 0.000 | 0.715 (0.674–0.756) | 6.7 |
| SOFA mean | 6.3 (3.9, 9.7) | 5.0 (3.1, 7.3) | 9.3 (6.3, 12.7) | 0.000 | 0.774 (0.737–0.811) | 5.8 |
| SOFA max | 8.0 (5.0, 12.0) | 6.7 (4.0, 10.0) | 12.0 (8.0, 15.0) | 0.000 | 0.767 (0.730–0.804) | 7.3 |
| △SOFA1 | 3.0 (2.0, 5.0) | 3.0 (1.0, 5.0) | 3.0 (2.0, 5.0) | 0.004 | 0.567 (0.521–0.613) | 2.5 |
| △SOFA2 | 0.0 (0.0, 2.0) | 0.0 (0.0, 1.0) | 1.0 (0.0, 3.8) | 0.000 | 0.634 (0.589–0.680) | 0.5 |
| LAC initial | 1.7 (1.1, 2.7) | 1.6 (1.1, 2.4) | 2.0 (1.2, 3.3) | 0.000 | 0.609 (0.564–0.655) | 1.9 |
| LAC mean | 1.6 (1.1, 2.4) | 1.4 (1.0, 2.0) | 2.0 (1.4, 3.4) | 0.000 | 0.697 (0.656–0.739) | 2.2 |
| LAC max | 2.3 (1.6, 3.5) | 2.0 (1.3, 2.8) | 2.8 (2.0, 6.4) | 0.000 | 0.699 (0.657–0.740) | 3.3 |
| △LAC1 | 1.1 (0.6, 2.1) | 0.9 (0.5, 1.6) | 1.5 (0.8, 3.4) | 0.000 | 0.650 (0.606–0.695) | 2.3 |
| △LAC2 | 0.2 (0.0, 0.9) | 0.0 (0.0, 0.6) | 0.5 (0.0, 2.2) | 0.000 | 0.630 (0.584–0.676) | 1.5 |
AUROC area under the receiver operating characteristic curve, APACHE II Acute Physiology and Chronic Health Evaluation II, ALB albumin, CI confidence interval, PCT procalcitonin, SOFA Sequential Organ Failure Assessment, LAC lactate, LAC initial lactate on the first day after admission, LAC mean average lactate for 7 consecutive days, LAC max maximum lactate for 7 consecutive days, LAC min minimum lactate for 7 consecutive days, △LAC1 LAC max – LAC min, △LAC2 LAC max – LAC initial, SOFA initial SOFA score on the first day after admission, SOFA mean average SOFA score for 7 consecutive days, SOFA max maximum SOFA score for 7 consecutive days, SOFA min minimum SOFA score for 7 consecutive days, △SOFA1 SOFA max – SOFA min, △SOFA2 SOFA max – SOFA initial. Data are expressed as medians (interquartile range). Comparison of the two groups was conducted using the Mann–Whitney test
Multivariable logistic regression model derivation and development of the SHMS
We used multivariate logistic regression to model the ability of the predictors to identify patients who have a high risk of in-hospital mortality. Binary logistic regression applied to predict mortality in patients with sepsis showed that among the APACHE II score, temperature, △SOFA2, △LAC 2, and SOFA mean score were independent predictors of sepsis mortality (Table 3). The regression model was constructed employing the forward elimination method.
Table 3.
Univariate and multivariate analysis of the risk factors for hospital mortality
| Variables | Univariate | Multivariate | ||
|---|---|---|---|---|
| OR (95%CI) | P value | OR (95%CI) | P value | |
| APACHE II | 1.216 (1.177–1.257) | 0.000 | 1.188 (1.144–1.235) | 0.000* |
| ALB | 0.962 (0.938–0.986) | 0.002 | NI | |
| Temperature | 1.139 (1.010–1.285) | 0.034 | 0.847 (0.724–0.991) | 0.038* |
| SOFA initial | 1.220 (1.167–1.276) | 0.000 | NI | |
| SOFA mean | 1.312 (1.247–1.381) | 0.000 | 1.089 (1.021–1.162) | 0.010* |
| SOFA max | 1.271 (1.215–1.329) | 0.000 | NI | |
| △SOFA1 | 1.097 (1.030–1.168) | 0.004 | NI | |
| △SOFA2 | 1.271 (1.173–1.377) | 0.000 | 1.193 (1.081–1.316) | 0.000* |
| LAC initial | 1.236 (1.135–1.345) | 0.000 | NI | |
| LAC mean | 1.743 (1.492–2.035) | 0.000 | NI | |
| LAC max | 1.285 (1.200–1.375) | 0.000 | NI | |
| △LAC1 | 1.267 (1.176–1.364) | 0.000 | NI | |
| △LAC2 | 1.423 (1.263–1.604) | 0.000 | 1.156 (1.027–1.301) | 0.017* |
NI not included in multivariate survival analysis. Risk is presented as hazard ratio and 95% confidence interval (CI) with an increment of 1 unit of the biomarker concentration. The following covariates, deemed important clinical variables, were considered in the multivariable models: APACHE II, temperature, SOFA mean, △SOFA2, △LAC2. *P < 0.05. APACHE II Acute Physiology and Chronic Health Evaluation II, ALB albumin, SOFA Sequential Organ Failure Assessment, LAC lactate, OR odds ratio
Regarding the multivariable analysis, four covariates remained in the final model (Table 4). A scoring system was developed to discriminate between patient survival and nonsurvival upon hospital discharge; this scoring system represented the final “SHMS”. The product term of △SOFA2 and SOFA mean score was 1.161, which suggested that interaction was not an issue. The score was derived as follows: xβ = −4.715 + (0.164 × APACHE II) + (0.171 × △SOFA2) + (0.145 × △LAC2) + (0.082 × SOFA mean). The calibration was also demonstrated to be accurate by the Hosmer–Lemeshow goodness-of-fit test (p = 0.59, 8° of freedom).
Table 4.
Multivariate analysis of the risk factors for development of model
| Variables | β | OR (95%CI) | P value |
|---|---|---|---|
| APACHE II | 0.164 | 1.178 (1.136–1.223) | 0.000* |
| △SOFA2 | 0.171 | 1.186 (1.076–1.307) | 0.001* |
| △LAC2 | 0.145 | 1.157 (1.029–1.299) | 0.014* |
| SOFA mean | 0.082 | 1.086 (1.019–1.157) | 0.012* |
| Constant | −4.715 | 0.378 (−) | 0.000* |
* P<0.05. APACHE II Acute Physiology and Chronic Health Evaluation II, CI confidence interval, SOFA Sequential Organ Failure Assessment, LAC lactate, OR odds ratio
Trend of the SOFA score as an independent risk factor in two groups
The data of each group did not satisfy the spherical symmetry of the covariance matrix. Greenhouse–Geisser correction showed an interaction between processing factors and time factors (F = 14.557, p < 0.001), indicating a trend in SOFA score changes with time; it can be considered that the SOFA score changes linearly with time. As shown in Table 5, the overall mean D1, D3, and D7 SOFA scores of the patients were different, with p values < 0.001, and the SOFA scores of the survival group were lower than those of the nonsurvival group.
Table 5.
The Sequential Organ Failure Assessment (SOFA) scores of survivors and nonsurvivors group in the D1, D3, and D7
| Time | (I) Group | (J) Group | Mean difference (I – J) | OR | P value | 95% CI | |
|---|---|---|---|---|---|---|---|
| Lower | Upper | ||||||
| D1 | Survivors | Nonsurvivors | −1.871 | 0.398 | 0.000 | −2.65 | −1.09 |
| Nonsurvivors | Survivors | 1.871 | 0.398 | 0.000 | 1.088 | 2.654 | |
| D3 | Survivors | Nonsurvivors | −2.338 | 0.388 | 0.000 | −3.1 | −1.58 |
| Nonsurvivors | Survivors | 2.338 | 0.388 | 0.000 | 1.575 | 3.101 | |
| D7 | Survivors | Nonsurvivors | −3.523 | 0.409 | 0.000 | −4.33 | −2.72 |
| Nonsurvivors | Survivors | 3.523 | 0.409 | 0.000 | 2.717 | 4.328 |
CI confidence interval, OR odds ratio
As seen in Table 6, the overall mean D1, D3, and D7 SOFA scores in the survival group were not equal (p < 0.001), and the nonsurvival group showed the opposite trend. The overall mean values of the D1, D3, and D7 SOFA scores were not statistically significant. The patients in the nonsurvival group showed no significant changes in the severity of disease within 7 days after admission.
Table 6.
The Sequential Organ Failure Assessment (SOFA) scores of the D1, D3, and D7 in the survivors and nonsurvivors
| Group | (I) Time | (J) Time | Mean difference (I – J) | OR | P value | 95% CI | |
|---|---|---|---|---|---|---|---|
| Lower | Upper | ||||||
| Survivors | D1 | D3 | 0.643 | 0.148 | 0.000 | 0.288 | 0.999 |
| D7 | 1.952 | 0.244 | 0.000 | 1.365 | 2.539 | ||
| D3 | D1 | −0.643 | 0.148 | 0.000 | −1.287 | −0.288 | |
| D7 | 1.309 | 0.195 | 0.000 | 0.841 | 1.777 | ||
| D7 | D1 | −1.952 | 0.244 | 0.000 | −2.539 | −1.365 | |
| D3 | −1.309 | 0.195 | 0.000 | −1.777 | −0.841 | ||
| Nonsurvivors | D1 | D3 | 0.176 | 0.181 | 0.993 | −0.259 | 0.612 |
| D7 | 0.301 | 0.299 | 0.948 | −0.419 | 1.021 | ||
| D3 | D1 | −0.176 | 0.181 | 0.993 | −0.612 | 0.259 | |
| D7 | 0.124 | 0.239 | 1.000 | −0.449 | 0.698 | ||
| D7 | D1 | −0.301 | 0.299 | 0.948 | −1.021 | 0.419 | |
| D3 | −0.124 | 0.239 | 1.000 | −0.698 | 0.449 |
CI confidence interval, OR odds ratio
The SOFA scores were markedly variable during sepsis episodes (Fig. 2). There were statistically significant differences between the downward trends for the predictors and hospital survival (p < 0.001 for D1–D3 trend and p < 0.001 for D3–D7 trend).
Figure 2.
Changes in Sequential Organ Failure Assessment (SOFA) scores on the first, third and seventh day of admission in the nonsurvival group and the survival group. D1 day 1, D2 day 2, D3 day 3
Differences in clinical outcome among three distinct situations of the same △SOFA score
The △SOFA score accounted for three distinct situations. The statistical results of D1–D3 and D1–D7 were the same. Comparison between the three groups showed that there were significant differences between the outcomes of the decreasing group and the increasing group, as well as between the increasing group and stable group, but there was no significant difference between the decreasing group and the stable group (Table 7).
Table 7.
Differences in clinical outcome among three distinct situations of the same delta Sequential Organ Failure Assessment (SOFA) score
| Same △SOFA | D1–D3 | D1–D7 | ||
|---|---|---|---|---|
| Survivors | Nonsurvivors | Survivors | Nonsurvivors | |
| Decreasing group | 201 | 58 | 179 | 45 |
| Stable group | 113 | 48 | 44 | 19 |
| Increasing group | 98 | 84 | 41 | 55 |
D1–D3: compared with decreasing group and stable group, decreasing group and increasing group, stable group and increasing group, p1 = 0.000 × 3, p2 = 0.009 × 3, p3 = 0.001 × 3, respectively; D1–D7:compared with decreasing group and stable group, decreasing group and increasing group, stable group and increasing group, p4 = 0.000 × 3, p5 = 0.089 × 3, p6 = 0.002 × 3, respectively
Predictive performance of the SHMS and cut-off values to discriminate nonsurvivors with sepsis
There was a prognostic value of the new risk factor, SHMS (based on APACHE II, △SOFA2, △LAC2, and SOFA mean), for hospital mortality. The AUROC of the sepsis mortality score and each of its constituent individual predictors of hospital mortality are shown in Table 8.
Table 8.
The receiver operating characteristic (ROC)analysis of APACHE II, △SOFA2, △LAC2, and SOFA mean score for prediction prognosis
| Variables | AUC | P value | 95% CI | Youden | Cut-off | Sensitivity | Specificity | NPV | PPV | |
|---|---|---|---|---|---|---|---|---|---|---|
| Lower | Upper | |||||||||
| APACHE II | 0.823 | 0.000 | 0.789 | 0.856 | 0.489 | 21.50 | 0.661 | 0.828 | 0.716 | 0.788 |
| △SOFA2 | 0.634 | 0.000 | 0.589 | 0.680 | 0.218 | 0.50 | 0.617 | 0.601 | 0.503 | 0.705 |
| △LAC2 | 0.630 | 0.000 | 0.584 | 0.676 | 0.233 | 1.45 | 0.315 | 0.918 | 0.716 | 0.671 |
| SOFA mean | 0.774 | 0.000 | 0.737 | 0.811 | 0.463 | 5.83 | 0.831 | 0.632 | 0.597 | 0.851 |
| SHMS | 0.851 | 0.000 | 0.821 | 0.882 | 0.550 | 0.33 | 0.778 | 0.772 | 0.692 | 0.841 |
AUROC area under the curve, APACHE II Acute Physiology and Chronic Health Evaluation II, SOFA Sequential Organ Failure Assessment, LAC lactate, CI confidence interval, NPV negative predictive value, PPV positive predictive value, SHMS Sepsis Hospital Mortality Score, SHMS = 0.164APACHEII + 0.171△SOFA2 + 0.145△LAC2 + 0.082SOFA mean
The distribution of scores calculated by the predictive formula in the experimental group and the validation group were shown in Fig. 3a, b. The AUC of the APACHE II score, △SOFA2, △LAC2, and SOFA mean for hospital mortality was 0.823, 0.634, 0.630, and 0.774, respectively, and the AUC was 0.851, which was improved when all 4 factors were combined (95% CI (0.821–0.882), standard error 0.033), suggesting very good model discrimination (Fig. 3c). The optimal cut-off value was 0.332 ≈ 0.33 (sensitivity: 78%, specificity: 77%, Youden’s index: 0.55). In all patients, considering a score threshold of ≤ 0.33, the positive predictive value was 84.1%, and the negative predictive value was 69.2%.
Figure 3.
Prognostic value of the new risk factor, Sepsis Hospital Mortality Score (SHMS), for hospital mortality. (a, b) The value of SHMS in the survival and nonsurvival groups; (c, d) comparison of receiver operating characteristic (ROC) analysis of the Acute Physiology and Chronic Health Evaluation II (APACHE II) score, △LAC2, △SOFA2, mean Sequential Organ Failure Assessment (SOFA) score and SHMS for the prediction of hospital mortality of patients with sepsis. LAC lactate
Two subgroups stratified by the severity of illness and their corresponding median survival times
The predicted value fluctuated between 0.001 and 0.996, and the cut-off value was 0.332, as described in Fig. 3a, b. The experimental group was divided into two subgroups according to the cut-off value, those with 0.01 < ŷ ≤ 0.332 and those with 0.332 < ŷ ≤ 0.996. As shown in Fig. 4, the log-rank values of the different cases were counted, and the following conclusions were drawn. Using a score threshold of < 0.322, 56 (16.1%) of the 348 nonsurvivors were correctly identified. In the Kaplan–Meier analysis, the median survival time in the high-risk group was significantly shorter than the low-risk group (11 vs. 15 days, p < 0.001) (Fig. 4a). If we reclassify the severity of the disease into low-, moderate-, and high-risk groups using another demarcation point, the curve indicates that the median survival time in the moderate- and low-risk groups coincided after 18 days (Fig. 4b).
Figure 4.
Kaplan–Meier plot showing survival according to increased sepsis hospital mortality score in the experimental group. (a) Kaplan–Meier plot showing the sepsis-related hospital mortality in the low-risk group: 0.001–0.332 and high-risk group: 0.332–0.996. The median survival times of the two groups were 15 days and 11 days, respectively (log-rank = 0.000). (b) Kaplan–Meier plot showing the sepsis-related hospital mortality in the low-risk group: 0.001–0.332, moderate-risk group: 0.332–0.800, and high-risk group: 0.332–0.996. The median survival times of the three groups was 15 days, 13 days, and 5 days, respectively (log-rank = 0.000)
Consistency of the diagnostic tests
Consistency of the diagnostic tests refers to the degree of consistency between the observations from the diagnostic tests and those from the validation group. The kappa statistic is usually used as an index to evaluate the degree of consistency. The value of the kappa statistic is usually in the range of 0–1. The larger the value, the higher the consistency level. In the validation group, 78 patients were predicted to die, and there were 60 true deaths. The kappa statistic of the deduced formula was 0.614, and the standard error was 0.060, showing good consistency. Furthermore, in the validation group (268 patients), the AUC was 0.899 when all four factors were combined (95% CI 0.801–0.897, standard error 0.025) (Fig. 3d).
Discussion
Early detection and the use of sepsis care bundles are associated with reduced mortality [8]. In this study, we assembled a cohort of 626 patients with sepsis and studied the effect of different predictors on their mortality with the overall goal of creating a prediction equation: the “SHMS”. The statistical results show that the APACHE II score, △SOFA2, △LAC2, and SOFA mean score were independently associated with hospital mortality in sepsis patients admitted to the ICU, and the new formula showed very good results in predicting hospital mortality (AUROC 0.851).
In our study, the SOFA scores were recorded for 7 days after admission. We wanted to explore an improved prediction method by calculating the diagnostic efficiency of the changes in the SOFA score, SOFA mean, SOFA initial, and SOFA max. Research [17] showed that the median SOFA value of in-hospital surviving patients (SOFA = 7) had a sensitivity of 95.3% and a specificity of 49.4% for hospital mortality. Our results are consistent with previous studies [8,18,19], which have shown that the mean SOFA score provides the best discrimination to predict hospital mortality (cut-off = 5.8, AUROC = 0.774, p = 0.000) compared with other scoring tools. We consider that SOFA means can reflect the severity of a patient’s condition for a period of time; it is more representative than the score at any time and more reflective of the patient’s response to treatment.
According to a study by Ferreira et al. [20], independent of the initial score, an increase in the SOFA score during the first 48 hours in the ICU predicts a mortality rate of at least 50% and differences in mortality were better predicted in the first 48 hours than in the subsequent 48 hours, which differ from our finding that the statistical results of D1–D3 and D1–D7 were the same. In our study, the patients in the nonsurvival group showed no significant changes in the severity of disease within 7 days of admission. Furthermore, the overall mean D1, D3, and D7 SOFA scores of the patients were different, and the SOFA scores of the surviving patients were lower than those in the nonsurvival group.
We believe that the △SOFA score accounts for three distinct situations: increasing scores, decreasing scores, and stable scores. Comparison between the three score groups showed that there were significant differences in the outcomes of the increasing group and the decreasing group and between the decreasing and stable group, but there was no significant difference between the increasing group and the stable group. The AUROC curve of the △SOFA score was less than 0.7. We consider the six systems in which SOFA scores are included, and the scores of the six systems varied from one system to another within 7 days, with the total score unchanged. Therefore, △SOFA should be combined with disease severity to better predict the outcomes of patients.
Aublanc and Richard [21] stressed that the basic requirement for a sepsis diagnosis is organ dysfunction, which may delay the early identification and treatment of sepsis, thus affecting the prognosis of patients. A combination of predictors may present a more reliable and objective guide for mortality prediction in sepsis patients. Recent studies also demonstrated that two prediction models that combined predictors performed better than routinely used clinical scores in predicting sepsis-related mortality [22, 23].
In clinical practice, many of the family members of the patient will consult the doctor for answers; questions usually do not involve the patient’s overall outcome, but revolve around how long the patient will live. Similar to the different median survival times for different types of tumors or tumor patients at different stages, our goal is to develop a prediction formula to classify sepsis patients according to different risk levels and then calculate the corresponding median survival time. This formula could provide theoretical support for the clinician to allocate limited medical resources and answer the questions of the patient’s family. For example, palliative therapy can be used for patients with a short median life expectancy.
In the validation group, the AUROC curve of the joint prediction index was 0.899, and the kappa statistic was 0.614, indicating good validation of the clinical utility of sepsis in the prediction of mortality in sepsis. The latest standard of sepsis is also applicable to the Chinese patient population to some extent and has a moderate effect in predicting patient outcomes. Our study had several limitations. Patient outcomes depended on patient management, which may have varied among different institutions. Although we attempted to control the confounding of other clinical variables by establishing a SHMS model with a logistic regression model, we were not to account for other unmeasured confounding or collinear effects. In addition, we did not record the lowest daily LAC value, which may explain the limited predictive power of LAC, and we did not obtain information on ICU-acquired sepsis. In addition, patients were followed only during hospitalization. We stratified the risk severity of the patient’s condition only into low-risk and high-risk. When risk stratification was performed with low-, moderate-, and high-risk groups, there was no significant difference in the corresponding median survival time between the low-risk group and the moderate-risk group. Therefore, we need more deliberate prospective cohort studies to verify the clinical significance of this formula.
Conclusions
The APACHE II score, △SOFA, △LAC, and SOFA mean score were independently associated with hospital mortality in sepsis patients and accurately predicted the hospital mortality rate and median survival time. Data on the median survival time in sepsis patients could be provided to clinicians to assist in the rational use of limited medical resources by facilitating prudent resource allocation.
Funding
This work was supported by the National Science and Technology Support Program (2012BAI11B05).
Availability of data and materials
The data sets are available from the corresponding author on request with justification.
Authors’ contributions
WL was involved in the acquisition of data, data processing, study design, statistical analysis and manuscript writing and drafting; MPW was involved in data processing and performed the statistical analysis; BZ was involved in the study design and data processing; YBZ and XMX were involved in the study design and the final revision of the manuscript; and all authors read and approved the final manuscript.
Ethics approval and consent to participate
The protocol used in this study was approved by the local ethics committee (2013FXHEC-KY018).
Consent for publication
Not applicable.
Conflicts of interest
On behalf of all authors, the corresponding author states that there are no conflicts of interest.
Acknowledgments
The authors would like to acknowledge the faculty and staff at the Xiang Ya Hospital, West China Hospital of SiChuan University, Chinese Academy of Medical Sciences Union Hospital, First Hospital of Jilin University, Peking Union Medical College Hospital, Peking University Third Hospital, General Hospital of Ningxia Medical University, Sino-Japanese Friendship Hospital, Ministry of Health, First Affiliated Hospital of China Medical University, Zhongshan Hospital Affiliated with Fu Dan University, FuXing Hospital Affiliated with Capital Medical University, Beijing Friendship Hospital Affiliated with Capital Medical University, Beijing Chao yang Hospital Affiliated with Capital Medical University, Beijing Tiantan Hospital Affiliated with Capital Medical University, Beijing Xuanwu Hospital Affiliated with Capital Medical University, Beijing Tongren Hospital Affiliated with Capital Medical University, First Affiliated Hospital of Jilin University, and Ningxia Medical University for their assistance in the conduct of this study.
References
- 1. De Backer D, Dorman T. Surviving sepsis guidelines: A continuous move toward better care of patients with sepsis. JAMA. 2017;317:807–8. [DOI] [PubMed] [Google Scholar]
- 2. Yongming Y, Yingyi L. Precision evaluation of immune status and its significance in sepsis after burns or trauma. Chin J Burns 2018;34:786–9. [DOI] [PubMed] [Google Scholar]
- 3. Esper AM, Martin GS. Extending international sepsis epidemiology: the impact of organ dysfunction. Crit Care. 2009;13:120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Harrison DA, Welch CA, Eddleston JM. The epidemiology of severe sepsis in England, Wales and Northern Ireland, 1996 to 2004: secondary analysis of a high quality clinical database, the ICNARC case mix programme database. Crit Care. 2006;10:R42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Yealy DM, Kellum JA, Huang DT, Barnato AE, Weissfeld LA, Pike F, et al. . A randomized trial of protocol-based care for early septic shock. N Engl J Med. 2014; 370: 1683–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Rowan KM, Angus DC, Bailey M, Barnato AE, Bellomo R, Canter RR, et al. . Early, goal-directed therapy for septic shock—a patient-level meta-analysis. N Engl J Med. 2017;376:2223–34. [DOI] [PubMed] [Google Scholar]
- 7. Cabral L, Afreixo V, Meireles R, Vaz M, Chaves C, Caetano M, et al. . Checking procalcitonin suitability for prognosis and antimicrobial therapy monitoring in burn patients. Burns Trauma. 2018;6:10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. . The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA. 2016;315:801–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Grooth HJ, Geenen IL, Girbes AR, Vincent JL, Parienti JJ, Oudemans-van Straaten HM. SOFA and mortality endpoints in randomized controlled trials: a systematic review and meta-regression analysis. Critical Care. 2017;21:38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Hayashida K, Kondo Y, Hara Y, Aihara M, Yamakawa K. Head-to-head comparison of procalcitonin and presepsin for the diagnosis of sepsis in critically ill adult patients: a protocol for a systematic review and meta-analysis. BMJ Open. 2017;7:e014305. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Moran JL, Santamaria J. Reconsidering lactate as a sepsis risk biomarker. PLoS One. 2017;12:e0185320. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Vincent JL, De Backer D. Circulatory shock. N Engl J Med. 2013;369:1726–34. [DOI] [PubMed] [Google Scholar]
- 13. García-Gallo JE, Fonseca-Ruiz NJ, Celi LA, Duitama-Muñoz JF. A machine learning-based model for 1-year mortality prediction in patients admitted to an intensive care unit with diagnosis of sepsis. Med Intensiva. 2018; pii: S0210-5691(18)30245-6. [DOI] [PubMed] [Google Scholar]
- 14. Concato J, Peduzzi P, Holford TR, Feinstein AR. Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol. 1995;48:1495–501. [DOI] [PubMed] [Google Scholar]
- 15. Zhang Y, Khalid S, Jiang L. Diagnostic and predictive performance of biomarkers in patients with sepsis in an intensive care unit. J Int Med Res. 2019;47:44–58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Rothman KJ. Epidemiology: An Introduction. New York: Oxford University Press, 2002. [Google Scholar]
- 17. Khwannimit B, Bhurayanontachai R, Vattanavanit V. Comparison of the accuracy of three early warning scores with SOFA score for predicting mortality in adult sepsis and septic shock patients admitted to intensive care unit. Heart Lung. 2019;48:240–4. [DOI] [PubMed] [Google Scholar]
- 18. Khwannimit B, Bhurayanontachai R, Vattanavanit V. Comparison of the performance of SOFA, qSOFA and SIRS for predicting mortality and organ failure among sepsis patients admitted to the intensive care unit in a middle-income country. J Crit Care. 2018;44:156–60. [DOI] [PubMed] [Google Scholar]
- 19. Szakmany T, Pugh R, Kopczynska M, Lundin RM, Sharif B, Morgan P, et al. . Defining sepsis on the wards: results of a multi-centre point-prevalence study comparing two sepsis definitions. Anaesthesia. 2018;73:195–04. [DOI] [PubMed] [Google Scholar]
- 20. Ferreira FL, Bota DP, Bross A, Mélot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA. 2001;286:1754–8. [DOI] [PubMed] [Google Scholar]
- 21. Aublanc M, Richard JC. Assessment of clinical criteria for sepsis—was the cart put before the horse? J Thorac Dis. 2016;8:E816–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Hu C, Zhou Y, Liu C, Kang Y. Pentraxin-3, procalcitonin and lactate as prognostic markers in patients with sepsis and septic shock. Oncotarget. 2018;9:5125–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Shukeri W, Ralib AM, Abdulah NZ, Mat-nor MB. Sepsis mortality score for the prediction of mortality in septic patients. J Crit Care. 2018;43:163–8. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data sets are available from the corresponding author on request with justification.